Grep

Feature Definition

A grep capability lets the model search file contents by pattern before reading or editing files in depth.

It is the highest-leverage narrowing tool in large repositories.

When used well, it converts broad intent into precise targets quickly.

Typical prompts that map naturally to grep:

• find all implementations of a behavior phrase
• locate use sites of a specific API symbol
• search for config keys or env vars
• identify TODO/FIXME clusters
• find regex-compatible log lines

The naive version of grep is easy.

The production version is hard.

Challenges that matter in practice:

• regex syntax and escaping
• path scoping and include/exclude rules
• huge match sets and context budget pressure
• cross-platform line endings
• inaccessible paths and partial errors
• stable output shape for downstream tooling

A robust grep tool must decide up front what it returns.

Two common return models:

• path-level: files that contain at least one match
• line-level: concrete line numbers and text snippets

Path-level is cheap and compact.

Line-level is richer, but potentially much larger.

Both are valid, provided the model understands which one it received.

Search tools also influence token economics.

A path-level grep followed by targeted reads usually beats directly streaming many match lines for large codebases.

But if the task requires exact match counts, line context, or direct replacements, line-level output can reduce round-trips.

Reference repos illustrate this design space:

• Aider: no model-callable grep tool; structural search via repomap and identifier heuristics
• Codex: experimental grep_files with path-level output
• OpenCode: grep tool with line-level grouping and metadata

See Ripgrep Integration for lower-level backend details shared by search tools.

---

See Also

• List and Glob for path discovery before content search.
• File Read for line-accurate inspection after matches are found.
• Ripgrep Integration for backend execution details shared across search pages.

---

Aider Implementation

Aider at commit b9050e1d5faf8096eae7a46a9ecc05a86231384b does not expose a model-callable grep tool.

Search behavior is distributed across repo mapping, identifier extraction, and command workflows.

No Model-Callable Grep Tool

Aider’s model interaction pattern is not built around JSON function calls for search.

The model cannot emit a typed call like:

grep(pattern="foo", include="*.py")

Instead, Aider injects a repository map and file contents into prompt context, then the model reasons from that material.

Command-level shell execution exists (/run, /git), but that is user-initiated command flow, not an autonomous structured grep tool.

RepoMap as Structural Search

Core logic lives in aider/repomap.py.

get_tags_raw() (repomap.py:279-360) performs:

1. language detection from filename (:280-283)
2. parser/language resolution via tree-sitter wrappers (:285-288)
3. query loading from *-tags.scm (:291-295)
4. source read (:296-299)
5. parse tree construction (:299)
6. query capture execution (:301-303)
7. capture classification into def / ref (:319-324)
8. yield Tag records with file/name/kind/line (:328-336)

When only definitions are available, Aider backfills references via Pygments tokenization (:343-360).

Rendered output is produced by render_tree() (repomap.py:710-746) and to_tree() (:748-784), which assemble context around lines of interest.

to_tree() truncates long lines to 100 chars (:781-783), providing compact structural snippets.

This is closer to symbol graph exploration than plain regex grep.

Identifier Heuristics

BaseCoder.get_ident_mentions() in aider/coders/base_coder.py:678-682 splits current user message text on non-word characters (re.split(r"\W+", text)).

Those mentions feed:

• filename matching (get_ident_filename_matches, :684-707)
• repo-map query hints (get_repo_map, :709-747)

get_repo_map() merges:

• file mentions
• identifier mentions
• chat file set
• non-chat file set

then queries repomap with hints.

Fallback behavior:

• hinted map attempt
• global map attempt
• unhinted map attempt

This mechanism gives “search-like” narrowing without raw regex scanning tool calls.

Practical effect:

• excellent symbol-orientation
• less direct support for arbitrary text regex tasks
• no standard path+line grep output contract

---

Codex Implementation

Codex at 4ab44e2c5cc54ed47e47a6729dfd8aa5a3dc2476 exposes grep_files as an experimental function tool.

Parameter Schema

Tool spec in core/src/tools/spec.rs:829-878 defines:

• pattern: string required
• include: string optional glob
• path: string optional file/dir path
• limit: number optional max result paths

Description states it finds files whose contents match the pattern and returns them ordered by modification time.

Registration is conditional in spec.rs:1511-1518:

experimental_supported_tools must include "grep_files".

If feature flags do not include that entry, the tool is absent from model-visible surface.

Execution Flow

Implementation: core/src/tools/handlers/grep_files.rs:1-270.

Detailed flow:

1. payload type check (ToolPayload::Function) (:48-55)
2. argument parse via shared helper (:57)
3. trim pattern and reject empty (:59-64)
4. reject zero limit (:66-70)
5. clamp limit to MAX_LIMIT = 2000 (:72)
6. resolve search path from turn context (:73)
7. verify path exists with async metadata (:75, :102-107)
8. normalize include glob by trimming empties (:77-83)
9. run ripgrep subprocess with timeout (:85-152)
10. parse stdout lines into string paths, truncating at limit (:154-170)
11. return:
    • No matches found. with success=false when empty
    • newline-separated path list with success=true otherwise

Ripgrep invocation arguments (:116-130):

• --files-with-matches
• --sortr=modified
• --regexp <pattern>
• --no-messages
• optional --glob <include>
• -- <search_path>

Timeout behavior:

• hard timeout at 30s (COMMAND_TIMEOUT, :22)
• timeout yields model-facing error rg timed out after 30 seconds (:131-135)

Launch failure behavior:

• model-facing error asks to ensure ripgrep is installed (:136-140)

Exit code handling:

• 0: parse and return results
• 1: return empty list (no matches)
• other: treat as error with stderr message

Output Contract

Codex returns path-level results only.

No line numbers.

No snippets.

No grouped formatting by file.

This is deliberate to keep payload compact and encourage read_file follow-up for details.

Advantages:

• predictable small outputs
• easier context budgeting
• fast path for candidate discovery

Tradeoff:

• second tool call needed for match context
• cannot directly count multiple matches per file from tool output

Experimental Tool Gating

Because grep_files is experimental-gated, behavior depends on model metadata/config layering.

ToolsConfig.experimental_supported_tools comes from model_info.experimental_supported_tools in spec.rs:102.

This allows server-side capability toggling per model.

Operational consequences:

• documentation must call out that availability is conditional
• prompts should be resilient when tool is missing
• fallback plan should include read_file + list_dir or shell tool in permitted environments

Test coverage in grep_files.rs:173-270 validates core behavior:

• parsing
• limit truncation
• glob filtering
• no-match handling
• ripgrep availability guard

---

OpenCode Implementation

OpenCode at 7ed449974864361bad2c1f1405769fd2c2fcdf42 implements grep in packages/opencode/src/tool/grep.ts:12-150.

Parameter Schema (Zod)

Schema fields:

• pattern: string required
• path?: string optional search root
• include?: string optional include glob

Description text (grep.txt:1-8) highlights:

• regex syntax support
• include filtering examples
• suggestion to use Bash+rg for match counting tasks
• guidance to use task agent for open-ended iterative searching

Ripgrep Command

grep.ts builds args (:39-45):

• -nH
• --hidden
• --no-messages
• --field-match-separator=|
• --regexp <pattern>
• optional --glob <include>
• <searchPath>

Process spawn (:46-50):

• executable path from Ripgrep.filepath() (:39)
• stdout/stderr piped
• abort signal wired

Exit code policy (:56-69):

• 0: matches found
• 1: no matches -> returns “No files found”
• 2 with output: partial success (e.g., inaccessible paths)
• 2 without output: treated as no matches
• other nonzero: throws failure with stderr

This is a practical handling of ripgrep’s nuanced exit semantics in real filesystems.

Parsing and Grouping

Output parsing details:

• split by \r?\n for cross-platform line endings (:73-75)
• split each line using | separator (:80-85)
• parse lineNum as integer (:83)
• reconstruct lineText from remaining parts (:84)
• stat each file for mtime sorting (:86-95)
• sort descending by mtime (:98)

Result shaping:

• cap result rows at limit = 100 (:100-103)
• line snippet max length MAX_LINE_LENGTH = 2000 (:10, :124-126)
• group output by file with path header (:115-123)
• include line entries like Line N: <text> (:126)

Truncation messaging:

• reports total matches
• states shown limit and hidden count (:129-133)

Partial error messaging:

• when exit code was 2, appends “Some paths were inaccessible and skipped” (:136-139)

Output and Metadata

OpenCode grep returns both narrative text and metadata.

Metadata includes:

• matches: totalMatches
• truncated: boolean

Title uses the pattern string.

This is richer than path-only output and can satisfy many tasks in one call, but can consume more tokens on broad patterns.

Permission layer:

Before search, ctx.ask is called with permission grep (:24-33), and external directory assertions are enforced (:37).

This keeps search behavior aligned with project boundary policy and user control.

---

Pitfalls & Hard Lessons

Pattern Escaping Across Shell and Regex

Regex strings and shell parsing are frequent failure points.

If commands are shell-joined rather than argv-array based, metacharacters can be interpreted unexpectedly.

Reference handling:

• Codex uses Command arg vectors
• OpenCode uses Bun.spawn([exe, ...args])

OpenOxide should preserve this invariant:

always pass grep args as structured array, never through shell interpolation.

Also consider model guidance examples for common escaping pitfalls:

• literal dot vs wildcard dot
• backslash doubling in JSON contexts
• word boundaries in different regex engines

Paths vs Line Matches

Path-level and line-level outputs serve different goals.

Confusion arises when prompts assume line-level context but tool only returns file paths.

Codex vs OpenCode contrast:

• Codex: path-level compactness
• OpenCode: line-level richness

Mitigation:

• tool schema should declare output mode clearly
• metadata should include mode
• model instruction should suggest next best tool based on selected mode

Timeouts and Partial Visibility

Long searches can timeout or partially fail without clear user awareness.

Codex explicitly emits timeout errors at 30 seconds.

OpenCode preserves partial-match output for ripgrep exit code 2 and flags inaccessible paths.

OpenOxide should preserve both capabilities:

• hard timeout guard
• partial-success reporting with explicit status

Avoid converting partial-success into silent no-match.

That causes incorrect model conclusions.

Binary and Huge File Behavior

Search backends can skip or alter behavior on binary files, ignored directories, or huge generated assets.

Potential issues:

• false negatives in minified or generated files
• skipped vendor/build directories depending on defaults
• surprising hidden-file inclusion/exclusion

Mitigation set:

• expose include/exclude controls explicitly
• surface key backend flags in debug metadata
• provide truncation and skipped-path indicators
• encourage two-phase search: coarse path discovery then precise read/grep narrowing

---

OpenOxide Blueprint

Tool Contract

Implement grep_files with explicit output mode.

Proposed schema:

• pattern: String required
• path: Option<String> absolute or workspace-relative
• include: Option<String> glob filter
• exclude: Option<Vec<String>> optional deny globs
• limit: Option<usize> default 100, max 2000
• mode: "paths" | "lines" default lines
• max_line_length: Option<usize> default 2000

Response shape:

• mode
• count_total
• count_returned
• truncated
• partial (true when some paths inaccessible)
• elapsed_ms
• results

Where results is:

• path list for mode=paths
• {path, line, text} tuples for mode=lines

Execution and Limits

Execution pipeline:

1. validate pattern non-empty
2. resolve and canonicalize path
3. enforce external-directory permission when needed
4. request grep permission with metadata
5. spawn ripgrep with argv args
6. apply timeout
7. parse output stream
8. apply caps and shaping
9. return structured result + concise human summary

Timeout policy:

• default 30s
• configurable upper bound by policy

Limit policy:

• request value clamped to hard max
• deterministic truncation

Error taxonomy should be typed:

• InvalidPattern
• InvalidPath
• PathNotFound
• PermissionDenied
• RipgrepUnavailable
• Timeout
• ExecutionFailed

Result Shaping

For mode=paths:

• unique normalized paths
• optional sort by mtime desc
• compact output optimized for follow-up reads

For mode=lines:

• grouped by file in text summary
• preserve line numbers
• line text truncation at configured boundary

Metadata should include diagnostic hints:

• whether exit status indicated partial filesystem errors
• number of ignored/skipped entries if known
• which include/exclude filters were applied

This helps the model decide next steps without guessing why expected matches were absent.

Crates

Core crates for Rust implementation:

• tokio::process for subprocess execution
• tokio::time for timeout handling
• serde + schemars for schema typing
• thiserror for error surface
• existing path normalization utility for absolute path enforcement

Optional later optimization:

• switch to ripgrep JSON output mode for structured parsing without fragile delimiters

Test plan should include:

• regex success and no-match
• include/exclude behavior
• path mode and line mode
• truncation correctness
• timeout path
• ripgrep missing path
• Windows and Unix line ending parsing
• partial filesystem access behavior

A grep tool is foundational.

Getting its contracts right improves every downstream read/edit decision and reduces wasted context budget across the entire agent loop.