Ripgrep Integration

Feature Definition

An AI coding agent needs to search through code. When the model wants to find all files that reference DatabasePool, or every place where a particular error string appears, or all .rs files containing async fn, it needs a fast, regex-capable search tool that respects .gitignore rules and returns results in a format the model can act on.

This is fundamentally different from repo mapping. Repo mapping is proactive — it builds a structural synopsis of the codebase before the model asks for anything. Code search is reactive — the model decides it needs to find something, invokes a search tool, and gets results back. Repo mapping uses tree-sitter to extract semantic tags (function definitions, class declarations). Code search uses regex pattern matching against raw file contents.

The hard parts are:

• Result volume: A naive grep -r "error" across a large monorepo can return tens of thousands of matches. The model’s context window can’t absorb that, and most of those matches are noise. The tool needs hard limits, intelligent truncation, and clear signals about whether results were cut off.
• Performance: Search must complete in seconds, not minutes. Ripgrep’s parallelized regex engine, memory-mapped I/O, and .gitignore-aware file walking make it the de facto choice in Codex and OpenCode.
• Ignore handling: The agent shouldn’t search inside node_modules/, .git/, or build artifacts. This means respecting .gitignore, .ignore, and potentially custom exclusion patterns.
• Output format: The model needs structured output — file paths, line numbers, matched text — not raw terminal output. How you format search results directly impacts whether the model can make use of them.
• Scope control: The model should be able to narrow searches by file glob (*.rs), directory path, and result count. Without this, searches on large codebases return too much irrelevant content.

None of the reference implementations invented their own search engine. Codex and OpenCode shell out to ripgrep (rg), while Aider relies on repo-map extraction and ranking instead of an LLM-callable grep tool. The differences are in how they wrap search: parameter schemas, result limits, output formatting, and error handling.

This page is the backend/search-engine layer. Tool-level contracts and model-facing behavior are documented in Grep, List, and Glob. For proactive structural context (rather than reactive regex search), see Repo Mapping.

---

Aider Implementation

Reference: references/aider/aider/ | Commit: b9050e1d5faf8096eae7a46a9ecc05a86231384b

No Ripgrep — Tree-sitter Instead

Aider does not expose a grep or ripgrep tool to the LLM. There is no search_files function call, no /grep slash command, and no subprocess invocation of rg anywhere in the codebase.

Instead, Aider’s approach to code search is entirely mediated through its repo mapping system (documented in Repo Mapping). When the user mentions an identifier like DatabasePool in their message, Aider’s get_ident_mentions() function in base_coder.py:678-682 extracts all word-boundary tokens from the user’s message using a simple regex split:

words = set(re.split(r"\W+", text))

These identifiers are then fed into the PageRank-based ranking system in repomap.py, which boosts files that define or reference those identifiers. The result is a repo map that surfaces relevant code — but it’s the map doing the searching, not a grep tool.

Slash Commands for Manual Search

Aider does provide a /run command (commands.py) that lets users execute arbitrary shell commands, including grep or rg. But this is a generic shell escape, not a structured search tool. The LLM doesn’t invoke it — the user does. Output from /run is appended to the chat as raw text with no parsing or truncation.

The /add command (commands.py:799-898) supports glob patterns for adding files to the chat context:

def glob_filtered_to_repo(self, pattern):
    raw_matched_files = list(Path(self.coder.root).glob(pattern))
    # Expand directories recursively
    # Filter to repo root
    # Filter by git tracked files

This uses Python’s pathlib.Path.glob(), not ripgrep. It’s for file discovery, not content search.

Why No Search Tool?

Aider’s design philosophy is that the LLM shouldn’t need to search — the repo map should preemptively surface everything relevant. The tag extraction system (tree-sitter + grep-ast), the PageRank ranking, and the token-budgeted rendering together form a proactive context system that makes reactive search less necessary. This works well for focused tasks where the user’s message contains enough signal to drive the ranking, but it means the model can’t explore the codebase on its own initiative.

---

Codex Implementation

Reference: references/codex/codex-rs/core/src/tools/handlers/grep_files.rs | Commit: 4ab44e2c5cc54ed47e47a6729dfd8aa5a3dc2476

grep_files Tool

Codex exposes grep_files as a first-class function-call tool that the model can invoke on any turn. The handler is GrepFilesHandler in grep_files.rs, a ToolHandler impl registered in the tool registry during session initialization.

Parameters (deserialized from the model’s JSON function call, lines 28-37):

struct GrepFilesArgs {
    pattern: String,                    // regex pattern (required)
    include: Option<String>,            // glob filter, e.g. "*.rs" (optional)
    path: Option<String>,              // search directory (optional)
    limit: usize,                       // max results, default 100, max 2000
}

Ripgrep invocation (lines 116-129):

rg --files-with-matches \
   --sortr=modified \
   --regexp PATTERN \
   --no-messages \
   [--glob GLOB] \
   -- PATH

Key flags:

• --files-with-matches: Returns only file paths, not matched lines. This is a deliberate design choice — Codex’s search tells the model which files match, not what lines match. The model then reads specific files with a separate read_file tool.
• --sortr=modified: Results sorted by modification time, newest first. Recently-modified files are more likely to be relevant to the current task.
• --no-messages: Suppresses ripgrep error messages (broken symlinks, permission denied) to avoid polluting the output.

Limits and timeouts (lines 20-22):

const DEFAULT_LIMIT: usize = 100;
const MAX_LIMIT: usize = 2000;
const COMMAND_TIMEOUT: Duration = Duration::from_secs(30);

The 30-second timeout prevents runaway searches on enormous repositories. The 100-result default keeps the model’s context manageable — if 100 files match a pattern, the model is almost certainly searching too broadly and should narrow the query.

Result parsing (lines 154-171):

Stdout is split by newlines, empty lines filtered out, and the result truncated at exactly limit entries. The output is a plain list of file paths, one per line. On no matches, the handler returns "No matches found." with success=false.

Error handling:

• Exit code 0 → success, parse results
• Exit code 1 → no matches (ripgrep convention), return empty
• Any other exit code → error with stderr message

Tool specification (in spec.rs:829-878):

fn create_grep_files_tool() -> ToolSpec {
    // Name: "grep_files"
    // Description: "Finds files whose contents match the pattern
    //              and lists them by modification time."
    // Required: ["pattern"]
}

The tool is registered with parallel_support: true, meaning the model can invoke multiple grep_files calls in a single turn without waiting for sequential results.

Design Philosophy

Codex’s search is intentionally coarse-grained. It answers “which files contain this pattern?” not “which lines match?” This pushes the model toward a two-step workflow: search to identify files, then read to understand them. The benefit is smaller search results that stay within token budget. The cost is an extra round-trip for every search.

---

OpenCode Implementation

Reference: references/opencode/packages/opencode/src/tool/grep.ts | Commit: 7ed449974864361bad2c1f1405769fd2c2fcdf42

Grep Tool

OpenCode’s grep tool is the most full-featured of the three. It returns matched lines with line numbers, groups results by file, sorts by modification time, and applies multiple layers of truncation.

Parameters (lines 12-18):

parameters: z.object({
    pattern: z.string(),           // regex pattern (required)
    path: z.string().optional(),   // search directory (optional)
    include: z.string().optional() // file glob, e.g. "*.{ts,tsx}" (optional)
})

Ripgrep invocation (lines 39-44):

rg -nH \
   --hidden \
   --no-messages \
   --field-match-separator=| \
   --regexp PATTERN \
   [--glob GLOB] \
   PATH

Key flags:

• -nH: Show line numbers (-n) and filenames (-H).
• --hidden: Include hidden files (dotfiles). Unlike Codex, OpenCode searches hidden files by default.
• --field-match-separator=|: Uses pipe as the delimiter between filename, line number, and matched text. This is critical for parsing — the default colon separator breaks on Windows paths and file contents that contain colons.

Output processing (lines 56-148):

1. Parse each line by splitting on | to extract filePath, lineNum, lineText
2. Collect mtime (modification time) for each unique file via fs.stat()
3. Sort files by mtime descending (newest first)
4. Group matches by file path
5. Format each match as Line {lineNum}: {text}
6. Truncate individual line text at 2,000 characters (line 10)
7. Hard cap at 100 total matches across all files (line 100-102)
8. Set truncated: true flag if limit was hit

Result format returned to the model:

Found 47 matches across 12 files (sorted by modification time):

src/session/prompt.ts
Line 142: async function buildPrompt(session: Session) {
Line 298: const messages = filterCompacted(session.id)

src/tool/grep.ts
Line 39: const args = ["-nH", "--hidden", ...
...

This format is significantly richer than Codex’s file-path-only output. The model gets immediate context about what matched, not just where.

Metadata returned alongside the text output:

metadata: {
    matches: totalMatches,    // total before truncation
    truncated: boolean        // whether limit was hit
}

Glob Tool

OpenCode also exposes a glob tool (tool/glob.ts) for file name matching:

parameters: z.object({
    pattern: z.string(),          // glob pattern (required)
    path: z.string().optional()   // search directory (optional)
})

This uses Ripgrep.files() — an async generator that spawns rg --files --glob PATTERN and streams file paths. Results are sorted by mtime descending and capped at 100 files. The output is a plain list of file paths.

Ripgrep Binary Management

OpenCode bundles its own ripgrep binary rather than relying on the system installation (file/ripgrep.ts:125-203):

1. Check for system rg via Bun.which("rg")
2. If not found, check for bundled binary at Global.Path.bin/rg
3. If neither exists, auto-download ripgrep 14.1.1 from GitHub releases
4. Platform detection covers arm64-darwin, arm64-linux, x64-darwin, x64-linux, x64-win32
5. Downloads are tar.gz (Unix) or zip (Windows), extracted automatically
6. Lazy initialization — download happens on first search, not at startup

This guarantees ripgrep availability regardless of the user’s system setup, at the cost of a one-time download on first use.

Output Truncation Pipeline

Beyond the 100-match hard cap in the grep tool itself, OpenCode applies a secondary truncation layer via tool/truncation.ts (lines 50-105):

const MAX_LINES = 2000
const MAX_BYTES = 50 * 1024  // 50 KB

If the tool output exceeds either limit, the overflow is saved to disk at Global.Path.data/tool-output/ with a 7-day retention policy. The truncated output includes a file path reference so the model can delegate to a Task tool to read the full output if needed.

---

Pitfalls & Hard Lessons

Result Volume Explosion

The most common failure mode across all three tools is a search pattern that matches too broadly. Searching for error in a large TypeScript project can return thousands of hits across node_modules/ (if not excluded), test fixtures, log files, and generated code. Both Codex and OpenCode solve this with hard caps (100-2000 results), but those caps mean the model may miss relevant matches buried after the cutoff. Sorting by mtime helps — recently modified files are more likely relevant — but it’s a heuristic, not a guarantee.

File-Only vs Line-Level Results

Codex’s --files-with-matches approach returns only file paths, forcing a two-step search-then-read workflow. This keeps search results compact but adds latency. OpenCode returns matched lines directly, which is more useful in a single round-trip but consumes more context. Neither approach is strictly better — it depends on how often the model needs the matched line content versus just needing to know which files to look at.

Hidden Files and Dotfiles

OpenCode searches hidden files by default (--hidden), which means it finds matches in .github/, .env.example, and other dotfiles. This is useful for configuration-related queries but can introduce noise. Codex doesn’t pass --hidden, so dotfiles are excluded unless explicitly globbed. Neither behavior is always correct.

Ripgrep Availability

Codex assumes ripgrep is installed on the system (rg in PATH). If it isn’t, grep_files fails silently or with an unhelpful error. OpenCode solves this by auto-downloading a pinned ripgrep version, but this adds complexity (platform detection, archive extraction, lazy initialization). Aider sidesteps the issue entirely by not using ripgrep.

Parse Failures on Unusual Output

Ripgrep’s output format can surprise parsers. Filenames containing the separator character (: or |), binary file warnings, and context lines (from -A/-B/-C flags) all need special handling. OpenCode’s choice of | as field separator via --field-match-separator is more robust than the default : but still breaks on filenames containing pipes. Codex avoids this entirely by using --files-with-matches, which only outputs paths.

Timeout Sensitivity

Codex’s 30-second timeout is generous for most searches but can be hit on very large repositories with broad patterns. There’s no incremental result delivery — if the timeout fires, the model gets nothing, not a partial result set. OpenCode doesn’t set a ripgrep-specific timeout, relying instead on its general tool timeout (2 minutes), which is more forgiving.

gitignore Edge Cases

Codex and OpenCode rely on ripgrep’s built-in .gitignore handling, which is excellent but not identical to git’s. Nested .gitignore files, negation patterns (!important.log), and the interaction between .gitignore and .ignore files can produce surprising results. Aider does not use ripgrep here; its /add path uses Path.glob() and then filters to tracked files.

---

OpenOxide Blueprint

Architecture

OpenOxide should expose two search tools to the model:

1. grep — Content search (regex against file contents, returns matched lines)
2. glob — File name search (glob patterns against file paths)

Both tools should be non-mutating, support parallel invocation, and return structured results with truncation metadata.

Ripgrep Integration

Use the grep crate (ripgrep’s library form) rather than shelling out to the rg binary. This eliminates the binary-availability problem entirely and gives us direct control over the search pipeline:

use grep_regex::RegexMatcher;
use grep_searcher::{Searcher, SearcherBuilder, Sink};
use ignore::WalkBuilder;

struct GrepResult {
    path: PathBuf,
    line_number: u64,
    line_text: String,
    mtime: SystemTime,
}

fn search(pattern: &str, root: &Path, glob: Option<&str>, limit: usize) -> Vec<GrepResult> {
    let matcher = RegexMatcher::new(pattern).unwrap();
    let walker = WalkBuilder::new(root)
        .hidden(true)          // include hidden files
        .git_ignore(true)      // respect .gitignore
        .git_global(true)      // respect global gitignore
        .build();

    let mut results = Vec::new();
    for entry in walker.flatten() {
        if results.len() >= limit { break; }
        // Apply glob filter if provided
        // Search file with matcher
        // Collect GrepResult entries
    }

    results.sort_by(|a, b| b.mtime.cmp(&a.mtime));
    results.truncate(limit);
    results
}

The ignore crate (also from the ripgrep ecosystem) handles .gitignore, .ignore, and hidden file filtering. Using it as a library means we get ripgrep’s exact ignore semantics without parsing subprocess output.

Crates

Crate	Purpose
grep-regex	Regex compilation for search patterns
grep-searcher	File content search with configurable sinks
ignore	.gitignore-aware directory walking
globset	Glob pattern compilation for file filters

Tool Parameters

#[derive(Deserialize)]
struct GrepParams {
    pattern: String,                    // regex (required)
    path: Option<String>,              // search root (optional, default CWD)
    include: Option<String>,           // file glob (optional)
    limit: Option<usize>,             // max results (optional, default 100, max 2000)
}

#[derive(Deserialize)]
struct GlobParams {
    pattern: String,                    // glob pattern (required)
    path: Option<String>,              // search root (optional, default CWD)
    limit: Option<usize>,             // max results (optional, default 100)
}

Output Format

Follow OpenCode’s approach — return matched lines with line numbers, grouped by file, sorted by mtime:

Found {n} matches across {m} files (sorted by modification time):

{relative_path_1}
Line {num}: {truncated_text}
Line {num}: {truncated_text}

{relative_path_2}
Line {num}: {truncated_text}
...

[Results truncated. {total} total matches found.]

Line text truncated at 2,000 characters. Total results capped at limit (default 100). Include a truncated: bool field in tool metadata.

Constants

const DEFAULT_LIMIT: usize = 100;
const MAX_LIMIT: usize = 2000;
const MAX_LINE_LENGTH: usize = 2000;   // truncate individual lines
const SEARCH_TIMEOUT: Duration = Duration::from_secs(30);

Timeout and Cancellation

Use tokio::time::timeout wrapping the search operation. If the timeout fires, return whatever results have been collected so far (partial results are better than nothing — this is where Codex’s all-or-nothing approach fails). Thread the CancellationToken from the session down to the walker so that session termination also stops in-progress searches.

Relationship to Repo Mapping

The grep tool and the repo map serve different purposes and should not share implementation. The repo map (documented separately) extracts semantic tags via tree-sitter and ranks them with PageRank. The grep tool does raw text search. They coexist: the repo map provides ambient context, while grep handles targeted searches the model initiates during a conversation.