OpenGrep

Advanced AST-aware code search with AI-powered insights. OpenGrep understands your code structure and provides intelligent search capabilities beyond simple pattern matching.

Features

• AST-Aware Search: Uses tree-sitter to understand code structure
• Multi-Language Support: Supports 15+ programming languages
• AI Integration: Optional OpenAI integration for intelligent insights
• High Performance: Parallel search with efficient file traversal
• Flexible Output: Text, JSON, HTML, XML, and CSV output formats
• Interactive Mode: Built-in interactive search interface
• Docker Ready: Containerized for easy deployment
• Configurable: Extensive configuration options

Quick Start

Installation

From crates.io (Recommended)

cargo install opengrep

From Source

# Clone the repository
git clone https://github.com/opengrep-org/opengrep.git
cd opengrep

# Build and install
cargo install --path .

Using Docker

# Pull the image
docker pull opengrep/opengrep:latest

# Run a search
docker run --rm -v $(pwd):/workspace opengrep/opengrep "TODO" src/

Basic Usage

# Search for a pattern in current directory
opengrep "TODO" .

# Case-insensitive regex search
opengrep -i -e "fn\s+\w+" src/

# Show AST context around matches
opengrep -a "struct.*User" src/

# Interactive mode
opengrep -i

# JSON output
opengrep -o json "error" src/ > results.json

Supported Languages

• Rust
• Python
• JavaScript/TypeScript
• Go
• Java
• C/C++
• C#
• Ruby
• Bash
• JSON
• TOML
• CSS

Advanced Examples

AST-Aware Search

# Find all function definitions
opengrep -a "function_item" src/

# Search with AST context display
opengrep --ast-context --max-ast-depth 2 "impl" src/

AI-Powered Analysis

# Enable AI insights (requires OPENAI_API_KEY)
export OPENAI_API_KEY="your-api-key"
opengrep --ai-insights "memory leak" src/

# Get AI explanations for matches
opengrep --ai-explain "unsafe" src/

Output Formats

# HTML report
opengrep -o html "TODO" src/ > report.html

# CSV for spreadsheet analysis
opengrep -o csv "function" src/ > functions.csv

# Structured JSON
opengrep -o json --stats "error" src/

Filtering and Performance

# Filter by programming language
opengrep -l rust -l python "struct" .

# Exclude directories
opengrep -x "target/**" -x "node_modules/**" "TODO" .

# Limit file size and use more threads
opengrep --max-file-size 1048576 -j 8 "pattern" .

Configuration

OpenGrep can be configured via CLI arguments or configuration files:

Configuration File

Create ~/.config/opengrep/config.toml:

[search]
ignore_case = false
regex = false
threads = 8
max_file_size = 10485760

[output]
color = true
line_numbers = true
before_context = 2
after_context = 2
show_ast_context = false

[ai]
model = "gpt-4o-mini"
enable_insights = false
enable_explanation = false
max_tokens = 1000

Environment Variables

export OPENAI_API_KEY="your-openai-api-key"
export OPENGREP_CONFIG="/path/to/config.toml"
export RUST_LOG="info"  # Logging level

Building

Prerequisites

• Rust 1.75+
• Git

Development Build

./build.sh build

Release Build

./build.sh build 1.0.0

All Build Commands

./build.sh clean     # Clean previous builds
./build.sh format    # Format code
./build.sh lint      # Run linter
./build.sh test      # Run tests
./build.sh docker    # Build Docker image
./build.sh install   # Install locally

Docker Usage

Build Docker Image

docker build -t opengrep:latest .

Run with Docker

# Basic search
docker run --rm -v $(pwd):/workspace opengrep:latest "pattern" /workspace

# Interactive mode
docker run --rm -it -v $(pwd):/workspace opengrep:latest -i

# With AI features
docker run --rm -v $(pwd):/workspace -e OPENAI_API_KEY opengrep:latest --ai-insights "pattern" /workspace

Testing

# Run all tests
cargo test

# Run specific test module
cargo test ast::tests

# Run with output
cargo test -- --nocapture

# Integration tests
cargo test --test integration_tests

API Integration

OpenGrep can be used as a library:

use opengrep::{Config, SearchEngine};
use std::path::PathBuf;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let config = Config::default();
    let engine = SearchEngine::new(config);
    
    let results = engine.search("TODO", &[PathBuf::from("src")]).await?;
    
    for result in results {
        println!("Found {} matches in {}", 
                 result.matches.len(), 
                 result.path.display());
    }
    
    Ok(())
}

FastAPI Integration

For web service integration, see the examples directory for FastAPI endpoint implementations.

Performance

OpenGrep is optimized for performance:

• Parallel Processing: Multi-threaded file traversal and searching
• Smart Filtering: Respect .gitignore and file type filters
• Memory Efficient: Streaming file processing
• AST Caching: Intelligent AST parsing and caching

Benchmarks

# Run benchmarks
cargo bench

# Compare with other tools
hyperfine 'opengrep "pattern" .' 'rg "pattern" .' 'grep -r "pattern" .'

Troubleshooting

Common Issues

1. AI features not working

   export OPENAI_API_KEY="your-api-key"
   

2. Out of memory errors

   opengrep --max-file-size 1048576 "pattern" .
   

3. Slow performance

   opengrep -j $(nproc) "pattern" .
   

4. Language not detected

   opengrep --list-languages  # See supported languages
   

Debug Mode

RUST_LOG=debug opengrep "pattern" .

Contributing

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Development Setup

git clone https://github.com/opengrep-org/opengrep.git
cd opengrep
./build.sh format lint test

License

This project is dual-licensed under either:

• MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)
• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)

at your option.

Acknowledgments

• tree-sitter for AST parsing
• tokio for async runtime
• clap for CLI interface
• OpenAI for AI integration

Support

• Documentation
• Issue Tracker
• Discussions
• Email: team@opengrep.dev