ReXile 🦎

A blazing-fast regex engine with 10-100x faster compilation speed

ReXile is a lightweight regex alternative that achieves exceptional compilation speed while maintaining competitive matching performance:

• ⚡ 10-100x faster compilation - Load patterns instantly
• 🚀 Competitive matching - 1.4-1.9x faster on simple patterns
• 🎯 Dot wildcard support - Full ., .*, .+ implementation with backtracking
• 📦 Only 2 dependencies - memchr and aho-corasick for SIMD primitives
• 🧠 Smart backtracking - Handles complex patterns with quantifiers
• 🔧 Perfect for parsers - Ideal for GRL, DSL, and rule engines

Key Features:

• ✅ Literal searches with SIMD acceleration
• ✅ Multi-pattern matching (alternations)
• ✅ Character classes with negation
• ✅ Quantifiers (*, +, ?)
• ✅ Non-greedy quantifiers (*?, +?, ??) - NEW in v0.2.1
• ✅ Dot wildcard (., .*, .+) with backtracking
• ✅ DOTALL mode ((?s)) - Dot matches newlines - NEW in v0.2.1
• ✅ Non-capturing groups ((?:...)) with alternations - NEW in v0.2.1
• ✅ Escape sequences (\d, \w, \s, etc.)
• ✅ Sequences and groups
• ✅ Word boundaries (\b, \B)
• ✅ Anchoring (^, $)
• ✅ Capturing groups - Auto-detection and extraction

🎯 Purpose

ReXile is a high-performance regex engine optimized for fast compilation:

• 🚀 Lightning-fast compilation - 10-100x faster than regex crate
• ⚡ Competitive matching - Faster on simple patterns, acceptable on complex
• 🎯 Ideal for parsers - GRL, DSL, rule engines with dynamic patterns
• 📦 Minimal dependencies - Only memchr + aho-corasick for SIMD primitives
• Memory efficient - 15x less compilation memory
• 🔧 Full control - Custom optimizations for specific use cases

Performance Highlights

Compilation Speed (vs regex crate):

• Pattern [a-zA-Z_]\w*: 104.7x faster 🚀
• Pattern \d+: 46.5x faster 🚀
• Pattern (\w+)\s*(>=|<=|==|!=|>|<)\s*(.+): 40.7x faster 🚀
• Pattern .*test.*: 15.3x faster
• Average: 10-100x faster compilation

Matching Speed:

• Simple patterns (\d+, \w+): 1.4-1.9x faster ✅
• Complex patterns with backtracking: 2-10x slower (acceptable for non-hot-path)
• Perfect trade-off for parsers and rule engines

Use Case Example (Load 1000 GRL rules):

• regex crate: ~2 seconds compilation
• rexile: ~0.02 seconds (100x faster startup!)

Memory Comparison:

• Compilation: 15x less memory (128 KB vs 1920 KB)
• Peak memory: 5x less in stress tests (0.12 MB vs 0.62 MB)
• Search operations: Equal memory efficiency

When to Use ReXile:

• ✅ Parsers & lexers (fast token matching + instant startup)
• ✅ Rule engines with dynamic patterns (100x faster rule loading)
• ✅ DSL compilers (GRL, business rules)
• ✅ Applications with many patterns (instant initialization)
• ✅ Memory-constrained environments (15x less memory)
• ✅ Non-hot-path matching (acceptable trade-off for 100x faster compilation)

🚀 Quick Start

use rexile::Pattern;

// Literal matching with SIMD acceleration
let pattern = Pattern::new("hello").unwrap();
assert!(pattern.is_match("hello world"));
assert_eq!(pattern.find("say hello"), Some((4, 9)));

// Multi-pattern matching (aho-corasick fast path)
let multi = Pattern::new("foo|bar|baz").unwrap();
assert!(multi.is_match("the bar is open"));

// Dot wildcard matching (with backtracking)
let dot = Pattern::new("a.c").unwrap();
assert!(dot.is_match("abc"));  // . matches 'b'
assert!(dot.is_match("a_c"));  // . matches '_'

// Greedy quantifiers with dot
let greedy = Pattern::new("a.*c").unwrap();
assert!(greedy.is_match("abc"));       // .* matches 'b'
assert!(greedy.is_match("a12345c"));   // .* matches '12345'

let plus = Pattern::new("a.+c").unwrap();
assert!(plus.is_match("abc"));         // .+ matches 'b' (requires at least one char)
assert!(!plus.is_match("ac"));         // .+ needs at least 1 character

// Non-greedy quantifiers (NEW in v0.2.1)
let lazy = Pattern::new(r"start\{.*?\}").unwrap();
assert_eq!(lazy.find("start{abc}end{xyz}"), Some((0, 10))); // Matches "start{abc}", not greedy

// DOTALL mode - dot matches newlines (NEW in v0.2.1)
let dotall = Pattern::new(r"(?s)rule\s+.*?\}").unwrap();
let multiline = "rule test {\n  content\n}";
assert!(dotall.is_match(multiline));    // (?s) makes .* match across newlines

// Non-capturing groups with alternation (NEW in v0.2.1)
let group = Pattern::new(r#"(?:"test"|foo)"#).unwrap();
assert!(group.is_match("\"test\""));    // Matches quoted "test"
assert!(group.is_match("foo"));         // Or matches foo

// Digit matching (DigitRun fast path - 1.4-1.9x faster than regex!)
let digits = Pattern::new("\\d+").unwrap();
let matches = digits.find_all("Order #12345 costs $67.89");
// Returns: [(7, 12), (20, 22), (23, 25)]

// Identifier matching (IdentifierRun fast path)
let ident = Pattern::new("[a-zA-Z_]\\w*").unwrap();
assert!(ident.is_match("variable_name_123"));

// Quoted strings (QuotedString fast path - 1.4-1.9x faster!)
let quoted = Pattern::new("\"[^\"]+\"").unwrap();
assert!(quoted.is_match("say \"hello world\""));

// Word boundaries
let word = Pattern::new("\\btest\\b").unwrap();
assert!(word.is_match("this is a test"));
assert!(!word.is_match("testing"));

// Anchors
let exact = Pattern::new("^hello$").unwrap();
assert!(exact.is_match("hello"));
assert!(!exact.is_match("hello world"));

Cached API (Recommended for Hot Paths)

For patterns used repeatedly in hot loops:

use rexile;

// Automatically cached - compile once, reuse forever
assert!(rexile::is_match("test", "this is a test").unwrap());
assert_eq!(rexile::find("world", "hello world").unwrap(), Some((6, 11)));

// Perfect for parsers and lexers
for line in log_lines {
    if rexile::is_match("ERROR", line).unwrap() {
        // handle error
    }
}

✨ Supported Features

Fast Path Optimizations (10 Types)

ReXile uses JIT-style specialized implementations for common patterns:

Regex Features

📊 Performance Benchmarks

Compilation Speed (Primary Advantage)

Pattern Compilation Benchmark (vs regex crate):

Average: 10-100x faster compilation - Perfect for dynamic patterns!

Matching Speed

Simple Patterns (Fast paths):

• Pattern \d+ on "12345": 1.4-1.9x faster ✅
• Pattern \w+ on "variable": 1.4-1.9x faster ✅
• Pattern "[^"]+" on quoted strings: Competitive ✅

Complex Patterns (Backtracking):

• Pattern a.+c on "abc": 2-5x slower (acceptable)
• Pattern .*test.* on long strings: 2-10x slower (acceptable)
• Trade-off: 100x faster compilation vs slightly slower complex matching

Use Case Performance

Loading 1000 GRL Rules:

• regex crate: ~2 seconds (2ms per pattern)
• rexile: ~0.02 seconds (20µs per pattern)
• Result: 100x faster startup! Perfect for parsers and rule engines.

Memory Comparison

Test 1: Pattern Compilation (10 patterns):

• regex: 1920 KB in 7.89ms
• ReXile: 128 KB in 370µs
• Result: 15x less memory, 21x faster ✨

Test 2: Search Operations (5 patterns × 139KB corpus):

• Both: 0 bytes memory delta
• Result: Equal efficiency ✅

Test 3: Stress Test (50 patterns × 500KB corpus):

• regex: 0.62 MB peak in 46ms
• ReXile: 0.12 MB peak in 27ms
• Result: 5x less peak memory, 1.7x faster ✨

When ReXile Wins

✅ Simple patterns (\d+, \w+) - 1.4-1.9x faster matching ✅ Fast compilation - 10-100x faster pattern compilation (huge win!) ✅ Identifiers ([a-zA-Z_]\w*) - 104.7x faster compilation ✅ Memory efficiency - 15x less for compilation, 5x less peak ✅ Instant startup - Load 1000 patterns in 0.02s vs 2s (100x faster) ✅ Dot wildcards - Full ., .*, .+ support with backtracking

When regex Wins

⚠️ Complex patterns with backtracking - ReXile 2-10x slower (acceptable trade-off) ⚠️ Alternations (when|then) - ReXile 2x slower ⚠️ Hot-path matching - For performance-critical matching, regex may be better

Architecture

Pattern → Parser → AST → Fast Path Detection → Specialized Matcher
                                                        ↓
                                     DigitRun (memchr SIMD scanning)
                                     IdentifierRun (direct byte scanning)
                                     QuotedString (memchr + validation)
                                     Alternation (aho-corasick automaton)
                                     Literal (memchr SIMD)
                                     ... 5 more fast paths

Run benchmarks yourself:

cargo run --release --example per_file_grl_benchmark
cargo run --release --example memory_comparison

📦 Installation

Add to your Cargo.toml:

[dependencies]
rexile = "0.2"

🎓 Examples

Literal Search

let p = Pattern::new("needle").unwrap();
assert!(p.is_match("needle in a haystack"));
assert_eq!(p.find("where is the needle?"), Some((13, 19)));

// Find all occurrences
let matches = p.find_all("needle and needle");
assert_eq!(matches, vec![(0, 6), (11, 17)]);

Multi-Pattern (Alternation)

// Fast multi-pattern search using aho-corasick
let keywords = Pattern::new("import|export|function|class").unwrap();
assert!(keywords.is_match("export default function"));

Anchored Patterns

// Must start with pattern
let starts = Pattern::new("^Hello").unwrap();
assert!(starts.is_match("Hello World"));
assert!(!starts.is_match("Say Hello"));

// Must end with pattern
let ends = Pattern::new("World$").unwrap();
assert!(ends.is_match("Hello World"));
assert!(!ends.is_match("World Peace"));

// Exact match
let exact = Pattern::new("^exact$").unwrap();
assert!(exact.is_match("exact"));
assert!(!exact.is_match("not exact"));

Cached API (Best for Repeated Patterns)

// First call compiles and caches
rexile::is_match("keyword", "find keyword here").unwrap();

// Subsequent calls reuse cached pattern (zero compile cost)
rexile::is_match("keyword", "another keyword").unwrap();
rexile::is_match("keyword", "more keyword text").unwrap();

📚 More examples: See examples/ directory for:

• basic_usage.rs - Core API walkthrough
• log_processing.rs - Log analysis patterns
• performance.rs - Performance comparison

Run examples with:

cargo run --example basic_usage
cargo run --example log_processing

🔧 Use Cases

ReXile is production-ready for:

✅ Ideal Use Cases

• Parsers and lexers - 21x faster pattern compilation, competitive matching
• Rule engines - Simple pattern matching in business rules (original use case!)
• Log processing - Fast keyword and pattern extraction
• Dynamic patterns - Applications that compile patterns at runtime
• Memory-constrained environments - 15x less compilation memory
• Low-latency applications - Predictable performance, no JIT warmup

🎯 Perfect Patterns for ReXile

• Fast compilation: All patterns compile 10-100x faster
• Simple matching: \d+, \w+ (1.4-1.9x faster matching)
• Identifiers: [a-zA-Z_]\w* (104.7x faster compilation!)
• Dot wildcards: ., .*, .+ with proper backtracking
• Keyword search: rule\s+, function\s+
• Many patterns: Load 1000 patterns instantly (100x faster startup)

⚠️ Consider regex crate for

• Complex alternations (ReXile 2x slower)
• Very sparse patterns (ReXile up to 1.44x slower)
• Unicode properties (\p{L} - not yet supported)
• Advanced features (lookahead, backreferences - not yet supported)

🤝 Contributing

Contributions welcome! ReXile is actively maintained and evolving.

Current focus:

• ✅ Core regex features complete
• ✅ Dot wildcard (., .*, .+) with backtracking - v0.2.0
• ✅ Capturing groups - Auto-detection and extraction - v0.2.0
• ✅ Non-greedy quantifiers (.*?, +?, ??) - v0.2.1
• ✅ DOTALL mode ((?s)) for multiline matching - v0.2.1
• ✅ Non-capturing groups ((?:...)) with alternations - v0.2.1
• ✅ 10-100x faster compilation
• 🔄 Advanced features: bounded quantifiers {n,m}, lookahead, Unicode support

How to contribute:

1. Check issues for open tasks
2. Run tests: cargo test
3. Run benchmarks: cargo run --release --example per_file_grl_benchmark
4. Submit PR with benchmarks showing performance impact

Priority areas:

• 📋 Bounded quantifiers ({n}, {n,m})
• 📋 More fast path patterns
• 📋 Unicode support
• 📋 Documentation improvements

📜 License

Licensed under either of:

• MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)
• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)

at your option.

🙏 Credits

Built on top of:

• memchr by Andrew Gallant - SIMD-accelerated substring search
• aho-corasick by Andrew Gallant - Multi-pattern matching automaton

Developed for the rust-rule-engine project, providing fast pattern matching for GRL (Grule Rule Language) parsing and business rule evaluation.

Performance Philosophy: ReXile achieves competitive performance through intelligent specialization rather than complex JIT compilation:

• 10 hand-optimized fast paths for common patterns
• SIMD acceleration via memchr
• Pre-built automatons for alternations
• Zero-copy iterator design
• Minimal metadata overhead

Status: ✅ Production Ready (v0.2.1)

• ✅ Compilation Speed: 10-100x faster than regex crate

• ✅ Matching Speed: 1.4-1.9x faster on simple patterns

• ✅ Memory: 15x less compilation, 5x less peak

• ✅ Features: Core regex + dot wildcard + capturing groups + non-greedy + DOTALL + non-capturing groups

• ✅ Testing: 84 unit tests + 13 group integration tests passing

• ✅ Real-world validated: GRL parsing, rule engines, DSL compilers