wardenclyffe

Wardenclyffe — A Tiny Query Engine in Rust

Wardenclyffe is a fast, lightweight command-line query engine written in Rust, capable of scanning CSV files (and soon Parquet) with a simple SQL-like filter language.

It includes:

• A custom query DSL

• A Pest-powered parser

• An AST (Abstract Syntax Tree)

• An expression evaluator

• A CSV + Parquet datasource abstraction

• Projection (SELECT)

• Execution timing

• Row/column filtering

✨ Features ✔ SQL-like filtering

Example:

1. Full Scan

wardenclyffe --file data.csv

2. Filter rows

wardenclyffe --file data.csv --filter "age > 25"

3. Filter + Projection

wardenclyffe --file data.csv \
  --filter "country = 'IN'" \
  --select "name,salary"

4. OR / AND logic

wardenclyffe --file data.csv \
  --filter "age > 25 AND salary > 40000"

🧠 Query Language (DSL)

The grammar is defined using Pest in src/query.pest.

Supported grammar:

expr        = or_expr
or_expr     = and_expr ( "OR" and_expr )*
and_expr    = cmp_expr ( "AND" cmp_expr )*
cmp_expr    = field op value
op          = ">" | "<" | "="
value       = number | 'string'

This DSL is transformed into an AST, then evaluated per row.

🛠 Internal Architecture

1. DataSource abstraction

enum DataSource {
    Csv(CsvSource),
    Parquet(ParquetSource),
}

2. Parser (Pest)

Converts the filter string into an AST.

3. AST

enum Expr {
    Cmp { field, op, value },
    And(Box<Expr>, Box<Expr>),
    Or(Box<Expr>, Box<Expr>),
}

4. . Evaluator

Executes the AST against each row.

5. Executor

• Coordinates:

• Row scanning

• Filter evaluation

• Projection

• Timing

Sample Output

run :

wardenclyffe --file data.csv --filter "age > 25"

Row { age: Some(30), name: Some("John"), country: Some("US"), salary: Some(60000) }
Row { age: Some(28), name: Some("Ravi"), country: Some("IN"), salary: Some(52000) }
...
Scanned 10 rows, matched 7 rows in 0.838 ms

📜 License

MIT License © 2025.

🤝 Contributing

PRs, issues, and feature requests are welcome.