sqjson

sqjson is a simple, embedded, file-based key-value database using JSON values and memory-mapped files
(like SQLite, but for JSON). Written in pure Rust with minimal dependencies (serde, memmap2, thiserror).

🚀 Features

• Embedded, single-file storage (.db)
• Fast memory-mapped I/O with memmap2
• Store structured data as JSON using serde_json
• Key-value API: put, get, delete, flush
• Field-level access: get_field(key, field)
• Secondary indexes for field-based queries: query(field, value)
• Range queries for numbers and strings: range_query(field, min, max)
• Filter records using a custom predicate: filter(|val| ...)
• Paginated queries: query_page(field, value, limit, offset)
• Export query results to JSON: export_query(field, value, path)
• Export full database snapshot: export_to_file(path)
• Show all records (show_all) for debugging
• Minimal dependencies (serde, memmap2, thiserror)
• Auto-growing data file (file expands transparently as more pages are written)

📦 Installation

Add to your Rust project with:

cargo add sqjson

Or add manually to your Cargo.toml:

[dependencies]
sqjson = "0.1"

🛠 Usage Examples

use sqjson::{YourDb, DbError};
use serde_json::json;

fn main() -> Result<(), DbError> {
    // Open or create the database
    let mut db = YourDb::open("jsondb.db")?;

    // Insert JSON records
    db.put("user:1", &json!({ "name": "Alice", "age": 30, "city": "NY" }))?;
    db.put("user:2", &json!({ "name": "Bob", "age": 25, "city": "LA" }))?;
    db.put("user:3", &json!({ "name": "Charlie", "age": 30, "city": "NY" }))?;
    db.put("user:4", &json!({ "name": "Diana", "age": 22, "city": "LA" }))?;

    // Persist changes
    db.flush()?;

    // Show all records
    println!("\n-- All Records --");
    db.show_all()?;

    // Get full record by key
    if let Some(user) = db.get("user:2")? {
        println!("\nFound user:2: {}", user);
    }

    // Get a specific JSON field
    if let Some(age) = db.get_field("user:1", "age")? {
        println!("user:1 age is: {}", age);
    }

    // Query by secondary index
    let users_age_30 = db.query("age", 30)?;
    println!("\nUsers with age 30: {:?}", users_age_30);

    // Filter records with a custom predicate
    let older_than_24 = db.filter(|val| val["age"].as_u64().unwrap_or(0) > 24)?;
    println!("\nUsers older than 24:");
    for (key, user) in older_than_24 {
        println!("{} => {}", key, user);
    }

     // Query by field value
    let users_age_30 = db.query("age", 30)?;
    println!("Users with age 30: {:?}", users_age_30);

    // Range query (numeric)
    let users_age_range = db.range_query("age", json!(25), json!(30))?;
    println!("Users with age 25–30: {:?}", users_age_range);


    // Paginated query (first 1 user in NY)
    let first_ny_user = db.query_page("city", "NY", 1, 0)?;
    println!("\nFirst user in NY: {:?}", first_ny_user);

    // Export query results to JSON
    db.export_query("city", "LA", "la_users.json")?;
    println!("Exported LA users to la_users.json");

    // Delete a record
    db.delete("user:3")?;
    println!("\nDeleted user:3");

    // Persist delete operation
    db.flush()?;

    // Export full DB
    db.export_to_file("backup.json")?;
    println!("\nExported full database to backup.json");

    Ok(())
}

🔧 API Overview

📁 File Format

Page 0: Stores the index (mapping keys to page IDs)

Page 1 and onwards: Store actual JSON-encoded data

Fixed page size (default 4096 bytes)

Note: While page size is fixed, the underlying database file now grows automatically in chunks as needed, so you are not limited by the initial file size.

Data stored as length-prefixed JSON blobs

📃 License

MIT OR Apache-2.0

👤 Author

Hafiz Ali Raza