Graph_D - Native Graph Database in Rust

A high-performance, memory-efficient native graph database implementation in Rust with built-in JSON support and ACID compliance.

🎯 Vision

Build a production-ready, memory-efficient native graph database that leverages Rust's safety guarantees and performance characteristics to provide:

• Native storage with index-free adjacency for O(1) traversal performance
• First-class JSON support for flexible node and relationship properties
• Memory efficiency comparable to SQLite for embedded applications
• Thread-safe, ACID-compliant operations with minimal overhead

✨ Features

Core Database Features

• 🚀 High Performance: 140K+ nodes/sec creation, 32M+ lookups/sec
• 💾 Memory Efficient: ~1.9KB per node including relationships and indexes
• 🔄 ACID Compliance: Full transaction support with multiple isolation levels
• 🔒 Concurrent Safe: Advanced locking with deadlock detection
• 📁 Persistent Storage: Memory-mapped files for durability

Query & Analytics

• 🔍 Advanced Queries: Traversal, filtering, aggregation, sorting
• 📊 Rich Aggregations: Count, sum, avg, min, max, group by, statistics
• 📄 Pagination: Efficient offset/limit support with sorting
• 🎯 Path Finding: BFS shortest path algorithms

Developer Experience

• 🦀 Rust Native: Zero-cost abstractions and memory safety
• 📖 Rich Documentation: Comprehensive examples and API docs
• 🧪 Well Tested: 32+ tests covering all major functionality
• ⚡ Async Ready: Built with tokio for async/await support

🚀 Quick Start

Installation

Add to your Cargo.toml:

[dependencies]
graph_d = "0.1.0"

Basic Usage

use graph_d::{Graph, Result};
use serde_json::json;

fn main() -> Result<()> {
    // Create a new in-memory graph
    let mut graph = Graph::new()?;
    
    // Create nodes with JSON properties
    let alice_id = graph.create_node([
        ("name".to_string(), json!("Alice")),
        ("age".to_string(), json!(30)),
        ("role".to_string(), json!("Engineer")),
    ].into())?;
    
    let bob_id = graph.create_node([
        ("name".to_string(), json!("Bob")),
        ("age".to_string(), json!(25)),
        ("role".to_string(), json!("Designer")),
    ].into())?;
    
    // Create relationships
    let rel_id = graph.create_relationship(
        alice_id,
        bob_id,
        "WORKS_WITH".to_string(),
        [("since".to_string(), json!("2023"))].into(),
    )?;
    
    // Query the graph
    if let Some(alice) = graph.get_node(alice_id)? {
        println!("Alice: {:?}", alice.properties);
    }
    
    // Find Alice's relationships
    let relationships = graph.get_relationships_for_node(alice_id)?;
    println!("Alice has {} relationships", relationships.len());
    
    Ok(())
}

💻 Command Line Interface

Graph_D also provides a standalone CLI binary for interactive database exploration, similar to sqlite3.

CLI Installation

# Install from crates.io (includes CLI)
cargo install graph_d --features cli

# Or build from source
cargo build --release --features cli

Interactive Shell

# Start with in-memory database
graph_d

# Open or create a database file
graph_d mydb.graphd

# Execute a single query and exit
graph_d mydb.graphd -c "MATCH (n:Person) RETURN n"

# Run queries from a script file
graph_d mydb.graphd -f queries.gql

Output Formats

# Table format (default) - human-readable
graph_d -c "MATCH (n) RETURN n LIMIT 5" -o table

# JSON format - for programmatic processing
graph_d -c "MATCH (n) RETURN n" -o json

# CSV format - for data export
graph_d -c "MATCH (n) RETURN n.name, n.age" -o csv

Shell Commands

When in interactive mode, these commands are available:

Example Session

$ graph_d mydb.graphd
Graph_D 0.1.0 - Type .help for help, .exit to exit

graph_d> CREATE (n:Person {name: 'Alice', age: 30});
1 row returned

graph_d> CREATE (n:Person {name: 'Bob', age: 25});
1 row returned

graph_d> MATCH (n:Person) RETURN n.name, n.age;
name  | age
------+----
Alice | 30
Bob   | 25
2 rows returned

graph_d> .exit
Goodbye!

📊 Performance Benchmarks

Tested on modern hardware with realistic workloads:

Scalability Results

• 100K nodes: Created in ~670ms
• 1M node target: Projected <7 seconds
• Memory efficiency: <1GB for 1M documents target achieved

🔍 Advanced Queries

Aggregations

use graph_d::query::{QueryBuilder, AggregateFunction};

let all_employees: Vec<_> = (1..=100).collect();
let query = QueryBuilder::new(&graph, all_employees);

// Count employees
let count = query.aggregate(AggregateFunction::Count)?;

// Average salary
let avg_salary = query.aggregate(AggregateFunction::Avg("salary".to_string()))?;

// Group by department
let groups = query.aggregate(AggregateFunction::GroupBy("department".to_string()))?;

Sorting & Pagination

use graph_d::query::{SortCriteria, Pagination};

// Sort by multiple criteria
let sorted = QueryBuilder::new(&graph, all_nodes)
    .sort(vec![
        SortCriteria::asc("department"),
        SortCriteria::desc("salary"),
    ])?
    .nodes()?;

// Paginated results
let page = query.sorted_page(
    vec![SortCriteria::asc("name")],
    Pagination::new(0, 10)
)?;

Graph Traversal

use graph_d::query::QueryBuilder;

// Find friends of friends
let friends_of_friends = QueryBuilder::from_node(&graph, alice_id)
    .outgoing("FRIENDS_WITH")?
    .outgoing("FRIENDS_WITH")?
    .nodes()?;

// Filter by properties
let senior_engineers = QueryBuilder::new(&graph, all_employees)
    .filter_by_property("department", &json!("Engineering"))?
    .filter_by_property("level", &json!("Senior"))?
    .nodes()?;

🔒 Concurrent Transactions

use graph_d::transaction::{TransactionManager, IsolationLevel, LockableResource};

let tx_manager = TransactionManager::new(IsolationLevel::ReadCommitted);

// Create concurrent transaction
let mut tx = tx_manager.begin_concurrent();

// Acquire locks
tx.read_lock(LockableResource::Node(1))?;
tx.write_lock(LockableResource::Node(2))?;

// Perform operations...

// Commit (automatically releases locks)
tx.commit()?;

💾 Persistent Storage

// Create persistent database
let mut graph = Graph::open("my_graph.db")?;

// Use normally...
graph.create_node(properties)?;

// Flush to disk
graph.storage.flush()?;

// Data persists across restarts
let graph2 = Graph::open("my_graph.db")?;

🏗️ Architecture

Storage Layer

• Memory-mapped files for efficient I/O
• Fixed-size records for predictable performance
• Index-free adjacency for O(1) traversals
• String interning for memory deduplication

Query Engine

• Fluent API for complex queries
• Lazy evaluation where possible
• Type-aware sorting for mixed data types
• Statistical functions for analytics

Transaction System

• MVCC with optimistic concurrency
• Deadlock detection via wait-for graphs
• Multiple isolation levels (Read Uncommitted → Serializable)
• Automatic lock management

📚 Examples

The examples/ directory contains comprehensive demonstrations:

Rust Library Examples

• getting_started.rs - Basic usage and quick start
• persistent_storage.rs - File-based persistence
• advanced_queries.rs - Complex query operations
• concurrent_transactions.rs - Multi-threaded usage
• gql_demo.rs - GQL query language examples
• memory_management.rs - Memory allocation patterns
• performance_test.rs - Benchmarking and profiling

CLI Examples

• cli_scripting.gql - GQL script file for batch operations
• cli_automation.sh - Shell script for CLI automation

Run Rust examples with:

cargo run --example getting_started
cargo run --example persistent_storage
cargo run --example concurrent_transactions

Run CLI examples with:

# Build CLI first
cargo build --features cli

# Run GQL script
./target/debug/graph_d mydb.graphd -f examples/cli_scripting.gql

# Run automation script
chmod +x examples/cli_automation.sh
./examples/cli_automation.sh

🧪 Testing

Run the comprehensive test suite:

# All tests
cargo test

# Benchmarks
cargo bench

# With output
cargo test -- --nocapture

🎯 Production Readiness

Current Status

• ✅ Core graph operations (CRUD)
• ✅ Advanced query engine with aggregations
• ✅ Concurrent transaction support with ACID compliance
• ✅ Multi-layered indexing system (Property, Range, Composite, Relationship)
• ✅ Memory-mapped persistence foundation
• ✅ GQL (Graph Query Language) parser, lexer, and executor
• ✅ Comprehensive test coverage (32+ tests)
• ✅ Performance benchmarks and optimization
• ✅ Complete rustdoc documentation with examples for all APIs

Roadmap

• 🔄 Performance Benchmarking Framework - Automated regression testing
• 📋 Schema Validation - Optional type constraints
• 📋 Enhanced Persistence - Complete crash recovery and integrity verification
• 📋 Clustering - Multi-node deployment support
• 📋 Backup/Restore - Data migration utilities

🤝 Contributing

Contributions are welcome! Please read our Contributing Guide and follow the Code of Conduct.

Development Setup

git clone https://github.com/your-org/graph_d
cd graph_d
cargo test
cargo run --example advanced_queries

📄 License

Licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT License (LICENSE-MIT)

at your option.

🙏 Acknowledgments

Built with ❤️ using:

• tokio - Async runtime
• parking_lot - High-performance synchronization
• serde_json - JSON serialization
• memmap2 - Memory mapping
• criterion - Benchmarking

Graph_D - Native graph database performance with Rust reliability. 🦀📊