brk_store

Blockchain-aware key-value storage wrapper for Bitcoin data

brk_store is a thin, typed wrapper around the Fjall embedded key-value store, specifically designed for Bitcoin blockchain data storage. It provides height-based versioning, batch operations, and optimized configurations for Bitcoin data patterns.

What it provides

• Blockchain-aware storage: Built-in height tracking and versioning for Bitcoin block processing
• Type-safe interface: Generic over key and value types with automatic serialization
• Batch operations: Efficient batched inserts/deletes with transactional support
• Metadata management: Automatic version checking and height progression tracking
• Performance optimization: Pre-configured for Bitcoin data patterns

Key Features

Height-Based Processing

• Height tracking: Each store tracks the last processed blockchain height
• Conditional operations: Insert only when blockchain height requires it
• Automatic versioning: Combines Fjall version + user version for compatibility

Batch Operations

• In-memory staging: Changes staged before atomic commit
• Transactional safety: All changes written atomically or none at all
• Efficient processing: Minimizes disk I/O through batching

Performance Optimization

• Optimized configurations: 32MB write buffers, 8MB memtables for Bitcoin data
• Configurable bloom filters: Optional for faster key lookups
• Memory efficiency: Reused read transactions, minimal overhead

Usage

Basic Operations

use brk_store::{Store, open_keyspace};
use brk_structs::{Height, Version};

// Open keyspace
let keyspace = open_keyspace(Path::new("./data"))?;

// Create typed store
let mut store: Store<String, u64> = Store::import(
    &keyspace,
    Path::new("./data"),
    "store_name",
    Version::ZERO,
    Some(true), // Enable bloom filters
)?;

// Conditional insertion based on height
store.insert_if_needed("key1".to_string(), 42u64, Height::new(800_000));

// Batch commit changes
store.commit(Height::new(800_000))?;

// Persist to disk
store.persist()?;

Querying Data

// Get value (checks pending puts first, then disk)
if let Some(value) = store.get(&"key1".to_string())? {
    println!("Value: {}", value);
}

// Check if store needs processing at height
if store.needs(Height::new(800_000)) {
    // Process this height
}

Managing Multiple Stores

use brk_store::AnyStore;

fn process_stores(stores: &mut [Box<dyn AnyStore>]) -> brk_error::Result<()> {
    let height = Height::new(800_000);
    
    for store in stores {
        if store.needs(height) {
            // Process data for this store
            store.commit(height)?;
        }
    }
    
    // Persist all stores
    for store in stores {
        store.persist()?;
    }
    
    Ok(())
}

Store Reset and Reindexing

// Reset store for complete reindexing
store.reset()?;

// Store automatically recreated on next import
let mut fresh_store: Store<String, u64> = Store::import(
    &keyspace,
    path,
    "store_name", 
    Version::ZERO,
    Some(true)
)?;

Store Lifecycle

1. Import: Create or open existing store with version checking
2. Insert: Add key-value pairs with height-based conditional insertion
3. Commit: Write batched changes to disk atomically
4. Persist: Force sync all data to storage
5. Reset: Clear all data for reindexing if needed

Type Requirements

Keys and values must implement:

• Debug + Clone + From<ByteView> + Ord for keys
• Debug + Clone + From<ByteView> for values

This enables automatic serialization and type-safe storage operations.

Version Management

• Automatic migration: Detects version mismatches and resets stores automatically
• File-based metadata: Stores version and height info for persistence
• Compatibility checking: Prevents data corruption from version incompatibilities

Dependencies

• fjall - Embedded key-value store engine
• byteview - Zero-copy byte view operations
• brk_structs - Bitcoin-aware type system
• brk_error - Unified error handling

This README was generated by Claude Code