walr

Ergonomic Write Ahead Log(WAL) using tokio

Requirements

• Rust

Installation

• cargo add walr

Usage

A WAL has three generics:

• Log: The type of the Log to write to disk

• State: The state wanting to be persisted

• ReplayContext: Optional context that can be provided to WAL::replay to use inside the replay_handler

Generally, you construct a WAL, call WAL::replay to replay the State from disk, start applying logs via WAL::log and WAL::log_many; then when WAL::should_checkpoint is true, you checkpoint the in-memory State via WAL::checkpoint. If your mutable action on State in not infallible, WAL::undo will undo the last log or log many operation.

Example (Persisting a basic Cache)

Log = CacheLog

State = Cache

ReplayContext = ()

type Cache = HashMap<String, String>;

#[derive(Serialize, Deserialize)]
enum CacheLog {
    Remove(String),
    Insert { key: String, value: String },
}

type ReplayContext = ();

Where the Log type captures all enumerates of mutable actions on State. Then, using these logs through replay with a replay handler, you can replay the State with the option to provide ReplayContext to aid in the replay process.

fn replay_handler(
    logs: Vec<CacheLog>,
    checkpoint: Arc<Mutex<Cache>>,
    context: Option<ReplayContext>,
) -> ReplayHandlerResult {
    Box::pin(async move {
        let mut checkpoint = checkpoint.lock().await;

        for log in logs {
            match log {
                CacheLog::Insert { key, value } => checkpoint.insert(key, value),
                CacheLog::Remove(key) => checkpoint.remove(&key),
            };
        }

        Ok(())
    })
}

See exmaple.rs for the full example.

See Full crate documentation here at docs.rs for all available functionality.

Roadmap

• Optimizations

Contributing

Open to contributions, please just create a PR/issue or reach out to me to discuss what you would like to add/change.

License

MIT License

Copyright (c) 2024 Robert Lopez

See LICENSE.md