WIP: this crate has not been tested yet, still in actively development.

Generic purpose append only log implementation.

English | 简体中文

Introducation

When developing infrastructure softwares, write-ahead log or append-only log plays an important role, and people re-implement same funcationalities multiple times, but actually, the core for append-only log is just write, replay, and rewrite.

This crate provides generic purpose append-only log implementation, there are three kinds of implementations based on std::fs::File, memory map, lockfree ARENA (SkipMap).

• aol::fs::AppendLog:

  Generic append-only log implementation based on std::fs::File.

  • It is good for:

    • The encoded entry size is smaller than 64 bytes.
    • Manifest file.
    • Write is not too frequently.

  • Pros:

    • It is growable, do not require pre-allocated.
    • Support automatically rewrite.
    • No holes in the file.

• aol::memmap::AppendLog:

  Generic append-only log implementation based on memmap.

  • It is good for:

    • Any append-only log, if you do not care about pre-allocating the file and you know you data will never larger than the pre-allocating size.

  • Pros:

    • Support automatic rewrite.
    • No holes in the file.
    • As this implementation is backed by an ARENA, no allocation required for both read and write.
    • Fast read and write performance, backed by memory map, no extra I/O required.

• aol::memmap::sync::AppendLog:

  Generic append-only log implementation based on lockfree ARENA based skl::SkipMap (support both in-memory and on-disk).

  • It is good for:

    • Append-only log which size cannot reach 4GB.
    • End-users who can manage the grow and rewrite by themselves.

  • Pros:

    • As this implementation is backed by an ARENA, no allocation required for both read and write.
    • Fast read and write performance, backed by memory map, no extra I/O required.
    • Lock-free and concurrent-safe.
    • Support both in-memory and on-disk.
    • Can be used in no_std environment.

Installation

[dependencies]
aol = "0.0.0"

License

aol is under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE, LICENSE-MIT for details.

Copyright (c) 2024 Al Liu.