wattle-appender

A feature-rich file appender for the tracing framework, providing flexible log rotation, compression, and both blocking and non-blocking write modes.

Features

• Flexible rotation strategies: Time-based (minutely, hourly, daily, weekly, monthly, yearly) and size-based rotation
• Dual write modes: Blocking for simplicity, non-blocking for high-performance async applications
• Compression support: Gzip, Zstd, XZ, and Zip compression for rotated log files
• Automatic cleanup: Limit retention by file count or age
• Symlink support: Automatically maintains a symlink pointing to the latest log file
• Smart file naming: Automatically generates timestamped and sequenced file names
• Thread-safe: Safe for concurrent use across multiple threads

Installation

Add this to your Cargo.toml:

[dependencies]
wattle-appender = "0.1"

Feature Flags

• default: Basic file appending with blocking mode only
• non-blocking: Enables async writing using crossbeam channels
• compression-gzip: Gzip compression support (via flate2)
• compression-zstd: Zstd compression support (via zstd)
• compression-xz: XZ compression support (via xz2)
• compression-zip: Zip compression support (via zip)
• full: Enables non-blocking and compression-zstd

To use all features:

[dependencies]
wattle-appender = { version = "0.1", features = ["full"] }

Quick Start

Basic Usage

use wattle_appender::FileAppender;
use std::io::Write;

fn main() -> std::io::Result<()> {
    let mut appender = FileAppender::new()
        .file_name("logs/app.log")
        .build()?;

    writeln!(appender, "Hello, world!")?;
    Ok(())
}

With Daily Rotation

use wattle_appender::FileAppender;

let appender = FileAppender::new()
    .file_name("logs/app.log")
    .daily_rotation()
    .max_backup(7)  // Keep last 7 days
    .build()
    .unwrap();

With Size-Based Rotation

use wattle_appender::FileAppender;

let appender = FileAppender::new()
    .file_name("logs/app.log")
    .size_limit(10 * 1024 * 1024)  // 10 MB
    .max_backup(5)  // Keep 5 backup files
    .build()
    .unwrap();

With Compression

use wattle_appender::FileAppender;

let appender = FileAppender::new()
    .file_name("logs/app.log")
    .daily_rotation()
    .zstd_compression()  // Requires "compression-zstd" feature
    .max_backup(30)
    .build()
    .unwrap();

Non-Blocking Mode

use wattle_appender::FileAppender;

// Requires "non-blocking" feature
let appender = FileAppender::new()
    .file_name("logs/app.log")
    .blocking(false)
    .daily_rotation()
    .build()
    .unwrap();

Integration with tracing

use tracing_subscriber::fmt;
use wattle_appender::FileAppender;

fn main() {
    let appender = FileAppender::new()
        .file_name("logs/app.log")
        .daily_rotation()
        .max_backup(7)
        .build()
        .unwrap();

    tracing_subscriber::fmt()
        .with_writer(appender)
        .init();

    tracing::info!("Application started");
}

File Naming Rules

The appender uses intelligent file naming based on your rotation configuration:

1. No Rotation (default)

When rotation is not configured, logs are written directly to the specified file:

FileAppender::new()
    .file_name("logs/app.log")
    .build()

Result: logs/app.log

2. Time-Based Rotation Only

When only time-based rotation is enabled, files are named with timestamps:

FileAppender::new()
    .file_name("logs/app.log")
    .daily_rotation()
    .build()

Files created:

• logs/app.log → Symlink to the current log file
• logs/app.log.2024-01-15_10-30-00.log → Actual log file with timestamp

Pattern: {file_name}.{YYYY-MM-DD_HH-MM-SS}.log

3. Size-Based Rotation Only

When only size-based rotation is enabled, files are numbered sequentially:

FileAppender::new()
    .file_name("logs/app.log")
    .size_limit(10 * 1024 * 1024)
    .build()

Files created:

• logs/app.log → Symlink to the current log file
• logs/app.log.1.log → First log file
• logs/app.log.2.log → Second log file (after size limit reached)
• logs/app.log.3.log → Third log file

Pattern: {file_name}.{N}.log

4. Combined Time and Size Rotation

When both time and size rotation are enabled, files include both timestamp and sequence number:

FileAppender::new()
    .file_name("logs/app.log")
    .daily_rotation()
    .size_limit(10 * 1024 * 1024)
    .build()

Files created:

• logs/app.log → Symlink to the current log file
• logs/app.log.2024-01-15_10-30-00.1.log → First file of the day
• logs/app.log.2024-01-15_10-30-00.2.log → Second file (after size limit)
• logs/app.log.2024-01-16_08-15-30.1.log → Next day's first file

Pattern: {file_name}.{YYYY-MM-DD_HH-MM-SS}.{N}.log

Note: Sequence numbers reset to 1 when a new time period begins.

5. With Compression

When compression is enabled, rotated files are automatically compressed:

FileAppender::new()
    .file_name("logs/app.log")
    .daily_rotation()
    .zstd_compression()
    .build()

Files created:

• logs/app.log → Symlink to current log file
• logs/app.log.2024-01-15_10-30-00.log.zst → Compressed rotated file

Compression extensions:

• Gzip: .gz
• Zstd: .zst
• XZ: .xz
• Zip: .zip

API Reference

FileAppender::new()

Creates a new builder for configuring the appender.

Builder Methods

File Configuration

• file_name(path: impl AsRef<Path>) - Set the base file path (required)
  • When rotation is disabled: This is the actual log file
  • When rotation is enabled: This becomes a prefix and symlink name

Write Mode

• blocking(bool) - Set blocking/non-blocking mode (default: true)
  • Requires non-blocking feature for false value

Time-Based Rotation

• minutely_rotation() - Rotate every minute
• hourly_rotation() - Rotate every hour
• daily_rotation() - Rotate daily at midnight
• weekly_rotation() - Rotate weekly
• monthly_rotation() - Rotate monthly
• yearly_rotation() - Rotate yearly
• never_rotation() - Disable time-based rotation (default)

Size-Based Rotation

• size_limit(bytes: u64) - Set maximum file size in bytes before rotation

Cleanup Options

• max_backup(count: usize) - Maximum number of backup files to keep
• max_age_days(days: usize) - Maximum age of backup files in days

Compression

• compress(bool) - Enable/disable compression (default: false)
• gzip_compression() - Use Gzip compression (requires compression-gzip feature)
• zstd_compression() - Use Zstd compression (requires compression-zstd feature)
• xz_compression() - Use XZ compression (requires compression-xz feature)
• zip_compression() - Use Zip compression (requires compression-zip feature)

Note: Calling any *_compression() method automatically enables compression.

Symlink

• symlink_latest(bool) - Enable/disable symlink to latest log (default: true)

Build

• build() - Construct the FileAppender (returns io::Result<FileAppender>)

Advanced Examples

Complete Configuration

use wattle_appender::FileAppender;

let appender = FileAppender::new()
    .file_name("logs/myapp.log")
    .blocking(false)              // Non-blocking mode
    .daily_rotation()             // Rotate daily
    .size_limit(50 * 1024 * 1024) // Also rotate at 50 MB
    .max_backup(30)               // Keep 30 backup files
    .max_age_days(90)             // Delete files older than 90 days
    .zstd_compression()           // Compress rotated files
    .symlink_latest(true)         // Create symlink to current file
    .build()
    .expect("Failed to create appender");

Multi-Threaded Usage

use std::sync::Arc;
use std::thread;
use wattle_appender::FileAppender;
use std::io::Write;

let appender = Arc::new(
    FileAppender::new()
        .file_name("logs/app.log")
        .daily_rotation()
        .build()
        .unwrap()
);

let mut handles = vec![];

for i in 0..10 {
    let appender_clone = Arc::clone(&appender);
    let handle = thread::spawn(move || {
        for j in 0..100 {
            let mut app = appender_clone.clone();
            writeln!(app, "Thread {} - Message {}", i, j).unwrap();
        }
    });
    handles.push(handle);
}

for handle in handles {
    handle.join().unwrap();
}

Hourly Rotation with Gzip

use wattle_appender::FileAppender;

let appender = FileAppender::new()
    .file_name("logs/hourly.log")
    .hourly_rotation()
    .gzip_compression()  // Requires "compression-gzip" feature
    .max_backup(24)      // Keep 24 hours
    .build()
    .unwrap();

Size-Only Rotation

use wattle_appender::FileAppender;

// Rotate only when file reaches 5 MB
let appender = FileAppender::new()
    .file_name("logs/size-based.log")
    .size_limit(5 * 1024 * 1024)
    .max_backup(10)
    .build()
    .unwrap();

How It Works

Rotation Trigger

The appender checks for rotation on every write operation:

1. Time-based: Compares current time with last rotation time
2. Size-based: Checks if current file size exceeds the limit
3. Combined: Rotation occurs if either condition is met

Rotation Process

When rotation is triggered:

1. Current file is closed (and compressed if enabled)
2. A new file is created with appropriate timestamp/sequence number
3. Symlink is updated to point to the new file
4. Old files are cleaned up based on max_backup and max_age_days

Cleanup Strategy

The cleanup process:

1. Scans directory for matching log files
2. Excludes the current file and symlink
3. Removes files older than max_age_days (if set)
4. Keeps only the most recent max_backup files (if set)
5. Sorts by modification time (oldest deleted first)

Thread Safety

• All write operations are protected by mutex locks
• Multiple threads can share the same FileAppender instance
• In non-blocking mode, a background worker thread handles actual writes

Performance Considerations

Blocking vs Non-Blocking

Blocking Mode:

• Simple and predictable
• Write operations block the calling thread
• Suitable for most applications

Non-Blocking Mode:

• Writes are sent to a channel and handled by a background thread
• Calling thread doesn't wait for disk I/O
• Ideal for high-throughput applications
• Small memory overhead for the channel buffer (1024 messages)

Compression Trade-offs

Compression reduces disk space but adds CPU overhead:

Algorithm	Speed	Compression Ratio	Best For
Gzip	Fast	Good	General use
Zstd	Very Fast	Excellent	Best balance
XZ	Slow	Best	Maximum compression
Zip	Fast	Good	Windows compatibility

Recommendation: Use Zstd for the best balance of speed and compression.

Limitations

• Unix-only symlinks: Symlink functionality requires Unix-like systems
• Non-atomic rotation: Brief write blocking occurs during rotation
• No log level filtering: Use tracing-subscriber's filtering instead
• Compression on rotation: Only rotated files are compressed, not the active file

Examples

Check the examples/ directory for more complete examples:

• basic_usage.rs - Simple file appender setup
• rotation_demo.rs - Demonstrates different rotation strategies
• advanced_usage.rs - Complete configuration with all features
• tracing_integration.rs - Integration with tracing framework

Run an example:

cargo run --example basic_usage
cargo run --example rotation_demo --features full

Testing

Run tests with all features:

cargo test --all-features

Run tests for specific features:

cargo test --features non-blocking
cargo test --features compression-zstd

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• Built for the tracing ecosystem
• Inspired by various logging solutions in the Rust community