tracing-better-stack

A tracing-subscriber layer for sending logs to Better Stack (formerly Logtail) via their HTTP API.

Features

• 🚀 Asynchronous & Non-blocking: Logs are sent in background without blocking your application
• 📦 Automatic Batching: Efficiently batches logs to reduce HTTP overhead
• 🔄 Retry Logic: Automatic retry with exponential backoff for failed requests
• 🎯 Structured Logging: Full support for structured fields and span context
• 🔒 Feature Flags: Opt-in to only the serialization format you need
• 🛡️ Production Ready: Comprehensive error handling and graceful degradation
• 🗜️ MessagePack Support: Choose between JSON and MessagePack serialization formats
• ⚡ Lazy Initialization: Handles tokio runtime initialization gracefully

Installation

Add this to your Cargo.toml:

[dependencies]
tokio = { version = "1", features = ["rt", "macros"] }
tracing = "0.1"
tracing-subscriber = "0.3"

# Default: uses MessagePack for better performance
tracing-better-stack = "0.1"

# Or explicitly choose MessagePack:
tracing-better-stack = { version = "0.1", features = ["message_pack"] }

# Or use JSON format:
tracing-better-stack = { version = "0.1", default-features = false, features = ["json"] }

Serialization Formats

Better Stack supports both JSON and MessagePack formats for log ingestion. This crate provides both options via mutually exclusive feature flags:

• MessagePack (default): More compact and efficient binary format
• JSON: Human-readable text format, useful for debugging

Using MessagePack (Default)

# This is the default, no special configuration needed
tracing-better-stack = "0.1"

Using JSON

tracing-better-stack = { version = "0.1", default-features = false, features = ["json"] }

Both formats send the same structured data to Better Stack, so you can switch between them without changing your logging code.

Quick Start

use tracing::{info, error};
use tracing_better_stack::{BetterStackLayer, BetterStackConfig};
use tracing_subscriber::prelude::*;

#[tokio::main]
async fn main() {
    // Get your Better Stack configuration from https://logs.betterstack.com/
    // Better Stack provides unique ingesting hosts for each source
    // e.g., "s1234567.us-east-9.betterstackdata.com"
    let ingesting_host = std::env::var("BETTER_STACK_INGESTING_HOST")
        .expect("BETTER_STACK_INGESTING_HOST must be set");
    let source_token = std::env::var("BETTER_STACK_SOURCE_TOKEN")
        .expect("BETTER_STACK_SOURCE_TOKEN must be set");

    // Create the Better Stack layer
    let better_stack_layer = BetterStackLayer::new(
        BetterStackConfig::builder(ingesting_host, source_token).build()
    );

    // Initialize tracing with Better Stack layer
    tracing_subscriber::registry()
        .with(better_stack_layer)
        .init();

    // Now your logs will be sent to Better Stack!
    info!("Application started");
    error!(user_id = 123, "Payment failed");
}

Configuration

The layer can be configured using the builder pattern:

use std::time::Duration;
use tracing_better_stack::{BetterStackLayer, BetterStackConfig};

// Both ingesting host and source token are required
// Better Stack provides unique hosts for each source
let layer = BetterStackLayer::new(
    BetterStackConfig::builder(
        "s1234567.us-east-9.betterstackdata.com",  // Your ingesting host
        "your-source-token"                        // Your source token
    )
        .batch_size(200)                                 // Max events per batch (default: 100)
        .batch_timeout(Duration::from_secs(10))          // Max time between batches (default: 5s)
        .max_retries(5)                                  // Max retry attempts (default: 3)
        .initial_retry_delay(Duration::from_millis(200)) // Initial retry delay (default: 100ms)
        .max_retry_delay(Duration::from_secs(30))        // Max retry delay (default: 10s)
        .include_location(true)                          // Include file/line info (default: true)
        .include_spans(true)                             // Include span context (default: true)
        .build()
);

Advanced Usage

With Console Output

Combine with other layers for both console and Better Stack logging:

use tracing_subscriber::prelude::*;
use tracing_better_stack::{BetterStackLayer, BetterStackConfig};

tracing_subscriber::registry()
    .with(BetterStackLayer::new(
        BetterStackConfig::builder(ingesting_host, source_token).build()
    ))
    .with(
        tracing_subscriber::fmt::layer()
            .with_target(false)
            .compact()
    )
    .init();

With Environment Filter

Control log levels using environment variables:

use tracing_subscriber::{prelude::*, EnvFilter};
use tracing_better_stack::{BetterStackLayer, BetterStackConfig};

tracing_subscriber::registry()
    .with(BetterStackLayer::new(
        BetterStackConfig::builder(ingesting_host, source_token).build()
    ))
    .with(EnvFilter::from_default_env())
    .init();

Structured Logging

Take advantage of tracing's structured logging capabilities:

use tracing::{info, span, Level};

// Log with structured fields
info!(
    user_id = 123,
    action = "purchase",
    amount = 99.99,
    currency = "USD",
    "Payment processed successfully"
);

// Use spans for context
let span = span!(Level::INFO, "api_request",
    endpoint = "/users",
    method = "GET"
);
let _enter = span.enter();

info!("Processing API request");
// All logs within this span will include the span's fields

How It Works

1. Event Collection: The layer captures tracing events and converts them to internal LogEvent structures
2. Batching: Events are collected into batches (up to 100 events or 5 seconds)
3. Serialization: Batches are serialized to either MessagePack (default) or JSON format
4. Async Sending: Batches are sent asynchronously via HTTPS to Better Stack
5. Retry Logic: Failed requests are retried with exponential backoff
6. Graceful Degradation: If Better Stack is unreachable, logs are dropped without affecting your application

Log Format

Logs are sent to Better Stack with the following structure:

{
  "timestamp": "2024-01-20T12:34:56.789Z",
  "level": "info",
  "message": "Your log message",
  "target": "your_module_name",
  "location": {
    "file": "src/main.rs",
    "line": 42
  },
  "fields": {
    "user_id": 123,
    "custom_field": "value"
  },
  "span": {
    "name": "request",
    "fields": {
      "request_id": "abc123"
    }
  }
}

This structure is preserved in both JSON and MessagePack formats.

Performance Considerations

• MessagePack vs JSON: MessagePack is ~30-50% more compact than JSON, reducing network overhead
• Zero-cost when unreachable: If Better Stack is down, the layer fails open with minimal overhead
• Efficient batching: Reduces network calls by batching multiple events
• Non-blocking: All network I/O happens in background tasks
• Lazy initialization: Tokio runtime is only accessed when needed

Need synchronous flushing?

Add a delay before program exit to ensure final logs are sent:

// At the end of main()
tokio::time::sleep(Duration::from_secs(2)).await;

Examples

Check out the examples directory for more usage patterns:

• basic.rs - Simple usage with environment variables
• with_filters.rs - Using with EnvFilter
• custom_config.rs - Advanced configuration

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.