tower-http-cache

Tower middleware for HTTP response caching with pluggable storage backends (in-memory, Redis, and more). tower-http-cache brings a production-grade caching layer to Tower/Axum/Hyper stacks, with stampede protection, stale-while-revalidate, header allowlisting, compression, and policy controls out of the box.

Features at a Glance

• ✅ Drop-in CacheLayer: wrap any Tower service; caches GET/HEAD by default.
• 🔒 Stampede protection: deduplicates concurrent misses and serves stale data while recomputing.
• ⏱ Flexible TTLs: positive/negative TTL, refresh-before-expiry window, stale-while-revalidate.
• 🔄 Auto-refresh: proactively refreshes frequently-accessed cache entries before expiration.
• 🎬 Chunk Caching: memory-efficient caching for large files with range request support.
• 🏷️ Cache Tags: group and invalidate related cache entries together.
• 🎯 Multi-Tier: hybrid L1/L2 caching for optimal performance and capacity.
• 📊 Admin API: REST endpoints for cache introspection and management.
• 🤖 ML-Ready Logging: structured logs with request correlation for ML training.
• 📦 Pluggable storage: in-memory (Moka), Redis, and Memcached backends with connection pooling.
• 📏 Policy guards: min/max body size, cache-control respect/override, custom method/status filters.
• 🧰 Custom keys: built-in extractors (path, path+query) plus custom closures.
• 📉 Observability hooks: optional metrics counters and tracing spans.

Installation

[dependencies]
tower-http-cache = "0.3"

# Enable Redis support if required
tower-http-cache = { version = "0.3", features = ["redis-backend"] }

# With admin API support
tower-http-cache = { version = "0.3", features = ["admin-api"] }

Quick Start

use std::time::Duration;
use tower::ServiceBuilder;
use tower_http_cache::prelude::*;

let cache_layer = CacheLayer::builder(InMemoryBackend::new(10_000))
    .ttl(Duration::from_secs(120))
    .negative_ttl(Duration::from_secs(10))
    .stale_while_revalidate(Duration::from_secs(30))
    .refresh_before(Duration::from_secs(5))
    .min_body_size(Some(1024))
    .max_body_size(Some(256 * 1024))
    .respect_cache_control(true)
    .build();

let svc = ServiceBuilder::new()
    .layer(cache_layer)
    .service(tower::service_fn(|_req| async {
        Ok::<_, std::convert::Infallible>(http::Response::new("hello world"))
    }));

Chunk Caching for Large Files

Efficiently cache and serve large files with byte-range support - perfect for video streaming:

use tower_http_cache::prelude::*;
use tower_http_cache::streaming::StreamingPolicy;
use std::time::Duration;

let cache_layer = CacheLayer::builder(InMemoryBackend::new(500))
    .policy(
        CachePolicy::default()
            .with_ttl(Duration::from_secs(3600))
            .with_streaming_policy(StreamingPolicy {
                enable_chunk_cache: true,
                chunk_size: 1024 * 1024,         // 1MB chunks
                min_chunk_file_size: 5 * 1024 * 1024, // Only chunk files >= 5MB
                ..Default::default()
            })
    )
    .build();

Benefits:

• 90% memory reduction for large file workloads
• Instant seeking for video streaming (no re-download)
• Range requests served directly from memory
• Only cache accessed chunks (partial file caching)

Example: See examples/chunk_cache_demo.rs for a complete working example.

Using the Redis backend

use std::time::Duration;
use tower_http_cache::prelude::*;

async fn build_redis_layer(redis_url: &str) -> CacheLayer<RedisBackend> {
    let client = redis::Client::open(redis_url).expect("valid Redis URL");
    let manager = client.get_tokio_connection_manager().await.expect("connect");

    CacheLayer::builder(RedisBackend::new(manager))
        .ttl(Duration::from_secs(30))
        .stale_while_revalidate(Duration::from_secs(10))
        .build()
}

Enabling Auto-Refresh

Auto-refresh proactively refreshes frequently-accessed cache entries before they expire, reducing cache misses and latency for hot endpoints:

use std::time::Duration;
use tower_http_cache::prelude::*;
use tower_http_cache::refresh::AutoRefreshConfig;

let cache_layer = CacheLayer::builder(InMemoryBackend::new(10_000))
    .ttl(Duration::from_secs(120))
    .refresh_before(Duration::from_secs(30))
    .auto_refresh(AutoRefreshConfig {
        enabled: true,
        min_hits_per_minute: 10.0,
        check_interval: Duration::from_secs(10),
        max_concurrent_refreshes: 5,
        ..Default::default()
    })
    .build();

// Initialize auto-refresh with the service instance
cache_layer.init_auto_refresh(my_service.clone()).await?;

Using Cache Tags

Group related cache entries and invalidate them together:

use tower_http_cache::prelude::*;
use tower_http_cache::tags::TagPolicy;

let cache_layer = CacheLayer::builder(backend)
    .policy(
        CachePolicy::default()
            .with_tag_policy(TagPolicy::new().with_enabled(true))
            .with_tag_extractor(|method, uri| {
                // Extract tags from request
                vec!["user:123".to_string(), "posts".to_string()]
            })
    )
    .build();

// Later: invalidate all entries with a tag
backend.invalidate_by_tag("user:123").await?;
backend.invalidate_by_tags(&["user:123", "posts"]).await?;

Multi-Tier Caching

Combine fast in-memory cache with larger distributed storage:

use tower_http_cache::backend::MultiTierBackend;

let backend = MultiTierBackend::builder()
    .l1(InMemoryBackend::new(1_000))        // Hot data (fast)
    .l2(RedisBackend::new(manager))          // Cold storage (large)
    .promotion_threshold(3)                   // Promote after 3 L2 hits
    .promotion_strategy(PromotionStrategy::HitCount)
    .write_through(true)
    .build();

let cache_layer = CacheLayer::builder(backend)
    .ttl(Duration::from_secs(300))
    .build();

Smart Streaming & Large File Handling

Automatically prevent large files from overwhelming your cache:

use tower_http_cache::streaming::StreamingPolicy;

let cache_layer = CacheLayer::builder(backend)
    .policy(
        CachePolicy::default()
            .with_streaming_policy(StreamingPolicy {
                enabled: true,
                max_cacheable_size: Some(1024 * 1024), // 1MB limit
                excluded_content_types: HashSet::from([
                    "application/pdf".to_string(),
                    "video/*".to_string(),
                    "audio/*".to_string(),
                    "application/zip".to_string(),
                ]),
                ..Default::default()
            })
    )
    .build();

Features:

• Automatic early detection via Content-Length and size_hint()
• Content-Type based filtering (skip PDFs, videos, archives by default)
• Protects multi-tier caches (large files excluded from L1)
• Prevents memory exhaustion from large response bodies
• Fully configurable per content-type and size

Admin API

Enable cache introspection and management endpoints:

use tower_http_cache::admin::{AdminConfig, admin_router};

let admin_config = AdminConfig::builder()
    .require_auth(true)
    .auth_token("your-secret-token")
    .build();

// Mount admin routes (Axum example)
let admin_routes = admin_router(backend.clone(), admin_config);
let app = Router::new()
    .nest("/admin/cache", admin_routes)
    .layer(cache_layer);

// Available endpoints:
// GET  /admin/cache/health
// GET  /admin/cache/stats
// GET  /admin/cache/hot-keys
// GET  /admin/cache/tags
// POST /admin/cache/invalidate

ML-Ready Structured Logging

Enable structured logging for ML model training:

use tower_http_cache::logging::MLLoggingConfig;

let cache_layer = CacheLayer::builder(backend)
    .policy(
        CachePolicy::default()
            .with_ml_logging(MLLoggingConfig {
                enabled: true,
                sample_rate: 1.0,        // Log 100% of operations
                hash_keys: true,          // Hash keys for privacy
                include_request_id: true, // Correlate with X-Request-ID
            })
    )
    .build();

// Logs will be emitted in JSON format:
// {
//   "timestamp": "2025-11-10T12:00:00Z",
//   "request_id": "550e8400-...",
//   "operation": "cache_hit",
//   "latency_us": 150,
//   "tags": ["user:123"],
//   "tier": "l1"
// }

Configuration Highlights

For the full API surface, see the generated docs: cargo doc --open.

Benchmarks

Benchmarks are powered by Criterion and can be reproduced with:

cargo bench --bench cache_benchmarks

Latest results (macOS / M3 Pro / Rust 1.85, redis-backend disabled unless noted):

Full raw output, including outlier analysis, is captured in initial_benchmark.md.

Testing & Tooling

# Library unit tests + integration tests
cargo test

# Redis integration tests
REDIS_URL=redis://127.0.0.1:6379/ cargo test --features redis-backend --tests redis_example

# Redis smoke test (launches example service, verifies cache hit/miss behaviour)
docker compose -f docker-compose.redis.yml up -d redis
python3 scripts/redis_smoke.py
docker compose -f docker-compose.redis.yml down

# Examples
cargo run --example axum_basic --features middleware
cargo run --example axum_custom --features middleware
cargo run --example redis_smoke --features redis-backend

Feature Flags

Minimum Supported Rust Version

MSRV: 1.75.0 (matching the crate's rust-version field). The MSRV will only increase with a minor version bump and will be documented in release notes.

Status

tower-http-cache is under active development. Expect API adjustments while we stabilize the 0.x series. Contributions and feedback are welcome—feel free to open an issue or PR! ***

License

This project is dual-licensed under either:

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)

You may choose either license to suit your needs. Unless explicitly stated otherwise, any contribution intentionally submitted for inclusion in the crate shall be dual-licensed as above, without additional terms or conditions.

Contributing

1. Fork and clone the repository.
2. Install prerequisites (cargo, rustup, and Docker if you plan to run Redis tests).
3. Run the checks:

   cargo fmt --all
   cargo clippy --all-targets --all-features
   cargo test
   python3 scripts/redis_smoke.py
   

4. Open a pull request with a succinct summary, test evidence, and (when applicable) benchmark output via cargo bench.

Bug reports and feature requests are welcome in the issue tracker. For larger design changes, please start a discussion thread to align on API shape before submitting code.