Garage SDK

An async Rust SDK for Garage (S3-compatible) that uploads files from paths, URLs, or bytes and returns a stable public URL for CDN or proxy-fronted access. Designed for production deployments where Garage sits behind a signing proxy (e.g., Envoy) or a public CDN base URL.

Need a production-ready Garage + Envoy setup? See docs/background.md for the full deployment guide.

Features

• Upload from multiple sources: Local files, URLs, or raw bytes
• Automatic content-type detection: Uses file extensions and MIME type guessing
• Builder pattern configuration: Flexible, type-safe configuration
• Proper error handling: Custom error types with detailed messages, no panics
• Configurable limits: Set max file size and download timeouts
• Tracing integration: Debug logging via the tracing crate
• Async/await: Built on tokio for async operations
• MSRV: Rust 1.92 (Edition 2024)

Installation

Add to your Cargo.toml:

[dependencies]
garage-sdk = "0.1"
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }

Quick Start

use garage_sdk::{GarageUploader, UploaderConfig};

#[tokio::main]
async fn main() -> Result<(), garage_sdk::Error> {
    // Configure the uploader
    let config = UploaderConfig::builder()
        .endpoint("https://s3.example.com")
        .bucket("my-bucket")
        .public_base_url("https://cdn.example.com")
        .credentials("access_key_id", "secret_access_key")
        .build()?;

    let uploader = GarageUploader::new(config)?;

    // Upload a local file
    let result = uploader.upload_from_path("./image.png").await?;
    println!("Uploaded to: {}", result.public_url);

    Ok(())
}

Configuration Options

Configuration Sources

You can load configuration in three primary ways:

• Environment variables: UploaderConfig::from_env() (recommended for K8s env injection)
• Secret files directory: UploaderConfig::from_secret_dir(...) (recommended for mounted secrets)
• Env with file fallback: UploaderConfig::from_env_or_secret_dir(...)

Using the Builder Pattern

use garage_sdk::UploaderConfig;
use std::time::Duration;

let config = UploaderConfig::builder()
    .endpoint("https://s3.example.com")      // Required: S3 endpoint
    .region("garage")                         // Optional: defaults to "garage"
    .bucket("my-bucket")                      // Required: target bucket
    .public_base_url("https://cdn.example.com") // Required: public CDN URL
    .key_prefix("uploads")                    // Optional: prefix for all keys
    .credentials("access_key", "secret_key")  // Required: AWS credentials
    .download_timeout(Duration::from_secs(60)) // Optional: defaults to 30s
    .max_file_size(50 * 1024 * 1024)          // Optional: defaults to 100MB
    .max_buffered_bytes(8 * 1024 * 1024)      // Optional: defaults to 8MB
    .build()?;

Using Environment Variables

use garage_sdk::UploaderConfig;

// Reads from environment variables:
// - GARAGE_ENDPOINT or S3_ENDPOINT
// - GARAGE_REGION or S3_REGION (optional)
// - GARAGE_BUCKET or S3_BUCKET
// - GARAGE_PUBLIC_URL or S3_PUBLIC_URL
// - GARAGE_KEY_PREFIX or S3_KEY_PREFIX (optional)
// - AWS_ACCESS_KEY_ID
// - AWS_SECRET_ACCESS_KEY

let config = UploaderConfig::from_env()?;

Kubernetes example:

env:
  - name: GARAGE_ENDPOINT
    value: "https://s3.example.com"
  - name: GARAGE_BUCKET
    valueFrom:
      secretKeyRef:
        name: garage-sdk
        key: bucket
  - name: GARAGE_PUBLIC_URL
    valueFrom:
      secretKeyRef:
        name: garage-sdk
        key: public_url
  - name: AWS_ACCESS_KEY_ID
    valueFrom:
      secretKeyRef:
        name: garage-sdk
        key: access_key_id
  - name: AWS_SECRET_ACCESS_KEY
    valueFrom:
      secretKeyRef:
        name: garage-sdk
        key: secret_access_key

Using Kubernetes Secret Files

Mount your secret as a volume (each key becomes a file), then load from the directory:

use garage_sdk::UploaderConfig;

let config = UploaderConfig::from_secret_dir("/var/run/secrets/garage")?;

Kubernetes example:

volumes:
  - name: garage-secrets
    secret:
      secretName: garage-sdk
containers:
  - name: app
    volumeMounts:
      - name: garage-secrets
        mountPath: /var/run/secrets/garage
        readOnly: true

Expected filenames:

• endpoint
• region (optional, defaults to garage)
• bucket
• public_url
• key_prefix (optional)
• access_key_id
• secret_access_key

Custom Secret Filenames

use garage_sdk::{SecretFileNames, UploaderConfig};

let names = SecretFileNames {
    endpoint: "s3_endpoint".into(),
    region: None,
    bucket: "s3_bucket".into(),
    public_url: "s3_public_url".into(),
    key_prefix: None,
    access_key_id: "s3_access_key_id".into(),
    secret_access_key: "s3_secret_access_key".into(),
};

let config = UploaderConfig::from_secret_dir_with_names("/var/run/secrets/garage", &names)?;

Or with a common prefix:

use garage_sdk::{SecretFileNames, UploaderConfig};

let names = SecretFileNames::with_prefix("s3_");
let config = UploaderConfig::from_secret_dir_with_names("/var/run/secrets/garage", &names)?;

Or with a common suffix:

use garage_sdk::{SecretFileNames, UploaderConfig};

let names = SecretFileNames::with_suffix("_secret");
let config = UploaderConfig::from_secret_dir_with_names("/var/run/secrets/garage", &names)?;

Or via the builder:

use garage_sdk::{SecretFileNamesBuilder, UploaderConfig};

let names = SecretFileNamesBuilder::new()
    .with_prefix("s3_")
    .endpoint("s3_endpoint")
    .bucket("s3_bucket")
    .public_url("s3_public_url")
    .access_key_id("s3_access_key_id")
    .secret_access_key("s3_secret_access_key")
    .region(None::<String>)
    .key_prefix(None::<String>)
    .build()?;

let config = UploaderConfig::from_secret_dir_with_names("/var/run/secrets/garage", &names)?;

Merge defaults without overriding explicit fields:

use garage_sdk::{SecretFileNames, SecretFileNamesBuilder, UploaderConfig};

let names = SecretFileNamesBuilder::new()
    .endpoint("custom_endpoint")
    .merge_defaults(SecretFileNames::with_prefix("s3_"))
    .build()?;

let config = UploaderConfig::from_secret_dir_with_names("/var/run/secrets/garage", &names)?;

Environment Variables with File Fallback

use garage_sdk::UploaderConfig;

let config = UploaderConfig::from_env_or_secret_dir("/var/run/secrets/garage")?;

Upload Methods

Upload from Local Path

let result = uploader.upload_from_path("./photo.jpg").await?;
println!("Public URL: {}", result.public_url);
println!("Key: {}", result.key);
println!("Size: {} bytes", result.size);
println!("Content-Type: {}", result.content_type);

Upload from URL

Downloads the content from a URL and uploads it to storage:

let result = uploader
    .upload_from_url("https://example.com/image.png")
    .await?;

Small downloads are buffered in memory by default (8 MB), while larger or unknown-size responses are streamed with the size cap enforced.

Download Buffering vs Streaming

upload_from_url buffers small downloads in memory and streams larger or unknown-size responses to avoid unbounded memory usage.

• Default buffer threshold: 8 MB (max_buffered_bytes)
• Hard size limit: 100 MB (max_file_size)

If Content-Length is present and below the threshold, the response is buffered. Otherwise, the response is streamed and the size cap is enforced during the read.

Upload Raw Bytes

let json_data = r#"{"message": "Hello!"}"#;
let result = uploader
    .upload_bytes(
        json_data.as_bytes().to_vec(),
        "application/json",
        Some("json"),
    )
    .await?;

Upload Result

All upload methods return an UploadResult:

pub struct UploadResult {
    pub bucket: String,       // The bucket name
    pub key: String,          // The object key
    pub public_url: String,   // The public CDN URL
    pub etag: Option<String>, // MD5 hash from S3
    pub content_type: String, // MIME type
    pub size: u64,            // File size in bytes
}

Extensibility

You can extend the SDK without changing core logic by plugging in your own implementations of the provided traits:

• Downloader: controls how remote URLs are fetched
• StorageClient: controls how objects are uploaded
• KeyGenerator: controls how object keys are generated

Use GarageUploader::with_components to supply custom implementations while keeping the public API unchanged.

Module Layout

src/
  config/
    mod.rs
    data.rs
  download/
    mod.rs
    impls.rs
  error/
    mod.rs
    types.rs
  keygen/
    mod.rs
    generator.rs
  storage/
    mod.rs
    client.rs
  types/
    mod.rs
    model.rs
  uploader/
    mod.rs
    client.rs
  lib.rs

Error Handling

The SDK uses custom error types for proper error handling:

use garage_sdk::Error;

match uploader.upload_from_path("./file.txt").await {
    Ok(result) => println!("Success: {}", result.public_url),
    Err(Error::FileRead { path, source }) => {
        eprintln!("Could not read file {}: {}", path, source);
    }
    Err(Error::S3Operation { operation, reason }) => {
        eprintln!("S3 {} failed: {}", operation, reason);
    }
    Err(Error::Config { message }) => {
        eprintln!("Configuration error: {}", message);
    }
    Err(e) => eprintln!("Error: {}", e),
}

Error Types

Using in Other Applications

As a Library Dependency

# In your application's Cargo.toml
[dependencies]
garage-sdk = { path = "../garage-sdk" }
# Or from a git repository:
# garage-sdk = { git = "https://github.com/boniface/garage-sdk" }

Example Integration

use garage_sdk::{GarageUploader, UploaderConfig, Error};

pub struct ImageService {
    uploader: GarageUploader,
}

impl ImageService {
    pub fn new() -> Result<Self, Error> {
        let config = UploaderConfig::from_env()?;
        let uploader = GarageUploader::new(config)?;
        Ok(Self { uploader })
    }

    pub async fn upload_user_avatar(&self, path: &str) -> Result<String, Error> {
        let result = self.uploader.upload_from_path(path).await?;
        Ok(result.public_url)
    }
}

Running the Example

# Set configuration and credentials
export GARAGE_ENDPOINT="https://s3.example.com"
export GARAGE_BUCKET="my-bucket"
export GARAGE_PUBLIC_URL="https://cdn.example.com"
export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"

# Optional inputs for extra examples
export GARAGE_EXAMPLE_FILE="/path/to/local/file.jpg"
export GARAGE_EXAMPLE_URL="https://example.com/image.png"

# Run the example
cargo run --features example

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

License

Licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.