telemetry-kit

https://docs.telemetry-kit.dev

🎉 Currently Working (v0.2.0-alpha.1)

The following features are fully implemented and tested:

• ✅ Interactive Consent Prompts: First-run consent dialogs for privacy compliance (NEW!)
• ✅ Smart Instrumentation Recommendations: AI-powered code analysis suggesting where to add telemetry (NEW!)
• ✅ Auto-Sync Background Task: Automatic event synchronization in the background
• ✅ SQLite → Service Sync Protocol: Offline-first event buffering with HMAC-SHA256 authentication
• ✅ Privacy-First User IDs: Anonymous client identifiers with client_ prefix (SHA-256 hashed machine IDs)
• ✅ Event Tracking: Command and feature event builders with fluent API
• ✅ Ingestion Server: Production-ready Rust server with PostgreSQL + Redis
• ✅ Rate Limiting: Token-based rate limits (Free/Pro/Business/Enterprise tiers)
• ✅ Replay Protection: Nonce-based duplicate request detection
• ✅ Batch Ingestion: 1-1000 events per request with partial success handling
• ✅ Docker Deployment: Complete docker-compose stack for local development

Quick Start with Working Features

# 1. Start the server
cd server
docker compose up -d

# 2. Run the end-to-end test
cd ..
cargo run --example e2e_sync_test

See DEPLOYMENT_GUIDE.md for complete setup instructions.

Auto-Sync Background Task (NEW! 🎉)

Events are now automatically synced in the background - no manual .sync() calls required!

use telemetry_kit::prelude::*;

#[tokio::main]
async fn main() -> Result<()> {
    // Initialize with auto-sync (enabled by default)
    let telemetry = TelemetryKit::builder()
        .service_name("my-app")?
        .with_sync_credentials(org_id, app_id, token, secret)?
        .auto_sync(true)              // Enable auto-sync (default: true)
        .sync_interval(60)            // Sync every 60 seconds (default)
        .sync_on_shutdown(true)       // Sync before exit (default: true)
        .build()?;

    // Track events - they sync automatically in the background
    telemetry.track_command("build", |event| {
        event.success(true).duration_ms(1234)
    }).await?;

    // Graceful shutdown with final sync
    telemetry.shutdown().await?;
    Ok(())
}

Features:

• Background tokio task syncs events at configurable intervals
• Graceful shutdown with optional final sync
• Respects DO_NOT_TRACK environment variable
• Exponential backoff on sync failures
• Thread-safe implementation

See examples/auto_sync.rs for a complete example.

CLI Tool (NEW! 🎉)

Manage telemetry configuration and operations from the command line.

# Install CLI
cargo install telemetry-kit --features cli

# Create a new project with telemetry pre-configured
telemetry-kit new my-app                    # CLI application
telemetry-kit new my-lib -p lib             # Library
telemetry-kit new my-api -p web --with-sync # Web service with sync

# Analyze code for instrumentation opportunities (NEW!)
telemetry-kit analyze                       # Analyze current directory
telemetry-kit analyze --detailed            # Show detailed recommendations with code snippets
telemetry-kit analyze --path src/           # Analyze specific directory
telemetry-kit analyze --format json         # Output as JSON

# Interactive setup (for existing projects)
telemetry-kit init

# View statistics
telemetry-kit stats

# Test sync credentials
telemetry-kit test

# Validate configuration
telemetry-kit validate

# Clean local events
telemetry-kit clean

Available Commands:

• new - Create new projects with telemetry pre-configured
• analyze - Get smart recommendations on where to add instrumentation (NEW!)
• init - Interactive project setup with credential configuration
• test - Validate sync credentials
• stats - View event statistics (total, synced, unsynced)
• sync - Manually trigger synchronization
• validate - Validate configuration
• clean - Clear local event database

See CLI.md for complete CLI documentation.

---

🎯 The Problem

As an open source maintainer, you want to understand:

• Which features are actually used vs ignored
• Where users encounter errors or confusion
• How your CLI tool performs in real-world environments
• Whether your library is used correctly

But current telemetry solutions are:

• Too complex: Setting up OpenTelemetry requires understanding multiple crates and writing verbose boilerplate
• Not opinionated: You don't know what to track or where to add instrumentation
• Privacy-blind: Easy to accidentally log PII, risking your users' trust
• CLI-unfriendly: Most tools designed for long-running services, not CLI applications
• Hard to self-host: Commercial solutions or complex infrastructure required
• Trust issues: Users disable telemetry because they don't trust what's collected

✨ The Solution

use telemetry_kit::prelude::*;

#[tokio::main]
#[telemetry_kit::instrumented]  // 🎉 That's it!
async fn main() {
    telemetry_kit::init()
        .service_name("my-awesome-cli")
        .endpoint("https://telemetry.myapp.com")
        .anonymous()
        .init();
    
    // Your code here - automatically instrumented!
}

What you get as an OSS maintainer:

• ✅ 3 lines instead of 50+: Sensible defaults, zero boilerplate
• 🎯 Smart instrumentation: Auto-detect CLI commands, errors, and performance bottlenecks
• 🔒 Privacy-first: Built-in anonymization, GDPR-compliant, earns user trust
• 🚀 CLI-optimized: Works with short-lived processes, offline-capable
• 📦 Self-hostable: Simple Docker-based collection server included
• 🤖 AI-suggested: Get recommendations on what to instrument
• 📊 GitHub Badges: Show usage metrics in your README
• 👥 Public Dashboards: Share anonymous analytics with your community

🌟 Key Features

For Open Source Maintainers

Understand Your Users Without Compromising Their Privacy:

// Add to your CLI tool or library
let telemetry = TelemetryKit::builder()
    .service_name("my-awesome-cli")?
    .service_version(env!("CARGO_PKG_VERSION"))
    .prompt_for_consent()?       // Ask users first
    .build()?;

// Track what matters
telemetry.track_command("build", |event| {
    event.success(true).duration_ms(1234)
}).await?;

What You Can Learn:

• 📊 Feature Usage: Which commands/features are popular vs unused
• ⚠️ Error Patterns: Where users struggle or encounter bugs
• ⚡ Performance: Real-world execution times and bottlenecks
• 🌍 Platform Distribution: OS/architecture breakdown
• 📈 Adoption Trends: New vs returning users, version migration

Build Trust With Your Community:

• 🔒 Transparent: Users see exactly what you collect
• 🛑 Respectful: DO_NOT_TRACK honored automatically
• ✅ Consent-First: Optional interactive prompts
• 📖 Open Source: Inspect the code, self-host if preferred
• 🎯 Anonymous: No PII, just anonymous usage patterns

Showcase Your Impact:

• 🏆 Add usage badges to your README:
• 📊 Share public dashboards with your community
• 📈 Show growth and adoption metrics to sponsors

Zero-Config Telemetry

// Literally one line
telemetry_kit::init().auto_configure()?;

Privacy Controls

telemetry_kit::init()
    .anonymous()                    // Anonymous user IDs
    .sanitize_emails()              // Hash email addresses
    .exclude_env_vars()             // Don't capture environment
    .gdpr_compliant()               // Full GDPR compliance
    .consent_prompt()               // Ask user on first run
    .init();

Smart Instrumentation (NEW! 🎉)

Enable automatic function instrumentation with the macros feature:

use telemetry_kit::prelude::*;

#[instrument]  // Auto-track duration
async fn fetch_data(url: &str) -> Result<Data, Error> {
    // Function execution is automatically timed
    // Works with both sync and async functions
    // Supports Result types for success/failure tracking
    let response = reqwest::get(url).await?;
    let data = response.json().await?;
    Ok(data)
}

Features:

• Automatic execution timing for all instrumented functions
• Works with async and sync functions
• Supports functions with or without Result return types
• Zero-overhead when macros feature is disabled
• Compile-time code generation

Enable with:

[dependencies]
telemetry-kit = { version = "0.2", features = ["macros"] }

See examples/instrument_macro.rs for a complete example.

Note: Currently the macro measures timing but doesn't send telemetry yet. Full telemetry integration coming soon!

CLI-Specific Features

use telemetry_kit::cli::*;

#[derive(Parser)]
#[command(name = "my-cli")]
struct Cli {
    #[command(subcommand)]
    command: Commands,
}

// Automatically tracks:
// - Which commands are used
// - Command duration
// - Success/failure rates
// - Anonymous usage patterns

🗺️ Roadmap

Phase 1: Foundation (v0.1.0) - Q1 2025

• Core telemetry abstraction over OpenTelemetry
• Privacy-first defaults (anonymization, sanitization)
• Basic instrumentation macros
• Simple OTLP export
• Documentation and examples

Phase 2: Developer Experience (v0.2.0) - Q2 2025

• CLI scaffolding tool (telemetry-kit init)
• Auto-detection of instrumentation points
• Smart suggestions (analyze code, suggest where to add tracking)
• Pre-built configuration templates
• VS Code extension for inline suggestions

Phase 3: CLI Optimization (v0.3.0) - Q2 2025

• Short-lived process handling
• Offline caching with automatic sync
• User consent flow (first-run prompts)
• Minimal overhead (<1ms impact)
• Graceful degradation when server unavailable

Phase 4: Self-Hosting (v0.4.0) - Q3 2025

• Simple collection server (Docker one-liner)
• Built-in dashboard for basic analytics
• SQLite/PostgreSQL storage backends
• REST API for custom integrations
• Pre-built dashboards for common patterns

Phase 5: Advanced Features (v0.5.0+) - Q4 2025

• AI-powered insights (usage patterns, anomaly detection)
• Anonymous user cohorts
• A/B testing support
• Feature flag integration
• Custom metric definitions
• Multi-project dashboards

📊 Comparison with Existing Solutions

Feature	telemetry-kit	OpenTelemetry	tracing	sentry-rust
Setup Complexity	⭐ Low (3 lines)	⚠️ High (50+ lines)	⚠️ Medium	⭐ Low
Privacy Built-in	✅ Yes	❌ No	❌ No	⚠️ Partial
CLI Optimized	✅ Yes	❌ No	❌ No	⚠️ Partial
Auto-instrumentation	✅ Yes	❌ No	❌ No	⚠️ Errors only
Self-hostable	✅ Included	⚠️ Complex	N/A	❌ Commercial
Smart Suggestions	✅ Yes	❌ No	❌ No	❌ No
Offline Support	✅ Yes	❌ No	N/A	⚠️ Limited
GDPR Compliant	✅ Built-in	⚠️ Manual	⚠️ Manual	⚠️ Manual

🚀 Quick Start

Installation

[dependencies]
telemetry-kit = "0.0.1"

Basic Usage

use telemetry_kit::prelude::*;

#[tokio::main]
async fn main() -> Result<()> {
    // Initialize with defaults
    let _guard = telemetry_kit::init()
        .service_name("my-app")
        .init()?;
    
    // Your application code
    do_work().await?;
    
    Ok(())
}

#[instrument]
async fn do_work() -> Result<()> {
    // Automatically tracked!
    Ok(())
}

CLI Application

use clap::Parser;
use telemetry_kit::cli::*;

#[derive(Parser)]
#[telemetry_kit::track_commands]  // Auto-track all commands!
struct Cli {
    #[command(subcommand)]
    command: Commands,
}

#[tokio::main]
async fn main() -> Result<()> {
    let cli = Cli::parse();
    
    telemetry_kit::init()
        .for_cli()                // CLI-specific optimizations
        .consent_on_first_run()   // Ask user permission
        .init()?;
    
    match cli.command {
        Commands::Build => build().await?,
        Commands::Deploy => deploy().await?,
    }
    
    Ok(())
}

🏗️ Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Your Application                        │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │   Macros     │  │  Middleware  │  │   Manual     │      │
│  │ #[instrument]│  │   Tracking   │  │   Tracking   │      │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘      │
│         └──────────────────┴──────────────────┘              │
└─────────────────────────────┬───────────────────────────────┘
                              │
                   ┌──────────▼──────────┐
                   │  telemetry-kit Core │
                   │                     │
                   │ • Privacy filters   │
                   │ • Anonymization     │
                   │ • Batching          │
                   │ • Sampling          │
                   └──────────┬──────────┘
                              │
        ┌─────────────────────┼─────────────────────┐
        │                     │                     │
┌───────▼────────┐  ┌─────────▼────────┐  ┌────────▼────────┐
│ OpenTelemetry  │  │  Simple Server   │  │   Custom        │
│   (OTLP)       │  │  (Self-hosted)   │  │   Backends      │
└────────────────┘  └──────────────────┘  └─────────────────┘

🔒 Privacy & Compliance

telemetry-kit is privacy-first by design:

• Anonymous by default: No PII collected unless explicitly enabled
• User consent: Built-in consent flow for CLI applications
• Data sanitization: Automatic scrubbing of sensitive data
• GDPR compliant: Right to erasure, data portability, consent management
• Transparent: Users can see exactly what data is collected
• Opt-out friendly: Easy to disable, respects DO_NOT_TRACK

Privacy Features

Interactive Consent Prompts (NEW!):

use telemetry_kit::prelude::*;

#[tokio::main]
async fn main() -> Result<()> {
    // Show interactive consent dialog on first run
    let telemetry = TelemetryKit::builder()
        .service_name("my-app")?
        .service_version("1.0.0")
        .prompt_for_consent()?  // Full prompt with privacy details
        .build()?;

    // Or use minimal one-liner prompt
    let telemetry = TelemetryKit::builder()
        .service_name("my-app")?
        .prompt_minimal()?      // Shorter prompt
        .build()?;

    Ok(())
}

Privacy Configuration:

use telemetry_kit::prelude::*;
use telemetry_kit::privacy::PrivacyConfig;

// Strict privacy mode (GDPR-compliant)
let telemetry = TelemetryKit::builder()
    .service_name("my-app")?
    .strict_privacy()  // Requires consent, sanitizes data, 30-day retention
    .build()?;

// Or configure individually
let telemetry = TelemetryKit::builder()
    .service_name("my-app")?
    .consent_required(true)
    .sanitize_paths(true)
    .sanitize_emails(true)
    .data_retention(90)  // Days
    .build()?;

Manage Consent via CLI:

telemetry-kit consent grant    # Enable telemetry
telemetry-kit consent deny     # Disable telemetry
telemetry-kit consent status   # Check current status

See examples/interactive_consent.rs and examples/privacy.rs for complete examples.

🤝 Contributing

We welcome contributions! This is a new project and we're building it in the open.

How to Contribute

1. Check out our Roadmap for planned features
2. Look at GitHub Issues for tasks
3. Read CONTRIBUTING.md for guidelines
4. Join discussions in GitHub Discussions

Areas We Need Help

• 🎨 Design: API design, ergonomics, developer experience
• 📝 Documentation: Examples, tutorials, guides
• 🔧 Implementation: Core features, integrations, tools
• 🧪 Testing: Unit tests, integration tests, real-world usage
• 🌍 Community: Blog posts, talks, spreading the word

📚 Documentation

User Documentation

• Quick Start Guide (coming soon)
• API Reference
• Privacy Guide (coming soon)
• CLI Best Practices (coming soon)
• Self-Hosting Guide (coming soon)

Project Documentation

• Roadmap - Feature roadmap and release plan
• Security Policy - Vulnerability disclosure and security practices
• SLSA Compliance - Supply chain security documentation
• Contributing Guide - How to contribute to the project
• Project Docs - Internal development documentation

💡 Inspiration

This project is inspired by:

• The simplicity of sentry-rust
• The power of OpenTelemetry
• The ergonomics of tracing
• The privacy-focus of Plausible Analytics
• The developer experience of Next.js Analytics

📄 License

This project is dual-licensed under your choice of:

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)

You may choose either license at your option.

Why Dual License?

We follow the Rust community standard of dual licensing to give you flexibility:

• Choose MIT if you prefer simplicity and maximum compatibility (including GPL2)
• Choose Apache 2.0 if you want explicit patent protection and contributor agreements

Both licenses are permissive and allow commercial use, modification, distribution, and private use.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

🙏 Acknowledgments

Built with ❤️ by Ibrahim Cesar and contributors.

Special thanks to:

• The OpenTelemetry project
• The Rust tracing ecosystem
• Everyone who provided feedback and ideas

GNU Terry Pratchett

This project includes the X-Clacks-Overhead: GNU Terry Pratchett header in all HTTP requests to keep the memory of Sir Terry Pratchett alive in the overhead of the internet. Learn more at gnuterrypratchett.com.

"A man is not dead while his name is still spoken." - Going Postal, Terry Pratchett

---

Status: 🚧 Early Development - API is unstable and will change

Current Version: 0.0.1 - Placeholder release for crate reservation

First Usable Release: v0.1.0 (Target: Q1 2025)

⭐ Star us on GitHub | 📖 Read the Docs | 💬 Discussions