chatpack

Token-efficient chat export processing for LLM and RAG pipelines.

Website | API Docs | Benchmarks | Export Guide

Table of Contents

• Overview
• Installation
• Quick Start
• Documentation
• Feature Flags
• Contributing
• License

Overview

Chatpack parses chat exports from Telegram, WhatsApp, Instagram, and Discord, converting them into token-efficient formats for LLM analysis and RAG ingestion.

Raw chat exports waste 80%+ of tokens on JSON structure, metadata, and formatting. Chatpack removes this noise, achieving 13x compression (92% token reduction) with CSV output.

┌─────────────────┐                   ┌─────────────────┐
│ Telegram JSON   │                   │ Clean CSV       │
│ WhatsApp TXT    │ ──▶ chatpack ──▶ │ 13x compression │
│ Instagram JSON  │                   │ LLM-ready       │
│ Discord Export  │                   │ RAG-optimized   │
└─────────────────┘                   └─────────────────┘

Token Compression Results

Installation

cargo add chatpack

Or add to Cargo.toml:

[dependencies]
chatpack = "0.5"

See Feature Flags for minimal installations.

Quick Start

use chatpack::prelude::*;
use chatpack::parser::{Platform, create_parser};

fn main() -> Result<()> {
    // Parse Telegram export
    let parser = create_parser(Platform::Telegram);
    let messages = parser.parse("export.json".as_ref())?;

    // Merge consecutive messages from same sender
    let merged = merge_consecutive(messages);

    // Write as CSV (13x compression)
    write_csv(&merged, "output.csv", &OutputConfig::new())?;

    println!("Processed {} messages", merged.len());
    Ok(())
}

Documentation

Feature Flags

Minimal installation example:

chatpack = { version = "0.5", default-features = false, features = ["telegram", "csv-output"] }

Minimum Supported Rust Version

Rust 2024 edition (rustc 1.85+). MSRV is tested in CI.

Contributing

Contributions welcome! Please:

1. Check existing issues
2. Open an issue before large changes
3. Run cargo test && cargo clippy before submitting PRs

License

MIT License. See LICENSE for details.

Try chatpack online - no installation required!