Charon

Modern Rust music source separation library using state-of-the-art ML inference

Charon is a pure-Rust implementation for audio source separation, inspired by Demucs but built entirely with modern Rust ML frameworks and performance optimizations from rust-imbalanced-learn.

No Python dependencies!

Features

• Pure Rust - No Python runtime required
• Multiple ML Backends
  • ONNX Runtime via ort (production-ready, hardware accelerated)
  • HuggingFace Candle (pure Rust, flexible)
• Complete Audio Processing
  • Decode any format with Symphonia (MP3, FLAC, OGG, WAV, etc.)
  • High-quality resampling with Rubato
  • Real-time processing support with CPAL
• Hardware Acceleration
  • CUDA / TensorRT (NVIDIA)
  • Metal (Apple Silicon)
  • Accelerate (macOS)
  • SIMD-optimized operations
• Parallel Processing - Multi-threaded with Rayon
• Performance Optimizations
  • KNN-based audio similarity search
  • SIMD-accelerated operations
  • Cache-friendly memory access patterns
• Progress Tracking - Built-in progress bars
• Easy API - Simple, ergonomic interface

Quick Start

Add to your Cargo.toml:

[dependencies]
charon-audio = "0.1"

Basic Usage

use charon_audio::{Separator, SeparatorConfig};

fn main() -> anyhow::Result<()> {
    // Create separator with ONNX model
    let config = SeparatorConfig::onnx("path/to/model.onnx")
        .with_shifts(1)
        .with_segment_length(10.0);
    
    let separator = Separator::new(config)?;
    
    // Separate audio file
    let stems = separator.separate_file("input.mp3")?;
    
    // Save individual stems
    stems.save_all("output_dir")?;
    
    // Or save specific stem
    stems.save("vocals", "vocals.wav")?;
    
    Ok(())
}

Advanced Audio Processing

use charon_audio::{AudioFile, AudioBuffer, AudioKNN, SimdOps};

fn main() -> anyhow::Result<()> {
    // Load and process audio
    let mut audio = AudioFile::read("song.mp3")?;
    audio.normalize();
    audio.apply_gain(-3.0); // Reduce by 3dB
    
    // Find similar audio segments using KNN
    let knn = AudioKNN::new(5);
    let segment = audio.data.slice(ndarray::s![.., 0..1000]);
    let similar = knn.find_similar_segments(
        segment,
        audio.data.view(),
        441  // hop size
    );
    
    // SIMD-optimized operations
    let mut samples = vec![1.0, 2.0, 3.0, 4.0];
    let window = vec![0.5, 0.5, 0.5, 0.5];
    SimdOps::multiply(&mut samples, &window);
    
    let rms = SimdOps::rms(&samples);
    println!("RMS: {}", rms);
    
    Ok(())
}

Batch Processing with Performance Optimization

use charon_audio::{utils, Separator, SeparatorConfig, BatchProcessor};

fn main() -> anyhow::Result<()> {
    // Find all audio files
    let files = utils::find_audio_files("input_dir", true)?;
    
    // Create separator with performance hints
    let config = SeparatorConfig::onnx("model.onnx")
        .with_shifts(1)
        .with_segment_length(10.0);
    
    let separator = Separator::new(config)?;
    
    // Parallel batch processing
    let batch_processor = BatchProcessor::new(10).with_threads(4);
    let results = batch_processor.process(files, |file| {
        separator.separate_file(&file)
    });
    
    println!("Processed {} files", results.len());
    Ok(())
}

Installation

Requirements

• Rust 1.70 or later
• Optional: CUDA toolkit for GPU acceleration

Basic Installation

cargo add charon-audio

With GPU Support

[dependencies]
charon-audio = { version = "0.1", features = ["cuda"] }

Feature Flags

• ort-backend (default) - ONNX Runtime backend
• candle-backend - HuggingFace Candle backend
• cuda - NVIDIA CUDA support
• tensorrt - NVIDIA TensorRT support
• metal - Apple Metal support
• accelerate - macOS Accelerate framework

Architecture

Core Modules

• audio - Audio I/O with Symphonia, high-quality resampling with Rubato
• models - ML model backends (ONNX Runtime, Candle)
• processor - Audio processing pipeline with segmentation and ensemble
• separator - High-level API for source separation
• performance - SIMD-optimized operations, KNN utilities, parallel batch processing
• utils - File system utilities, memory estimation

Performance Optimizations

Charon integrates high-performance patterns from rust-imbalanced-learn:

use charon::{PerformanceHints, PerformanceHint};

let hints = PerformanceHints::new()
    .with_hint(PerformanceHint::Parallel)
    .with_hint(PerformanceHint::Vectorize)
    .with_hint(PerformanceHint::CacheFriendly);

Use Cases

Music Production

• Isolate vocals for remixes
• Extract drums for sampling
• Create karaoke tracks
• Analyze individual instruments

Audio Research

• Source separation experiments
• Music information retrieval
• Audio similarity search
• Dataset preparation

Content Creation

• Podcast editing
• Video production
• Live streaming
• Audio restoration

Performance

Charon achieves significant performance improvements over Python-based solutions:

Benchmarks on M1 MacBook Pro with optimized release builds

Examples

Run Examples

# Basic separation
cargo run --example separate -- input.mp3 output/ model.onnx

# Real-time processing (conceptual)
cargo run --example realtime

# Batch processing
cargo run --example batch -- input_dir/ output_dir/ model.onnx

Model Downloads

Charon works with ONNX-format audio separation models:

1. Export from PyTorch Demucs: Convert existing Demucs models to ONNX
2. Use pre-trained ONNX models: Download from model repositories
3. Train custom models: Use Burn or Candle for training

Integration with Rust ML Ecosystem

With Linfa

use linfa::prelude::*;
use charon::AudioBuffer;

let features = extract_audio_features(&audio_buffer);
let dataset = Dataset::new(features, labels);

With SmartCore

use smartcore::linalg::basic::matrix::DenseMatrix;
use charon::AudioFile;

let audio = AudioFile::read("song.mp3")?;
let features = audio.data.into_raw_vec();

With rust-imbalanced-learn

use imbalanced_sampling::prelude::*;
use charon::AudioBuffer;

// Use SMOTE for augmenting audio training data
let smote = SmoteStrategy::new(5);
let (x_balanced, y_balanced) = smote.resample(x.view(), y.view(), &config)?;

Testing

# Run all tests
cargo test --all

# Run with verbose output
cargo test --all -- --nocapture

# Run specific test
cargo test --test audio_tests

Documentation

Full API documentation is available at docs.rs/charon.

Generate local documentation:

cargo doc --open --no-deps

Roadmap

• ONNX Runtime backend
• Audio I/O with Symphonia
• High-quality resampling
• SIMD-optimized operations
• KNN-based audio similarity
• Parallel batch processing
• Candle backend implementation
• Real-time CPAL integration
• Pre-trained model zoo
• WebAssembly support

Contributing

Contributions are welcome! Please read our Contributing Guide.

Areas for Contribution

• Additional ML backends (tch-rs, tract)
• More audio processing algorithms
• Pre-trained model integration
• Performance optimizations
• Documentation improvements

License

Licensed under the MIT License. See LICENSE for details.

Acknowledgements

• Demucs - Original music source separation research
• ONNX Runtime - Cross-platform ML inference
• HuggingFace Candle - Minimalist ML framework for Rust
• Symphonia - Pure Rust audio decoding
• Rust ML community for the amazing ecosystem