OxiCode

A modern binary serialization library for Rust - the successor to bincode.

About

OxiCode is a compact encoder/decoder pair that uses a binary zero-fluff encoding scheme. The size of the encoded object will be the same or smaller than the size that the object takes up in memory in a running Rust program.

This project serves as the spiritual successor to bincode, maintaining 100% binary compatibility while introducing modern improvements and advanced features that make it 150% better.

Features

Core Features (100% Bincode Compatible)

• Compact encoding: Minimal overhead in serialized format
• Fast: Optimized for performance with zero-copy operations where possible
• Flexible: Support for various encoding configurations
• Safe: Strict no-unwrap policy, comprehensive error handling
• Modern: Built with latest Rust practices and 2021 edition features
• no_std support: Works in embedded and resource-constrained environments
• Bincode compatibility: 100% binary format compatibility with bincode 2.0

150% Enhancement Features (Beyond Bincode)

• ⚡ SIMD Optimization: Hardware-accelerated array encoding (2-4x speedup)
• 🗜️ Compression: LZ4 (fast) and Zstd (better ratio) support
• 📦 Schema Evolution: Version tracking and automatic migration
• 🌊 Streaming: Chunked encoding/decoding for large datasets
• ⏱️ Async Streaming: Non-blocking async I/O with tokio
• ✅ Validation: Constraint-based validation middleware

See Feature Comparison below for detailed breakdown.

Why OxiCode?

While bincode has served the Rust community well, OxiCode brings:

1. 100% Binary Compatibility: Drop-in replacement with identical binary format
2. Modern Rust practices: Built from the ground up with Rust 2021 edition
3. Safety first: Strict no-unwrap policy throughout the codebase
4. Better error handling: More informative error messages and comprehensive error types
5. Advanced features: SIMD, compression, streaming, async, validation - features bincode lacks
6. Active maintenance: Dedicated to long-term support and evolution

Installation

Add this to your Cargo.toml:

[dependencies]
oxicode = "0.1"

# With serde support (for serde::Serialize/Deserialize types)
oxicode = { version = "0.1", features = ["serde"] }

# Optional features
oxicode = { version = "0.1", features = ["simd", "compression", "async-tokio"] }

Feature Flags

default = ["std", "derive"]
std = ["alloc"]              # Standard library support
alloc = []                   # Heap allocations (for no_std + alloc)
derive = []                  # Derive macros for Encode/Decode
serde = []                   # Serde integration (optional)
simd = []                    # SIMD-accelerated array encoding
compression-lz4 = []         # LZ4 compression (fast)
compression-zstd = []        # Zstd compression (better ratio)
compression = ["compression-lz4"]  # Default compression
async-tokio = ["tokio"]      # Async streaming with tokio
async-io = ["futures-io"]    # Generic async IO traits

Quick Start

use oxicode::{Encode, Decode};

#[derive(Encode, Decode, PartialEq, Debug)]
struct Point {
    x: f32,
    y: f32,
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let point = Point { x: 1.0, y: 2.0 };

    // Encode to bytes
    let encoded = oxicode::encode_to_vec(&point)?;

    // Decode from bytes
    let (decoded, _): (Point, _) = oxicode::decode_from_slice(&encoded)?;

    assert_eq!(point, decoded);
    Ok(())
}

Using with Serde

OxiCode provides optional serde integration for types that implement serde::Serialize and serde::Deserialize:

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug, PartialEq)]
struct Person {
    name: String,
    age: u32,
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let person = Person {
        name: "Alice".to_string(),
        age: 30,
    };

    // Encode using serde integration
    let encoded = oxicode::serde::encode_to_vec(&person, oxicode::config::standard())?;

    // Decode using serde integration
    let (decoded, _): (Person, _) = oxicode::serde::decode_from_slice(&encoded, oxicode::config::standard())?;

    assert_eq!(person.name, decoded.name);
    assert_eq!(person.age, decoded.age);
    Ok(())
}

Enable serde feature in Cargo.toml:

[dependencies]
oxicode = { version = "0.1", features = ["serde"] }
serde = { version = "1.0", features = ["derive"] }

Configuration

OxiCode supports various encoding configurations:

use oxicode::config;

// Standard configuration (default): little-endian + varint
let cfg = config::standard();

// Legacy bincode 1.0-compatible: little-endian + fixed-int
let cfg = config::legacy();

// Custom configuration
let cfg = config::standard()
    .with_big_endian()
    .with_fixed_int_encoding()
    .with_limit::<1048576>(); // 1MB limit

// Use with encoding/decoding
let bytes = oxicode::encode_to_vec_with_config(&value, cfg)?;
let (decoded, _) = oxicode::decode_from_slice_with_config(&bytes, cfg)?;

Advanced Features

SIMD-Accelerated Arrays

Hardware acceleration for large array operations (2-4x speedup):

use oxicode::{Encode, Decode};

#[derive(Encode, Decode)]
struct LargeDataset {
    readings: Vec<f64>,  // SIMD-accelerated when feature enabled
}

// Enable with features = ["simd"]
// Auto-detects CPU capabilities (SSE2, AVX2, AVX-512)

See examples/simd_arrays.rs for detailed usage.

Compression

Reduce size with LZ4 or Zstd compression:

use oxicode::compression::{CompressedEncoder, CompressedDecoder, CompressionType};

// LZ4 - fast compression
let mut encoder = CompressedEncoder::new(writer, CompressionType::Lz4)?;
value.encode(&mut encoder)?;

// Zstd - better compression ratio
let mut encoder = CompressedEncoder::new(writer, CompressionType::Zstd(10))?;
value.encode(&mut encoder)?;

See examples/compression.rs for detailed usage.

Streaming Serialization

Process large datasets incrementally:

use oxicode::streaming::{StreamingEncoder, StreamingDecoder};

// Encode items one at a time
let mut encoder = StreamingEncoder::new(writer, config)?;
for item in large_dataset {
    encoder.write_item(&item)?;
}
encoder.finish()?;

// Decode items incrementally
let mut decoder = StreamingDecoder::new(reader, config)?;
while let Some(item) = decoder.read_item::<MyType>()? {
    process(item);
}

See examples/streaming.rs for detailed usage.

Async Streaming

Non-blocking async I/O with tokio:

use oxicode::streaming::AsyncStreamingEncoder;

// Async encoding
let mut encoder = AsyncStreamingEncoder::new(writer, config);
for item in dataset {
    encoder.write_item(&item).await?;
}
let writer = encoder.finish().await?;

See examples/async_streaming.rs for detailed usage.

Validation Middleware

Validate data during decoding:

use oxicode::validation::{Validator, Constraints};

// Create validator with constraints
let mut validator = Validator::new();
validator.add_constraint("name", Constraints::max_len(100));
validator.add_constraint("age", Constraints::range(Some(0), Some(120)));

// Validate decoded data
validator.validate(&person)?;

See examples/validation.rs for detailed usage.

Schema Evolution

Version your data formats and migrate gracefully:

use oxicode::versioning::{Version, VersionedEncoder};

let version = Version::new(1, 0, 0);
let mut encoder = VersionedEncoder::new(writer, version, config)?;
value.encode(&mut encoder)?;

// Decoder automatically validates version compatibility

See examples/versioning.rs for detailed usage.

Migration from bincode

OxiCode is 100% binary-compatible with bincode. Migration is straightforward:

// Before (bincode 2.0)
use bincode::{Encode, Decode, config};
let bytes = bincode::encode_to_vec(&value, config::standard())?;
let (decoded, _) = bincode::decode_from_slice(&bytes, config::standard())?;

// After (oxicode) - same API!
use oxicode::{Encode, Decode, config};
let bytes = oxicode::encode_to_vec(&value, config::standard())?;
let (decoded, _) = oxicode::decode_from_slice(&bytes, config::standard())?;

Binary data is 100% compatible - you can mix libraries:

• Data encoded with bincode can be decoded with oxicode ✓
• Data encoded with oxicode can be decoded with bincode ✓

For detailed migration guide, see MIGRATION.md.

Feature Comparison

Project Status

🎯 Version 0.1.0 - Production Ready

All core features and enhancements complete. See CHANGELOG.md for details.

Statistics (as of 2025-12-28):

• Lines of Code: 10,860 (Rust source, excluding tests)
• Files: 61 Rust files
• Test Coverage: 211 tests passing (100% pass rate)
  • 18 binary compatibility tests (100% byte-for-byte identical to bincode)
  • 193+ feature and integration tests
• Type Coverage: 112+ types with full Encode/Decode support
• Binary Compatibility: 100% verified through cross-library testing
• Code Quality: ✓ Zero unwrap(), ✓ Zero warnings, ✓ All files < 2000 lines

Project Structure

This is a workspace with the following crates:

• oxicode: Main library crate
• oxicode_derive: Procedural macros for deriving Encode/Decode
• oxicode_compatibility: Compatibility tests and bincode interop

Development Principles

OxiCode follows strict development principles:

• No warnings policy: All code must compile without warnings
• No unwrap policy: All error cases must be properly handled
• Latest crates policy: Use latest versions of dependencies
• Workspace policy: Proper workspace structure with shared dependencies
• Refactoring policy: Keep individual files under 2000 lines

Performance

OxiCode is designed for performance:

• SIMD acceleration: 2-4x speedup for large arrays (with simd feature)
• Zero-copy deserialization: Where possible
• Efficient varint encoding: For integers
• Minimal allocations: During encoding/decoding
• Benchmark suite: Included in benches/

Run benchmarks:

cargo bench

Testing

# Run all tests
cargo nextest run --all-features

# Run specific feature tests
cargo test --features simd
cargo test --features compression
cargo test --features async-tokio

# Run with no-std
cargo test --no-default-features --features alloc

Examples

The examples/ directory contains comprehensive examples:

• basic_usage.rs - Simple encoding/decoding
• configuration.rs - Configuration options
• zero_copy.rs - Zero-copy deserialization
• simd_arrays.rs - SIMD-accelerated arrays
• compression.rs - LZ4 and Zstd compression
• streaming.rs - Chunked streaming
• async_streaming.rs - Async tokio streaming
• validation.rs - Validation middleware
• versioning.rs - Schema evolution

Run examples:

cargo run --example basic_usage
cargo run --example simd_arrays --features simd
cargo run --example compression --features compression
cargo run --example async_streaming --features async-tokio

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

Licensed under the MIT license. See LICENSE for details.

Acknowledgments

This project builds upon the excellent work done by the bincode team and community. We're grateful for their contributions to the Rust ecosystem.

Related Projects

• SciRS2 - Scientific computing library
• NumRS2 - Numerical computing library
• ToRSh - PyTorch-like tensor library
• OxiRS - RDF and SPARQL library
• QuantRS2 - Quantum computing library