clickhouse-rowbinary

Fast, streaming encoder/decoder for ClickHouse RowBinary format.

Available as both a Rust crate and Python package (with Rust-powered performance).

Features

• Full type support: All ClickHouse types including Decimal, DateTime64, UUID, IPv4/IPv6, Nullable, Array, Map, Tuple, LowCardinality, Nested, JSON, and Dynamic
• Three format variants: RowBinary, RowBinaryWithNames, RowBinaryWithNamesAndTypes
• Streaming APIs: Read/write rows incrementally without loading entire datasets into memory
• Schema validation: Strict type checking with clear error messages
• Zero-copy where possible: Efficient memory usage for high-throughput scenarios

Installation

Python

pip install clickhouse-rowbinary

Rust

[dependencies]
clickhouse_rowbinary = "0.1"

Quick Start

Python

from clickhouse_rowbinary import Schema, RowBinaryWriter, RowBinaryReader

# Define schema
schema = Schema.from_clickhouse([
    ("id", "UInt32"),
    ("name", "String"),
    ("score", "Float64"),
])

# Write rows
writer = RowBinaryWriter(schema)
writer.write_row({"id": 1, "name": b"Alice", "score": 95.5})
writer.write_row({"id": 2, "name": b"Bob", "score": 87.0})
data = writer.take()

# Read rows back
reader = RowBinaryReader(data, schema)
for row in reader:
    print(f"{row['name'].decode()}: {row['score']}")

Rust

use clickhouse_rowbinary::{RowBinaryFormat, RowBinaryValueWriter, Schema, Value};

let schema = Schema::from_type_strings(&[
    ("id", "UInt32"),
    ("name", "String"),
    ("score", "Float64"),
])?;

let mut writer = RowBinaryValueWriter::new(
    Vec::new(),
    RowBinaryFormat::RowBinary,
    schema.clone(),
);
writer.write_row(&[
    Value::UInt32(1),
    Value::String(b"Alice".to_vec()),
    Value::Float64(95.5),
])?;

let payload = writer.into_inner();

Supported Types

Format Variants

• RowBinary: Raw row data only (most compact, requires schema)
• RowBinaryWithNames: Includes column names header
• RowBinaryWithNamesAndTypes: Self-describing with names and types

from clickhouse_rowbinary import Format

# For self-describing data
writer = RowBinaryWriter(schema, format=Format.RowBinaryWithNamesAndTypes)
writer.write_header()
writer.write_rows(rows)

Documentation

• Python: See the Python package documentation for detailed Python API reference
• Rust: See USAGE.md for advanced Rust patterns (streaming, batching, Zstd compression)
• API Reference: docs.rs/clickhouse_rowbinary

Error Handling

Both implementations provide specific exception/error types:

Running Tests

# Rust tests
cargo test --lib

# Python tests
uv run pytest tests/python

# Full validation (requires ClickHouse)
./validate.sh

License

MIT OR Apache-2.0