autocompress-rs

Automatically select suitable decoder from magic bytes or encoder from file extension. This library also provides I/O thread pool to perform decompression and compression in background threads.

Supported file formats

• Gzip
• Zlib (Cannot suggest format from magic bytes and file extension)
• BZip2
• XZ
• Z-standard

Feature flags

• gzip : Gzip format support
• bgzip : bgzip format support
• bzip2 : Bzip2 format support
• xz : XZ format support
• zstd : Zstd format support
• rayon : Off-load compression and decompression process to another thread using rayon
• tokio : Async reader and writer support with tokio
• tokio_fs: Enable autodetect_async_open function

Example

Read from a file

use std::io::prelude::*;
use autocompress::autodetect_open;

fn main() -> anyhow::Result<()> {
    let mut reader = autodetect_open("testfiles/pg2701.txt.xz")?;
    let mut buf = Vec::new();
    reader.read_to_end(&mut buf)?;
    Ok(())
}

Write to a file

use std::io::prelude::*;
use autocompress::{autodetect_create, CompressionLevel};

fn main() -> anyhow::Result<()> {
    let mut writer = autodetect_create("target/doc-index.xz", CompressionLevel::Default)?;
    writer.write_all(&b"Hello, world\n"[..])?;
    Ok(())
}

Compress file in parallel

use std::io::prelude::*;
use autocompress::{autodetect_parallel_create, CompressionLevel};

fn main() -> anyhow::Result<()> {
  let mut writer = autodetect_parallel_create("target/doc-index2.xz", CompressionLevel::Default)?;
  writer.write_all(&b"Hello, world\n"[..])?;
  Ok(())
}