zarrs_icechunk

Crates.iozarrs_icechunk
lib.rszarrs_icechunk
version0.3.1
created_at2024-10-17 20:51:18.044877+00
updated_at2025-07-21 22:17:26.442321+00
descriptionicechunk store support for the zarrs crate
homepagehttps://zarrs.dev
repositoryhttps://github.com/zarrs/zarrs_icechunk
max_upload_size
id1413611
size186,778
Lachlan Deakin (LDeakin)

documentation

https://docs.rs/zarrs_icechunk

README

zarrs_icechunk

Latest Version icechunk 0.3 zarrs_icechunk documentation msrv build

icechunk store support for the zarrs Rust crate.

Icechunk is a transactional store that enables git-like version control of Zarr hierarchies.

zarrs_icechunk can read data in a range of archival formats (e.g., netCDF4, HDF5, etc.) that are converted to icechunk-backed "virtual Zarr datacubes" via VirtualiZarr (example below).

Version Compatibility Matrix

See doc/version_compatibility_matrix.md.

Examples

Basic Usage and Version Control

use icechunk::{Repository, RepositoryConfig, repository::VersionInfo};
use zarrs_icechunk::AsyncIcechunkStore;

// Create an icechunk repository
let storage = icechunk::new_in_memory_storage().await?;
let config = RepositoryConfig::default();
let repo = Repository::create(Some(config), storage, HashMap::new()).await?;

// Do some array/metadata manipulation with zarrs, then commit a snapshot
let session = repo.writable_session("main").await?;
let store = Arc::new(AsyncIcechunkStore::new(session));
let array: Array<AsyncIcechunkStore> = ...;
let snapshot0 = store.session().write().await.commit("Initial commit", None).await?;

// Do some more array/metadata manipulation, then commit another snapshot
let session = repo.writable_session("main").await?;
let store = Arc::new(AsyncIcechunkStore::new(session));
let array: Array<AsyncIcechunkStore> = ...;
let snapshot1 = store.session().write().await.commit("Update data", None).await?;

// Checkout the first snapshot
let session = repo.readonly_session(&VersionInfo::SnapshotId(snapshot0)).await?;
let store = Arc::new(AsyncIcechunkStore::new(session));
let array: Array<AsyncIcechunkStore> = ...;

Virtualise NetCDF as Zarr (via VirtualiZarr)

Decode a virtual Zarr array /examples/data/test.icechunk.zarr:

cargo run --example virtualizarr_netcdf

This references /examples/data/test[0,1].nc hosted in this repository over HTTP. /examples/data/test.icechunk.zarr was created with /examples/virtualizarr_netcdf.py.

Licence

zarrs_icechunk is licensed under either of

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Commit count: 0

cargo fmt