prefix-sum-vec

Crates.ioprefix-sum-vec
lib.rsprefix-sum-vec
version0.1.2
sourcesrc
created_at2022-10-11 13:23:09.197129
updated_at2023-01-23 14:59:03.747414
descriptionCompressed storage for highly repeating elements, with `O(log n)` lookups
homepage
repository
max_upload_size
id685351
size20,168
Simonas Kazlauskas (nagisa)

documentation

https://docs.rs/prefix-sum-vec/

README

prefix-sum-vec

A Rust crate for compressed storage of highly repeating elements, with O(log n) lookups

What is this?

The data structure provided by this crate allow users to space-efficiently store repeating sequences of the same value. An example of this sort of data comes up in WebAssembly, where the locals for a function are represented as a sequence of types and their repetition count. The binary encoding of function locals such as these

(locals i32 i32 i32 i32 i64 i64 f64)

is actually encoded as a sequence along the lines of 0x04 i32 0x02 i64 0x01 f64. Were the decoded representation of the locals naively stuff everything into a vector, a potential denial-of-service hazard could arise. A crafted input with a representation of millions of locals in just a couple of bytes of encoded space would force a naive implementation to allocate gigabytes of memory space.

The data structure provided by this crate stores just one copy of the repeating element alongside with a prefix-sum of the indices at which the element can be found. So, given the locals example above, the storage would look something like this:

[(4, i32), (6, i64), (7, f64)]

From there on, looking up an element with an index 5 is a binary search away over the prefix-sum indices.

Usage

First, specify this crate in your Cargo.toml:

[dependencies]
prefix-sum-vec = "0.1"

This crate aims to be extremely lightweight and straightforward. As such there are no optional features to enable at this time. Then, refer to the documentation of the PrefixSumVec type to discover its usage in your code.

License

This project is licensed under the Apache 2.0 OR BSD 3-clause license at your choice.

Commit count: 0

cargo fmt