| Crates.io | sharded-vec-writer |
| lib.rs | sharded-vec-writer |
| version | 0.4.0 |
| created_at | 2024-08-10 12:08:16.745928+00 |
| updated_at | 2025-08-22 01:35:28.048373+00 |
| description | Write parts of a Vec from different threads |
| homepage | |
| repository | https://github.com/davidlattimore/sharded-vec-writer |
| max_upload_size | |
| id | 1332338 |
| size | 38,119 |
This crate is intended for use when you want to build a Vec<T> and have separate threads
initialise separate parts of the vec.
Example usage:
use sharded_vec_writer::VecWriter;
let shard_sizes = [4, 3, 7];
let total_size = shard_sizes.iter().sum();
// Create the vec with sufficient capacity for whatever we'd like to put in it.
let mut data = Vec::with_capacity(total_size);
// Create a writer - this mutably borrows the vec.
let mut writer = VecWriter::new(&mut data);
// Split the writer into some number of shards up to the capacity of the Vec.
let mut shards = writer.take_shards(shard_sizes.into_iter());
std::thread::scope(|scope| {
// Write to the shards. This can be done from separate threads.
for shard in &mut shards {
scope.spawn(|| {
for i in 0..shard.len() {
shard.push(i);
}
});
}
});
// This resizes the Vec to have length `total_size`. The shards need to all have been fully
// written, otherwise this will panic. For a non-panicking version, you can use
// `try_return_shards`.
writer.return_shards(shards);
assert_eq!(data, &[0, 1, 2, 3, 0, 1, 2, 0, 1, 2, 3, 4, 5, 6]);
Licensed under either of Apache License, Version 2.0 or MIT license at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Wild by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.