Crates.io | batch-channel |
lib.rs | batch-channel |
version | 0.4.2 |
source | src |
created_at | 2023-08-03 03:20:13.185419 |
updated_at | 2024-10-31 03:55:14.922449 |
description | async channel that reduces overhead by reading and writing many values at once |
homepage | |
repository | https://github.com/chadaustin/batch-channel |
max_upload_size | |
id | 933353 |
size | 119,123 |
async MPMC channel that reduces overhead by reading and writing many values at once.
Sometimes large volumes of small values are farmed out to workers through a channel. Consider directory traversal: each readdir() call produces a batch of directory entries. Batching can help the consumption side too. Consider querying SQLite with received values -- larger batches reduce the amortized cost of acquiring SQLite's lock and bulk queries can be issued.
One can imagine natural, currently unimplemented, extensions to this crate:
futures::sink::Sink
and futures::stream::Stream
Ask if they would be helpful.