Crates.io | byteserde_derive |
lib.rs | byteserde_derive |
version | 0.6.2 |
source | src |
created_at | 2023-05-17 23:09:10.701197 |
updated_at | 2024-01-14 18:07:18.378983 |
description | A procedural macro for mapping byte streams to/from arbitrary struct types with focus on performance |
homepage | |
repository | https://github.com/softstream-link/byteserde |
max_upload_size | |
id | 867284 |
size | 64,454 |
The motivation for this product is two fold:
To be able to map a byte slice &[u8]
, typically acquired from a network, into a rust struct
datamodel by simply using derive macro
annotations and attributes auto-generate necessary code.
This comes very handy when the byte slice
is not serialized using one of the existing and widely available protocols. Ex: An application specific C-Struct.
To be the fastest
byte stream serializer/deserializer on the market for latency sensitive use cases. Benchmark results below show a performance summary of serializing & deserializing a Reference Struct using different frameworks available:
byteserde
- ~15ns
read/writebincode
- ~15ns
read / ~100ns
writermp-serde
- ~215ns
read/writeserde_json
- ~600ns
read/write - understandably slow due to strings usage
this document contains benchmark details.
use
and when to avoid
this frameworkIf you work with network protocols that deliver data in a byte stream
format that is not matching one of the widely available standards, ex: bincode
, protobuf
. Use this product to efficiently map your byte stream
into a rust struct
.
You have a latency sensitive usecase. Note that this protocol does not add any schema information during serialization process and hence is equivalent of dumping the memory layout of the struct without padding
Example of protocols that are a perfect fit for this framework.
byte stream
is serialized or deserialized using a wideley available standard avoid this framework and instead the that respective standard to work with the byte stream
#[derive(ByteSerializeStack)]
- generates ByteSerializeStack trait
#[derive(ByteSerializeHeap)]
- generates ByteSerializeHeap trait
#[derive(ByteDeserializeSlice)]
- generates ByteDeserializeSlice<T>
trait
#[derive(ByteSerializedSizeOf)]
- generates ByteSerializedSizeOf trait - this trait provides an associated
method byte_size()
which gives you a struct
memory size in bytes without alignment. However it does not support types which heap allocate, ex: Vectors, Strings, or their derivations.
#[derive(ByteSerializedLenOf)]
- generates ByteSerializedLenOf trait - this trait provides an instance
method byte_len(&self)
which gives you memory size in bytes without alignment of specific instance. It exists specifically to deal with types that ByteSerializedSizeOf trait
does not support
ByteSerializerStack<CAP>
- provides ultra fast serializer into a pre allocated byte array
[u8; CAP]
on stack
, hence the name, it is very fast but at the cost of you needing to specify the size of the LARGEST struct
you will attempt to serialize. If you reach the boundary of this preallocated byte array, your serialization will fail. This utility provides a reset features, which moves the internal counter to the begining, and allows you to recycle the buffer multiple times.
struct
s that implement ByteSerializeStack traitByteSerializerHeap - provides a fast enough for most speed by serializing into a byte vector
Vec<u8>
, hence the name. This utility trades some performance in return for not having to worry about knowing the LARGEST struct
size in advance.
struct
s that implement ByteSerializeHeap traitByteDeserializerSlice - takes a byte stream
&[u8]
irrespctive of heap vs stack allocation and turns it into a struct
works for struct
s that implement ByteDeserializeSlice<T>
trait
byte stream
, follow example section for more details.Please refer to this document for a number of comprehensive examples and features overview.