Rust crate for dYdX Chain protobufs

Usage as a dependency

Cargo.toml

[dependencies]
dydx-proto = "0.4.0"

Note: by default, rust stub files are not rebuilt (see Q&A below)

For more idiomatic Rust you can use conversions (try_into and into) for the following types:

• prost_types::Timestamp -> std::time::SystemTime
• prost_types::Duration-> std::time::Duration

Local development

Prerequisites

1. Rust
2. Buf - to resolve 3rd-party dependencies for protobuf files
3. protoc - to compile protobuf files with their 3rd-party dependencies
4. cargo deny - to check for security/license/sources issues

Then for a code (re-)generation run

V4_PROTO_REBUILD=1 cargo build -vv

Before publishing make sure to run (and fix all warnings and errors)

cargo fmt
cargo clippy
cargo deny check licenses advisories sources

Q&A

1. Why do we put autogenerated files to the crate (and git) and do not (re-)generate them at compilation?

   For several reasons:

   • reproducibility of the dependency
   • to avoid external dependencies for the lib users (protoc and buf are only needed for code generation)

   But if a user wants to (re-)generate at compilation time, he/she can set an environment variable V4_PROTO_REBUILD (to any value).

2. Why do I need a protoc for this crate development? I thought prost-build crate generates everything natively with Rust?

   The main work (parsing, linking, etc. - have a look https://protobuf.com/docs/descriptors) is done by protoc. The result of the protoc work is a "file descriptor" (think of it as IR assembly language like LLVM IR) - a binary file. This file descriptor is an input for a language-specific code generator like prost. Think of prost crate as a compiler target which generates a ISA-specific "assembly" (in our case, Rust) as an output. prost-build always used the protoc but since version 0.11 of prost-build it requires protoc (the protobuf compiler) to be already installed on the system - before the protoc could be compiled during the prost-build build (https://github.com/tokio-rs/prost/blob/v0.10.4/prost-build/build.rs#L77).

3. Why do we use tonic-build crate and not just prost-build?

   prost-build generates only serialization-deserialization stubs for messages, but we also need a client implementation (generated by tonic-build) because packages in other language implementations of v4-chain have ones.

4. Why do we need buf?

   Buf is a tool whose primary function is to resolve dependencies in protobuf files. Protobuf specifications can refer to 3rd-party protobuf specifications and use types declared there. Basically, buf builds a list of all used protobuf files, downloads them, and allows exporting (=copying) them to a specified directory. The proto files in this repository and downloaded 3rd-party proto files (aka "includes") are an input for the protoc.