# ` proto-gen`
**Protobuf to `Rust` code generation using tonic-build**
[](https://embark.dev)
[](https://discord.gg/dAuKfZS)
[](https://deps.rs/repo/github/EmbarkStudios/proto-gen)
[](https://github.com/EmbarkStudios/proto-gen/actions)
## What
The repo contains a a cli that uses tonic-build to generate rust-code from protobuf-files.
[tonic-build](https://docs.rs/tonic-build/latest/tonic_build/) already does this, the cli is a front-end to
that with some added code to make sure that the generated files are placed in a valid path, and takes care of the
module structuring.
## Why
[prost-build](https://docs.rs/prost-build/latest/prost_build/) used to ship with `cmake` which we would like to avoid.
`cmake` was used to build `protoc` which was then used for the proto-to-rust codegen.
The final decision from the prost maintainers side is that the user should provide their own protoc and check in the code
instead of building it in a `build.rs`, to make that process simpler, this cli was created.
## How
Uses [a local version of protoc on path](https://grpc.io/docs/protoc-installation/) to generate rust code form a
workspace specification passed to the CLI. The CLI can also be used to validate generated Rust code against source
protocol files, for example, in CI.
## Considerations
There are quite a few problems with this tool, for now it remains the best alternative for our Protobuf pipeline,
some things to take into account follows.
### Requires manual installations on the user's machine
Running this tool requires at minimum `protoc` to be installed on the user's machine,
many protocol buffers found in the wild have dependencies in the form of other `.proto` files. Which also
have to be sourced.
Many `.proto` files that you might want to build have dependencies on the `google.protobuf`
[types](https://protobuf.dev/reference/protobuf/google.protobuf/),
many `.proto` authors assume that these are available to the user, which may or may not be true.
There has been discussions about vendoring them here since they are so common but for now no vendoring will be done,
since they are not mandatory in many cases.
A middle-ground is to vendor the dependency protos yourself, and run this tool with the directory where you placed those
`.proto` files specified (`-d my-dep-dir`).
### Version incompatibility if tonic version has changed (even minor versions)
On tonic minor versions sometimes comments or other non-functional changes are made, these will cause the
`validate` command to reject old protos generated by a lower version of this tool, requiring regeneration.
It could be argued that therefore a minor version change in `tonic` becomes a major version change
for this tool, although that may be a bit philosophical since the generated code runs the same.
### Doc comments are executed
Tonic converts protobuf comments to doc comments, if these include code examples, cargo will attempt to execute them
when running `cargo test` (`doc-test` stage).
This behaviour is different form using `tonic-build`, since those doc-comments are generated where cargo doesn't try
to run them.
Running them is almost always guaranteed to fail in the best case (because they will be non-rust code or not code at all),
and be a lurking security vulnerability in the worst case where someone have inserted malicious Rust-code in a doc-comment.
Ideally it'd be easier to disable running doc-test on a per-module basis. Placing generated protos in a separate lib
which doesn't run doc-comments is a safer choice, although doc-tests being opt-in would be significantly safer.
### Usage
Generate types from a `.proto` into an project.
#### Example in this project
```bash
cargo r -r -p proto-gen -- generate -d examples/example-project/proto -f examples/example-project/proto/my-proto.proto -o examples/example-project/src/proto_types
```
This will generate Rust code from the proto specified in `examples/example-project/proto/my-proto.proto` and place it
in `examples/example-project/src/proto_types`.
```bash
cargo r -r -p proto-gen -- validate -d examples/example-project/proto -f examples/example-project/proto/my-proto.proto -o examples/example-project/src/proto_types
```
This will also generate Rust code (to a temporary directory) and the run a diff against the code contained in `examples/example-project/src/proto_types`.
If it finds any diffs it will exit with error code 1 and a message.
If we want to use includes, the directory to include needs to be specified.
```bash
cargo r -r -p proto-gen -- generate -d examples/example-project/dep_protos -d examples/example-project/proto_with_deps -f examples/example-project/proto_with_deps/my-proto.proto -o examples/example-project/src/proto_types
```
Here we're passing -d twice, once to include the dependency protos, and one to include the protos we want to generate.
```bash
cargo r -r -p proto-gen -- validate -d examples/example-project/dep_protos -d examples/example-project/proto_with_deps -f examples/example-project/proto_with_deps/my-proto.proto -o examples/example-project/src/proto_types
```
## Contributing
[](CODE_OF_CONDUCT.md)
We welcome community contributions to this project.
Please read our [Contributor Guide](CONTRIBUTING.md) for more information on how to get started.
Please also read our [Contributor Terms](CONTRIBUTING.md#contributor-terms) before you make any contributions.
Any contribution intentionally submitted for inclusion in an Embark Studios project, shall comply with the Rust standard licensing model (MIT OR Apache 2.0) and therefore be dual licensed as described below, without any additional terms or conditions:
### License
This contribution is dual licensed under EITHER OF
- Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or )
- MIT license ([LICENSE-MIT](LICENSE-MIT) or )
at your option.
For clarity, "your" refers to Embark or any other licensee/user of the contribution.