# API style guidelines

Generally follow guidance at https://cloud.google.com/apis/design/, in
particular for proto3 as described at:

* https://cloud.google.com/apis/design/proto3
* https://cloud.google.com/apis/design/naming_convention
* https://developers.google.com/protocol-buffers/docs/style

A key aspect of our API style is maintaining stability by following the [API versioning
guidelines](API_VERSIONING.md). All developers must familiarize themselves with these guidelines,
any PR which makes breaking changes to the API will not be merged.

In addition, the following conventions should be followed:

* Every proto directory should have a `README.md` describing its content. See
  for example [envoy.service](envoy/service/README.md).

* The data plane APIs are primarily intended for machine generation and consumption.
  It is expected that the management server is responsible for mapping higher
  level configuration concepts to concrete API concepts. Similarly, static configuration
  fragments may be generated by tools and UIs, etc. The APIs and tools used
  to generate xDS configuration are beyond the scope of the definitions in this
  repository.

* Use [wrapped scalar
  types](https://github.com/google/protobuf/blob/master/src/google/protobuf/wrappers.proto)
  if there is any potential need for a field to have a default value that does not
  match the proto3 defaults (0/false/""). For example, new features whose
  default value may change in the future or security mitigations that should be
  default safe in the future but are temporarily not enabled.

* Use a `[#not-implemented-hide:]` `protodoc` annotation in comments for fields that lack Envoy
  implementation. These indicate that the entity is not implemented in Envoy and the entity
  should be hidden from the Envoy documentation.

* For extensions that are a work-in-progress or the config proto documentation is hidden with
  `[#not-implemented-hide:]`, set its `status` field to `wip` in `extensions_metadata.yaml`.

* Use a `(xds.annotations.v3.file_status).work_in_progress`,
  `(xds.annotations.v3.message_status).work_in_progress`, or
  `(xds.annotations.v3.field_status).work_in_progress` option annotation for files,
  messages, or fields, respectively, that are considered work in progress and are not subject to the
  threat model or the breaking change policy. This is similar to the work-in-progress/alpha tagging
  of extensions described below, but allows tagging protos that are used as part of the core API
  as work in progress without having to break them into their own file.

* Always use plural field names for `repeated` fields, such as `filters`.

* Due to the fact that we consider JSON/YAML to be first class inputs, we cannot easily change a
  a singular field to a repeated field (both due to JSON/YAML array structural differences as well
  as singular vs. plural field naming). If there is a reasonable expectation that a field may need
  to be repeated in the future, but we don't need it to be repeated right away, consider making it
  repeated now but using constraints to enforce a maximum repeated size of 1. E.g.:

  ```proto
  repeated OutputSink sinks = 1 [(validate.rules).repeated = {min_items: 1, max_items: 1}];
  ```

* Always use upper camel case names for message types and enum types without embedded
  acronyms, such as `HttpRequest`.

* Prefer multiple fields with defined precedence over boolean overloads of fields or
  `oneof`. For example, prefer:

  ```proto
  // Simple path matcher. If regex_path is set, this field is not used.
  string simple_path = 1;
  // Regex path matcher. If set, takes precedence over simple_path.
  string regex_path = 2;
  ```

  to

  ```proto
  string path = 1;
  bool path_is_regex = 2;
  ```

  or

  ```
  oneof path_specifier {
    string simple_path = 1;
    string regex_path = 2;
  }
  ```

  This is more efficient on the wire. It also allows new alternatives to be
  added later in a way that allows control planes to be backward-compatible.

* The API includes two types for representing [percents](envoy/type/percent.proto). `Percent` is
  effectively a double value in the range 0.0-100.0. `FractionalPercent` is an integral fraction
  that can be used to create a truncated percentage also in the range 0.0-100.0. In high performance
  paths, `FractionalPercent` is preferred as randomness calculations can be performed using integral
  modulo and comparison operations only without any floating point conversions. Typically, most
  users do not need infinite precision in these paths.

* For enum types, if one of the enum values is used for most cases, make it the
  first enum value with `0` numeric value. Otherwise, define the first enum
  value like `TYPE_NAME_UNSPECIFIED = 0`, and treat it as an error. This design
  pattern forces developers to explicitly choose the correct enum value for
  their use case, and avoid misunderstanding of the default behavior.

* For time-related fields, prefer using the well-known types `google.protobuf.Duration` or
  `google.protobuf.Timestamp` instead of raw integers for seconds.

* If a field is going to contain raw bytes rather than a human-readable string, the field should
  be of type `bytes` instead of `string`.

* Proto fields should be sorted logically, not by field number.

## Package organization

API definitions are layered hierarchically in packages from top-to-bottom as following:
- `envoy.extensions` contains all definitions for the extensions, the package should match the structure of the `source` directory.
- `envoy.service` contains gRPC definitions of supporting services and top-level messages for the services.
e.g. `envoy.service.route.v3` contains RDS, `envoy.service.listener.v3` contains LDS.
- `envoy.config` contains other definitions for service configuration, bootstrap and some legacy core types.
- `envoy.data` contains data format declaration for data types that Envoy produces.
- `envoy.type` contains common protobuf types such as percent, range and matchers.

Extensions should use the regular hierarchy. For example, configuration for network filters belongs
in a package under `envoy.extensions.filter.network`.

## Adding an extension configuration to the API

Extensions must currently be added as v3 APIs following the [package
organization](#package-organization) above.
To add an extension config to the API, the steps below should be followed:

1. Place the v3 extension configuration `.proto` in `api/envoy/extensions` or `api/contrib/envoy/extensions`, e.g.
   `api/envoy/extensions/filters/http/foobar/v3/foobar.proto` together with an initial BUILD file:
   ```bazel
   load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")

   licenses(["notice"])  # Apache 2

   api_proto_package(
       deps = ["@com_github_cncf_xds//udpa/annotations:pkg"],
   )
   ```
1. If this is still WiP and subject to breaking changes, please tag it
   `option (xds.annotations.v3.file_status).work_in_progress = true;` and
   optionally hide it from the docs (`[#not-implemented-hide:]`).
1. Make sure your proto imports the v3 extension config proto (`import "udpa/annotations/status.proto";`)
1. Make sure your proto is tracked as ready to be used
   (`option (udpa.annotations.file_status).package_version_status = ACTIVE;`).
   This is required to automatically include the config proto in [api/versioning/BUILD](versioning/BUILD) under `active_protos`.
1. Add a reference to the v3 extension config in [api/BUILD](BUILD) under `v3_protos`.
1. Update [source/extensions/extensions_metadata.yaml](../source/extensions/extensions_metadata.yaml) or [contrib/extensions_metadata.yaml](../contrib/extensions_metadata.yaml)
   with the category, [security posture, and status](../EXTENSION_POLICY.md#extension-stability-and-security-posture).
   * Any extension category added to `extensions_metadata.yaml` should be annotated in precisely one proto file, associated with a field of a proto message. e.g.
     ```proto
     message SomeMessage {
       // An ordered list of http filters
       // [#extension-category: envoy.http.filters]
       repeated core.v3.TypedExtensionConfig http_filter_extensions = 1;
     }
     ```
   * Each extension added to `extensions_metadata.yaml` should have precisely one proto file annotated with the extension name. e.g.
     ```proto
     // [#protodoc-title: Your New Filter]
     // [#extension: envoy.http.filters.your_new_filter]

     // YourFilterConfig is the configuration for a YourFilter (write real documentation here).
     message YourFilterConfig {
     }
     ```
1. If you introduce a new extension category, you'll also need to add its name
   under `categories` in: [tools/extensions/extensions_schema.yaml](../tools/extensions/extensions_schema.yaml).
1. Update
   [source/extensions/extensions_build_config.bzl](../source/extensions/extensions_build_config.bzl) or [contrib/contrib_build_config.bzl](../contrib/contrib_build_config.bzl)
   to include the new extension.
1. If the extension is not hidden, find or create a docs file with a toctree
   and reference your proto to make sure users can navigate to it from the API docs
   (and to not break the docs build), like [docs/root/api-v3/admin/admin.rst](../docs/root/api-v3/admin/admin.rst).
1. Run `./tools/proto_format/proto_format.sh fix`. **Before running the script**, you will need to **commit your local changes**. By adding the commit, the tool will recognize the change, and will regenerate the `BUILD` file and reformat `foobar.proto` as needed. If you have not followed any of the above steps correctly `proto_format.sh` may remove some of the files that you added. If that is the case you can revert to the committed state, and try again once any issues are resolved.
1. See the [key-value-store PR](https://github.com/envoyproxy/envoy/pull/17745/files) for an example of adding a new extension point to common.

## API annotations

A number of annotations are used in the Envoy APIs to provide additional API
metadata. We describe these annotations below by category.

### Field level
* `[deprecated = true]` to denote fields that are deprecated in a major version.
  These fields are slated for removal at the next major cycle and follow the
  [breaking change policy](../CONTRIBUTING.md#breaking-change-policy).
* `[envoy.annotations.disallowed_by_default = true]` to denote fields that have
  been disallowed by default as per the [breaking change policy](../CONTRIBUTING.md#breaking-change-policy).
* `[(udpa.annotations.field_migrate).rename = "<new field name>"]` to denote that
  the field will be renamed to a given name in the next API major version.
* `[(udpa.annotations.sensitive) = true]` to denote sensitive fields that
  should be redacted in output such as logging or configuration dumps.
* [PGV annotations](https://github.com/bufbuild/protoc-gen-validate) to denote field
  value constraints.

### Enum value level
* `[(udpa.annotations.enum_value_migrate).rename = "new enum value name"]` to denote that
  the enum value will be renamed to a given name in the next API major version.

### Message level
* `option (udpa.annotations.versioning).previous_message_type = "<message type
  name>";` to denote the previous type name for an upgraded message. You should
  never have to write these manually, they are generated by `protoxform`.

### Service level
* `option (envoy.annotations.resource).type = "<resource type name>";` to denote
  the resource type for an xDS service definition.

### File level
* `option (udpa.annotations.file_migrate).move_to_package = "<package name>";`
  to denote that in the next major version of the API, the file will be moved to
  the given package. This is consumed by `protoxform`.
* `option (xds.annotations.v3.file_status).work_in_progress = true;` to denote a
  file that is still work-in-progress and subject to breaking changes.

## Principles

The following principles should be adhered to when extending or modifying the
xDS APIs:

* The xDS APIs have a logical distinction between transport and data model:
  - The xDS transport protocol describes the network transport on which xDS
    configuration resources are delivered to clients. A versioned gRPC streaming
    protocol with support for ACK/NACK is provided by xDS; this is known as the
    xDS transport protocol (xDS-TP). xDS configuration resources can also be
    delivered on other transports, e.g. HTTP or filesystem, with some
    limitations (e.g. no version feedback).
  - The xDS data model describes the xDS configuration resources themselves,
    e.g. listeners, route configurations, clusters, endpoints, secrets.

* The xDS APIs are directionally client and server neutral. While many aspects
  of the APIs reflect the history of their origin as Envoy's control plane APIs,
  API decisions going forward should reflect the principle of client neutrality.

* The xDS APIs are expressed canonically as [Proto3](https://developers.google.com/protocol-buffers/docs/proto3).
  Both JSON and YAML are also supported formats, with the standard JSON-proto3
  conversion used during client configuration ingestion.

* xDS APIs are eventual consistency first. For example, if RDS references a
  cluster that has not yet been supplied by CDS, it should be silently ignored
  and traffic not forwarded until the CDS update occurs. Stronger consistency
  guarantees are possible if the management server is able to sequence the xDS
  APIs carefully (for example by using the ADS API below). By following the
  `[CDS, EDS, LDS, RDS]` sequence for all pertinent resources, it will be
  possible to avoid traffic outages during configuration update.

* The API is primarily intended for machine generation and consumption. It is
  expected that the management server is responsible for mapping higher level
  configuration concepts to API responses. Similarly, static configuration
  fragments may be generated by templating tools, etc. With that consideration,
  we also aim to have API artifacts readable by humans for debugging and
  understanding applied configuration. This implies that APIs do not have to
  have ergonomics as the main driver, but should still be reasonable to read by
  humans. The APIs and tools used to generate xDS configuration are beyond the
  scope of the definitions in this repository.

* All supported transports (xDS-TP, HTTP, filesystem) support basic singleton xDS
  subscription services CDS/EDS/LDS/RDS/SDS. Advanced APIs such as HDS, ADS and
  EDS multi-dimensional LB are xDS-TP only. This avoids having to map
  complicated bidirectional stream semantics onto REST, etc..

* Versioning follows the scheme described [here](API_VERSIONING.md). A key
  principle that we target is that API consumers should not be exposed to
  breaking changes where there is no substantial gain in functionality,
  performance, security or implementation simplification. We will tolerate
  technical debt in the API itself, e.g. in the form of vestigial deprecated
  fields or reduced ergonomics in order to meet this principle.

* Namespaces for extensions, metadata, etc. use a reverse DNS naming scheme,
  e.g. `com.google.widget`, `com.lyft.widget`. Client built-ins may be prefixed
  with client name, e.g. `envoy.foo`, `grpc.bar`.