QUIC as a game networking protocol

QUIC seems really attractive as a game networking protocol because most of the hard-work is done by the protocol specification and the implementation (here Quinn). No need to reinvent the wheel once again on error-prones subjects such as a UDP reliability wrapper, encryption & authentication mechanisms or congestion-control.

Most of the features proposed by the big networking libs are supported by default through QUIC. As an example, here is the list of features presented in GameNetworkingSockets:

• Connection-oriented API (like TCP) -> by default
• ... but message-oriented (like UDP), not stream-oriented -> by default (*)
• Supports both reliable and unreliable message types -> by default
• Messages can be larger than underlying MTU. The protocol performs fragmentation, reassembly, and retransmission for reliable messages -> by default (frag & reassembly is not done by the protocol for unreliable packets)
• A reliability layer [...]. It is based on the "ack vector" model from DCCP (RFC 4340, section 11.4) and Google QUIC and discussed in the context of games by Glenn Fiedler [...] -> by default.
• Encryption. [...] The details for shared key derivation and per-packet IV are based on the design used by Google's QUIC protocol -> by default
• Tools for simulating packet latency/loss, and detailed stats measurement -> Not by default
• Head-of-line blocking control and bandwidth sharing of multiple message streams on the same connection. -> by default
• IPv6 support -> by default
• Peer-to-peer networking (NAT traversal with ICE + signaling + symmetric connect mode) -> Not by default
• Cross platform -> by default, where UDP is available

-> Roughly 9 points out of 11 by default.

(*) Almost, when sharing a QUIC stream, reliable messages need to be framed.

Features

Quinnet can be used as a transport layer. It currently features:

• A Client plugin which can:
  • Connect/disconnect to/from one or more server
  • Send & receive unreliable and ordered/unordered reliable messages
• A Server plugin which can:
  • Accept client connections & disconnect them
  • Send & receive unreliable and ordered/unordered reliable messages
• Both client & server accept custom protocol structs/enums defined by the user as the message format, as well as raw bytes.
• Communications are encrypted, and the client can authenticate the server.

Although Quinn and parts of Quinnet are asynchronous, the APIs exposed by Quinnet for the client and server are synchronous. This makes the surface API easy to work with and adapted to a Bevy usage. The implementation uses tokio channels internally to communicate with the networking async tasks.

Quickstart

Client

• Add the QuinnetClientPlugin to the bevy app:

 App::new()
        // ...
        .add_plugins(QuinnetClientPlugin::default())
        // ...
        .run();

• You can then use the Client resource to connect, send & receive messages:

fn start_connection(client: ResMut<QuinnetClient>) {
    client
        .open_connection(ClientConnectionConfiguration {
            addr_config: ClientAddrConfiguration::from_ips(
                SERVER_HOST,
                SERVER_PORT,
                LOCAL_BIND_IP,
                0,
            ),
            cert_mode: CertificateVerificationMode::SkipVerification,
            defaultables: Default::default(),
        });
    
    // Once connected, you will receive a ConnectionEvent

• To process server messages, you can use a bevy system such as the one below. The function receive_message is generic, here ServerMessage is a user provided enum deriving Serialize and Deserialize.

fn handle_server_messages(
    mut client: ResMut<QuinnetClient>,
    /*...*/
) {
    while let Some(message) = client.connection_mut().try_receive_message() {
        match message {
            // Match on your own message types ...
            ServerMessage::ClientConnected { client_id, username} => {/*...*/}
            ServerMessage::ClientDisconnected { client_id } => {/*...*/}
            ServerMessage::ChatMessage { client_id, message } => {/*...*/}
        }
    }
}

Server

• Add the QuinnetServerPlugin to the bevy app:

 App::new()
        /*...*/
        .add_plugins(QuinnetServerPlugin::default())
        /*...*/
        .run();

• You can then use the Server resource to start the listening server:

fn start_listening(mut server: ResMut<QuinnetServer>) {
    server
        .start_endpoint(ServerEndpointConfiguration {
            addr_config: EndpointAddrConfiguration::from_ip(Ipv6Addr::UNSPECIFIED, 6000),
            cert_mode: CertificateRetrievalMode::GenerateSelfSigned {
                server_hostname: Ipv6Addr::LOCALHOST.to_string(),
            },
            defaultables: Default::default(),
        });
}

• To process client messages & send messages, you can use a bevy system such as the one below. The function receive_message is generic, here ClientMessage is a user provided enum deriving Serialize and Deserialize.

fn handle_client_messages(
    mut server: ResMut<QuinnetServer>,
    /*...*/
) {
    let mut endpoint = server.endpoint_mut();
    for client_id in endpoint.clients() {
        while let Some(message) = endpoint.try_receive_message(client_id) {
            match message {
                // Match on your own message types ...
                ClientMessage::Join { username} => {
                    // Send a messsage to 1 client
                    endpoint.try_send_message(client_id, ServerMessage::InitClient {/*...*/});
                    /*...*/
                }
                ClientMessage::Disconnect { } => {
                    // Disconnect a client
                    endpoint.disconnect_client(client_id);
                    /*...*/
                }
                ClientMessage::ChatMessage { message } => {
                    // Send a message to a group of clients
                    endpoint.try_send_group_message(
                            client_group, // Iterator of ClientId
                            ServerMessage::ChatMessage {/*...*/}
                        );
                    /*...*/
                }           
            }
        }
    }
}

You can also use endpoint.broadcast_message, which will send a message to all connected clients. "Connected" here means connected to the server plugin, which happens before your own app handshakes/verifications if you have any. Use send_group_message if you want to control the recipients.

Channels

There are currently 3 types of channels available when you send a message:

• OrderedReliable: ensure that messages sent are delivered, and are processed by the receiving end in the same order as they were sent (exemple usage: chat messages)
• UnorderedReliable: ensure that messages sent are delivered, in any order (exemple usage: an animation trigger)
• Unreliable: no guarantees on the delivery or the order of processing by the receiving end (exemple usage: an entity position sent every ticks)

When you open a connection/endpoint, some channels are created directly according to the given SendChannelsConfiguration.

// Default channels configuration contains only 1 channel of the OrderedReliable type,
// akin to a TCP connection.
let channels_config = SendChannelsConfiguration::default();
// Creates 2 OrderedReliable channels, and 1 unreliable channel,
// with channel ids being respectively 0, 1 and 2.
let channels_config = SendChannelsConfiguration::from_configs(vec![
    ChannelConfig::default_ordered_reliable(),
    ChannelConfig::default_ordered_reliable(),
    ChannelConfig::default_reliable()]);

Each channel is identified by its own ChannelId. Among those, there is a default channel which will be used when you don't specify the channel. At startup, the first opened channel becomes the default channel.

let connection = client.connection();
// No channel specified, default channel is used
connection.send_message(message);
// Specifying the channel id
connection.send_message_on(channel_id, message);
// Changing the default channel
connection.set_default_channel(channel_id);

In some cases, you may want to create more than one channel instance of the same type. As an example, using multiple OrderedReliable channels to avoid some Head of line blocking issues. Although channels can be defined through a SendChannelsConfiguration, they can also currently be opened & closed at any time. You may have up to 256 differents channels opened simultaneously.

// If you want to create more channels
let chat_channel = client.connection().open_channel(ChannelConfig::default()).unwrap();
client.connection().send_message_on(chat_channel, chat_message);

On the server, channels are created and closed at the endpoint level and exist for all current & future clients.

let chat_channel = server.endpoint().open_channel(ChannelConfig::default()).unwrap();
server.endpoint().send_message_on(client_id, chat_channel, chat_message);

Certificates and server authentication

Bevy Quinnet (through Quinn & QUIC) uses TLS 1.3 for authentication, the server needs to provide the client with a certificate confirming its identity, and the client must be configured to trust the certificates it receives from the server.

Here are the current options available to the server and client plugins for the server authentication:

• Client :
  • Skip certificate verification (messages are still encrypted, but the server is not authentified)
  • Accept certificates issued by a Certificate Authority (implemented in Quinn, using rustls)
  • Trust on first use certificates (implemented in Quinnet, using rustls)
• Server:
  • Generate and issue a self-signed certificate
  • Issue an already existing certificate (CA or self-signed)

On the client:

    // To accept any certificate
    client.open_connection(/*...*/ CertificateVerificationMode::SkipVerification);
    // To only accept certificates issued by a Certificate Authority
    client.open_connection(/*...*/ CertificateVerificationMode::SignedByCertificateAuthority);
    // To use the default configuration of the Trust on first use authentication scheme
    client.open_connection(/*...*/ CertificateVerificationMode::TrustOnFirstUse(TrustOnFirstUseConfig {
            // You can configure TrustOnFirstUse through the TrustOnFirstUseConfig:
            // Provide your own fingerprint store variable/file,
            // or configure the actions to apply for each possible certificate verification status.
            ..Default::default()
        }),
    );

On the server:

    // To generate a new self-signed certificate on each startup 
    server.start_endpoint(/*...*/ CertificateRetrievalMode::GenerateSelfSigned { 
        server_hostname: Ipv6Addr::LOCALHOST.to_string(),
    });
    // To load a pre-existing one from files
    server.start_endpoint(/*...*/ CertificateRetrievalMode::LoadFromFile {
        cert_file: "./certificates.pem".into(),
        key_file: "./privkey.pem".into(),
    });
    // To load one from files, or to generate a new self-signed one if the files do not exist.
    server.start_endpoint(/*...*/ CertificateRetrievalMode::LoadFromFileOrGenerateSelfSigned {
        cert_file: "./certificates.pem".into(),
        key_file: "./privkey.pem".into(),
        save_on_disk: true, // To persist on disk if generated
        server_hostname: Ipv6Addr::LOCALHOST.to_string(),
    });

See more about certificates in the certificates readme

Examples

cargo run --example chat-server --features=bincode-messages

cargo run --example chat-client --features=bincode-messages

cargo run --example breakout --features=bincode-messages

Examples can be found in the examples directory.

Replicon integration

Bevy Quinnet can be used as a transport in bevy_replicon with the provided bevy_replicon_quinnet.

Compatible Bevy versions

Misc

Cargo features

Find the list and description in cargo.toml

Logs

For logs configuration, see the unoffical bevy cheatbook.

Limitations

• QUIC is not available directly in a Browser (it is used by Browsers but not exposed directly as an API). WebTransport is starting to be available on Browsers, however this crate does not provide nor use a WebTransport implementation, and as such, does not support Browser target.

Credits

Thanks to the Renet crate for the initial inspiration on the high level API.

License

This crate is free and open source. All code in this repository is dual-licensed under either:

• MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)
• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)

at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.