tokio-console subscriber

📡️ A tracing-subscriber Layer for collecting tokio-console telemetry.

Website | Chat | API Documentation

Overview

tokio-console is a debugging and profiling tool for asynchronous Rust applications, which collects and displays in-depth diagnostic data on the asynchronous tasks, resources, and operations in an application. The console system consists of two primary components:

• instrumentation, embedded in the application, which collects data from the async runtime and exposes it over the console's wire format
• consumers, such as the tokio-console command-line application, which connect to the instrumented application, receive telemetry data, and display it to the user

This crate implements the instrumentation-side interface using data emitted by the async runtime using the tracing. It provides a type implementing the Layer trait from tracing-subscriber, for collecting and aggregating the runtime's tracing data, and a gRPC server that exports telemetry to clients.

Getting Started

To instrument your asynchronous application, you must be using an async runtime that supports the tracing instrumentation required by the console. Currently, the only runtime that implements this instrumentation is Tokio version 1.7.0 and newer.

Enabling Tokio Instrumentation

⚠️ Currently, the tracing support in the tokio runtime is considered experimental. In order to use console-subscriber with Tokio, the following is required:

• Tokio's optional tracing dependency must be enabled. For example:

  [dependencies]
  # ...
  tokio = { version = "1.15", features = ["full", "tracing"] }
  

• The tokio_unstable cfg flag, which enables experimental APIs in Tokio, must be enabled. It can be enabled by setting the RUSTFLAGS environment variable at build-time:

  $ RUSTFLAGS="--cfg tokio_unstable" cargo build
  

  or, by adding the following to the .cargo/config.toml file in a Cargo workspace:

  [build]
  rustflags = ["--cfg", "tokio_unstable"]
  

  If you're using a workspace, you should put the .cargo/config.toml file in the root of your workspace. Otherwise, put the .cargo/config.toml file in the root directory of your crate.

  Putting .cargo/config.toml files below the workspace or crate root directory may lead to tools like Rust-Analyzer or VSCode not using your .cargo/config.toml since they invoke cargo from the workspace or crate root and cargo only looks for the .cargo directory in the current & parent directories. Cargo ignores configurations in child directories. More information about where cargo looks for configuration files can be found here.

  Missing this configuration file during compilation will cause tokio-console to not work, and alternating between building with and without this configuration file included will cause full rebuilds of your project.

• The tokio and runtime tracing targets must be enabled at the TRACE level.

  • If you're using the console_subscriber::init() or console_subscriber::Builder APIs, these targets are enabled automatically.

  • If you are manually configuring the tracing subscriber using the EnvFilter or Targets filters from tracing-subscriber, add "tokio=trace,runtime=trace" to your filter configuration.

  • Also, ensure you have not enabled any of the compile time filter features in your Cargo.toml.

Required Tokio Versions

Because instrumentation for different aspects of the runtime is being added to Tokio over time, the latest Tokio release is generally recommended to access all of the console's functionality. However, it should generally be compatible with earlier Tokio versions, although some information may not be available. A minimum version of Tokio v1.0.0 or later is required to use the console's task instrumentation.

Other instrumentation is added in later Tokio releases:

• Tokio v1.7.0 or later is required to record task waker instrumentation (such as waker counts, clones, drops, et cetera).

• Tokio v1.12.0 or later is required to record tasks created by the Runtime::block_on and Handle::block_on methods.

• Tokio v1.13.0 or later is required to track tokio::time resources, such as sleep and Interval.

• Tokio v1.15.0 or later is required to track tokio::sync resources, such as Mutexes, RwLocks, Semaphores, oneshot channels, mpsc channels, et cetera.

• Tokio v1.21.0 or later is required to use newest task::Builder::spawn* APIs.

Adding the Console Subscriber

If the runtime emits compatible tracing events, enabling the console is as simple as adding the following line to your main function:

hds_console_subscriber::init();

This sets the default tracing subscriber to serve console telemetry (as well as logging to stdout based on the RUST_LOG environment variable). The console subscriber's behavior can be configured via a set of environment variables.

For programmatic configuration, a builder interface is also provided:

use std::time::Duration;

hds_console_subscriber::ConsoleLayer::builder()
    // set how long the console will retain data from completed tasks
    .retention(Duration::from_secs(60))
    // set the address the server is bound to
    .server_addr(([127, 0, 0, 1], 5555))
    // ... other configurations ...
    .init();

The layer provided by this crate can also be combined with other Layers from other crates:

use tracing_subscriber::prelude::*;

// spawn the console server in the background,
// returning a `Layer`:
let console_layer = hds_console_subscriber::spawn();

// build a `Subscriber` by combining layers with a
// `tracing_subscriber::Registry`:
tracing_subscriber::registry()
    // add the console layer to the subscriber
    .with(console_layer)
    // add other layers...
    .with(tracing_subscriber::fmt::layer())
 // .with(...)
    .init();

Using other runtimes

If you are using a custom runtime that supports tokio-console, you may not need to enable the tokio_unstable cfg flag. In this case, you need to enable cfg console_without_tokio_unstable for console-subscriber to disable its check for tokio_unstable.

Crate Feature Flags

This crate provides the following feature flags and optional dependencies:

• parking_lot: Use the parking_lot crate's locks, rather than std::sync. Using parking_lot may result in improved performance, especially in highly concurrent applications. Disabled by default.

Getting Help

First, see if the answer to your question can be found in the API documentation. If the answer is not there, there is an active community in the Tokio Discord server. We would be happy to try to answer your question. You can also ask your question on the discussions page.

Contributing

🎈 Thanks for your help improving the project! We are so happy to have you! We have a contributing guide to help you get involved in the Tokio console project.

Supported Rust Versions

The Tokio console is built against the latest stable release. The minimum supported version is 1.64. The current Tokio console version is not guaranteed to build on Rust versions earlier than the minimum supported version.

License

This project is licensed under the MIT license.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Tokio by you, shall be licensed as MIT, without any additional terms or conditions.