benchmark-rs

Crates.iobenchmark-rs
lib.rsbenchmark-rs
version0.1.1
sourcesrc
created_at2023-04-21 04:49:11.941203
updated_at2023-08-13 11:45:56.208189
descriptionBenchmarks for Rust libraries
homepagehttps://www.navigatorsguild.io/
repositoryhttps://github.com/navigatorsguild/benchmark-rs
max_upload_size
id845022
size136,070
Giora Kosoi (giora-kosoi-ng)

documentation

README

Maintenance

benchmark-rs

A benchmarking library for Rust package authors.

Benchmark-rs crate provides tools for Rust package authors to evaluate performance of their implementation for varying workloads with varying configurations, find performance regressions between versions and also some crude facilities to measure execution times of code blocks.

For example, if you implement a concurrent algorithm with an expectation of reduced execution time when more CPU cores are added, benchmark-rs will help validate that the concurrency has the desired effect and that there is no anomaly for varying data sets, that is, the algorithm performs consistently within boundaries defined for it.

However, this doesn't have to be a fancy algorithm that we want to test, some banal looking code can exhibit radically different behavior for different data sets, depending on database queries, buffer sizes, number of open files, etc. We may want to compare the behavior of our code with different querying strategies or different buffer sizes for a range of data sets to find the best fit for our use-case.

Another common use-case is to verify that there is no regression in performance introduced by the new code. Benchmark-rs supports comparison to previous results with user defined equality threshold.

Design

Benchmark-rs is designed to evaluate performance of subjects processing a set of workloads with modifiable configurations. For example if we write a protocol parser we may define our workload points as 10MB, 20MB, ..., 100MB input sizes. Workloads are user defined in benchmark-rs. Any type W: Clone + Display can be used to specify workloads. It could be an integer that specifies the size of the workload, a path to a file or a key that we can use to fetch the workload from the benchmark configuration. The Display is required for the workload point to produce the key in result series, so it is recommended to keep it short and descriptive. See examples below.

Each benchmark is repeated repeat times for every workload point after it was ramped up for that point. Upon completion the summary is available as JSON or as CSV.

Issues

Issues are welcome and appreciated. Please submit to https://github.com/navigatorsguild/benchmark-rs/issues

Examples

A real world example can be found at command-executor project blocking_queue.rs benchmark and at Benchmarks wiki page which was built from the generated data.

link

Simple Benchmark

A simple benchmark that measures execution time for increasing workloads. In this case the workload is simulated by by a u64 value passed to thread::sleep function

use std::thread;
use std::time::Duration;
use benchmark_rs::benchmarks::Benchmarks;
use benchmark_rs::stopwatch::StopWatch;

fn example(_stop_watch: &mut StopWatch, _config: &str, work: u64) -> Result<(), anyhow::Error> {
    thread::sleep(Duration::from_millis(work));
    Ok(())
}

fn main() -> Result<(), anyhow::Error> {
    let mut benchmarks = Benchmarks::new("Example");
    benchmarks.add("A Simple Benchmark", example, "No Configuration", (1..=10).collect(), 2, 1)?;
    benchmarks.run()?;

    let summary = benchmarks.summary_as_json();
    println!("Summary: {summary}");
    Ok(())
}

Benchmark Workloads

A more complex example that shows how to use Benchmark configuration and how to control the stopwatch from within the benchmark to avoid measuring the housekeeping tasks.

Benchmark Workloads Example 
use std::collections::BTreeMap;
use std::fmt::{Display, Formatter};
use std::thread;
use std::time::Duration;

use benchmark_rs::benchmarks::Benchmarks;
use benchmark_rs::stopwatch::StopWatch;

#[derive(Clone)]
struct Config {
    // simulate available resources - CPU cores, memory buffers, etc.
    pub resources: u32,
    pub workloads: BTreeMap<u64, Duration>,
}

impl Display for Config {
    fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
        let keys: Vec<String> = self.workloads.keys().map(|k| k.to_string()).collect();
        write!(f, "{}", keys.join(", "))
    }
}

fn example(stop_watch: &mut StopWatch, config: Config, work: u64) -> Result<(), anyhow::Error> {
    stop_watch.pause();
    // perform potentially lengthy preparation that will not reflect in the measurement
    // ...
    // fetch the workload definition from configuration associated with the 'work' point
    let sleep_time = config.workloads.get(&work).unwrap().clone();
    // resume the stopwatch to measure the actual work
    stop_watch.resume();
    // perform measured computation
    thread::sleep(sleep_time / config.resources);
    stop_watch.pause();
    // perform potentially lengthy cleanup
    // ...
    Ok(())
}

fn main() -> Result<(), anyhow::Error> {
    let mut benchmarks = Benchmarks::new("benchmark-workloads");
    let workloads: BTreeMap<u64, Duration> =
        (0..=10).map(|i| (i, Duration::from_millis(i))).collect();
    benchmarks.add(
        "benchmark-workload-1",
        example,
        Config {
            resources: 1,
            workloads: workloads.clone(),
        },
        (1..=10).collect(),
        2,
        1,
    )?;

    benchmarks.add(
        "benchmark-workload-2",
        example,
        Config {
            resources: 2,
            workloads: workloads.clone(),
        },
        (1..=10).collect(),
        2,
        1,
    )?;

    benchmarks.run()?;

    let summary = benchmarks.summary_as_json();
    println!("Benchmark summary in JSON format.");
    println!("Summary:");
    println!("{summary}");
    println!();

    println!("Benchmark series in CSV format.");
    let csv_data = benchmarks.summary_as_csv(true, false);
    for (k, v) in csv_data {
        println!("Benchmark name: {k}");
        for line in v {
            println!("{line}")
        }
        println!();
    }

    Ok(())
}
Benchmark Workloads Summary 
{
  "name": "benchmark-workloads",
  "created_at": "2023-08-13 04:09:13.923036",
  "series": {
    "benchmark-workload-2": {
      "name": "benchmark-workload-2",
      "config": "0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10",
      "runs": [
        [
          "1",
          {
            "name": "benchmark-workload-2",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 639791,
            "min_sec": 0.000639791,
            "min_str": "00:00:00.000",
            "max_nanos": 644250,
            "max_sec": 0.00064425,
            "max_str": "00:00:00.000",
            "median_nanos": 642020,
            "median_sec": 0.00064202,
            "median_str": "00:00:00.000",
            "std_dev": 3152.9891373108153,
            "std_dev_sec": 3.1529891373108154e-6,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "2",
          {
            "name": "benchmark-workload-2",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 1264124,
            "min_sec": 0.001264124,
            "min_str": "00:00:00.001",
            "max_nanos": 1264250,
            "max_sec": 0.00126425,
            "max_str": "00:00:00.001",
            "median_nanos": 1264187,
            "median_sec": 0.001264187,
            "median_str": "00:00:00.001",
            "std_dev": 89.09545442950498,
            "std_dev_sec": 8.909545442950498e-8,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "3",
          {
            "name": "benchmark-workload-2",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 1891833,
            "min_sec": 0.001891833,
            "min_str": "00:00:00.001",
            "max_nanos": 1892875,
            "max_sec": 0.001892875,
            "max_str": "00:00:00.001",
            "median_nanos": 1892354,
            "median_sec": 0.001892354,
            "median_str": "00:00:00.001",
            "std_dev": 736.8052659963826,
            "std_dev_sec": 7.368052659963826e-7,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "4",
          {
            "name": "benchmark-workload-2",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 2566417,
            "min_sec": 0.002566417,
            "min_str": "00:00:00.002",
            "max_nanos": 2576416,
            "max_sec": 0.002576416,
            "max_str": "00:00:00.002",
            "median_nanos": 2571416,
            "median_sec": 0.002571416,
            "median_str": "00:00:00.002",
            "std_dev": 7070.360705084288,
            "std_dev_sec": 7.0703607050842884e-6,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "5",
          {
            "name": "benchmark-workload-2",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 2928666,
            "min_sec": 0.002928666,
            "min_str": "00:00:00.002",
            "max_nanos": 3174917,
            "max_sec": 0.003174917,
            "max_str": "00:00:00.003",
            "median_nanos": 3051791,
            "median_sec": 0.003051791,
            "median_str": "00:00:00.003",
            "std_dev": 174125.75197396852,
            "std_dev_sec": 0.00017412575197396852,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "6",
          {
            "name": "benchmark-workload-2",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 3377791,
            "min_sec": 0.003377791,
            "min_str": "00:00:00.003",
            "max_nanos": 3784625,
            "max_sec": 0.003784625,
            "max_str": "00:00:00.003",
            "median_nanos": 3581208,
            "median_sec": 0.003581208,
            "median_str": "00:00:00.003",
            "std_dev": 287675.0802172479,
            "std_dev_sec": 0.00028767508021724787,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "7",
          {
            "name": "benchmark-workload-2",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 4404499,
            "min_sec": 0.004404499,
            "min_str": "00:00:00.004",
            "max_nanos": 4915541,
            "max_sec": 0.004915541,
            "max_str": "00:00:00.004",
            "median_nanos": 4660020,
            "median_sec": 0.00466002,
            "median_str": "00:00:00.004",
            "std_dev": 361361.26367113565,
            "std_dev_sec": 0.00036136126367113564,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "8",
          {
            "name": "benchmark-workload-2",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 4882584,
            "min_sec": 0.004882584,
            "min_str": "00:00:00.004",
            "max_nanos": 5029209,
            "max_sec": 0.005029209,
            "max_str": "00:00:00.005",
            "median_nanos": 4955896,
            "median_sec": 0.004955896,
            "median_str": "00:00:00.004",
            "std_dev": 103679.53179147754,
            "std_dev_sec": 0.00010367953179147753,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "9",
          {
            "name": "benchmark-workload-2",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 5638959,
            "min_sec": 0.005638959,
            "min_str": "00:00:00.005",
            "max_nanos": 5643416,
            "max_sec": 0.005643416,
            "max_str": "00:00:00.005",
            "median_nanos": 5641187,
            "median_sec": 0.005641187,
            "median_str": "00:00:00.005",
            "std_dev": 3151.5749237484424,
            "std_dev_sec": 3.1515749237484423e-6,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "10",
          {
            "name": "benchmark-workload-2",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 5086250,
            "min_sec": 0.00508625,
            "min_str": "00:00:00.005",
            "max_nanos": 5587292,
            "max_sec": 0.005587292,
            "max_str": "00:00:00.005",
            "median_nanos": 5336771,
            "median_sec": 0.005336771,
            "median_str": "00:00:00.005",
            "std_dev": 354290.19585927017,
            "std_dev_sec": 0.00035429019585927015,
            "std_dev_str": "00:00:00.000"
          }
        ]
      ]
    },
    "benchmark-workload-1": {
      "name": "benchmark-workload-1",
      "config": "0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10",
      "runs": [
        [
          "1",
          {
            "name": "benchmark-workload-1",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 1259375,
            "min_sec": 0.001259375,
            "min_str": "00:00:00.001",
            "max_nanos": 1276125,
            "max_sec": 0.001276125,
            "max_str": "00:00:00.001",
            "median_nanos": 1267750,
            "median_sec": 0.00126775,
            "median_str": "00:00:00.001",
            "std_dev": 11844.038584874672,
            "std_dev_sec": 0.000011844038584874672,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "2",
          {
            "name": "benchmark-workload-1",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 2513584,
            "min_sec": 0.002513584,
            "min_str": "00:00:00.002",
            "max_nanos": 2520875,
            "max_sec": 0.002520875,
            "max_str": "00:00:00.002",
            "median_nanos": 2517229,
            "median_sec": 0.002517229,
            "median_str": "00:00:00.002",
            "std_dev": 5155.515541631118,
            "std_dev_sec": 5.155515541631118e-6,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "3",
          {
            "name": "benchmark-workload-1",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 3755750,
            "min_sec": 0.00375575,
            "min_str": "00:00:00.003",
            "max_nanos": 3784000,
            "max_sec": 0.003784,
            "max_str": "00:00:00.003",
            "median_nanos": 3769875,
            "median_sec": 0.003769875,
            "median_str": "00:00:00.003",
            "std_dev": 19975.766568519968,
            "std_dev_sec": 0.00001997576656851997,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "4",
          {
            "name": "benchmark-workload-1",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 5003833,
            "min_sec": 0.005003833,
            "min_str": "00:00:00.005",
            "max_nanos": 5030084,
            "max_sec": 0.005030084,
            "max_str": "00:00:00.005",
            "median_nanos": 5016958,
            "median_sec": 0.005016958,
            "median_str": "00:00:00.005",
            "std_dev": 18562.26011292806,
            "std_dev_sec": 0.00001856226011292806,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "5",
          {
            "name": "benchmark-workload-1",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 5518500,
            "min_sec": 0.0055185,
            "min_str": "00:00:00.005",
            "max_nanos": 6283084,
            "max_sec": 0.006283084,
            "max_str": "00:00:00.006",
            "median_nanos": 5900792,
            "median_sec": 0.005900792,
            "median_str": "00:00:00.005",
            "std_dev": 540642.5311867353,
            "std_dev_sec": 0.0005406425311867353,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "6",
          {
            "name": "benchmark-workload-1",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 6942292,
            "min_sec": 0.006942292,
            "min_str": "00:00:00.006",
            "max_nanos": 7541458,
            "max_sec": 0.007541458,
            "max_str": "00:00:00.007",
            "median_nanos": 7241875,
            "median_sec": 0.007241875,
            "median_str": "00:00:00.007",
            "std_dev": 423674.3416564189,
            "std_dev_sec": 0.00042367434165641895,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "7",
          {
            "name": "benchmark-workload-1",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 8784417,
            "min_sec": 0.008784417,
            "min_str": "00:00:00.008",
            "max_nanos": 8812875,
            "max_sec": 0.008812875,
            "max_str": "00:00:00.008",
            "median_nanos": 8798646,
            "median_sec": 0.008798646,
            "median_str": "00:00:00.008",
            "std_dev": 20122.84477900677,
            "std_dev_sec": 0.00002012284477900677,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "8",
          {
            "name": "benchmark-workload-1",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 9426875,
            "min_sec": 0.009426875,
            "min_str": "00:00:00.009",
            "max_nanos": 11596125,
            "max_sec": 0.011596125,
            "max_str": "00:00:00.011",
            "median_nanos": 10511500,
            "median_sec": 0.0105115,
            "median_str": "00:00:00.010",
            "std_dev": 1533891.3850889183,
            "std_dev_sec": 0.0015338913850889183,
            "std_dev_str": "00:00:00.001"
          }
        ],
        [
          "9",
          {
            "name": "benchmark-workload-1",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 9498333,
            "min_sec": 0.009498333,
            "min_str": "00:00:00.009",
            "max_nanos": 9953333,
            "max_sec": 0.009953333,
            "max_str": "00:00:00.009",
            "median_nanos": 9725833,
            "median_sec": 0.009725833,
            "median_str": "00:00:00.009",
            "std_dev": 321733.5854398791,
            "std_dev_sec": 0.0003217335854398791,
            "std_dev_str": "00:00:00.000"
          }
        ],
        [
          "10",
          {
            "name": "benchmark-workload-1",
            "ramp_up": 1,
            "repeat": 2,
            "min_nanos": 11674000,
            "min_sec": 0.011674,
            "min_str": "00:00:00.011",
            "max_nanos": 12627167,
            "max_sec": 0.012627167,
            "max_str": "00:00:00.012",
            "median_nanos": 12150583,
            "median_sec": 0.012150583,
            "median_str": "00:00:00.012",
            "std_dev": 673990.849303238,
            "std_dev_sec": 0.0006739908493032379,
            "std_dev_str": "00:00:00.000"
          }
        ]
      ]
    }
  }
}
Benchmark Workloads Series

Benchmark name: benchmark-workload-2

point,ramp_up,repeat,min_sec,max_sec,median_sec,std_dev_sec
1,1,2,0.00063575,0.000637376,0.000636563,0.0000011497556262093261
2,1,2,0.001265333,0.001269584,0.001267458,0.0000030059109268240135
3,1,2,0.001890958,0.001892333,0.001891645,0.0000009722718241315028
4,1,2,0.002512042,0.002604791,0.002558416,0.0000655834468482711
5,1,2,0.003129084,0.00314525,0.003137167,0.000011431088224661727
6,1,2,0.003092583,0.003767833,0.003430208,0.0004774738539962162
7,1,2,0.004055333,0.004084166,0.004069749,0.000020388009821951726
8,1,2,0.004171542,0.005756541,0.004964041,0.0011207635410738967
9,1,2,0.004549334,0.005149792,0.004849563,0.0004245879236177119
10,1,2,0.006351,0.007404083,0.006877541,0.000744642130452273

Benchmark name: benchmark-workload-1

point,ramp_up,repeat,min_sec,max_sec,median_sec,std_dev_sec
1,1,2,0.001265417,0.001266709,0.001266063,0.0000009135819612930194
2,1,2,0.002518501,0.002534999,0.00252675,0.000011665847676015662
3,1,2,0.003762958,0.004072459,0.003917708,0.00021885025588401765
4,1,2,0.004471833,0.004559375,0.004515604,0.00006190154183863275
5,1,2,0.005676917,0.005765751,0.005721334,0.00006281512379992577
6,1,2,0.006610875,0.007539833,0.007075354,0.0006568725012374928
7,1,2,0.007831792,0.008021959,0.007926875,0.0001344683752579022
8,1,2,0.009668583,0.009832959,0.009750771,0.00011623138426431993
9,1,2,0.009677333,0.011178501,0.010427917,0.0010614860725002473
10,1,2,0.011440417,0.012526,0.011983208,0.0007676231008408358
Find Regressions Example 
use std::collections::BTreeMap;
use std::fmt::{Display, Formatter};
use std::thread;
use std::time::Duration;

use benchmark_rs::benchmarks::Benchmarks;
use benchmark_rs::stopwatch::StopWatch;
use rand::Rng;

#[derive(Clone)]
struct Config {
    // simulate available resources - CPU cores, memory buffers, etc.
    pub resources: u32,
    pub workloads: BTreeMap<u64, Duration>,
}

impl Display for Config {
    fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
        for k in self.workloads.keys() {
            write!(f, "{k} ")?;
        }
        Ok(())
    }
}

fn example(_stop_watch: &mut StopWatch, config: Config, work: u64) -> Result<(), anyhow::Error> {
    let sleep_time = config.workloads.get(&work).unwrap().clone();
    thread::sleep(sleep_time / config.resources);
    Ok(())
}

fn modified_example(
    _stop_watch: &mut StopWatch,
    config: Config,
    work: u64,
) -> Result<(), anyhow::Error> {
    // we introduce a random deviation in modified example to simulate
    // regression introduced by a code change
    let deviation = Duration::from_millis(rand::thread_rng().gen_range(0..10));
    let sleep_time = config.workloads.get(&work).unwrap().clone();
    thread::sleep(sleep_time / config.resources + deviation);
    Ok(())
}

fn main() -> Result<(), anyhow::Error> {
    let mut previous_benchmarks = Benchmarks::new("benchmarks");
    let workloads: BTreeMap<u64, Duration> = (1..=10)
        .map(|i| (i, Duration::from_millis(i * 25)))
        .collect();
    previous_benchmarks.add(
        "benchmark-1",
        example,
        Config {
            resources: 1,
            workloads: workloads.clone(),
        },
        (1..=10).collect(),
        5,
        3,
    )?;
    previous_benchmarks.run()?;
    let previous_summary = previous_benchmarks.summary_as_json();

    let mut current_benchmarks = Benchmarks::new("benchmarks");
    current_benchmarks.add(
        "benchmark-1",
        modified_example,
        Config {
            resources: 1,
            workloads: workloads.clone(),
        },
        (1..=10).collect(),
        5,
        3,
    )?;
    current_benchmarks.add(
        "benchmark-2",
        modified_example,
        Config {
            resources: 2,
            workloads: workloads.clone(),
        },
        (1..=10).collect(),
        5,
        3,
    )?;
    current_benchmarks.run()?;

    // compare results of this run with the results of previous runs with threshold of 5 percent
    let analysis_result = current_benchmarks.analyze(Some(previous_summary), 5.0)?;
    println!("Analysis result:");
    println!("{}", analysis_result.to_string());
    assert!(!analysis_result.divergent_series().is_empty());
    Ok(())
}
Find Regressions Output

Analysis result. Current and previous values are in nanosecond units, the change is in percents.

  {
    "name": "benchmarks",
    "new_series": [
      "benchmark-2"
    ],
    "equal_series": {},
    "divergent_series": {
      "benchmark-1": {
        "8": {
          "Equal": {
            "point": "8",
            "previous": 200686042,
            "current": 206411375,
            "change": 2.852880520709064
          }
        },
        "7": {
          "Equal": {
            "point": "7",
            "previous": 176700166,
            "current": 183395333,
            "change": 3.788998704166474
          }
        },
        "10": {
          "Equal": {
            "point": "10",
            "previous": 251701083,
            "current": 256856000,
            "change": 2.0480313149864315
          }
        },
        "2": {
          "Equal": {
            "point": "2",
            "previous": 52684667,
            "current": 55052334,
            "change": 4.494034288951653
          }
        },
        "5": {
          "Equal": {
            "point": "5",
            "previous": 127510583,
            "current": 130817709,
            "change": 2.5936090340046434
          }
        },
        "6": {
          "Equal": {
            "point": "6",
            "previous": 152803084,
            "current": 158141750,
            "change": 3.4938208446106955
          }
        },
        "9": {
          "Equal": {
            "point": "9",
            "previous": 225706250,
            "current": 229522083,
            "change": 1.6906191122310474
          }
        },
        "1": {
          "Greater": {
            "point": "1",
            "previous": 26413209,
            "current": 35264166,
            "change": 33.50958605597674
          }
        },
        "4": {
          "Equal": {
            "point": "4",
            "previous": 104201208,
            "current": 109166958,
            "change": 4.7655397622645665
          }
        },
        "3": {
          "Equal": {
            "point": "3",
            "previous": 79801875,
            "current": 84060834,
            "change": 5.336915955922095
          }
        }
      }
    }
  }

Similar Projects

License: MIT OR Apache-2.0

Commit count: 19

cargo fmt