## Criterion benchmarks for Tonic

### Running the benchmarks 
From the root Tonic directory, `cargo bench`

After running, the reports can be found in `tonic/target/criterion/report/index.html`

[Gnuplot](http://www.gnuplot.info/) is required for graph generation.  If gnuplot is not installed, Criterion will display: `Gnuplot not found, disabling plotting` at the console.

### Notes 
1) Currently, these benchmarks only test the performance of constructing Tonic Requests and Responses, not over-the-wire throughput. 
2) The `thrpt` value generated by Criterion is simply a measure of bytes consumed by the target function.
3) As we are not testing tonic-build compile time, the tests reference pre-compiled .rs files in 'benchmarks/compiled_protos'.
4) The original proto files are in the `proto` directory for reference.  
5) This used the Criterion 3.0 `Criterion Group` functionality. Details here: https://docs.rs/criterion/0.3.0/criterion/  

### Interpreting Results

Criterion is particularly useful for establishing a first-run baseline and then comparing after code-changes - e.g. `Performance has regressed` below. 

```bash
Request_Response/request/100000                                                                             
                        time:   [2.7231 us 2.7588 us 2.7969 us]
                        thrpt:  [33.298 GiB/s 33.758 GiB/s 34.200 GiB/s]
                 change:
                        time:   [+16.073% +17.871% +19.980%] (p = 0.00 < 0.05)
                        thrpt:  [-16.653% -15.162% -13.847%]
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
```