Track your Criterion benchmarks

🦀 Track the results of `cargo bench` over time with Bencher

How It Works

Run your benchmarks

Run your benchmarks locally or in CI using your favorite benchmarking tools. The bencher CLI simply wraps your existing benchmark harness and stores its results.

Track your benchmarks

Track the results of your benchmarks over time. Monitor, query, and graph the results using the Bencher web console based on the source branch and testbed.

Catch performance regressions

Catch performance regressions in CI. Bencher uses state of the art, customizable analytics to detect performance regressions before they make it to production.

You are in good company

Track your Criterion benchmarks

Track the results of your Criterion benchmarks over time. Monitor, query, and graph the results using the Bencher web console based on the source branch and testbed.

$ bencher run --adapter rust_criterion "cargo bench"

Adapter::Magic (JSON)
time:   [3.4623 µs 3.4632 µs 3.4641 µs]
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe

Adapter::Json
time:   [3.4793 µs 3.4796 µs 3.4800 µs]
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe

Adapter::Magic (Rust)
time:   [14.721 µs 14.726 µs 14.730 µs]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

Adapter::Rust
time:   [14.881 µs 14.884 µs 14.887 µs]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

Bencher New Report:
{
  "branch": "main",
  "end_time": "2024-03-10T20:34:00.981095Z",
  "hash": "a7d081e048b77c5d4bfc0fb66f48cbf05cb49464",
  "results": [ ... ],
  "settings": {},
  "start_time": "2024-03-10T20:33:57.722930Z",
  "testbed": "localhost"
}

Get Performance Insights Directly in Your Workflow

Software performance pull request comments allow you to quickly analyze your PR's performance and risk without leaving your workflow. Fail the build! Keep your codebase fast and your users happy.

Bencher Report

Branch	254/merge
Testbed	ubuntu-latest

🚨 1 ALERT: Threshold Boundary Limit exceeded!

Benchmark	Measure Units	View	Benchmark Result (Result Δ%)	Upper Boundary (Limit %)
Adapter::Json	Latency nanoseconds (ns)	📈 plot 🚨 alert 🚷 threshold	3,445.60 (+1.52%)	3,362.07 (102.48%)

Click to view all benchmark results

Benchmark	Latency	Benchmark Results nanoseconds (ns) (Result Δ%)	Upper Boundary nanoseconds (ns) (Limit %)
Adapter::Json	📈 view plot 🚨 view alert 🚷 view threshold	3,445.60 (+1.52%)	3,362.07 (102.48%)
Adapter::Magic (JSON)	📈 view plot 🚷 view threshold	3,431.40 (+0.69%)	3,596.95 (95.40%)
Adapter::Magic (Rust)	📈 view plot 🚷 view threshold	22,095.00 (-0.83%)	24,732.80 (89.33%)
Adapter::Rust	📈 view plot 🚷 view threshold	2,305.70 (-2.76%)	2,500.49 (92.21%)
Adapter::RustBench	📈 view plot 🚷 view threshold	2,299.90 (-3.11%)	2,503.41 (91.87%)