Continuous benchmarking on bare metal

A benchmark result (called a Metric) is a single point-in-time measurement. Each benchmark run produces one Metric, so 5 benchmarks × 10 runs = 50 Metrics. Even if your harness samples a benchmark 1,000 times internally for accuracy, that still counts as 1 Metric per run. A benchmark with multiple measures (like latency and throughput) creates one Metric per measure per run, so 5 benchmarks × 2 measures × 10 runs = 100 Metrics.

A Public Project is visible to anyone who can reach your Bencher instance. On Bencher Cloud, all Public Projects are listed here. A Private Project is only visible to members of your organization and requires a paid plan (Pro or Enterprise).

On Bencher Cloud, the Pro plan is $20/month and includes $20 of usage credit. Private Metrics ($0.01 each) and Bencher Bare Metal runner time ($1.00/hour) both draw from this single monthly credit, so most teams pay exactly $20. If you go past the included credit, additional usage is billed at the same rates, with no spend cliff. Credit is granted fresh each month and does not roll over. Bencher Self-Hosted Metrics are billed annually based on a licensed quantity.

Yes. Pro starts with a 1-month free trial: your first month's $20 base fee is waived and you get $20 of usage credit to spend across Metrics and Bare Metal. A credit card is required to start the trial. After 30 days it converts to the standard $20/month. There is no per-seat pricing on any plan.

No. Metrics from Public Projects are always free and unlimited. Only Private Project Metrics draw from your Pro usage credit. The Free plan supports Public Projects only, with daily rate limits.

• Unclaimed Project: 255 Metrics per day
• Free plan: 65,535 Metrics per day
• Bencher Plus plan: No rate limit

Community support (Free): GitHub issues and our Discord community.

Priority support (Pro): Email support with responses within 1 business day.

Dedicated support (Enterprise): Dedicated Slack channel with direct access to Bencher engineering, priority on bug fixes and feature requests, and dedicated onboarding.

Yes. You can upgrade, downgrade, or cancel your plan at any time. On Bencher Cloud, metered billing stops at the end of the current period. Bencher Self-Hosted licenses aren't refunded mid-term, but you can downgrade or cancel before your next renewal.

Your Projects and benchmark results (called Metrics) remain accessible after cancellation. Public Projects stay publicly visible unless you delete them, and Private Projects stay private. You can export your data via the Bencher API at any time.

The Bencher License is heavily inspired by the GitLab License. You can use Bencher Self-Hosted in development, testing, and production without a Bencher Plus license, equivalent to Bencher Cloud on the Free plan.

Bencher Plus features for Self-Hosted are available on the Enterprise (On-Prem) plan. Contact us to set up a license.

On Bencher Self-Hosted, new Metrics stop being accepted once you hit your licensed quantity limit. You can increase your licensed quantity at any time. We recommend setting your quantity with an extra margin to avoid interruption.

Bencher is the first Continuous Benchmarking tool that runs your benchmarks on the exact same bare metal hardware both locally and in CI. Developers and agents can compare work in progress against any commit in your project's history.

Locally, keep multitasking. No stopping what you're doing, pulling an old branch, and waiting on a comparison.

In CI, trust the numbers. No noisy neighbors, throttling, or host migrations.

No. You can run benchmarks right away without creating an account, with a 1 minute timeout per run. A Free account raises that timeout to 5 minutes and increases your daily rate limits. No credit card required either way. Check out the Bencher Bare Metal Quickstart to run your first benchmark in minutes.

Bencher Cloud On-Demand runners use the intel-v1 Spec: Linux on x86_64 with 4 CPUs, 48 GiB of memory, and 128 GiB of disk. Code runs in a Firecracker microVM sandbox for isolation. See the full Spec reference for details. Bencher Cloud Enterprise customers can request Dedicated and Custom runners. Contact us for more details.

On Bencher Self-Hosted, you can create server-scoped Bare Metal runners, and Enterprise (On-Prem) users can also create organization-scoped runners.

Bencher Cloud On-Demand runners support Linux x86_64 and (coming soon) ARM64, sandboxed in Firecracker. Dedicated and Custom runners can run without the Firecracker sandbox on Linux (x86_64 or ARM64), macOS, and Windows. Contact us for more details.

Bencher Self-Hosted runners can run on Linux (x86_64 or ARM64), macOS, and Windows.

See the bare metal runner reference for details.

On Bencher Cloud, Bare Metal On-Demand runner time costs $1.00/hr ($0.01666/min) and draws from the same monthly Pro usage credit as private Metrics. One credit pool covers both, so you reason about a single number. You're only billed for the minutes your benchmarks are actively running, and overage past the included credit is billed at the same rate.

Bencher Self-Hosted users do not pay for runner minutes.

The Pro plan is $20/month and includes $20 of usage credit shared across Metrics and runner time. There is no per-seat pricing and no separate runner minimum.

• On-Demand (Pro, Enterprise): billed for the minutes your benchmarks are actively running, drawn from your monthly credit
• Dedicated (Enterprise): flat monthly rate
• Custom (Enterprise): usage-based

Bencher Self-Hosted users do not pay for runner minutes. Contact us for more details.

Free users get one concurrent Bare Metal job with a 5 minute timeout and Standard queue priority. This is the same for both Bencher Cloud and Bencher Self-Hosted.

For Bencher Cloud On-Demand runners, you're running on the same Bare Metal machines as Pro and Enterprise customers, just with lower queue position during busy periods.

Bencher Bare Metal runners usually execute your already-compiled benchmark binaries. Compilation happens locally or in your CI pipeline and is typically the longest step, so most benchmark suites fit well under the 5 minute timeout.

Pro and Enterprise jobs are prioritized ahead of Free jobs in the runner queue and share equal priority with each other. All Bencher Cloud On-Demand jobs run on the same Bare Metal hardware. During idle periods Free jobs run immediately. During busy periods paid jobs are scheduled first, and Free jobs wait for the next available slot.