Continuous benchmarking on bare metal

A benchmark series is a unique combination of benchmark, measure, and testbed. It is the unit the Pro plan is billed on. A series is active in a month if it reports at least one result that month, no matter how many results or runs. So 5 benchmarks × 2 measures × 1 testbed is 10 series whether you run them once or a thousand times. Active series are counted across both public and private projects.

A benchmark is a named performance test. A measure is the unit a benchmark reports, such as latency or throughput. A benchmark can report one or more measures. A testbed is the environment the benchmark runs on, like a specific machine or class of CI runner.

A Public Project is visible to anyone who can reach your Bencher instance. On Bencher Cloud, all Public Projects are listed here. A Private Project is only visible to members of your organization and requires a paid plan (Pro or Enterprise).

On Bencher Cloud, the Pro plan is a flat monthly fee based on your active benchmark series:

Active series	Price
0 to 250	$100 / month
251 to 375	$150 / month
376 to 500	$200 / month
501 or more	Get in touch

Bencher Bare Metal runner time ($1.00/hour) is billed separately on top, except during your first month free trial. Bencher Self-Hosted is billed annually based on a licensed quantity.

Yes. Pro starts with a 1-month free trial: your first month's base fee is waived and Bencher Bare Metal runner minutes are included, so your first month is free. A credit card is required to start the trial. After 30 days it converts to the standard monthly fee for your active series. There is no per-seat pricing on any plan.

The Free plan supports Public Projects only, free of charge, with daily rate limits. On the Pro plan, your active benchmark series are counted across all your projects, both public and private.

• Unclaimed Project: 255 Metrics per day
• Free plan: 65,535 Metrics per day
• Bencher Plus plan: No rate limit

Community support (Free): GitHub issues and our Discord community.

Priority support (Pro): Email support with responses within 1 business day.

Dedicated support (Enterprise): Dedicated Slack channel with direct access to Bencher engineering, priority on bug fixes and feature requests, and dedicated onboarding.

Yes. You can upgrade, downgrade, or cancel your plan at any time. On Bencher Cloud, metered billing stops at the end of the current period. Bencher Self-Hosted licenses aren't refunded mid-term, but you can downgrade or cancel before your next renewal.

Your Projects and benchmark results (called Metrics) remain accessible after cancellation. Public Projects stay publicly visible unless you delete them, and Private Projects stay private. You can export your data via the Bencher API at any time.

The Bencher License is heavily inspired by the GitLab License. You can use Bencher Self-Hosted in development, testing, and production without a Bencher Plus license, equivalent to Bencher Cloud on the Free plan.

Bencher Plus features for Self-Hosted are available on the Enterprise (On-Prem) plan. Contact us to set up a license.

On Bencher Self-Hosted, new Metrics stop being accepted once you hit your licensed quantity limit. You can increase your licensed quantity at any time. We recommend setting your quantity with an extra margin to avoid interruption.

Bencher is the first Continuous Benchmarking tool that runs your benchmarks on the exact same bare metal hardware both locally and in CI. Developers and agents can compare work in progress against any commit in your project's history.

Locally, keep multitasking. No stopping what you're doing, pulling an old branch, and waiting on a comparison.

In CI, trust the numbers. No noisy neighbors, throttling, or host migrations.

No. You can run benchmarks right away without creating an account, with a 1 minute timeout per run. A Free account raises that timeout to 5 minutes and increases your daily rate limits. No credit card required either way. Check out the Bencher Bare Metal Quickstart to run your first benchmark in minutes.

Bencher Cloud On-Demand runners use the intel-v1 Spec: Linux on x86_64 with 4 CPUs, 48 GiB of memory, and 128 GiB of disk. Code runs in a Firecracker microVM sandbox for isolation. See the full Spec reference for details. Bencher Cloud Enterprise customers can request Dedicated and Custom runners. Contact us for more details.

On Bencher Self-Hosted, you can create server-scoped Bare Metal runners, and Enterprise (On-Prem) users can also create organization-scoped runners.

Bencher Cloud On-Demand runners support Linux x86_64 and (coming soon) ARM64, sandboxed in Firecracker. Dedicated and Custom runners can run without the Firecracker sandbox on Linux (x86_64 or ARM64), macOS, and Windows. Contact us for more details.

Bencher Self-Hosted runners can run on Linux (x86_64 or ARM64), macOS, and Windows.

See the bare metal runner reference for details.

On Bencher Cloud, Bare Metal On-Demand runner time costs $1.00/hr ($0.01666/min), billed separately on top of your Pro series fee. You're only billed for the minutes your benchmarks are actively running. During your first month free trial, runner minutes are included.

Bencher Self-Hosted users do not pay for runner minutes.

The Pro plan starts at $100/month for your active benchmark series. There is no per-seat pricing and no separate runner minimum.

• On-Demand (Pro, Enterprise): billed for the minutes your benchmarks are actively running
• Dedicated (Enterprise): flat monthly rate
• Custom (Enterprise): usage-based

Bencher Self-Hosted users do not pay for runner minutes. Contact us for more details.

Free users get one concurrent Bare Metal job with a 5 minute timeout and Standard queue priority. This is the same for both Bencher Cloud and Bencher Self-Hosted.

For Bencher Cloud On-Demand runners, you're running on the same Bare Metal machines as Pro and Enterprise customers, just with lower queue position during busy periods.

Bencher Bare Metal runners usually execute your already-compiled benchmark binaries. Compilation happens locally or in your CI pipeline and is typically the longest step, so most benchmark suites fit well under the 5 minute timeout.

Pro and Enterprise jobs are prioritized ahead of Free jobs in the runner queue and share equal priority with each other. All Bencher Cloud On-Demand jobs run on the same Bare Metal hardware. During idle periods Free jobs run immediately. During busy periods paid jobs are scheduled first, and Free jobs wait for the next available slot.