Pricing

What is a benchmark result?

A benchmark result is referred to as a Metric.

What is a Metric?

A Metric is a single, point-in-time benchmark result. Each time a benchmark is run, it creates at least one new Metric.

For example, if you have 5 benchmarks that measure latency then they would create 5 Metrics each time they run. If you ran your benchmarks 10 times, you would then have 50 Metrics:

5 benchmarks x 10 runs = 50 Metrics

Even if your benchmark harness runs your benchmarks 1,000 times to get a more accurate result, you would still only have 50 Metrics.

Some benchmarks may create more than one new Metric per run.

For example, if you have 5 benchmarks that each have 2 measures, latency and throughput, then they would create 10 Metrics each time they run. If you ran your benchmarks 10 times, you would then have 100 Metrics:

5 benchmarks x 2 measures x 10 runs = 100 Metrics

How are Metrics billed?

On Bencher Cloud, Metrics are billed either:

• Monthly based on metered usage
• Annually based on a licensed quantity

For example, if you create 5,280 Metrics in a particular calendar month with metered usage then you would be billed for 5,280 Metrics that month.

Bencher Self-Hosted Metrics are always billed annually based on a licensed quantity.

For example, if you create 5,280 Metrics within a given year, then you would need to have a licensed quantity of say 6,000 Metrics.

What is a Public Project?

A Public Project is visible to anyone who has access to your Bencher instance.

On Bencher Cloud, all Public Projects are available here . If you are using Bencher Self-Hosted, then anyone with access to your network will likewise be able to see all Public Projects.

What is a Private Project?

A Private Project is only visible to members of your organization.

In order to create and use Private Projects, your organization must have an active Bencher Plus plan.

Are Metrics for Public Projects counted?

Yes, if you have a valid Bencher Plus plan then Metrics from both your Public Projects and Private Projects are counted.

What happens if I reach my licensed quantity limit?

Once you reach your licensed quantity limit, no new Metrics will be accepted.

No need to panic, you can always increase your licensed quantity. It is best to give yourself an extra margin when setting your licensed quantity though.

How does the Bencher License work for Bencher Self-Hosted?

The Bencher License is heavily inspired by the GitLab License. You can use Bencher Self-Hosted in development, testing, and production without a Bencher Plus license. This is equivalent to using Bencher Cloud on the Free plan. If you would like access to any of the Bencher Plus features (Team or Enterprise), then you will need to purchase a Bencher Plus license.