Bencher Self-Hosted with Docker Quick Start


What is Bencher?

Bencher is a suite of continuous benchmarking tools. Have you ever had a performance regression impact your users? Bencher could have prevented that from happening. Bencher allows you to detect and prevent performance regressions before they make it to production.

  • Run: Run your benchmarks locally or in CI using your favorite benchmarking tools. The bencher CLI simply wraps your existing benchmark harness and stores its results.
  • Track: Track the results of your benchmarks over time. Monitor, query, and graph the results using the Bencher web console based on the source branch, testbed, benchmark, and measure.
  • Catch: Catch performance regressions in CI. Bencher uses state of the art, customizable analytics to detect performance regressions before they make it to production.

For the same reasons that unit tests are run in CI to prevent feature regressions, benchmarks should be run in CI with Bencher to prevent performance regressions. Performance bugs are bugs!


Bencher Self-Hosted

Bencher is open source and self-hostable. If you are interested in using Bencher Cloud, check out the Bencher Cloud Quick Start tutorial. This tutorial will get you setup using Bencher Self-Hosted with Docker.

🐰 Once you feel comfortable using Bencher Self-Hosted, consider checking out the following resources:


Install Docker

In order to run the Console and API servers in this tutorial you will need to have docker installed. Check to see if you have docker installed. Run:

Terminal window
docker --version

You should see something like:

Docker version 20.10.17, build 100c701

It is okay if your version number is different. It’s just important that this command works. If not follow the instructions for installing docker.


Install bencher CLI

Select your operating system and run the provided command to install the bencher CLI. For more details, see the bencher CLI install documentation.

Terminal window

Now, lets check that you have the bencher CLI installed. Run:

Terminal window
bencher --version

You should see:

bencher 0.5.0

Run bencher up

With docker and the bencher CLI installed, you can now run the Console and API servers with the bencher up CLI subcommand. Run:

Terminal window
bencher up

You should see something like:

Pulling `ghcr.io/bencherdev/bencher-api:latest` image...
Creating `bencher_api` container...
Starting `bencher_api` container...
Pulling `ghcr.io/bencherdev/bencher-console:latest` image...
Creating `bencher_console` container...
Starting `bencher_console` container...
🐰 Bencher Self-Hosted is up and running!
Console Server: http://localhost:3000
API Server: http://localhost:61016
Press Ctrl+C to stop Bencher Self-Hosted.
🐰 Bencher Self-Hosted logs...
Jan 08 16:49:07.727 INFO 🐰 Bencher API Server v0.5.0
...

Again, it is okay if your output is different. It’s just important that this command works.

🐰 If you get an error from ghcr.io saying: "authentication required" Try running: docker logout ghcr.io


Select your Benchmark Harness

If you already have benchmarks written, select your programming language and benchmarking harness from the list below. Otherwise, just skip this step. For more details, see the benchmark harness adapters documentation.

C#
C++
Go
Java
JavaScript
Python
Ruby
Rust
Shell
JSON

Track your Benchmarks

You are now ready to track your benchmark results! To do so, you will use the bencher run CLI subcommand to run your benchmarks and collect the results. Run:

Terminal window

You may need to modify the benchmark command to match your setup. If you don’t have any benchmarks yet, you can just use the bencher mock subcommand as your benchmark command to generate some mock data. If everything works as expected, the end of the output should look something like this:

View results:
- bencher::mock_0 (Latency): https://localhost:3000/perf/project-abc4567-wxyz123456789?branches=88d5192d-5cd1-47c6-a817-056e5968737c&heads=657a8ee9-1f30-49d4-bd9b-ceed02576d7e&testbeds=f3a5db46-a57e-4caf-b96e-f0c1111eaa67&benchmarks=f7022024-ae16-4782-8f0d-869d65a82930&measures=775999d3-d705-482f-acd8-41947f8e0fbc&start_time=1741390156000&end_time=1743982156000&report=709d3476-51a4-4939-9584-75d9a2c04c54
- bencher::mock_1 (Latency): https://localhost:3000/perf/project-abc4567-wxyz123456789?branches=88d5192d-5cd1-47c6-a817-056e5968737c&heads=657a8ee9-1f30-49d4-bd9b-ceed02576d7e&testbeds=f3a5db46-a57e-4caf-b96e-f0c1111eaa67&benchmarks=7a823440-216f-482d-a05f-8bf75e865bba&measures=775999d3-d705-482f-acd8-41947f8e0fbc&start_time=1741390156000&end_time=1743982156000&report=709d3476-51a4-4939-9584-75d9a2c04c54
- bencher::mock_2 (Latency): https://localhost:3000/perf/project-abc4567-wxyz123456789?branches=88d5192d-5cd1-47c6-a817-056e5968737c&heads=657a8ee9-1f30-49d4-bd9b-ceed02576d7e&testbeds=f3a5db46-a57e-4caf-b96e-f0c1111eaa67&benchmarks=8d9695ff-f352-4781-9561-3c69012fd9fe&measures=775999d3-d705-482f-acd8-41947f8e0fbc&start_time=1741390156000&end_time=1743982156000&report=709d3476-51a4-4939-9584-75d9a2c04c54
- bencher::mock_3 (Latency): https://localhost:3000/perf/project-abc4567-wxyz123456789?branches=88d5192d-5cd1-47c6-a817-056e5968737c&heads=657a8ee9-1f30-49d4-bd9b-ceed02576d7e&testbeds=f3a5db46-a57e-4caf-b96e-f0c1111eaa67&benchmarks=8ef6e256-8084-4afe-a7cf-eaa46384c19d&measures=775999d3-d705-482f-acd8-41947f8e0fbc&start_time=1741390156000&end_time=1743982156000&report=709d3476-51a4-4939-9584-75d9a2c04c54
- bencher::mock_4 (Latency): https://localhost:3000/perf/project-abc4567-wxyz123456789?branches=88d5192d-5cd1-47c6-a817-056e5968737c&heads=657a8ee9-1f30-49d4-bd9b-ceed02576d7e&testbeds=f3a5db46-a57e-4caf-b96e-f0c1111eaa67&benchmarks=1205e35a-c73b-4ff9-916c-40838a62ae0b&measures=775999d3-d705-482f-acd8-41947f8e0fbc&start_time=1741390156000&end_time=1743982156000&report=709d3476-51a4-4939-9584-75d9a2c04c54
Claim this project: https://localhost:3000/auth/signup?claim=d4b0cd5a-8422-40af-9872-8e18d5d062c4

You can now view the results for each of your benchmarks in the browser. Click or copy and paste the links from View results. To claim these results, click or copy and paste the Claim this project link into your browser.



🐰 Congrats! You tracked your first benchmark results with Bencher Self-Hosted! 🎉


Keep Going: How to Claim Benchmark Results ➡



Published: Sat, August 12, 2023 at 4:07:00 PM UTC | Last Updated: Sun, April 6, 2025 at 6:30:00 PM UTC