How to Claim Benchmark Results with Bencher
Most benchmark results are ephemeral. They disappear as soon as your terminal reaches its scrollback limit. Some benchmark harnesses let you cache results, but most only do so locally. Bencher allows you to track your benchmarks from both local and CI runs and compare the results, while still using your favorite benchmark harness. Bencher has a few features that make it easy to track your local benchmark results that need to be taken into consideration when transitioning to track benchmarks in CI.
unclaimed Projects
Bencher allows you to track benchmarks results without creating an account.
When you invoke the bencher run CLI subcommand
without setting the --token option,
Bencher will create a new unclaimed on-the-fly Project
for your benchmark results.
If you followed the Quick Start or Self-Hosted Quick Start tutorials,
you were taking advantage of this functionality.
Anyone can view and post benchmark results to an unclaimed Project.
In order to claim an unclaimed Project, you will need to create an account.
If you follow the Claim this project signup link at the end of your
bencher run output,
you will automatically claim the Project when your account is created.
If you already have a Bencher account,
then you can navigate to the Project’s public Perf Page
and click the Claim this project button at the top of the page.
Similarly, if you already have an API token,
you can use the API token with the the --token option
the next time you invoke bencher run
to claim the Project.
🐰 IMPORTANT: Once a Project is
claimed, it cannot be posted to anonymously. You must use an API token with the the--tokenoption.
Create an API Token
In order to use the
bencher run CLI subcommand
with the the --token option,
you will need to create an API token.
Click to add an API Token
Steps:
- Navigate to the Bencher Console.
- Hover over your name in top right corner.
- A dropdown menu should appear. Select
Tokens. - Once on the API Tokens page, click the
➕ Addbutton.
On-the-Fly Projects
Bencher allows you to create both claimed and unclaimed Projects on-the-fly.
When you invoke the bencher run CLI subcommand
without setting the --project option or if the slug specified does not match an existing Project,
Bencher will create a new on-the-fly Project
for your benchmark results.
If you followed the Quick Start or Self-Hosted Quick Start tutorials,
you were taking advantage of this functionality.
If the --project option is not specified, then an on-the-fly Project slug will be generated for you based on:
- The name of the parent directory for the
gitrepository, if it is available. - The 7 digit hexadecimal short hash for the initial commit of the
gitrepository, if it is available. - A 13 digit alphanumeric fingerprint of the local machine, for supported operating systems.
For example, a generated on-the-fly Project slug might look like: project-abc4567-wxyz123456789
🐰 IMPORTANT: If you are relying on an on-the-fly Project slug and you want to track benchmark results across different machines, you must use the
--projectoption to specify the Project slug. Otherwise, a new on-the-fly Project may be created for each different machine.
🐰 Congrats! You have learned how to claim your benchmark results! 🎉