Branch Selection with bencher run

A Branch is the git ref (ie branch name or tag). When using the bencher run CLI subcommand, select a Branch with the --branch option or the BENCHER_BRANCH environment variable. The current git hash can also be specified with the --hash option. If a Branch is not specified then the current git branch name is used if it is available. Otherwise, main is used as the default Branch. Similarly, if the git hash is not specified then the current git hash is used if it is available.

When Continuous Benchmarking, that is benchmarking in CI, you will often want to create or rebase a branch on an already existing Branch. This is called a Branch Start Point. A Start Point is another Branch at a specific version (and git hash, if available). When using the bencher run CLI subcommand, select a Start Point with the --start-point option. Additionally, use the --start-point-hash option to specify a specific git hash for the Start Point. The amount of data that is copied over from the Start Point is controlled with the --start-point-max-versions option. Similarly, Thresholds can be copied over from the Start Point with the --start-point-clone-thresholds flag. To prevent drift, use the --start-point-reset flag to always reset the Branch to the Start Point.

--branch <BRANCH>


To select a Branch, set the --branch option or the BENCHER_BRANCH environment variable to the name, slug, or UUID for the Branch. If the value specified is a name or slug and the Branch does not already exist, it will be created for you. However, if the value specified is a UUID then the Branch must already exist. If both are specified, the --branch option takes precedence over the BENCHER_BRANCH environment variable.

If neither are specified, the Bencher CLI tries to find the current git branch. It starts by looking for a git repository in the current working directory. If unsuccessful, it continues to its parent directory and retries all the way up to the root directory. If a git repository is found, then the current short branch name is used. However, if a git repository is not found then main is used as the default Branch.

--hash <HASH>


A 40-character SHA-1 git commit hash. If two reports have the same branch and hash, they will be considered to be from the same commit. Therefore, they will have the same branch version number.

If not provided, the Bencher CLI tries to find the current git hash. It starts by looking for a git repository in the current working directory. If unsuccessful, it continues to its parent directory and retries all the way up to the root directory. If a git repository is found, then the current branch’s HEAD git hash is used.

--start-point <BRANCH>


Use the specified Branch name, slug, or UUID as the Start Point for the branch option. For GitHub this is usually the base branch for a Pull Request, and for GitLab this is usually the target branch for a Merge Request. The primary use case for --start-point is Statistical Continuous Benchmarking.

  • If the specified Start Point is an empty string, then this option is ignored.
  • If branch does not exist, it will be created using this Start Point.
  • If branch does exist and the Start Point is the same, then this option is ignored.
  • If branch does exist and the Start Point is different, a new branch Head will be created using this updated Start Point.
  • If branch does not exist and the Start Point does not exist, then the branch will be created without any historical data.
  • If branch does exist and the Start Point does not exist, then this option is ignored.

If a new branch is created with a Start Point, then the most recent data and optionally Thresholds from the specified Start Point are copied over. See the --start-point-max-versions option and the --start-point-clone-thresholds option for more details.

For example, if feature_branch does not yet exist, but main does exist and has historical data. Then using --branch feature_branch --start-point main would:

  1. Create a new Branch named feature_branch
  2. Copy the most recent data from main to feature_branch
  3. bencher run would proceed using feature_branch

🐰 Note: Whenever using environment variables they should be wrapped in double quotes (ie --start-point "$MY_BRANCH_ENV_VAR"), if you would like bencher run to not error when MY_BRANCH_ENV_VAR is undefined.

--start-point-hash <HASH>


Use the specified full Git hash as the Start Point for the branch option. This option requires the --start-point option to be set.

  • If branch does not exist, it will be created from start-point at the specified hash.
  • If branch does exist and both start-point and the specified hash are the same, then this option is ignored.
  • If branch does exist and either the start-point or the specified hash are different, a new branch Head will be created from start-point at the specified hash.
  • If branch does not exist and the start-point does not exist at the specified hash, then the branch will be created without any historical data.
  • If branch does exist and the start-point does not exist at the specified hash, then this option is ignored.

If a new branch is created with a Start Point, then all the most recent data and optionally Thresholds from the start-point at the specified hash are copied over. See the --start-point-max-versions option and the --start-point-clone-thresholds option for more details.

For example, if feature_branch exists with its Start Point being the main Branch at hash 32aea434d751648726097ed3ac760b57107edd8b then using --branch feature_branch --start-point main --start-point-hash 68b12758509ca5e2b1e3983525625e532eb5f562 would:

  1. Remove the old feature_branch Head
  2. Create a new feature_branch Head
  3. Copy of all the most recent data from main at hash 68b12758509ca5e2b1e3983525625e532eb5f562 to the new feature_branch Head
  4. bencher run would then proceed using the new feature_branch Head

--start-point-max-versions <COUNT>


Use the specified maximum number of historical branch versions to include from the Start Point for the branch option. Any versions beyond this number will be omitted. The default is 255 versions. This option requires the --start-point option to be set.

For example, if feature_branch exists with its Start Point being the main Branch and main has 300 versions, then using --branch feature_branch --start-point main --start-point-max-versions 100 would only include the most recent 100 versions of main in feature_branch.

--start-point-clone-thresholds


Clone all of the Thresholds from the Start Point for the branch option. This flag requires the --start-point option to be set.

  • If a Threshold does not exist, it will be created from the start-point Threshold.
  • If a Threshold does exist and it is the same as the start-point Threshold, then that Threshold is effectively ignored.
  • If a Threshold does exist and its model is different from the start-point Threshold, then the old model will be replaced with the start-point Threshold model.
  • If a Threshold does exists and the start-point Threshold does not exist, then the Threshold model is removed.

For example, if feature_branch exists with its Start Point being the main Branch and main has a Threshold for the built-in localhost Testbed and latency Measure, then using --branch feature_branch --start-point main --start-point-clone-thresholds would create a new Threshold for feature_branch with the localhost Testbed and latency Measure.

--start-point-reset


Reset the specified branch. When the --start-point option is set, the branch will be reset to the specified Start Point. This is useful to prevent drift in Statistical Continuous Benchmarking data. The primary use case for --start-point-reset without specifying a Start Point is Relative Continuous Benchmarking.

  • If branch does not exit and no Start Point is specified, the branch will be created without any historical data.
  • If branch does exist and no Start Point is specified, a new branch Head will be created without any historical data.
  • If branch does not exist and a Start Point is specified, the branch will be created using the Start Point.
  • If branch does exist and a Start Point is specified, a new branch Head will be created using the Start Point.

If a new branch is created with a Start Point, then all the most recent data and optionally Thresholds from the start-point at the specified hash are copied over. See the --start-point-max-versions option and the --start-point-clone-thresholds option for more details.

For example, if feature_branch exists without a Start Point then using --branch feature_branch --start-point-reset would:

  1. Remove the old feature_branch Head
  2. Create a new feature_branch Head with no historical data
  3. bencher run would then proceed using the new feature_branch Head

🐰 Congrats! You have learned all about Branch selection! 🎉


Keep Going: Thresholds & Alerts ➡


Published: Sat, August 12, 2023 at 4:07:00 PM UTC | Last Updated: Thu, February 20, 2025 at 10:36:00 PM UTC