How to use Bencher in GitLab CI/CD

benchmark_with_bencher:
    image: rust:bullseye
    variables:
      BENCHER_PROJECT: save-walter-white
      BENCHER_API_TOKEN: ${{ secrets.BENCHER_API_TOKEN }}
      BENCHER_ADAPTER: json
      BENCHER_TESTBED: bullseye
      BENCHER_VERSION: 0.3.18
    before_script:
      - wget https://github.com/bencherdev/bencher/releases/download/v${BENCHER_VERSION}/bencher_${BENCHER_VERSION}_amd64.deb
      - dpkg -i bencher_${BENCHER_VERSION}_amd64.deb
    script:
      - |
        bencher run \
        --if-branch "$CI_COMMIT_REF_NAME" \
        --else-if-branch "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME" \
        --else-if-branch main \
        --err \
        "bencher mock"
  1. Create a GitLab CI/CD file (ex: .gitlab-ci.yml)
  2. Create a GitLab CI/CD job (ex: benchmark_with_bencher)
  3. Select a Docker image to run your job in (ex: image: rust:bullseye)
  4. The Project must already exist. Set the --project flag or the BENCHER_PROJECT environment variable to the Project slug or UUID. (ex: BENCHER_PROJECT: save-walter-white)
  5. Add BENCHER_API_TOKEN to your project’s CI/CD secrets, under Settings -> CI / CD -> Variables -> Add Variable
  6. The API token must already exist. Set the --token flag or the BENCHER_API_TOKEN environment variable to the API token. (ex: BENCHER_API_TOKEN: ${{ secrets.BENCHER_API_TOKEN }})
  7. Optional: Set the --adapter flag or the BENCHER_ADAPTER environment variable to the desired adapter name. If this is not set, then the magic Adapter will be used. See benchmark harness adapters for a full overview. (ex: BENCHER_ADAPTER: json)
  8. Optional: Set the --testbed flag or the BENCHER_TESTBED environment variable to the Testbed slug or UUID. The Testbed must already exist. If this is not set, then the localhost Testbed will be used. (ex: BENCHER_TESTBED: bullseye)
  9. Optional: Create an environment variable to make it easy to keep track the CLI version being used. (ex: BENCHER_VERSION: 0.3.18)
  10. Download and install the Bencher CLI from a .deb.
  11. Track your benchmarks with the bencher run CLI subcommand:
    1. There are several options for setting the project branch. See branch selection for a full overview. The provided command uses GitLab CI/CD predefined variables and it tries to:
      1. Use the current branch if it already exists. (ex: --if-branch "$CI_COMMIT_REF_NAME")
      2. Create a clone of MR target branch data and thresholds if it already exists. (ex: --else-if-branch "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME")
      3. Otherwise, create a clone of the main branch data and thresholds. (ex: --else-if-branch main)
    2. Set the command to fail if an Alert is generated. In order for an Alert to be generated, a Threshold must already exist. (ex: --err)
    3. Run your benchmarks and generate a Report from the results. (ex: "bencher mock")


🐰 Congrats! You have learned how to use Bencher in GitLab CI/CD! 🎉


Keep Going: Benchmarking Overview ➡