How to use Bencher in GitLab CI/CD
benchmark_with_bencher:
image: rust:bullseye
variables:
BENCHER_PROJECT: save-walter-white
BENCHER_API_TOKEN: ${{ secrets.BENCHER_API_TOKEN }}
BENCHER_ADAPTER: json
BENCHER_TESTBED: bullseye
BENCHER_VERSION: 0.3.18
before_script:
- wget https://github.com/bencherdev/bencher/releases/download/v${BENCHER_VERSION}/bencher_${BENCHER_VERSION}_amd64.deb
- dpkg -i bencher_${BENCHER_VERSION}_amd64.deb
script:
- |
bencher run \
--if-branch "$CI_COMMIT_REF_NAME" \
--else-if-branch "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME" \
--else-if-branch main \
--err \
"bencher mock"
- Create a GitLab CI/CD file (ex:
.gitlab-ci.yml
) - Create a GitLab CI/CD
job
(ex:benchmark_with_bencher
) - Select a Docker image to run your job in (ex:
image: rust:bullseye
) - The Project must already exist. Set the
--project
flag or theBENCHER_PROJECT
environment variable to the Project slug or UUID. (ex:BENCHER_PROJECT: save-walter-white
) - Add
BENCHER_API_TOKEN
to your projectβs CI/CD secrets, underSettings -> CI / CD -> Variables -> Add Variable
- The API token must already exist. Set the
--token
flag or theBENCHER_API_TOKEN
environment variable to the API token. (ex:BENCHER_API_TOKEN: ${{ secrets.BENCHER_API_TOKEN }}
) - Optional: Set the
--adapter
flag or theBENCHER_ADAPTER
environment variable to the desired adapter name. If this is not set, then themagic
Adapter will be used. See benchmark harness adapters for a full overview. (ex:BENCHER_ADAPTER: json
) - Optional: Set the
--testbed
flag or theBENCHER_TESTBED
environment variable to the Testbed slug or UUID. The Testbed must already exist. If this is not set, then thelocalhost
Testbed will be used. (ex:BENCHER_TESTBED: bullseye
) - Optional: Create an environment variable to make it easy to keep track the CLI version being used. (ex:
BENCHER_VERSION: 0.3.18
) - Download and install the Bencher CLI from a
.deb
. - Track your benchmarks with the
bencher run
CLI subcommand:- There are several options for setting the project branch. See branch selection for a full overview. The provided command uses GitLab CI/CD predefined variables and it tries to:
- Use the current branch if it already exists. (ex:
--if-branch "$CI_COMMIT_REF_NAME"
) - Create a clone of MR target branch data and thresholds if it already exists. (ex:
--else-if-branch "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME"
) - Otherwise, create a clone of the
main
branch data and thresholds. (ex:--else-if-branch main
)
- Use the current branch if it already exists. (ex:
- Set the command to fail if an Alert is generated. In order for an Alert to be generated, a Threshold must already exist. (ex:
--err
) - Run your benchmarks and generate a Report from the results. (ex:
"bencher mock"
)
- There are several options for setting the project branch. See branch selection for a full overview. The provided command uses GitLab CI/CD predefined variables and it tries to:
π° Congrats! You have learned how to use Bencher in GitLab CI/CD! π