Run REST API
Create a run
⚠️ Breaking Changes: The Bencher API is in beta and under active development. We may introduce breaking changes in future releases. We will announce any breaking changes in the release notes.
Create a run. The user does not need have an account yet or be authenticated. The project may or may not exist yet.
Headers
Content-Type string
Set to application/json.
Authorization string
Set to Bearer BENCHER_API_TOKEN, where BENCHER_API_TOKEN is a vaild Bencher API token.
Body Parameters
branchstring
Branch UUID, slug, or name. If the branch is not provided or does not exist, it will be created.
contextobject
Context for the report.
Properties of context
end_timestring
End time for the report. Must be an ISO 8601 formatted string.
hashstring
Full `git` commit hash. All reports with the same `git` commit hash will be considered part of the same branch version. This can be useful for tracking the performance of a specific commit across multiple testbeds.
idempotency_keystring
Optional idempotency key for deduplicating run submissions. If provided, a duplicate submission with the same key within the same project will return the existing report instead of creating a new one.
jobobject
Runner job configuration. When present, the run is executed on a remote bare metal runner instead of locally.
Properties of job
allow_failureboolean
Allow benchmark failure without short-circuiting iterations
backdatestring
Backdate the report start time
build_timeboolean
Track the build time of the benchmark command
cmdarray of strings
Command override (like Docker CMD)
entrypointarray of strings
Container entrypoint override (like Docker ENTRYPOINT)
envEnvironment variables passed to the container
file_pathsarray of strings
File paths to collect from the VM after job completion
file_sizeboolean
Track the file size of the output files instead of parsing their contents
imagestring
OCI image reference (e.g. "alpine:3.18", "ghcr.io/owner/repo:v1", "image@sha256:abc...")
iterinteger
Number of benchmark iterations for the runner to execute
specHardware spec slug or UUID to run on
timeoutinteger
Maximum execution time in seconds
projectProject UUID or slug. If the project is not provided or does not exist, it will be created.
resultsarray of strings
An array of benchmarks results in Bencher Metric Format (BMF).
settingsobject
Settings for how to handle the results.
Properties of settings
adapterstring
The benchmark harness adapter for parsing the benchmark results. If no adapter is specified, then the Magic adapter will be used.
One of: magic, json, rust, rust_bench, rust_criterion, rust_iai, rust_gungraun, cpp, cpp_google, cpp_catch2, go, go_bench, java, java_jmh, c_sharp, c_sharp_dot_net, js, js_benchmark, js_time, python, python_asv, python_pytest, ruby, ruby_benchmark, shell, shell_hyperfine, dart, dart_benchmark_harness
averagestring
Benchmark harness suggested central tendency (ie average). Some benchmarking harnesses provide multiple averages, such as mean and median.
One of: mean, median
foldstring
Fold multiple results into a single result using the selected operation. This can be useful for taking the min, max, mean, or median of the benchmark results.
One of: min, max, mean, median
spec_resetboolean
When set to `true`, clear the hardware spec from the testbed.
start_pointobject
The start point for the report branch. If the branch does not exist, the start point will be used to create a new branch. If the branch already exists and the start point is not provided, the current branch will be used. If the branch already exists and the start point provided is different, a new branch head will be created from the new start point. If a new branch or new branch head is created with a start point, historical branch versions from the start point branch will be shallow copied over to the new branch. That is, historical metrics data for the start point branch will appear in queries for the branch. For example, pull request branches often use their base branch as their start point branch. If a new branch is created, it is not kept in sync with the start point branch.
Properties of start_point
branchstring
The UUID, slug, or name of the branch to use as the start point.
clone_thresholdsboolean
If set to `true`, the thresholds from the start point branch will be deep copied to the branch. This can be useful for pull request branches that should have the same thresholds as their target branch. Requires the `branch` field to be set.
hashstring
The full git hash of the branch to use as the start point. Requires the `branch` field to be set.
max_versionsinteger
The maximum number of historical branch versions to include. Versions beyond this number will be omitted. The default is 255. Requires the `branch` field to be set.
resetboolean
Reset the branch head to an empty state. If the start point `branch` is specified, the new branch head will begin at that start point. Otherwise, the branch head will be reset to an empty state.
start_timestring
Start time for the report. Must be an ISO 8601 formatted string.
testbedstring
Testbed UUID, slug, or name. If the testbed is not provided or does not exist, it will be created.
thresholdsobject
Thresholds to use for the branch, testbed, and measures in the report. If a threshold does not exist, it will be created. If a threshold exists and the model is different, it will be updated with the new model. If a measure name or slug is provided, the measure will be created if it does not exist.
Properties of thresholds
modelsobject
Map of measure UUID, slug, or name to the threshold model to use. If a measure name or slug is provided, the measure will be created if it does not exist.
See the Thresholds documentation for a full overview.
resetboolean
Reset all thresholds for the branch and testbed. Any models present in the `models` field will still be updated accordingly. If a threshold already exists and is not present in the `models` field, its current model will be removed.
/v0/run | Bencher Cloud | https://api.bencher.dev |
| Bencher Self-Hosted | http://localhost:61016 |
Bencher CLI
bencher runView OpenAPI Spec