Bencher runner CLI

The runner binary is the agent that executes Bare Metal benchmark Jobs. It is the in-depth reference for setting up a Self-Hosted Runner. The binary provides two subcommands:

  • runner up: Start the runner, polling for and executing benchmark Jobs.
  • runner run: Pull an Image and execute it once on the local host, for testing.

🐰 The runner binary is distinct from the bencher runner subcommands. The runner binary operates a Runner host, while bencher runner manages Runner resources through the REST API.

runner up

Start the runner, polling for and executing benchmark Jobs. This is the long-running command used to operate a Self-Hosted Runner. The runner opens a single WebSocket connection to the API server, claims Jobs that match its Specs, executes them, and reports the results back.

runner up [OPTIONS]

Options

--host <HOST>

The Bencher API server to connect to. By default, https://api.bencher.dev is used. Can also be set with the BENCHER_HOST environment variable.

--runner <RUNNER>

The UUID or slug of the Runner to operate as. Can also be set with the BENCHER_RUNNER environment variable.

--key <KEY>

The Runner authentication key (bencher_runner_...) returned when the Runner was created. Can also be set with the BENCHER_RUNNER_KEY environment variable.

--poll-timeout <POLL_TIMEOUT>

The long-poll timeout in seconds while waiting for a Job, between 1 and 900. By default, 55 is used.

--danger-allow-no-sandbox

Allow executing Jobs without a Sandbox. Without this flag, a Job whose Spec has no Sandbox is rejected at runtime. Non-sandboxed Jobs run directly on the host, so only enable this for trusted workloads. Can also be set with the BENCHER_DANGER_ALLOW_NO_SANDBOX environment variable.

--sandbox-log-level <SANDBOX_LOG_LEVEL>

The log level for the sandbox process. By default, warning is used.

--no-auto-update

Disable automatic updates from the server. By default, the runner updates itself between Jobs when the server offers a new version. Can also be set with the BENCHER_NO_AUTO_UPDATE environment variable.

--max-download-size <MAX_DOWNLOAD_SIZE>

The maximum download size in bytes for self-update binaries. By default, 500 MiB is used. Conflicts with --no-auto-update.

--max-output-size <MAX_OUTPUT_SIZE>

The maximum size in bytes for collected stdout and stderr. By default, 25 MiB is used.

--max-file-count <MAX_FILE_COUNT>

The maximum number of output files to decode. By default, 255 is used.

--max-symlinks <MAX_SYMLINKS>

The maximum number of symlinks to follow during path resolution, matching the Linux kernel MAXSYMLINKS limit. By default, 40 is used. Only used in non-sandboxed mode.

--grace-period <GRACE_PERIOD>

The grace period in seconds after the benchmark exits before final output collection.

Host Tuning

Before each benchmark, the runner applies host tuning to reduce measurement noise. By default it disables ASLR, the NMI watchdog, SMT / hyper-threading, and turbo boost; sets the CPU scaling governor to performance, swappiness to 10, and perf_event_paranoid to -1. The flags below keep individual optimizations at their host defaults instead.

--no-tuning

Disable all host tuning optimizations.

--aslr

Keep ASLR enabled (default: disabled for benchmarks).

--nmi-watchdog

Keep the NMI watchdog enabled (default: disabled for benchmarks).

--smt

Keep SMT / hyper-threading enabled (default: disabled for benchmarks).

--turbo

Keep turbo boost enabled (default: disabled for benchmarks).

--swappiness <SWAPPINESS>

Set the swappiness value. By default, 10 is used.

--governor <GOVERNOR>

Set the CPU scaling governor. By default, performance is used.

--perf-event-paranoid <PERF_EVENT_PARANOID>

Set the perf_event_paranoid value. By default, -1 is used.

--help

Print help.

runner run

Pull an Image, create a rootfs, and execute it once on the local host. Unlike runner up, this does not connect to an API server or claim a Job. It runs a single Image directly, which is useful for testing and debugging a Runner host or an Image before submitting real Jobs.

runner run --image <IMAGE> [OPTIONS]

Options

--image <IMAGE>

The OCI Image to run, as a local path or a registry reference.

--token <TOKEN>

A JWT token for registry authentication when pulling the Image.

--sandbox <SANDBOX>

The sandbox mode for benchmark execution. Use firecracker for a Firecracker microVM (Linux only). Omit for non-sandboxed host execution.

--timeout <TIMEOUT>

The execution timeout in seconds. By default, 300 is used.

--iter <ITER>

The number of benchmark iterations to execute. By default, 1 is used.

--allow-failure

Allow a benchmark failure without short-circuiting the remaining iterations.

--entrypoint <ENTRYPOINT>

Override the container entrypoint. May be specified multiple times.

--cmd <CMD>

Override the container command. May be specified multiple times.

--env <ENV>

Set an environment variable in KEY=VALUE format. May be specified multiple times.

--network

Enable network access inside the VM.

--output <OUTPUT>

An output file path inside the guest to collect. May be specified multiple times.

--vcpus <VCPUS>

Override the number of vCPUs (for testing).

--memory <MEMORY>

Override the memory in MiB (for testing).

--disk <DISK>

Override the disk size in MiB (for testing).

--max-output-size <MAX_OUTPUT_SIZE>

The maximum size in bytes for collected stdout and stderr. By default, 25 MiB is used.

--max-file-count <MAX_FILE_COUNT>

The maximum number of output files to decode. By default, 255 is used.

--max-symlinks <MAX_SYMLINKS>

The maximum number of symlinks to follow during path resolution, matching the Linux kernel MAXSYMLINKS limit. By default, 40 is used. Only used in non-sandboxed mode, so it conflicts with --sandbox.

--grace-period <GRACE_PERIOD>

The grace period in seconds after the benchmark exits before final output collection. By default, 1 is used.

--sandbox-log-level <SANDBOX_LOG_LEVEL>

The log level for the sandbox process. Requires --sandbox. By default, warning is used.

Host Tuning

Before each benchmark, the runner applies host tuning to reduce measurement noise. By default it disables ASLR, the NMI watchdog, SMT / hyper-threading, and turbo boost; sets the CPU scaling governor to performance, swappiness to 10, and perf_event_paranoid to -1. The flags below keep individual optimizations at their host defaults instead.

--no-tuning

Disable all host tuning optimizations.

--aslr

Keep ASLR enabled (default: disabled for benchmarks).

--nmi-watchdog

Keep the NMI watchdog enabled (default: disabled for benchmarks).

--smt

Keep SMT / hyper-threading enabled (default: disabled for benchmarks).

--turbo

Keep turbo boost enabled (default: disabled for benchmarks).

--swappiness <SWAPPINESS>

Set the swappiness value. By default, 10 is used.

--governor <GOVERNOR>

Set the CPU scaling governor. By default, performance is used.

--perf-event-paranoid <PERF_EVENT_PARANOID>

Set the perf_event_paranoid value. By default, -1 is used.

--help

Print help.


Published: Fri, June 19, 2026 at 8:00:00 AM UTC