Bencher Self-Hosted


Bencher is open source. It is available as a SaaS, Bencher Cloud, and as a self-hostable set of container images:

Deployment

The easiest way to deploy Bencher Self-Hosted is using a triptych of bencher CLI subcommands:

  • bencher up

    bencher up

    Create and start the Bencher Self-Hosted containers. This is similar to running docker compose up.

    bencher up [OPTIONS] [SERVICE]

    Service

    Select the containers to run. The possible choices are: all, api, and console. By default all is chosen. Similar to the SERVICE argument for docker compose up.

    Options

    --detach

    Run in detached mode. The containers will be run in the background. Similar to the --detach flag for docker compose up.

    --pull <WHEN>

    Pull images before running. The possible choices are: always, missing, and never. By default always is chosen. Similar to the --pull option for docker compose up.

    --tag <TAG>

    Specify the image tag for the containers. By default the image tag matches the bencher CLI version. The most recent bencher CLI version is v0.4.36.

    --console-port <PORT>

    Specify a port number for the Console server container. By default, port 3000 is used. Similar to the port only form of the --expose option for docker run.

    --api-port <PORT>

    Specify a port number for the API server container. By default, port 61016 is used. Similar to the port only form of the --expose option for docker run.

    --console-env <KEY_VALUE>

    Pass an environment variable to the Console server container. The expected format is KEY=value. This can be useful when working with the Console Server configuration. Similar to the --env option for docker run.

    --api-env <KEY_VALUE>

    Pass an environment variable to the API server container. The expected format is KEY=value. This can be useful when working with the API Server configuration. Similar to the --env option for docker run.

    --console-volume <HOST_CONTAINER>

    Pass a mount volume to the Console server container. The expected format is /host/path:/container/path. This can be useful when working with the Console Server configuration. Similar to the --volume option for docker run.

    --api-volume <HOST_CONTAINER>

    Pass a mount volume to the API server container. The expected format is /host/path:/container/path. This can be useful when working with the API Server configuration. Similar to the --volume option for docker run.

    --help

    Print help.

  • bencher logs

    bencher logs

    View output from the Bencher Self-Hosted containers, if bencher up was run with the --detach flag. This is similar to running docker compose logs.

    bencher up [OPTIONS] [SERVICE]

    Service

    Select the containers to run. The possible choices are: all, api, and console. By default all is chosen. Similar to the SERVICE argument for docker compose logs.

    Options

    --help

    Print help.

  • bencher down

    bencher down

    Stop and remove the Bencher Self-Hosted containers, if bencher up was run with the --detach flag. This is similar to running docker compose down.

    bencher down [OPTIONS] [SERVICE]

    Service

    Select the containers to run. The possible choices are: all, api, and console. By default all is chosen. Similar to the SERVICE argument for docker compose down.

    Options

    --help

    Print help.

See the Bencher Compose reference for more details.

Alternatively, you could just use docker compose to orchestrate the containers.

docker-compose.yml
services:
bencher_api_compose:
image: ghcr.io/bencherdev/bencher-api:latest
container_name: bencher_api
ports:
- 61016:61016
volumes:
# Mount the configuration directory
- "./services/api/etc:/etc/bencher"
# Mount the database directory
- "./services/api/data:/var/lib/bencher/data"
# environment:
# Set configuration string or file path
# - BENCHER_CONFIG=
# - BENCHER_CONFIG_PATH=
bencher_console_compose:
image: ghcr.io/bencherdev/bencher-console:latest
container_name: bencher_console
ports:
- 3000:3000
environment:
# Set the API URL to the localhost IPv4 address
- BENCHER_API_URL=http://127.0.0.1:61016
# This is needed for Docker-to-Docker communication when using a local `BENCHER_API_URL`
# Do not set this when using a remote `BENCHER_API_URL`, such as https://bencher.example.com
- INTERNAL_API_URL=http://host.docker.internal:61016
depends_on:
- bencher_api_compose

See the Docker Compose reference example for more details.

HTTPS

In order to setup HTTPS can use Bencher Self-Hosted behind a proxy with HTTPS enabled or use a domain name with a valid SSL certificate.

Host URL

When using Bencher Self-Hosted, you will need to set the --host option to the URL of your Bencher API server for most bencher CLI subcommands.

For example, if you were using the bencher run CLI subcommand with a Bencher API server at https://api.bencher.example.com then you could write something like this:

Terminal window
bencher run \
--host https://api.bencher.example.com \
--project save-walter-white-1234abcd \
--branch main \
--testbed ci-runner \
--adapter json \
bencher mock

Custom CA Certificates

By default, the bencher CLI loads certificates from the bundled webpki-roots crate. The webpki-roots are a reliable set of trust roots from Mozilla, and including them in bencher improves portability and performance. This is especially true on macOS, where reading the system trust store incurs a significant delay.

However, in some cases, you may want to use the platform’s native certificate store, especially if you’re relying on a corporate trust root that’s included in your system’s certificate store for a mandatory proxy or self-signed Bencher Self-Hosted connections. To instruct bencher to use the system’s trust store, run bencher with the --native-tls flag.

If a direct path to the certificate is required, set the SSL_CERT_FILE environment variable to the path of the certificate bundle, to instruct bencher to use that file instead of the system’s trust store. This is often the case in CI environments where the system’s trust store is not available.

If client certificate authentication (mTLS) is desired, set the SSL_CLIENT_CERT environment variable to the path of the PEM formatted file containing the certificate followed by the private key.

Finally, if you’re using a setup in which you want to trust a self-signed certificate or otherwise disable certificate verification, you can instruct bencher to allow insecure connections to dedicated hosts via the --insecure-host flag. This flag is only applicable to HTTPS connections, as HTTP connections are inherently insecure.

WARNING: Only use --insecure-host in a secure network with verified sources, as it bypasses SSL verification and could expose you to man-in-the-middle attacks.

Backups

The easiest way to backup your Bencher Self-Hosted API server is using the bencher server backup CLI subcommand. The user must be an admin on the server to use this subcommand.

bencher server backup

bencher server backup

Backup the API server database.

Options

--host <URL>

The Bencher Self-Hosted API server URL. For example, https://api.bencher.example.com.

--token <TOKEN>

Either the --token option or the BENCHER_API_TOKEN environment variable must be set to a valid API token. If both are specified, the --token option takes precedence over the BENCHER_API_TOKEN environment variable.

--compress

Compress the database backup with gzip. This operation runs first.

--data-store <DATA_STORE>

Save the database backup to this data store. This operation runs second.

Supported data stores:

  • aws_s3: Backup to AWS S3

Use the database.data_store section of the API server configuration to setup the data store for your Bencher Self-Hosted API server.

--remove

Remove the local copy of the database backup. This operation runs third.

Disaster Recovery

In order to prevent data loss in between backups, Bencher offers a built-in disaster recovery solution. This feature requires an active Bencher Plus license.

Supported replication schemes:

  • file: Replicate to a local file path
  • sftp: Replicate over SFTP
  • s3: Replicate to any S3 compatible blob storage
  • abs: Replicate to Azure Blob Storage
  • gcs: Replicate to Google Cloud Storage

Use the plus.disaster_recovery section of the API server configuration to setup disaster recovery for your Bencher Self-Hosted API server.



Published: Mon, October 14, 2024 at 6:20:00 AM UTC | Last Updated: Sat, October 19, 2024 at 7:30:00 PM UTC