Bencher Metric Format (BMF)

Built-in benchmark harness adapters convert benchmark output into Bencher Metric Format (BMF) JSON. Custom benchmarking harnesses should output their results as BMF JSON and use the json adapter. See how to track custom benchmarks for a full overview. To generate mock BMF JSON data, use the bencher mock CLI subcommand.

Example Bencher Metric Format (BMF) JSON

{
  "benchmark_name": {
    "latency": {
      "value": 88.0,
      "lower_value": 87.42,
      "upper_value": 88.88
    },
    "throughput": {
      "value": 5.55,
      "lower_value": 3.14,
      "upper_value": 6.30
    }
  },
  "/bin/sh -c ./my_build_script.sh": {
    "build-time": {
      "value": 87.0
    }
  },
  "my_binary": {
    "file-size": {
      "value": 42.0
    }
  }
}

In this example, there are three Benchmarks:

• benchmark_name Benchmark with two Measures:
  • Latency (latency)
  • Throughput (throughput)
• /bin/sh -c ./my_build_script.sh Benchmark with one Measure to track build time (build-time)
• my_binary Benchmark with one Measure to track file size (file-size)

Order of Precedence

The order of precedence for identifying a Benchmark is as follows:

1. UUID
2. Kebab case name with a -bencher-ignore suffix that is stripped to suppress alerts (max length 1024 characters)
3. slug (max length 64 characters)
4. name (max length 1024 characters)

The order of precedence for identifying a Measure is as follows:

1. UUID
2. slug (max length 64 characters)
3. name (max length 64 characters)

Bencher Metric Format (BMF) JSON Schema

This is the JSON schema for Bencher Metric Format (BMF) JSON:

{
  "$id": "https://bencher.dev/bmf.json",
  "$schema": "http://json-schema.org/draft-07/schema",
  "type": "object",
  "patternProperties": {
    ".+": {
      "type": "object",
      "patternProperties": {
        ".+": {
          "type": "object",
          "properties": {
            "value": {
              "type": "number"
            },
            "lower_value": {
              "type": "number"
            },
            "upper_value": {
              "type": "number"
            }
          },
          "required": ["value"]
        }
      }
    }
  }
}

Schema Versions:

• Latest: https://bencher.dev/bmf.json
• v0 (latest): https://bencher.dev/v0/bmf.json