Bencher Changelog
  
    v0.5.7
- Fix workspace level feature unification for non-Bencher Plus builds (Thank you @flokli)
 
v0.5.6
- Fix Iai-Callgrind adapter parsing bug (Thank you @gamma0987)
 
- Add alias for Iai-Callgrind to Gungraun rename, (
rust_iai_callgrind -> rust_gungraun) 
v0.5.5
- Add 
Login with Google as an authentication option 
- Allow the 
--file option to be specified multiple times to track multiple result files 
v0.5.4
- Fix fork pull request example (Thank you @ruyadorno)
 
- Improve database backup to run in the background
 
- Improve server stats to run in the background
 
- Add branch head version to the Console UI
 
- BREAKING CHANGE Creating an on-the-fly project when the 
CI environment variable is set to true will now require the --ci-on-the-fly flag (Thank you @jaqx0r) 
v0.5.3
- Improve 
iai-callgrind adapter (Thank you gamma0987) 
v0.5.2
- Fix empty slug validation bug
 
v0.5.1
- Fix horizontal wiggle in mobile view of the Console UI
 
- Add daily rate limiting to all resource creation
 
- Add 
plus.rate_limiting section to the API server configuration 
v0.5.0
- Allow 
bencher run to create a project if it does not exist for both anonymous and authenticated users 
- Deprecate the 
--no-hash flag for bencher run 
- Move to 
Linux, macOS, and Windows as the default Testbeds for bencher run 
- Add log scaling to Perf Images (Thank you @brianheineman)
 
- Add daily rate limiting for Metrics for Organizations without a Bencher Plus plan
 
v0.4.37
- Add dimension search in Threshold creation UI
 
- Add Threshold list to Branch, Testbed, and Measure pages in Console UI
 
- Add unit abbreviation to Perf Report tables in Console UI and PR comments (Thank you @BD103)
 
- Add support for GitHub Enterprise Server to 
bencher CLI (Thank you @btoic) 
v0.4.36
- Fix CORS failures when API server returns an error
 
- Catch errors when Console WASM fails to load
 
- Add 
--insecure-host flag to bencher CLI to allow insecure connections to Bencher Self-Hosted API servers 
- Add 
--native-tls flag to bencher CLI to load TLS certificates from the platform’s native certificate store 
- Add 
--timeout option to bencher CLI to set the timeout for all API requests 
- Add 
smtp.insecure_host option to Bencher Self-Hosted API server configuration to allow insecure connections to SMTP servers (Thank you @btoic) 
v0.4.35
- Add “Dismiss All” Alerts button to Console UI
 
- Fix infinite progress bar when Perf Plot is not initialized
 
- Show collapsed key for Pinned Plots
 
- Show selected Branches, Testbeds, and Benchmarks at the top of their Perf Plot tabs
 
- Improve Reports table view
 
- Show “Recent Reports” table for Branches and Testbeds
 
- Use exponential scaling for Perf Plot y-axis when there is a > 10x relative difference (Thank you @brianheineman)
 
- Improve new contributor onboarding experience (Thank you @brianheineman)
 
v0.4.34
- Add 
--head option to bencher branch view CLI subcommand 
- Allow selection of up to two Measures in Perf Images
 
- Fix the Perf Plot Pinned tab
 
- Add 
--build-time and --file-size to PR comment ID 
- Omit Threshold warning from PR comment when 
--ci-only-thresholds is used 
- Fix flashing of “No data found” in Perf Plot
 
v0.4.33
- Fix plot image y-axis labels
 
- Show correctly scaled units in Perf Plot hover box
 
- Factor out common dimensions in Perf Plot key
 
- Allow selection of up to two Measures in Perf Plots
 
v0.4.32
- Fix benchmark result units in PR comments (Thank you @dklassic)
 
v0.4.31
- Fix Report measure count bug
 
- Auto-scale units in Perf Plots, Perf Images, Reports, and PR comments
 
v0.4.30
- Fix Bencher Self-Hosted Console build bug since 
v0.4.24 
v0.4.29
- Improve Pinned Plots loading fallback
 
- Return 404 Not Found for all resource endpoints when checking for both existence and permissions
 
- Update CLI install scripts to be able to set the 
BENCHER_VERSION 
v0.4.28
- Remove lazy loading of embedded perf plots due to Safari bug
 
- Move Pinned Plots out of 
iframe tags 
- Fix custom pinned plot window size bug
 
v0.4.27
- Limit GitHub Action Check to 65,535 characters
 
- Fix CLI HTML table output bug for Benchmarks with different Measures
 
v0.4.26
- Fix GitHub Action summary bug (Thank you @dklassic)
 
v0.4.25
- Add GitHub Action summary when using 
--github-actions option 
- Add ability to track build times with the new 
--build-time flag 
v0.4.24
v0.4.23
- Fix Console Alert Perf Plot button bug
 
- Fix benchmark.js adapter (
js_benchmark) float parsing bug (Thank you @arv) 
- Fix Catch2 adapter (
cpp_catch2) long name parse bug 
- BREAKING CHANGE The Catch2 adapter (
cpp_catch2) now includes the test case name in the benchmark name (ex: Test Case: Benchmark Name) 
- Fix Benchmark results table x-overflow bug when there were several Measures
 
v0.4.22
- Remove bloom filter in Perf Query when there is no branch head specified
 
v0.4.21
- BREAKING CHANGE Update the Branch JSON to include the concept of a Head which now contains the Start Point
 
- Changing a branch’s start point will now update the branch head instead of creating a new branch and archiving the old one
 
- Deprecate 
--branch-start-point, --branch-start-point-hash, and --branch-reset in favor of --start-point, --start-point-hash, and --start-point-reset for bencher run 
- Introduce a 
max_versions for Branch Start Points, default to 255 
- Add the 
--start-point-max-versions option to bencher run 
- Thresholds are no longer cloned based off of a branch start point
 
- Add the 
--start-point-clone-thresholds flag to bencher run 
- Add Thresholds to Report JSON
 
- Add 
--threshold-measure, --threshold-test, --threshold-min-sample-size, --threshold-max-sample-size, --threshold-window, --threshold-lower-boundary, --threshold-upper-boundary, and --thresholds-reset to bencher run 
- Fix public plots pages to not require authentication
 
- Add 
--measure option to bencher mock to mock multiple measures 
- Add a 
bencher archive and bencher unarchive subcommand to archive and unarchive dimensions 
- Fix broken default version tag for 
bencher up (Thank you @mwasplund) 
- Add 
--api-port and --console-port options to bencher up to set the API and Console UI ports 
v0.4.20
- Add public pages for Reports, Branches, Testbeds, Benchmarks, Measures, and Plots
 
- Add links between all dimension pages
 
- BREAKING CHANGE Change the Report results format to mirror Bencher Metric Format, from measures owning benchmarks to benchmarks owning measures
 
- Remove user email from Report results
 
- Add a table view for Reports in the Console UI
 
- Update PR comments to be more concise
 
- Remove deprecated 
statistic field from Threshold in favor of model 
- Remove deprecated server endpoint 
/v0/server/endpoint in favor of /v0/server/config/console 
- Track global bus events for 
iai-callgrind adapter (Thank you @OmarTawfik) 
v0.4.19
- Add the 
--format option to bencher run 
- BREAKING CHANGE Remove the Report JSON from the default 
bencher run output (now use --format json) 
- BREAKING CHANGE Remove the 
--html flag from bencher run (now use --format html) 
- Standardize all 
bencher CLI default values 
- Allow 
--branch-reset to be used with --branch-start-point 
- Update archived branches renaming scheme
 
v0.4.18
- Add the concept of 
archived to Branches, Testbeds, Benchmarks, and Measures 
- Remove Alert stats endpoint
 
- Fix 
+ Add API token UI permissions bug 
- Filter for 
archived resources in the Console UI 
- Archive and unarchive resources in the Console UI
 
- Make Start Points for Branches fallible
 
- Parse results for multiple tools for 
iai-callgrind adapter (Thank you @OmarTawfik) 
v0.4.17
- Revamp UI for looking at resource details
 
- Remove cached Organization when logging out
 
v0.4.16
- Track Branches for pinned Plots across changes in their Start Point
 
- Update pinned Plots when theme changes
 
- Remove 15 second timeout for all CLI commands
 
- Show total number of pages for paginated lists in Console UI
 
- Add the ability to create and delete Organizations
 
v0.4.15
- Fix 404 error for dedicated Metric pages in Console
 
- Create Branch and Testbed, if they do not exist for a Report on the API server
 
- Remove Branch and Testbed 
soft creation 
- Add Docker daemon check to CLI commands
 
v0.4.14
- Fix iai-callgrind adapter bug
 
- Obfuscate user emails in API error messages
 
- Include Metric UUID in Report and Perf results
 
- Add a 
GET Metric API endpoint and CLI subcommand 
- Add a dedicated Metric page when clicking on a Metric dot in a Perf Plot
 
- Enable 
busy_timeout for SQLite connections to prevent database is locked errors 
- Add replica settings for disaster recovery
 
v0.4.13
- Add 
Required to required query params in docs (Thank you eddyashton) 
- Auto-log out in Console UI for expired sessions
 
- Fix wordmark color for default theme initialization
 
- Add 
bencher plot subcommand to manage pinned plots to CLI 
- Add 
--api-env/--console-env and --api-volume/--console-volume to bencher up CLI command (Thank you ricardaxel) 
- Add 
service argument to bencher up and bencher down to only control a single container 
v0.4.12
- Fix cross-project query bug
 
v0.4.11
- Reduce Console navbar, menu, and panel flickering on page load
 
- Fix 
Add, Edit, Delete button permissions bug (Thank you eddyashton for reporting this bug) 
- Add the ability to “pin” Perf Plots to create a dashboard of pinned plots
 
- Add a button to clear each dimension separately in the Perf Plot
 
v0.4.10
- Expand 
--file-size option for bencher run to be used multiple times to track multiple file sizes 
- Add dark mode to Bencher Console UI
 
v0.4.9
- Automatically find the 
git branch name for bencher run if one is not specified 
- Add the 
--file-size option to bencher run to track file size (ie binary size) 
- Allow custom Measures in Report results to be created on the fly
 
- Fix parsing new libtest bench Metrics as floats instead of integers (#390)
 
v0.4.8
- Add new user onboarding flow
 
- Fix server stats endpoint bug
 
- A very special thank you to hamirmahal for their first contribution!
 
v0.4.7
- Redirect to public Perf Page for Bencher Console links if user is not logged in
 
- Fix jittering in Perf Plot tabs when loading and changing pages
 
- Fix PR comment tags when changing branch start points
 
- Move benchmark results under a 
details tag for HTML comment output 
- Add an Alert summary table for HTML comment output
 
- Add a warning if no Thresholds are found for HTML comment output
 
- Deprecated: 
--ci-no-metrics flag no longer effects HTML comment output 
v0.4.6
- Add perf plot images as OpenGraph images
 
- Add GET 
/v0/users endpoint for server admins 
- Improve perf plot query performance by 120x or more by adding indexes and a materialized view
 
v0.4.5
- Move Bencher Cloud out of BETA
 
- Add tab loading progress bar to perf pages
 
- Add 
--branch-start-point and --branch-start-point-hash options to bencher run to track rebasing and other branch changes 
- Deprecated: The 
bencher run options and flags --if-branch, --else-if-branch, --else-branch, and --endif-branch
have been deprecated in favor of using --branch and --branch-start-point 
- BREAKING CHANGE 
--if-branch will now hard fail instead of skipping the run if not specified or if an empty string is provided, and it will always be created even without --else-branch set 
- BREAKING CHANGE 
--else-if-branch will only use the first branch specified as the start point and it must exist, if it is a non-empty string 
- Update branch date time plot to use local time and version plot to be discrete along the x-axis
 
v0.4.4
- Fix resource contention during large Perf queries
 
- Remove 
public query param from /v0/projects endpoint 
- BREAKING CHANGE Move all CLI commands using 
--org, --project, and --user over to positional arguments, except bencher run and bencher threshold create 
- Add a 
bencher report create CLI subcommand 
bencher run now tries to auto-fill the git commit hash for the report 
- BREAKING CHANGE Rename 
statistic to model and remove API endpoint 
- Add “Click to view Threshold” for all boundary limits and add a public view Threshold page
 
- Add PATCH endpoint for user account settings
 
- BREAKING CHANGE Remove 
/v0/server/ping endpoint 
- Deprecate 
/v0/server/endpoint in favor of /v0/server/config/console and BREAKING CHANGE remove bencher server endpoint CLI command 
v0.4.3
- BREAKING CHANGE Limit branch names to 256 characters and all slug lengths to 64 characters. Any existing values greater than these new limits are set to their respective UUID.
 
- Add shortcut URLs for user pages:
/console/users/tokens -> /console/users/USER-SLUG/tokens 
/console/users/tokens/add -> /console/users/USER-SLUG/tokens/add 
/console/users/settings -> /console/users/USER-SLUG/settings 
 
- Improve 
bencher run error messages to include the command being run (idea credit to dholroyd: #335) 
- Prevent pwn requests when using 
bencher run 
- Add an API version check to all Bencher CLI commands that use the backend
 
v0.4.2
- Fix Perf Plot view reduce bug
 
- Add a 
View button to Perf Plot for Reports, Branches, Testbeds, and Benchmarks 
- Add 
search query param to GET list endpoints for all named resources 
- Add search for a named resources in Console UI and Perf Plot dimensions
 
- Add date based search for Reports in Console UI and Perf Plot
 
- Add search for public projects
 
- Move Bencher API server to “distroless” Docker image, 
bencher-api that is 10x smaller 
- Move Bencher Console UI to “distroless” Docker image, 
bencher-console that is 1/3 smaller 
- Fix Console server side rendering bug when using local URLs inside of Docker
 
- First time contributor JonasWanke added the Iai-Callgrind benchmark harness adapter
 
v0.4.1
- Add a “Show all”/“Hide all” toggle for perf plot results (idea credit to jerome-benoit)
 
- Add an 
--exec flag to bencher run to run commands as executables (no shell) and by default when there is more than one argument 
- Add new Thresholds: Static, Percentage, Log Normal, Interquartile Range, and Delta Interquartile Range
 
- Include Iteration number for Report results
 
- Redirect to newly created resources in Console UI
 
v0.4.0
- BREAKING CHANGES Remove backwards compatibility for renames:
Metric (lower_bound => lower_value and upper_bound => upper_value) 
Measure (metric_kind => measure in Report and Threshold; metric_kinds => measures in Perf) 
Boundary (average => baseline) 
 
- Add 
bencher up, bencher down, and bencher logs commands to CLI to easily run Bencher Self-Hosted 
- Branches and Testbeds are now created if they do not exit for 
bencher run if the value passed is a slug or name (UUIDs are still required to already exist) 
v0.3.26
- Add OAuth2 based login for GitHub
 
- Allow users to accept invitations while logged in
 
- Run smoke tests on Linux, MacOS, and Windows
 
- Release Linux, MacOS, and Windows CLI builds for both x86_64 and ARM64
 
- Add convenience script to install 
bencher CLI on Linux, MacOS, and Windows 
v0.3.25
- Allow custom CI ID to be an unbounded length
 
v0.3.24
- Add URL fragment links to all headings 🔗
 
- Fix browser navigation to replace history for automatic query args, like pagination
 
- Scroll to top of page when changing paginated pages
 
- Allow Branch, Testbed, and Measure names to be used in addition to slugs and UUIDs for Report (
GET list & POST), Threshold (GET list & POST), and Branch (POST start point) endpoints 
- Validate Branch and Testbed exist before running benchmarks with 
bencher run 
v0.3.23
- Move CLI over to relaxed response parsing and add 
--strict flag to enable strict response parsing 
- Add 
--quiet flag to bencher run to enable quite mode and only output the final Report JSON 
v0.3.22
- The Boundary Limits object is now optional for Perf query results and Reports
 
- Update all 
POST API endpoints with creation semantics to return a 201 status code 
- Update all 
PUT and PATCH API endpoints with immediate update semantics to return a 200 status code 
- Update all 
DELETE API endpoints to return a 204 status code and an empty body 
v0.3.21
- Add Hyperfine adapter
 
- Rename 
Metric Kind to Measure 
v0.3.20
- Rename 
average to baseline for the Boundary Limits 
v0.3.19
- Include the 
average used for the Boundary Limits 
- Remove 
--ci-public-links for bencher run and simply use the project visibility to determine PR links 
- Replace 
--ci-with-metrics with --ci-no-metrics for bencher run and make showing benchmark metrics and boundary limits on PRs the default 
v0.3.18
- Add public Alert pages
 
- Add a 
--ci-public-links option to bencher run to only post public links to PRs 
- Add a 
--ci-with-metrics option to bencher run to display benchmark metrics and boundary limits on PRs 
- Update GitHub Actions PR documentation
 
v0.3.17
- Fix word break CSS in console UI
 
- Add status bar for perf plot loading
 
- Fix perf plot x-axis label
 
- Improve embedded perf plot layout
 
v0.3.16
- Improve 
bencher_client ergonomics for use in rustls 
- Fix navbar bug on public pages while logged in
 
- Add Bencher Learn section to the site
 
- Set max perf query permutations to 
255 
- BREAKING CHANGE: Move from single 
metric_kind to array of metric_kinds for perf plot query 
v0.3.15
v0.3.14
- Move default Threshold Boundaries from 
0.9 to 0.95 
- Redirect to login for an authorized users in Console UI
 
- Improve API server error messages
 
- Fixed the UI y-axis scale issue… again
 
- Add Bencher Plus licenses
 
v0.3.13
- Add ability for 
bencher run to read output from a file without running a command argument 
- Improve Report creation error messages
 
- Add filtering Reports by Branch, Testbed, and start/end time
 
- Add one month historical default when viewing Reports in the Perf Plot
 
- Add endpoint 
/v0/server/spec to view OpenAPI spec JSON 
- Remove old UI source and scripts
 
v0.3.12
- Change Metric 
lower_bound and upper_bound to lower_value and upper_value respectively 
- Add ability to visualize 
lower_value and upper_value in Perf Plot 
v0.3.11
- Add strongly typed IDs for database entities
 
- Remove deprecated configuration keys (
endpoint => console.url and secret_key => security.secret_key) 
v0.3.10
- Add optional error monitoring with Sentry
 
- Start API error message improvement
 
- Fix Threshold dimension selection bug in UI
 
- Display benchmark harness results in realtime via 
bencher run 
- A special thank you to @robertorosmaninho for bug hunting 👾
 
v0.3.9
- Move UI over to Astro with stronger TypeScript
 
- Add check for failing Netlify deployments
 
- Move default Threshold Boundaries from 
0.95 to 0.9 
v0.3.8
- Segment PR comments by the combination of Project, Branch, Testbed, and Adapter
 
- Add a custom PR comment ID argument 
--ci-id to bencher run for complex scenarios 
- Another special thank you to @GitGab19 for all the feedback
 
v0.3.7
- Fix slug length validation bug
 
- A special thank you to @GitGab19 for all the feedback
 
v0.3.6
- Fix multi-Metric Kind bug for Perf Plot Report selection
 
- Require minimum and maximum sample sizes to be greater than or equal to two
 
- Allow system Metric Kinds (ie Latency, Throughput, etc) to be deleted and recreated via a later Report
 
- Comment on GitHub pull requests using 
--github-actions argument for bencher run 
- Only post CI integrations when a Threshold is set for a Metric Kind, Branch, and Testbed with 
ci-only-thresholds 
- Only start posting CI integrations when an Alert is generated with 
--ci-only-on-alert 
- Fix Threshold boundary calculation bug when standard deviation is zero
 
- Add a warning ⚠️ in the Perf Plot UI when a Boundary Limit is not calculated
 
- A special thank you to @untitaker for all the feedback
 
v0.3.5
- Visualize Threshold lower and upper boundaries and Alerts in Perf Plot
 
- Add ability to dismiss and reactivate Alerts
 
- Add ability to update Thresholds in the UI
 
- Fix Threshold 
PUT bug 
- Add deletion capabilities to the UI
 
- Add help page to public UI
 
v0.3.4
- Revamp Quick Start tutorial
 
- Add Docker Self-Hosted tutorial
 
- Fix public perf pages bug
 
- Add Reports tab to perf pages
 
- Add ability to clear the perf plot
 
- Add ability to set Project URL to 
null 
- Add Project permissions API endpoint
 
- Remove redundant Project API endpoints
 
- Add ability to update Organizations, Projects, Metric Kinds, Branches, Testbeds, and Benchmarks in UI
 
v0.3.3
- Move API server config key 
endpoint to console.url 
- Change API from 
/v0/server/config/endpoint to /v0/server/endpoint to get console.url 
- Add dedicated 
POST API endpoint for Benchmarks 
- Add 
PATCH API endpoints for Organizations, Projects, Metric Kinds, Branches, Testbeds, and Benchmarks 
- Add 
PUT API endpoint for Thresholds 
- Add 
DELETE API endpoint for Metric Kinds, Branches, Testbeds, Benchmarks, and Thresholds 
- Create two default Thresholds for all new Projects
 
- Add a Go benchmark example
 
- Fix Go auxillary data parse error
 
v0.3.2
- Decrement all proceeding Branch versions after the final Report is deleted for a Branch version
 
- Add pagination to all GET list API endpoints, 
bencher CLI, and console UI 
- Move all 
bencher CLI API calls to using client auto-generated from OpenAPI spec 
- Improve 
bencher CLI error messages 
v0.3.1
- Add 
--backdate argument to bencher run for backdating Reports 
- Add 
created and modified timestamps as applicable to all entities and their JSON 
- Add ability to delete a Report
 
- Add ability to delete a Project
 
- A special thank you to @wuputah and the Hydra team for all the feedback
 
v0.3.0
- Add demo video to landing page
 
- Add 
bencher run documentation 
- Refactor Report, Alert, and Perf JSON to be more powerful
 
- All Report and Perf metrics now show the boundary limits used if a Threshold is set
 
- Display URL to visualize report in UI after 
bencher run 
- Change 
magnitude to pow for bencher mock 
- A very special thank you to @MichaelScofield for being Bencher’s second community contributor!
 
v0.2.46
- Update roadmap
 
- Add Iai benchmark harness adapter
 
- Move from GitPod over to Dev Containers/GitHub Codespaces
 
- Fix thresholds with no historical metrics bug
 
- A very special thank you to @osiewicz for being Bencher’s first community contributor!
 
v0.2.45
- Add servers to Swagger API docs
 
- Fix perf view refresh bug
 
- Add an order of 
--magnitude argument to bencher mock command 
- Add an API endpoint for threshold statistics and a CLI 
statistic subcommand 
- Upgrade the 
bencher perf CLI flag --table to accept an optional style parameter 
- Fix long text wrapping in UI deck view, both header and card contents
 
- Autogenerate TypeScript types from Rust type definitions
 
- Massive thank you to @vient for all the feedback
 
v0.2.44
- Fix UI expired token error
 
- Move the CLI linker over to use Zig in order to pin a specific 
glibc version 
- Move linker in CI over to 
mold for increased parallelism 
- Fix cargo workspace 
default-features = false warnings 
- Add project stewardship commitments
 
- Massive thank you to @brainstorm for all the feedback
 
v0.2.43
- Add a 
--table option to the bencher perf command to display query results as a table 
- Add the ability to toggle the perf plot x-axis from report date time to branch version number
 
v0.2.42
- Create and publish Bencher CLI GitHub Action
 
- Fix both UI and perf image y-axis sizing once and for all… hopefully
 
v0.2.41
- Move perf plot images from a screenshot to server side rendering
 
- Add a share mechanism to perf pages with a perf image preview
 
v0.2.40
- Add an optional 
--file argument to bencher run to read file output 
- Move perf query API endpoint from a POST to GET
 
- Add an endpoint to GET a screenshot of a public perf plot
 
v0.2.39
- Update readme to add Discord invite
 
- Allow certain benchmarks to be ignored by Threshold checks
 
- BREAKING CHANGE: Make Project names unique within an organization and delete existing duplicates
 
- Add paid plans and metered billing for Bencher Cloud
 
- Change perf pages to use milliseconds as opposed to date/time stamp
 
v0.2.38
- Fix CLI runtime dependency error over the past five releases
 
- BREAKING CHANGE: Make Metric Kind, Branch, and Testbed names unique and delete existing duplicates
 
- Add a soft branch creation option to eliminate race conditions in CI
 
- Add supported languages to landing page
 
- Lots of new docs!
 
v0.2.33
- Add new 
average setting for adapters to choose between mean/standard deviation and median/interquartile range when possible 
- Increase CLI default retry to ten attempts three seconds apart (30 second window)
 
- Add ability to clone thresholds when creating a new branch from a start point and make this the default behavior for 
--else-if-branch in bencher run CLI 
- Moved the plot key to below the plot all the time in preparation for screenshots/embedding
 
v0.2.32
- Add STARTTLS option for SMTP connections
 
- Add 
throughput as a new default metric kind with operations / second (ops/s) as units 
- Add Java JMH, C# BenchmarkDotNet, JavaScript Benchmark.js, JavaScript console.time/timeEnd, Python airspeed velocity, Python pytest-benchmark, and Ruby Benchmark adapters
 
- Add resend email button to UI auth confirm page
 
- Change C++ Catch2 adapter to use standard deviation for 
lower_bound and upper_bound 
- Fix table and perf panel formatting for long, single word values
 
v0.2.31
- Prevent leaking secrets by obfuscating them in the server logs (ie 
************) 
- Move run failure detection to the CLI, adapter parsing will no longer attempt error detection
 
- New harness specific adapter targets (ie 
rust_bench and rust_criterion) 
- Add additional options to 
bencher mock, --fail and --flaky to easily test parsing failing and flaky benchmarks respectively 
- Wrote continuous benchmarking and how to install the CLI pages
 
- Update server framework, dropshot, and Rust to 
1.67.0 
- Two new C++ adapters for Google Benchmark and Catch2
 
- Add an adapter for 
go test -bench 
v0.2.30
- Added 
--if-else-branch, --else-branch, and --endif-branch flags to to bencher run CLI 
- Wrote a branch management explanation page for using 
bencher run 
- Completely refactored docs UI configuration to make it easier to add new pages, one place instead of three
 
- Made the Diataxis convention explicit in the docs
 
- Fixed bounds bug in 
bencher mock 
- Added a 
/v0/server/backup route and bencher server backup CLI command to perform database backups with options to compress, push to a remote data store (currently only AWS S3), and remove the local copy 
v0.2.29
- Fixed and refixed Criterion empty benchmark name bug
 
- Type level checking for benchmark names and metric kind resource IDs
 
- Added documentation: Quick Start, Prior Art, and GitHub Actions how to
 
- Prettier and eslint added to CI/CD
 
- Stubbed out UI tests with 
vitest in CI/CD 
v0.2.28
- Centralized all validation logic between frontend and backend via Rust + WASM modules
 
- All pages in the UI now have basic functionality
 
- Added public project perf pages
 
v0.2.27
- Fixed the UI y-axis scale cutoff issue! Note to self: RTFD
 
- Fixed the UI documentation + routing bug! It turns out 
SwaggerUIBundle was the solution. 
- Added Metric Kind list, add, and view to UI
 
- In perf queries allowed for both UUIDs and slugs for branches and testbeds
 
- Added bencher.dev to Google Search Console and Analytics
 
- Changed API server default URL to always be localhost:3000
 
- Moved Docker UI build stage into its own 
builder.Dockerfile, mirroring the API setup 
- Created a better default for the UI’s backend API URL, basically 
this-host:61016 
- Created a walking skeleton of adding Rust + WASM modules to the UI in both dev and CI/CD
 
- Fixed UI logout + routing “too many redirects” bug
 
- Changed notifications from state based to URL query param based in the UI
 
- Refactored how pathname and redirects are handled in the UI to increase encapsulation
 
- Added the current version to the UI 
BETA tag 
- Created a repo shortcut at bencher.dev/repo
 
     
              
              Published: Fri, October 27, 2023 at 8:40:00 AM UTC | Last Updated: Sun, February 16, 2025 at 6:23:00 PM UTC