Releases: ordo-one/package-benchmark
1.3.1
1.3.0
What's Changed
Added the ability to track ARC traffic (retain/release) with three new metrics:
/// Number of retains (ARC)
case retainCount
/// Number of releases (ARC)
case releaseCount
/// ABS(retains-releases) - if this is non-zero, it would typically mean the benchmark has a retain cycle (use Memory Graph Debugger to troubleshoot) or that startMeasurement/stopMeasurement aren't used properly
case retainReleaseDelta
There's also a convenience BenchmarkMetric.arc
that will add all three of those metrics.
Sample output:
Full Changelog: 1.2.0...1.3.0
1.2.0
1.1.2
1.1.1
1.1.0
1.0.2
1.0.1
1.0.0
What's Changed
Added support for absolute benchmark thresholds.
Significant cleanup of dependencies, API naming and reduced boilerplate needed for benchmarks.
See benchmark samples update commit for some examples;
This needs small adoptions of existing benchmarks:
- Dependencies
Replace:
.product(name: "BenchmarkSupport", package: "package-benchmark"),
with
.product(name: "Benchmark", package: "package-benchmark"),
.product(name: "BenchmarkPlugin", package: "package-benchmark"),
- Change boilerplate from:
import BenchmarkSupport
@main extension BenchmarkRunner {}
@_dynamicReplacement(for: registerBenchmarks)
func benchmarks() {
...
}
to
import Benchmark
let benchmarks = {
...
}
BenchmarkResult.PercentileThresholds
->BenchmarkThresholds
- feat: Eliminating use of @dynamicReplacement, auto generate boilerplate, update required dependencies by @hassila in #111
- feat: Adding support for absolute thresholds by @hassila in #116
- chore: Move to booleans instead of ints, fixes #114 by @hassila in #118
- fix: Remove target name from benchmarkrunner as some characters causes issues by @hassila in #119
Full Changelog: 0.9.0...1.0.0
0.9.0
A major release of Benchmark which includes several new features, bug fixes - but most importantly, significantly improved documentation thanks to @heckj who's made a great set of contributions in this release, thanks!
There are some source-breaking changes in this release as the API is cleaned up aiming for a 1.0 soonish (will capture feedback especially on the output formats and fine tune that a bit more, let us know how it works for you).
It's basically simple search-and-replace for any existing benchmarks:
benchmark.throughputIterations -> benchmark.scaledIterations
Benchmark.defaultConfiguration.desiredDuration -> Benchmark.defaultConfiguration.maxDuration
Benchmark.defaultConfiguration.desiredIterations -> Benchmark.defaultConfiguration.maxIterations
throughputScalingFactor -> scalingFactor
Major improvements are greatly expanded export formats (JMH, percentiles, etc), documentation and the new check
command which can be used to check for regressions in CI with much improved and understandable output compared to 0.8.0.
What's Changed
- feat: Add support for histogram export, clean up command line usage for release by @hassila in #64
- chore: Change naming of desiredIterations/desiredDuration -> maxIterations/maxDuration by @hassila in #67
- fix: Let maxDuration affect the true wall clock time, not the accumulated run time by @hassila in #70
- fix: fixed up external references after rename by @heckj in #73
- docs: use DocC for the documentation by @heckj in #63
- chore: Make CPU time units consistent and polish scaling factor usage, improve debugging by @hassila in #76
- feat: adds devContainer support by @heckj in #75
- chore: migrated jemalloc feature to a broader set by @heckj in #80
- feat: add encoded histogram export by @heckj in #84
- chore: Documentation update by @hassila in #86
- chore: Some initial refactoring to bring down complexity of benchmark runner, improved debugging experience by @hassila in #77
- fix: doc cleanup after refactor and reorg by @heckj in #87
- docs: reworked the core documentation pages by @heckj in #93
- feat: Split compare / check to two different baseline commands by @hassila in #89
- chore: Improve documentation by @hassila in #98
- fix: Clean up required baseline parameters for consistency, cleanup of parameter passing to child process by @hassila in #96
- feat: Display test names properly with metric grouping, add support for on-the-fly specification of metrics. by @hassila in #97
- fix: Tab separated value output was broken by @hassila in #99
- docs: Update documentation for new metrics option by @hassila in #100
- feat: Add support for percentiles TSV output by @hassila in #102
- feat: Add support throwable benchmarks, add startup/shutdown hooks by @hassila in #103
- feat: histogram percentile export fixes by @hassila in #104
New Contributors
Full Changelog: 0.8.0...0.9.0