1.11.0
1.11.0 (2023-09-14)
Features
Its desirable to be able to signal from CI if a threshold has been improved too and not only regressed, so we now return exit code 0 only for exact equal checks and 2 for regressions and 4 for improvements for baseline checks and absolute threshold comparisons.
Added support for signposts (#183) which makes profiling from Instruments easier to analyze.
The easiest way to access Instruments and profile a benchmark, is to simply quit Xcode and open the package with:
open --env BENCHMARK_DISABLE_JEMALLOC=1 Package.swift
This works around the Xcode bug that makes it impossible to use jemalloc and one can simply build for Profile and use instruments as usual - now with signpost information per benchmark run:
Added new objectAllocCount ARC stat counter and fixed the delta calculation to also take this initial refcount into account so retain/release deltas should be closer to reality (still sometimes non-zero due to some missing runtime hooks).
Also added support for optionally returning setupState from the benchmark setup hooks, to allow access to state that can be reused per benchmark iteration which is provided as an additional second parameter to the benchmark closure.
This can easily be done like this:
Benchmark("SetupTeardown6") { benchmark, setupState in
print("Array of integers: \(setupState)")
} setup: {
[1, 2, 3]
}