Releases: ordo-one/package-benchmark
Releases · ordo-one/package-benchmark
1.5.0
Added additional ways to customize startup/teardown code for benchmarks
There are now multiple ways to setup shared benchmark setup/teardown code (the old startupHook/shutdownHooks are deprecated and will be removed in a future major release, please use Benchmark.setup and Benchmark.teardown instead).
First off, one can easily setup global shared state that can be reused by just defining it in the closure:
import Benchmark
let benchmarks = {
let mySharedSetup = [1, 2, 3, 4]
Benchmark("Minimal benchmark") { benchmark in
// Some work to measure here, use mySharedSetup
}
Benchmark("Minimal benchmark 2") { benchmark in
// Some work to measure here, use mySharedSetup here too
}
}Secondly, one can have setup/teardown closures that are shared across all benchmarks:
import Benchmark
let benchmarks = {
Benchmark.setup = { print("global setup closure, used for all benchmarks") }
Benchmark.teardown = { print("global teardown closure, used for all benchmarks") }
Benchmark("Minimal benchmark") { benchmark in
// Some work to measure here, use mySharedSetup
}
Benchmark("Minimal benchmark 2") { benchmark in
// Some work to measure here, use mySharedSetup here too
}
}Thirdly, one can have setup/teardown closures as part of the configuration for a subset of benchmarks
import Benchmark
func setupFunction() {
}
func teardownFunction() {
}
let benchmarks = {
// only shared setup
Benchmark("Minimal benchmark",
configuration: .init(setup: setupFunction, teardown: teardownFunction)) { benchmark in
// Some work to measure here
}
Benchmark("Minimal benchmark 2",
configuration: .init(setup: setupFunction, teardown: teardownFunction)) { benchmark in
// Some work to measure here
}
}Finally, one can have setup/teardown closures that are specific to a given benchmark
import Benchmark
let benchmarks = {
Benchmark("Minimal benchmark") { benchmark in
} setup: {
// do setup for this benchmark here
} teardown: {
// do teardown here
}
}All of these setup/teardown hooks can be combined, the order of execution is:
- Global setup
- Configuration provided setup
- Closure provided setup
with teardown in reverse order.
So to use all hooks at the same time:
import Benchmark
func sharedSetup() {
}
func sharedTeardown() {
}
let benchmarks = {
Benchmark.setup = { print("global setup closure, used for all benchmarks") }
Benchmark.teardown = { print("global teardown closure, used for all benchmarks") }
Benchmark("Minimal benchmark",
configuration: .init(setup: setupFunction, teardown: teardownFunction)) { benchmark in
} setup: {
// do setup for this benchmark here
} teardown: {
// do teardown here
}
}1.5.0 (2023-04-21)
Features
1.4.8
1.4.7
1.4.6: fix: Fix Swift 5.7 toolchain support (#149)
Workaround https://github.com/apple/swift/issues/60876 so we can build with Swift 5.7 again
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.4.0 (2023-03-28)
New command for faster creation of benchmarks introduced:
swift package --allow-writing-to-package-directory benchmark init MyNewBenchmarkThis will:
- Create the path
Benchmarks/MyNewBenchmark - Add the file
Benchmarks/MyNewBenchmark/MyNewBenchmark.swiftwith the required boilerplate - Add the executable target for the benchmark to the end of your Package.swift file
You still need to add the dependency to the benchmark package first as previously.
Sample:
swift package --allow-writing-to-package-directory benchmark init MyNewBenchmark
swift package benchmark --target MyNewBenchmark
...
Building for debugging...
Build complete! (0.33s)
Building benchmark targets in release mode for benchmark run...
Building MyBenchmarkTarget
Build complete!
==================
Running Benchmarks
==================
100% [------------------------------------------------------------] ETA: 00:00:00 | MyBenchmarkTarget:SomeBenchmark
====================================================================================================
Baseline 'Current run'
====================================================================================================
Host 'max.local' with 10 'arm64' processors with 64 GB memory, running:
Darwin Kernel Version 22.3.0: Mon Jan 30 20:38:37 PST 2023; root:xnu-8792.81.3~2/RELEASE_ARM64_T6000
=================
MyBenchmarkTarget
=================
SomeBenchmark
╒════════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric │ p0 │ p25 │ p50 │ p75 │ p90 │ p99 │ p100 │ Samples │
╞════════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Malloc (total) │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ 10000 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Memory (resident peak) (K) │ 7618 │ 7639 │ 7655 │ 7655 │ 7671 │ 7671 │ 7671 │ 10000 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Throughput (# / s) (M) │ 24 │ 23 │ 12 │ 12 │ 11 │ 8 │ 0 │ 9999 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (total CPU) (ns) │ 500 │ 583 │ 584 │ 625 │ 750 │ 1208 │ 12791 │ 10000 │
├────────────────────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Time (wall clock) (ns) │ 41 │ 42 │ 83 │ 83 │ 84 │ 125 │ 5875 │ 9999 │
╘════════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛