Skip to content

Releases: ordo-one/package-benchmark

1.3.1

28 Mar 08:26
bec70e9
Compare
Choose a tag to compare

1.3.1 (2023-03-28)

Bug Fixes

1.3.0

27 Mar 09:58
e449f1c
Compare
Choose a tag to compare

What's Changed

Added the ability to track ARC traffic (retain/release) with three new metrics:

    /// Number of retains (ARC)
    case retainCount
    /// Number of releases (ARC)
    case releaseCount
    /// ABS(retains-releases) - if this is non-zero, it would typically mean the benchmark has a retain cycle (use Memory Graph Debugger to troubleshoot) or that startMeasurement/stopMeasurement aren't used properly
    case retainReleaseDelta

There's also a convenience BenchmarkMetric.arc that will add all three of those metrics.

Sample output:

image

Full Changelog: 1.2.0...1.3.0

1.2.0

22 Mar 16:46
93b8ae9
Compare
Choose a tag to compare

1.2.0 (2023-03-22)

To run benchmarks without jemalloc installed:

BENCHMARK_DISABLE_JEMALLOC=1 swift package benchmark

Features

  • minor: Make it possible to build and run without jemalloc available (#127) (93b8ae9)

1.1.2

22 Mar 14:17
3023ab1
Compare
Choose a tag to compare

1.1.2 (2023-03-22)

1.1.1

22 Mar 12:43
866db52
Compare
Choose a tag to compare

What's Changed

  • chore: Restructure into single target to simplify documentation and avoiding reexports by @hassila in #125

Full Changelog: 1.1.0...1.1.1

1.1.0

22 Mar 09:01
82f5b55
Compare
Choose a tag to compare

1.1.0 (2023-03-22)

Features

  • minor: Add convenience blackHole() to Benchmark, fix up documentation links (#124) (82f5b55)

1.0.2

22 Mar 08:02
6c80fcf
Compare
Choose a tag to compare

What's Changed

Full Changelog: 1.0.1...1.0.2

1.0.1

21 Mar 14:09
325fe5b
Compare
Choose a tag to compare

What's Changed

  • ci: 🔄 synced file(s) with ordo-one/public-repository-templates by @ordo-ci in #121
  • fix: Update parameter naming after refactor by @hassila in #120

Full Changelog: 1.0.0...1.0.1

1.0.0

21 Mar 13:31
5248a57
Compare
Choose a tag to compare

What's Changed

Added support for absolute benchmark thresholds.

Significant cleanup of dependencies, API naming and reduced boilerplate needed for benchmarks.
See benchmark samples update commit for some examples;

This needs small adoptions of existing benchmarks:

  1. Dependencies
    Replace:
.product(name: "BenchmarkSupport", package: "package-benchmark"),

with

.product(name: "Benchmark", package: "package-benchmark"),
.product(name: "BenchmarkPlugin", package: "package-benchmark"),
  1. Change boilerplate from:
import BenchmarkSupport
@main extension BenchmarkRunner {}

@_dynamicReplacement(for: registerBenchmarks)
func benchmarks() {
...
}

to

import Benchmark

let benchmarks = {
...
}
  1. BenchmarkResult.PercentileThresholds -> BenchmarkThresholds
  • feat: Eliminating use of @dynamicReplacement, auto generate boilerplate, update required dependencies by @hassila in #111
  • feat: Adding support for absolute thresholds by @hassila in #116
  • chore: Move to booleans instead of ints, fixes #114 by @hassila in #118
  • fix: Remove target name from benchmarkrunner as some characters causes issues by @hassila in #119

Full Changelog: 0.9.0...1.0.0

0.9.0

15 Mar 13:01
3b3d46e
Compare
Choose a tag to compare

A major release of Benchmark which includes several new features, bug fixes - but most importantly, significantly improved documentation thanks to @heckj who's made a great set of contributions in this release, thanks!

There are some source-breaking changes in this release as the API is cleaned up aiming for a 1.0 soonish (will capture feedback especially on the output formats and fine tune that a bit more, let us know how it works for you).

It's basically simple search-and-replace for any existing benchmarks:

benchmark.throughputIterations -> benchmark.scaledIterations
Benchmark.defaultConfiguration.desiredDuration -> Benchmark.defaultConfiguration.maxDuration
Benchmark.defaultConfiguration.desiredIterations -> Benchmark.defaultConfiguration.maxIterations
throughputScalingFactor -> scalingFactor

Major improvements are greatly expanded export formats (JMH, percentiles, etc), documentation and the new check command which can be used to check for regressions in CI with much improved and understandable output compared to 0.8.0.

What's Changed

  • feat: Add support for histogram export, clean up command line usage for release by @hassila in #64
  • chore: Change naming of desiredIterations/desiredDuration -> maxIterations/maxDuration by @hassila in #67
  • fix: Let maxDuration affect the true wall clock time, not the accumulated run time by @hassila in #70
  • fix: fixed up external references after rename by @heckj in #73
  • docs: use DocC for the documentation by @heckj in #63
  • chore: Make CPU time units consistent and polish scaling factor usage, improve debugging by @hassila in #76
  • feat: adds devContainer support by @heckj in #75
  • chore: migrated jemalloc feature to a broader set by @heckj in #80
  • feat: add encoded histogram export by @heckj in #84
  • chore: Documentation update by @hassila in #86
  • chore: Some initial refactoring to bring down complexity of benchmark runner, improved debugging experience by @hassila in #77
  • fix: doc cleanup after refactor and reorg by @heckj in #87
  • docs: reworked the core documentation pages by @heckj in #93
  • feat: Split compare / check to two different baseline commands by @hassila in #89
  • chore: Improve documentation by @hassila in #98
  • fix: Clean up required baseline parameters for consistency, cleanup of parameter passing to child process by @hassila in #96
  • feat: Display test names properly with metric grouping, add support for on-the-fly specification of metrics. by @hassila in #97
  • fix: Tab separated value output was broken by @hassila in #99
  • docs: Update documentation for new metrics option by @hassila in #100
  • feat: Add support for percentiles TSV output by @hassila in #102
  • feat: Add support throwable benchmarks, add startup/shutdown hooks by @hassila in #103
  • feat: histogram percentile export fixes by @hassila in #104

New Contributors

  • @heckj made their first contribution in #73

Full Changelog: 0.8.0...0.9.0