You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've noticed - without taking a look at the code - that the benchmarks round numbers too much, or have a lower precision than i expect.
For example if a threshold tolerance is set to 3%, and the result is 2.8% above the thresholds, the benchmark will fail because it'll round 2.8 to 3%, then fail the benchmark since 3% == 3%.
Or for the faster of the benchmarks, I'd like to have microsecond precision instead of millisecond, so i don't need to use a 6% threshold tolerance just because the benchmark could move a bit and round it up to the next 10-millisecond.
The text was updated successfully, but these errors were encountered:
I've noticed - without taking a look at the code - that the benchmarks round numbers too much, or have a lower precision than i expect.
For example if a threshold tolerance is set to 3%, and the result is 2.8% above the thresholds, the benchmark will fail because it'll round 2.8 to 3%, then fail the benchmark since 3% == 3%.
Or for the faster of the benchmarks, I'd like to have microsecond precision instead of millisecond, so i don't need to use a 6% threshold tolerance just because the benchmark could move a bit and round it up to the next 10-millisecond.
The text was updated successfully, but these errors were encountered: