Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark results not consistent ? #29

Open
hasangenc0 opened this issue Feb 8, 2020 · 4 comments
Open

Benchmark results not consistent ? #29

hasangenc0 opened this issue Feb 8, 2020 · 4 comments
Labels
bug Something isn't working

Comments

@hasangenc0
Copy link

When I write my solution on the top of the other solutions the winner becomes my solution.

Code Result

But if I write my solution after others the winner is changing.

Code Result

Even if I write my solution twice the results are different.

Code Result

The results actually can be normal because of the behavior of this kind of performance analysis. I'm just wondering these tests are reliable.
Maybe the usage of Benchmark.prototype.hz is wrong 🤔

@yavuzkoca
Copy link
Contributor

Actually its quite normal since the solutions are pretty close to each other. However, since the solutions are tested in dockerized environment it would minimize the difference

@hasangenc0
Copy link
Author

It is right but the problem is when I right my solution at the top "always" my solution wins and if I write the bottom the winner is changing. And the difference between these even between the duplicate solutions is 2x.

@yavuzkoca
Copy link
Contributor

I don't think that it would depend order of the solutions but maybe there is a glitch

@Acanguven Acanguven added the bug Something isn't working label Feb 9, 2020
@Acanguven
Copy link
Owner

I am not sure if this is a bug, I will investigate this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants