Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(tm2/pkg/cmap): add benchmarks to show true impact of contention #3540

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

odeke-em
Copy link
Contributor

The Go standard library's sync.Map is touted as great for cases with high load and is commonly known knowledge but the benchmark that I am committing shows otherwise that for this library's usage, it is so much more expensive hence this benchmark will avoid someone committing sync.Map without seeing the true implications.

$ benchstat map_w_mutex.txt stdlib_sync_map.txt
name                               old time/op    new time/op    delta
CMapConcurrentInsertsDeletesHas-8     1.72s ±11%     1.92s ± 3%   +11.66%  (p=0.000 n=10+9)
CMapHas-8                             109ns ± 9%     118ns ± 3%    +8.26%  (p=0.002 n=10+8)

name                               old alloc/op   new alloc/op   delta
CMapConcurrentInsertsDeletesHas-8    1.18GB ± 2%    3.21GB ± 3%  +172.09%  (p=0.000 n=10+10)
CMapHas-8                             16.0B ± 0%     16.0B ± 0%      ~     (all equal)

name                               old allocs/op  new allocs/op  delta
CMapConcurrentInsertsDeletesHas-8      824k ± 0%     4433k ± 0%  +437.89%  (p=0.000 n=10+10)
CMapHas-8                              2.00 ± 0%      1.60 ±38%      ~     (p=0.065 n=9+10)

Updates #3505

The Go standard library's sync.Map is touted as great for cases with
high load and is commonly known knowledge but the benchmark that I am
committing shows otherwise that for this library's usage, it is so much
more expensive hence this benchmark will avoid someone committing
sync.Map without seeing the true implications.

```shell
$ benchstat map_w_mutex.txt stdlib_sync_map.txt
name                               old time/op    new time/op    delta
CMapConcurrentInsertsDeletesHas-8     1.72s ±11%     1.92s ± 3%   +11.66%  (p=0.000 n=10+9)
CMapHas-8                             109ns ± 9%     118ns ± 3%    +8.26%  (p=0.002 n=10+8)

name                               old alloc/op   new alloc/op   delta
CMapConcurrentInsertsDeletesHas-8    1.18GB ± 2%    3.21GB ± 3%  +172.09%  (p=0.000 n=10+10)
CMapHas-8                             16.0B ± 0%     16.0B ± 0%      ~     (all equal)

name                               old allocs/op  new allocs/op  delta
CMapConcurrentInsertsDeletesHas-8      824k ± 0%     4433k ± 0%  +437.89%  (p=0.000 n=10+10)
CMapHas-8                              2.00 ± 0%      1.60 ±38%      ~     (p=0.065 n=9+10)
```

Updates gnolang#3505
@Gno2D2
Copy link
Collaborator

Gno2D2 commented Jan 17, 2025

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):
  • IGNORE the bot requirements for this PR (force green CI check)
  • The pull request description provides enough details
Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)

☑️ Contributor Actions:
  1. Fix any issues flagged by automated checks.
  2. Follow the Contributor Checklist to ensure your PR is ready for review.
    • Add new tests, or document why they are unnecessary.
    • Provide clear examples/screenshots, if necessary.
    • Update documentation, if required.
    • Ensure no breaking changes, or include BREAKING CHANGE notes.
    • Link related issues/PRs, where applicable.
☑️ Reviewer Actions:
  1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.
📚 Resources:
Debug
Automated Checks
Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 The pull request was created from a fork (head branch repo: odeke-em/gno)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

  • Any user with comment edit permission
The pull request description provides enough details

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 Not (🔴 Pull request author is a member of the team: core-contributors)
    └── 🟢 Not (🔴 Pull request author is user: dependabot[bot])

Can be checked by

  • team core-contributors

@odeke-em odeke-em changed the title test(tm2/pkg/cmap): adds benchmarks to show true impact of contention test(tm2/pkg/cmap): add benchmarks to show true impact of contention Jan 17, 2025
Copy link

codecov bot commented Jan 17, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

@notJoon notJoon added the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📦 🌐 tendermint v2 Issues or PRs tm2 related review/triage-pending PRs opened by external contributors that are waiting for the 1st review
Projects
Status: In Review
Development

Successfully merging this pull request may close these issues.

3 participants