Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backport: Optimize chained hyperslab selection. #1032

Merged
merged 2 commits into from
Jul 26, 2024

Conversation

1uc
Copy link
Collaborator

@1uc 1uc commented Jul 25, 2024

A common pattern for creating semi-unstructured selection is to use many
(small) RegularHyperSlab and chain them:

HyperSlab hyperslab;
for(auto slab : regular_hyper_slabs) {
  hyperslab |= slab;
}

This eventually triggers calling:

for(auto slab : regular_hyper_slabs) {
  auto [offset, stride, counts, blocks] = slab;
  H5Sselect_hyperslab(space_id, offset, stride, counts, block);
}

Measurements show that this has runtime that's quadratic in the number
of regular hyper slabs. This starts becoming prohibitive at 10k - 40k
slabs.

We noticed that H5Scombine_select does not suffer from the same
performance issue. This allows us to optimize (long) chain of Op::Or
using divide and conquer.

The current implementation only optimizes streaks of Op::Or.

@1uc 1uc changed the title 1uc/backport optimize hyperslab selection Optimize chained hyperslab selection. Jul 25, 2024
@1uc 1uc changed the title Optimize chained hyperslab selection. backport: Optimize chained hyperslab selection. Jul 25, 2024
@1uc 1uc force-pushed the 1uc/backport-optimize-hyperslab-selection branch from 7bd8150 to 6e7b550 Compare July 25, 2024 15:38
Copy link

codecov bot commented Jul 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.24%. Comparing base (84d9ce8) to head (a7b22df).

Additional details and impacted files
@@            Coverage Diff             @@
##             v2.x    #1032      +/-   ##
==========================================
+ Coverage   86.13%   86.24%   +0.10%     
==========================================
  Files          90       90              
  Lines        5858     5902      +44     
==========================================
+ Hits         5046     5090      +44     
  Misses        812      812              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@1uc 1uc force-pushed the 1uc/backport-optimize-hyperslab-selection branch from 6e7b550 to a7b22df Compare July 26, 2024 06:03
A common pattern for creating semi-unstructured selection is to use many
(small) RegularHyperSlab and chain them:

```
HyperSlab hyperslab;
for(auto slab : regular_hyper_slabs) {
  hyperslab |= slab;
}
```

This eventually triggers calling:
```
for(auto slab : regular_hyper_slabs) {
  auto [offset, stride, counts, blocks] = slab;
  H5Sselect_hyperslab(space_id, offset, stride, counts, block);
}
```

Measurements show that this has runtime that's quadratic in the number
of regular hyper slabs. This starts becoming prohibitive at 10k - 40k
slabs.

We noticed that `H5Scombine_select` does not suffer from the same
performance issue. This allows us to optimize (long) chain of `Op::Or`
using divide and conquer.

The current implementation only optimizes streaks of `Op::Or`.
@1uc 1uc force-pushed the 1uc/backport-optimize-hyperslab-selection branch from a7b22df to 47b2bf2 Compare July 26, 2024 08:59
@1uc 1uc marked this pull request as ready for review July 26, 2024 09:01
@1uc 1uc merged commit 05d6402 into v2.x Jul 26, 2024
32 checks passed
@1uc 1uc deleted the 1uc/backport-optimize-hyperslab-selection branch July 26, 2024 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants