[WIP] Unify Random-Access API and Streaming API into Series::snapshots() #1592
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
franzpoeschel
added
discussion
api: breaking
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
franzpoeschel
force-pushed
the
topic-iterator
branch
from
0ee77a2
to
73336c9
Compare
franzpoeschel
force-pushed
the
topic-iterator
branch
2 times, most recently
from
1e862b7
to
ab0991d
Compare
src/snapshots/StatefulIterator.cpp
Outdated
Comment on lines
100
to
235
| switch (series.iterationEncoding()) | ||
| { | ||
| using IE = IterationEncoding; | ||
| case IE::fileBased: | ||
| series.readFileBased(); | ||
| break; | ||
| case IE::groupBased: | ||
| case IE::variableBased: { | ||
| Parameter<Operation::OPEN_FILE> fOpen; | ||
| fOpen.name = series.get().m_name; | ||
| series.IOHandler()->enqueue(IOTask(&series, fOpen)); | ||
| series.IOHandler()->flush(internal::defaultFlushParams); | ||
| using PP = Parameter<Operation::OPEN_FILE>::ParsePreference; | ||
| switch (*fOpen.out_parsePreference) | ||
| { | ||
| case PP::PerStep: | ||
| series.advance(AdvanceMode::BEGINSTEP); | ||
| series.readGorVBased( | ||
| /* do_always_throw_errors = */ false, /* init = */ true); | ||
| break; | ||
| case PP::UpFront: | ||
| series.readGorVBased( | ||
| /* do_always_throw_errors = */ false, /* init = */ true); | ||
| /* | ||
| * In linear read mode (where no parsing at all is done upon | ||
| * constructing the Series), it might turn out after parsing | ||
| * that what we expected to be a group-based Series was in fact | ||
| * a single file of a file-based Series. | ||
| * (E.g. when opening "data00000100.h5" directly instead of | ||
| * "data%T.h5") | ||
| * So we need to check the current value of | ||
| * `iterationEncoding()` once more. | ||
| */ | ||
| if (series.iterationEncoding() != IterationEncoding::fileBased) | ||
| { | ||
| series.advance(AdvanceMode::BEGINSTEP); | ||
| } | ||
| break; | ||
| } | ||
| data.parsePreference = *fOpen.out_parsePreference; | ||
| break; | ||
| } | ||
| } |
Check notice
Code scanning / CodeQL
Long switch case Note
franzpoeschel
force-pushed
the
topic-iterator
branch
3 times, most recently
from
3845ed7
to
b5e4217
Compare
franzpoeschel
force-pushed
the
topic-iterator
branch
from
b5e4217
to
9987204
Compare
src/Series.cpp
Outdated
| Access at, | ||
| std::string const &options, | ||
| // Either an MPI_Comm or none, the template works for both options | ||
| MPI_Communicator &&...comm) |
Check notice
Code scanning / CodeQL
Unused local variable Note
franzpoeschel
force-pushed
the
topic-iterator
branch
5 times, most recently
from
667ee24
to
f5d8433
Compare
franzpoeschel
force-pushed
the
topic-iterator
branch
from
f5d8433
to
26e5a49
Compare
Comment on lines
+88
to
+117
| // increment/decrement | ||
| // ChildClass &operator++(); | ||
| // ChildClass &operator--(); | ||
| // ChildClass &operator++(int); | ||
| // ChildClass &operator--(int); |
Check notice
Code scanning / CodeQL
Commented-out code Note
Comment on lines
+82
to
+109
| // dereference | ||
| // value_type const &operator*() const = 0; | ||
| // value_type &operator*() = 0; |
Check notice
Code scanning / CodeQL
Commented-out code Note
| @@ -185,6 +196,7 @@ | |||
| } | |||
| // ~Series intentionally not yet called | |||
|
|
|||
| // std::cout << "READ " << filename << std::endl; | |||
Check notice
Code scanning / CodeQL
Commented-out code Note test
| @@ -163,6 +173,7 @@ | |||
| auxiliary::getEnvNum("OPENPMD_TEST_NFILES_MAX", 1030); | |||
| std::string filename = | |||
| "../samples/many_iterations/many_iterations_%T." + ext; | |||
| // std::cout << "WRITE " << filename << std::endl; | |||
Check notice
Code scanning / CodeQL
Commented-out code Note test
This was referenced Mar 19, 2024
franzpoeschel
force-pushed
the
topic-iterator
branch
from
19395b0
to
5b85307
Compare
for more information, see https://pre-commit.ci
Proper return() is supported beginning with CMake 3.25 only
for more information, see https://pre-commit.ci
franzpoeschel
force-pushed
the
topic-iterator
branch
from
a73c4c6
to
d278794
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
description soon
TODO:
usingdefinitions, empty headerssnapshots(Snapshots::RandomAccess)vssnapshots(Snapshots::CollectiveAccess)or sosnapshots().begin()-> reset to start, add sth likesnapshots().current()defer_iteration_parsingin READ_LINEAR modeDiff: franzpoeschel/openPMD-api@shared-attributable-data...topic-iterator