Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adios2 warn groupbased encoding #1498

Merged
merged 15 commits into from
Dec 22, 2023

Conversation

franzpoeschel
Copy link
Contributor

@franzpoeschel franzpoeschel commented Aug 8, 2023

Should not go into 0.15 release cycle since it advises to use variable-based encoding

  • Merge Warn on BP5+Blosc in ADIOS2 v2.9 up to patch level 1 #1497 first
  • Add some environment variable or similar to deactivate this The warning is now only printed for the new ADIOS2 encoding (group table), so this does not affect legacy workflows
  • Maybe adapt tests Postponed to when variable-based encoding becomes more useful
  • Entirely forbid usage without steps? Would make things easier
    EDIT: adios2.usesteps option is now removed, is automatically used if WriteIterations is used in the frontend
  • For a less abrupt transition, apply warnings only in new schema (i.e. group table enabled)

More context: #1457

The warning is printed only when using the (currently opt-in) group table feature, for the sake of a smoother transition for users.
We should soon make group tables opt-out. A good time would be when we have better support for variable-based encoding.

Breaking because: It's no longer possible to open ADIOS2 group-based files with linear access mode if they don't use the group table feature.

@ax3l
Copy link
Member

ax3l commented Aug 9, 2023

@franzpoeschel ready for rebase :)

@franzpoeschel franzpoeschel force-pushed the adios2-warn-groupbased-encoding branch from 09f3077 to 97d2d7d Compare August 10, 2023 09:44
@franzpoeschel
Copy link
Contributor Author

@ax3l You recently mentioned a workflow that you have where you repeatedly close and reopen an Iteration to write data to it. Since this workflow implies that you cannot use variable-based encoding, I assume that you use file-based encoding instead?
I want to avoid discontinuing group-based encoding in ADIOS2 while there are workflows which actually use it.

@franzpoeschel franzpoeschel force-pushed the adios2-warn-groupbased-encoding branch from 97d2d7d to ad5b6b7 Compare August 15, 2023 14:38
@franzpoeschel
Copy link
Contributor Author

I would like to adapt all examples and tests to this change. Since this requires more features for file-based enconding (e.g. reopening a closed iteration) and for variable-based encoding (e.g. random-access) alike, I will postpone this PR for now.

@franzpoeschel franzpoeschel force-pushed the adios2-warn-groupbased-encoding branch 4 times, most recently from 185b37f to 88cc81f Compare October 19, 2023 13:50
@franzpoeschel franzpoeschel force-pushed the adios2-warn-groupbased-encoding branch 3 times, most recently from d89e566 to 6e68d77 Compare November 20, 2023 15:53
@ax3l ax3l requested review from ax3l and guj December 22, 2023 13:15
@ax3l ax3l self-assigned this Dec 22, 2023
@ax3l ax3l added the api: breaking breaking API changes label Dec 22, 2023
@ax3l ax3l merged commit e668e86 into openPMD:dev Dec 22, 2023
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants