Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optional debugging output for AbstractIOHandlerImpl::flush() #1495

Merged
merged 2 commits into from
Aug 10, 2023
Merged

Optional debugging output for AbstractIOHandlerImpl::flush() #1495

merged 2 commits into from
Aug 10, 2023

Conversation

franzpoeschel
Copy link
Contributor

I've used something like this probably a hundred times already, this PR puts debugging output (disabled by default) for the backend IO tasks into AbstractIOHandlerImpl::flush().

To be enabled by manually putting a #define OPENPMD_DEBUG 1 in the corresponding .cpp file.
This can be seen as a first step toward proper debug logging that users can enable e.g. via an environment variable.

Sample output:

Filters: write_test                                                                                 [1185/1896]
[0->0xe1e938] CREATE_FILE: serial_write
[0xe1e938->0xe1eb08] CREATE_PATH: /data/
[0xe1eb08->0xe32088] CREATE_PATH: 1
[0->0xe1e938] WRITE_ATT: (STRING) particlesPath
[0xe32088->0xe31fb8] CREATE_PATH: particles/
[0xe31fb8->0xe16f88] CREATE_PATH: e
[0xe16f88->0xe2e768] CREATE_PATH: position
[0xe2e768->0xe2d598] CREATE_DATASET: x
[0xe2e768->0xe2d598] WRITE_DATASET
[0xe2e768->0xe2d598] WRITE_DATASET
[0xe2e768->0xe2d598] WRITE_DATASET
[0xe2e768->0xe2d598] WRITE_DATASET
[0xe2e768->0xe2d598] WRITE_ATT: (DOUBLE) unitSI
[0xe16f88->0xe2e768] WRITE_ATT: (FLOAT) timeOffset
[0xe16f88->0xe2e768] WRITE_ATT: (ARR_DBL_7) unitDimension
[0xe16f88->0xe2df78] CREATE_PATH: positionOffset
[0xe2df78->0xe2fa18] CREATE_DATASET: x
[0xe2df78->0xe2fa18] WRITE_DATASET
[0xe2df78->0xe2fa18] WRITE_DATASET
[0xe2df78->0xe2fa18] WRITE_DATASET
[0xe2df78->0xe2fa18] WRITE_DATASET
[0xe2df78->0xe2fa18] WRITE_ATT: (DOUBLE) unitSI
[0xe16f88->0xe2df78] WRITE_ATT: (FLOAT) timeOffset
[0xe16f88->0xe2df78] WRITE_ATT: (ARR_DBL_7) unitDimension
[0xe1eb08->0xe32088] WRITE_ATT: (DOUBLE) dt
[0xe1eb08->0xe32088] WRITE_ATT: (DOUBLE) time
[0xe1eb08->0xe32088] WRITE_ATT: (DOUBLE) timeUnitSI
[0xe1eb08->0xe1b8a8] CREATE_PATH: 2
[0xe1b8a8->0xe30498] CREATE_PATH: particles/
[0xe30498->0xe1a6b8] CREATE_PATH: e
[0xe1a6b8->0xe2f0d8] CREATE_PATH: position
[0xe2f0d8->0xe2f3a8] CREATE_DATASET: x
[0xe2f0d8->0xe2f3a8] WRITE_DATASET
[0xe2f0d8->0xe2f3a8] WRITE_DATASET
[0xe2f0d8->0xe2f3a8] WRITE_DATASET
[0xe2f0d8->0xe2f3a8] WRITE_DATASET
[0xe2f0d8->0xe2f3a8] WRITE_ATT: (DOUBLE) unitSI
[0xe1a6b8->0xe2f0d8] WRITE_ATT: (FLOAT) timeOffset
[0xe1a6b8->0xe2f0d8] WRITE_ATT: (ARR_DBL_7) unitDimension
[0xe1a6b8->0xe32498] CREATE_PATH: positionOffset
[0xe32498->0xe32778] CREATE_DATASET: x
[0xe32498->0xe32778] WRITE_DATASET
[0xe32498->0xe32778] WRITE_DATASET
[0xe32498->0xe32778] WRITE_DATASET
[0xe32498->0xe32778] WRITE_DATASET
[0xe32498->0xe32778] WRITE_ATT: (DOUBLE) unitSI
[0xe1a6b8->0xe32498] WRITE_ATT: (FLOAT) timeOffset
[0xe1a6b8->0xe32498] WRITE_ATT: (ARR_DBL_7) unitDimension

github-advanced-security[bot]
github-advanced-security bot found potential problems Aug 7, 2023
View reviewed changes
src/IO/AbstractIOHandlerImpl.cpp
#endif
}

std::future<void> AbstractIOHandlerImpl::flush()

Check warning

Code scanning / CodeQL

Poorly documented large function Warning

Poorly documented function: fewer than 2% comments for a function of 380 lines.
@ax3l ax3l self-requested a review August 9, 2023 22:10
@ax3l ax3l self-assigned this Aug 9, 2023
ax3l
ax3l approved these changes Aug 9, 2023
View reviewed changes
Copy link
Member

@ax3l ax3l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great!

Since it is not performance critical, one could also consider controlling this with an environment option (verbosity).

That way, no recompile needs to be done.

@franzpoeschel
Copy link
Contributor Author

This is great!

Since it is not performance critical, one could also consider controlling this with an environment option (verbosity).

That way, no recompile needs to be done.

I had not added that because of our plans to introduce proper logging at some point, but adding an environment variable is probably a good idea. This way, we can easily ask users to give us more details in bug reports without recompiling anything.
In that case, I would maybe even include this in 0.15.2?

@franzpoeschel franzpoeschel added this to the 0.15.2 milestone Aug 10, 2023
ax3l
ax3l approved these changes Aug 10, 2023
View reviewed changes
Copy link
Member

@ax3l ax3l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent, thank you!

@ax3l ax3l merged commit 0387b6f into openPMD:dev Aug 10, 2023
28 checks passed
eschnett added a commit to eschnett/openPMD-api that referenced this pull request Sep 5, 2023
@eschnett
Merge branch 'dev' into eschnett/julia-bindings
724806e 
* dev:
  Fix CMake: HDF5 Libs are PUBLIC (openPMD#1520)
  Fix `chmod` in `download_samples.sh` (openPMD#1518)
  CI: Old CTest (openPMD#1519)
  Python: Fix ODR Violation (openPMD#1521)
  replace extent in weighting and displacement (openPMD#1510)
  CMake: Warn and Continue on Empty HDF5_VERSION (openPMD#1512)
  Replace openPMD_Datatypes global with function (openPMD#1509)
  Streaming examples: Set WAN as default transport (openPMD#1511)
  TOML Backend (openPMD#1436)
  make it possible to manually set chunks when loading dask arrays (openPMD#1477)
  [pre-commit.ci] pre-commit autoupdate (openPMD#1504)
  Optional debugging output for AbstractIOHandlerImpl::flush() (openPMD#1495)
  Python: 3.8+ (openPMD#1502)

# Conflicts:
#	.github/workflows/linux.yml
#	src/binding/python/Series.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ax3l ax3l ax3l approved these changes

Assignees

@ax3l ax3l

Labels
internal
Projects
None yet
Milestone
0.15.2
Development

Successfully merging this pull request may close these issues.
2 participants
@franzpoeschel @ax3l