Use parallel I/O in Github Actions CI tests #277
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Now that NCDatasets is an optional dependency (coupled using an 'extension'), we do not need to use an old version of the HDF5 library provided by Julia, `HDF5_jll` (cf issue #148). Using the latest version will allow us to activate parallel I/O in the Github Actions CI jobs.
Is good to have a couple of tests that us `parallel_io=false` (the restart interpolation already did), so that even when parallel I/O is available, the branch that does not use it gets tested.
NCDatasets was already included for the serial tests. Now that we use the Julia-provided MPI for the parallel tests, it is also possible to install NCDatasets for the parallel tests.
It is good to still test this option even when parallel I/O is available.
If NCDatasets is installed (so the NetCDF I/O extension is activated), then use NetCDF for a few of the debug checks, to test that it is working properly.
By mistake, one command (creating the `mpiexecjl` executable) was run with `--check-bounds=no`. This might trigger re-precompilation of packages, so is inefficient at best.
At the moment installing NCDatasets (and therefore activating the NetCDF I/O extension) causes precompilation to hang in the debug checks CI job. Not sure why this is, but skip NCDatasets for now to avoid it.
johnomotani
force-pushed
the
ci-parallel-io
branch
from
7969b51 to
69faf43
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Now that NCDatasets is an optional dependency (coupled using an 'extension'), we do not need to use an old version of the HDF5 library provided by Julia,
HDF5_jll(cf #148). Using the latest version allows us to activate parallel I/O in the Github Actions CI jobs.As we use the Julia-provided MPI for the CI jobs, we can use NCDatasets at the same time (although this is not possible 'normally', see #254).
Explicitly sets
parallel_io=falsein one randomly-chosen test (the Harrison-Thompson one), to make sure that this code gets tested even when parallel I/O is available. Also setsparallel_io=falsein a few debug checks, and uses NetCDF (if avaliable) in a few other debug checks, to improve the code coverage.