Store conserved variable values to HDF5 #314
Conversation
There was a problem hiding this comment.
I have run some tests using BlastWave, ClusterMerger, CR_Diffusion, JetICMWall, MHD_OrszagTangVortex and Plummer to check values in HDF5 data and check the Record__Conservation can be reproduced after restart.
There is only one small issue:
- For the
BlastWavetest problem (serial by default) with--bitwise_reproducibility=trueandOPT__CK_CONSERVATIONturned on. TheRecord__Conservationof the restarted run is different (order of machine precision) from theRecord__Conservationof the fresh run.- However, if the tests are run with MPI (
--mpi=true), then the restartedRecord__Conservationis the same as the fresh run. - This issue also happens in the main branch: The recorded total momentum is different after the restart.
- However, if the tests are run with MPI (
And I have one minor suggestion:
- Since the variables
XXX_RefandXXX_H5Refhave become global and are not insideAux_Check_Conservation(), how about adjusting the variable names or adding some comments to describe what kind of references they are to avoid confusion?
|
Add a simple check of storing the reference value at |
Conflicts: src/Auxiliary/Aux_Check_Conservation.cpp
|
@hsinhaoHHuang The file |
Done! |
|
@hsinhaoHHuang I have added the force record reference value code. |
There was a problem hiding this comment.
After your latest changes, it (Plummer test problem) will show this error if OPT__CK_CONSERVATION is off.
[eureka04:64557] *** An error occurred in MPI_Allreduce
[eureka04:64557] *** reported by process [140131862052865,1719644775776256]
[eureka04:64557] *** on communicator MPI_COMM_WORLD
[eureka04:64557] *** MPI_ERR_TRUNCATE: message truncated
[eureka04:64557] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
[eureka04:64557] *** and potentially your MPI job)
|
@hsinhaoHHuang Thanks for reviewing. I have fixed the bug caused by |
There was a problem hiding this comment.
Thanks for the update!
I have finished further tests on restarting simulations from runs with OPT__CK_CONSERVATION on or off initially. The output Record__Conservation and those stored reference values in HDF5 data are all the same as the fresh runs.
I only have one final suggestion:
- Since
ANGMOM_ORIGIN_*will always be used to calculate the conservation reference, how about also always recording them inRecord__Noteat src/Auxiliary/Aux_TakeNote.cpp Line.1545? It is possible thatOPT__CK_CONSERVATIONis off at first and is turned on during the restart, and user may want to look up what origin (which could possibly be reset during restart) is used to calculate the angular momenta reference.
|
@hsinhaoHHuang Thanks for the review again! I have updated the code according to your suggestions. |
…/gamer-fork into save_conserved_vars
Conflicts: src/Output/Output_DumpData_Total_HDF5.cpp
Conflicts: src/Main/Main.cpp src/Output/Output_DumpData_Total_HDF5.cpp
|
@ChunYen-Chen Please resolve the conflicts. |
Conflicts: src/Output/Output_DumpData_Total_HDF5.cpp
Use the solution 1 mentioned in #305 to resolve #305. Now,
*_Refis assigned only at the start of the simulation, otherwise, it is loaded from HDF5.