Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/AquaCropv7.2 integration #1633

Open
wants to merge 107 commits into
base: master
Choose a base branch
from

Conversation

lbusschaert
Copy link
Contributor

Description

This PR introduces the implementation of a crop model (AquaCropv7.2) in LIS, along with the LDT parameter processing. The implementation was done within the KU Leuven team of Prof. Gabriëlle De Lannoy, mainly by Michel Bechtold and myself.
Research has been done with a preliminary version of AquaCropv7.0 in LIS:

The objective was now to develop a final working version in LIS.

I was visiting the NASA-LIS team in early 2024 and prepared this implementation in collaboration with Sujay Kumar and David Mocko. The implementation is now finalized and has been carefully tested and compared to AquaCrop reference output.

AquaCropv7.2 (ac72) is now part of the surfacemodels/land/. The implementation is documented in
LIS_AC_documentation.pdf

We acknowledge that this is a large PR and we greatly appreciate the time you spend on it. Do not hesitate to contact me in case of any questions ([email protected]).

Testcase

Test cases for LDT and LIS have been added under the ldt/testcases and lis/testcases folders. The data for the test cases are stored in a Zenodo repository organized in the requested format: https://zenodo.org/records/14035685

Testing was done for the generic crop having a 365-day cycle
for 4 restart times (1 Feb, 1 Aug, 1 Dec, 1 Jan next sim), as well as
for the AquaCrop Maize file planted on March 22nd.
Minor differences are supposably introduced by precision differences
(saving restart variables as single precision while AC uses doubles) but
have no impact on the final biomass, yield, Tact, Eact.

Note that the restart is not working for
- perennial crops
- GDD crops (will be fixed later)
- salinity
- (may need a more elaborate list somewhere)

No irrigation nor groundwater files were used as it has been decided
that those options will be forced from LIS. Irrigation will be
introduced in the next developments. Groundwater will not be introduced
for the time being.
@emkemp
Copy link
Contributor

emkemp commented Jan 27, 2025

An update:

(1) Adding a call to initializeGlobalStrings in ac72_setup clears the problem reported on Friday.
(2) It was also necessary to change SetTemperatureRecord, SetRainRecord, and SetEToRecord to copy each structure element separately rather than all at once. I do not know what the official Fortran standard says on this, but that's the workaround for our compiler on Discover.
(3) It was also necessary to add a call to SetTnxReferenceFileFull in ac72_setup to initialize the TnxReferenceFileFull global variable before use.

The code now fails in ac72_setup.F90 when accessing an uninitialized local variable (NrRuns).

            AC72_struc(n)%AC72(t)%NrRuns = NrRuns

Given the number of uninitialized variables I've encountered so far, my conclusion is the code needs more work from the developers, and that updates (and probably a rerun of the provided test case) should be performed before providing this to NASA again.

I can provide the code changes I made on my end upon request.

The compiler flags I used to detect these problems are:

CFLAGS = -c -g -Wall -Wcast-qual -Wcheck -Wdeprecated -Wextra-tokens -Wformat -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wpointer-arith -Wremarks -Wreturn-type -Wshadow -Wsign-compare -Wstrict-prototypes -Wtrigraphs -Wuninitialized -Wunused-function -Wunused-parameter -Wunused-variable -Wwrite-strings -fcheck=conversions,stack,uninit -fp-stack-check -fp-trap=common -fp-trap-all=common -ftrapv -traceback -DIFC -DLINUX -DLIS_JULES
FFLAGS77 = -c -g -check bounds,format,output_conversion,pointers,stack,uninit -fp-stack-check -ftrapuv -traceback -nomixed-str-len-arg -names lowercase -convert little_endian -assume byterecl -DSPMD -DHIDE_SHR_MSG -DNO_SHR_VMATH -DIFC -DLINUX -I$(MOD_ESMF) -DUSE_INCLUDE_MPI -I$(INC_ECCODES) -I$(INC_NETCDF) -I$(INC_HDFEOS) -I$(INC_HDF4) -I$(INC_HDF5) -DLIS_JULES
FFLAGS = -c -g -check bounds,format,output_conversion,pointers,stack,uninit -fp-stack-check -ftrapuv -u -traceback -fpe0 -nomixed-str-len-arg -names lowercase -convert little_endian -assume byterecl -DSPMD -DHIDE_SHR_MSG -DNO_SHR_VMATH -DIFC -DLINUX -I$(MOD_ESMF) -DUSE_INCLUDE_MPI -I$(INC_ECCODES) -I$(INC_NETCDF) -I$(INC_HDFEOS) -I$(INC_HDF4) -I$(INC_HDF5) -DLIS_JULES

Note that compiling without the -check uninit flag causes the program to assume all variables have valid values. If -O0, the compiler will often set values to zero regardless of whether it makes sense to do so. For -O1 and above, the values in uninitialized variables are totally random (whatever happened to be in that particular piece of memory when it was assigned to the variable).

@lbusschaert
Copy link
Contributor Author

Thank you for taking the time to find the problem, @emkemp! It is really appreciated.

We can take it back from here and review all the uninitialized variables. I'll update this PR and re-run the test case.

I will let you know when it is done.

Passes test case with strict check compilation
Fixed restart bug to make sure that the soil moisture is not reset
to FC. Test case runs with strict checks. Last step is to compile
and run with recent compiler
Working version on wICE (was not able to compile with strict
checks but -O0 works)
@lbusschaert
Copy link
Contributor Author

lbusschaert commented Feb 18, 2025

Hi David and Eric,

The code is now ready for you to be tested again. Main modifications include:

  • the initialization of previously uninitialized variables (checked with strict checks compilation)
  • the passing of derived type elements (when they contain a string) individually, rather than the whole structure at once.

Since we are already doing research with the code, I took the opportunity to already include two bug fixes:

  • bug fix related to irrigation file management
  • bug fix related to restarting

The output of the test case (TARGET_OUTPUT) only slightly changed. The largest relative difference is ~10^(-4) but I suspect this can be due to a difference in compilation/system. The testcase output has been updated on Zenodo (https://zenodo.org/records/14887278, version 2.1.0). Only the aquacropv72test_output.tar.gz was updated, the ldt and lis input files remain unchanged.

The test case was generated using the following lis.configure

FC              = mpiifx
FC77            = mpiifx
LD              = mpiifx
CC              = mpiicx
AR              = ar
MOD_ESMF        = /apps/leuven/rocky8/icelake/2023b/software/ESMF/8.6.1-intel-2023b/mod
LIB_ESMF        = /apps/leuven/rocky8/icelake/2023b/software/ESMF/8.6.1-intel-2023b/lib
INC_JPEG2000      = /apps/leuven/rocky8/icelake/2023b/software/JasPer/4.0.0-GCCcore-13.2.0/include/
LIB_JPEG2000      = /apps/leuven/rocky8/icelake/2023b/software/JasPer/4.0.0-GCCcore-13.2.0/lib64/
INC_ECCODES     = /apps/leuven/rocky8/icelake/2023b/software/ecCodes/2.31.0-iimpi-2023b/include/
LIB_ECCODES     = /apps/leuven/rocky8/icelake/2023b/software/ecCodes/2.31.0-iimpi-2023b/lib/
INC_NETCDF      = /apps/leuven/rocky8/icelake/2023b/software/netCDF/4.9.2-iimpi-2023b/include/
LIB_NETCDF      = /apps/leuven/rocky8/icelake/2023b/software/netCDF/4.9.2-iimpi-2023b/lib/
INC_HDF4        = 
LIB_HDF4        = 
INC_HDF5        = /apps/leuven/rocky8/icelake/2023b/software/HDF5/1.14.3-iimpi-2023b/include/
LIB_HDF5        = /apps/leuven/rocky8/icelake/2023b/software/HDF5/1.14.3-iimpi-2023b/lib/
INC_HDFEOS      = 
LIB_HDFEOS      = 
LIB_MINPACK     = 
INC_CRTM        = 
LIB_CRTM        = 
INC_PROF_UTIL   = 
LIB_PROF_UTIL   = 
INC_CMEM        = 
LIB_CMEM        = 
LIB_LAPACK      = 
INC_PETSC       = 
LIB_PETSC       = 
CFLAGS          = -c  -O3 -fp-model precise -traceback -DIFC -DLINUX -DLIS_JULES
FFLAGS77        = -c  -O3 -fp-model precise -traceback -nomixed-str-len-arg -names lowercase -convert big_endian -assume byterecl -DSPMD -DHIDE_SHR_MSG -DNO_SHR_VMATH -DIFC -DLINUX -I$(MOD_ESMF) -DUSE_INCLUDE_MPI -I$(INC_ECCODES) -I$(INC_NETCDF) -I$(INC_HDF5) -DLIS_JULES
FFLAGS          = -c  -O3 -fp-model precise -u -traceback -fpe0 -nomixed-str-len-arg -names lowercase -convert big_endian -assume byterecl -DSPMD -DHIDE_SHR_MSG -DNO_SHR_VMATH -DIFC -DLINUX -I$(MOD_ESMF) -DUSE_INCLUDE_MPI -I$(INC_ECCODES) -I$(INC_NETCDF) -I$(INC_HDF5) -DLIS_JULES
LDFLAGS         =  -L$(LIB_ESMF) -lesmf -lpioc -lstdc++ -limf -lrt -lmpi -lmkl -L$(LIB_ECCODES) -leccodes_f90 -leccodes -L$(LIB_JPEG2000) -ljasper -L$(LIB_NETCDF) -lnetcdff -lnetcdf -L$(LIB_HDF5) -lhdf5_fortran -lhdf5_hl -lhdf5 -lz
LIS_LIB_FLAGS   = -lesmf -lstdc++ -limf -lrt -leccodes_f90 -leccodes -ljasper -lnetcdff -lnetcdf -lhdf5_fortran -lhdf5_hl -lhdf5
LIS_LIB_PATHS   = -L$(LIB_ESMF) -L$(LIB_ECCODES) -L$(LIB_JPEG2000) -L$(LIB_NETCDF) -L$(LIB_HDF5)

Let me know if you need any extra input from my side, or if the code would crash again on the Discover nodes.

Many thanks!

@emkemp
Copy link
Contributor

emkemp commented Feb 21, 2025

Hello:

I successfully ran both LDT and LIS, and found minimal output differences with the latest targets. I also ran LIS in debug mode and found no issues.

I cannot create a LDT executable in debug mode (due to unrelated code -- the executable takes too much memory at link time). However, the compiler did report some unused variables. I will prepare updates to remove them and save a little memory.

@emkemp
Copy link
Contributor

emkemp commented Feb 27, 2025

Hi @lbusschaert

Can you please open up permissions in your GitHub repo so I can upload changes? @jvgeiger and I are doing a code review and there are some (mostly cosmetic) changes that we would like to make. But I can't push updates to your repo.

@lbusschaert
Copy link
Contributor Author

Hi @emkemp,

I've made you both collaborators to my repo. I think you should be able to push changes now. If not, please let me know.

Let me know if I can help with anything else.

Thanks

@emkemp
Copy link
Contributor

emkemp commented Feb 28, 2025

Hi @lbusschaert

Hmm, I'm still being blocked.

[emkemp@discover31 /discover/nobackup/projects/usaf_lis/emkemp/pr/1633/LISF/ldt]$ git push
ERROR: Permission to lbusschaert/LISF.git denied to emkemp.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

@jvgeiger Do I need to push this to my personal GitHub repo first, and then push from there?

@lbusschaert
Copy link
Contributor Author

Hi @emkemp,

You are both still marked as "pending invite", maybe you could try to accept it first?

image

@emkemp
Copy link
Contributor

emkemp commented Feb 28, 2025

Hi @lbusschaert

That was it! And my push went through (removing unused local variables).

I'll work on additional updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation Update to documentation (User's Guide or within the code) enhancement New feature or request NotReady
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants