Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New BNU soil type and VIIRS vegetation type data #843

Merged
merged 21 commits into from
Sep 26, 2023
Merged

New BNU soil type and VIIRS vegetation type data #843

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
f9e63a3
Add logic to ignore soil and clay percentage records.
GeorgeGayno-NOAA May 5, 2023
26eeef1
Merge branch 'develop' into bugfix/bnu_soil
GeorgeGayno-NOAA May 31, 2023
ecf2442
Update documentation and script logs for new files.
GeorgeGayno-NOAA Jun 1, 2023
400cc13
Merge branch 'develop' into bugfix/bnu_soil
GeorgeGayno-NOAA Aug 11, 2023
084d51c
Some script updates.
GeorgeGayno-NOAA Aug 14, 2023
98caf8b
Update orion driver script to use additional resources
GeorgeGayno-NOAA Aug 15, 2023
6179b24
Update jet grid driver script to request more resources
GeorgeGayno-NOAA Aug 15, 2023
fc17593
Minor updates to the wcoss2 driver grid script.
GeorgeGayno-NOAA Aug 15, 2023
22dadfe
Merge branch 'develop' into bugfix/bnu_soil
GeorgeGayno-NOAA Aug 16, 2023
6f3fe9e
Update some script comments. Update sfc_climo_gen script to use
GeorgeGayno-NOAA Aug 16, 2023
64fdfd7
Update comments in the sfc_climo_gen utility script.
GeorgeGayno-NOAA Aug 16, 2023
d1e4f51
Update "c96.viirs.vegt" regression test to use the new
GeorgeGayno-NOAA Aug 18, 2023
54b88b6
Update "c96.viirs.vegt" test to use new BNU soil
GeorgeGayno-NOAA Aug 18, 2023
cf4cda9
Rename "c96.viirs.vegt" test to "c96.viirs.bnu".
GeorgeGayno-NOAA Aug 18, 2023
093b3e3
Merge branch 'develop' into bugfix/bnu_soil
GeorgeGayno-NOAA Aug 21, 2023
4bc4e85
Adjust requested resources for Orion grid_gen regression
GeorgeGayno-NOAA Aug 21, 2023
397cd43
Request additional resources for Jet grid_gen regression
GeorgeGayno-NOAA Aug 21, 2023
5ea89bc
Update 'readthedocs' for the new substrate temperature data.
GeorgeGayno-NOAA Aug 21, 2023
76a3b02
Update for 'v3' of data.
GeorgeGayno-NOAA Sep 21, 2023
1e7c9fc
Increase stack size for creating grids on Orion.
GeorgeGayno-NOAA Sep 22, 2023
62294be
Update link_fixdirs.sh to point to new fixed data directory
GeorgeGayno-NOAA Sep 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 2 additions & 7 deletions docs/source/ufs_utils.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -539,19 +539,14 @@ The surface climatological data is located here `./fix/fix_sfc_climo <https://no
* N Hemis 30 sec - soil_type.statsgo.nh.30s.nc
* Global 30 sec - soil_type.statsgo.30s.nc
* Categorical BNU soil type
* Global 30-second - soil_type.bnu.30s.nc
* Global 30-second - soil_type.bnu.v2.30s.nc
* Categorical IGBP vegetation type
* MODIS-based global 0.05-degree - vegetation_type.modis.igbp.0.05.nc
* MODIS-based global 0.03-degree - vegetation_type.modis.igbp.0.03.nc
* MODIS-based CONUS 30 sec - vegetation_type.modis.igbp.conus.30s.nc
* MODIS-based N Hemis 30 sec - vegetation_type.modis.igbp.nh.30s.nc
* MODIS-based global 30 sec - vegetation_type.modis.igbp.30s.nc
* NESDIS VIIRS-based global 0.10-degree - vegetation_type.viirs.igbp.0.1.nc
* NESDIS VIIRS-based global 0.05-degree - vegetation_type.viirs.igbp.0.05.nc
* NESDIS VIIRS-based global 0.03-degree - vegetation_type.viirs.igbp.0.03.nc
* NESDIS VIIRS-based CONUS 30-second - vegetation_type.viirs.igbp.conus.30s.nc
* NESDIS VIIRS-based N HEMIS 30-second - vegetation_type.viirs.igbp.nh.30s.nc
* NESDIS VIIRS-based global 30-second - vegetation_type.viirs.igbp.30s.nc
* NESDIS VIIRS-based global 30-second - vegetation_type.viirs.v2.igbp.30s.nc
* Global 0.144-degree monthly vegetation greenness in percent - vegetation_greenness.0.144.nc

The files that define the model grid. All NetCDF.
Expand Down
15 changes: 5 additions & 10 deletions driver_scripts/driver_grid.hera.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
#SBATCH -o log.fv3_grid_driver
#SBATCH -e log.fv3_grid_driver
#SBATCH --nodes=1 --ntasks-per-node=24
##SBATCH --partition=bigmem
#SBATCH --partition=bigmem
#SBATCH -q debug
#SBATCH -t 00:20:00

Expand Down Expand Up @@ -86,30 +86,25 @@ export vegsoilt_frac='.false.' # When .false., output dominant soil and
# the dominant category. A Fortran logical,
# so include the dots.

export veg_type_src="modis.igbp.0.05" # Vegetation type data.
export veg_type_src="viirs.v2.igbp.30s" # Vegetation type data.
# For viirs-based vegetation type data, set to:
# 1) "viirs.igbp.0.1" for global 0.10-deg data
# 2) "viirs.igbp.0.05" for global 0.05-deg data
# 3) "viirs.igbp.0.03" for global 0.03-deg data
# 4) "viirs.igbp.conus.30s" for CONUS 30s data
# 5) "viirs.igbp.nh.30s" for NH 30s data
# 6) "viirs.igbp.30s" for global 30s data
# 1) "viirs.v2.igbp.30s" for global 30s data
# For the modis-based data, set to:
# 1) "modis.igbp.0.05" for global 0.05-deg data
# 2) "modis.igbp.0.03" for global 0.03-deg data
# 3) "modis.igbp.conus.30s" for CONUS 30s data
# 4) "modis.igbp.nh.30s" for N Hemis 30s data
# 5) "modis.igbp.30s" for global 30s data

export soil_type_src="statsgo.0.05" # Soil type data.
export soil_type_src="bnu.v2.30s" # Soil type data.
# For STATSGO data
# 1) "statsgo.0.05" for global 0.05-deg data
# 2) "statsgo.0.03" for global 0.03-deg data
# 3) "statsgo.conus.30s" for CONUS 30s data
# 4) "statsgo.nh.30s" for NH 30s data
# 5) "statsgo.30s" for global 30s data
# For Beijing Norm. Univ. data
# 1) "bnu.30s" for global 30s data.
# 1) "bnu.v2.30s" for global 30s data.

if [ $gtype = uniform ]; then
export res=96
Expand Down
21 changes: 8 additions & 13 deletions driver_scripts/driver_grid.jet.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@
#SBATCH --open-mode=truncate
#SBATCH -o log.fv3_grid_driver
#SBATCH -e log.fv3_grid_driver
#SBATCH --nodes=1 --ntasks-per-node=24
#SBATCH --nodes=3 --ntasks-per-node=10
#SBATCH --partition=xjet
#SBATCH -q batch
#SBATCH -t 00:10:00
#SBATCH -q debug
#SBATCH -t 00:30:00

#-----------------------------------------------------------------------
# Driver script to create a cubic-sphere based model grid on Jet.
Expand Down Expand Up @@ -84,30 +84,25 @@ export vegsoilt_frac='.false.' # When true, outputs percent of each
# outputs the dominant category. A
# Fortran logical, so include the dots.

export veg_type_src="modis.igbp.0.05" # Vegetation type data.
export veg_type_src="viirs.v2.igbp.30s" # Vegetation type data.
# For viirs-based vegetation type data, set to:
# 1) "viirs.igbp.0.1" for global 0.10-deg data
# 2) "viirs.igbp.0.05" for global 0.05-deg data
# 3) "viirs.igbp.0.03" for global 0.03-deg data
# 4) "viirs.igbp.conus.30s" for CONUS 30s data
# 5) "viirs.igbp.nh.30s" for NH 30s data
# 6) "viirs.igbp.30s" for global 30s data
# 1) "viirs.v2.igbp.30s" for global 30s data
# For the modis-based data, set to:
# 1) "modis.igbp.0.05" for global 0.05-deg data
# 2) "modis.igbp.0.03" for global 0.03-deg data
# 3) "modis.igbp.conus.30s" for CONUS 30s data
# 4) "modis.igbp.nh.30s" for N Hemis 30s data
# 5) "modis.igbp.30s" for global 30s data

export soil_type_src="statsgo.0.05" # Soil type data.
export soil_type_src="bnu.v2.30s" # Soil type data.
# For Beijing Normal Univ. data, set to:
# 1) "bnu.v2.30s" for global 30s data.
# For STATSGO soil type data, set to:
# 1) "statsgo.0.05" for global 0.05-deg data
# 2) "statsgo.0.03" for global 0.03-deg data
# 3) "statsgo.conus.30s" for CONUS 30s data
# 4) "statsgo.nh.30s" for NH 30s data
# 5) "statsgo.30s" for global 30s data
# For Beijing Normal Univ. data, set to:
# 1) "bnu.30s" for global 30s data.

if [ $gtype = uniform ]; then
export res=96
Expand Down
19 changes: 7 additions & 12 deletions driver_scripts/driver_grid.orion.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
#SBATCH --open-mode=truncate
#SBATCH -o log.fv3_grid_driver
#SBATCH -e log.fv3_grid_driver
#SBATCH --nodes=1 --ntasks-per-node=24
#SBATCH --nodes=2 --ntasks-per-node=15
#SBATCH -q debug
#SBATCH -t 00:30:00

Expand Down Expand Up @@ -71,7 +71,7 @@ module list
# Set grid specs here.
#-----------------------------------------------------------------------

export gtype=regional_esg # 'uniform', 'stretch', 'nest',
export gtype=uniform # 'uniform', 'stretch', 'nest',
# 'regional_gfdl', 'regional_esg'

export make_gsl_orog=false # When 'true' will output 'oro' files for
Expand All @@ -83,30 +83,25 @@ export vegsoilt_frac='.false.' # When true, outputs percent of each
# outputs the dominant category. A
# Fortran logical, so include the dots.

export veg_type_src="modis.igbp.0.05" # Vegetation type data.
export veg_type_src="viirs.v2.igbp.30s" # Vegetation type data.
# For viirs-based vegetation type data, set to:
# 1) "viirs.igbp.0.1" for global 0.10-deg data
# 2) "viirs.igbp.0.05" for global 0.05-deg data
# 3) "viirs.igbp.0.03" for global 0.03-deg data
# 4) "viirs.igbp.conus.30s" for CONUS 30s data
# 5) "viirs.igbp.nh.30s" for NH 30s data
# 6) "viirs.igbp.30s" for global 30s data
# 1) "viirs.v2.igbp.30s" for global 30s data
# For the modis-based data, set to:
# 1) "modis.igbp.0.05" for global 0.05-deg data
# 2) "modis.igbp.0.03" for global 0.03-deg data
# 3) "modis.igbp.conus.30s" for CONUS 30s data
# 4) "modis.igbp.nh.30s" for N Hemis 30s data
# 5) "modis.igbp.30s" for global 30s data

export soil_type_src="statsgo.0.05" # Soil type data.
export soil_type_src="bnu.v2.30s" # Soil type data.
# For Beijing Normal Univ. data, set to:
# 1) "bnu.v2.30s" for global 30s data.
# For STATSGO soil type data, set to:
# 1) "statsgo.0.05" for global 0.05-deg data
# 2) "statsgo.0.03" for global 0.03-deg data
# 3) "statsgo.conus.30s" for CONUS 30s data
# 4) "statsgo.nh.30s" for NH 30s data
# 5) "statsgo.30s" for global 30s data
# For Beijing Normal Univ. data, set to:
# 1) "bnu.30s" for global 30s data.

if [ $gtype = uniform ]; then
export res=96
Expand Down
19 changes: 7 additions & 12 deletions driver_scripts/driver_grid.wcoss2.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
#PBS -e log
#PBS -q debug
#PBS -A GFS-DEV
#PBS -l walltime=00:15:00
#PBS -l walltime=00:30:00
#PBS -N make_grid
#PBS -l select=1:ncpus=24:mem=500GB

Expand Down Expand Up @@ -70,7 +70,7 @@ module list
# Set grid specs here.
#-----------------------------------------------------------------------

export gtype=regional_esg # 'uniform', 'stretch', 'nest',
export gtype=uniform # 'uniform', 'stretch', 'nest',
# 'regional_gfdl', 'regional_esg'

export make_gsl_orog=false # When 'true' will output 'oro' files for
Expand All @@ -82,30 +82,25 @@ export vegsoilt_frac='.false.' # When true, outputs percent of each
# outputs the dominant category. A
# Fortran logical, so include the dots.

export veg_type_src="modis.igbp.0.05" # Vegetation type data.
export veg_type_src="viirs.v2.igbp.30s" # Vegetation type data.
# For viirs-based vegetation type data, set to:
# 1) "viirs.igbp.0.1" for global 0.10-deg data
# 2) "viirs.igbp.0.05" for global 0.05-deg data
# 3) "viirs.igbp.0.03" for global 0.03-deg data
# 4) "viirs.igbp.conus.30s" for CONUS 30s data
# 5) "viirs.igbp.nh.30s" for NH 30s data
# 6) "viirs.igbp.30s" for global 30s data
# 1) "viirs.v2.igbp.30s" for global 30s data
# For the modis-based data, set to:
# 1) "modis.igbp.0.05" for global 0.05-deg data
# 2) "modis.igbp.0.03" for global 0.03-deg data
# 3) "modis.igbp.conus.30s" for CONUS 30s data
# 4) "modis.igbp.nh.30s" for N Hemis 30s data
# 5) "modis.igbp.30s" for global 30s data

export soil_type_src="statsgo.0.05" # Soil type data
export soil_type_src="bnu.v2.30s" # Soil type data
# For Beijing Normal Univ. data, set to:
# 1) "bnu.v2.30s" for global 30s data.
# For STATSGO soil type data, set to:
# 1) "statsgo.0.05" for global 0.05-deg data
# 2) "statsgo.0.03" for global 0.03-deg data
# 3) "statsgo.conus.30s" for CONUS 30s data
# 4) "statsgo.nh.30s" for NH 30s data
# 5) "statsgo.30s" for global 30s data
# For Beijing Normal Univ. data, set to:
# 1) "bnu.30s" for global 30s data.

if [ $gtype = uniform ]; then
export res=96
Expand Down
2 changes: 1 addition & 1 deletion sorc/sfc_climo_gen.fd/search.f90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ subroutine search (field, mask, idim, jdim, tile, field_name)
case ('vegetation_type') ! vegetation type
default_value = float(3)
case default
print*,'- FATAL ERROR IN ROUTINE SEARCH. UNIDENTIFIED FIELD : ', field
print*,'- FATAL ERROR IN ROUTINE SEARCH. UNIDENTIFIED FIELD : ', field_name
call mpi_abort(mpi_comm_world, 77, ierr)
end select

Expand Down
21 changes: 16 additions & 5 deletions sorc/sfc_climo_gen.fd/source_grid.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ subroutine define_source_grid(localpet, npets, input_file)

integer, intent(in) :: localpet, npets

character(len=50) :: field_names_save(100)
character(len=50) :: vname

integer :: dimid, dims(1), ncid, status
Expand Down Expand Up @@ -160,10 +161,9 @@ subroutine define_source_grid(localpet, npets, input_file)
! number of variables minus 5.
!-----------------------------------------------------------------------

num_fields = num_vars - 5
num_records = num_vars * num_time_recs

allocate(field_names(num_fields))
! NOTE: the new BNU soil type data contains extra records for
! sand and clay percentages. These extra records are not need yet,
! so add logic to temporarily ignore them.

count = 0
do n = 1, num_vars
Expand All @@ -176,12 +176,23 @@ subroutine define_source_grid(localpet, npets, input_file)
if (trim(vname) == 'lon_corner') cycle
if (trim(vname) == 'lat') cycle
if (trim(vname) == 'lat_corner') cycle
if (trim(vname) == 'clay_lev1') cycle
if (trim(vname) == 'clay_top') cycle
if (trim(vname) == 'sand_lev1') cycle
if (trim(vname) == 'sand_top') cycle

count = count + 1
field_names(count) = vname
field_names_save(count) = vname

enddo

num_fields = count
num_records = num_vars * num_time_recs

allocate(field_names(num_fields))

field_names = field_names_save(1:num_fields)

if(localpet==0) print*,'- FIELDS TO BE PROCESSED: ', field_names

if (localpet == 0) then
Expand Down
2 changes: 1 addition & 1 deletion ush/sfc_climo_gen.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ fi
cat << EOF > ./fort.41
&config
input_facsf_file="${input_sfc_climo_dir}/facsf.1.0.nc"
input_substrate_temperature_file="${input_sfc_climo_dir}/substrate_temperature.2.6x1.5.nc"
input_substrate_temperature_file="${input_sfc_climo_dir}/substrate_temperature.gfs.0.5.nc"
input_maximum_snow_albedo_file="${input_sfc_climo_dir}/maximum_snow_albedo.0.05.nc"
input_snowfree_albedo_file="${input_sfc_climo_dir}/snowfree_albedo.4comp.0.05.nc"
input_slope_type_file="${input_sfc_climo_dir}/slope_type.1.0.nc"
Expand Down
27 changes: 12 additions & 15 deletions util/sfc_climo_gen/sfc_gen.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,7 @@
#
# veg_type_src - Input vegetation type data. Choices are:
# For viirs-based vegetation type data, set to:
# - "viirs.igbp.0.1" for global 0.10-deg data
# - "viirs.igbp.0.05" for global 0.05-deg data
# - "viirs.igbp.0.03" for global 0.03-deg data
# - "viirs.igbp.conus.30s" for CONUS 30s data
# - "viirs.igbp.nh.30s" for NH 30s data
# - "viirs.igbp.30s" for global 30s data
# - "viirs.v2.igbp.30s" for global 30s data
# For the modis-based vegetation data, set to:
# - "modis.igbp.0.05" for global 0.05-deg data
# - "modis.igbp.0.03" for global 0.03-deg data
Expand All @@ -53,14 +48,14 @@
# - "modis.igbp.30s" for global 30s data
#
# soil_type_src - Input soil type data. Choices are:
# For Beijing Norm. Univ. soil type data
# - "bnu.v2.30s" for global 30s data
# For STATSGO soil type data
# - "statsgo.0.05" for global 0.05-deg data
# - "statsgo.0.03" for global 0.03-deg data
# - "statsgo.conus.30s" for CONUS 30s data
# - "statsgo.nh.30s" for NH 30s data
# - "statsgo.30s" for global 30s data
# For Beijing Norm. Univ. soil type data
# - "bnu.30s" for global 30s data
#
# vegsoilt_frac - When .true., output the fraction of each
# vegetation and soil type and the dominant
Expand All @@ -70,23 +65,25 @@

set -x

#export res=96
export res=96.mx100
export res=768.mx025

#HALO=4
#export GRIDTYPE=regional
#FIX_REG=/lfs/h2/emc/stmp/$LOGNAME/fix.reg

export veg_type_src="modis.igbp.0.05"
export veg_type_src="viirs.v2.igbp.30s"

export soil_type_src="statsgo.0.05"
export soil_type_src="bnu.v2.30s"

export WORK_DIR=/lfs/h2/emc/stmp/$LOGNAME/work.sfc
export SAVE_DIR=/lfs/h2/emc/stmp/$LOGNAME/sfc.C${res}
export WORK_DIR=/scratch1/NCEPDEV/stmp2/$LOGNAME/work.sfc
export SAVE_DIR=/scratch1/NCEPDEV/stmp2/$LOGNAME/sfc.C${res}

export FIX_FV3=${BASE_DIR}/fix/orog/C${res}

export vegsoilt_frac=.true.
# Requires much more resources when true. On hera, use 6 nodes,
# 12 tasks per node. On WCOSS2, use 5 nodes, 12 tasks per node.

export vegsoilt_frac=.false.

#------------------------------------------------------------------------
#------------------------------------------------------------------------
Expand Down
Loading