Feature/gefs v13 coupled #1
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
On branch feature/gefs_v13_coupled modified: parm/config/config.base.emc.dyn modified: ush/rocoto/setup_workflow_fcstonly.py modified: ush/rocoto/workflow_utils.py
Updated transfer list files from WCOSS2 ops. - In move to WCOSS2 the transfer*list files were moved into a new transfer folder under the upper-level parm folder. - The transfer*list files were updated to clean out unneeded paths and the beginning of the paths were updated from: `com/gfs/_ENVIR_` to: `_COMROOT_/gfs/_SHORTVER_` Refs: NOAA-EMC#419
In the workflow refactoring, the addition of postsnd to the task list when bufrsnd is true was inadvertently left out. It is now added back in.
On branch feature/gefs_v13_789_setupexp modified: parm/config/config.base.emc.dyn Refs: NOAA-EMC#789
Merge changes associated with g-w issue NOAA-EMC#521 from g-w branch feature/ufsda_gdasapp into develop. feature/ufsda_gdasapp contains g-w extensions for JEDI based atmospheric DA. Specifically, this PR adds the option to add JEDI based variational and/or ensemble DA jobs to replace GSI based DA jobs. The toggling on/off of JEDI_VAR and JEDI_ENS jobs is controlled via two new variables added to `config.base.emc.dyn` and `config.base.nco.static` ``` # DA engine export DO_JEDIVAR="NO" export DO_JEDIENS="NO" ``` When both variables are `NO`, the global workflow uses GSI based DA jobs. Thus, the PR does not alter the default behavior of the develop global workflow. When `DO_JEDIVAR=YES`, GSI jobs `anal` and `analdiag` are replaced by JEDI_VAR jobs `atmanalprep`, `atmanalrun`, and `atmanalpost`. When `DO_JEDIENS=YES`, GSI jobs `eobs`, `ediag`, and `eupd` are replaced by JEDI_ENS jobs `atmensanalprep`, `atmensanalrun`, and `atmensanalpost`. `checkout.sh`, `build_all.sh`, and `link_workflow.sh` are updated to clone, build, and install the GDASapp in the global workflow. Local directory `sorc/gdas.cd` contains the GDASApp superstructure plus the relevant components of JEDI needed to run GDASApp. Closes NOAA-EMC#521
On branch feature/gefs_v13_coupled Changes to be committed: modified: jobs/JGLOBAL_WAVE_INIT modified: jobs/rocoto/coupled_ic.sh modified: jobs/rocoto/waveinit.sh modified: parm/config/config.base.emc.dyn modified: parm/config/config.coupled_ic modified: parm/config/config.nsst modified: parm/config/config.wave modified: ush/forecast_postdet.sh modified: ush/forecast_predet.sh modified: workflow/applications.py modified: workflow/hosts/hera.yaml modified: workflow/rocoto/workflow_tasks.py modified: workflow/rocoto/workflow_xml.py
On branch feature/gefs_v13_coupled Changes to be committed: modified: workflow/rocoto/workflow_tasks.py
On branch feature/gefs_v13_coupled Changes to be committed: modified: jobs/rocoto/coupled_ic.sh modified: ush/forecast_postdet.sh
On branch feature/gefs_v13_coupled modified: ush/forecast_postdet.sh
…OAA-EMC#920) * WCOSS2 updates to gempak ush scripts - Add /gempak subfolder where needed in gempak ush scripts. - Remove unneeded commented out path settings from older iterations. * Removing older release notes - Cleaning out older GFS version release notes; includes current GFSv16.2.1 release notes, will commit GFSv16.3 release notes with implementation this fall. - Will then keep only the latest release notes moving forward. Refs: NOAA-EMC#419
The PR contains changes to optimize the DA clone, build, and link. Changes are made to `checkout.sh`, `build_all.sh`, and `link_workflow.sh` in the g-w `sorc/` directory. These changes are in g-w branch `feature/clone` Two arguments are added to `checkout.sh` to allow the user to specify which DA package to build the global workflow with. These options are - `-g`: clone from the [GSI](https://github.com/NOAA-EMC/GSI) repo and build the g-w for GSI-based DA - `-u`: clone from the [GDASApp](https://github.com/NOAA-EMC/GDASApp) repo and build the g-w for UFS-based DA If no option is specified, `checkout.sh` does not clone any DA and DA related repos. This is the default behavior of `checkout.sh`. (_DA related_ repos include [GLDAS](https://github.com/NOAA-EMC/GLDAS), [GSI-utils](https://github.com/NOAA-EMC/GSI-utils), and [GSI-Monitor](https://github.com/NOAA-EMC/GSI-Monitor).) `build_all.sh` is modified to detect which repos and have been cloned and to build accordingly. `link_workflow.sh` is modified to detect which directories are present and link/copy accordingly. Closes NOAA-EMC#930
On branch feature/gefs_v13_coupled modified: ush/forecast_postdet.sh
…AA-EMC#922) Removes code related to decommissioned HPC platforms WCOSS 1 (Dell & Cray) and Theia. Some references remain in scripts outside the global-workflow repo that are cloned as part of `checkout.sh`. Scripts from the `driver` directory that were hard-wired for one of the WCOSS1 platforms are also removed. Additionally, this commit also switches to using serial netCDF for resolutions C48. C96, C192. Running with parallel netCDF (on Hera) gave errors when testing at C96 for the deterministic forecast. If someone gives a very compelling reason to use parallel netCDF at these resolutions as default, I would be very interested in what they have to say. Closes NOAA-EMC#680
…OAA-EMC#933) Replaces the dependency of `gdasechgres` on `gdasefmn` with `gdasefcs01`. Presently, `gdasechgres` has 2 dependencies: - `gdasfcst` - deterministic forecast - `gdasefmn` - ensemble forecasts (all of them). The work done in `gdasechgres` actually depends only on the `mem001/atmos/gdas.tHHz.atmf006.nc`. This file is used as a template as well as obtaining `hgtsfc`. As such, there is no reason to depend on the entire ensemble of forecasts to be complete before `gdasechgres` can start.
This file was missed in the initial cleanup.
GLDAS requires the namelist definition for the topo directory to have the trailing slash. Fixes NOAA-EMC#1063
The ocean resolution for atmostphere C48 should by 5 deg, not 4 deg. Fixes NOAA-EMC#1054
…-EMC#1065) Added a $cyc subdirectory to the default file location for monitor data in rocoto. This ensures no file name collisions occur in the output monitor files. This change was also recently made to GFSv16.3.0 package @ 4335ef2. Additionally a problem with the definition of the previous cycle (m1) for the DA monitors was identified and corrected. This corrects the MinMon's output data. Fixes NOAA-EMC#1055
Updates the UFS version. This captures the conversion of UFS module from TCL to lua. A couple of the CICE namelist variables are no longer valid in this version, so they are removed. Due to memory limitations on Hera and the increased memory requirements of GOCART, the number of threads is increased there when running the forecast with aerosols. Also added a temporary block to delete any existing gocart output files. The ability to clobber files was deactivated a while ago and I got fed up with forecast jobs failing on retry.
Splits the existing rocoto cycle definitions up to offer better job control. This means that only the jobs that are due to run will appear in a cycle's job list from rocotostat/rocotoviewer. It also allows for the removal of some of the cycleexist dependencies that were there solely to prevent the job from running in the half cycle. A side effect of this change is that the half-cycle will be recognized as a completed cycle, fixing the bug with archive jobs starting in the fourth cycle (NOAA-EMC#1003). The gdas cycledef has been split into a `gdas_half` for the first half- cycle and `gdas` for the other GDAS cycles. Tasks that run during that first half-cycle therefore run on two cycledefs. For gfs, instead of slicing perpindicular to time, a new cycledef `gfs_cont` (continuity) was created in parallel to the existing gfs cycledef that omits the first cycle. This was done since only one job (`aerosol_init`) currently skips the first cycle, and this prevents the need to provide two cycledefs for every gfs task but one. Since some time math is now being done on sdate in workflow_xml.py, we now keep those as datetime objects and only convert to string when writing the cycledef strings. In order to access the pygw utilities in the workflow directory, a symlink is created in `workflow` pointing to the pygw location in `ush`. A better solution may be found in the future. Fixes NOAA-EMC#1003
No settings modifications were necessary for this update. This advance of the UFS version (finally) fixes the problem with missing PV and sigma levels.
This PR updates the gfs-utils version (af933d3) and ush script symlinks/paths. This will resolve issue NOAA-EMC#1059 until `finddate.sh` can be used via the `prod_util` module. * New gfs-utils hash: af933d3 * A number of ush scripts were moved into the new gfs-utils repo but not added to `link_workflow.sh`. This PR adds them to `link_workflow.sh`. * Several workflow scripts are updated to point to the gfs-utils ush symlinks under the top level `/ush` folder instead of the prior `/util/ush` folder. Fixes NOAA-EMC#1059
The recent update of the gfs_utils version in PR NOAA-EMC#1082 introduced some gfs_util that weren't accounted for in the original PR. gfs_utils changed the name of the script that determines the machine to `detect_machine.sh`, so scripts that call that script had to be updated. The variable name holding the machine name has also changed (from `$target` to `$MACHINE_ID`) and it may now include a compiler at the end, so changes were necessary to account for that. The WW3 build is changed completely to use the UFS modules to be consistent with the rest of UFS instead of maintaining separate modules in workflow that may use different module versions. Also reverts an inadvertent removal of execute permissions for `checkout.sh` Fixes NOAA-EMC#1086
The new detect_machine script in gfs-utils requires COMPILER be set. Also added the gfs-utils scripts to the git ignore list. Fixes NOAA-EMC#1090
* Add WCOSS2 BASE_CPLIC to config.coupled_ic * Remove errant ) from WCOSS2 hosts file for COMINsyn * Updates to config.fv3 from WCOSS2 testing * Add "is_exclusive" setting to config.resources * Add WCOSS2 to machine npe_node_max check in config.resources * Update resources in config.resources * Update build_ufs.sh to set S2SW as default app on WCOSS2 * Set hpssarch to NO by default on WCOSS2 Refs NOAA-EMC#419
Updates the GDASapp version to a newer version that builds correctly. The former version no longer builds because submodules point to develop branches that are no longer compatible. Moves module loads out of the j-jobs and into their appropriate place in the rocoto job. A new load module script is added to handle the different module set needed for UFSDA. Also temporarily turns off strict mode for the UFSDA jobs to avoid PS1 unbound error in conda. Fixes NOAA-EMC#1043 Fixes NOAA-EMC#1067
Adds module-setup calls and updates gfs-utils to reset modules when building. The gfs-utils update also fixes the build issue on Orion by updating the stack version to 1.2. Brings all build scripts into full shellcheck compliance. Also removes the workflow utils modulefiles since they are no longer needed after moving those programs to gfs-utils. Fixes NOAA-EMC#1093
Updates workflow to allow early cycle EnKF. Refs: NOAA-EMC#1021
…al assimilation (NOAA-EMC#1104) Adds the ability to run fv3jedi_var.x with the GSI static B.
Updates remaining shell scripts in sorc directory to pass shellcheck without any warnings. Refs: NOAA-EMC#397
UFS DA unified forward operator (UFO) validation uses `crtm/2.3.0` for radiances. UFS DA scripts which exercise UFO for radiance assimilation should use the same CRTM coefficients. UFS DA scripts `exgdas_global_atmos_analysis_run.sh` and `exgdas_global_atmos_ensanal_run.sh` currently use CRTM coefficients from `crtm/2.3.0_jedi`. This is not correct. The CRTM coefficient path for the two UFS DA analysis scripts in question has been updated in [`feature/ufsda_crtm`.](https://github.com/NOAA-EMC/global-workflow/tree/feature/ufsda_crtm) Fixes NOAA-EMC#1110
* Update multi-prog in HERA.env and ORION.env * Update launcher commands in HERA.env and ORION.env * Adjust C96 & C48 eobs resources in config.resources Refs NOAA-EMC#1060
…OAA-EMC#1122) Allow running ensemble forecasts in the early cycle (`CDUMP=gfs`) to an arbitrary forecast length controlled by `FHMAX_ENKF_GFS`
* Correct wave_mpmd setting in JGLOBAL_WAVE_INIT * Update Jet and S4 env files for mpmd_opt * Rename mpmd to mpmd_opt in WCOSS2 env file
New hpc-stack location on Orion managed by Hang Lei: /apps/contrib/NCEP/hpc-stack/libs/hpc-stack/modulefiles/stack Refs NOAA-EMC#1119
The logical test used to set gsi namelist variable `nhr_obsbin` was not consistent with other checks in the script which use variable `l4densvar`. This relocates the setting for `nhr_obsbin` to be consistent with other tests using `l4densvar`, which will allow the appropriate error messages to be output by the executable. Fixes NOAA-EMC#1123 Fixes [NCO bugzilla NOAA-EMC#1196](http://www2.spa.ncep.noaa.gov/bugzilla/show_bug.cgi?id=1196)
C48 forecasts currently fail on WCOSS2 when running with two threads. Two threads are likely not really needed, so the forecast job is reduced to single-threaded on all platforms, at least until the problem can be corrected. Also moves the block capping the write block size to the node size outside the resolution-specific block, since that is needed for all resolutions. Fixes NOAA-EMC#1129
Updates the archive jobs to the standard j-job/exscript format. Fixes: NOAA-EMC#1051 Refs: NOAA-EMC#720
…1133) `ush/forecast_predet.sh` contains an unnecessary sCDATE assignment. Therefore this line is safe to be removed. Fixes NOAA-EMC#956
Adds additional settings for ensemble forecast group size and number of members since gfs CDUMP (GEFS) will have different values than GDAS. NMEM_EFCS sets the number of members for GEFS (default: 30) NMEM_EFCSGRP_GFS sets the number of members per job for GEFS (default: 1) Fixes NOAA-EMC#1117
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Checklist