Optimized ww3_outp for the netcdf point output #1365

AliS-Noaa · 2025-02-07T20:16:13Z

Pull Request Summary

Optimized the ww3_outp for netcdf point output

Description

Following the #1355 and @dkokron work, the ww3_outp in improved to read per time step NetCDF output and create different point output.

Issue(s) addressed

Addressing issue #876

Commit Message

Optimized ww3_outp for the netcdf point output

Check list

Branch is up to date with the authoritative repository (NOAA-EMC) develop branch.
Checked the checklist for a developer submitting to develop.
If a version number update is required, checked the updating version number checklist.
If a new feature was added, a regression test for testing the new feature is added.

Testing

How were these changes tested? Full regression tests ran, please see the matrix.
Are the changes covered by regression tests? (If not, why? Do new tests need to be added?) regression test, ww3_ufs1.1 is modified to test the new ww3_outp implementation for the global-workflow.
Have the matrix regression tests been run (if yes, please note HPC and compiler)?
Please indicate the expected changes in the regression test output, (Note the list of known non-identical tests.) For ww3_ufs1.1 regression test, 3 different ww3_outp.inp are added to produce .spec, .bull, and .cbull point outputs similar to global-workflow.
Please provide the summary output of matrix.comp (matrix.Diff.txt, matrixCompFull.txt and matrixCompSummary.txt):

********************* non-identical cases ***************************

mww3_test_03/./work_PR2_UQ_MPI_d2 (16 files differ)
mww3_test_03/./work_PR2_UNO_MPI_d2 (9 files differ)
mww3_test_03/./work_PR2_UQ_MPI_e (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2_c (17 files differ)
mww3_test_03/./work_PR3_UQ_MPI_d2 (15 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2_c (9 files differ)
mww3_test_03/./work_PR1_MPI_d2 (9 files differ)
mww3_test_03/./work_PR3_UNO_MPI_d2 (16 files differ)
mww3_test_03/./work_PR1_MPI_e (1 files differ)
mww3_test_03/./work_PR3_UNO_MPI_e_c (1 files differ)
mww3_test_03/./work_PR3_UQ_MPI_e (1 files differ)
mww3_test_03/./work_PR2_UNO_MPI_e (1 files differ)
mww3_test_09/./work_MPI_ASCII (0 files differ)
ww3_tp2.10/./work_MPI_OMPH (7 files differ)
ww3_tp2.16/./work_MPI_OMPH (4 files differ)
ww3_tp2.6/./work_ST4_ASCII (0 files differ)
ww3_ufs1.1/./work_unstr_b (1 files differ)
ww3_ufs1.1/./work_unstr_a (1 files differ)
ww3_ufs1.1/./work_unstr_c (1 files differ)
ww3_ufs1.3/./work_a (3 files differ)
matrixCompFull.txt
matrixCompSummary.txt
matrixDiff.txt

… current capabilities

…nt output

…t naming accordingly

…ore clean up

JessicaMeixner-NOAA

Thanks for all of your efforts for bringing theses changes into the develop branch from production v16 branch.

I've left a few comments in the code. We either need to create an issue to add the two new variables for reading the nml file or we need to add that to the code here now. Additionally as mentioned in the code, we need to describe this new variable in the example inp files. Lastly, a regression test for this new feature needs to be added. I will help with that and make a PR to your branch by Monday.

model/src/w3iopomd.F90

model/src/ww3_outp.F90

model/src/w3iopomd.F90

JessicaMeixner-NOAA · 2025-02-07T20:32:03Z

model/src/ww3_outp.F90

@@ -2674,7 +2839,7 @@ SUBROUTINE W3EXPO
         '    Hs   Tp  dir |'/              &
         ' |  hour |  (m)  - - |    (m)  (s) (d) |',              &
         '    (m)  (s) (d) |',              &
-         '    (m)  (s) (d) |',              &
+         '    (m)  (s) (d) |',              &    


remove extra space at end of line here.

JessicaMeixner-NOAA · 2025-02-10T13:18:06Z

model/src/w3iopomd.F90

+    ELSE
+      filename = FNMPRE(:LEN_TRIM(FNMPRE))//'out_pnt.'//FILEXT(:LEN_TRIM(FILEXT))//'.nc'
+    END IF
+


@AliS-Noaa - Particularly because the code will fail without a very informative error, before trying to open the file, adding in a check to see if the file exists and if it does not, providing an informative error statement before existing would be helpful. Please let me know if you need any assistance adding this.

…g the prefix to W3IOPON/W3IOPON_READ;some more cleaning

model/src/w3iopomd.F90

model/src/ww3_outp.F90

JessicaMeixner-NOAA · 2025-02-10T16:33:16Z

regtests/ww3_ufs1.1/input_unstr/ww3_outp.inp

 $
-  20210401 000000  3600.  100
+  20210401 000000  3600.  25  1 


Please check the permissions changing on this file. Also, please add a prefix here so that we can test that feature.

Can we add a second ww3_outp to do both bull/cbull and spec files to test both features?

JessicaMeixner-NOAA · 2025-02-10T16:34:15Z

regtests/ww3_ufs1.1/input_unstr/ww3_outp.inp

 $   flags for spectrum and source terms (C*21, 3I, 6L)
-$ - Bin frequencies in Hz for all bins.
+$ - Bin frequenies in Hz for all bins.


spelling frequenies- > frequencies

JessicaMeixner-NOAA · 2025-02-10T16:34:27Z

regtests/ww3_ufs1.1/input_unstr/ww3_outp.inp

 $
 $ The transfer file contains records with the following contents.
 $
-$ - File ID in quotes, number of frequencies, directions and points,
+$ - File ID in quotes, nubmer of frequencies, directions and points,


spelling nubmer -> number

model/src/ww3_outp.F90

…ting spelling

JessicaMeixner-NOAA · 2025-02-11T15:03:45Z

model/src/w3iopomd.F90

-
+      END IF      
+
+      !Variables read based on time (IPASS):      


!Variables read based on time (IPASS):
->
!Variables read based on time (itime):

JessicaMeixner-NOAA · 2025-02-11T15:03:58Z

model/src/w3iopomd.F90

-    END IF 
-
+    END IF
+    


remove extra space here

@JessicaMeixner-NOAA This is done and its ready for review.

JessicaMeixner-NOAA · 2025-02-12T13:34:41Z

@AliS-Noaa - Thanks for the last update you made. My regression tests before your update were as expected. We'll hold off on a final round of testing until we've finished testing in g-w. In the meantime, please don't forget to fill out the testing portion of the PR when you have finalized things.

AliS-Noaa · 2025-02-14T19:04:20Z

@JessicaMeixner-NOAA I tested this with updated G-W and it is ready for the review.

JessicaMeixner-NOAA · 2025-02-14T19:09:08Z

Thank you @AliS-Noaa - Starting review and testing now!! Thanks for all of this! It's really appreciated.

JessicaMeixner-NOAA · 2025-02-19T13:29:55Z

@AliS-Noaa - In operations when I un-tar the spec files or other tar-balls, I have:
gfswave.NAME.spec or gadswave.NAME.spec
however, I do not get this in this branch.

While we can leave off the prefix for the YYYYMMDD.HHMMSS.out_pnt.ww3 file which will simplify for example the regression test, I think this needs to be re-added to have correctly named output to match what we are obtaining in operations. I think it'll be more efficient to have these files output with the correct name versus having to rename them all before tarring.

AliS-Noaa · 2025-02-19T15:22:03Z

@AliS-Noaa - In operations when I un-tar the spec files or other tar-balls, I have: gfswave.NAME.spec or gadswave.NAME.spec however, I do not get this in this branch.

While we can leave off the prefix for the YYYYMMDD.HHMMSS.out_pnt.ww3 file which will simplify for example the regression test, I think this needs to be re-added to have correctly named output to match what we are obtaining in operations. I think it'll be more efficient to have these files output with the correct name versus having to rename them all before tarring.

@JessicaMeixner-NOAA so do you need to bring back the optional prefix to the code and also the g-w workflow script? I can work on it now.

JessicaMeixner-NOAA · 2025-02-19T16:26:57Z

@AliS-Noaa - In operations when I un-tar the spec files or other tar-balls, I have: gfswave.NAME.spec or gadswave.NAME.spec however, I do not get this in this branch.
While we can leave off the prefix for the YYYYMMDD.HHMMSS.out_pnt.ww3 file which will simplify for example the regression test, I think this needs to be re-added to have correctly named output to match what we are obtaining in operations. I think it'll be more efficient to have these files output with the correct name versus having to rename them all before tarring.

@JessicaMeixner-NOAA so do you need to bring back the optional prefix to the code and also the g-w workflow script? I can work on it now.

Yes, thank you! Please let me know when you're ready for re-review

JessicaMeixner-NOAA · 2025-02-24T14:15:34Z

@AliS-Noaa - I'm working to determine why I'm getting tiny differences in the output between a global-workflow case using your branch merged to dev-ufs-weather-model versus a "baseline" case. I'll post more information when I have it. The WW3 regression tests themselves pass. I hope to have more information this afternoon or tomorrow morning.

JessicaMeixner-NOAA · 2025-02-26T14:55:05Z

@AliS-Noaa and I have been chatting offline. The remaining issue here is when running in the full workflow, we expect to get the same answers however everything is matching except for the partition number in the bull file. This is being investigated before this PR is merged.

AliS-Noaa added 7 commits January 30, 2025 18:40

reading per time netcdf in ww3_outp, it needs more work to merge with…

360d738

… current capabilities

Reading per time step netcdf point output data and processing the poi…

48cc032

…nt output

Adding proper optional dynpnt and prefix inline and modifing the inpu…

ebb7fe6

…t naming accordingly

adding BIN2NC flag for dynpnt=1 for general data

220e998

adding error for binary per-time-step point output

82b4d62

adding more concise per-time-step w3iopon_read and savepoint changes

2aff3e4

removed the savepoint modification; corrected the indentations; and m…

dbc2a96

…ore clean up

AliS-Noaa marked this pull request as draft February 7, 2025 20:33

JessicaMeixner-NOAA reviewed Feb 7, 2025

View reviewed changes

JessicaMeixner-NOAA reviewed Feb 10, 2025

View reviewed changes

Modifying ww3_ufs1.1 regtest;even more concise W3IOPON_READ and addin…

3918202

…g the prefix to W3IOPON/W3IOPON_READ;some more cleaning

JessicaMeixner-NOAA reviewed Feb 10, 2025

View reviewed changes

AliS-Noaa added 3 commits February 10, 2025 20:02

fixing minor issues on ww3_outp.in

b14b4db

Merge remote-tracking branch 'EMC/develop' into para_outp

e4dd760

taking out prefix; adding different ww3_outp.in to ww3_ufs1.1; correc…

9bc9983

…ting spelling

JessicaMeixner-NOAA self-requested a review February 11, 2025 14:10

JessicaMeixner-NOAA reviewed Feb 11, 2025

View reviewed changes

very minor wording changes

16baac7

AliS-Noaa marked this pull request as ready for review February 11, 2025 15:37

adding the dynpnt=0 condition for time dependent file check for ITYPE=0

67d032d

Merge remote-tracking branch 'EMC/develop' into para_outp

ac6649e

AliS-Noaa and others added 2 commits February 19, 2025 11:55

Merge branch 'NOAA-EMC:develop' into para_outp

2c8a1ff

retreiving the point output prefix

aaf7fa2

mingchen-NOAA mentioned this pull request Feb 21, 2025

Write log files for all output to indicate it's finished #1378

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimized ww3_outp for the netcdf point output #1365

Optimized ww3_outp for the netcdf point output #1365

AliS-Noaa commented Feb 7, 2025 •

edited

Loading

JessicaMeixner-NOAA left a comment

JessicaMeixner-NOAA Feb 7, 2025

JessicaMeixner-NOAA Feb 10, 2025

JessicaMeixner-NOAA Feb 10, 2025

JessicaMeixner-NOAA Feb 10, 2025

JessicaMeixner-NOAA Feb 10, 2025

JessicaMeixner-NOAA Feb 10, 2025

JessicaMeixner-NOAA Feb 11, 2025

JessicaMeixner-NOAA Feb 11, 2025

AliS-Noaa Feb 11, 2025

JessicaMeixner-NOAA commented Feb 12, 2025

AliS-Noaa commented Feb 14, 2025

JessicaMeixner-NOAA commented Feb 14, 2025

JessicaMeixner-NOAA commented Feb 19, 2025

AliS-Noaa commented Feb 19, 2025

JessicaMeixner-NOAA commented Feb 19, 2025

JessicaMeixner-NOAA commented Feb 24, 2025

JessicaMeixner-NOAA commented Feb 26, 2025

Optimized ww3_outp for the netcdf point output #1365

Are you sure you want to change the base?

Optimized ww3_outp for the netcdf point output #1365

Conversation

AliS-Noaa commented Feb 7, 2025 • edited Loading

Pull Request Summary

Description

Issue(s) addressed

Commit Message

Check list

Testing

JessicaMeixner-NOAA left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JessicaMeixner-NOAA commented Feb 12, 2025

AliS-Noaa commented Feb 14, 2025

JessicaMeixner-NOAA commented Feb 14, 2025

JessicaMeixner-NOAA commented Feb 19, 2025

AliS-Noaa commented Feb 19, 2025

JessicaMeixner-NOAA commented Feb 19, 2025

JessicaMeixner-NOAA commented Feb 24, 2025

JessicaMeixner-NOAA commented Feb 26, 2025

AliS-Noaa commented Feb 7, 2025 •

edited

Loading