[1pt] PR: Rating curves adjustments with ml-bathymetry data and optimized mannN

[hhs732]

This PR focuses on adjusting rating curves by using bathymetric data and optimized channel roughness values. The bathymetry data includes eHydro surveys and AI-based datasets created for all NWM streams. New manning roughness values were developed for each feature-id using a differential evolution objective function (OF). The OF minimizes the number of the false_positives and false_negatives cells in our flood inundation maps where we have test cases across the CONUS.

Even though the Python scripts of roughness manning number optimization were not included in this branch, optimized roughness values can be found here: /fim-data/inputs/rating_curve/variable_roughness/mannings_optz_fe_clusters_so3.csv. Detailed python scripts also can be found here: /fim-data/outputs/heidi-mannN-optimization/projects/bathy_mannN_projects/dev-bathymetric-adjustment-mannN-optz/.

Changes

• src/bathymetric-adjustment.py: correct_rating_for_ai_based_bathymetry function was added to the script. This function processes AI-based bathymetry data and adjusts rating curves using this data. Also apply_src_adjustment_for_bathymetry function was added to prioritize USACE eHydro over AI-based bathymetry dataset. The multi-processing function multi_process_hucs was updated based on the latest code.

• src/bash_variables.env: New variables and their paths were added. Also, a new input file with the nwm feature_ids and optimized channel roughness and overbank roughness attributes was created and stored here:
  /fim-data/inputs/rating_curve/variable_roughness/mannings_optz_fe_clusters_so3.csv
  The locations of these files were also added to the bash_variables.env.

Here is a list of new/updated input files:

1. /fim-data/inputs/rating_curve/variable_roughness/mannings_optz_fe_clusters_so3.csv
   This CSV file contains the new optimized roughness values. It will replace this file:
   vmann_input_file=${inputsDir}/rating_curve/variable_roughness/mannings_global_nwm3.csv

2. bathy_file_aibased=${inputsDir}/bathymetry/ml_outputs_v1.01.parquet
   This file contains the ml-bathymetry and manning roughness values data.

3. bathy_file_ehydro=${inputsDir}/bathymetry/final_bathymetry_ehydro.gpkg
   We already had this file, the name of the variable has changed from bathymetry_file to bathy_file_ehydro, and it was updated.

• fim_post_processing.sh: New arguments were added.

Testing

This PR has been tested on 11 HUC8s around the Illinois River and Ohio River.

Deployment Plan (For developer use)

How do the changes affect the product?

• Code only?
• If applicable, has a deployment plan be created with the deployment person/team?
• Require new or adjusted data inputs? Does it have start, end and duration code (in UTC)?
• If new or updated data sets, has the FIM code been updated and tested with the new/adjusted data (subset is fine, but must be a subset of the new data)?
• Require new pre-clip set?
• Has new or updated python packages?

Issuer Checklist (For developer use)

You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.

• Informative and human-readable title, using the format: [_pt] PR: <description>
• Links are provided if this PR resolves an issue, or depends on another other PR
• If submitting a PR to the dev branch (the default branch), you have a descriptive Feature Branch name using the format: dev-<description-of-change> (e.g. dev-revise-levee-masking)
• Changes are limited to a single goal (no scope creep)
• The feature branch you're submitting as a PR is up to date (merged) with the latest dev branch
• pre-commit hooks were run locally
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future todos are captured in comments
• CHANGELOG updated with template version number, e.g. 4.x.x.x
• Add yourself as an assignee in the PR as well as the FIM Technical Lead

Merge Checklist (For Technical Lead use only)

• Update CHANGELOG with latest version number and merge date
• Update the Citation.cff file to reflect the latest version number in the CHANGELOG
• If applicable, update README with major alterations

[RobHanna-NOAA]

Ah. Looks like we have new inputs (and/or) some updated ones, correct?

[hhs732]

Hi Rob, yes that is correct.

…

On Fri, Nov 8, 2024 at 1:56 PM Rob Hanna - NOAA ***@***.***> wrote: Ah. Looks like we have new inputs (and/or) some updated ones, correct? — Reply to this email directly, view it on GitHub <#1340 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFRLX2UVV2DFIDLD565RASDZ7UXP5AVCNFSM6AAAAABROJK2XCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRVHAYTAOJSGM> . You are receiving this because you were assigned.Message ID: ***@***.***>

[RobHanna-NOAA]

ok. awesome. Can you make a list of which files/folders so I know where to get them and get them moved around. Is it ok for me to start copying them around now? or not quite yet?

[hhs732]

Hi Rob,
There are three input files that I added/changed:

1. /fim-data/inputs/rating_curve/variable_roughness/mannings_optz_fe_clusters_so3.csv
   This CSV file contains the new optimized roughness values. It will replace this file:
   vmann_input_file=${inputsDir}/rating_curve/variable_roughness/mannings_global_nwm3.csv

2. bathy_file_aibased=${inputsDir}/bathymetry/ml_outputs_v1.01.parquet
   This file contains the ml-bathymetry and manning roughness values data.

3. bathy_file_ehydro=${inputsDir}/bathymetry/final_bathymetry_ehydro.gpkg
   We already had this file, the name of the variable has changed from bathymetry_file to bathy_file_ehydro, and it was updated.

[mluck]

The new files have been uploaded to all of the environments.