A Novel Community Streamflow Evaluation System (CSES) to evaluate hydrological model performance using a standardized NHDPlus data model.
CSES evaluates modeled streamflow to a repository of over 5,000 in situ USGS monitoring sites, with interactive visualizations supporting an in-depth analysis.
Given the launch of the Cooperative Institute for Research to Operations in Hydrology (CIROH) in April, 2020, CIROH scientists from 28 different academic, government, and private are working to improve the understanding of hydrologic processes, operational hydrologic forecasting techniques and workflows, community water modeling, translation of forecasts to actionable products, and use of water predictions in decision making. National-scale streamflow modeling remains a modern challenge, as changes in the underlying hydrology from land use and land cover (LULC) change, anthropogentic streamflow modification, and general process components (reach length, hydrogeophysical processes, precipitation, temperature, etc) greatly influence hydrological modeling. To benchmark model performance, characterize improvements in hydrological modeling formuations, and generate reproducible science, the team at the Alabama Water Institute (AWI) developed Community Streamflow Evaluation System (CSES) to originally characterize the water supply forecasting skill of the National Water Model v2.1 (NWM v3.0 coming soon!) in the Great Salt Lake Basin. The tool quickly scaled and with the support of the Earth Science Information Partners (ESIP), provided an opportunity to turn the tool into a novel evaluation platform for CONUS-wide applications. The package contains three key methods for evaluation: state-based LULC, HUC level analysis, and USGS station-based analysis. Below is a description of each method and application. Designed to assess NWM version 2.1 retrospective performance, by using the exemplified data model the tool can evaluate other model predictions, with the motivation to improve regionally dominant hydrological modeling skill. By using CSES, researchers can identify locations where a model may benefit from further training/calibration/parameterization or a need for new model processes/features (e.g., integration of reservoir release operations) to ultimately create new post-processing methods and/or hydrological modeling formulations to improve streamflow prediction capabilities with respect to modeling needs (e.g., stormflow, supply, emergency management, flooding, etc).
Each streamflow evaluation method requires similar inputs, including a start date, end date, and model. There are currently three different evaluation classes, each providing the user with a unique method for evaluating streamflow modeling performance:
- Class Eval_State(): Modeled Streamflow Evaluation by StreamStats
- Class HUC_Eval(): Modeled Streamflow Evaluation by Hydrologic Unit Code (HUC)
- Class Reach_Eval(): NHD - USGS Streamflow Evaluation
For all examples, the predictions are from the NWM v2.1 retrospective. Please see the Examples folder for more information on applying each specific class.
To determine how LULC affects the predictive performance of streamflow models, Community Streamflow Evaluation System (CSES) uses StreamStats to categorize the watershed upstream of each USGS monitoring site by watershed characteristics. Please see the** State Land Use - Land Cover Evaluation.md** readme to use the tool.
Running the Community Streamflow Evaluation System LULC_Eval class loads, processes, and visualizes model performance for the state, category, and size of interest
By clicking on a marker a popup of the modeled vs. observed performance at the inputted temporal frequency will appear
The Community Streamflow Evaluation System supports an interactive engagement with model results
The HUC_Eval class allows the user to evaluate modeled streamflow with observed in situ NWIS monitoring sites for watershed(s) of interest. The user can input multiple watersheds (e.g., Great Salt Lake: ['1601', '1602']) The user must enter a start date, end date, watersheds, and model to compare (NWM v2.1 is set up). NWM retrospective data spans from 1980 - 2020, and USGS/NWIS data is location-dependent.
The HUC_Eval class loads all USGS and colocated modeled NHD reaches for evaluation. Color coding of the markers allows for quick identification of poor and well-performing reaches and clicking on the markers will put up a modeled vs. observed graph at the desired temporal resolution.
Similar to the State_Eval class, the HUC_Eval class supports a more in-depth graphical analysis of the modeled vs. observed using the holoviews package
The Reach_Eval class allows the user to evaluate modeled streamflow with selected NWIS monitoring sites of interest. The user can input multiple USGS sites (e.g., ['02378780', '02339495', '02342500']) Similar to the other classes, enter a start date, end date, and model to compare (NWM v2.1 is set up). NWM retrospective data spans from 1980 to 2020, and USGS/NWIS data is location-dependent
The Reach_Eval class quickly loads and maps the selected USGS streamflow monitoring locations, along with the colocated model predictions of NHD reaches. The color code of the marker indicates the model performance at the respective USGS monitoring station site, and by clicking on a marker, the interactive map produces a graph at the desired temporal resolution.
Similar to the State_Eval and HUC_Eval classes, the Reach_Eval class supports a more in-depth graphical analysis of the modeled vs. observed using the holoviews package
Community Streamflow Evaluation System (CSES) leverages USGS/NWIS observations from 1980-2020 and colocated and while all data is publically available through the respective agencies, we found the download time to be preventative for a timely model evaluation. The Alabama Water Institute at the University of Alabama hosts NWM v2.1 retrospective for all colocated USGS monitoring stations at a daily temporal resolution and provides the data free of charge via access to Amazon AWS S3 cloud storage. CSES can quickly access observed and predicted data supporting a fast and repeatable tool for evaluating modeled streamflow performance.
Python: Version 3.9.12.
The included requirements.txt file should set up the correct Community Streamflow Evaluation System (CSES) environment. To use CSES, create a virtual Python environment and run the requirement.txt file to ensure all package versions are correct. To get started, click the Here
Please reach out the AWI team so we can put your model results in our AWS S3 database. We currently have a simple data model and are looking into more computationally efficient methods to expedite interactivity and the overall hydrological model evaluation experience.
Reach out to the AWI team and we can identify meaningful areas to grow Community Streamflow Evaluation System (CSES). Community is in the name for a reason and we intend to integrate more hydrological modeling components into the tool kit as funding allows. Specific areas of contribution include snow, atmospheric forcings, and other model outputs. Let's advance the community modeling paradigm together!
Funding to support this research was awarded to the Cooperative Institute for Research to Operations in Hydrology (CIROH) through the NOAA Cooperative Agreement with The University of Alabama (NA22NWS4320003). This work is based on CIROH's Community Streamflow Evaluation System (CSES) provided by the ESIP Lab with support from the National Aeronautics and Space Administration (NASA), National Oceanic and Atmospheric Administration (NOAA) and the United States Geologic Survey (USGS).