Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIT L1B - validate standard rates data #1351

Open
wants to merge 6 commits into
base: dev
Choose a base branch
from

Conversation

vmartinez-cu
Copy link
Contributor

@vmartinez-cu vmartinez-cu commented Feb 10, 2025

This PR adds a test to validate the L1B standard rates data. As part of this work, the helper test functions file l1a_validation.py was updated to apply to L1B validation as well.

Closes #960

Updated Files

  • l1a_validation.py
    • Renamed file to l1_validation.py
    • Updated file to support both L1A and L1B validation since some of the validation data reformatting work is common for both.
    • Added constants to the top of the file for readability
    • Added a helper function to reformat/prepare standard rates validation data for comparison with dataset
    • Split out consolidation of sector rate fields in the validation data into it's own function
    • Added type hints
    • Cleaned up documentation
  • test_hit_l1b.py
    • added a test to validate standard rates product and use helper functions to carry this out
  • test_hit_l1a.py
    • minor updates to reflect changes to the helper functions file

-Add test to validate l1b standard rates product
-Rename l1a_validation.py helper file to l1_validation.py to support
 both l1a and l1b validation tests
-Update functions in helper file and add function to prepare
 standard rates validation data for comparison with product
-Add TODOs
@vmartinez-cu vmartinez-cu self-assigned this Feb 10, 2025
@vmartinez-cu vmartinez-cu added Ins: HIT Related to the HIT instrument Level: L1 Level 1 processing Req: Data Product Validation labels Feb 10, 2025
@vmartinez-cu vmartinez-cu requested review from sdhoyt, a team, laspsandoval and torimarbois and removed request for a team February 10, 2025 17:45
@vmartinez-cu
Copy link
Contributor Author

The checks are failing because the validation file is barely too large to add to the repo. Waiting for an updated file from HIT

@vmartinez-cu vmartinez-cu requested review from a team, nkerman, subagonsouth and tech3371 and removed request for laspsandoval, torimarbois and a team February 10, 2025 18:07
@greglucas
Copy link
Collaborator

greglucas commented Feb 10, 2025

The checks are failing because the validation file is barely too large to add to the repo. Waiting for an updated file from HIT

@vmartinez-cu, this might be a great use-case for the remote file updates @bourque did in #1336

@vmartinez-cu
Copy link
Contributor Author

vmartinez-cu commented Feb 10, 2025

The checks are failing because the validation file is barely too large to add to the repo. Waiting for an updated file from HIT

@vmartinez-cu, this might be a great use-case for the remote file updates @bourque did in #1336

Yes, I was thinking that after Matthew shared his work during the tag up today. Though I think the problem is that all the numbers in the file have way more decimal places than defined for this product which results in a large file. When I reduce them to just one decimal place, the file size goes down by half. I'm following up with HIT on this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ins: HIT Related to the HIT instrument Level: L1 Level 1 processing Req: Data Product Validation
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Validate HIT L1B Standard Rates Data
2 participants