Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

README's Operational dataset example does not produce valid urls #61

Open
jarrodcolburn opened this issue Aug 23, 2024 · 2 comments
Open

Comments

@jarrodcolburn
Copy link

Running the example in the ReadMe's Operational dataset

nwmurl/README.md

Lines 86 to 113 in 553cab7

1. For Operational dataset:
```
import nwmurl
start_date = "202201120000"
end_date = "202201130000"
fcst_cycle = [0,8]
lead_time = [1,18]
varinput = 1
geoinput = 1
runinput = 1
urlbaseinput = 2
meminput = 1
write_to_file = False
file_list = nwmurl.generate_urls_operational(
start_date,
end_date,
fcst_cycle,
lead_time,
varinput,
geoinput,
runinput,
urlbaseinput,
meminput,
write_to_file
)
```

results the the following list of files

https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/nwm.20220112/short_range/nwm.t00z.short_range.channel_rt.f001.conus.nc
https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/nwm.20220112/short_range/nwm.t00z.short_range.channel_rt.f018.conus.nc
https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/nwm.20220112/short_range/nwm.t08z.short_range.channel_rt.f001.conus.nc
https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/nwm.20220112/short_range/nwm.t08z.short_range.channel_rt.f018.conus.nc
https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/nwm.20220113/short_range/nwm.t00z.short_range.channel_rt.f001.conus.nc
https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/nwm.20220113/short_range/nwm.t00z.short_range.channel_rt.f018.conus.nc
https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/nwm.20220113/short_range/nwm.t08z.short_range.channel_rt.f001.conus.nc
https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/nwm.20220113/short_range/nwm.t08z.short_range.channel_rt.f018.conus.nc

However, these^^ links don't seem to have valid urls.


Looking at the like the structure, https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/ , it appears to have changed. Such that the Date is not a folder, but rather prefixed the filename.

So currently result of a URL generated (from example above)....

https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/nwm.20220112/short_range/nwm.t00z.short_range.channel_rt.f001.conus.nc

Should probably produce (expected result)...

https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/short_range/nwm.20220112.t00z.short_range.channel_rt.f001.conus.nc

or to compare only the path after https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/post-processed/WMS/

nwm.20220112/short_range/nwm.t00z.short_range.channel_rt.f001.conus.nc

vs. (Date moved ---->)

short_range/nwm.20220112.t00z.short_range.channel_rt.f001.conus.nc

@jameshalgren
Copy link
Member

@jarrodcolburn
Way too long responding here. Good bug report. Working on a fix.

ping @karnesh, @JordanLaserGit re #62

@jameshalgren
Copy link
Member

jameshalgren commented Oct 30, 2024

I don't think we have ever had proper resolution of the Post-Processed URLs.
That needs to be fixed per #10.

And, at some point, there was probably a valid option in the url_base_option 2, but that seems to have moved and we didn't notice the readme getting out of sync. @karnesh can you help here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants