Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter data by observation time upon loading #2974

Open
adybbroe opened this issue Nov 8, 2024 · 1 comment
Open

Filter data by observation time upon loading #2974

adybbroe opened this issue Nov 8, 2024 · 1 comment
Assignees
Labels
component:readers enhancement code enhancements, features, improvements

Comments

@adybbroe
Copy link
Contributor

adybbroe commented Nov 8, 2024

Feature Request

Is your feature request related to a problem? Please describe.

So, this issue came up when working with the early Arctic Weather Satellite (AWS1) data. AWS1 was launched Aug 16, 2024. Data are not yet released, but I got a few orbits for the EUM conf 2024. But the issue applies to all data and in particular to all global polar orbiting satellites - for instance the NOAA GAC AVHRR data, where files may span an entire (or a bit more) orbit.

Suppose you want to look at the area where the orbit starts and ends, like in the case of the image below:

aws_mw183_humidity_a_20240913_1912_nsper_swe

The start of this orbit is 19:12 UTC, but the end is ~95 minutes later. So if you want a snapshot of the data over an area of interest you do not want to blend the start and the end of the orbit, either you want just the beginning, or just then end!

So, simply reading all data and remap to the area, as in the below examples doesn't really work:

aws_mw183_humidity_a_20240913_2225_arctic_south
aws_mw183_humidity_a_20240913_2048_arctic_south

Describe the solution you'd like

So, what I would have liked in this case, is an option to read only data applicable to a certain time window, so for example:

scn = Scene(filenames=FILENAMES, reader='aws_l1b_nc')
composites = ['mw183_humidity']
scn.load(composites, start_time=datetime(2024, 9, 13, 19, 12), end_time=datetime(2024, 9, 13, 19, 22))

Describe any changes to existing user workflow
Don't believe there should be any changes needed.

Additional context
I know that we have usually the opportunity to do this filtering by choosing only the (granule) files that are relevant for what we want to do. That is fine when the data are segmented in a few minutes of observation data, but doesn't work when data comes as one file with an entire orbit.

@adybbroe adybbroe self-assigned this Nov 8, 2024
@adybbroe adybbroe added enhancement code enhancements, features, improvements component:readers labels Nov 8, 2024
@djhoese
Copy link
Member

djhoese commented Nov 11, 2024

Theoretically this should be possible as part of xarray's "coords" interface. That is, if you know the times you want and the reader adds a time coordinate to data_arr.coords["time"] or something similar, then you should be able to use xarray's slicing/isel/loc or whatever it is called to slice out the portion you want based on that coordinate variable. Some complications:

  1. Most readers don't add this time coordinate.
  2. I don't think we've established a consistent naming for the variable regardless.
  3. Because we store geolocation information in a non-xarray-standard .attrs["area"], you'd have to get the slicing information (somehow) for the data and then apply it to the SwathDefinition separately. So slice the data DataArray, then slice the SwathDefinition, and finally re-assign that swath definition to the new DataArray's .attrs["area"]. I don't think the Satpy Scene currently allows high-level access to .loc for the all DataArrays in the Scene as we have multiple resolutions in one Scene container and doing that type of indexing would be difficult.

Xarray indexing and selecting:

https://docs.xarray.dev/en/stable/user-guide/indexing.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:readers enhancement code enhancements, features, improvements
Projects
None yet
Development

No branches or pull requests

2 participants