Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Couldn't find this package on pypi at advertised URL, install from source? #2

Open
rsignell opened this issue Aug 14, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@rsignell
Copy link

rsignell commented Aug 14, 2024

I read with interest the kerchunk feature of xstac:
https://github.com/stac-utils/xstac?tab=readme-ov-file#kerchunk-support
but when I looked for how to enable the example, I could not find the referenced package on pypi: https://pypi.org/project/stactools-noaa-nwm/

I tried searching pypi for "nwm" and didn't see any likely looking packages either.

Do we need to install from source, or is this package deprecated in favor of some other package?

@rsignell rsignell added the bug Something isn't working label Aug 14, 2024
@TomAugspurger
Copy link
Member

The PyPI link probably came from the repo template. #1 has the initial prototype for this package, but it wasn't pushed through to completion.

I read with interest the kerchunk feature of xstac:

FWIW, my thinking on embedding "chunk manifests" (to use virtual zarr's term) in the STAC metadata has shifted a bit since implementing that in xstac. The chunk manifests can get quite large, which may not be the best user experience.

If I had to guess, I think it will be more common to store just the metadata needed to build an xarray Dataset from a STAC item or items (things like the variables, dims, attrs, encoding, shape, chunksize), and you'd include a STAC Asset linking to the chunk manifests. You'll hopefully be able to delay reading the Chunk Manifests, until you actually need to read data (on your Dask worker, for example; you'd read the chunk manifest, which will tell you what additional reads you need to make for the actual data).

@rsignell
Copy link
Author

@TomAugspurger , okay, thanks, so basically like having STAC assets pointing to a kerchunk reference json or set of Parquet files, right?

@TomAugspurger
Copy link
Member

Yep, that's the idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants