-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
User-generated Collections #376
Comments
Note that this is again a step further from simply loading a result, may not be possible to do this within the platform project. |
What exactly is the difference (from your point of view)? Both the user's results and collections need to be loaded into a data cube, so is there a big difference whether they are exposed via the results endpoints or the collections endpoints? Once we are at the point to expose all results as collections instead of items (see #359), it seems to be just a small additional step. Edit: Removed outdated stuff |
Below, I'm discussing some options for user-generated collections that we can consider for openEO (and SAP07 in openEO Platform @christophreimer). Options:
General
The collection metadata is generated by the back-end, but can usually be overriden. Some general thoughts:
Each option (except 5) creates collections in the
We should try to align the endpoints with OGC APIs and/or STAC Transaction extension (if available). 1. File Format
|
Thanks for putting this together, we also want to move forward with this!
I see most promise in a combination of options 3 and 4. So basically an API that aligns with STAC transaction to allow management of the collection, and a specific process to add job results to that collection. |
Thank you for these points. Comments below.
Good to know. I'll add this above. So then it's a less internal thing and reading such collections is more related to load_stac rather than load_collection, I guess.
Hmm, this sounds like a back-end issue that should be handled internally and not by the API. On the other hand, it could be generally useful if users could combine multiple results. But then I'm wondering about the consistency of the data. How can this be ensured? It feels cleaner and safer to just allow a "controlled entity" like the aggregator to combine data under a single collection, if at all.
I'm not sure whether I made it clear enough in my post, but unfortunately STAC Transactions are currently only defined on the item level. So you can't create a collection through the API and you'd need to push all items individually (or as item collection?), which is rather bad. We'd need to define collection-level transactions and contribute them back to STAC/OGC, which will be a rather lengthy process, I assume. It seems unfeasible during the runtime of SAP07, at least. |
Another example of multiple jobs contributing to a collection is when you add a new product for instance every 10 days, as new satellite observations come in. Here I don't see how we could make that work with a single batch job. |
That's a good point. So saving to an existing collection should add data instead of overwriting it or throwing an error. |
@GeraldIr fyi |
Idea from the openEO community call:
|
`save_results` returns the STAC resource instead of boolean `true` Open-EO/openeo-api#376
Proposed extension for Collection Transactions in STAC APIs: |
* Add `export_collection`, `export_workspace`, `stac_update`; `save_results` returns the STAC resource instead of boolean `true` Open-EO/openeo-api#376 * Update stac_update/modify * Added details about STAC support. * Update meta/implementation.md Co-authored-by: Matthias Mohr <m.mohr@uni-muenster.de> --------- Co-authored-by: Michele Claus <31700619+clausmichele@users.noreply.github.com>
How to make user-generated or user-uploaded collections available? We could make them available in /collections (after authentication). Currently load_result and load_uploaded_files are an option, but maybe it's also just easier to make it available via collections or a new endpoint. This is very much open to discussion...
The text was updated successfully, but these errors were encountered: