Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable warnings #480

Merged
merged 43 commits into from
May 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
f8171c0
Xarray-style options
jsignell Apr 13, 2023
19cb420
Refine options and add mechanism to easily respond to events based on…
jsignell Apr 13, 2023
c0877f2
Fix types
jsignell Apr 14, 2023
2de758e
Fix message
jsignell Apr 14, 2023
40c5eaf
Use warnings rather than options
jsignell Apr 14, 2023
671d945
Expect warnings
jsignell Apr 14, 2023
1a63a2c
Add another warning and put back assert_conforms_to
jsignell Apr 14, 2023
9001e78
Rollback to passing tests
jsignell Apr 14, 2023
1ae1e0f
Apply suggestions from code review
jsignell Apr 17, 2023
20d427f
Get rid of ignore_conformance
jsignell Apr 18, 2023
9af577a
Let users control conformsTo
jsignell Apr 20, 2023
193b521
Fix up tests
jsignell Apr 20, 2023
e1661df
Merge branch 'main' into warnings
jsignell Apr 20, 2023
60cb2b8
Rewrite a cassette
jsignell Apr 20, 2023
7efd55f
Rewrite a cassette
jsignell Apr 20, 2023
618324f
More tests to demonstrate helper methods
jsignell Apr 20, 2023
eaa85ab
Tests for altering conforms_to via cli
jsignell Apr 21, 2023
85c4697
Add cassettes
jsignell Apr 21, 2023
e1fbd66
fix, tests: rewrite some cassettes, filter warning
gadomski Apr 21, 2023
d747b93
Merge branch 'main' into warnings
gadomski Apr 21, 2023
025dd80
Merge branch 'main' into warnings
jsignell Apr 24, 2023
5b84cf2
Put back in `ignore_conformance`
jsignell Apr 24, 2023
67a5983
Apply suggestions from code review
jsignell Apr 24, 2023
df9d698
Make warnings easier to use
jsignell Apr 24, 2023
38a2ba3
Add _supports_collections
jsignell Apr 24, 2023
0accb69
Better examples
jsignell Apr 24, 2023
de3eff6
Don't deprecate stac_io, add warnings for conformance
jsignell Apr 24, 2023
9e3cefd
Make sure stacklevel is always 2
jsignell Apr 24, 2023
1109d13
Try again to record the cassette
jsignell Apr 24, 2023
de34f4c
Docs and simplify a bit
jsignell Apr 24, 2023
c14ce2d
Merge branch 'main' into warnings
jsignell Apr 25, 2023
d3fb5ba
Merge branch 'main' into warnings
jsignell Apr 25, 2023
9804aad
Merge branch 'main' into warnings
gadomski Apr 27, 2023
98cc2a1
tests: rewrite cassette
gadomski Apr 27, 2023
718d23f
Start working on docs
jsignell Apr 27, 2023
b1aa915
Respond to PR comments
jsignell Apr 27, 2023
037368e
Finish first pass at docs
jsignell Apr 27, 2023
d0c9092
Merge branch 'main' into warnings
jsignell May 1, 2023
a34e592
Update docs/usage.rst
jsignell May 4, 2023
bfa8546
Don't capture warnings in logs
jsignell May 4, 2023
60a3ac6
Update changelog
jsignell May 4, 2023
3493978
Merge branch 'main' into warnings
jsignell May 4, 2023
8031adc
Merge branch 'main' into warnings
jsignell May 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
### Added

- Timeout option added to `Client.open` [#463](https://github.com/stac-utils/pystac-client/pull/463)
- Support for fetching catalog queryables [#477](https://github.com/stac-utils/pystac-client/pull/477)
- PySTAC Client specific warnings [#480](https://github.com/stac-utils/pystac-client/pull/480)

### Changed

Expand All @@ -26,9 +28,10 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Three tests that were false positives due to out-of-date cassettes [#491](https://github.com/stac-utils/pystac-client/pull/491)
- Max items checks when paging [#492](https://github.com/stac-utils/pystac-client/pull/492)

### Added
### Removed

- ``pystac_client.conformance.CONFORMANCE_URIS`` dictionary [#480](https://github.com/stac-utils/pystac-client/pull/480)

- Support for fetching catalog queryables [#477](https://github.com/stac-utils/pystac-client/pull/477)

## [v0.6.1] - 2023-03-14

Expand Down
8 changes: 8 additions & 0 deletions docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,11 @@ Exceptions
:members:
:undoc-members:
:show-inheritance:

Warnings
--------

.. automodule:: pystac_client.warnings
:members:
:undoc-members:
:show-inheritance:
45 changes: 20 additions & 25 deletions docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,16 @@ the screen (if supported by the STAC API).

.. code-block:: console

$ stac-client search https://earth-search.aws.element84.com/v0 -c sentinel-s2-l2a-cogs --bbox -72.5 40.5 -72 41 --matched
2179 items matched
$ stac-client search https://earth-search.aws.element84.com/v1 -c sentinel-2-l2a --bbox -72.5 40.5 -72 41 --matched
3141 items matched

If the same URL is to be used over and over, define an environment
variable to be used in the CLI call:

.. code-block:: console

$ export STAC_API_URL=https://earth-search.aws.element84.com/v0
$ stac-client search ${STAC_API_URL} -c sentinel-s2-l2a-cogs --bbox -72.5 40.5 -72 41 --datetime 2020-01-01/2020-01-31 --matched
$ export STAC_API_URL=https://earth-search.aws.element84.com/v1
$ stac-client search ${STAC_API_URL} -c sentinel-2-l2a --bbox -72.5 40.5 -72 41 --datetime 2020-01-01/2020-01-31 --matched
48 items matched

Without the ``--matched`` switch, all items will be fetched, paginating
Expand All @@ -38,7 +38,7 @@ another process such as

.. code-block:: console

$ stac-client search ${STAC_API_URL} -c sentinel-s2-l2a-cogs --bbox -72.5 40.5 -72 41 --datetime 2020-01-01/2020-01-31 | stacterm cal --label platform
$ stac-client search ${STAC_API_URL} -c sentinel-2-l2a --bbox -72.5 40.5 -72 41 --datetime 2020-01-01/2020-01-31 | stacterm cal --label platform

.. figure:: images/stacterm-cal.png
:alt:
Expand All @@ -48,7 +48,7 @@ output to stdout, but instead will be saved to the specified file.

.. code-block:: console

$ stac-client search ${STAC_API_URL} -c sentinel-s2-l2a-cogs --bbox -72.5 40.5 -72 41 --datetime 2020-01-01/2020-01-31 --save items.json
$ stac-client search ${STAC_API_URL} -c sentinel-2-l2a --bbox -72.5 40.5 -72 41 --datetime 2020-01-01/2020-01-31 --save items.json

If the Catalog supports the `Query
extension <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/query>`__,
Expand Down Expand Up @@ -79,12 +79,12 @@ than once to use additional operators.

.. code-block:: console

$ stac-client search ${STAC_API_URL} -c sentinel-s2-l2a-cogs --bbox -72.5 40.5 -72 41 --datetime 2020-01-01/2020-01-31 -q "eo:cloud_cover<10" --matched
$ stac-client search ${STAC_API_URL} -c sentinel-2-l2a --bbox -72.5 40.5 -72 41 --datetime 2020-01-01/2020-01-31 -q "eo:cloud_cover<10" --matched
10 items matched

.. code-block:: console

$ stac-client search ${STAC_API_URL} -c sentinel-s2-l2a-cogs --bbox -72.5 40.5 -72 41 --datetime 2020-01-01/2020-01-31 -q "eo:cloud_cover<10" "eo:cloud_cover>5" --matched
$ stac-client search ${STAC_API_URL} -c sentinel-2-l2a --bbox -72.5 40.5 -72 41 --datetime 2020-01-01/2020-01-31 -q "eo:cloud_cover<10" "eo:cloud_cover>5" --matched
4 items matched

Python
Expand All @@ -97,36 +97,31 @@ specific STAC API (use the root URL):

from pystac_client import Client

client = Client.open("https://earth-search.aws.element84.com/v0")
client = Client.open("https://earth-search.aws.element84.com/v1")

Create a search:

.. code-block:: python

my_search = client.search(
search = client.search(
max_items=10,
collections=['sentinel-s2-l2a-cogs'],
bbox=[-72.5,40.5,-72,41])
print(f"{my_search.matched()} items found")
collections=['sentinel-2-l2a'],
bbox=[-72.5,40.5,-72,41]
)
print(f"{search.matched()} items found")

The ``items()`` iterator method can be used to iterate through all resulting items.

.. code-block:: python

for item in my_search.items():
for item in search.items():
print(item.id)

To convert all of Items from a search as a single `PySTAC
ItemCollection <https://pystac.readthedocs.io/en/latest/api/pystac.html#pystac.ItemCollection>`__,
you must first do a limited iteration on the iterator to get a list of Items, and then
create an ItemCollection with that. The ``ItemCollection`` can then be saved as a
GeoJSON FeatureCollection.

Save all found items as a single FeatureCollection:
Use `item_collection()` to convert all Items from a search into a single `PySTAC
ItemCollection <https://pystac.readthedocs.io/en/latest/api/pystac.html#pystac.ItemCollection>`__.
The ``ItemCollection`` can then be saved as a GeoJSON FeatureCollection.

.. code-block:: python

from pystac import ItemCollection

my_itemcollection = ItemCollection(items = list(my_search.items()))
my_itemcollection.save_object('my_itemcollection.json')
item_collection = search.item_collection()
item_collection.save_object('my_itemcollection.json')
2 changes: 1 addition & 1 deletion docs/tutorials/aoi-coverage.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@
"\n",
"# Create a Client and an ItemSearch representing our search\n",
"# No search operations will be performed until we call the items() method\n",
"client = Client.open(URL, ignore_conformance=False)\n",
"client = Client.open(URL)\n",
"item_search = client.search(\n",
" collections=[\"sentinel-2-l2a\"], intersects=intersects_geometry, max_items=100\n",
")"
Expand Down
5 changes: 3 additions & 2 deletions docs/tutorials/cql2-filter.ipynb
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "e06a27bf",
"metadata": {},
Expand All @@ -9,7 +10,7 @@
"\n",
"This notebook demonstrates the use of pystac-client to use [CQL2 filtering](https://github.com/radiantearth/stac-api-spec/tree/master/fragments/filter). The server needs to support this and advertise conformance as the `https://api.stacspec.org/v1.0.0-rc.1/item-search#filter` class in the `conformsTo` attribute of the root API.\n",
"\n",
"**This should be considered an experimental feature. This notebook uses the Microsoft Planetary Computer API, as it is currently the only public CQL2 implementation. The Planetary Computer API also does not yet advertise the correct conformance class, thus the `ignore_conformance` keyword is specified in the `Client.open` function below.**"
"**This should be considered an experimental feature. This notebook uses the Microsoft Planetary Computer API, as it is currently the only public CQL2 implementation.**"
]
},
{
Expand Down Expand Up @@ -61,7 +62,7 @@
"# custom headers\n",
"headers = []\n",
"\n",
"cat = Client.open(URL, headers=headers, ignore_conformance=True)"
"cat = Client.open(URL, headers=headers)"
]
},
{
Expand Down
108 changes: 61 additions & 47 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,37 +39,35 @@ includes convenience methods and attributes for:

* Checking conformance to various specs
* Querying a search endpoint (if the API conforms to the STAC API - Item Search spec)
* Getting jsonschema of queryables from `/queryables` endpoint (if the API conforms
to the STAC API - Filter spec)

The preferred way to interact with any STAC Catalog or API is to create an
:class:`pystac_client.Client` instance with the ``pystac_client.Client.open`` method
on a root Catalog. This calls the :meth:`pystac.STACObject.from_file` except
on a root Catalog. This calls the :meth:`pystac.STACObject.from_file` except it
properly configures conformance and IO for reading from remote servers.

The following code creates an instance by making a call to the Microsoft Planetary
Computer root catalog.

.. code-block:: python

>>> from pystac_client.Client import Client
>>> api = Client.open('https://planetarycomputer.microsoft.com/api/stac/v1')
>>> api.title
'microsoft-pc'
>>> from pystac_client import Client
>>> catalog = Client.open('https://planetarycomputer.microsoft.com/api/stac/v1')
>>> catalog.title
'Microsoft Planetary Computer STAC API'

Some functions, such as ``Client.search`` will throw an error if the provided
Catalog/API does not support the required Conformance Class. In other cases,
such as ``Client.get_collections``, API endpoints will be used if the API
conforms, otherwise it will fall back to default behavior provided by
:class:`pystac.Catalog`.

Users may optionally provide an ``ignore_conformance`` argument when opening,
in which case pystac-client will not check for conformance and will assume
this is a fully featured API. This can cause unusual errors to be thrown if the API
does not in fact conform to the expected behavior.

In addition to the methods and attributes inherited from :class:`pystac.Catalog`,
this class offers more efficient methods (if used with an API) for getting collections
and items, as well as a search capability, utilizing the
:class:`pystac_client.ItemSearch` class.
When a ``Client`` does not conform to a particular Conformance Class, an informative
warning is raised. Similarly when falling back to the :class:`pystac.Catalog`
implementation a warning is raised. You can control the behavior of these warnings
using the standard :py:mod:`warnings` or special context managers :func:`pystac_client.warnings.strict` and
from :func:`pystac_client.warnings.ignore`.

API Conformance
---------------
Expand All @@ -83,53 +81,69 @@ A STAC API is a STAC Catalog that is required to advertise its capabilities in a
`conformsTo` field and implements the `STAC API - Core` spec along with other
optional specifications:

* `STAC API - Core <https://github.com/radiantearth/stac-api-spec/tree/master/core>`__
* `STAC API - Item Search <https://github.com/radiantearth/stac-api-spec/tree/master/item-search>`__
* `Fields Extension <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/fields>`__
* `Query Extension <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/query>`__
* `Sort Extension <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/sort>`__
* `Context Extension <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/context>`__
* `Filter Extension <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/filter>`__
* `STAC API - Features <https://github.com/radiantearth/stac-api-spec/tree/master/ogcapi-features>`__ (based on
* `CORE <https://github.com/radiantearth/stac-api-spec/tree/master/core>`__
* `ITEM_SEARCH <https://github.com/radiantearth/stac-api-spec/tree/master/item-search>`__
* `FIELDS <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/fields>`__
* `QUERY <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/query>`__
* `SORT <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/sort>`__
* `CONTEXT <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/context>`__
* `FILTER <https://github.com/radiantearth/stac-api-spec/tree/master/fragments/filter>`__
* `COLLECTIONS <https://github.com/radiantearth/stac-api-spec/tree/master/collections>`__ (based on
the `Features Collection section of OGC APO - Features <http://docs.opengeospatial.org/is/17-069r3/17-069r3.html#_collections_>__`)
* `FEATURES <https://github.com/radiantearth/stac-api-spec/tree/master/ogcapi-features>`__ (based on
`OGC API - Features <https://www.ogc.org/standards/ogcapi-features>`__)

The :meth:`pystac_client.Client.conforms_to` method is used to check conformance
against conformance classes (specs). To check an API for support for a given spec,
pass the `conforms_to` function the :class:`ConformanceClasses` attribute as a
parameter.
pass the `conforms_to` function the name of a :class:`ConformanceClasses`.

.. code-block:: python

>>> from pystac_client import ConformanceClasses
>>> api.conforms_to(ConformanceClasses.STAC_API_ITEM_SEARCH)
>>> catalog.conforms_to("ITEM_SEARCH")
True

If the API does not advertise conformance with a particular spec, but it does support
it you can update `conforms_to` on the client object. For instance in `v0` of earth-search
there are no ``"conformsTo"`` uris set at all. But they can be explicitly set:

.. code-block:: python

>>> catalog = Client.open("https://earth-search.aws.element84.com/v0")
<stdin>:1: NoConformsTo: Server does not advertise any conformance classes.
>>> catalog.conforms_to("ITEM_SEARCH")
False
>>> catalog.add_conforms_to("ITEM_SEARCH")

Note, updating ``"conformsTo"`` does not change what the server supports, it just
changes PySTAC client's understanding of what the server supports.

CollectionClient
++++++++++++++++

STAC APIs may provide a curated list of catalogs and collections via their ``"links"``
attribute. Links with a ``"rel"`` type of ``"child"`` represent catalogs or collections
provided by the API. Since :class:`~pystac_client.Client` instances are also
:class:`pystac.Catalog` instances, we can use the methods defined on that class to
get collections:
STAC APIs may optionally implement a ``/collections`` endpoint as describe in the
`STAC API - Collections spec
<https://github.com/radiantearth/stac-api-spec/tree/master/collections>`__. This endpoint
allows clients to search or inspect items within a particular collection.

.. code-block:: python

>>> child_links = api.get_links('child')
>>> len(child_links)
12
>>> first_child_link = api.get_single_link('child')
>>> first_child_link.resolve_stac_object(api)
>>> first_collection = first_child_link.target
>>> first_collection.title
'Landsat 8 C1 T1'
>>> catalog = Client.open('https://planetarycomputer.microsoft.com/api/stac/v1')
>>> collection = catalog.get_collection("sentinel-2-l2a")
>>> collection.title
'Sentinel-2 Level-2A'

:class:`pystac_client.CollectionClient` overrides :meth:`pystac.Collection.get_items`.
PySTAC will get items by iterating through all children until it gets to an ``item`` link.
PySTAC client will use the API endpoint instead: `/collections/<collection_id>/items`
(as long as `STAC API - Item Search spec
<https://github.com/radiantearth/stac-api-spec/tree/master/item-search>`__ is supported).

.. code-block:: python

CollectionClient overrides the :meth:`pystac.Collection.get_items` method. PySTAC will
get items by iterating through all children until it gets to an `item` link. If the
`CollectionClient` instance contains an `items` link, this will instead iterate through
items using the API endpoint instead: `/collections/<collection_id>/items`. If no such
link is present it will fall back to the PySTAC Collection behavior.
>>> item = next(collection.get_items(), None)

Note that calling list on this iterator will take a really long time since it will be retrieving
every itme for the whole ``"sentinel-2-l2a"`` collection.

ItemSearch
++++++++++
Expand All @@ -149,8 +163,8 @@ requests to a service's "search" endpoint. This method returns a
.. code-block:: python

>>> from pystac_client import Client
>>> api = Client.open('https://planetarycomputer.microsoft.com/api/stac/v1')
>>> results = api.search(
>>> catalog = Client.open('https://planetarycomputer.microsoft.com/api/stac/v1')
>>> results = catalog.search(
... max_items=5
... bbox=[-73.21, 43.99, -73.12, 44.05],
... datetime=['2019-01-01T00:00:00Z', '2019-01-02T00:00:00Z'],
Expand Down Expand Up @@ -307,7 +321,7 @@ modify the STAC objects returned by the STAC API.

>>> from pystac_client import Client
>>> import planetary_computer, requests
>>> api = Client.open(
>>> catalog = Client.open(
... 'https://planetarycomputer.microsoft.com/api/stac/v1',
... modifier=planetary_computer.sign_inplace,
... )
Expand Down
Loading