Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NOIRLab advanced search #1701

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
2 changes: 1 addition & 1 deletion CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ esa/xmm-newton
noirlab
^^^^^^^

- Module added to access the NOIRLab (formally NOAO) archive. [#1638]
- Module added to access the NOIRLab (formally NOAO) archive. [#1638, #1701]


Service fixes and enhancements
Expand Down
7 changes: 2 additions & 5 deletions astroquery/noirlab/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from astropy import config as _config
# Licensed under a 3-clause BSD style license - see LICENSE.rst
"""NSF's OIR Lab Astro Data Archive(Beta)
-----------------------------------------
"""NSF's OIR Lab Astro Data Archive (Beta)
------------------------------------------

The NSF's OIR Lab Astro Data Archive (formerly NOAO Science Archive)
pothiers marked this conversation as resolved.
Show resolved Hide resolved
provides access to data taken with more than 40 telescope and
Expand Down Expand Up @@ -54,9 +54,6 @@
Universities for Research in Astronomy (AURA), Inc. under a
cooperative agreement with the National Science Foundation.


See also: gemini, nrao

"""


Expand Down
93 changes: 85 additions & 8 deletions astroquery/noirlab/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

This does DB access through web-services.
"""

import astropy.io.fits as fits
import astropy.table
from ..query import BaseQuery
from ..utils import async_to_sync
Expand All @@ -19,8 +19,6 @@ class NoirlabClass(BaseQuery):

TIMEOUT = conf.timeout
NAT_URL = conf.server
ADS_URL = f'{NAT_URL}/api/adv_search/fasearch'
SIA_URL = f'{NAT_URL}/api/sia/voimg'

def __init__(self, which='file'):
"""Return object used for searching the NOIRLab Archive.
Expand All @@ -31,13 +29,18 @@ def __init__(self, which='file'):
is only the case with some pipeline processed files.
"""
self._api_version = None
self._adsurl = f'{self.NAT_URL}/api/adv_search'

if which == 'hdu':
self.url = f'{self.NAT_URL}/api/sia/vohdu'
elif which == 'file':
self.url = f'{self.NAT_URL}/api/sia/voimg'
self.siaurl = f'{self.NAT_URL}/api/sia/vohdu'
self._adss_url = f'{self._adsurl}/hasearch'
self._adsc_url = f'{self._adsurl}/core_hdu_fields'
self._adsa_url = f'{self._adsurl}/aux_hdu_fields'
else:
self.url = f'{self.NAT_URL}/api/sia/voimg'
self.siaurl = f'{self.NAT_URL}/api/sia/voimg'
self._adss_url = f'{self._adsurl}/fasearch'
self._adsc_url = f'{self._adsurl}/core_file_fields'
self._adsa_url = f'{self._adsurl}/aux_file_fields'

super().__init__()

Expand Down Expand Up @@ -66,6 +69,15 @@ def _validate_version(self):
f'{self.api_version} from the API.')
raise Exception(msg)

def service_metadata(self, cache=True):
"""Denotes a Metadata Query: no images are requested; only metadata
should be returned. This feature is described in more detail in:
http://www.ivoa.net/documents/PR/DAL/PR-SIA-1.0-20090521.html#mdquery
"""
url = f'{self.siaurl}?FORMAT=METADATA&format=json'
response = self._request('GET', url, timeout=self.TIMEOUT, cache=cache)
return response.json()[0]

@class_or_instance
def query_region(self, coordinate, radius=0.1, cache=True):
pothiers marked this conversation as resolved.
Show resolved Hide resolved
"""Query for NOIRLab observations by region of the sky.
Expand All @@ -90,12 +102,77 @@ def query_region(self, coordinate, radius=0.1, cache=True):
"""
self._validate_version()
ra, dec = coordinate.to_string('decimal').split()
url = f'{self.url}?POS={ra},{dec}&SIZE={radius}&format=json'
url = f'{self.siaurl}?POS={ra},{dec}&SIZE={radius}&format=json'
response = self._request('GET', url,
timeout=self.TIMEOUT,
cache=cache)
response.raise_for_status()
Comment on lines 103 to 109
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest replacing all of this with:

response = self.query_region_async(coordinate, radius=radius, cache=cache)

return astropy.table.Table(data=response.json())

def core_fields(self, cache=True):
"""List the available CORE fields. CORE fields are faster to search
than AUX fields.."""
response = self._request('GET', self._adsc_url,
timeout=self.TIMEOUT,
cache=cache)
response.raise_for_status()
return response.json()

def aux_fields(self, instrument, proctype, cache=True):
"""List the available AUX fields. AUX fields are ANY fields in the
Archive FITS files that are not core DB fields. These are generally
common to a single Instrument, Proctype combination. AUX fields are
slower to search than CORE fields. """
url = f'{self._adsa_url}/{instrument}/{proctype}/'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since instrument and proctype are user-entered parameters, please document them here (and maybe consider adding some sort of validity check)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these functions are supported by web-services which do the validation. We have people accessing directly via the web-services. Doing validation here would be redundant (and require DB access since what is valid depends on DB. I've added note in doc that says the acceptable values come from CATEGORICALS method.

Copy link
Contributor

@keflavich keflavich May 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need the parameters documented here, in the docstring.

If a function takes parameters, as this one does, those parameters need to be documented in the docstring, i.e.:

Parameters
----------
instrument : str
    Instrument name, see <categoricals?> for details

etc.

response = self._request('GET', url, timeout=self.TIMEOUT, cache=cache)
response.raise_for_status()
return response.json()

def categoricals(self, cache=True):
"""List the currently acceptable values for each 'categorical field'
associated with Archive files. A 'categorical field' is one in
which the values are restricted to a specific set. The specific
set may grow over time, but not often. The categorical fields are:
collection, instrument, obs_mode, proc_type, prod_type, site, survey,
telescope.
"""
url = f'{self._adsurl}/cat_lists/?format=json'
response = self._request('GET', url, timeout=self.TIMEOUT, cache=cache)
response.raise_for_status()
return response.json()

@class_or_instance
def query_metadata(self, qspec, limit=1000, cache=True):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could also be async'd, but nbd if it isn't

it does, however, need some docs!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, this function requires a docstring.

self._validate_version()
url = f'{self._adss_url}/?limit={limit}'

if qspec is None:
jdata = {"outfields": ["md5sum", ], "search": []}
else:
jdata = qspec

response = self._request('POST', url, json=jdata, timeout=self.TIMEOUT)
response.raise_for_status()
return astropy.table.Table(rows=response.json())
pothiers marked this conversation as resolved.
Show resolved Hide resolved

def retrieve(self, fileid, cache=True):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need docstring

url = f'{self.NAT_URL}/api/retrieve/{fileid}/'
hdul = fits.open(url)
return hdul

def version(self, cache=False):
url = f'{self.NAT_URL}/api/version/'
response = self._request('GET', url, timeout=self.TIMEOUT, cache=cache)
response.raise_for_status()
return response.json()

def get_token(self, email, password, cache=True):
url = f'{self.NAT_URL}/api/get_token/'
response = self._request('POST', url,
json={"email": email, "password": password},
timeout=self.TIMEOUT)
response.raise_for_status()
return response.json()


Noirlab = NoirlabClass()
Loading