Releases: LSSTDESC/dataregistry
Releases · LSSTDESC/dataregistry
v1.0.2
v1.0.1
- Expand support to Python 3.12
v1.0.0
Release v1.0.0
First stable release. Changes in CHANGELOG
. Release v1.0.0 is available in the DESC python_bleed
environment.
v0.4.0
Version 0.4.0
Version 0.4.0 focuses around being able to manipulate data already within the dataregistry, i.e., adding the ability to delete and modify previous datasets.
Changelog for developers:
Registrar
now has a class for each table. They inherit from aBaseTable
class, this means that shared functions, like deleting entries, are available for all tables. (#92)- Working with tables via the python interface has slightly different syntax (see user changelog below). (#92)
is_valid
is removed as adataset
property. It has been replaced withstatus
which is a bitmask (bit 0="valid", bit 1= "deleted" and bit 2="archived"), so now datasets can a combination of multiple states. (#93)archive_date
,archive_path
,delete_date
,delete_uid
andmove_date
have been added as newdataset
fields. (#93)- Database version bumped to
2.0.1
(#93) dataset
entries can be deleted (see below) (#94)- The CI for the CLI is now pure Python (i.e., there is no more bash script to ingest dummy entries into the registry for testing).
- Can no longer "bump" a dataset that has a version suffix (trying to do so will raise an error). If a user wants to make a new version of a dataset with a suffix they can still do so by manually specifying the version and suffix (#97 ).
- Dataset entries can be modified (see below, #100)
Changelog for users:
- All database tables (
dataset
,execution
, etc) have a more universal syntax. The functionality is still accessed via theRegistrar
class, but now for example to register a dataset it'sRegistrar.dataset.register()
, similarly for an executionRegistrar.execution.register()
(#92). The docs and tutorials have been updated (#95). dataset
entries can now be deleted using theRegistrar.dataset.delete(dataset_id=...)
function. This will also delete the raw data within theroot_dir
. Note that the entry in the database will always remain (with an updatedstatus
field to indicate it has been deleted). (#94)- Documentation has been updated to make things a bit clearer. Now split into more focused tutorials (#95).
- Certain dataset quantities can be modified after registration (#100). Documentation has been updated with examples.
Release v0.3.0
v0.3.0 Release notes
For users
access_API_configuration
field added to thedataset
table. When registering a dataset you can now specify a text configuration file that will be read in as raw text and associated with the dataset. This is similar to theconfiguration
column in theexecution
table.- The data registry path to where the data is stored now includes the
schema
name, i.e.,<root_dir>/<schema>/<owner_type>/<owner>/<relative_path>
- There is now a check to make sure copying data was successful. If there is an entry in the
dataset
table withis_valid=False
it means that the data copying fromold_location
failed for some reason. If the data failed to copy when the dataset was intended to be overwritten, it will not corrupt the original data, the original data is kept. - Can now specify a "site" rather than a manual
root_dir
. For now the only site is "nersc", which points to the common shared space on NERSC for the dataregistry. Ifroot_dir=None
for theDataRegistry
class, "nersc" is assumed to be the default.
For developers
- Database version is updated to version 2.0.0
- Implemented production schema.
- Can now link dependencies to datasets in other schemas (i.e., to production).
- Schema has been moved to a yaml file, which is called upon in the code. This saves dataset description duplication.
- Use pytest fixtures to create temporary files and directories for copying dummy files into the registry during CI and for creating dummy config files.
- The original
root_dir
a dataset was ingested into is now stored as a field in thedataset
table.
v0.2.2: Merge pull request #35 from LSSTDESC/u/jrbogart/beta_default
Ready for beta testers