Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#38: rescore, restore, backup utils #44

Draft
wants to merge 25 commits into
base: main
Choose a base branch
from

Conversation

mohit-s96
Copy link
Member

@mohit-s96 mohit-s96 commented Apr 10, 2023

closes #38

Important

These utilities work on a server that has the changes from https://github.com/RESOStandards/reso-certification/pull/1319 merged.

Basic Usage

Backup a given Reso Cert API server

node index.js backup -u <server base url> -p <path to the directory where we want the reports backed up> -d -w

-d and-w are optional flags that specify whether to 'only backup DD/DA reports (-d)' or to 'only backup webAPI reports (-w)'. Omitting/Including both will back up everything.

Example usage:

node index.js backup -u http://localhost -p ~/Downloads

Restore from a server backup

node index.js restore -u <server base url> -p <path to the server backup directory> --restoreFromBackup

The --restoreFromBackup or -r flag tells the tool to restore from the backed-up server data rather than raw reports.

Example usage:

node index.js restore -u http://localhost -p ~/Downloads/reso-server-backup/data_dictionary-1.7 -r

Run a transformation on all reports of a given cert API server

node index.js transform -u <server base url> -p <path to the directory where we want the reports backed up> -r -b

The -b or --backup flag creates a backup of all the DD/DA reports from the given server before starting the transformation.

The -p or --pathToBackup option is required when using the backup flag.

The -r or --rescore flag is a transformation that adjusts the EMPTY_LIST/NULL_VALUE counts, rescores the report, and posts them back to the cert API server.

The -e or --runEtl flag is another transformation that takes in a DA report, converts it to raw form, and runs the ETL method on the report. This transformation will always be run last if more than one transformation is provided.

Note: It is advisable to always create a backup before running any transformations.

Example usage

node index.js transform -u http://localhost -r -b -p ~/Documents

@mohit-s96 mohit-s96 changed the title #38: rescore/retore/backup utils #38: rescore/restore/backup utils Apr 10, 2023
@mohit-s96 mohit-s96 changed the title #38: rescore/restore/backup utils #38: rescore, restore, backup utils Apr 10, 2023
@mohit-s96 mohit-s96 self-assigned this Apr 12, 2023
- results are now saved as they are downloaded
- added conflict handling in case of absent usi for single and multiple systems
data-access/usi-mapping.js Outdated Show resolved Hide resolved
mohit-s96 added 4 commits May 4, 2023 11:59
we check if the availability or frequency count on the field is greater than that of the NULL_VALUE or EMPTY_LIST and only then we reduce those counts. Prevents multiple count reductions in case of more than one run.
@mohit-s96 mohit-s96 linked an issue Jul 11, 2023 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add transformation for removing bins ETL adjustments EMPTY_LIST and NULL_VALUE
2 participants