-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
#38: rescore, restore, backup utils #44
Draft
mohit-s96
wants to merge
25
commits into
main
Choose a base branch
from
38-etl-adjustments-empty-list-null-value
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mohit-s96
changed the title
#38: rescore/retore/backup utils
#38: rescore/restore/backup utils
Apr 10, 2023
mohit-s96
changed the title
#38: rescore/restore/backup utils
#38: rescore, restore, backup utils
Apr 10, 2023
- results are now saved as they are downloaded - added conflict handling in case of absent usi for single and multiple systems
darnjo
reviewed
May 4, 2023
we check if the availability or frequency count on the field is greater than that of the NULL_VALUE or EMPTY_LIST and only then we reduce those counts. Prevents multiple count reductions in case of more than one run.
now points to latest commit containing the bins removal
All prod reports have USI now. This file is no longer needed.
useful in case of an error where the program gets stuck due to network issues. we can now skip the reports already downloaded and continue where we the program hanged.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes #38
Important
These utilities work on a server that has the changes from https://github.com/RESOStandards/reso-certification/pull/1319 merged.
Basic Usage
Backup a given Reso Cert API server
-d
and-w
are optional flags that specify whether to 'only backup DD/DA reports (-d
)' or to 'only backup webAPI reports (-w
)'. Omitting/Including both will back up everything.Example usage:
node index.js backup -u http://localhost -p ~/Downloads
Restore from a server backup
The
--restoreFromBackup
or-r
flag tells the tool to restore from the backed-up server data rather than raw reports.Example usage:
node index.js restore -u http://localhost -p ~/Downloads/reso-server-backup/data_dictionary-1.7 -r
Run a transformation on all reports of a given cert API server
The
-b
or--backup
flag creates a backup of all the DD/DA reports from the given server before starting the transformation.The
-p
or--pathToBackup
option is required when using the backup flag.The
-r
or--rescore
flag is a transformation that adjusts the EMPTY_LIST/NULL_VALUE counts, rescores the report, and posts them back to the cert API server.The
-e
or--runEtl
flag is another transformation that takes in a DA report, converts it to raw form, and runs the ETL method on the report. This transformation will always be run last if more than one transformation is provided.Note: It is advisable to always create a backup before running any transformations.
Example usage
node index.js transform -u http://localhost -r -b -p ~/Documents