-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RDRP-1142_freezing_amendments #418
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could we use this as an opportunity to replace this list with getting the cols from the config?
Also, the tests seem to be failing....
src/freezing/freezing_compare.py
Outdated
"706", | ||
"707", | ||
"709", | ||
"711", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this looks rough, doesn't it? Can you find a way of using what we have in the dev config to get this list?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deriving this list from the dev_config.yaml
would be a great challenge. The reason is that this list of columns comes from an excel file shared between Sarah M and Jen C. I would suggest, as you have mentioned, moving this list to the config.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
consistency_checks:
2xx_totals:
purchases_split: ["222", "223", "203"]
sal_oth_expend: ["202", "203", "204"]
research_expend: ["205", "206", "207", "204"]
capex: ["219", "220", "209", "210"]
intram: ["204", "210", "211"]
funding: ['212', '214', '216', '242', '250', '243', '244', '245', '246', '247', '248', '249', '218']
ownership: ['225', '226', '227', '228', '229', '237', '218']
equality: ['211', '218']
3xx_totals:
purchases: ['302', '303', '304', '305']
4xx_totals:
emp_civil: ['405', '407', '409', '411']
emp_defence: ['406', '408', '410', '412']
5xx_totals:
hc_res_m: ['501', '503', '505', '507']
hc_res_f: ['502', '504', '506', '508']
Use this part of the config
and this code in utils/ breakdown_validation.py
def get_equality_dicts(config: dict, sublist: str = "default") -> dict:
"""
Get the equality checks for the construction data.
Args:
config (dict): The config dictionary.
Returns:
dict
"""
# use the config to get breakdown totals
all_checks_dict = config["consistency_checks"]
# isolate the relationships suitlable for checking in the construction module
if sublist == "default":
wanted_dicts = [key for key in all_checks_dict.keys() if "xx_totals" in key]
… call config to get cols to check
src/dev_config.yaml
Outdated
@@ -30,6 +30,9 @@ freezing_paths: | |||
freezing_changes_to_review_path: "changes_to_review" | |||
freezing_amendments_path: "freezing_updates" | |||
freezing_additions_path: "freezing_updates" | |||
freezing_columns_to_check: | |||
non_numeric_columns: ["200", "201", "601", "604"] | |||
non_numeric_columns: ["202", "203", "204", "205", "206", "207", "209", "210", "211", "212", "214", "216", "218", "219", "220", "221", "222", "223", "225", "226", "227", "228", "229", "237", "242", "243", "244", "245", "246", "247", "248", "249", "250", "405", "406", "407", "408", "409", "410", "411", "412", "501", "502", "503", "504", "505", "506", "507", "508", "602", "701", "702", "703", "704", "705", "706", "707", "709", "711"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi Ryan,
Instead of this, please could you use the code I put in my second review? Then we've got everything for the questions in the same place, and if we need to update, we only need to update in one place, thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Of course, no worries.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some columns will ne hard coding in the config as they do not appear within config["consistency_checks"], but are specified by the business area to checked, for example 219.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the use of config["consistency_checks"]
for gathering the columns freezing amendments is applied to:
Employing get_equality_dicts
to gather columns for freezing comparison yields a comprehensive list of all values within config["consistency_checks"]
. Splitting this list to match numeric and non-numeric columns to be checked (defined by the business area) requires either in the config or a script to contain these numeric and non-numeric column names, thus making the entire idea of removing hardcoded numeric and non-numeric from the config/script redundant.
…d how freezing cols are derived
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a few thoughts... I haven't finished reviewing yet.
} | ||
} | ||
|
||
return config |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do you think it would be useful to have more columns to check? One col is not much....
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it not needed. The unit test already checks 4 columns. Before my changes the unit test only checked 3 (and 3 must have been deemed satisfactory).
addressed PR comments
changes to address PR comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a few more comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
more comments on test
"consistency_checks": { | ||
"2xx_totals": { | ||
"purchases_split": [], | ||
"sal_oth_expend": ["203"], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add 202 ?
Pull Request submission
Insert detailed bullet points about your changes here!
Insert any instructions to help the reviewer, e.g. "install new requirements from
requirements.txt
"*Let the reviewer know what data files are needed (and if applicable, where they are to be found)
Closes or fixes
Closes #
Code
Documentation
Any new code includes all the following forms of documentation:
Args
andreturns
for all major functionsData
Testing
Peer Review Section
requirements.txt
Final approval (post-review)
The author has responded to my review and made changes to my satisfaction.
Review comments
Insert detailed comments here!
These might include, but not exclusively:
that it is likely to interact with?)
works correctly?)
Your suggestions should be tailored to the code that you are reviewing.
Be critical and clear, but not mean. Ask questions and set actions.