Add a way to pass command to the script from a file #34
Conversation
src/cleanup/cleanup_script.py
Outdated
|
|
||
| Example: 1, 3 => 1.25 |
There was a problem hiding this comment.
| Example: 1, 3 => 1.25 |
| """ | ||
| Read commands and their arguments from a file. | ||
|
|
||
| Expected file format: | ||
| - Each command on its own line | ||
| - Each line's values are separated by commas (csv file ideal) | ||
| - The first value of each line is the command, the rest are its arguments (if any) | ||
| """ |
There was a problem hiding this comment.
Can you provide a sample file that meets these criteria?
There was a problem hiding this comment.
Is tests/test_commands.csv an instance?
There was a problem hiding this comment.
Yes it was, but I've since replaced it with a file in src/cleanup that I actually used to clean up the latest dataset.
| command_lines = [] | ||
| with open(filename, 'r') as file: | ||
| for line in file: | ||
| line = line.rstrip('\n') | ||
| command_lines.append(line.split(',')) | ||
|
|
||
| for line in command_lines: | ||
| command = line[0] | ||
| columns = line[1:] | ||
| if getattr(args, command, None) == None: | ||
| setattr(args, command, columns) |
There was a problem hiding this comment.
@pneumatick Would this be too complicated to add to the project?
| parser.add_argument( | ||
| '--commandfile', | ||
| help="Use commands and arguments from a comma separated file. Manually entered commands will override those in the file." | ||
| ) |
There was a problem hiding this comment.
This seems like a good use case for a mutually exclusive group.
There was a problem hiding this comment.
From what I could glean from the documentation (which is fairly poor for this specific function) and people trying to use mutually exclusive groups to mimic the behavior of this command, argparse wouldn't be able to deal with it properly. From what I've read there isn't a way to add commands to a group and then add that group to a mutually exclusive group. In other words, every command other than --input, --output, and --commandfile would be placed in a group which would then be added to the mutually exclusive group, and then --commandfile would be added to the mutually exclusive group as well. I don't think that that is possible.
There was a problem hiding this comment.
I see. I guess you could add the logic you need and throw exceptions when needed?
if args.foo and (args.bar or args.baz):
parser.error('--foo cannot be used with --bar or --baz')There was a problem hiding this comment.
@pneumatick Any thoughts on this?
Co-authored-by: Carlos Paniagua <[email protected]>
| if args.commandfile != None: | ||
| args = commands_from_file(args) | ||
| if args.removews != None: | ||
| df = remove_whitespace(df, args.removews) | ||
| if args.tobool != None: |
There was a problem hiding this comment.
| if args.commandfile != None: | |
| args = commands_from_file(args) | |
| if args.removews != None: | |
| df = remove_whitespace(df, args.removews) | |
| if args.tobool != None: | |
| if args.commandfile is not None: | |
| args = commands_from_file(args) | |
| if args.removews is not None: | |
| df = remove_whitespace(df, args.removews) | |
| if args.tobool is not None: |
There was a problem hiding this comment.
It's been a while since it happened so I forget exactly what the issue was, but I had tried that initially and it caused the script to fail to execute certain commands. The script does need some slight modifications later anyway so I'll revisit that idea at that point.
| parser.add_argument( | ||
| '--commandfile', | ||
| help="Use commands and arguments from a comma separated file. Manually entered commands will override those in the file." | ||
| ) |
There was a problem hiding this comment.
@pneumatick Any thoughts on this?
| command_lines = [] | ||
| with open(filename, 'r') as file: | ||
| for line in file: | ||
| line = line.rstrip('\n') | ||
| command_lines.append(line.split(',')) | ||
|
|
||
| for line in command_lines: | ||
| command = line[0] | ||
| columns = line[1:] | ||
| if getattr(args, command, None) == None: | ||
| setattr(args, command, columns) |
There was a problem hiding this comment.
@pneumatick Would this be too complicated to add to the project?
closes #30