2.0 - Test Config - Run

[pflarr]

Test Run Scripts and Configuration

When a test is to be run, a script will be generated (dependent on the scheduler), to setup the environment for the test, and run it. Several changes to the test section of test configs are necessary to support this. All of this is towards the goal of reducing the need for separate test wrapper scripts.

Cmd -> Cmds

We can now allow a sequence of shell (bash) commands for each test run. These can be used to generate the test data set, change directories as needed, and to add any other extra steps to the test execution.

If any of these commands should fail, the test should fail and the line of cmds that failed should be logged.

Test Args

Test args are now to be included directly in the test command lines. If multiple permutations of test args are needed, this can be accomplished with dynamic string substitution. The addition of multiple commands makes this necessary, as it's now unclear to which cmd test_args would apply.

env

A dictionary of key->value pairs, which will be exported as environment variables in the test run script.

modules

A list of modules to load, in the given order. See #51.

Other

• Move the 'scheduler' key to the test_config root.