Code coverage

[caleb-sitton-inl]

---

Pull Request Description

What issue does this change request address?

#376

What are the significant changes in functionality due to this change request?

This request implements the necessary changes to support functionality of a code coverage script (HERON/check_py_coverage.sh). A screenshot of a coverage report produced by this script is attached below. Running the script also prints the location of the complete resulting html report. The script checks the coverage both of HERON functions called before RAVEN outer is run as well as HERON functions called within the run of RAVEN inner (as a side note, these are mostly dispatch-related functions).

Pre-RAVEN outer coverage tracking

• A new coverage script has been added that runs RAVEN/run_tests with HERON tests only and provides the python command that is eventually passed as an argument to RAVEN/rook/main.py (note, rook uses this to set the python command in the testers). The source for coverage measurement is all files in the HERON/src/ directory and in the HERON/templates/ directory, as well as those files in any subdirectory of these two that has an __init__.py file inside it. Exceptions to this are explicitly listed as HERON/src/ARMABypass.py and the three twin_pyomo tests in HERON/src/dispatch/.
• A configuration file has been added to support the above mentioned coverage script.
• The README.md file in HERON/tests/ has been updated to document information about code coverage.
• The HERONIntegration tester has been modified to pass a command line argument to the HERON/heron shell script with a python command for running HERON/src/main.py.
• The HERON/heron shell script has been modified to use the python command provided in the new optional command line argument if it exists.

Coverage tracking during RAVEN inner

• The HERONIntegration tester has been modified to pass a second command line argument to the HERON/heron shell script providing the python command with which to run RAVEN/raven_framework.py on the RAVEN inner input xml script. Both command line arguments (this and the argument mentioned above to specify the python command for running HERON/src/main.py) are set to the same string, namely the python_command variable in the tester.
• This second command line argument, the python command for running RAVEN inner, is passed unmodified through the HERON/heron shell script to HERON/src/main.py. The HERON/src/main.py script has been modified to recognize this argument and set a variable in HERON/src/Cases.py to its value.
• The HERON/src/Cases.py script has been modified with an initialization, getter, and setter for the above mentioned variable.
• The HERON/templates/template_driver.py has been modified to check whether the case has a specified python command for running RAVEN (the variable in the above bullet). If it exists, a line is added to the outer.xml that prepends the python command to the command to run the RAVEN inner.

---

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

• 1. Review all computer code.
• 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
• 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
• 4. Automated Tests should pass.
• 5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large tes.
• 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
• 7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
• 8. If an analytic test is changed/added, the the analytic documentation must be updated/added.
• 9. If any test used as a basis for documentation examples have been changed, the associated documentation must be reviewed and assured the text matches the example.

[GabrielSoto-INL]

there might be a path issue in the main script, otherwise this work is looking great!

[GabrielSoto-INL]

just wondering if there is a scenario where the --python-comand and --python_command_for_raven would ever be different?

[caleb-sitton-inl]

I don't know of a current practical scenario where a user would want to make them different. To provide a relatively impractical example, this could allow users to check what functions in HERON are called from within the raven inner. This would be done by using some form of coverage run as the python_command_for_raven, but keeping python_command as something like python. The idea was to maximize the flexibility of the new functionality for future use cases. If this flexibility is removed, some of the changed files could be simplified, which could be worth it. Also, there would be no reduction of flexibility from the current code; prior to this PR, it has assumed that the python commands will be the same.

[GabrielSoto-INL]

I tried running this on my local machine, I believe the path to the establish_conda_env.sh here is set up to run with the HERON plugin version (i.e., the script thinks it is in raven/plugins/HERON and wants to find the raven/scripts/establish_conda_env.sh). this should be more generalized to run in a git cloned HERON version

[GabrielSoto-INL]

Changes LGTM