Feature/multi loader

[nosnelmil]

Summary

A wrapper around loader to run multiple studies at once with additional features like validation, dry-run and log collection

Implementation Notes ⚒️

• Multi-Loader Entry Point:

  • MultiLoaderRunner houses the main logic of the multi-loader and exposes two execution points: RunDryRun and RunActual
  • multi_loader.go in tools/multiloader/ initializes the MultiLoaderRunner and calls the appropriate execution point

• Experiment Execution Flow:

  • The multi-loader runner is instantiated with the provided configuration path
  • Execution is split into three phases: pre-execution, execution, and post-execution
    • Pre-Execution: Executes global pre-scripts, sets up directories, and unpacks sub-experiments.
    • Execution: Runs experiments with the generated configurations and appropriate flags and collect loader.go logs.
    • Post-Execution: Executes experiment-specific post-scripts and performs necessary cleanup tasks.
  • If the execution is a dry run, the same flow is followed, but the dryRun flag is set to true when calling loader.go

• Post Execution:

  • Global post-scripts are executed after the experiments finish
  • Executes Make Clean
  • Temporary folders are removed

External Dependencies 🍀

• N/A

Breaking API Changes ⚠️

• N/A

Simply specify none (N/A) if not applicable.

[leokondrashov]

This is the first brief look at the code.

Please consider moving the multi-loader exclusive part away from the loader's common package. There is no point in storing a structure for a multi-loader there.

Another thing is gofmt; please set the VS Code to run it on all file saves. There are issues with alignment and whitespace around parentheses.

Please add GitHub workflows for the tests and add e2e tests. You can add separate workflow files for both. In e2e, please check the output hierarchy and the different settings used in the experiments.

Please review each function and ensure that the names make sense to someone who reads them for the first time.

For the failing spellcheck, you will need to add all the failing words (unless you misspelled them) into .github/configs/wordlist.txt.

[leokondrashov]

Great job. I've made couple of comments about minor things, please fix