[WIP] branch with discrete crystal env experiments

[vict0rsch]

This is for running discrete env experiments.

This branch may not ever be merged, TBD.

Changes of interest:

• (See [👶 PR] Improve Launcher to handle SLURM_TMPDIR #228) launch.py now handles code_dir: $SLURM_TMPDIR or --code_dir='$SLURM_TMPDIR'
  • If this is used, the sbatch script will:
    1. git clone the repo
    2. git checkout whatever value is in git_checkout: or --git_checkout=. If git_checkout is empty, the current branch name is used
    3. 💥 From the command-line, single quotes are mandatory around '$SLURM_TMPDIR' otherwise bash will try to evaluate it instead of giving the raw string to Python
• (See [👶 PR] Handle logger notes in wandb #229) There is a new logger.notes key that can be used to describe a wandb experiment

Note: You can use environment variables in hydra (via OmegaConf), in particular if you want to see the $SLURM_JOB_ID somewhere you can do this:

${user.logdir.root}/workshop23/discrete-matbench/${oc.env:SLURM_JOB_ID,local}/${now:%Y-%m-%d_%H-%M-%S}

^ Here ${oc.env:SLURM_JOB_ID,local} will resolve to local if $SLURM_JOB_ID does not exist

[alexhernandezgarcia]

@vict0rsch is there anything here that is not in more recent branches and is worth keeping?

[vict0rsch]

I would keep the commits related to launch.py but I can cherry pick and make that in a new PR

[vict0rsch]

Moved to #292