Install and configure SimFS in passive mode for testing the one month run on Piz Daint

Installing SimFS

1. SSH to Piz Daint

2. Install LUA

mkdir <dir where you want to install SimFS -- better NOT in $SCRATCH>
cd <dir where you want to install SimFS> 
wget https://www.lua.org/ftp/lua-5.3.4.tar.gz
tar -xvf lua-5.3.4.tar.gz
cd lua-5.3.4
make linux
export LUA_PATH=$(pwd)/src
cd ..

3. Clone this repo (ask Salvo to give you access rights)

git clone https://github.com/SalvatoreDiGirolamo/SimFS -b testing
cd SimFS

4. Configure SimFS

cat PATHS.in.daint | sed -e "s%__SDAVI_DIR__%$(pwd)/%" > PATHS.in

4. Compile DV and DVLib

./build_dv.sh
./build_dvlib_x86.sh --netcdf /opt/cray/pe/netcdf/4.4.1.1.3/generic
echo "export SIMFS_DIR=$(pwd)/" >> ~/.bashrc
echo "export PATH=\$PATH:\$SIMFS_DIR/build/" >> ~/.bashrc
exec bash

Virtualizing COSMO 2km

Prepare the simulation

1. Clone Salvo's cosmo-pompa (small change to run script in 04_lm_f -- needs PR)

cd <dir where you want cosmo -- probably somewhere in $SCRATCH>
git clone https://github.com/SalvatoreDiGirolamo/cosmo-pompa -b crclim

2. Executable and config file generation. The way I usually do it is:

cd cosmo-pompa/cosmo/test/climate/crClim2km_DVL/
cp -r /project/pr04/salvodg/one_month_testing/bin/ .
./run_daint.sh 
#wait that the script submits all the jobs -- there may be errors when launching the analysis tools

#stop the all jobs
scancel $USER

but probably there is a better way.

3. Copy input and restart files of the 2km simulation (stored in $PROJECT)

#This may take some time
cp /project/pr04/crclim_onemonth/output/lm2lm/* output/lm2lm/

#This takes time as well :(
cp -r /project/pr04/salvodg/cosmo_scratch_bak/restarts/ 04_lm_f/

4. Initialize the SimFS context

cd 04_lm_f/
simfs crclim_fine init $SIMFS_DIR/dv_config_files/crCLIM/lmf.dv

Run

This is the part that you have to repeat to perform new tests (and maybe re-copy the lm2lm/ if it got wiped from $SCRATCH).

1. Modify INPUT_IO as needed

2. Start DV

ssh daint101
screen
mkdir -p $SIMFS_DIR/logs
export SIMFS_LOG_LEVEL=2
export SIMFS_LOG="ERROR,INFO,CLIENT,SIMULATOR,CACHE,PREFETCHER"
simfs crclim_fine start_passive > $SIMFS_DIR/logs/simfs.log 2> $SIMFS_DIR/logs/simfs.err
#Press CTRL + A + D to detach
exit

3. Start the simulation job

#default duration is 1h, so it will be killed
sbatch job_withsimfs

Once the simulation finishes:

ssh daint101
screen -r
#Press CTRL + c to terminate DV
exit
exit

Processing Logs

For processing the logs you need to download them on your local machine and install R (if not already installed).

1. Install R (Ubuntu/Debian):

sudo apt-get install r-base

2. Download (scp) logs from $SIMFS_DIR/logs on Piz Daint to a local folder (will call it <logdir> in the following)

3. Download log_processing script from https://www.inf.ethz.ch/personal/salvatore.digirolamo/

#untar
tar -xvf logproc.tar
cd logproc

#process
./process_logs.sh <logdir>/simfs.log

It should print the table with the statistics computed from the logs and output a plot (in the same folder).