Using Latte on Cori

This guide walks through setting up Latte on the Cori supercomputer.

Quick links

• Module Dependencies
• Building Julia
• Building Latte
• CIFAR-10 Example
• Tiny-imagenet Example
• ImageNet Example

Module Dependencies

Recommend adding these to your ~/.bash_profile.ext instead of running this command every startup

module load python gcc autoconf cmake hdf5-parallel impi

Building Julia

git clone https://github.com/JuliaLang/julia.git
cd julia
git checkout tags/v0.4.3
printf "USEICC = 1\nUSEIFC = 1\nUSE_INTEL_MKL = 1\nUSE_INTEL_MKL_FFT = 1\nUSE_INTEL_LIBM = 1\n" >> Make.user

As of Feb 2016, this patch is required to build libuv. Check this julia issue for updates.

diff --git a/deps/Makefile b/deps/Makefile
index 051b23d..08b9d97 100644
--- a/deps/Makefile
+++ b/deps/Makefile
@@ -751,7 +751,7 @@ $(LIBUV_SRC_DIR)/config.status: $(LIBUV_SRC_DIR)/configure
        touch -c $(LIBUV_SRC_DIR)/Makefile.in
        touch -c $(LIBUV_SRC_DIR)/configure
        cd $(LIBUV_SRC_DIR) && \
-       ./configure --with-pic $(CONFIGURE_COMMON) $(UV_FLAGS)
+       ./configure --with-pic --disable-dtrace $(CONFIGURE_COMMON) $(UV_FLAGS)
        touch -c $@
 $(UV_SRC_TARGET): $(LIBUV_SRC_DIR)/config.status
        $(MAKE) -C $(LIBUV_SRC_DIR) $(UV_MFLAGS)

Build

make -j8

Add the build directory to your path, i.e.

export PATH=$HOME/julia:$PATH

Building Latte

$ julia -e 'Pkg.checkout("CompilerTools")'
$ julia -e 'Pkg.checkout("ParallelAccelerator")'
$ julia -e 'Pkg.clone("https://github.com/IntelLabs/Latte.jl")'

Build Latte's dependencies with MPI enabled

# Build the IO library
cd ~/.julia/v0.4/Latte/deps
cd IO
cmake -DLATTE_MPI=ON . && make
cp libLatteIO.so ../

# Build the communication library
cd ../communication
cmake . && make
cp libLatteComm.so ../

CIFAR-10 Example

Download and convert the dataset into hdf5

cd ~/.julia/v0.4/Latte/examples/cifar10/data
./get_data.sh $SCRATCH

Create a SLURM batch script called train_vgg_mini.sh

#!/bin/bash -l
#SBATCH -p debug
#SBATCH -N 2
#SBATCH -t 02:00:00
#SBATCH -J train_vgg_mini_cifar
srun -n 2 julia vgg-mini.jl

Submit a job

sbatch train_vgg_mini.sh

Tiny-imagenet Example

Build the image conversion utility

cd ~/.julia/v0.4/Latte/utils/converter
cmake . && make
cp convert_imagenet ../../bin

Download and convert the dataset

cd ~/.julia/v0.4/Latte/examples/tiny-imagenet/data
./get-data.sh $SCRATCH

ImageNet Example

Visit the ImageNet website to download the ILSVRC2012 training and validation images for Task 1 to $SCRATCH.