MuSTEM5.3_PGIcommunity19.4

Repository for my source folder of MuSTEMv5.3 with working single Makefile for build with PGI Community 19.4 on Linux and Windows.

This folder consists of the MuSTEMv5.3 "Source" folder plus my own Makefile and pgi.env to build MuSTEM v5.3 on Windows10 and Ubuntu18.04 with PGI Community 19.4 for an RTX 2080Ti graphics card that requires Cuda 10.0 as a minimum.

Built TOK with

• Windows10 with Turing architecture NVidia GPU (RTX 2080Ti) with PGI Community compiler v19.4 and slightly modified version 5.3 of MuSTEM from github https://github.com/HamishGBrown/MuSTEM
• and Ubuntu 18.04 with Turing architecture NVidia GPU (RTX 2080Ti) with PGI Community compiler v19.4 and slightly modified version 5.3 of MuSTEM from github https://github.com/HamishGBrown/MuSTEM

Changes to original version 5.3 source code:

• OPEN (6, CARRIAGECONTROL = "FORTRAN") not supported by PGI 19.4
• line 145 in mustem.f90: change "OPEN (6, CARRIAGECONTROL = "FORTRAN")" to "Open(6)" was necessary to get it to compile. Then added *"call flush" * to enable overwriting on command line after an "*open (6,1,advance=no)*"
• added "call flush" to m_absorption.f90, m_multislice.f90, m_potential.f90, m_user_input.f90, s_absorptive_stem.f90, s_qep_stem.f90 and s_qep_tem.f90
• backward slash was hard-coded as path separator
  In order to make it OS transparent I added -DLIN preprocessor directive to Makefile and added a *"path_sep"* variable to m_multislice.f90 and m_potential.f90 to pick forward or backward slash as path separator depending on -DLIN preprocessor directive.
• PGI 19.4 memory allocation problem due to if(.not. ) one-liner statements
  PGI version > 18.1 gets fluffed by if(.not. ) statements on one line (without: then, newline, endif), see issue 17 on MuSTEM github => Replaced all one-liner if(.not. ) statements with full length statements

Prerequisites

Windows

Install

• MS Windows SDK
• Visual Studio Community 2017
• CUDA 10.1, PGI 19.4 Community

Download FFTW3 pre-compiled libraries and then create import libraries with

lib /machine:x64 /def:libfftwfl-3.def
lib /machine:x64 /def:libfftw3l-3.def
lib /machine:x64 /def:libfftw3-3.def

Linux

Install

• CUDA dependencies
• CUDA 10.1; "source cuda10.env"
• PGI 19.4 Community; "source pgi.env"
• compile FFTW3 libraries (or use OS supplied libraries)

pgi.env:
export PGI=/opt/pgi;
export LM_LICENSE_FILE="$LM_LICENSE_FILE":/opt/pgi/license.dat;
export PATH=/opt/pgi/linux86-64/19.4/bin:$PATH;
export PATH=/opt/openmpi_4.0.1_pgi/bin:$PATH

cuda.env:
export PATH=$PATH:/usr/local/cuda-10.1/bin
export CUDADIR=/usr/local/cuda-10.1
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-10.1/lib64

Build

Note to self: Don't forget to run "make clean" between builds

Windows

• open PGI cygwin terminal (this has all necessary environment variables set)
• edit top of Makefile to configure type of build (GPU/CPU,single,dble,Win/Lin,paths)
• "make"
• before a new build run "make clean" but don't forget to move the new executable out of the current folder
• add the fftw3 DLLs to your library search path!

Linux

• open bash terminal
• "source pgi.env"
• edit top of Makefile to configure type of build (GPU/CPU,single,dble,Win/Lin,paths)
• "make"
• before a new build run "make clean" but don't forget to move the new executable out of the current folder

Testing

I have added a folder "Testing" with sub-folders for Windows and Linux versions of the MuSTEM tutorial input files (DOS/Unix format).