Performance Analysis

You are here: Home > Developer Documentation > Performance Analysis

---

This page presents information on using performance analysis tools with PIConGPU.

Score-P

Update early 2016: Michael Sippel's Gist

Score-P is a measurement infrastructure combining several open-source performance analysis tools. It enables to trace and profile massively-parallel applications, including hybrid MPI+CUDA programs.

PIConGPU has cmake support for Score-P. When building and installing the measurement tool, be sure to enable support for MPI and CUDA (and CUPTI)

<user>:<scorep-build-dir>$ ./configure ... --enable-mpi --enable-cuda

and set the SCOREP_ROOT environment variable to the Score-P installation directory. When configuring PIConGPU, pass the -DSCOREP_ENABLE=ON as a configuration flag:

# important for the linker to keep CUDA instrumentation in
<user>:<pic-build-dir>$ export SCOREP_WRAPPER_INSTRUMENTER_FLAGS="--mpp=mpi --cuda --verbose"
<user>:<pic-build-dir>$ <pic-src-dir>/configure <pic-paramset> -c "-DSCOREP_ENABLE=ON"

Before executing PIConGPU, several Score-P environment variables must be set in your batch environment template script. Some template scripts already provide these environment variables, e.g. jaguar/batch_scorep_profile.tpl. For your own script, set at least the following (buffer sizes may vary):

export SCOREP_ENABLE_TRACING=yes
export SCOREP_CUDA_ENABLE=yes,flushatexit
export SCOREP_CUDA_BUFFER=200M
export SCOREP_TOTAL_MEMORY=1G
export SCOREP_FILTERING_FILE=!TBG_dstPath/tbg/scorep.filter

When successfull, a new directory called scorep-* is created which contains the trace file trace.otf2. The trace can than be visualized with Vampir.

Further information:

• Score-P website
• Online documentation
• Score-P training material slides