deploy: 2236274

_sources/olcf-andes.rst.txt

@@ -2,7 +2,7 @@ Batch Visualization on Andes
 ============================
 
 It is best to work on ``andes.olcf.ornl.gov``.  You will want to setup
-a new env for andes.  We'll call it ``andes_env``.
+a new env for andes.  We'll call it ``myenv``.
 
 You need to load python with anaconda support there:
 
@@ -24,6 +24,11 @@ this will modify your `.bashrc`, adding code that is specific to andes.
    The version of ``conda`` install on andes is very own, so it is best
    to install all other packages using pip in your new environment.
 
+.. note::
+
+   OLCF also has ``miniforge`` installed on Andes, although the documentation
+   is out of date regarding that: https://docs.olcf.ornl.gov/software/python/index.html
+
 To activate the environment, do:
 
 .. prompt:: bash
@@ -43,7 +48,7 @@ you need to do:
 
 .. prompt:: bash
 
-   source activate andes_env
+   source activate myenv
 
 Then you can run a python script that does visualization as with the
 following submission script::
@@ -56,7 +61,7 @@ following submission script::
 
     cd $SLURM_SUBMIT_DIR
 
-    source activate andes_env
+    source activate myenv
 
     srun python vol-xrb-enuc.py flame_wave_1000Hz_25cm_smallplt203204
 

_sources/olcf-compilers.rst.txt

@@ -67,18 +67,17 @@ load modules:
 .. prompt:: bash
 
    module load PrgEnv-gnu
-   module load cray-mpich/8.1.28
+   module load cray-mpich/8.1.27
    module load craype-accel-amd-gfx90a
-   module load amd-mixed/6.1.3
+   module load amd-mixed/6.0.0
    module unload darshan-runtime
 
-this will load ROCm 6.1.3
+this will load ROCm 6.0.0
 
 .. note::
 
-   In the past, tabulated rates seem to exhibit a strange slow down on
-   Frontier, so it is best to run test with and without rate
-   tabulation to see if there is a performance issue.
+   Tabulate rates seem to exhibit a strange slow down on Frontier, so
+   it is best to run without rate tabulation.
 
 build via:
 

_sources/olcf-workflow.rst.txt

@@ -502,23 +502,6 @@ https://github.com/AMReX-Astro/workflow/blob/main/job_scripts/frontier/frontier.
 Also see the WarpX docs: https://warpx.readthedocs.io/en/latest/install/hpc/frontier.html
 
 
-GPU-aware MPI
-^^^^^^^^^^^^^
-
-Some codes run better with GPU-aware MPI.  To enable this add the following to your
-submission script:
-
-.. code:: bash
-
-   export MPICH_GPU_SUPPORT_ENABLED=1
-   export FI_MR_CACHE_MONITOR=memhooks
-
-and set the runtime parameter:
-
-.. code::
-
-   amrex.use_gpu_aware_mpi=1
-
 Job Status
 ^^^^^^^^^^
 

olcf-andes.html

@@ -92,7 +92,7 @@
   <section id="batch-visualization-on-andes">
 <h1>Batch Visualization on Andes<a class="headerlink" href="#batch-visualization-on-andes" title="Link to this heading">¶</a></h1>
 <p>It is best to work on <code class="docutils literal notranslate"><span class="pre">andes.olcf.ornl.gov</span></code>.  You will want to setup
-a new env for andes.  We’ll call it <code class="docutils literal notranslate"><span class="pre">andes_env</span></code>.</p>
+a new env for andes.  We’ll call it <code class="docutils literal notranslate"><span class="pre">myenv</span></code>.</p>
 <p>You need to load python with anaconda support there:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><style type="text/css">
 span.prompt1:before {
@@ -108,6 +108,11 @@ <h1>Batch Visualization on Andes<a class="headerlink" href="#batch-visualization
 <p>The version of <code class="docutils literal notranslate"><span class="pre">conda</span></code> install on andes is very own, so it is best
 to install all other packages using pip in your new environment.</p>
 </div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>OLCF also has <code class="docutils literal notranslate"><span class="pre">miniforge</span></code> installed on Andes, although the documentation
+is out of date regarding that: <a class="reference external" href="https://docs.olcf.ornl.gov/software/python/index.html">https://docs.olcf.ornl.gov/software/python/index.html</a></p>
+</div>
 <p>To activate the environment, do:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="nb">source</span><span class="w"> </span>activate<span class="w"> </span>myenv</span>
 </pre></div></div><p>You can then install yt from source:</p>
@@ -116,7 +121,7 @@ <h1>Batch Visualization on Andes<a class="headerlink" href="#batch-visualization
 <span class="prompt1">pip<span class="w"> </span>install<span class="w"> </span>.</span>
 </pre></div></div><p>Each time you log in, if you want to use this environment,
 you need to do:</p>
-<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="nb">source</span><span class="w"> </span>activate<span class="w"> </span>andes_env</span>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1"><span class="nb">source</span><span class="w"> </span>activate<span class="w"> </span>myenv</span>
 </pre></div></div><p>Then you can run a python script that does visualization as with the
 following submission script:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>#!/bin/bash
@@ -127,7 +132,7 @@ <h1>Batch Visualization on Andes<a class="headerlink" href="#batch-visualization
 
 cd $SLURM_SUBMIT_DIR
 
-source activate andes_env
+source activate myenv
 
 srun python vol-xrb-enuc.py flame_wave_1000Hz_25cm_smallplt203204
 </pre></div>

olcf-compilers.html

@@ -146,16 +146,15 @@ <h2>Frontier<a class="headerlink" href="#frontier" title="Link to this heading">
 <p>see: <a class="reference external" href="https://docs.olcf.ornl.gov/systems/frontier_user_guide.html#programming-environment">https://docs.olcf.ornl.gov/systems/frontier_user_guide.html#programming-environment</a></p>
 <p>load modules:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>PrgEnv-gnu</span>
-<span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>cray-mpich/8.1.28</span>
+<span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>cray-mpich/8.1.27</span>
 <span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>craype-accel-amd-gfx90a</span>
-<span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>amd-mixed/6.1.3</span>
+<span class="prompt1">module<span class="w"> </span>load<span class="w"> </span>amd-mixed/6.0.0</span>
 <span class="prompt1">module<span class="w"> </span>unload<span class="w"> </span>darshan-runtime</span>
-</pre></div></div><p>this will load ROCm 6.1.3</p>
+</pre></div></div><p>this will load ROCm 6.0.0</p>
 <div class="admonition note">
 <p class="admonition-title">Note</p>
-<p>In the past, tabulated rates seem to exhibit a strange slow down on
-Frontier, so it is best to run test with and without rate
-tabulation to see if there is a performance issue.</p>
+<p>Tabulate rates seem to exhibit a strange slow down on Frontier, so
+it is best to run without rate tabulation.</p>
 </div>
 <p>build via:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">make<span class="w"> </span><span class="nv">USE_HIP</span><span class="o">=</span>TRUE</span>

olcf-workflow.html

@@ -70,7 +70,6 @@
 <li class="toctree-l3"><a class="reference internal" href="#frontier">Frontier</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="#machine-details">Machine details</a></li>
 <li class="toctree-l4"><a class="reference internal" href="#submitting-jobs">Submitting jobs</a></li>
-<li class="toctree-l4"><a class="reference internal" href="#gpu-aware-mpi">GPU-aware MPI</a></li>
 <li class="toctree-l4"><a class="reference internal" href="#job-status">Job Status</a></li>
 <li class="toctree-l4"><a class="reference internal" href="#job-chaining">Job Chaining</a></li>
 <li class="toctree-l4"><a class="reference internal" href="#debugging">Debugging</a></li>
@@ -601,19 +600,6 @@ <h3>Submitting jobs<a class="headerlink" href="#submitting-jobs" title="Link to
 <a class="reference external" href="https://github.com/AMReX-Astro/workflow/blob/main/job_scripts/frontier/frontier.slurm">https://github.com/AMReX-Astro/workflow/blob/main/job_scripts/frontier/frontier.slurm</a></p>
 <p>Also see the WarpX docs: <a class="reference external" href="https://warpx.readthedocs.io/en/latest/install/hpc/frontier.html">https://warpx.readthedocs.io/en/latest/install/hpc/frontier.html</a></p>
 </section>
-<section id="gpu-aware-mpi">
-<h3>GPU-aware MPI<a class="headerlink" href="#gpu-aware-mpi" title="Link to this heading">¶</a></h3>
-<p>Some codes run better with GPU-aware MPI.  To enable this add the following to your
-submission script:</p>
-<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nb">export</span><span class="w"> </span><span class="nv">MPICH_GPU_SUPPORT_ENABLED</span><span class="o">=</span><span class="m">1</span>
-<span class="nb">export</span><span class="w"> </span><span class="nv">FI_MR_CACHE_MONITOR</span><span class="o">=</span>memhooks
-</pre></div>
-</div>
-<p>and set the runtime parameter:</p>
-<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>amrex.use_gpu_aware_mpi<span class="o">=</span><span class="m">1</span>
-</pre></div>
-</div>
-</section>
 <section id="job-status">
 <h3>Job Status<a class="headerlink" href="#job-status" title="Link to this heading">¶</a></h3>
 <p>You can check on the status of your jobs via:</p>

olcf.html

@@ -115,7 +115,6 @@ <h1>Working at OLCF<a class="headerlink" href="#working-at-olcf" title="Link to
 <li class="toctree-l2"><a class="reference internal" href="olcf-workflow.html#frontier">Frontier</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="olcf-workflow.html#machine-details">Machine details</a></li>
 <li class="toctree-l3"><a class="reference internal" href="olcf-workflow.html#submitting-jobs">Submitting jobs</a></li>
-<li class="toctree-l3"><a class="reference internal" href="olcf-workflow.html#gpu-aware-mpi">GPU-aware MPI</a></li>
 <li class="toctree-l3"><a class="reference internal" href="olcf-workflow.html#job-status">Job Status</a></li>
 <li class="toctree-l3"><a class="reference internal" href="olcf-workflow.html#job-chaining">Job Chaining</a></li>
 <li class="toctree-l3"><a class="reference internal" href="olcf-workflow.html#debugging">Debugging</a></li>