Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updated docs and graph functionality in symmer base #178

Merged
merged 3 commits into from
Jan 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 29 additions & 3 deletions docs/source/HPC.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,40 @@
Multiprocessing and HPC
=======================

Currently `symmer <https://github.com/UCL-CCS/symmer>`_ uses `ray <https://github.com/ray-project/ray>`_ to accelerate
Currently `symmer <https://github.com/UCL-CCS/symmer>`_ uses `ray <https://github.com/ray-project/ray>`_ by default to accelerate
the codebase by distributing problems over multiple cores if available. For standard use this works seamlessly and the user
doesn't need to do anything. However, when deploying on HPC systems a couple of things
need to be set. For further information see `link <https://docs.ray.io/en/latest/cluster/vms/user-guides/community/slurm.html>`_.
Below we comment on how to implement these tools on a SLURM system.
Below we comment on how to implement these tools on a SLURM system when ray is used. However, first we show an easier fix that will work on all linux based systems,
but can have issues with windows OS.

+++++
SLURM
Easy fix
+++++
Note these fixes can be implemented for local use on a laptop/desktop too. When importing symmer the following code
should be run before any other symmer functionality is used:

.. code-block:: bash

from symmer import process
process.method = 'mp' # for multiprocessing to be used instead of ray

An alternate fix is to turn multiprocessing off via:

.. code-block:: bash

from symmer import process
process.method = 'single_thread' # stops all multiprocessing

Finally, ray can be turned back by doing

.. code-block:: bash

from symmer import process
process.method = 'ray' # makes ray the multiprocessing library (set to this by default)

+++++
Ray and SLURM
+++++
Note these steps are **NOT** required for local use on a laptop/desktop.

Expand Down
Loading
Loading