Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added documentation on how to build on top of EESSI #118

Merged
merged 7 commits into from
Nov 7, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions docs/using_eessi/building_on_eessi.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Building software on top of EESSI

## Building software on top of EESSI with EasyBuild
Building on top of EESSI with EasyBuild is relatively straightforward. One crucial feature is that EasyBuild supports building against operating system libraries that are _not_ in a standard prefix (such as `/usr/lib`). This is required when building against EESSI, since all of the software in EESSI is build against the [compatibility layer](../compatibility_layer.md).

### Starting the EESSI software environment
Start your environment as described [here](../using_eessi/setting_up_environment.md)

### Configure EasyBuild
To configure EasyBuild, first, check out the Github repository. We advice you check out the branch corresponding to the version of EESSI you would like to use.
casparvl marked this conversation as resolved.
Show resolved Hide resolved

If you are unsure which version you are using, you can run
```
echo ${EESSI_PILOT_VERSION}
```
to check it.

To build on top of e.g. version `2023.06` of the EESSI software stack, we check it out, and go into that directory:

```
git clone https://github.com/EESSI/software-layer/ --branch 2023.06
cd software-layer
```
Then, you have to pick a working directory (that you have write access to) where EasyBuild can do the build, and an install directory, where EasyBuild can install it. In this example, we use `/tmp/easybuild` as our working directory, and `$HOME/.local/easybuild` as our installpath:
```
export WORKDIR=/tmp/easybuild
casparvl marked this conversation as resolved.
Show resolved Hide resolved
export EASYBUILD_INSTALLPATH="${HOME}/.local/easybuild"
casparvl marked this conversation as resolved.
Show resolved Hide resolved
source configure_easybuild
```
Next, you load the EasyBuild module that you want to use, e.g.
```
module load EasyBuild/4.8.1
ocaisa marked this conversation as resolved.
Show resolved Hide resolved
```
Finally, you can check the current configuration for EasyBuild using
```
eb --show-config
```

!!! Note
We use EasyBuild's default behaviour in optimizing for the host architecture. Since the EESSI initialization script also loads the EESSI stack that is optimized for your host architecture, this matches nicely. However, if you work on a cluster with heterogeneous node types, you have to realize you can only use these builds on the same architecture as where you build them. You can use different `EASYBUILD_INSTALLPATH`s if you want to build for different host architectures.
casparvl marked this conversation as resolved.
Show resolved Hide resolved

### Building
Now, you are ready to build. For example, at the time of writing, `netCDF-4.9.0-gompi-2022a.eb` was not in the EESSI environment yet, so you can build it yourself:
```
eb netCDF-4.9.0-gompi-2022a.eb
```

!!! Note
If this netCDF module is available by the time you are trying, you can force a local rebuild by adding the `--force` argument in order to experiment with building locally, or pick a different EasyConfig to build.

### Adding your module directory to the `MODULEPATH`
Finally, we need to make sure this module is available on our `MODULEPATH`. The path where EasyBuild installs modules by default is `${EASYBUILD_INSTALLPATH}/modules/all`. You can add it to your modulepath by running:
casparvl marked this conversation as resolved.
Show resolved Hide resolved
```
module use ${EASYBUILD_INSTALLPATH}/modules/all
```
You may want to do this as part of your `.bashrc`.

Now, we should be able to load our newly build module:
```
module load netCDF/4.9.0-gompi-2022a
casparvl marked this conversation as resolved.
Show resolved Hide resolved
```

!!! Note
Be careful adding to the `MODULEPATH` in your `.bashrc` if you are on a cluster with heterogeneous architectures. You don't want to pick up on a module that was not compiled for the correct architectures accidentally.

## Manually building software op top of EESSI
Building software on top of EESSI would require your linker to use the same system-dependencies as the software in EESSI does. In other words: it requires you to link against libraries from the compatibility layer, instead of from your host OS.

While we plan to support this in the future, manually building on top of EESSI is not supported yet.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ nav:
- Using EESSI:
- using_eessi/setting_up_environment.md
- using_eessi/basic_commands.md
- using_eessi/building_on_eessi.md
- using_eessi/eessi_demos.md
- Getting support: support.md
- Meetings:
Expand Down