From b6d418bf573cb30680963133defff2fb1ea96fed Mon Sep 17 00:00:00 2001 From: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com> Date: Thu, 10 Oct 2024 14:14:24 +0200 Subject: [PATCH 01/19] Add `dev.eessi.io.md` entry to repository list --- docs/repositories/dev.eessi.io.md | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 docs/repositories/dev.eessi.io.md diff --git a/docs/repositories/dev.eessi.io.md b/docs/repositories/dev.eessi.io.md new file mode 100644 index 000000000..8cf10a381 --- /dev/null +++ b/docs/repositories/dev.eessi.io.md @@ -0,0 +1,9 @@ +# `dev.eessi.io` repository + +## What is `dev.eessi.io`? + +`dev.eessi.io` is the development repository of EESSI. With it, developers can build and deploy non-production ready versions of their software to a CernVM-FS repository. +This way, development version can easily be tested on systems where `dev.eessi.io` is available. + +On a system with `dev.eessi.io` mounted access is possible with `module use /cvmfs/dev.eessi.io/versions/2023.06/modules/all`. Then, all that is left is +try out the development software! From 51421c0d4d7b09e349699b89cf1fd3f61b6d59bd Mon Sep 17 00:00:00 2001 From: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com> Date: Thu, 10 Oct 2024 15:15:20 +0200 Subject: [PATCH 02/19] Add `adding_development_software.md` `dev.eessi.io` overview and instructions --- .../adding_development_software.md | 80 +++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 docs/adding_software/adding_development_software.md diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md new file mode 100644 index 000000000..755d12232 --- /dev/null +++ b/docs/adding_software/adding_development_software.md @@ -0,0 +1,80 @@ +# Adding software to `dev.eessi.io` + +## What is `dev.eessi.io`? + +`dev.eessi.io` is the development repository of EESSI. With it, developers can build and deploy non-production ready versions of their software to a CernVM-FS repository. This way, development version can easily be tested on systems where `dev.eessi.io` is available. + +## Adding software + +Using `dev.eessi.io` is similar to using EESSI's production repository `software.eessi.io`. Software builds are triggered by a bot listening to pull requests in GitHub repository (at the moment https://github.com/EESSI/dev.eessi.io). This repository is organised by project, where corresponding easystack files and easyconfig files are placed. + +``` +dev.eessi.io +├── project1 +│   ├── easyconfigs +│   └── easystacks +└── project2 + ├── easyconfigs + └── easystacks +``` + +Creating a PR that adds an entry to an easystack file under on of the projects will allow authorised users to trigger +a build with the command `bot: build` through a GitHub comment. + +## Building development versions + +### easyconfig files and `--software-commit` +The approach to build and install software is similar to that of `software.eessi.io`. +It requires an easyconfig file which for `dev.eessi.io` need not be part of https://easybuilders/easybuild-easyconfigs +as these easyconfigs can simply be placed under `project/easyconfigs`. + +To allow for development builds, we leverage the `--software-commit` functionality (requires EasyBuild v4.9.3 or higher). This lets us build a given application from +a specific commit in repository. This can also be done from a fork, by changing the `github_account` field in the easyconfig file. +We've created a template for `ESPResSo` based on the standard eaasyconfig of the most recent version. The relevant fields are: + +``` python +easyblock = 'CMakeMake' + +name = 'ESPResSo' +version = '4.2.2' +versionsuffix = '-%(software_commit)s' + +homepage = 'https://espressomd.org/wordpress' +description = """A software package for performing and analyzing scientific Molecular Dynamics simulations.""" + +github_account = 'espressomd' +source_urls = ['https://github.com/%(github_account)s/%(name)s/archive/'] + +sources = ['%(software_commit)s.tar.gz'] +``` + +One can also make new changes to the easyconfig file, for example, if the new functionality requires new build or +runtime dependencies, patches, flags, etc. It's a good idea to try installing from a specific commit locally first, +to at least see if everything is parsed correctly and confirm that the right sources are being downloaded. + +### Easystack files and triggering builds + +After the easyconfig file has been created and added to `projectX/easyconfigs`, an easystack file that picks it up +needs to be in place. This easystack file must follow a naming convention: `software-eb-X.Y.Z-dev.yml`, +where X.Y.Z correspond to the EasyBuild version used to install the software. +Following our example for `ESPREsSo`, it would look like: + +``` yml +easyconfigs: + - ESPResSo-4.2.2-foss-2023a-software-commit.eb: + options: + software-commit: 2ba17de6096933275abec0550981d9122e4e5f28 # release 4.2.2 +``` + +The `ESPResSo-4.2.2-foss-2023a-software-commit.eb` would be the easyconfig file added in the step above. +Note the option passing the `software-commit` for the development version that should be built. +For the sake of this example, the chosen commit actually corresponds to the 4.2.2 release. + +To trigger a build, all one needs to do is open a PR with the changes adding the easyconfig and easystack +files and commenting `bot: build`. This can only be done by previously authorized users. +The current build cluster builds for the `zen2` CPU microarchitecture, but this is likely to change. + +Once a build is complete and the `bot:deploy` label is added, a staging PR can be merged to deploy the +application to the `dev.eessi.io` cvmfs repository. On a system with `dev.eessi.io` mounted, then all +that is left is to `module use /cvmfs/dev.eessi.io/versions/2023.06/modules/all` and try out the software! + From 5e353f805416aefcdb70fba7733ea010d678ac1b Mon Sep 17 00:00:00 2001 From: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com> Date: Thu, 10 Oct 2024 15:19:10 +0200 Subject: [PATCH 03/19] Add `dev.eessi.io` pages to side bar --- mkdocs.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mkdocs.yml b/mkdocs.yml index ea5a7c6c7..587905e19 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -28,6 +28,7 @@ nav: - Production: repositories/software.eessi.io.md - RISC-V: repositories/riscv.eessi.io.md - Pilot: repositories/pilot.md + - Development repository: repositories/dev.eessi.io.md - Installation and configuration: - Is EESSI already installed?: getting_access/is_eessi_accessible.md - Native: getting_access/native_installation.md @@ -63,6 +64,7 @@ nav: - Contribution policy: adding_software/contribution_policy.md - Open PR: adding_software/opening_pr.md - Debugging: adding_software/debugging_failed_builds.md + - Development repository: adding_software/adding_development_software.md - For maintainers: - BTD bot: bot.md - Building software: adding_software/building_software.md From f0313447f05b4bd5b8b4eed9827d4ac383eaaacd Mon Sep 17 00:00:00 2001 From: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com> Date: Thu, 10 Oct 2024 15:21:33 +0200 Subject: [PATCH 04/19] Update deploy.yml Build page from dev branch --- .github/workflows/deploy.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml index 38db175d5..06fff08b2 100644 --- a/.github/workflows/deploy.yml +++ b/.github/workflows/deploy.yml @@ -2,7 +2,7 @@ name: deploy documentation (only on push to main branch) on: push: - branches: main + branches: main, dev.eessi.io # Declare default permissions as read only. permissions: read-all jobs: From 8ad2f4722159fa2c8952b306c2727da173ffbc0a Mon Sep 17 00:00:00 2001 From: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com> Date: Thu, 10 Oct 2024 15:35:27 +0200 Subject: [PATCH 05/19] Update deploy.yml --- .github/workflows/deploy.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml index 06fff08b2..38db175d5 100644 --- a/.github/workflows/deploy.yml +++ b/.github/workflows/deploy.yml @@ -2,7 +2,7 @@ name: deploy documentation (only on push to main branch) on: push: - branches: main, dev.eessi.io + branches: main # Declare default permissions as read only. permissions: read-all jobs: From cf4048b8bf8b46ae00c368c723dd0b878bc49af2 Mon Sep 17 00:00:00 2001 From: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com> Date: Thu, 10 Oct 2024 16:00:09 +0200 Subject: [PATCH 06/19] Improve `dev.eessi.io` description Co-authored-by: Kenneth Hoste --- docs/repositories/dev.eessi.io.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/repositories/dev.eessi.io.md b/docs/repositories/dev.eessi.io.md index 8cf10a381..9b6e367df 100644 --- a/docs/repositories/dev.eessi.io.md +++ b/docs/repositories/dev.eessi.io.md @@ -3,7 +3,7 @@ ## What is `dev.eessi.io`? `dev.eessi.io` is the development repository of EESSI. With it, developers can build and deploy non-production ready versions of their software to a CernVM-FS repository. -This way, development version can easily be tested on systems where `dev.eessi.io` is available. +This way, development versions of software can easily be tested on systems where the `dev.eessi.io` CernVM-FS repository is available. On a system with `dev.eessi.io` mounted access is possible with `module use /cvmfs/dev.eessi.io/versions/2023.06/modules/all`. Then, all that is left is try out the development software! From 520bda1fb24b8c8f3b5646a4dbd377d051fbf716 Mon Sep 17 00:00:00 2001 From: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com> Date: Fri, 11 Oct 2024 11:27:51 +0200 Subject: [PATCH 07/19] Apply suggestions from code review Co-authored-by: Kenneth Hoste --- docs/adding_software/adding_development_software.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index 755d12232..8571f90df 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -18,10 +18,10 @@ dev.eessi.io └── easystacks ``` -Creating a PR that adds an entry to an easystack file under on of the projects will allow authorised users to trigger +Creating a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) that adds an entry to an easystack file under one of the projects will allow authorized users to trigger a build with the command `bot: build` through a GitHub comment. -## Building development versions +Deploying pre-release builds of software ### easyconfig files and `--software-commit` The approach to build and install software is similar to that of `software.eessi.io`. @@ -49,12 +49,12 @@ sources = ['%(software_commit)s.tar.gz'] ``` One can also make new changes to the easyconfig file, for example, if the new functionality requires new build or -runtime dependencies, patches, flags, etc. It's a good idea to try installing from a specific commit locally first, +runtime dependencies, patches, configuration options, etc. It's a good idea to try installing from a specific commit locally first, to at least see if everything is parsed correctly and confirm that the right sources are being downloaded. ### Easystack files and triggering builds -After the easyconfig file has been created and added to `projectX/easyconfigs`, an easystack file that picks it up +After the easyconfig file has been created and added to the `easyconfigs` subdirectory, an [easystack file](https://docs.easybuild.io/easystack-files) that picks it up needs to be in place. This easystack file must follow a naming convention: `software-eb-X.Y.Z-dev.yml`, where X.Y.Z correspond to the EasyBuild version used to install the software. Following our example for `ESPREsSo`, it would look like: From c6b3ac05588141dcc09d32d31ab03e2179bcf77e Mon Sep 17 00:00:00 2001 From: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com> Date: Fri, 11 Oct 2024 12:04:48 +0200 Subject: [PATCH 08/19] Improve repository description Co-authored-by: Kenneth Hoste --- docs/repositories/dev.eessi.io.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/repositories/dev.eessi.io.md b/docs/repositories/dev.eessi.io.md index 9b6e367df..680f5a861 100644 --- a/docs/repositories/dev.eessi.io.md +++ b/docs/repositories/dev.eessi.io.md @@ -2,7 +2,7 @@ ## What is `dev.eessi.io`? -`dev.eessi.io` is the development repository of EESSI. With it, developers can build and deploy non-production ready versions of their software to a CernVM-FS repository. +`dev.eessi.io` is the development repository of EESSI. With it, developers can deploy pre-release builds of their software to EESSI. This way, development versions of software can easily be tested on systems where the `dev.eessi.io` CernVM-FS repository is available. On a system with `dev.eessi.io` mounted access is possible with `module use /cvmfs/dev.eessi.io/versions/2023.06/modules/all`. Then, all that is left is From 964ab5b91bf1fa795b09a25514e5a64652113b61 Mon Sep 17 00:00:00 2001 From: Pedro Santos Neves <10762799+Neves-P@users.noreply.github.com> Date: Fri, 11 Oct 2024 12:10:54 +0200 Subject: [PATCH 09/19] Highlight easystack naming convention Co-authored-by: Kenneth Hoste --- .../adding_development_software.md | 24 +++++++++++-------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index 8571f90df..94051c516 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -55,16 +55,20 @@ to at least see if everything is parsed correctly and confirm that the right sou ### Easystack files and triggering builds After the easyconfig file has been created and added to the `easyconfigs` subdirectory, an [easystack file](https://docs.easybuild.io/easystack-files) that picks it up -needs to be in place. This easystack file must follow a naming convention: `software-eb-X.Y.Z-dev.yml`, -where X.Y.Z correspond to the EasyBuild version used to install the software. -Following our example for `ESPREsSo`, it would look like: - -``` yml -easyconfigs: - - ESPResSo-4.2.2-foss-2023a-software-commit.eb: - options: - software-commit: 2ba17de6096933275abec0550981d9122e4e5f28 # release 4.2.2 -``` +needs to be in place. + +!!! note "Naming convention for easystack files" + + The easystack files must follow a naming convention: `software-eb-X.Y.Z-dev.yml`, + where X.Y.Z correspond to the EasyBuild version used to install the software. + Following our example for `ESPREsSo`, it would look like: + + ``` yml + easyconfigs: + - ESPResSo-4.2.2-foss-2023a-software-commit.eb: + options: + software-commit: 2ba17de6096933275abec0550981d9122e4e5f28 # release 4.2.2 + ``` The `ESPResSo-4.2.2-foss-2023a-software-commit.eb` would be the easyconfig file added in the step above. Note the option passing the `software-commit` for the development version that should be built. From d9683f9649de5ab72bc92cfb91f094ad31c1e76b Mon Sep 17 00:00:00 2001 From: Neves-P Date: Mon, 14 Oct 2024 15:30:55 +0200 Subject: [PATCH 10/19] dev.eessi.io doc improvements given dev.eessi.io-example WIP --- .../adding_development_software.md | 33 +++++++++---------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index 94051c516..35a560a0b 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -2,20 +2,16 @@ ## What is `dev.eessi.io`? -`dev.eessi.io` is the development repository of EESSI. With it, developers can build and deploy non-production ready versions of their software to a CernVM-FS repository. This way, development version can easily be tested on systems where `dev.eessi.io` is available. +`dev.eessi.io` is the [development repository of EESSI](repositories/dev.eessi.io.md). ## Adding software -Using `dev.eessi.io` is similar to using EESSI's production repository `software.eessi.io`. Software builds are triggered by a bot listening to pull requests in GitHub repository (at the moment https://github.com/EESSI/dev.eessi.io). This repository is organised by project, where corresponding easystack files and easyconfig files are placed. +Using `dev.eessi.io` is similar to using EESSI's production repository `software.eessi.io`. Software builds are triggered by a [bot](https://www.eessi.io/docs/bot/) listening to pull requests in [GitHub repositories](https://github.com/search?q=org%3AEESSI+dev.eessi.io&type=repositories). Each repository , where corresponding easystack files and easyconfig files are placed. ``` -dev.eessi.io -├── project1 -│   ├── easyconfigs -│   └── easystacks -└── project2 - ├── easyconfigs - └── easystacks +dev.eessi.io-example +├── easyconfigs +└── easystacks ``` Creating a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) that adds an entry to an easystack file under one of the projects will allow authorized users to trigger @@ -28,7 +24,7 @@ The approach to build and install software is similar to that of `software.eessi It requires an easyconfig file which for `dev.eessi.io` need not be part of https://easybuilders/easybuild-easyconfigs as these easyconfigs can simply be placed under `project/easyconfigs`. -To allow for development builds, we leverage the `--software-commit` functionality (requires EasyBuild v4.9.3 or higher). This lets us build a given application from +To allow for development builds, we leverage the `--software-commit` functionality (requires [EasyBuild](https://easybuild.io/) v4.9.3 or higher). This lets us build a given application from a specific commit in repository. This can also be done from a fork, by changing the `github_account` field in the easyconfig file. We've created a template for `ESPResSo` based on the standard eaasyconfig of the most recent version. The relevant fields are: @@ -36,7 +32,7 @@ We've created a template for `ESPResSo` based on the standard eaasyconfig of the easyblock = 'CMakeMake' name = 'ESPResSo' -version = '4.2.2' +version = 'devel' versionsuffix = '-%(software_commit)s' homepage = 'https://espressomd.org/wordpress' @@ -50,7 +46,9 @@ sources = ['%(software_commit)s.tar.gz'] One can also make new changes to the easyconfig file, for example, if the new functionality requires new build or runtime dependencies, patches, configuration options, etc. It's a good idea to try installing from a specific commit locally first, -to at least see if everything is parsed correctly and confirm that the right sources are being downloaded. +to at least see if everything is parsed correctly and confirm that the right sources are being downloaded. Note that while you can +also build software from + ### Easystack files and triggering builds @@ -59,18 +57,19 @@ needs to be in place. !!! note "Naming convention for easystack files" - The easystack files must follow a naming convention: `software-eb-X.Y.Z-dev.yml`, - where X.Y.Z correspond to the EasyBuild version used to install the software. - Following our example for `ESPREsSo`, it would look like: + The easystack files must follow a naming convention and be named something + like: `software-eb-X.Y.Z-dev.yml`, where X.Y.Z correspond to the EasyBuild + version used to install the software. Following our example for + `ESPREsSo`, it would look like: ``` yml easyconfigs: - - ESPResSo-4.2.2-foss-2023a-software-commit.eb: + - ESPResSo-devel-foss-2023a-software-commit.eb: options: software-commit: 2ba17de6096933275abec0550981d9122e4e5f28 # release 4.2.2 ``` -The `ESPResSo-4.2.2-foss-2023a-software-commit.eb` would be the easyconfig file added in the step above. +The `ESPResSo-devel-foss-2023a-software-commit.eb` would be the easyconfig file added in the step above. Note the option passing the `software-commit` for the development version that should be built. For the sake of this example, the chosen commit actually corresponds to the 4.2.2 release. From c1ed77be5829ef5018df9edbad2104e683fa0a9e Mon Sep 17 00:00:00 2001 From: Neves-P Date: Mon, 14 Oct 2024 16:27:47 +0200 Subject: [PATCH 11/19] Process review comments Still todo: changes reflecting that cvmfs path includes project specific directory --- .../adding_development_software.md | 37 ++++++++++--------- docs/repositories/dev.eessi.io.md | 12 +++++- 2 files changed, 31 insertions(+), 18 deletions(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index 94051c516..1aabdd581 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -2,23 +2,25 @@ ## What is `dev.eessi.io`? -`dev.eessi.io` is the development repository of EESSI. With it, developers can build and deploy non-production ready versions of their software to a CernVM-FS repository. This way, development version can easily be tested on systems where `dev.eessi.io` is available. +`dev.eessi.io` is the [development repository of EESSI](repositories/dev.eessi.io.md). ## Adding software -Using `dev.eessi.io` is similar to using EESSI's production repository `software.eessi.io`. Software builds are triggered by a bot listening to pull requests in GitHub repository (at the moment https://github.com/EESSI/dev.eessi.io). This repository is organised by project, where corresponding easystack files and easyconfig files are placed. +Using `dev.eessi.io` is similar to using EESSI's production repository `software.eessi.io`. +Software builds are triggered by a [bot](https://www.eessi.io/docs/bot/) listening to pull +requests in [GitHub repositories](https://github.com/search?q=org%3AEESSI+dev.eessi.io&type=repositories). +Each GitHub repository, where corresponding easystack files and easyconfig files are placed. + +To see this in practice, refer to the [dev.eessi.io-example GitHub repository](https://github.com/EESSI/dev.eessi.io-example). In this GitHub repository you will find templates for some software installations and the appropriate directory structure, that is: ``` -dev.eessi.io -├── project1 -│   ├── easyconfigs -│   └── easystacks -└── project2 - ├── easyconfigs - └── easystacks +dev.eessi.io-example +├── easyconfigs +└── easystacks ``` -Creating a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) that adds an entry to an easystack file under one of the projects will allow authorized users to trigger +Creating a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) +that adds an entry to an easystack file under one of the projects will allow authorized users to trigger a build with the command `bot: build` through a GitHub comment. Deploying pre-release builds of software @@ -28,7 +30,7 @@ The approach to build and install software is similar to that of `software.eessi It requires an easyconfig file which for `dev.eessi.io` need not be part of https://easybuilders/easybuild-easyconfigs as these easyconfigs can simply be placed under `project/easyconfigs`. -To allow for development builds, we leverage the `--software-commit` functionality (requires EasyBuild v4.9.3 or higher). This lets us build a given application from +To allow for development builds, we leverage the `--software-commit` functionality (requires [EasyBuild](https://easybuild.io/) v4.9.3 or higher). This lets us build a given application from a specific commit in repository. This can also be done from a fork, by changing the `github_account` field in the easyconfig file. We've created a template for `ESPResSo` based on the standard eaasyconfig of the most recent version. The relevant fields are: @@ -36,7 +38,7 @@ We've created a template for `ESPResSo` based on the standard eaasyconfig of the easyblock = 'CMakeMake' name = 'ESPResSo' -version = '4.2.2' +version = 'devel' versionsuffix = '-%(software_commit)s' homepage = 'https://espressomd.org/wordpress' @@ -59,18 +61,19 @@ needs to be in place. !!! note "Naming convention for easystack files" - The easystack files must follow a naming convention: `software-eb-X.Y.Z-dev.yml`, - where X.Y.Z correspond to the EasyBuild version used to install the software. - Following our example for `ESPREsSo`, it would look like: + The easystack files must follow a naming convention and be named something + like: `software-eb-X.Y.Z-dev.yml`, where X.Y.Z correspond to the EasyBuild + version used to install the software. Following our example for + `ESPREsSo`, it would look like: ``` yml easyconfigs: - - ESPResSo-4.2.2-foss-2023a-software-commit.eb: + - ESPResSo-devel-foss-2023a-software-commit.eb: options: software-commit: 2ba17de6096933275abec0550981d9122e4e5f28 # release 4.2.2 ``` -The `ESPResSo-4.2.2-foss-2023a-software-commit.eb` would be the easyconfig file added in the step above. +The `ESPResSo-devel-foss-2023a-software-commit.eb` would be the easyconfig file added in the step above. Note the option passing the `software-commit` for the development version that should be built. For the sake of this example, the chosen commit actually corresponds to the 4.2.2 release. diff --git a/docs/repositories/dev.eessi.io.md b/docs/repositories/dev.eessi.io.md index 680f5a861..a748cb8fa 100644 --- a/docs/repositories/dev.eessi.io.md +++ b/docs/repositories/dev.eessi.io.md @@ -1,4 +1,4 @@ -# `dev.eessi.io` repository +# Development repository (`dev.eessi.io`) ## What is `dev.eessi.io`? @@ -7,3 +7,13 @@ This way, development versions of software can easily be tested on systems where On a system with `dev.eessi.io` mounted access is possible with `module use /cvmfs/dev.eessi.io/versions/2023.06/modules/all`. Then, all that is left is try out the development software! + +## Question or problems + +If you have any questions regarding EESSI, or if you experience a problem in accessing or using it, +please [open a support request](../support.md). If you experience issues with the development repository, feel free to use the #dev.eessi.io channel +of the EESSI Slack. + +## Infrastructure status + +The status of the CernVM-FS infrastructure for the production repository is shown at [https://status.eessi.io](https://status.eessi.io/). \ No newline at end of file From 42cc427172be25bfb88f8286a1bed3d78b07934f Mon Sep 17 00:00:00 2001 From: Neves-P Date: Thu, 17 Oct 2024 17:22:16 +0200 Subject: [PATCH 12/19] More progress on docs --- .../adding_development_software.md | 32 +++++++++++++++---- 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index 1aabdd581..f98715c1f 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -54,10 +54,27 @@ One can also make new changes to the easyconfig file, for example, if the new fu runtime dependencies, patches, configuration options, etc. It's a good idea to try installing from a specific commit locally first, to at least see if everything is parsed correctly and confirm that the right sources are being downloaded. -### Easystack files and triggering builds +### Installation details -After the easyconfig file has been created and added to the `easyconfigs` subdirectory, an [easystack file](https://docs.easybuild.io/easystack-files) that picks it up -needs to be in place. +There are a few additional details one should keep in mind when building for `dev.eessi.io` when compared to the [production repository](repositories/software.eessi.io.md). + +#### Versions, dependencies, tags + +Standard EESSI installations are + +#### Patch files + +If your specific development build requires patch files, you should add these to the `easyconfigs/` directory. If the necessary patch is part of an EasyBuild release, then this may not be necessary, as these will be directly taken from EasyBuild. + +#### Checksums + +EasyBuild's easyconfig files typically contain [checksums](https://docs.easybuild.io/writing-easyconfig-files/?h=checksums#common_easyconfig_param_sources_checksums) as their use is highly recommended. +By default, EasyBuild will compute the checksums of sources and patch files it needs for a given installation, and compare them with the values in the easyconfig file. Because builds +for `dev.eessi.io` change much more often, hard coded checksums become a problem, as they'd need to be updated with every new build. + +#### Easystack files +After an easyconfig file has been created and added to the `easyconfigs` subdirectory, an [easystack file](https://docs.easybuild.io/easystack-files) that picks it up +needs to be in place so that a build can be triggered. !!! note "Naming convention for easystack files" @@ -73,15 +90,18 @@ needs to be in place. software-commit: 2ba17de6096933275abec0550981d9122e4e5f28 # release 4.2.2 ``` -The `ESPResSo-devel-foss-2023a-software-commit.eb` would be the easyconfig file added in the step above. -Note the option passing the `software-commit` for the development version that should be built. +The `ESPResSo-devel-foss-2023a-software-commit.eb` would be the easyconfig file added in our example step above. +Note the option passing the `software-commit` for the development version that should be built. For the sake of this example, the chosen commit actually corresponds to the 4.2.2 release. +### Triggering builds + To trigger a build, all one needs to do is open a PR with the changes adding the easyconfig and easystack files and commenting `bot: build`. This can only be done by previously authorized users. -The current build cluster builds for the `zen2` CPU microarchitecture, but this is likely to change. +The current build cluster builds only for the `zen2` CPU microarchitecture, but this is likely to change. Once a build is complete and the `bot:deploy` label is added, a staging PR can be merged to deploy the application to the `dev.eessi.io` cvmfs repository. On a system with `dev.eessi.io` mounted, then all that is left is to `module use /cvmfs/dev.eessi.io/versions/2023.06/modules/all` and try out the software! +There is currently no initialisation script or module for `dev.eessi.io`, but this feature is coming soon. \ No newline at end of file From f6e22519b81db4a8f678b02ed3e5c7ba0117b248 Mon Sep 17 00:00:00 2001 From: Neves-P Date: Fri, 18 Oct 2024 18:23:29 +0200 Subject: [PATCH 13/19] Merge branch 'dev.eessi.io' of https://github.com/Neves-P/docs into HEAD --- .../adding_development_software.md | 27 ++++++++++++++----- 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index f98715c1f..71a4df92d 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -50,27 +50,42 @@ source_urls = ['https://github.com/%(github_account)s/%(name)s/archive/'] sources = ['%(software_commit)s.tar.gz'] ``` +!!! note `--software-commit` disables `--robot` + + Using `--software-commit` disables the use of `--robot`, so make sure that you explictly include + new dependencies that might need to be installed. Otherwise, the easyconfig files won't be found. + One can also make new changes to the easyconfig file, for example, if the new functionality requires new build or runtime dependencies, patches, configuration options, etc. It's a good idea to try installing from a specific commit locally first, to at least see if everything is parsed correctly and confirm that the right sources are being downloaded. ### Installation details -There are a few additional details one should keep in mind when building for `dev.eessi.io` when compared to the [production repository](repositories/software.eessi.io.md). +While the prcess to build for `dev.eessi.io` is similar to the one for the [production repository](repositories/software.eessi.io.md) there +are a few additional details to keep in mind. #### Versions, dependencies, tags -Standard EESSI installations are +Installations to the EESSI production repository refer to specific versions of applications, however, Development builds can't follow the same +approach as they are most often not pegged to a release. Because of this, it is possible to add a descriptive "version" name to the `version` field +in the easyconfig file for a give (set of) installations. Note that some applications are built with custom easyblocks which may +use the `version` field to determine how the installation is meant to work (e.g., versions >2 need to copy files from to a new directory). Make sure +that you account for this otherwise you may install software differently than intended. If you encounter issues, you can open an issue in our +[support portal](https://gitlab.com/eessi/support#eessi-support-portal). #### Patch files -If your specific development build requires patch files, you should add these to the `easyconfigs/` directory. If the necessary patch is part of an EasyBuild release, then this may not be necessary, as these will be directly taken from EasyBuild. +If your specific development build requires patch files, you should add these to the `easyconfigs/` directory. If the necessary patch is part of an +EasyBuild release, then this may not be necessary, as these will be directly taken from EasyBuild. If it is a new patch that is not on an EasyBuild +release, then include it in the `easyconfigs/` directory. #### Checksums -EasyBuild's easyconfig files typically contain [checksums](https://docs.easybuild.io/writing-easyconfig-files/?h=checksums#common_easyconfig_param_sources_checksums) as their use is highly recommended. -By default, EasyBuild will compute the checksums of sources and patch files it needs for a given installation, and compare them with the values in the easyconfig file. Because builds -for `dev.eessi.io` change much more often, hard coded checksums become a problem, as they'd need to be updated with every new build. +EasyBuild's easyconfig files typically contain [checksums](https://docs.easybuild.io/writing-easyconfig-files/?h=checksums#common_easyconfig_param_sources_checksums) + as their use is highly recommended. By default, EasyBuild will compute the checksums of sources and patch files it needs for + a given installation, and compare them with the values in the easyconfig file. Because builds for `dev.eessi.io` change much + more often, hard coded checksums become a problem, as they'd need to be updated with every new build. For this reason, we + recommend not including checksums in your development easyconfig files (unless you need to, for a specific reason). #### Easystack files After an easyconfig file has been created and added to the `easyconfigs` subdirectory, an [easystack file](https://docs.easybuild.io/easystack-files) that picks it up From 0e949865fff68fcab897b7b7d4dfc53125d6c55c Mon Sep 17 00:00:00 2001 From: Neves-P Date: Fri, 18 Oct 2024 18:51:37 +0200 Subject: [PATCH 14/19] better formatting and removing duplicate information --- .../adding_development_software.md | 42 ++++++++++++------- 1 file changed, 26 insertions(+), 16 deletions(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index 71a4df92d..d0cc33cb9 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -2,16 +2,17 @@ ## What is `dev.eessi.io`? -`dev.eessi.io` is the [development repository of EESSI](repositories/dev.eessi.io.md). +`dev.eessi.io` is the [development repository of EESSI](../repositories/dev.eessi.io.md). ## Adding software Using `dev.eessi.io` is similar to using EESSI's production repository `software.eessi.io`. Software builds are triggered by a [bot](https://www.eessi.io/docs/bot/) listening to pull requests in [GitHub repositories](https://github.com/search?q=org%3AEESSI+dev.eessi.io&type=repositories). -Each GitHub repository, where corresponding easystack files and easyconfig files are placed. +These builds require custom easyconfig and easystack files, which should be in specific directories. -To see this in practice, refer to the [dev.eessi.io-example GitHub repository](https://github.com/EESSI/dev.eessi.io-example). In this GitHub repository you will find templates for some software installations and the appropriate directory structure, that is: +To see this in practice, refer to the [dev.eessi.io-example GitHub repository](https://github.com/EESSI/dev.eessi.io-example). + In this GitHub repository you will find templates for some software installations with the appropriate directory structure, that is: ``` dev.eessi.io-example @@ -19,16 +20,11 @@ dev.eessi.io-example └── easystacks ``` -Creating a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) -that adds an entry to an easystack file under one of the projects will allow authorized users to trigger -a build with the command `bot: build` through a GitHub comment. - -Deploying pre-release builds of software - ### easyconfig files and `--software-commit` The approach to build and install software is similar to that of `software.eessi.io`. -It requires an easyconfig file which for `dev.eessi.io` need not be part of https://easybuilders/easybuild-easyconfigs -as these easyconfigs can simply be placed under `project/easyconfigs`. +It requires one or more easyconfig files. Easybuild files used for building for `dev.eessi.io` +do not need to be a part of an [EasyBuild release](https://easybuilders/easybuild-easyconfigs), unlike builds for +`software.eessi.io`. In this case, the development easyconfigs can simply be placed under `project/easyconfigs`. To allow for development builds, we leverage the `--software-commit` functionality (requires [EasyBuild](https://easybuild.io/) v4.9.3 or higher). This lets us build a given application from a specific commit in repository. This can also be done from a fork, by changing the `github_account` field in the easyconfig file. @@ -50,7 +46,7 @@ source_urls = ['https://github.com/%(github_account)s/%(name)s/archive/'] sources = ['%(software_commit)s.tar.gz'] ``` -!!! note `--software-commit` disables `--robot` +!!! warning "`--software-commit` disables `--robot`" Using `--software-commit` disables the use of `--robot`, so make sure that you explictly include new dependencies that might need to be installed. Otherwise, the easyconfig files won't be found. @@ -61,18 +57,32 @@ to at least see if everything is parsed correctly and confirm that the right sou ### Installation details -While the prcess to build for `dev.eessi.io` is similar to the one for the [production repository](repositories/software.eessi.io.md) there +While the process to build for `dev.eessi.io` is similar to the one for the [production repository](../repositories/software.eessi.io.md) there are a few additional details to keep in mind. -#### Versions, dependencies, tags +#### Versions Installations to the EESSI production repository refer to specific versions of applications, however, Development builds can't follow the same approach as they are most often not pegged to a release. Because of this, it is possible to add a descriptive "version" name to the `version` field -in the easyconfig file for a give (set of) installations. Note that some applications are built with custom easyblocks which may +in the easyconfig file for a give (set of) installations. + +Note that some applications are built with custom easyblocks which may use the `version` field to determine how the installation is meant to work (e.g., versions >2 need to copy files from to a new directory). Make sure -that you account for this otherwise you may install software differently than intended. If you encounter issues, you can open an issue in our +that you account for this, otherwise you may install software differently than intended. If you encounter issues, you can open an issue in our [support portal](https://gitlab.com/eessi/support#eessi-support-portal). +#### Installing dependencies + +Installations in `dev.eessi.io` are done _on top_ of `software.eessi.io`. That means if your development build depends on some application that is +already installed in `software.eessi.io`, then that will simply be used. However, if you need to add a new dependency, then this must included as +part of the build. That means including an easybuild file for it, and adding it to the right easystack file. + +#### Using commit IDs or tags for `--software-commit` + +Installing with `--software-commit` requires that you include either a commit ID or a tag. The installation procedure will use the commit ID or tag to +obtain the source from the right state. Because tags can be changed to point to a different commit ID, we recommend you avoid using them but stick with +the commit ID itself. You can then include this in the `versionsuffix` on your easyconfig file, to generate a unique (if ugly) module name. + #### Patch files If your specific development build requires patch files, you should add these to the `easyconfigs/` directory. If the necessary patch is part of an From b8517b6e15070402f79c4f106819955197af7968 Mon Sep 17 00:00:00 2001 From: Neves-P Date: Fri, 18 Oct 2024 19:23:00 +0200 Subject: [PATCH 15/19] Add quick start guide --- .../adding_development_software.md | 25 +++++++++++++++---- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index d0cc33cb9..f78173577 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -20,7 +20,22 @@ dev.eessi.io-example └── easystacks ``` -### easyconfig files and `--software-commit` +### Quick steps to build for `dev.eessi.io` + +- Obtain commit ID from GitHub or GitLab repository with source to build +- Fork the project's `dev.eessi.io` repository on GitHub, or checkout to a new branch if you can do so. +- If needed, prepare an easyconfig template using `--software-commit` and add it to `easyconfigs/` +- Add an easystack file in `easystacks/` that with the easyconfig file above, add the +commit ID to `software-commit` under `options`. +- Open a PR from the fork or branch to the main branch of the application's `dev.eessi.io` GitHub repository. +- Instruct the bot to start a build by adding a comment with `bot: build`. +- Confirm the build worked correctly. If so, you can deploy the software by adding the label `bot:build` to the +pull request. +- Once the staging PR is approved, the development build will become available on `dev.eessi.io` in a few minutes! + +### Installation details + +#### easyconfig files and `--software-commit` The approach to build and install software is similar to that of `software.eessi.io`. It requires one or more easyconfig files. Easybuild files used for building for `dev.eessi.io` do not need to be a part of an [EasyBuild release](https://easybuilders/easybuild-easyconfigs), unlike builds for @@ -55,7 +70,6 @@ One can also make new changes to the easyconfig file, for example, if the new fu runtime dependencies, patches, configuration options, etc. It's a good idea to try installing from a specific commit locally first, to at least see if everything is parsed correctly and confirm that the right sources are being downloaded. -### Installation details While the process to build for `dev.eessi.io` is similar to the one for the [production repository](../repositories/software.eessi.io.md) there are a few additional details to keep in mind. @@ -64,7 +78,7 @@ are a few additional details to keep in mind. Installations to the EESSI production repository refer to specific versions of applications, however, Development builds can't follow the same approach as they are most often not pegged to a release. Because of this, it is possible to add a descriptive "version" name to the `version` field -in the easyconfig file for a give (set of) installations. +in the easyconfig file for a given (set of) installations. Note that some applications are built with custom easyblocks which may use the `version` field to determine how the installation is meant to work (e.g., versions >2 need to copy files from to a new directory). Make sure @@ -75,7 +89,7 @@ that you account for this, otherwise you may install software differently than i Installations in `dev.eessi.io` are done _on top_ of `software.eessi.io`. That means if your development build depends on some application that is already installed in `software.eessi.io`, then that will simply be used. However, if you need to add a new dependency, then this must included as -part of the build. That means including an easybuild file for it, and adding it to the right easystack file. +part of the build. That means including an easystack file for it, and adding it to the right easystack file. #### Using commit IDs or tags for `--software-commit` @@ -121,7 +135,8 @@ For the sake of this example, the chosen commit actually corresponds to the 4.2. ### Triggering builds -To trigger a build, all one needs to do is open a PR with the changes adding the easyconfig and easystack +We use the [EESSI build-test-deploy bot](../bot.md) to handle software builds. +All one needs to do is open a PR with the changes adding the easyconfig and easystack files and commenting `bot: build`. This can only be done by previously authorized users. The current build cluster builds only for the `zen2` CPU microarchitecture, but this is likely to change. From 90854848dd3a4b497be47e00854b3641f22974ff Mon Sep 17 00:00:00 2001 From: Neves-P Date: Mon, 21 Oct 2024 13:40:15 +0200 Subject: [PATCH 16/19] Add "in development" warning --- docs/adding_software/adding_development_software.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index f78173577..4dbb10eed 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -1,5 +1,14 @@ # Adding software to `dev.eessi.io` +!!! warning "`dev.eessi.io` is still in active development and focused on MultiXscale" + + The `dev.eessi.io` repository and functionality is still in its earlie stages. + The repository, build, and deploy procedures are functional, but may change + often for the time being. + + Our focus is currently on including and supporting developers and applications in the + [MultiXscale CoE](https://multixscale.eu). + ## What is `dev.eessi.io`? `dev.eessi.io` is the [development repository of EESSI](../repositories/dev.eessi.io.md). @@ -63,7 +72,7 @@ sources = ['%(software_commit)s.tar.gz'] !!! warning "`--software-commit` disables `--robot`" - Using `--software-commit` disables the use of `--robot`, so make sure that you explictly include + Using `--software-commit` disables the use of `--robot`, so make sure that you explicitly include new dependencies that might need to be installed. Otherwise, the easyconfig files won't be found. One can also make new changes to the easyconfig file, for example, if the new functionality requires new build or From b6a877e03f2f02d1dac0a34e81eb26c3854c348c Mon Sep 17 00:00:00 2001 From: Neves-P Date: Mon, 21 Oct 2024 13:58:13 +0200 Subject: [PATCH 17/19] Lint --- docs/adding_software/adding_development_software.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index 4dbb10eed..ff4222806 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -31,15 +31,14 @@ dev.eessi.io-example ### Quick steps to build for `dev.eessi.io` -- Obtain commit ID from GitHub or GitLab repository with source to build +- Obtain commit ID from GitHub or GitLab repository with source to build. - Fork the project's `dev.eessi.io` repository on GitHub, or checkout to a new branch if you can do so. - If needed, prepare an easyconfig template using `--software-commit` and add it to `easyconfigs/` - Add an easystack file in `easystacks/` that with the easyconfig file above, add the commit ID to `software-commit` under `options`. - Open a PR from the fork or branch to the main branch of the application's `dev.eessi.io` GitHub repository. - Instruct the bot to start a build by adding a comment with `bot: build`. -- Confirm the build worked correctly. If so, you can deploy the software by adding the label `bot:build` to the -pull request. +- Confirm the build worked correctly. If so, you can deploy the software by adding the label `bot:build` to the pull request. - Once the staging PR is approved, the development build will become available on `dev.eessi.io` in a few minutes! ### Installation details From 5c8ebb9c30ecea7eda02241b271cf6af42b7d0a0 Mon Sep 17 00:00:00 2001 From: Neves-P Date: Mon, 21 Oct 2024 15:30:43 +0200 Subject: [PATCH 18/19] Fix broken link --- docs/adding_software/adding_development_software.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index ff4222806..38ee2b4b8 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -46,7 +46,7 @@ commit ID to `software-commit` under `options`. #### easyconfig files and `--software-commit` The approach to build and install software is similar to that of `software.eessi.io`. It requires one or more easyconfig files. Easybuild files used for building for `dev.eessi.io` -do not need to be a part of an [EasyBuild release](https://easybuilders/easybuild-easyconfigs), unlike builds for +do not need to be a part of an [EasyBuild release](https://github.com/easybuilders/easybuild-easyconfigs), unlike builds for `software.eessi.io`. In this case, the development easyconfigs can simply be placed under `project/easyconfigs`. To allow for development builds, we leverage the `--software-commit` functionality (requires [EasyBuild](https://easybuild.io/) v4.9.3 or higher). This lets us build a given application from From 81533f8146a4aed5391f25d768ad524a07146eec Mon Sep 17 00:00:00 2001 From: Kenneth Hoste Date: Sat, 16 Nov 2024 13:25:32 +0100 Subject: [PATCH 19/19] minor fixes/changed to docs for dev.eessi.io repo --- .../adding_development_software.md | 38 +++++++++---------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/adding_software/adding_development_software.md b/docs/adding_software/adding_development_software.md index 38ee2b4b8..b3716b02b 100644 --- a/docs/adding_software/adding_development_software.md +++ b/docs/adding_software/adding_development_software.md @@ -2,8 +2,8 @@ !!! warning "`dev.eessi.io` is still in active development and focused on MultiXscale" - The `dev.eessi.io` repository and functionality is still in its earlie stages. - The repository, build, and deploy procedures are functional, but may change + The `dev.eessi.io` repository and functionality is still in its early stages. + The repository itself and build + deploy procedure for it are functional, but may change often for the time being. Our focus is currently on including and supporting developers and applications in the @@ -45,9 +45,9 @@ commit ID to `software-commit` under `options`. #### easyconfig files and `--software-commit` The approach to build and install software is similar to that of `software.eessi.io`. -It requires one or more easyconfig files. Easybuild files used for building for `dev.eessi.io` +It requires one or more easyconfig files. Easyconfig files used for building for `dev.eessi.io` do not need to be a part of an [EasyBuild release](https://github.com/easybuilders/easybuild-easyconfigs), unlike builds for -`software.eessi.io`. In this case, the development easyconfigs can simply be placed under `project/easyconfigs`. +`software.eessi.io`. In this case, the development easyconfigs can be located under `easyconfigs/` in the `dev.eessi.io` repository being used. To allow for development builds, we leverage the `--software-commit` functionality (requires [EasyBuild](https://easybuild.io/) v4.9.3 or higher). This lets us build a given application from a specific commit in repository. This can also be done from a fork, by changing the `github_account` field in the easyconfig file. @@ -74,22 +74,22 @@ sources = ['%(software_commit)s.tar.gz'] Using `--software-commit` disables the use of `--robot`, so make sure that you explicitly include new dependencies that might need to be installed. Otherwise, the easyconfig files won't be found. -One can also make new changes to the easyconfig file, for example, if the new functionality requires new build or +You can also make additional changes to the easyconfig file, for example, if the new functionality requires new build or runtime dependencies, patches, configuration options, etc. It's a good idea to try installing from a specific commit locally first, to at least see if everything is parsed correctly and confirm that the right sources are being downloaded. -While the process to build for `dev.eessi.io` is similar to the one for the [production repository](../repositories/software.eessi.io.md) there +While the process to build for `dev.eessi.io` is similar to the one for the [production repository](../repositories/software.eessi.io.md), there are a few additional details to keep in mind. -#### Versions +#### Software version -Installations to the EESSI production repository refer to specific versions of applications, however, Development builds can't follow the same -approach as they are most often not pegged to a release. Because of this, it is possible to add a descriptive "version" name to the `version` field +Installations to the EESSI production repository refer to specific versions of applications. However, development builds can't follow the same +approach as they are most often not pegged to a release. Because of this, it is possible to include a descriptive "version" label to the `version` parameter in the easyconfig file for a given (set of) installations. -Note that some applications are built with custom easyblocks which may -use the `version` field to determine how the installation is meant to work (e.g., versions >2 need to copy files from to a new directory). Make sure +Note that some applications are built with custom easyblocks, which may +use the `version` parameter to determine how the installation is meant to work (for example, recent versions need to copy files from to a new directory). Make sure that you account for this, otherwise you may install software differently than intended. If you encounter issues, you can open an issue in our [support portal](https://gitlab.com/eessi/support#eessi-support-portal). @@ -97,13 +97,13 @@ that you account for this, otherwise you may install software differently than i Installations in `dev.eessi.io` are done _on top_ of `software.eessi.io`. That means if your development build depends on some application that is already installed in `software.eessi.io`, then that will simply be used. However, if you need to add a new dependency, then this must included as -part of the build. That means including an easystack file for it, and adding it to the right easystack file. +part of the build. That means including an easyconfig file for it, and adding it to the right easystack file. #### Using commit IDs or tags for `--software-commit` -Installing with `--software-commit` requires that you include either a commit ID or a tag. The installation procedure will use the commit ID or tag to -obtain the source from the right state. Because tags can be changed to point to a different commit ID, we recommend you avoid using them but stick with -the commit ID itself. You can then include this in the `versionsuffix` on your easyconfig file, to generate a unique (if ugly) module name. +Installing with `--software-commit` requires that you include either a commit ID or a tag. The installation procedure will use this to +obtain the sources for the build. Because tags can be changed to point to a different commit ID, we recommend you avoid using them and sticking to +the commit ID itself. You can then include this in the `versionsuffix` on your easyconfig file, to generate a unique (though "ugly") module name. #### Patch files @@ -130,23 +130,23 @@ needs to be in place so that a build can be triggered. version used to install the software. Following our example for `ESPREsSo`, it would look like: - ``` yml + ```yaml easyconfigs: - ESPResSo-devel-foss-2023a-software-commit.eb: options: software-commit: 2ba17de6096933275abec0550981d9122e4e5f28 # release 4.2.2 ``` -The `ESPResSo-devel-foss-2023a-software-commit.eb` would be the easyconfig file added in our example step above. +`ESPResSo-devel-foss-2023a-software-commit.eb` would be the name of the easyconfig file added in our example step above. Note the option passing the `software-commit` for the development version that should be built. -For the sake of this example, the chosen commit actually corresponds to the 4.2.2 release. +For the sake of this example, the chosen commit actually corresponds to the [4.2.2 release of ESPResSo](https://github.com/espressomd/espresso/releases/tag/4.2.2). ### Triggering builds We use the [EESSI build-test-deploy bot](../bot.md) to handle software builds. All one needs to do is open a PR with the changes adding the easyconfig and easystack files and commenting `bot: build`. This can only be done by previously authorized users. -The current build cluster builds only for the `zen2` CPU microarchitecture, but this is likely to change. +The current build cluster for `dev.eessi.io` builds only for the `zen2` CPU microarchitecture, but this is likely to change. Once a build is complete and the `bot:deploy` label is added, a staging PR can be merged to deploy the application to the `dev.eessi.io` cvmfs repository. On a system with `dev.eessi.io` mounted, then all