Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Capture and automatically load dependencies in modulefiles #5

Merged
merged 14 commits into from
Mar 10, 2021
Merged

Capture and automatically load dependencies in modulefiles #5

merged 14 commits into from
Mar 10, 2021

Conversation

terhorstd
Copy link
Collaborator

@terhorstd terhorstd commented Oct 23, 2020

This is fixes #3, by injecting multiple prereq statements into the modulefile template during its instantiation.

Issues with the multi-line prereq replacement have presumably been solved.

  • The mechanism to replace multi-line variables does not work in the "old bash" branch of the template instantiation.
  • Ideally the planfile could list 'build-time only' dependencies that should not be explicitly required at run time (e.g. compilers). (should be done in separate PR)

@terhorstd terhorstd marked this pull request as draft October 23, 2020 10:17
@terhorstd
Copy link
Collaborator Author

duplicate of #1

@terhorstd terhorstd closed this Oct 23, 2020
@terhorstd
Copy link
Collaborator Author

Re-opening from here to remove feature branches from central repository.

@terhorstd
Copy link
Collaborator Author

terhorstd commented Mar 4, 2021

There is also one new package "bats" which is used for testing (see #20 ). Though these are seemingly unrelated files the framework was essential to find the possible errors, so it ended up in this PR not the other. This also has the nice side effect that bats is already available when tests are added later.
The "activation" of this feature in all module files is also done in a more structured manner in #20.

@terhorstd terhorstd marked this pull request as ready for review March 4, 2021 22:10
@terhorstd terhorstd added the enhancement New feature or request label Mar 4, 2021
@terhorstd terhorstd mentioned this pull request Mar 4, 2021
@terhorstd terhorstd requested a review from stinebuu March 5, 2021 13:45
Copy link
Contributor

@stinebuu stinebuu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @terhorstd! This works for me, but I had to load quite a lot of modules, including ones that are not necessary for running NEST, like Cmake etc. This is not a problem, but something we can work on I think.

When building NEST, I loaded 3 modules (including CMake), and some of these loaded modules behind the scenes. When I then did module load nest-simulator/2.20.1, I then had to load all the hidden ones, because they where detected before the modules I had loaded manually. Also, when loading NEST, I got a message saying Lmod has detected the following error: Cannot load module "nest-simulator/2.20.1/default". At least one of these module(s) must be loaded: and then one module. It would be nice if in the future all of the modules needed where listed at the same time.

@terhorstd
Copy link
Collaborator Author

terhorstd commented Mar 10, 2021

Thanks for the review, @stinebuu !
Yes, currently there is no distinction between build-time and run-time dependencies. cmake is a very good example. Maybe we can add some information to the plan files that mask out build-time-only dependencies. I opened issue #21 for this.
Regarding the module loading, we can easily change prereq to module load, then it would automatically load dependent modules. I'll create a new PR for that, then we can decouple the discussion and look into it there. (-> #22)

@terhorstd terhorstd merged commit b1764fd into INM-6:master Mar 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Modulefile templates should automatically require dependant modules
2 participants