Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature request]: Generic Cluster Config #342

Open
TimothyWillard opened this issue Oct 11, 2024 · 2 comments
Open

[Feature request]: Generic Cluster Config #342

TimothyWillard opened this issue Oct 11, 2024 · 2 comments
Assignees
Labels
good first issue Good for newcomers. low priority Low priority. meta/workflow Relating to CI / issue templates / testing frameworks / etc.

Comments

@TimothyWillard
Copy link
Contributor

Label

good first issue, meta/workflow

Priority Label

low priority

Is your feature request related to a problem? Please describe.

Currently we have hard coded knowledge about rockfish and to a lesser extent longleaf throughout the repo. Which makes updating said info or adding new HPC environments painful. Has been described in other detail here: #329 (comment).

Is your feature request related to a new application, scenario round, pathogen? Please describe.

n/a

Describe the solution you'd like

Place config files, maybe yaml or something else, in a central place keyed on environment name. Where these should go is still an open question, viable options are:

  1. Place somewhere in this repository, or
  2. Create a new repository.

I have a preference for (2) since this repository is already a bit bloated and updates to config would be small and need to be quickly done. Some information that would be helpful to know about each HPC environment in these config files would be:

  1. Preferred module versions for gcc, git, anaconda, etc.
  2. Some light information the the directory structure (i.e. where to normally expect flepiMoP and a project repo).

Further work would be required to incorporate this info into our various scripts.

@github-actions github-actions bot added good first issue Good for newcomers. low priority Low priority. meta/workflow Relating to CI / issue templates / testing frameworks / etc. question There is an outstanding question about this issue. labels Oct 11, 2024
@TimothyWillard TimothyWillard removed the question There is an outstanding question about this issue. label Oct 11, 2024
@jcblemai
Copy link
Collaborator

All of that is very good, thanks. I wonder if we should even go to the route of having this configuration provide the ability to put a user-defined script for each cluster that is sourced at the start of the batch job (in case e.g LD_LIBRARY_PATH changes are made, or to change the shwordsplit for zsh/bash incompatibility.

I'm in favor of having it in this repo, but only slightly. Easier oversight.

@jcblemai
Copy link
Collaborator

#191 wanted to link this issue to it as well.

TimothyWillard added a commit that referenced this issue Nov 1, 2024
Draft solution to GH-342 with the ability to extend to other types of
generic config/info in the future. Add `gempyor.info` module with
exported function `get_cluster_info` which pulls and parses an
information yaml into a pydantic model.
TimothyWillard added a commit that referenced this issue Nov 4, 2024
Draft solution to GH-342 with the ability to extend to other types of
generic config/info in the future. Add `gempyor.info` module with
exported function `get_cluster_info` which pulls and parses an
information yaml into a pydantic model.
TimothyWillard added a commit that referenced this issue Nov 8, 2024
Draft solution to GH-342 with the ability to extend to other types of
generic config/info in the future. Add `gempyor.info` module with
exported function `get_cluster_info` which pulls and parses an
information yaml into a pydantic model.
@TimothyWillard TimothyWillard self-assigned this Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers. low priority Low priority. meta/workflow Relating to CI / issue templates / testing frameworks / etc.
Projects
None yet
Development

No branches or pull requests

2 participants