Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EMCEE Batch Submit CLI Tool #394

Open
wants to merge 78 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
ce065d4
Jinja2 templating infrastructure
TimothyWillard Oct 29, 2024
8ecc3e0
Logging infrastructure for `gempyor`
TimothyWillard Oct 29, 2024
1afe3c8
Add private `shutil.which` wrapper
TimothyWillard Oct 30, 2024
cd3ca2d
Only propagate logger in pytest
TimothyWillard Oct 30, 2024
e6c86ef
Add `_sbatch` to `_slurm`
TimothyWillard Oct 31, 2024
51ae598
Add `gempyor.utils._git_head`
TimothyWillard Oct 31, 2024
d642739
Add `write_manifest` for batch manifest files
TimothyWillard Oct 31, 2024
75304cd
Move `manifest` module into `batch`
TimothyWillard Oct 31, 2024
3eb1cb2
Moved `_slurm` module into `batch`
TimothyWillard Oct 31, 2024
06dbf0d
Initial `JobSize` class
TimothyWillard Oct 31, 2024
10faf77
Add `JobSize.size_from_jobs_sims_blocks`
TimothyWillard Oct 31, 2024
498197c
Reformat new files with line length 92
TimothyWillard Oct 31, 2024
d9b03a7
Add `_format_cli_options` helper
TimothyWillard Nov 1, 2024
23b315e
Draft `_sbatch_template` function
TimothyWillard Nov 1, 2024
ff2d1f5
Initial pass at generic cluster config
TimothyWillard Nov 1, 2024
05c119b
Initial EMCEE inference sbatch script
TimothyWillard Nov 1, 2024
d98eb80
Add path modifications to cluster info
TimothyWillard Nov 4, 2024
abd92e6
Skeleton of sbatch script
TimothyWillard Nov 4, 2024
e3e373d
Apply black formatter
TimothyWillard Nov 4, 2024
dc67254
Add `log_cli_inputs` utility
TimothyWillard Nov 5, 2024
3089358
Add `gempyor.batch._job_name`
TimothyWillard Nov 5, 2024
d6773ff
Add `_resolve_batch_system` utility
TimothyWillard Nov 5, 2024
42ceda3
Initial work on `flepimop batch` CLI
TimothyWillard Nov 5, 2024
ed4ddc7
Minor additions to `flepimop batch`
TimothyWillard Nov 5, 2024
382a616
Add `JobTimeLimit` class
TimothyWillard Nov 5, 2024
28965ff
Add `from_per_simulation_time` method
TimothyWillard Nov 6, 2024
772e872
Begin constructing sbatch call
TimothyWillard Nov 6, 2024
a8ab8db
Add custom duration click param type
TimothyWillard Nov 6, 2024
8db1e4a
Fix bug in time defaults, add in cluster info
TimothyWillard Nov 6, 2024
5a95683
Write config for batch, work on batch script
TimothyWillard Nov 6, 2024
3fc27b9
Implement options for `flepimop-calibrate`
TimothyWillard Nov 6, 2024
c40aed9
Add `--email` to `flepimop batch`
TimothyWillard Nov 6, 2024
458ac00
Correct `param_decls` arg to `click.Option`
TimothyWillard Nov 7, 2024
b4c028d
New lines in `emcee_inference` template
TimothyWillard Nov 7, 2024
1e7e1e4
Add slurm+cluster option value error unit test
TimothyWillard Nov 7, 2024
a18af5f
Add `BatchSystem` enum
TimothyWillard Nov 7, 2024
b0d1852
Greatly simplify `JobSize` for first draft
TimothyWillard Nov 7, 2024
770d6f6
Add `JobResources` class
TimothyWillard Nov 7, 2024
e2f3e13
Further unit tests of `JobResources`
TimothyWillard Nov 8, 2024
c710522
Remove unneeded `warnings` import
TimothyWillard Nov 8, 2024
fcc1dbc
Rename `flepimop batch` to `submit`
TimothyWillard Nov 8, 2024
a893b00
Dump config to actual rather than temp file
TimothyWillard Nov 8, 2024
d5eab61
Remove `--nthin` from `flepimop submit`
TimothyWillard Nov 8, 2024
1316ebe
Optional overrides to job resource presets
TimothyWillard Nov 8, 2024
d4a86a9
Initial pass at incorporating R inference
TimothyWillard Nov 11, 2024
82dfe8f
Clean whitespace in `_sbatch` logging
TimothyWillard Nov 11, 2024
3d5484b
Add `_local` internal utility
TimothyWillard Nov 11, 2024
a2fe5df
Splitting out templates
TimothyWillard Nov 11, 2024
641bc70
Initial template testing
TimothyWillard Nov 11, 2024
4eecc0b
Add exact results for conda activate template
TimothyWillard Nov 12, 2024
a842342
Add `_local_template` utility
TimothyWillard Nov 12, 2024
398c352
Incorporate batch system into `JobSize`
TimothyWillard Nov 12, 2024
75448f7
Add `conda init` shim
TimothyWillard Nov 12, 2024
e3e4adc
Split env vars out into its own template
TimothyWillard Nov 12, 2024
e9bcb23
Now supports `--local` and non-EMCEE inference
TimothyWillard Nov 12, 2024
08ec61e
Fix bug in `--skip-manifest` flag
TimothyWillard Nov 12, 2024
b6c95ca
Add git branch checkout to preserve run.
TimothyWillard Nov 12, 2024
3fb4dbe
Remove custom interpreter for sbatch
TimothyWillard Nov 12, 2024
93ab660
Legacy inference use slurm array jobs
TimothyWillard Nov 14, 2024
0490895
Override nodes for legacy inference on slurm
TimothyWillard Nov 14, 2024
7e7efd5
Infer cluster from FQDN
TimothyWillard Nov 14, 2024
821210d
Add `--samples` option to `flepimop submit`
TimothyWillard Nov 14, 2024
9353b66
Add detailed documentation for `flepimop submit`
TimothyWillard Nov 14, 2024
a56a4a8
Add custom click param type for memory
TimothyWillard Nov 14, 2024
b3961f6
Replace memory option `IntRange` with `MEMORY_MB`
TimothyWillard Nov 14, 2024
0fa1f9b
Document info/shared_cli, MemoryParamType int
TimothyWillard Nov 15, 2024
6e38dd2
Assume unitless numbers are in default unit
TimothyWillard Nov 25, 2024
4f936db
Move `get_cluster_info` call into `_click_submit`
TimothyWillard Nov 25, 2024
8d7b3d8
Helpful logging if using `--local` on a cluster
TimothyWillard Nov 25, 2024
5027499
Add args documentation for `_submit_scenario_job`
TimothyWillard Nov 26, 2024
d549728
Remove temporary comments
TimothyWillard Nov 26, 2024
b76bb3b
Add `FLEPI_PATH` to GH action, fix Jinja2 loader
TimothyWillard Nov 26, 2024
2f46391
Add `Self` shim for python 3.10
TimothyWillard Nov 26, 2024
c14552e
Escape `\` in `flepimop submit --help` man page
TimothyWillard Nov 26, 2024
a64ca7e
Provide only pkg name to Jinja2 `PackageLoader`
TimothyWillard Nov 26, 2024
7c03736
Prefer `FileSystemLoader` over `PackageLoader`
TimothyWillard Nov 26, 2024
0149bc7
Set `include-package-data` to `true`
TimothyWillard Nov 26, 2024
ad11fae
Include templates with `include-data` directive
TimothyWillard Dec 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/gempyor-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ jobs:
shell: bash
- name: Run gempyor tests
run: |
export FLEPI_PATH=$(pwd)
cd flepimop/gempyor_pkg
pytest --exitfirst
shell: bash
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,6 @@ flepimop/gempyor_pkg/.coverage

# Environment variables
.env

# info/ directory
info/**/*.json
5 changes: 4 additions & 1 deletion flepimop/gempyor_pkg/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,13 @@ dependencies = [
"emcee",
"graphviz",
"h5py",
"Jinja2",
"matplotlib",
"numba>=0.53.1",
"numpy",
"pandas",
"pyarrow",
"pydantic",
"scipy",
"seaborn",
"sympy",
Expand All @@ -51,7 +53,8 @@ gempyor-simulate = "gempyor.simulate:_deprecated_simulate"

[tool.setuptools]
package-dir = {"" = "src"}
include-package-data = false
include-package-data = true
package-data = { "gempyor" = ["templates/*.j2"] }

[tool.setuptools.packages.find]
where = ["src"]
Expand Down
120 changes: 120 additions & 0 deletions flepimop/gempyor_pkg/src/gempyor/_jinja.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
"""
Internal Jinja2 template rendering utilities.

This module contains utilities for finding and rendering Jinja2 templates. This module
is tightly coupled to the organization of the package and not intended for external use.
"""

# Exports
__all__ = []


# Imports
from os.path import dirname
from pathlib import Path
from tempfile import mkstemp
from typing import Any

from jinja2 import Environment, FileSystemLoader, PackageLoader, Template


# Globals
try:
_jinja_environment = Environment(
loader=FileSystemLoader(dirname(__file__).replace("\\", "/") + "/templates")
)
except ValueError:
_jinja_environment = Environment(loader=PackageLoader("gempyor"))


# Functions
def _get_template(name: str) -> Template:
"""
Get a jinja template by name.

Args:
name: The name of the template to pull.

Returns:
A jinja template object corresponding to `name`.

Examples:
>>> _get_template("test_template.j2")
<Template 'test_template.j2'>
"""
return _jinja_environment.get_template(name)


def _render_template(name: str, data: dict[str, Any]) -> str:
"""
Render a jinja template by name.

Args:
name: The name of the template to pull.
data: The data to pass to the template when rendering.

Returns:
The rendered template as a string.

Examples:
>>> _render_template("test_template.j2", {"name": "Bob"})
'Hello Bob!'
"""
return _get_template(name).render(data)


def _render_template_to_file(name: str, data: dict[str, Any], file: Path) -> None:
"""
Render a jinja template and save to a file.

Args:
name: The name of the template to pull.
data: The data to pass to the template when rendering.
file: The file to save the rendered template to.

Examples:
>>> from pathlib import Path
>>> file = Path("hi.txt")
>>> _render_template_to_file("test_template.j2", {"name": "Jane"}, file)
>>> file.read_text()
'Hello Jane!'
"""
with file.open(mode="w", encoding="utf-8") as f:
f.write(_render_template(name, data))


def _render_template_to_temp_file(
name: str,
data: dict[str, Any],
suffix: str | None = None,
prefix: str | None = None,
) -> Path:
"""
Render a jinja template and save to a temporary file.

Args:
name: The name of the template to pull.
data: The data to pass to the template when rendering.
suffix: The suffix of the temporary file, such as an extension. Passed on to
`tempfile.mkstemp`.
prefix: The prefix of the temporary file. Passed on to `tempfile.mkstemp`.

Returns:
The file containing the rendered template as a `Path` object.

See Also:
[`tempfile.mkstemp`](https://docs.python.org/3/library/tempfile.html#tempfile.mkstemp)

Examples:
>>> file = _render_template_to_temp_file(
... "test_template.j2", {"name": "John"}, suffix=".txt", prefix="foo_"
... )
>>> file
PosixPath('/var/folders/2z/h3pc0p7s3ng1tvxrgsw5kr680000gp/T/foo_ocaomg4k.txt')
>>> file.read_text()
'Hello John!'
"""
_, tmp = mkstemp(suffix=suffix, prefix=prefix, text=True)
tmp = Path(tmp)
_render_template_to_file(name, data, tmp)
return tmp
Loading