diff --git a/.gitignore b/.gitignore index 570e312..5a91287 100644 --- a/.gitignore +++ b/.gitignore @@ -7,5 +7,5 @@ __pycache__/ env/ # Test coverage output -/.coverage -/coverage.xml +.coverage +coverage.xml diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index d38cdfb..580dd2c 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -6,6 +6,12 @@ repos: - id: end-of-file-fixer - id: check-json - id: check-yaml +- repo: https://github.com/anaconda/pre-commit-hooks + rev: v24.5.2 + hooks: + - id: cog + files: README.md|Makefile + - id: generate-renovate-annotations - repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks rev: v2.13.0 hooks: diff --git a/Makefile b/Makefile index 0a3eca1..2092159 100644 --- a/Makefile +++ b/Makefile @@ -34,4 +34,7 @@ type-check: ## Run static type checks test: ## Run all the unit tests $(conda_run) pytest +cog-readme: ## Run cog on the README.md to generate command output + $(conda_run) run-cog README.md + .PHONY: $(MAKECMDGOALS) diff --git a/README.md b/README.md index 98681c0..0c31877 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,59 @@ An example usage is shown below: ] ``` +The hook is backed by a CLI command, whose help output is reproduced below: + + + +```shell +Usage: generate-renovate-annotations [OPTIONS] ENV_FILES... COMMAND [ARGS]... + + Generate Renovate comments for a list of conda environment files. + For each file, we: + + • Run a command to ensure the environment is created/updated + • Extract a list of installed packages in that environment, including pip + • Generate a Renovate annotation comment, including the package name and + channel. This step also allows for overriding the index of pip packages. + • Pin the exact installed version of each dependency. + +╭─ Arguments ──────────────────────────────────────────────────────────────────╮ +│ * env_files ENV_FILES... A list of conda environment files, │ +│ typically passed in from pre-commit │ +│ automatically │ +│ [default: None] │ +│ [required] │ +╰──────────────────────────────────────────────────────────────────────────────╯ +╭─ Options ────────────────────────────────────────────────────────────────────╮ +│ --internal-pip-package TEXT One or more packages to pull │ +│ from the │ +│ --internal-pip-index-url │ +│ [default: None] │ +│ --internal-pip-index-url TEXT An optional extra pip index URL, │ +│ used in conjunction with the │ +│ --internal-pip-package option │ +│ --create-command TEXT A command to invoke at each │ +│ parent directory of all │ +│ environment files to ensure the │ +│ conda environment is created and │ +│ updated │ +│ [default: make setup] │ +│ --environment-selector TEXT A string used to select the │ +│ conda environment, either │ +│ prefix-based (recommended) or │ +│ named │ +│ [default: -p ./env] │ +│ --disable-environment-creation If set, environment will not be │ +│ created/updated before │ +│ annotations are added. │ +│ --help Show this message and exit. │ +╰──────────────────────────────────────────────────────────────────────────────╯ +``` + ## run-cog The `run-cog` hook can be used to run the [`cog`](https://nedbatchelder.com/code/cog) tool automatically to generate code when committing a file. @@ -92,14 +145,15 @@ import os, sys; sys.path.insert(0, os.path.join(os.getcwd(), "dev")) from generate_makefile_targets_table import main; main() ]]] --> -| Target | Description | -|-----------------|-----------------------------------------------| -| `help` | Display help on all Makefile targets | -| `setup` | Setup local conda environment for development | -| `install-hooks` | Download + install all pre-commit hooks | -| `pre-commit` | Run pre-commit against all files | -| `type-check` | Run static type checks | -| `test` | Run all the unit tests | +| Target | Description | +|-----------------|--------------------------------------------------------------------------| +| `help` | Display help on all Makefile targets | +| `setup` | Setup local conda environment for development | +| `install-hooks` | Download + install all pre-commit hooks | +| `pre-commit` | Run pre-commit against all files | +| `type-check` | Run static type checks | +| `test` | Run all the unit tests | +| `cog-readme` | Run cog on the README.md to generate command output | > **Note:** Interestingly, the table above is generated by the `cog` hook defined in this repo :smile: diff --git a/dev/generate_cli_output.py b/dev/generate_cli_output.py new file mode 100644 index 0000000..ffdad57 --- /dev/null +++ b/dev/generate_cli_output.py @@ -0,0 +1,18 @@ +import shlex +import subprocess + +import cog + +OUTPUT_STR_FORMAT = """\ +```{language} +{text} +```\ +""" + + +def main(command: str, language: str = "shell"): + """Run a command in a subprocess and send the resulting output to cog's output, wrapped in a shell code block.""" + raw_text = subprocess.check_output(shlex.split(command), text=True) + output = OUTPUT_STR_FORMAT.format(language=language, text=raw_text.strip()) + for line in output.splitlines(): + cog.outl(line.rstrip()) diff --git a/dev/generate_makefile_targets_table.py b/dev/generate_makefile_targets_table.py index 866305a..d1a6b1e 100644 --- a/dev/generate_makefile_targets_table.py +++ b/dev/generate_makefile_targets_table.py @@ -1,3 +1,4 @@ +import re import subprocess from textwrap import dedent from typing import NamedTuple @@ -34,6 +35,10 @@ def main(): f"|{'-'*(max_target_len + 2)}|{'-'*(max_description_len + 2):{max_description_len}s}|" ) for t in makefile_targets: + # In GitHub Actions, we get superfluous targets like `make[1]`, so ignore those + if re.search(r"make\[[0-9]+]", t.target): + continue + target_str = f"`{t.target}`" cog.outl( f"| {target_str:{max_target_len}s} | {t.description:{max_description_len}s} |" diff --git a/pyproject.toml b/pyproject.toml index 65c9fa6..cf0dbed 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -18,7 +18,7 @@ requires-python = ">=3.8" version = "0.1.0" [project.scripts] -generate-renovate-annotations = "anaconda_pre_commit_hooks.add_renovate_annotations:main" +generate-renovate-annotations = "anaconda_pre_commit_hooks.add_renovate_annotations:app" run-cog = "anaconda_pre_commit_hooks.run_cog:main" [tool.mypy] diff --git a/src/anaconda_pre_commit_hooks/add_renovate_annotations.py b/src/anaconda_pre_commit_hooks/add_renovate_annotations.py index bd75830..db1217f 100755 --- a/src/anaconda_pre_commit_hooks/add_renovate_annotations.py +++ b/src/anaconda_pre_commit_hooks/add_renovate_annotations.py @@ -28,6 +28,9 @@ IndexOverrides = dict[PackageName, IndexUrl] +app = typer.Typer(rich_markup_mode="markdown", add_completion=False) + + class Dependency(TypedDict): name: str channel: str @@ -218,16 +221,62 @@ def parse_pip_index_overrides( return pip_index_overrides +@app.callback(invoke_without_command=True, no_args_is_help=True) def cli( - env_files: list[Path], - internal_pip_package: Annotated[Optional[list[str]], typer.Option()] = None, - internal_pip_index_url: Annotated[str, typer.Option()] = "", - create_command: Annotated[str, typer.Option()] = DEFAULT_CREATE_COMMAND, - environment_selector: Annotated[str, typer.Option()] = DEFAULT_ENVIRONMENT_SELECTOR, + env_files: Annotated[ + list[Path], + typer.Argument( + help="A list of conda environment files, typically passed in from pre-commit automatically" + ), + ], + internal_pip_package: Annotated[ + Optional[list[str]], + typer.Option( + help="One or more packages to pull from the --internal-pip-index-url" + ), + ] = None, + internal_pip_index_url: Annotated[ + str, + typer.Option( + help="An optional extra pip index URL, used in conjunction with the --internal-pip-package option" + ), + ] = "", + create_command: Annotated[ + str, + typer.Option( + help="A command to invoke at each parent directory of all environment files to ensure the conda environment is created and updated" + ), + ] = DEFAULT_CREATE_COMMAND, + environment_selector: Annotated[ + str, + typer.Option( + help="A string used to select the conda environment, either prefix-based (recommended) or named" + ), + ] = DEFAULT_ENVIRONMENT_SELECTOR, disable_environment_creation: Annotated[ - bool, typer.Option("--disable-environment-creation") + bool, + typer.Option( + "--disable-environment-creation", + help="If set, environment will not be created/updated before annotations are added.", + ), ] = False, ) -> None: + """Generate Renovate comments for a list of `conda` environment files. + + For each file, we: + + * Run a command to ensure the environment is created/updated + + * Extract a list of installed packages in that environment, including pip + + * Generate a Renovate annotation comment, including the package name and channel. + + This step also allows for overriding the index of pip packages. + + * Pin the exact installed version of each dependency. + + """ + # Construct a mapping of package name to index URL based on CLI options pip_index_overrides = parse_pip_index_overrides( internal_pip_index_url, internal_pip_package or [] @@ -247,7 +296,3 @@ def cli( add_comments_to_env_file( env_file, deps, pip_index_overrides=pip_index_overrides ) - - -def main() -> None: - typer.run(cli) # pragma: nocover