Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Run hooks on self and add really nice CLI output to README.md #72

Merged
merged 14 commits into from
May 10, 2024
Merged
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,5 @@ __pycache__/
env/

# Test coverage output
/.coverage
/coverage.xml
.coverage
coverage.xml
6 changes: 6 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ repos:
- id: end-of-file-fixer
- id: check-json
- id: check-yaml
- repo: https://github.com/anaconda/pre-commit-hooks
rev: v24.5.2
hooks:
- id: cog
files: README.md|Makefile
- id: generate-renovate-annotations
- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
rev: v2.13.0
hooks:
Expand Down
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,7 @@ type-check: ## Run static type checks
test: ## Run all the unit tests
$(conda_run) pytest

cog-readme: ## Run cog on the README.md to generate command output
$(conda_run) run-cog README.md

.PHONY: $(MAKECMDGOALS)
70 changes: 62 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,59 @@ An example usage is shown below:
]
```

The hook is backed by a CLI command, whose help output is reproduced below:

<!-- [[[cog
#import os, sys; sys.path.insert(0, os.path.join(os.getcwd(), "dev"))
#from generate_cli_output import main
#main(command="generate-renovate-annotations --help")
]]] -->
<!-- [[[end]]] -->
```shell
Usage: generate-renovate-annotations [OPTIONS] ENV_FILES... COMMAND [ARGS]...

Generate Renovate comments for a list of conda environment files.
For each file, we:

• Run a command to ensure the environment is created/updated
• Extract a list of installed packages in that environment, including pip
• Generate a Renovate annotation comment, including the package name and
channel. This step also allows for overriding the index of pip packages.
• Pin the exact installed version of each dependency.

╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ * env_files ENV_FILES... A list of conda environment files, │
│ typically passed in from pre-commit │
│ automatically │
│ [default: None] │
│ [required] │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --internal-pip-package TEXT One or more packages to pull │
│ from the │
│ --internal-pip-index-url │
│ [default: None] │
│ --internal-pip-index-url TEXT An optional extra pip index URL, │
│ used in conjunction with the │
│ --internal-pip-package option │
│ --create-command TEXT A command to invoke at each │
│ parent directory of all │
│ environment files to ensure the │
│ conda environment is created and │
│ updated │
│ [default: make setup] │
│ --environment-selector TEXT A string used to select the │
│ conda environment, either │
│ prefix-based (recommended) or │
│ named │
│ [default: -p ./env] │
│ --disable-environment-creation If set, environment will not be │
│ created/updated before │
│ annotations are added. │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```

## run-cog

The `run-cog` hook can be used to run the [`cog`](https://nedbatchelder.com/code/cog) tool automatically to generate code when committing a file.
Expand Down Expand Up @@ -92,14 +145,15 @@ import os, sys; sys.path.insert(0, os.path.join(os.getcwd(), "dev"))
from generate_makefile_targets_table import main; main()
]]] -->
<!-- THE FOLLOWING CODE IS GENERATED BY COG VIA PRE-COMMIT. ANY MANUAL CHANGES WILL BE LOST. -->
| Target | Description |
|-----------------|-----------------------------------------------|
| `help` | Display help on all Makefile targets |
| `setup` | Setup local conda environment for development |
| `install-hooks` | Download + install all pre-commit hooks |
| `pre-commit` | Run pre-commit against all files |
| `type-check` | Run static type checks |
| `test` | Run all the unit tests |
| Target | Description |
|-----------------|--------------------------------------------------------------------------|
| `help` | Display help on all Makefile targets |
| `setup` | Setup local conda environment for development |
| `install-hooks` | Download + install all pre-commit hooks |
| `pre-commit` | Run pre-commit against all files |
| `type-check` | Run static type checks |
| `test` | Run all the unit tests |
| `cog-readme` | Run cog on the README.md to generate command output |
<!-- [[[end]]] -->

> **Note:** Interestingly, the table above is generated by the `cog` hook defined in this repo :smile:
18 changes: 18 additions & 0 deletions dev/generate_cli_output.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import shlex
import subprocess

import cog

OUTPUT_STR_FORMAT = """\
```{language}
{text}
```\
"""


def main(command: str, language: str = "shell"):
"""Run a command in a subprocess and send the resulting output to cog's output, wrapped in a shell code block."""
raw_text = subprocess.check_output(shlex.split(command), text=True)
output = OUTPUT_STR_FORMAT.format(language=language, text=raw_text.strip())
for line in output.splitlines():
cog.outl(line.rstrip())
5 changes: 5 additions & 0 deletions dev/generate_makefile_targets_table.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import re
import subprocess
from textwrap import dedent
from typing import NamedTuple
Expand Down Expand Up @@ -34,6 +35,10 @@ def main():
f"|{'-'*(max_target_len + 2)}|{'-'*(max_description_len + 2):{max_description_len}s}|"
)
for t in makefile_targets:
# In GitHub Actions, we get superfluous targets like `make[1]`, so ignore those
if re.search(r"make\[[0-9]+]", t.target):
continue

target_str = f"`{t.target}`"
cog.outl(
f"| {target_str:{max_target_len}s} | {t.description:{max_description_len}s} |"
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ requires-python = ">=3.8"
version = "0.1.0"

[project.scripts]
generate-renovate-annotations = "anaconda_pre_commit_hooks.add_renovate_annotations:main"
generate-renovate-annotations = "anaconda_pre_commit_hooks.add_renovate_annotations:app"
run-cog = "anaconda_pre_commit_hooks.run_cog:main"

[tool.mypy]
Expand Down
65 changes: 55 additions & 10 deletions src/anaconda_pre_commit_hooks/add_renovate_annotations.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@
IndexOverrides = dict[PackageName, IndexUrl]


app = typer.Typer(rich_markup_mode="markdown", add_completion=False)


class Dependency(TypedDict):
name: str
channel: str
Expand Down Expand Up @@ -218,16 +221,62 @@ def parse_pip_index_overrides(
return pip_index_overrides


@app.callback(invoke_without_command=True, no_args_is_help=True)
def cli(
env_files: list[Path],
internal_pip_package: Annotated[Optional[list[str]], typer.Option()] = None,
internal_pip_index_url: Annotated[str, typer.Option()] = "",
create_command: Annotated[str, typer.Option()] = DEFAULT_CREATE_COMMAND,
environment_selector: Annotated[str, typer.Option()] = DEFAULT_ENVIRONMENT_SELECTOR,
env_files: Annotated[
list[Path],
typer.Argument(
help="A list of conda environment files, typically passed in from pre-commit automatically"
),
],
internal_pip_package: Annotated[
Optional[list[str]],
typer.Option(
help="One or more packages to pull from the --internal-pip-index-url"
),
] = None,
internal_pip_index_url: Annotated[
str,
typer.Option(
help="An optional extra pip index URL, used in conjunction with the --internal-pip-package option"
),
] = "",
create_command: Annotated[
str,
typer.Option(
help="A command to invoke at each parent directory of all environment files to ensure the conda environment is created and updated"
),
] = DEFAULT_CREATE_COMMAND,
environment_selector: Annotated[
str,
typer.Option(
help="A string used to select the conda environment, either prefix-based (recommended) or named"
),
] = DEFAULT_ENVIRONMENT_SELECTOR,
disable_environment_creation: Annotated[
bool, typer.Option("--disable-environment-creation")
bool,
typer.Option(
"--disable-environment-creation",
help="If set, environment will not be created/updated before annotations are added.",
),
] = False,
) -> None:
"""Generate Renovate comments for a list of `conda` environment files.

For each file, we:

* Run a command to ensure the environment is created/updated

* Extract a list of installed packages in that environment, including pip

* Generate a Renovate annotation comment, including the package name and channel.

This step also allows for overriding the index of pip packages.

* Pin the exact installed version of each dependency.

"""

# Construct a mapping of package name to index URL based on CLI options
pip_index_overrides = parse_pip_index_overrides(
internal_pip_index_url, internal_pip_package or []
Expand All @@ -247,7 +296,3 @@ def cli(
add_comments_to_env_file(
env_file, deps, pip_index_overrides=pip_index_overrides
)


def main() -> None:
typer.run(cli) # pragma: nocover
Loading