Introduce a new dependabot-related pip-compile GH actions workflow [pydantic saga] #765
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Note that I also considered force pushing directly from the workflow to the same branch, but I originally faced some permissions issue. Admittedly, that could have been due to missing GH token permissions (addressed), but then I'd be most likely spawning a new concurrent gating job out of a different workflow just because we'd be force pushing a pip-compile change. Anyway, I reckoned such approach would have been obfuscating and opaque so I went with with a clearer approach requiring a bit of manual intervention though. |
|
ping. |
This workflow is a direct consequence of the asynchronous release schedule of pydantic and pydantic core and the fact that pydantic is always pinned to a particular pydantic-core version. Dependabot doesn't see these transitive relations and so can't properly update the versions in this case (it always assumes the latest for every dependency). This will naturally lead to broken CI making these version updates impossible to merge. Since our project directly only cares about pydantic and not pydantic-core, we can ignore pydantic-core updates (future patch) and run a dedicated workflow on every dependabot pull request that would check whether any additional changes (i.e. transitive dependency version locks) to our requirements files are needed. If so, then the GitHub actions bot will comment on the pull request that a change to these files is needed and will provide a patch to the reviewer to apply and update the pull request. The workflow is only executed when changes to the requirements files are proposed (realistically only by dependabot). Note that it's not possible to specify the source branch as the workflow trigger, only the target branch, and so that could not have been used as a better filter for dependabot-proposed pull requests specifically. It is run using a Python Alpine docker image, saves the git diff produced by pip-compile to the default github actions environment followed by a github script action that will pop the diff out of the environment and use it to comment on the pull request. References: - https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/workflow-commands-for-github-actions#multiline-strings - https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/using-conditions-to-control-job-execution - https://github.com/actions/github-script?tab=readme-ov-file#comment-on-an-issue - actions/github-script#247 (comment) - actions/github-script#220 (comment) Signed-off-by: Erik Skultety <[email protected]>
pydantic and pydantic-core have different release cycles and since a pydantic release is always locked to a particular pydantic-core version, a dependabot update to both will cause install conflicts pretty much every time because dependabot doesn't see transitive dependency relations. This patch makes it so that pydantic-core's updates are ignored by dependabot (because it's a transitive dependency for us) and we're only going to consider pydantic. This however, will still break installs because pip-compile needs to be invoked to re-pin this new pydantic version to a correct pydantic-core version. That is handled by the newly introduced github dependabot workflow 'dependabot-pipcompile.yml' introduced in a past patch. Signed-off-by: Erik Skultety <[email protected]>
eskultety
force-pushed
the
dependabot-pydantic-saga
branch
from
7293a4c to
2570ebe
Compare
|
I wonder if we could use this workflow to completely replace dependabot and regenerate |
As appealing as that might sound (especially to me since I'm not a fan of dependabot), you'd lose the one useful bit of information dependabot provides during these updates - changelog! I can only wish the industry would use the dedicated OCI labels for this purpose in container images as well so that the hashes wouldn't be completely opaque leaving us with no other choice but to blindly just hit merge if the CI passes. Well, it is what it is for container images... |
74510b4
This should finally resolve the problem of pydantic async release cycles for
pydanticandpydantic-corecausing CI issues preventing merges of pydantic-related changes (see patch 2 for detailed explanation).The effect of the workflow is illustrated here: eskultety#33 (comment)
Note, after the workflow, manual intervention is still going to be needed, because the only thing the workflow does is issue a comment, but then it's just a matter of applying the patch from the comment and force pushing to dependabot's branch and that should be it.
Depends on #764
Maintainers will complete the following section
Note: if the contribution is external (not from an organization member), the CI
pipeline will not run automatically. After verifying that the CI is safe to run:
/ok-to-test(as is the standard for Pipelines as Code)