Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for in tree actions from all configured branches #73

Merged
merged 4 commits into from
Sep 10, 2024

Conversation

bhearsum
Copy link
Contributor

I believe this change is overall a good one, but it has some possibly controversial aspects and/or downsides. I'm interested in any and all feedback or pushback. There might be some ways this can be done without some or all of the downsides.

Most notably, this change more or less requires that a GITHUB_TOKEN is set in the environment to run ci-admin successfully, otherwise you'll run into rate limits when branches are fetched from Github.

The other, less serious, downside is because we generate hooks for all globbed branches, we end up generating many, many superfluous ones for repos like https://github.com/mozilla/application-services, which have many old and now-unused release branches. One way we could improve this is to not use the branches list to determine which branches get action hooks, but have them specified separately. This has its own downside of requiring us to maintain a separate list, and possibly not working automatically when new release branches are created.

The ci-admin diff for this is giant and weird because so many new hooks are being created. One thing I did for additional sanity checking was to diff the list of resource names, which showed a whole bunch of new hooks being added. I'd like to diff hook bodies as well, but https://bugzilla.mozilla.org/show_bug.cgi?id=1905339 is making that difficult.

This is built on a few other PRs; only the most recent 3 commits are the relevant changes.

@bhearsum bhearsum force-pushed the hook-branches branch 2 times, most recently from 7da942a to 2139bf6 Compare August 16, 2024 15:30
@bhearsum
Copy link
Contributor Author

Update: I did some additionally sanity checking by comparing the json form with and without this change with this script. That showed that the only changes to hooks that already exist are to their descriptions, as expected:

~/tmp/2024-08-16 ❯ python parse.py before.json after.json
project-app-services/in-tree-action-1-generic/733b05722e's description differs
project-app-services/in-tree-action-1-release-promotion/733b05722e's description differs
project-app-services/in-tree-action-3-generic/d34f673940's description differs
project-app-services/in-tree-action-3-release-promotion/d34f673940's description differs
project-comm/in-tree-action-3-backfill/3855bf7d13's description differs
project-comm/in-tree-action-3-generic/3855bf7d13's description differs
project-comm/in-tree-action-3-merge-automation/3855bf7d13's description differs
project-comm/in-tree-action-3-purge-caches/3855bf7d13's description differs
project-comm/in-tree-action-3-release-promotion/3855bf7d13's description differs
project-comm/in-tree-action-3-retrigger-decision/3855bf7d13's description differs
project-gecko/in-tree-action-2-backfill/f526da500b's description differs
project-gecko/in-tree-action-2-generic/f526da500b's description differs
project-gecko/in-tree-action-2-googleplay/f526da500b's description differs
project-gecko/in-tree-action-2-merge-automation/f526da500b's description differs
project-gecko/in-tree-action-2-purge-caches/f526da500b's description differs
project-gecko/in-tree-action-2-release-promotion/f526da500b's description differs
project-gecko/in-tree-action-2-retrigger-decision/f526da500b's description differs
project-gecko/in-tree-action-2-scriptworker-canary/f526da500b's description differs
project-gecko/in-tree-action-3-backfill/f526da500b's description differs
project-gecko/in-tree-action-3-generic/f526da500b's description differs
project-gecko/in-tree-action-3-googleplay/f526da500b's description differs
project-gecko/in-tree-action-3-merge-automation/f526da500b's description differs
project-gecko/in-tree-action-3-purge-caches/f526da500b's description differs
project-gecko/in-tree-action-3-release-promotion/f526da500b's description differs
project-gecko/in-tree-action-3-retrigger-decision/f526da500b's description differs
project-gecko/in-tree-action-3-scriptworker-canary/f526da500b's description differs
project-mozillavpn/in-tree-action-1-generic/db45fbd59d's description differs
project-mozillavpn/in-tree-action-1-release-promotion/db45fbd59d's description differs
project-mozillavpn/in-tree-action-3-generic/db45fbd59d's description differs
project-mozillavpn/in-tree-action-3-release-promotion/db45fbd59d's description differs
project-releng/in-tree-action-3-generic/808e8c8213's description differs
project-releng/in-tree-action-3-generic/e4fd8f3800's description differs
project-releng/in-tree-action-3-generic/febcafe5a5's description differs
project-scriptworker/in-tree-action-3-generic/3cbdeb9e53's description differs
project-translations/in-tree-action-1-generic/c49d0f0c50's description differs
project-translations/in-tree-action-1-train/c49d0f0c50's description differs

With all of this in mind, this should be perfectly safe to take, assuming the approach is desired.

@bhearsum bhearsum marked this pull request as ready for review September 3, 2024 17:34
@bhearsum bhearsum requested a review from a team as a code owner September 3, 2024 17:34
Copy link
Contributor

@jcristau jcristau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me.

Copy link
Contributor

@ahal ahal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the concept, but have suggestion for the implementation that feels worth pursuing.

src/ciadmin/generate/branches.py Show resolved Hide resolved
This secret exists automatically (see https://docs.github.com/en/actions/security-for-github-actions/security-guides/automatic-token-authentication#about-the-github_token-secret). We need it set in a to avoid rate limits when fetching lists of branches for repositories in a subsequent commit.
This is a basic implementation that should be fine for now. At some point we'll want to convert this to query as an App so that we can support private repositories, but I don't think it's worth the effort at the moment, as it will require additional plumbing for the app secret, installing to a bunch of repos, etc.
…ories

There's a fair amount going on here, but most of it is moving around code (most notably the `branch_name` check in `update_resources` needs to happen before we try to check the level of a branch, as not all branches that exist in Github will be present here). The functional changes here invert our iteration over branches: rather than just assuming all non-globbed ones exist in Github, we fetch the list from Github and check each one to see if it matches one that's configured in projects.yml.
@bhearsum bhearsum merged commit eca8f82 into mozilla-releng:main Sep 10, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants