Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ci): no more docker builds on PR-related events #27146

Merged
merged 5 commits into from
Feb 22, 2024

Conversation

mistercrunch
Copy link
Member

SUMMARY

Speaking with the devexp committee, and in the recent community hall
meeting, it's pretty clear we don't need to build the whole docker
matrix on every PR event. It's somewhat expensive and isn't useful.

One of the reason we did it, from what I've gathered, is we want to
catch if some change breaks the docker build for whatever reason. Now
we may not need to actively prevent this pre-merge as this is unlikely,
yet we can still catch it on master as I'm keeping the push events on
master.

Note that:

  • I'm also moving the master push build to be multi-platform (amd+arm)
    as we started doing this for the release builds, which is nice as
    the same image can be used on different archs
  • The PR events were not "pushed" or cached in our shared, remote
    dockerhub-hosted repo anyways, they were just kind of dry-runs

Oh one more thing, notice the no-op.yml changes? That's because the
required checks related to branch protection are always based on what's
defined on .asf.yml ON master, meaning i need to add dummy/no-op
checks to get this merged, and those can be cleaned up later on in a
follow up PR. Let me know if you know a better way to deal with this.

Speaking with the devexp committee, and in the recent community hall
meeting, it's pretty clear we don't need to build the whole docker
matrix on every PR event. It's somewhat expensive and isn't useful.

One of the reason we did it, from what I've gathered, is we want to
catch if some change breaks the docker build for whatever reason. Now
we may not need to actively prevent this pre-merge as this is unlikely,
yet we can still catch it on `master` as I'm keeping the push events on
master.

Note that:
- I'm also moving the master push build to be multi-platform (amd+arm)
  as we started doing this for the release builds, which is nice as
  the same image can be used on different archs
- The PR events were not "pushed" or cached in our shared, remote
  dockerhub-hosted repo anyways, they were just kind of dry-runs

Oh one more thing, notice the no-op.yml changes? That's because the
required checks related to branch protection are always based on what's
defined on .asf.yml ON `master`, meaning i need to add dummy/no-op
checks to get this merged, and those can be cleaned up later on in a
follow up PR. Let me know if you know a better way to deal with this.
@craig-rueda
Copy link
Member

How will users be able to test Dockerfile changes without running anything at all? I agree that the whole matrix probably doesn't need to be built, but what's stopping us from just allowing one path at a min to run? We could also remove it from the "required" checks as to not be a blocker

@pull-request-size pull-request-size bot added size/L and removed size/M labels Feb 20, 2024
@github-actions github-actions bot added the doc Namespace | Anything related to documentation label Feb 20, 2024
@mistercrunch
Copy link
Member Author

I agree @craig-rueda and adapted to run the "ci" build preset on pull_request, this will only --load since we don't have the creds in that context, but it should validate the build on both amd a arm, which we'll do systematically now across the board as the build matrix and resulting set of tags was getting confusing.

Copy link
Member

@craig-rueda craig-rueda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@dpgaspar dpgaspar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will probably help on #26997. I think that docker-compose.yaml -> x-superset-image: &superset-image apachesuperset.docker.scarf.sh/apache/superset:${TAG:-latest-dev} should point to master-dev now

proposing #27179 (if it makes sense)

@mistercrunch mistercrunch merged commit f1cd8cc into master Feb 22, 2024
28 of 29 checks passed
@rusackas rusackas deleted the no_docker_on_prs branch February 22, 2024 19:20
@mistercrunch mistercrunch restored the no_docker_on_prs branch February 27, 2024 02:19
sfirke pushed a commit to sfirke/superset that referenced this pull request Mar 22, 2024
@rusackas rusackas deleted the no_docker_on_prs branch April 16, 2024 16:54
vinothkumar66 pushed a commit to vinothkumar66/superset that referenced this pull request Nov 11, 2024
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 4.1.0 labels Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels doc Namespace | Anything related to documentation github_actions Pull requests that update GitHub Actions code size/L 🚢 4.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants