Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use docker in frontend GHA to parallelize work #31490

Merged
merged 26 commits into from
Jan 8, 2025

Conversation

mistercrunch
Copy link
Member

@mistercrunch mistercrunch commented Dec 17, 2024

Optimizing/parallelizing work around frontend builds.

  • docker build is typically much faster than CI builds, able to reuse layers
  • build happening only once, before parallelizing the work on top
  • sharding jest tests, running 8 in parallel (!)
  • maintained the codecov coverage reporting though upload is broken both in master and here, requires more attention

Clock time savings from 25 to 5-10 minutes depending on docker layer cache

Machine time savings also significant VS parallelizing the build steps.

Oh also, bycatch fixing the multiple alembic/db migrations heads currently in master

workflow topology

Screenshot 2025-01-07 at 3 20 27 PM

@github-actions github-actions bot added doc Namespace | Anything related to documentation github_actions Pull requests that update GitHub Actions code labels Dec 17, 2024
@mistercrunch mistercrunch force-pushed the faster_frontend_builds branch from c310511 to 042e732 Compare December 17, 2024 01:52
@github-actions github-actions bot added preset-io and removed doc Namespace | Anything related to documentation labels Dec 17, 2024
@mistercrunch mistercrunch force-pushed the faster_frontend_builds branch from b6b3a99 to 6f9bd98 Compare December 17, 2024 19:49
@mistercrunch mistercrunch force-pushed the faster_frontend_builds branch from ad8b086 to 06f044c Compare December 19, 2024 07:30
@mistercrunch mistercrunch force-pushed the faster_frontend_builds branch from 904ec1e to 49d8fa9 Compare December 21, 2024 19:32
@mistercrunch mistercrunch marked this pull request as ready for review January 5, 2025 01:25
Copy link

@korbit-ai korbit-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've completed my review and didn't find any issues... but I did find this unicorn.

                    /
               ,.. /
             ,'   ';
  ,,.__    _,' /';  .
 :','  ~~~~    '. '~
:' (   )         )::,
'. '. .=----=..-~  .;'
 '  ;'  ::   ':.  '"
   (:   ':    ;)
    \\   '"  ./
     '"      '"

Need a new review? Comment /korbit-review on this PR and I'll review your latest changes.

Korbit Guide: Usage and Customization

Interacting with Korbit

  • You can manually ask Korbit to review your PR using the /korbit-review command in a comment at the root of your PR.
  • You can ask Korbit to generate a new PR description using the /korbit-generate-pr-description command in any comment on your PR.
  • Too many Korbit comments? I can resolve all my comment threads if you use the /korbit-resolve command in any comment on your PR.
  • Chat with Korbit on issues we post by tagging @korbit-ai in your reply.
  • Help train Korbit to improve your reviews by giving a 👍 or 👎 on the comments Korbit posts.

Customizing Korbit

  • Check out our docs on how you can make Korbit work best for you and your team.
  • Customize Korbit for your organization through the Korbit Console.

Current Korbit Configuration

General Settings
Setting Value
Review Schedule Automatic excluding drafts
Max Issue Count 10
Automatic PR Descriptions
Issue Categories
Category Enabled
Naming
Database Operations
Documentation
Logging
Error Handling
Systems and Environment
Objects and Data Structures
Readability and Maintainability
Asynchronous Processing
Design Patterns
Third-Party Libraries
Performance
Security
Functionality

Feedback and Support

Note

Korbit Pro is free for open source projects 🎉

Looking to add Korbit to your team? Get started with a free 2 week trial here

@mistercrunch mistercrunch force-pushed the faster_frontend_builds branch from eeb5fb2 to e782250 Compare January 5, 2025 08:40
@mistercrunch
Copy link
Member Author

NOTE: this needs to work in forks/without dockerhub creds to be viable

@mistercrunch mistercrunch force-pushed the faster_frontend_builds branch 2 times, most recently from 55fcae1 to 4c54dc4 Compare January 7, 2025 23:15
@mistercrunch mistercrunch force-pushed the faster_frontend_builds branch from 4c54dc4 to b61e546 Compare January 7, 2025 23:47
@github-actions github-actions bot added the risk:db-migration PRs that require a DB migration label Jan 8, 2025
@mistercrunch mistercrunch merged commit 0eca79c into master Jan 8, 2025
52 of 53 checks passed
@mistercrunch mistercrunch deleted the faster_frontend_builds branch January 8, 2025 00:36
mistercrunch added a commit that referenced this pull request Jan 8, 2025
I just merged #31490 and realize
it fails when no frontend changes are found. It will run the first
workflow and fail on subsequent.

This PR makes it such that the first workflow emits an output to tell
the other workflows to skip if needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies:npm github_actions Pull requests that update GitHub Actions code preset-io review:draft risk:db-migration PRs that require a DB migration size/L
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants