-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(schema-compiler): Boost models transpilation 10-13x times (using SWC instead of Babel) #9225
Open
KSDaemon
wants to merge
14
commits into
master
Choose a base branch
from
feat/compilation-swc-backend-native
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #9225 +/- ##
=======================================
Coverage 83.57% 83.57%
=======================================
Files 227 227
Lines 81627 81627
=======================================
Hits 68216 68216
Misses 13411 13411
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR brings further improvements into data model compilation.
Cube transpilers are rewritten in Rust and are called asynchronously, moving transpilation jobs to the internal native threads under the hood. More improvements are achieved by caching the transpilation metadata between files transpile calls within one compilation phase.
This implementation supersedes other approaches taken in #9217 as it boosts performance more.
There were a bunch of iterations, research, and improvements. But long story short: now js files transpilation is 11−14 times faster than it was before!! 🚀
Here are a few numbers and charts for comparison:
Input: 300 js files with ~400 members each
We have 3 phases of compilation. That's why there are 3 messages with times for each phase.
Original master: synchronous sequential traversing in main node.js thread.
In total, it's ~36 seconds.
What is more important — we observe event loop delays up to 50,000 ms!
And here is what we have now: asynchronous parallel processing in native worker threads. (defaults to 4 libuv threads)
In total, it's ~2,8 seconds.
Comparing to master: 36 / 2.8 = 12.8x!!!
What's also important is that I can't get rid of event loop delays totally, but now there are smaller portions of delays, and they are way shorter (5,000 VS 50,000).
Check List