AST dis- and reassembler: an alternative for hydrating #1632
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DRAFT: But feel free to look through. I need to add tests and a comparison of the resource consumption of Hydrator vs. Dis-/Reassembler.
Hydrating is currently used to send an entire AST (including the CST) from the async parser worker back to the main language server thread. This is done by transforming the AST/CST into serializeable representation, send it via messaging to the parent thread and transforming back this serializable representation back to an AST/CST structure. We end up (more or less) with 4 copies of the same structure. Especially for big files, this can become a bottleneck, since worker threads seem to have a memory limit of 4 GB.
With the approach given by this PR, I try to serialize the build process of the AST/CST, ending up in only 2 copies (more or less). On the worker thread side the AST gets traversed and disassembled to build instructions. The instructions are send over to the parent which can start building single AST and CST nodes.
The structure of the instruction flow is like this:
And that is the whole magic.
Currently there are only 5 tests that were used before for the hydrator.
I think I need to add more, such that every instruction was tested at least once (that is why it is still a draft).
Plus, I do not know anything about performance yet. I was just writing down the code. I have some data which I can use to test against, but let's see when I find the time for it.