-
Notifications
You must be signed in to change notification settings - Fork 289
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace ANTLR with hand-rolled parser #917
base: main
Are you sure you want to change the base?
Conversation
50f636e
to
c9d76f5
Compare
pkl-core/src/main/java/org/pkl/core/ast/builder/AstBuilderNew.java
Outdated
Show resolved
Hide resolved
pkl-core/src/main/java/org/pkl/core/ast/builder/AstBuilderNew.java
Outdated
Show resolved
Hide resolved
|
||
@Override | ||
public ExpressionNode visitAmendsExpr(Amends expr) { | ||
// parentExpr is always New, Amends or Parenthesized. The parser makes sure of it |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this comment, but it would be useful to link through to the method where this behaviour is expected in the parser.
pkl-core/src/main/java/org/pkl/core/ast/builder/AstBuilderNew.java
Outdated
Show resolved
Hide resolved
pkl-core/src/test/kotlin/org/pkl/core/newparser/ParserComparisonTest.kt
Outdated
Show resolved
Hide resolved
pkl-core/src/test/files/LanguageSnippetTests/input/basic/amendsChains.pkl
Show resolved
Hide resolved
This comment was marked as resolved.
This comment was marked as resolved.
3afe063
to
8b440d4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did a first pass! Still have much more to review.
children.addAll(pars); | ||
children.addAll(members); | ||
return children; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we make this allocation free?
One option is to just make children
a field on the class.
Alternatively, maybe this should instead be visitChildren
. In that case, this becomes:
<T> @Nullable T visitChildren(ParserVisitor<T> visitor) {
for (var parameter : parameters) {
parameter.accept(visitor);
}
for (var member : members) {
member.accept(visitor);
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I kept the children()
method there but refactored the visitor not call it anymore. So no allocations are being made in our visitors.
Fixes #906
Fixes #888
Fixes #927
Fixes #931
Fixes #932
Incompatible changes:
Reason: not requiring semicolons or newlines requires too much backtracking in the parser, degrading performance. Also both our IntelliJ plugin and the LSP don't allow this syntax and show an error.
Our current ANTLR parser is quite slow, and, depending on the codebase, can be the slowest part of running Pkl.
Some results from parsing all snippet tests in pkl-core showing ~100x performance improvement:
ANTLR elapsed: 7122ms (7.1s)
New parser elapsed: 78.73ms
This is still a draft, many tests are still failing and repl parsing is still using the old parser.