Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The ANTLRv4 grammar is now equivalent to the ANTLRv3 grammar #4424

Merged
merged 5 commits into from
Feb 26, 2025

Conversation

mike-lischke
Copy link
Member

  • The LexBasic.g4 grammar has been removed and the few used rules have been taken over into ANTLRv4Lexer.g4. This lowers complexity and increases readability of error messages (e.g. missing ';' instead of missing SEMI).
  • The wrong definition of element options for action (and sempred) blocks has been fixed.
  • The accepted token types list for element options has been fixed.
  • A predefined token file is now part of the set to make sure tokens appear in the right order (for tree walkers that were ported from ANTLRv3).

This grammar is now used in antlr-ng for generating parsers, so it should also appear as the official grammar (since the one in ANTLRv4 is actually a v3 grammar).

mike-lischke and others added 3 commits February 24, 2025 17:16
- The LexBasic.g4 grammar has been removed and the few used rules have been taken over into ANTLRv4Lexer.g4. This lowers complexity and increases readability of error messages (e.g. missing ';' instead of missing SEMI).
- The wrong definition of element options for action (and sempred) blocks has been fixed.
- A predefined token file is now part of the set to make sure tokens appear in the right order (for tree walkers that were ported from ANTLRv3).

This grammar is now used in antlr-ng for generating parser, so it should also appear as the official grammar (since the one in ANTLRv4 is actually a v3 grammar).
@mike-lischke mike-lischke changed the title The grammar is now equivalent to the ANTLRv3 grammar The ANTLRv4 grammar is now equivalent to the ANTLRv3 grammar Feb 24, 2025
@kaby76
Copy link
Contributor

kaby76 commented Feb 24, 2025

We're going to need the predefined.tokens file in grammars-v4/antlr/antlr4/predefined.tokens since it's source code for the grammar and antlr4 fails without the file. Github Actions says success, but the build failed. It will be fixed with the antlr-ng PR #4385. Thanks.

@mike-lischke
Copy link
Member Author

Oh, I didn't know tokens files are ignored. Should I force-add it or do you have a different solution?

@kaby76
Copy link
Contributor

kaby76 commented Feb 24, 2025

The .tokens file isn't ignored apparently. The old "antlr4" tool requires the file, fails with error(114): ANTLRv4Lexer.g4:54:17: cannot find tokens file .\predefined.tokens if it's not there. This is because of the options { tokenVocab = predefined; } you added in this PR. So, we will need that added please.

@teverett
Copy link
Member

@mike-lischke thanks!

@teverett teverett merged commit 31cfc54 into antlr:master Feb 26, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants