Extend or customize syntax highlighting

[EDKarlsson]

I only started using this yesterday so I may have not stumbled on it. Is there a way to customize or extend the generated syntax highlighting? I'd prefer to not have everything go under "keyword.control". I know I could add semantic highlighting. do a embedded language or injection. I tried injection but I'm new to using that feature of vscode extensions. My use case is having 2 different styles of comments.

Comment belongs to the type it is being referred to while a note should be ignored.
Note starts with //* and comment has only 1 /

terminal COMMENT: /\/\*[\s\S]*\*\//
hidden terminal NOTE: /\/\/*[\s\S]*?\*\//;

This results in anything within /* .... */ not having any highlighting, well it does have highlighting that follows the patterns of the language and but no specific rule in the "repository". I tried adding a return to the COMMENT terminal but didn't do anything. It would be nice to be able to define the scope of a rule within the grammar.

[msujew]

Hi @EDKarlsson, the generated textMate syntax highlighting file is basically just a starting help for new grammars. Anyone who wants more fine-grained control over their syntactical highlighting should deactivate the textMate option in their langium-config.json and write/modify their textMate file by hand. It's basically impossible to infer accurate syntactical highlighting just based on a grammar. That's also what we did for the Langium syntax highlighting.

Regarding your comments: I'm pretty sure that this style of commenting will lead to a lot of problems down the line. Your COMMENT terminal contains a greedy [\s\S]* regex (compare to the non-greedy [\s\S]*? regex), which basically parses/lexes as "read everything in the file until the end of time itself". Have you confirmed that this doesn't introduce any parsing errors? Additionally, it will definitely override the NOTE terminal, since they are ambiguous, so that one will never be lexed. I would recommend you to remove the COMMENT terminal. Just use the NOTE terminal and call the findCommentNode function to find comments for a CST node (you can use astNode.$cstNode to get the CST node of an AST node).

It would be nice to be able to define the scope of a rule within the grammar.

I think this is (*drum roll*) out of scope for Langium. Our grammar language is already complicated enough as it is, and syntax highlighting can be easily done manually, so I don't really see a reason to include this into the grammar language.

[EDKarlsson]

That works as well. I couldn't find any documentation on deactivating the tmLanguage generation. Thanks for answering my question!

I did try it and it does not overwrite the NOTE terminal. Unfortunately I do not have control of the grammar. This is a fun side project implementing SysML v2. I got decently far with my first attempt using textX (https://github.com/textX/textX) but it doesn't have any concept of fragments and rules started becoming unwieldy just to avoid creating an element representation of a rule

[EDKarlsson]

Here is an example of it working. I deactivated the tmLanguage generation which is a perfectly acceptable solution. There isn't documentation about doing so being okay. Though I will probably have to generate it until I've finished with the entire language spec since it would be a hassle to constantly have to add each rule to the the syntax grammar.

[EDKarlsson]

Found an easy workaround to keep auto generated syntax highlighting and having a custom one by using https://www.npmjs.com/package/json-merger and adding a build step. I have Langium write to sysml.generated.tmlLanguage.json then use this template.tmLanguage.json and finally write it to sysml.tmLanguage.json

{
    "$merge": {
        "source": {
            "$import": "syntaxes/sysml.generated.tmLanguage.json"
        },
        "with": {
            "patterns": {
                    "$concat": [{
                         "include": "#sysml-comments"
                    }
                ],
            },
            "repository": {
                "sysml-comments": {
                    "patterns": [
                        {
                            "name": "string.unquoted.sysml",
                            "begin": "/\\*",
                            "beginCaptures": {
                                "0": {
                                    "name": "punctuation.definition.string.begin.sysml"
                                }
                            },
                            "end": "\\*/",
                            "endCaptures": {
                                "0": {
                                    "name": "punctuation.definition.string.end.sysml"
                                }
                            }
                        }
                    ]
                }
            }
        }
    }
}

[msujew]

Here is an example of it working.

Oh I see now, I didn't notice that the NOTE and COMMENT terminals started differently /* and //*. Yeah, now it all makes sense.

Found an easy workaround to keep auto generated syntax highlighting and having a custom one

Cool, that's good to know. We might want to include that in the documentation on textMate grammars that's supposed to come eventually.