[proposal] Introducing TextMate grammar system for syntax highlighting ? #9087
Comments
skywind3000
changed the title
skywind3000
changed the title
|
Thank you for starting this discussion. I had a vague plan to look into integrating treesitter, it is good to know it also has disadvantages. Vscode is widely used, thus if it uses TextMate then there must be something good about it. Comments welcome. |
|
I'll just comment that I would take these comments about tree-sitter with a significant heap of salt. |
|
The might be some misunderstanding here. Tree-sitter in neovim doesn't use an external process like coc.nvim. The parser runtime is a C library embedded into the editor itself (in total not more LOC:s than syntax.c + highlight.c in vim itself), and parses the buffer in memory without copy and produces a syntax tree that in-process plugins can use (for highlighting but also for other purposing like text objects). |
|
Right now the biggest problem with syntax highlighting is how inconsistent and unpredictable it is among different languages. An unified interface will be more than worth the effort. TextMate will probably be better for keeping the syntax system more integrated & backwards compatible than using something like treesitter. Also the modular and overengineered plugin architecture of treesitter would be a huge departure from the way it is done right now, so we should be a little cautious about how much functionality to reimplement. |
|
@bfredl How much longer does it take to load a larger file like src/evalfunc.c in Neovim when tree-sitter is enabled, compared to the default syntax highlighting? I'm assuming that default syntax highlighting is disabled for filetypes where tree-sitter is supported. |
|
@bfrg |
|
treesitter is more than just syntax highlighting, it's also useful for text objects for example. TextMate system is old, Sublime Text has been mentioned but it left it years ago to use its own syntax engine. Does it make sense to adopt a system that is already waning? And how big is its library if it must be included? Also when saying that a system is more performant, some source/benchmark should be provided. Is it TextMate more performant than treesitter? Who says so? |
|
@bfredl , thanks for figuring it out, and I made a new revision: list of tree-sitter disadvantages for syntax highlighting:
The biggest risk is the horrible parser quality, over 100+ open issues for parsers: examples for quality issues:
examples for performance issues:
The parser quality problem is totally out of control, nearly impossible for us to fix all the parsers one by one. It appears that the performance issues of traditional vim syntax highlighting are not fixed by tree-sitter, but more issues are introduced by it. Therefore, syntax highlighting should not rely on such unreliable parsers. One more thing, parsers are not only hard to implement but also hard to extend/improve. most of the time, only the author can do this. But json syntax files are much easier for everyone. One last thing, parser binaries are big in size:
The average size of each parser is around 500KB, vim 8.2.3582 has 644 syntax files, that means you need extra 322MB disk space to save the parse binaries. Vim is shipped every where, including systems that do not have gcc/clang compilers installed, and systems with limited storage space, that means it is hard to build the parsers on demand or ship the pre-build binaries with vim itself. |
|
@mg979 For modern editors, regex based syntax highlighting is still the foundation, and semantic highlighting is just the decoration. I am talking about the foundation not the decoration.
I think I made my point in the previous post: #9087 (comment) Everyone knows, but nobody dares to say, that NeoVim users are struggling in the mud of treesitter parsers right now, no need for vim users to experience such horrible things again. If someone really care about semantic syntax highlighting, indentation or textobj, they can still use LSP as well, LSP does a good job for all these things. There are plenty of LSP solutions for both vim & neovim. Let's back to textmate, the core part of textmate syntax system is oniguruma, which is open source and well maintained by the community. known editors / ides supporting textmate grammar:
The syntax rendering of those editors/IDEs above is really fluent and proved by both time and massive users, while treesitter are still in test and can freeze nvim when parsing large files. (also check the performance issues in the previous post) Monarch was initially built to support languages in VS Code. Then, they decided to switch for TextMate as well because of reasons listed above: microsoft/vscode#174 (comment) . Some details:
it is easy to implement textmate syntax highlightingThe tokenizer of vscode/textmate is: And here is the wrapper in javascript, it's neatly written and not hard to understand: All we need to do is rewriting the javascript wrapper in C,
No more than 4854 lines (including comments) in javascript/typescript And thousands of textmate syntax files are ready to use. |
Tests excluded, it's 3779 lines of code (source: |
|
Why not Sublime grammar instead of TextMate grammar? It seems more powerful, and easier to read.
Is it because there have been fewer |
|
@lacygoill maybe textmate grammar is a little easier ? because there are reference implementations: But sublime is closed source ? we need write everything from scratch ?? edit: if sublime 3 grammar is also based on oniguruma, maybe things can become a little easier ? anyway, both textmate/sublime solution are better than tree-sitter. |
Good point. I forgot that sublime was closed source. Is TextMate much better (readibility, reliability, performance) than our current syntax highlighting mechanism? Just for TypeScript alone, there have been 754 reported bugs, 41 remaining open currently. Assuming we support TextMate, what would happen to our current issues related to syntax highlighting? Do we close them, and tell their authors to use the new syntax highlighting mechanism? If the users find issues in TextMate grammar files, do we accept their reports on this bug tracker? IOW, is it going to help reduce the number of remaining open issues here? |
|
Because TypeScript is a new language that evolves quickly ? Oniguruma + json like config certainly has better performance and reliability than current vim's mechanism. People seldom encounter such issues in syntax highlighting when using textmate/sublime2/vscode/eclipse/jetbrains. Sublime's grammar seems more readable and powerful than textmate, maybe oniguruma+config can achieve such thing ? |
|
I remember an issue where Vim was very slow when adding/removing text properties on With regards to how people perceive the current syntax highlighting as being too slow, I wonder which part of the issue comes from the syntax highlighting itself, and which part from something else like (too much redraw).
That's interesting. I hope it's really thanks to their own syntax highlighting mechanism, and not some other optimizations (like multithreading). |
|
A couple of remarks:
I think performance of vim syntax highlighting could be improved before trying alternatives, for example:
|
|
I want to add that we currently have no safe-guards for tree-sitter that are applied for regex-based highlighting like limiting the line number or doing background parsing like Atom would do.
I think background syntax highlighting (if you refer to asynchronous or separate threads highlighting) is neither implemented for tree-sitter nor for traditional vim highlighting. The possibility to make a fast thread-safe copy of the parsing state for tree-sitter or any other kind of multithreading is not used at the moment in Neovim. Many of the issues you cited complained about features missing due to missing About quality of the grammars, you surely have different trade-offs. VS-Code has significant more users than Atom and Nightly-Neovim. Tree-sitter parses the whole document which can help with complex syntax constructs and large-scale structure. However, it will easier get confused when it sees something that cannot be handled be the language grammar (preproc-constructs or non-standard language extensions) while regexes with a more local view are often still ok. The error recovering capabilities vary a lot on how the concrete grammar is written. Tree-sitter provides something in-between regex highlighting and LSP-like semantic highlighting, so it might not be necessary if the two latter are available for a language. Distributing binaries is another challenge for tree-sitter. Arbitrary code execution through custom scanners enables highest flexibility but may also pose a security risk though if the parsers are not self-generated and the scanner code is not reviewed. |
|
For those who haven't seen it, this is an excellent introduction to Tree-sitter, by the author: https://www.youtube.com/watch?v=Jes3bD6P0To&ab_channel=StrangeLoopConference tl;dr: Tree-sitter is a (portable, dependency-free) C library which (conceptually) takes a grammar (expressed in JavaScript) and a source file, and returns a parse tree for the source file with respect to the grammar. The big selling point is that TS (claims that it) can handle syntax errors well (still return a reasonable parse tree) and that it is incremental (returns new parse trees efficiently/quickly given some code edits and previous trees). Parsers for different languages are provided by the community and while I haven't seen this first-hand, I find it easy to believe that many of them are not great. But the project is much younger than TextMate, and GitHub uses it for its on-web syntax highlighting so there might be some corporate support there. Personally, the thing I would be most excited about seeing is Vim exposing a representation of the syntax tree which can be used not just for syntax coloring but also for semantic editing (expand visual selection one AST node up, copy function body, etc.). IDK how well the Vim architecture supports this today. But in theory you could then plug in whatever parse-tree-generator you choose (Tree-sitter or TextMate). If you are using an LSP language server, it's true that the LS can give you a parse tree (one which is even more accurate, esp. in the case of context-sensitive grammars like C++), but a language server will always be slower (it will do more than a parser, for example it will resolve cross-file deps and so on) and therefore will have to be async and higher-latency. So I think there is room for both a fast incremental parse system (like Tree-sitter) and LSP support (for things like go-to-definition and find usage). See also this discussion in the VSCode repo: microsoft/vscode#50140 |
|
As someone who has spent months writing and maintaining TextMate and tree-sitter grammars for real-world languages, let me tell you that the TextMate grammar system is totally broken, at least from a 2021 perspective. TextMate grammars are a nightmare to maintain and impossible to get right. Out of desperation, I even developed my own macro system (just like the authors of TypeScript's TextMate grammar), and it was still a nightmare. tree-sitter is in a completely different league. It's a top-notch incremental parser that can be used for accurate (!) syntax highlighting, code folding, code formatting, etc. tree-sitter grammars are dramatically easier to write and maintain, and it's actually possible to get them right. GitHub has been using tree-sitter for a while, and VSCode is also starting to use it (see https://github.com/microsoft/vscode-anycode). Betting on TextMate grammars in 2021 would be an engineering crime. |
I am not sure how much of your hyperbolic speech can be deemed accurate, but from what I can see one of the biggest problem with tree-sitter is the general low quality of parsers contributed by different people as pointed out by the OP. "Top-notch" is not the way I would describe it. Which certainly needs to be taken into account as it would require a vast amount of effort to deal with these issues Vim would inherit as a result of undertaking the HUGE project of integrating tree-sitter. I can't speak for textmate grammar for lack of familiarity. Personally my biggest problem with tree-sitter (at least the way neovim does it) is it's dependency on the environment (gcc/clang), large binary size and the do-it-all mentality which suits neovim but definitely does not feel like the "vim way". |
|
As someone who has spent months writing and maintaining TextMate and
tree-sitter grammars for real-world languages, let me tell you that
the TextMate grammar system is totally broken, at least from a 2021
perspective. TextMate grammars are a nightmare to maintain and
_impossible_ to get right. Out of desperation, I even developed my own
macro system (just like the authors of TypeScript's TextMate grammar),
and it was still a nightmare.
tree-sitter is in a completely different league. It's a top-notch
incremental parser that can be used for accurate (!) syntax
highlighting, code folding, code formatting, etc. tree-sitter grammars
are dramatically easier to write and maintain, and it's actually
possible to get them right. GitHub has been using tree-sitter for a
while, and VSCode is also starting to use it (see
https://github.com/microsoft/vscode-anycode).
Betting on TextMate grammars in 2021 would be an engineering crime.
Thanks for your opinion. Making it easier/simpler/better to write a
parser is an important goal. So we should look at the best way to use
tree-sitter. That it compiles each parser into an executable seems like
a disadvantage. Perhaps this is OK for often used languages, but a way
to add a parser at runtime would be really useful.
…--
TIM: Too late.
ARTHUR: What?
TIM: There he is!
[They all turn, and see a large white RABBIT lollop a few yards out of the
cave. Accompanied by terrifying chord and jarring metallic monster noise.]
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
/// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
|
If tree-sitter is top-notch, how come an ubiquitous and highly popular language like python is broken in it since quite a while? |
|
I think this discussion is devolving more and more from the purely technical and into prejudices. It is very important here to distinguish
I think Vim should at this stage focus on 1. to make a reasoned decision (while it of course makes good sense -- and would make me very happy -- to take Neovim's approach and decisions for 2. into account; admitting that the two projects have different needs). And I find it highly disingenuous to point fingers at 3. while ignoring that the quality of TextMate grammars (and, indeed, Vim's bundled syntax files) varies wildly as well. It's clear that (just like Neovim) you cannot simply switch engines but have to support both (on a per-language basis) for some time until the replacement catches up. |
|
I was obviously talking about the engine, which is what matters in the long run. Regarding existing grammars, the difference is that tree-sitter grammars can be improved relatively easily because they can be reasoned about. On the other hand, improving real-world TextMate grammars is anywhere from difficult to impossible. (Often, fixing one problem causes an inexplicable problem somewhere else, which is only discovered later.) I can't comment on integration aspects. I'm not even a Vim user. But as a language/tooling developer myself, I feel strongly that it's time to move past TextMate grammars, which is why I offered my insights. Good luck! |
@jgb Indentation has nothing to do with tree-sitter itself. There is a very ad-hoc implementation of using the parsed tree as indentexpr. Python indentation is not working because this implementation just considers the current syntax node you are currently on which is nothing in case of the Python parser because the relevant syntax node ended in the previous line when you start a new one. One would have to add a rule that respects this case or tune the general logic at this point. You always have to write some system that translates your parsed representation to indents. The quality of this translation says nothing about the quality of the representation itself. |
|
As someone who recently spent some time writing a TreeSitter grammar, I have also become less enthusiastic about the project. I watched the author’s presentation a while ago and it sounded like the greatest invention since sliced bread, but in practice it doesn’t always work that well. The biggest obstacle in my opinion is languages with preprocessors (e.g. C and C++). This isn’t something I had considered initially, but it is simply impossible to parse those languages with TreeSitter because you’re dealing with a language within a language. Now before someone mentions this: I know TreeSitter supports injections, e.g. JavaScript in HTML, but that’s not the same thing because, as I understand, each injection is essentially its own “program”. It’s fundamentally not possible to parse pre-processed languages with a context-free grammar. If you think about it, conditional compilation is as context-sensitive as it gets. I’m talking about constructs like this: #if FLAG
if (foo) {
#endif
bar;
#if FLAG
}
#endifOr this: #define BEGIN_FUNC void func() {
#define END_FUNC }
BEGIN_FUNC
bla;
END_FUNCOr this: #define RENAME(x) renamed_ ## x
void RENAME(my_func) {
bla;
}How is TreeSitter supposed to generate an AST for such code if it doesn’t interpret the macros? It’s simply impossible. And often this will result in parse errors. Now, TreeSitter is in theory “fault tolerant”, so it should be able to recover from errors, but I’ve found that it often recovers in a weird, unpredictable way that causes syntax highlighting to be messed up. It gets even worse when we’re talking about using it for features like syntax-aware selections, indentations and folds: Just forget about it. All TreeSitter grammars for preprocessed languages contain hacks to work around this issue, but they never work 100%. They just handle a few special cases, but blow up in the general case. The next problem is that parsing is incredibly slow. I benchmarked parsing a 4 MB file and it took over a second. Depending on where you are coming from, that might not sound too bad, but 4 MB a second really isn’t impressive when you consider that modern RAM can handle tens of gigabytes per second. Quite frankly, I’m not sure this “incremental parsing” approach is all that useful when the implementation is so slow in practice. I guarantee I could write a hand-rolled parser that would just reparse the entire file on every edit and it would still be orders of magnitudes faster. I’ve also found that syntactic highlighting doesn’t actually add that much value over a simple lexer, but it is significantly more complex. Semantic highlighting on the other hand is even more complex, but it also adds a lot of value. If I had to rate the cost-benefit relationship, I’d say: lexer > semantic > syntactic. If I had to design a syntax highlighting system from scratch, I’d probably just go with a simple C API, something like this: typedef enum {TOK_IDENT, TOK_STRING, TOK_OPERATOR, ...} Token;
void highlight_tokens(const char *buf, size_t len, Token *tokens, const void *input_state, void *output_state, size_t state_size);You just pass a chunk of data to the parser and then it returns a buffer with a character class for each character (or maybe an array of ranges, see also LSP for a similar approach). This is the most general form, giving you the greatest amount of flexibility. You could hand-roll a parser, or build one based on regexes or TreeSitter grammars or whatever. It doesn’t restrict you to a particular system. I’d even consider getting rid of the state persistence stuff and just pass one large buffer containing the entire file and reparse the whole file every time. Because in the general case, you have to do it anyway. Consider putting a comment |
|
Anyone who eagerly promotes tree-sitter here should answer my questions above first. Repeating its advantages a thousand times do not mean that these fatal problems will disappear. Tree-sitter is not a new thing, no need to be so excited. Remember that Atom has adopted tree-sitter early in 2018. Compared with excited neovim users, the atom community is very calm about this "new" feature. I don't need a better highlighter at the cost of perfomance and flexibility. Because I am suffering performance issues right now and all I want is a fast & static regex-based highlighting system. @lacygoill you claimed in this comment that the problem was caused by "drawing too much". That's not true, I have done a bisect investigation in this problem here: And found that there was a big performance regression after Still unsolved today, see its latest comment ! |
Treesitter provides a tool called Playground. It displays the whole AST with highlight groups. A user has to open this window, place the cursor on the symbol in source file so that relevant part of AST is highlighted, and thus you know the highlight group, and the range of text (in source file) it encompasses. On a general note, treesitter grammar bugs can be hard to fix. This gets harder if the author also used an external ad-hoc parser (written in C) in addition to treesitter's own parser. For non CFG like markdown using an external parser is not uncommon. |
|
The current highlight groups are somewhat of a blunt instrument. Setting |
|
For the record, this is why Neovim has added a (larger) number of standard groups for tree-sitter highlighting (inspired by TextMate/Sublime scopes, which Helix uses as well): https://neovim.io/doc/user/treesitter.html#treesitter-highlight-groups All official tree-sitter queries are enforced by CI to use these (only) for consistency (although the fallback mechanism makes it possible to gracefully handle further specialization and provides an automatic language-specialization for colorschemes to use). Your needs are of course different, but I'm pretty happy with it, and it might contain helpful inspiration. |
|
I'll comment only once more so as not to further disturb the great number of people likely following this conversation: my suggestion of expanding the standardized highlight group set was partially inspired by tree-sitter. I use Vim 9.1 (and vim9 script) so tree-sitter isn't a solution for me although I have not once felt the power and performance of the current syntax highlighting system to be insufficient (however, I do recognize some of the regex improvements suggested in comments above). In fact, Vim 9.1 performs exceptionally. I wholeheartedly believe in Vim's highly considered approach, addressing shortcomings of pioneer solutions ("doing it right," as Bram had said). With that, I assume a major change (behind a feature flag or otherwise) is not around the corner. If, however, the interface (highlight groups, in this case) can be determined with the level of rigor we've come to expect from changes in the Vim project, syntax maintainers would ideally eliminate reliance on bespoke highlight groups. This would result in a superior experience for both syntax maintainers and end users without the need for immediate introduction of a new parsing system. In the event of deeper systemic changes (introduction of systems akin to TextMate or tree-sitter), the highlight group interface would remain the same and end users would benefit immediately from any performance- or matching-related improvements. |
|
In the meantime... textmate highlighter: |
|
checking on treesitter status. sqlite3.c (222k lines of code)
tinywl.c (1000 lines of code)
my own thoughts if the aim of VIM is..
|
idk man, but if you are manually editing a file with 200k loc of c code, instead of programatically generating it, feels like you gotta be doing something wrong imho. i tried opening the treesitter c parser, and yes, it gets a little laggy. i did :TSDisable and then it got fast. it has 600k loc. |
|
Again, please evaluate the technology itself, not Neovim's (work in progress) implementation -- at least without educating yourself about the implementation (you can ask, you know!) In particular, we have not yet implemented a timeout such as the regex syntax engine has -- so you are comparing apples and oranges. It's on our todo list but not a very high priority since tree-sitter is so fast in general that you only really need it for "monster files" such as this. (Also make sure to always test the latest nightly version -- we provide appimages from our releases page -- since we make constant performance improvements.) |
|
Just so you know, I do love treesitter and I use it with neovim. I've experimented it with vim also: |
|
I'm not accusing you of anything; but as you can see from the discussion here, many comments are made without thought. And I do think "checking on treesitter status", followed by these examples, is misleading (even if not deliberately). So it's important to provide context if you want to have a meaningful discussion. If you don't do it, I will (to the limit of my ability and interest). |
|
The main drawback still applies: reliance on an external program. LSP is worse tho because it depends on an external program and a build configuration... and the toolchain... and the build system ;-) Especially with Linux kernel tree this quickly becomes apparent and it starts show its ugly face. TS is not as bad but is still quite faulty design given that it is not well integrated to the source code of vim. |
|
This is not to compare to neovim but it is fair to state that it does not have fully builtin TS support. You need a separate plugin nvim-treesitter for compiling parser and generally make it usable. I use this only to point out that there is no a proven example to this day of vim or any of it s forks demonstrating fully integrated TS support. Also lot of TS complexity comes from being cross-tool solution. If anything I'd love to see vim pursue to a non-generic designed for vim grammar solution, maybe something that would take advantage of the new vim9script. |
Writing a c++ parser in vim9script? |
Integrating Treesitter fully into Vim is not a good design choice. Treesitter uses a different parser for each language, which needs to be generated using a Treesitter seems to be an unsuitable solution for a lightweight editor like Vim. Its main selling point is the availability of an Abstract Syntax Tree (AST). While I wrote a small plugin to display devdoc and found that Treesitter could have made my job easier (I had to use Pandoc to generate an AST instead), it is still an overly complex solution if its primary role is to color syntax, a non-essential feature in my opinion. Other use cases, such as additional text objects or scaffolding for refactoring, are already covered by basic Vim functionalities. On the other hand, TextMate also has its issues. Its grammar differs significantly from Vim's syntax grammar, which is closely aligned with Vim's regex grammar. Integrating TextMate into Vim feels like forcing a square peg into a round hole unless Vim's regex grammar artifacts are modified to reflect TextMate's regex grammar. |
Why in the heck would you do that? |
For the memes, of course |
Sorry bro, I ignore memes. Obviously parser generator (just like with even flex and bison) needs to be written with something else than vim9script. Just thought that it might be usable for snippets of logic but obviously could be wrong too, since not a contributor. |
Agree with you. Tree-sitter is made to work with multiple editors. Vim would benefit most something with similar high-level ideas but made only for vim. Lot of meat and cruft in tree-sitter comes from it being "too generic". Nice idea but the implementation is not that great and adds bunch of extra dependencies. If I recall correctly, e.g. installing neovim pulled tree-sitter, which pulled whole nodejs as dep, which was just terrible. I mean for an "ubiquitos" test editor used in terminal you have to have nodejs installed to your machine. I'd like keep my computer nodejs crap free. |
You don't. (Please, get your facts right before making a claim.) |
But some of its parsers AFAIK anyway. You are correct about package deps. I.e. you don't need it but make tree-sitter usable, system in practice needs to have npm, right? Or can you get 100% user experience without npm? Getting facts exactly right is pretty hard in the case of neovim is hard because the tree-sitter implementation is half-broken, given that it is unusable without https://github.com/nvim-treesitter/nvim-treesitter. I guess one can sum that plugin as part of usable tree-sitter functionality. |
|
Yes, you can absolutely do without nodejs installed. And you do not need that plugin, anymore than you need vim-polyglot in Vim. (Source: Maintainer for both Neovim and nvim-treesitter.) And I would appreciate you not throwing around words like "broken" without actual experience. |
Node.js is required to parse the JavaScript grammar file when creating a new parser, as noted in the dependencies section. However, end users only need the compiled object file or C source file containing the language-specific parser. Therefore, if my recollection is correct, end users don't need to install Node.js. |
I'm unfamiliar what polyglot is. I only use a few plugins and most of them are from tpope: Just hoping that neither TS or LSP will never land as any sort of features to Vim. I'm happy to let NeoVIM to keep them. Since not a developer I leave it here ;-) And IMHO Vim does not need to compete with NeoVIM anyhow. Many systems programmers (like me) appreciate it being as low-level as it is because ubiquitos property is more essential than TS and LSP combined togethter. |
welcome you to continue to use vim native syntax hl method, it is mostly strong and stable, though maybe there some issues with specific terminal etc, maybe. |
|
This gentleman took a crack at a plugin for TextMate and Vim. Look what he has accomplished: https://github.com/icedman/vim-textmate |
I also took a crack at nvim plugin for textmate.. https://github.com/icedman/nvim-textmate 70+ more from the nvim users :) |
and others
Current problem
The current syntax highlighting system is very slow, and there are noticeable lags when scrolling large C++ files which contain complex syntax elements.
Issues of tree-sitter
Previously, most people suggest something like nvim-treesitter which will analyze source code in a background treesitter process and render keywords in the foreground with text-property.
But is it a good idea? I don't really think so,
at least 4 disadvantages for treesitter solutions:
changetickincrease to prevent such things, which is a little flaky.Background syntax highlighter is still immature, there are still many other strange issues in nvim-treesitter:
https://github.com/nvim-treesitter/nvim-treesitter/issues
If we introduce something like this, we shall take all these issues into account.
TextMate grammar system
Syntax highlighting is the most important part of an editor, better not rely on any uncontrollable external programs.
We need some new things that can satisfy such goals below:
And TextMate's grammar engine is really a good candidate which is widely used in many IDE/editors, including vscode (see syntax-highlight-guide for details), sublime and many others.
The grammar can be defined in JSON, that means can be translated into viml or just plain JSON files.
Possible Solution
We can specify which grammar engine to use for the given buffer:
And some new command can be used to change grammar engine:
for example, the snippet below can be included in the head of syntax files:
And lots of existing vscode/textmate syntax files can be reused with minimal modification.
The text was updated successfully, but these errors were encountered: