Tree sitter grammar

[amitu]

We should create it as many editors support syntax highlighting based on it.

https://www.jonashietala.se/blog/2024/03/19/lets_create_a_tree-sitter_grammar/

[siddhantk232]

I've done some work on this, definitely to not the point where I can open source it...hold on... I'll just put this under the org real quick. https://github.com/fastn-stack/tree-sitter-ftd/

[amitu]

This is awesome!

[amitu]

https://github.com/fastn-stack/tree-sitter-ftd/blob/main/grammar.js is at P1 level. P1 will not give great syntax highlighting etc, as it does not know when a component is getting defined or when we are importing something etc. Can we parse to our AST?

[siddhantk232]

the work you've linked is just me playing around with the treesitter DSL. What do you mean by P1? could you link some resources I can read on this. Also, is there some doc on our AST other than the source?

[siddhantk232]

tree-sitter also supports code folding in text editors which will be a good addition. I've talked about this before here: FifthTry/ui#58 (comment)

[siddhantk232]

tree-sitter grammar can also be used for syntax highlighting code on web pages. We can get rid of prism js files in fastn by introducing this as a compile time dep along with smaller css file. Github uses TS to highlight code on their website. They also have https://github.com/github-linguist/linguist where they allow accepting TS grammars to add a new language to GitHub.

[siddhantk232]

TS grammars are also suitable to use in LSP servers, which we should definitely work on, would be great to have autocomplete, auto import, snippets and docs inside the editor. Vim/Neovim, Emacs and Vscode supports have native support for LSP. We can also compile a lsp server to WASM and use it in our private editor product on the client side (not sure about the memory consumption issues).