Syntax Highlighting for Markdown Documentation Comments

[Number-3434]

Please add syntax highlighting for documentation comments.

For example, the following:

/// ## Does something useful, involving [value]
///
/// ### Reason for This Method
/// This method is purely intended for demonstrative purposes.
/// Note the lack of any body, and the use of `...`
/// 
/// ### Here's a [table](https://www.markdownguide.org/extended-syntax/#tables):
/// 
/// | Nice Table | Value |
/// |   :---:    | :---: |
/// | Some Text  |  foo  |
/// | More Text  |  bar  |
/// 
/// **See also:**
/// - [num], some useful points here
///   - [int], a child of [num]
///   - [double], also a child of [num]
/// - [bool], a basic boolean
void usefulMethod(int value) { ... }

...should be coloured like:

## Does something useful, involving [value]

### Reason for This Method
This method is purely intended for demonstrative purposes.
Note the lack of any body, and the use of `...`

### Here's a [table](https://www.markdownguide.org/extended-syntax/#tables):

| Nice Table | Value |
|   :---:    | :---: |
| Some Text  |  foo  |
| More Text  |  bar  |

**See also:**
- [num], some useful points here
  - [int], a child of [num]
  - [double], also a child of [num]
- [bool], a basic boolean

...but using the local syntax highlighting rules.
Rendered as:

Does something useful, involving [value]

Reason for This Method

This method is purely intended for demonstrative purposes.
Note the lack of any body, and the use of ...

Here's a table:

Nice Table	Value
Some Text	foo
More Text	bar

See also:

• [num], some useful points here
  • [int], a child of [num]
  • [double], also a child of [num]
• [bool], a basic boolean

[DanTup]

@bwilkerson @srawlins any thoughts on this? On one hand, reproducing bits of markdown syntax highlighting inside Dart feels odd, but on the other hand there's already parsing of the markdown and we do already highlight references in the code.

Comparing this in a Dart comment to the VS Code built-in markdown I think it's mainly headers and links that could benefit from some highlighting. In theory we could produce tokens for those (perhaps just adding custom semantic token modifiers that can be defined in Dart-Code to map them onto some styles).

(probably it'd require offset/lengths from the markdown parser though, something we don't currently have)

[bwilkerson]

I think it would be nice to have, but I don't think it's very high priority for us given the other work that's on our list. I'd certainly be happy to work with an external contributor that was interested in adding it.

[srawlins]

I love the idea. There is some precedent; I dunno about other languages, but CodeMirror has had support for language blocks within language blocks, which is how some templating languages are done, like Mustache.

I dunno how we'd do it as our markdown parser doesn't offer up this data. See dart-lang/tools#1283 and dart-lang/tools#1287 (which references #3330 which is a subset of this issue).

[DanTup]

Yeah, I think it'd be better to do this when those markdown issues are done than something else (like regex). I don't currently plan to work on this, but it's useful having some extra context and opinions on issues when coming back to them (or to help if others are considering contributing).

[Number-3434]

Okay, so on hold for now?

Maybe could you add highlighting for dart code blocks in documentation comments as a starting point?

[bwilkerson]

Okay, so on hold for now?

Sorry, but yes.

Maybe could you add highlighting for dart code blocks in documentation comments as a starting point?

I believe we'd need the offset/length information from dart-lang/tools#1287 before we could do that.

[Number-3434]

Okay, so on hold for now?

Sorry, but yes.

Maybe could you add highlighting for dart code blocks in documentation comments as a starting point?

I believe we'd need the offset/length information from dart-lang/tools#1287 before we could do that.

Yeah, I feel that issue should have some more attention.

[Number-3434]

I just found out about embedded languages.

This means you could mark documentation comments as an embedded language to get the syntax highlighting.

The only snag is I'm not too sure how easy it will be then to highlight references.

[DanTup]

Unfortunately those embedded languages aren't supported with semantic tokens (see microsoft/vscode#163292).

That solution has the drawback of being VS Code-only, but if the server generated the tokens it could work for all clients.

[Number-3434]

Unfortunately those embedded languages aren't supported with semantic tokens (see microsoft/vscode#163292).

That solution has the drawback of being VS Code-only, but if the server generated the tokens it could work for all clients.

Okay, but personally I would prefer embedded languages if possible, as that would provide custom snippets, and Intellisense suggestions from VSCode, along with any other extensions / settings the user may have with them.

[Number-3434]

Oops, sorry about that

[DanTup]

Okay, but personally I would prefer embedded languages if possible, as that would provide custom snippets, and Intellisense suggestions from VSCode, along with any other extensions / settings the user may have with them.

Yeah, that's fair. Although right now unfortunately we can't do either (VS Code / LSP don't support embedding a language when using semantic tokens, and the markdown parser doesn't have enough info for us to do proper syntax highlighting). If one of them becomes possible before the other, it's something to consider even if the other way has some additional benefits.

[Number-3434]

Okay, thanks! I've taken a look at the VSCode support for semantic tokens in embedded languages, and given it my upvote. Thanks!