Non-verbatim fenced code blocks - with Markdown formatting

[dmitryperets]

Hi,

In Pandoc Markdown, fenced code blocks are treated verbatim. Hence, all the Markdown syntax is ignored.
This means that we cannot do any Markdown-style formatting inside fenced code blocks.

For example:

```
This is some ===highlight===
   This is indentation with some **bold** text
      This is some [custom style added by the user]{.customStyle}
```

None of the above will work, because the text inside is not treated as Markdown, so it will just appear verbatim.

But what if I do want it to be treated as Markdown?

I came up with a sort-of solution, tailored to my needs - since I am rendering into HTML.
So I can use a RawBlock to treat the text inside the fenced code block directly as HTML.
This LUA filter does the trick:

function CodeBlock(elem)
  html = "<pre><code>"..elem.text.."</code></pre>"
  return pandoc.RawBlock('html', html)
end

Unfortunately, this still doesn't let me apply the styling using Markdown syntax. I have to use HTML syntax:

```
This is some <mark>highlight</mark>
   This is indentation with some <b>bold</b> text
      This is some <span class="customStyle">custom style added by the user</span>
```

This works, but I hate using the target format (HTML) in my markup files.

Wouldn't it be a good enhancement request to support the non-verbatim fenced code blocks?
I think it is somewhat similar to the idea of parsed raw blocks (#7753), just in this case the desired Format is "markdown".

Or maybe I am missing a simpler solution to my problem?

[jgm]

You could use pandoc.read to parse the text contents of your code block as markdown, then pandoc.write to convert it to HTML, and include that in the raw block with surround pre tags...

[dmitryperets]

@jgm can I preserve the identation somehow?

Tried this:

function CodeBlock(elem)
  md = pandoc.read(elem.text)
  html = "<pre><code>"..pandoc.write(md, 'html').."</code></pre>"
  return pandoc.RawBlock('html', html)
end

It loses the indentation:

<pre><code><p>This is some <em>italics</em> This is indentation with some
<strong>bold</strong> text This is some <span class="green">custom style
added by the user</span></p></code></pre>

And another problem is with extra spaces between words... Even if I add | in front of each line (to preserve indentation), I still lose the extra spaces/tabs between words, as if.

[jgm]

You might try just using a line block (look up in manual). This will preserve indentation and spacing and allow markdown formatting. Might be what you want.

[dmitryperets]

Unfortunately, while the line blocks preserve indentation and allow markdown formatting, they do not preserve spacing...

| This is some _italics_ and     some    extra    spaces    between    words
|   This is indentation with some **bold** text  
|      This is some [custom style added by the user]{.green}  

The above is rendered as:

<div class="line-block">This is some <em>italics</em> and some extra
spaces between words<br />
  This is indentation with some <strong>bold</strong> text<br />
     This is some <span class="green">custom style added by the
user</span></div>

Note that the extra spaces between the words are lost... I'd need to manually escape them.

That's exactly what I am struggling with... cannot get all the three elements at the same time (indentation + spacing + markdown formatting).

[jgm]

You can preserve spacing too if you use \ which will give you a nonbreaking space.

[dmitryperets]

For the reference - this LUA filter eventually did the job for me, so it seems:

function CodeBlock(elem)
  local escaped_spaces = elem.text:gsub(' ', '\\ ')
  local md = pandoc.read(escaped_spaces)
  local html = "<pre><code>"..pandoc.write(md, 'html', pandoc.WriterOptions {prefer_ascii=true}).."</code></pre>"
  html = html:gsub('&#xA0;', ' ')
  return pandoc.RawBlock('html', html)
end

The idea is to escape the spaces, so that pandoc.read() doesn't get rid of them when parsing the text as markdown. This preserves all the indentation and spacing in the AST returned by read().
Then we write the result as HTML using pandoc.write(). The indendation and the spaces are then rendered as non-breaking spaces.
It's already fine, but if we want to make sure that the resulting HTML looks neat to the eye, then we also reverse the substitution (the last ":gsub" line), to return to the normal spaces.

End-result:

<pre><code>This is some <em>italics</em> and     some    extra    spaces    between    words
   This is indentation with some <b>bold</b> text
      This is some <span class="green">custom style added by the user</span></code></pre>

This is displayed properly (with all the formatting, indentation and spacing!).
And also WeasyPrint renders a good PDF out of it - which is my end-goal.