Alternative frontmatter end / body start indicator

[AvdN]

I propose to change the end frontmatter end indictor from --- to --- |, or at least allow the presence of the |. This proposal does not affect the initial --- in the .md files.

The rationale for this is that this little change makes the combination of the frontmatter and the markdown body into a valid multi-document YAML file that can be processed, and generated using any, standards conforming, YAML parser, without the need to explicitly split the header from body. The only restriction for this to work for most standard conforming parsers is that the body (i.e. the markdown) should not contain lines starting with ... or --- followed by whitespace, but even this is not a requirement for all parsers (assuming you tell them up front how many documents are in the stream).

For those not familiar with the YAML specification:

• the --- at the beginning of a line, followed by whitespace, is the end-of-directives marker in YAML. In the absence of directives ( %YAML, %TAG) there is no need for a preceding document in a YAML document stream to be ended with an end-of-document directive (...), and documents can just be seperated by the end-of-directives marker.
• YAML documents starting with --- | consist of a single literal scalar. Such a "root-level" scalar, does not have to be indented and should be loaded keeping the linebreaks intact.
• YAML is a superset of JSON since the 1.2 specification (from 2009), so that existing standards conforming utilities/parsers can be used to easily update the header information.

If acceptable I can write a PR for this change.

[kristoff-it]

Thank you for this well explained proposal.

Unfortunately the choice of JSON for the frontmatter is only temporary and in the future we'll switch to a custom format that won't be compatible with YAML.

If you can give me an idea of the usecase / workflow that you had in mind (ie the reason why it would have been nice to use YAML tooling to generate the content file), I can then keep that in mind when designing the new system. This is not a promise that your use case will be fully supported by Zine in the future, but telling me about real world use cases never hurts :^)

Thank you!

[AvdN]

I generate all of my Markdown (and previously reStructuredText) files that contains code and the results of this code using ryd. That includes all my answers on StackOverflow since a few years, as well as the documentation for all but very old Python packages that I created ( over 250, a part of which are open source).

The .ryd files are multi-document YAML files with tags on the various documents that tell the program what to do with the part ( plain text (MarkDown, SO or reST, depending on the frontmatter document, Python code, Zig code, compiler output, captured stdout, check output against expected output etc.), that generate an appropriate output file. That way I am a sure that code in the documenation, or as part of an SO answer, actually corresponds to the program without me having to do any copy-pasting, and without having to do in-Markdown/reST updates of code and output segments. I used in-file updating before developing ryd, but that sometimes led to scrambled files and me having to revert to the latest committed version ...

So one thing I wanted to look into, is hooking that up ryd to zine, and there is no problem having ryd generate the front-matter seperately. But I have some tools that can go over the front-matter of a range of .ryd files and update fields (and also e.g. translate the reST text parts of a file to markdown (piping through pandoc), without having to do anything with the code and output related elements in the YAML document stream. And those tools could be used for the current frontmattered zine .md files.

I never understood why the R-docs and several other that have YAML frontmatter did not add the space + | to make the files acceptable to a YAML parser. But as the documentation already indicated JSON is temporary, and I assume you are waiting for being able to use the .zon format once a seperate parser for that is availble. Fully understandable.

I don't want to push things, but you could tag the frontmatter. If it stays essentially (nested) key-value pairs you could have the current

--- !JSON
{
  "title": "Hello World!",
  "date": "2020-07-06T00:00:00",
  "author": "Sample Author",
  "draft": false,
  "layout": "page.html",
  "tags": ["tag1", "tag2", "tag3"]
}  
--- |
# Home

Hello world!

or (done by hand no guarantee to be correct, not even sure if anything but structs or strings are allowed as value)

--- !ZON |
{
  .title: "Hello World!",
  .date: "2020-07-06T00:00:00",
  .author: "Sample Author",
  .draft: false,
  .layout: "page.html",
  .tags: .{ "tag1", "tag2", "tag3" },
}  
--- | 
# Home

Hello world!

(note the | after !ZON)

Both could be parsed with a YAML parser, and updated (at least if I would have a .zon loader/dumper), and the current tools are easily apated to the these "separators".

And yes, if you develop and maintain a YAML parser, everything looks like a YAML document stream >:-)