Merge pull request #463 from pillar-markup/dev

Merge dev to master

doc/api.md

@@ -0,0 +1 @@
+# Microdown APIThere is a lot of functionality in Microdown, and this document gives an overview of the Microdown architecture.- Architecture - key objects in Microdown- Microdown facade methods- Resource references and resolvers. Dealing with links, images, and included documentsSeveral parts of Microdown allows for extending the Microdown tool suite. [Extension has is own documentation page](extension.md).The full suite of microdown tools are not loaded by default. In particular there is also support for - Translation of Microdown to Latex and Beamer- Translation of Microdown to HTML- Translation from Pillar to Microdown- Misc other tools## ArchitectureThe overall idea in Microdown is like this:![](https://www.planttext.com/api/plantuml/png/VP1D2u9048Rl-oi6xq9q35c4EdHG2fJkYmwofBioEqj2zDzJAoNxcECy3xmlR-nO4Vkc5iATjMaLgGO82rQcgX6k0leZwqsvjMIGOBqIDo5c8qXrGRQq5nF0IvzWPZqL256KCMbJIGaBOMSBtw3XNia9KSe5px4RsC5pw_c39egn-uttUPeiwRDH6CevUmD7HGvf5ARle8pn6pXffzb-uOy2VuInmZiVvelHbFtcTm00)&commentYou can edit the figure using this source on planttext.com @startumlskinparam rectangle {    roundCorner 20}rectangle "Microdown"  {    rectangle Source <<String>>     rectangle Document <<Tree>>    rectangle Text <<Output>>    rectangle Latex <<Output>>    rectangle HTML <<Output>>    Source --> Document : Parser    Document --> Text : Visitor    Document --> Latex : Visitor    Document --> HTML : Visitor    }@enduml &The format of the source code is explained in the [section on syntax](syntax.md).The parser translates the source code into an abstract syntax tree which we refer to as a "document". Translation into various output formats are done using visitors. By default only the visitor for translation into pharo `Text` is in the image. Sometimes `Text` is refered to as `RichText` to emphasize its difference from plain character `String`.## Microdown facade### Parse and output generationThe class `Microdown` has facade methods for the main actions in the architecture.- `Microdown class >> #parse:` translates a microdown source string into a document.- `Microdown class >> #asRichText:` translates a document into rich text. `asRichText:` can also accept a source string directly, cirumventing the `parse:` method. There are two facade methods for _resolution_. - `Microdown class >> #resolveDocument:withBase:` and - `Microdown class >> #resolveString:withBase:`The will be explained in the next section ## Resolution and referencesReferences to other documents or images can be either _absolute_ or _relative_. A typical absolute reference is `[Pharo](https:pharo.org)` or `![](file:/path/to/image.png)`. Absolute references cause no problems for the visitors to find the image or follow the link.However, when writing larger documents it is customary to keep images for the document in a seperate image folder, and refer to the images as `![](img/figure2.png)`. Such a reference is _relative_. It is relative to to location of the document containing the reference.**Resolution** is the process of replacing all relative references with absolute references. This process takes two inputs:- A document D containing relative references- The absolute location of document D (the base of the document).The two methods - `Microdown class >> #resolveDocument:withBase:` and - `Microdown class >> #resolveString:withBase:`are the prefered way to resolve documents.### Resource referencesIf we go one level below this, references are first order objects, all subclasses of `MicResourceReference`.In the standard image there are three kinds of absolute references:- http (and https) - for example: 'https://pharo.org/web/files/pharo.png'- file - for example 'file:/user/joe/docs/api.md'- the pharo image - for example 'pharo:///Object/iconNamed:/thumbsUp' (![](pharo:///Object/iconNamed:/thumbsUp))#### Creating referencesThe prefered way is to use the extension method `asMicResourceReference`, which is defined for- String - for example `'file:/user/joe/docs/api.md' asMicResourceReference`- ZnUrl - for example `(ZnUri fromString: 'https://pharo.org/web/files/pharo.png') asMicResourceReference`- FileReference - for example `(FileSystem memory) asMicResourceReference`#### Using referencesThere are two main key methods on references:- `loadImage` - returns a `Form` by reading the image (read using `ImageReadWriter class >> #formFromStream:`)- `loadMicrodown` - loads _and resolves_  a document#### BackgroundThe resolution is done using `ZnUrl >> #withRelativeReference:`, which implements the full resolution standard [RFC 3986 Section 5](https://datatracker.ietf.org/doc/html/rfc3986#section-5).##### Ambiguity of file uriIn pharo there is an issue that uri ' file://path/to/my/document.md' is ambiguous, because it can not be determined in which file system the path is supposed to be resolved (memory or disk).  This problem is dealt with, in the following manner:- `' file://path/to/my/document.md' asMicResourceReference` will always use the disk file system- `aFileReference asMicResourceReference` will create a `MicFileResourceReference` which remembers the file system in the file reference.- `aMicFileResourceReference loadDocument` will use the right filesystem for resolution.

doc/extension.md

@@ -0,0 +1 @@
+# Extension mechanisms

doc/readme.md

@@ -0,0 +1,3 @@
+# Microdown
+
+Microdown is the Pharo version of markdown. In general it is very close to github markdown, with two major differences and a number of smaller ones.Principal differences:- Microdown has a number of extension points to allow new functionality to be added without introducing new markdown syntax. In paticular Microdown has support for Latex, Pharo defined color highlighting of embedded code, inclusion of one document into an other- Microdown does not allow embedded html, as microdown is not restricted to being translated into html## Syntax[This document](syntax.md) defines the syntax of Microdown[This document](extension.md) defines the extension mechanisms of Microdown## APIAn important aspect of Microdown is its API, which allow you to work with Microdown from within Pharo.[This document](api.md) explaines the overall architecture of Microdown and the key methods.thods.

doc/syntax.md

@@ -0,0 +1,2 @@
+# Microdown Syntax
+The syntax covers two distict categories - inline and paragraphs. - Inline commands cover things like emphasis and links.- Paragraphs are commands that span several lines of text (lists, code examples, etc).## Inline syntax### Text emphasis- **bold** is done by `**bold**` (You can get \** by `\\**`)  - _italics_ is done by `_italics_` (should be underline, but now it stuck, there is no underline)- ~strike~ is done by `~strike~`- `inline code` is done by   \`inline code\`### Links and Figure- [Link](https://Pharo.org) is done as `[Link](https://Pharo.org)`. The produced link will open in the standard browser (Pharo `WebBrowser class >> #openOn:`)- ![alt text](https://pharo.org/web/files/pharo-logo-small.png) is done using ` ![alt text](https://pharo.org/web/files/pharo-logo-small.png) `.  Often larger figures are done by placing using this syntax on a line by itself### Microdown specials- $x^2$ - inline $\LaTeX$ is done using `$x^2$` (and `$LaTeX$`).- {{**bold**}} can be done also using raw### Microdown inline extensionsThere is potentially an endless number of extensions one would like to add to Microdown. Rather than keep inventing new syntax, the generic syntax for inline syntax is:`{!extensionName|parameter1=value1&parameter2=value2&parameter3=value3!}`. What the extension does will typically depend on the vistor ($\LaTeX$ generation, Text generation, HTML generation etc)Some extensions are already defined in the Microdown library:- **inputFile|path=uri** - insert the contents of the microdown document at uri at this place.- **footnote|note=some note which goes to the foot**  - adds a footnote the the generated document- **citation|ref=somewhere so others can find it** - adds a reference to the generated document## Microdown Paragraphs### HeadersHeaders are done by lines starting with `#`. One `#` is the largest header, six `######` is the smallest header. The header text is written after the `#` signs.`##### This is a header level 5, done as `##### This is a header level 5` ### CommentsIt is possible to put comments in the Microdown source. Lines starting with `%` are creates a comment paragraph, but most tools ignore them.% They really are### Quote blocks> Lines starting with `>` are supposed to be rendered in an indented manner> At present the in-image rendering leaves something to be desired>  But it works for nowEach source line which is part of the quote block need to start with a `>`### Unordered listsUnordered list are made by prefixing a line with '-' or `*`.* item 1- item 2* It is possible to make a long item  with several lines of text by indenting the following lines   by 2 spaces (two!, not three or one or tab)#### Indented lists* Item 1  * by prefixing the `-` or `*` by two spaces, one indent the list by one level    * Item 1.1.1  * Item 1.2* Againg, the indentation is two spaces!    * If you indent by say 4 spaces instead of two, it is just seen as a new line in the preceding bullet point### Ordered listsOrdered lists are done by prefixing a line with a digit followed by `.` or `)`.2. First item4) The numbers need not be in order, the rendering will start the numbering from one and increment### Mixed listsIt is possible to mix the two types of lists1. This is first item, made using `1. This is first item`   - sub item   * Indenting sublist is tricky, but the bullet (`*` or '-') must be matched up under the first letter in the line `This is the ...`      In this case 3 spaces of indentation `1._`1. Next numbered item### Horizontal lines***Are created by a single line starting with at least three `*` - this works: `***` as well as `*************`*************### Code blocksTo show code one use a tripple  backquote```text  ```pharo	Metacello new		repository: 'github://svenvc/zinc/repository';		baseline: 'ZincHTTPComponents'; 		load.  ``````produces:```pharo  Metacello new    repository: 'github://svenvc/zinc/repository';    baseline: 'ZincHTTPComponents';    load.```#### Predefind language modifiersThe `pharo` modifier used above is good for pharo expressions. Often one will show a full method,  and should use `method` to get a better highlighting and coloring of the method header```text  ```method    accept: aVisitor           ^ aVisitor visitCode: self  ``````produces:```methodaccept: aVisitor 	^ aVisitor visitCode: self```Plain unformatted text us made by using the `text` modifier, which will just print the body as is.```text  ```text    accept: aVisitor           ^ aVisitor visitCode: self  ``````produces:```textaccept: aVisitor 	^ aVisitor visitCode: self```### TablesIt is possible to write tables in markup. These tables are intended to be short and not too wide. The cells of the tables can only be markedup with inline markup.```text| Header1 | Header2 | Header 3 || --- | ---- | --- || cell 1 1 | cell 1 2 | cell 1 3 || cell 2 1 | cell 2 2 | cell 2 3 |```produces:| Header1 | Header2 | Header 3 || --- | ---- | --- || cell 1 1 | cell 1 2 | cell 1 3 || cell 2 1 | cell 2 2 | cell 2 3 |**Notice**, unlike some other markdowns, Microdown requires lines to start with `|`. There has to be at least the `---` in the line after the header. Leaving out the second line merely produce a table without header highlighting.| Header1 | Header2 | Header 3 || cell 1 1 | cell 1 2 | cell 1 3 || cell 2 1 | cell 2 2 | cell 2 3 |

src/BaselineOfMicrodown/BaselineOfMicrodown.class.st

@@ -13,12 +13,8 @@ BaselineOfMicrodown >> baseline: spec [
 
 	<baseline>
 	spec for: #common do: [
-		spec
-			baseline: 'PillarCore'
-			with: [
-				spec  
-					loads: #('Pillar-Core');
-					repository: 'github://pillar-markup/pillar:dev-8' ].
+		self pillar: spec.
+		self xmlParserHtml: spec.				
 
 		spec
 			package: #Microdown;
@@ -35,6 +31,16 @@ BaselineOfMicrodown >> baseline: spec [
 			package: #'Microdown-MathFlaky-Tests'
 				with: [ spec requires: #( #'Microdown-RichTextComposer-Tests' ) ];
 
+			package: #'Microdown-RichTextPresenter'
+				with: [ spec requires: #( #Microdown ) ];
+			package: #'Microdown-RichTextPresenter-Tests'
+				with: [ spec requires: #( #'Microdown-RichTextPresenter' ) ];
+
+			package: #'Microdown-DocumentBrowser'
+				with: [ spec requires: #( #Microdown #'Microdown-RichTextPresenter') ];
+			package: #'Microdown-DocumentBrowser-Tests'
+				with: [ spec requires: #( #'Microdown-DocumentBrowser' ) ];
+
 			package: #'Microdown-Transformer'
 				with: [ spec requires: #( #Microdown ) ];
 			package: #'Microdown-Transformer-Tests'
@@ -48,7 +54,7 @@ BaselineOfMicrodown >> baseline: spec [
 			package: #'Microdown-HTMLExporter'
 				with: [ spec requires: #( #Microdown #'Microdown-LaTeXExporter') ];
 			package: #'Microdown-HTMLExporter-Tests'
-				with: [ spec requires: #( #'Microdown-HTMLExporter' 'Microdown-Tests') ];				
+				with: [ spec requires: #( #'Microdown-HTMLExporter' 'Microdown-Tests' 'XMLParserHTML') ];			
 
 			package: #'Microdown-LaTeXExporter'
 				with: [ spec requires: #( #Microdown ) ];
@@ -74,22 +80,59 @@ BaselineOfMicrodown >> baseline: spec [
 				with: [ spec requires: #( #'Microdown-Pillar' ) ];	
 
 			package: #'Microdown-PrettyPrinter-Tests'
-				with: [ spec requires: #( #'Microdown-PrettyPrinter' #'Microdown-Tests') ].
+				with: [ spec requires: #( #'Microdown-PrettyPrinter' #'Microdown-Tests') ];
+
+			package: 'Microdown-Macrodown'
+				with: [ spec requires: #( 'Microdown') ];
+			package: #'Microdown-Macrodown-Pillar'
+				with: [ spec requires: #( 'Microdown-Macrodown' 'PillarCore') ];
+			package: 'Microdown-Macrodown-Tests'
+				with: [ spec requires: #( 'Microdown-Macrodown' 'Microdown-Tests') ].
+
+
 
 		"I do not want group without tests for now"
 		spec
 			group: 'Core' with: #('Microdown');
 			group: 'Tests' with: #('Core' 'Microdown-Tests');
 			group: 'RichText' with: #('Core' 'Microdown-RichTextComposer' );
-			group: 'Extensions' with: #(   
+			group: 'Extensions' with: #(
+				#'Microdown-RichTextPresenter'
+				#'Microdown-RichTextPresenter-Tests'
+				#'Microdown-DocumentBrowser'
+				#'Microdown-DocumentBrowser-Tests' 
 				#'Microdown-Evaluator'
 				#'Microdown-Evaluator-Tests'
 				#'Microdown-PrettyPrinter'
 				#'Microdown-PrettyPrinter-Tests'
 				#'Microdown-HTMLExporter'
+				#'Microdown-HTMLExporter-Tests'				
 				#'Microdown-LaTeXExporter'
 				#'Microdown-LaTeXExporter-Tests'
 				#'Microdown-Transformer'
 				#'Microdown-Transformer-Tests');
-			group: 'All' with: #('Core' 'Tests' 'Extensions' #'Microdown-Pharo-Tools' 'RichText') ]
+			group: 'Macrodown' with: #('Core' 'Microdown-Pillar' 'Microdown-Macrodown' 'Microdown-Macrodown-Tests');
+			group: 'All' with: #('Core' 'Tests' 'Extensions' 'Macrodown' 'Microdown-Pharo-Tools' 'RichText') ]
+]
+
+{ #category : #'external projects' }
+BaselineOfMicrodown >> pillar: spec [
+
+	spec
+		baseline: 'PillarCore'
+		with: [
+			spec  
+				loads: #('Pillar-Core');
+				repository: 'github://pillar-markup/pillar:dev-8' ]
+]
+
+{ #category : #'external projects' }
+BaselineOfMicrodown >> xmlParserHtml: spec [
+
+	spec
+		baseline: 'XMLParserHTML'
+		with: [ 
+			spec
+				loads: #('Core');
+				repository: 'github://pharo-contributions/XML-XMLParserHTML/src' ]
 ]

src/BaselineOfMicrodownH5P/BaselineOfMicrodownH5P.class.st

@@ -23,14 +23,21 @@ Class {
 BaselineOfMicrodownH5P >> baseline: spec [
 	<baseline>
 
+	spec
+		for: #(unix osx)
+		do: [ spec 
+					baseline: 'OSSubprocess' 
+					with: [ spec repository: 'github://pharo-contributions/OSSubprocess:v1.4.0/repository' ] ].
+
 	spec for: #'common' do: [
+		"probably not needed with the latest version e.g. jpeg production."
 		spec 
 			baseline: 'Mustache'
 			with: [ spec
 						repository: 'github://noha/mustache:v1.0/repository';
 						loads: #('Core' 'Tests') ].
 		spec 
 			package: #'Microdown-H5P' 
-			with: [ spec requires: #('Mustache') ]
+			with: [ spec requires: #('Mustache' 'OSSubprocess') ]
 	]
 ]