Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Markdown parser phase 2 #1480

Merged
merged 108 commits into from
Dec 13, 2022
Merged
Show file tree
Hide file tree
Changes from 82 commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
6a14315
New markdown config profiles structure
miteshashar Sep 19, 2022
7c2215b
Remove smartquotes. Correct profiles for update and venue
miteshashar Sep 20, 2022
a458903
Generic markdown profiles: basic, document, text-field. Future provis…
miteshashar Sep 20, 2022
b0c830f
Merge branch 'main' into markdown
miteshashar Sep 21, 2022
cc499bb
Merge branch 'main' into markdown
miteshashar Sep 21, 2022
5f9ac1f
Merge branch 'main' into markdown
miteshashar Sep 22, 2022
6cc62a0
Introduced MarkdownColumnNative, that uses the new markdown config pr…
miteshashar Sep 22, 2022
d874fd7
Use MarkdownColumnNative for venue(basic) and venue_room(document, fo…
miteshashar Sep 22, 2022
0f2852f
Removed funnel.utils.markdown.extmap #1483
miteshashar Sep 22, 2022
08f67d5
Revamp markdown tests for new config profile structure #1486
miteshashar Sep 23, 2022
6065bc7
Merge branch 'main' into markdown
miteshashar Sep 26, 2022
1e05ab1
Effort to clean up markdown tests by segregating and moving all debug…
miteshashar Sep 29, 2022
780c676
Merge branch 'main' into markdown
miteshashar Oct 3, 2022
f80f787
Merge branch 'markdown' into markdown-tests
miteshashar Oct 3, 2022
83a42ce
Merge branch 'main' into markdown-1493
miteshashar Oct 11, 2022
e01c620
Corrected rendering issues with markdown-it-py plugin for ins #1493
miteshashar Oct 11, 2022
2628e2d
Cleaned up and reintroduced markdown-it-py plugin for sub & del.
miteshashar Oct 11, 2022
4c61e7f
Ported markdown-it-sup #1493
miteshashar Oct 14, 2022
6708e79
Markdown-it-py sub tag plugin now ported from javascript. Correction …
miteshashar Oct 16, 2022
db0c476
Plugin for mark tag using == #1493
miteshashar Oct 17, 2022
4ff75c1
Merge branch 'main' into markdown-1496
miteshashar Oct 19, 2022
522f114
Markdown-it-py plugin for embeds, ported from mdit_py_plugins.contain…
miteshashar Oct 19, 2022
6f0c9da
Small correction to 522f114 #1496
miteshashar Oct 20, 2022
289dda2
Markmap, Mermaid and Vega integration handled with new Markdown #1496
miteshashar Oct 27, 2022
499566e
Resize markmap container on window resize #1496
miteshashar Oct 27, 2022
450e31c
Use document profile for Proposal.body #1485
miteshashar Oct 27, 2022
5a3f3f3
Merge branch 'main' into markdown
miteshashar Oct 27, 2022
40d6b1b
Resolves some of the issues in review for #1480
miteshashar Oct 28, 2022
1b868d3
Classes for markdown profiles. #1480
miteshashar Oct 30, 2022
faf7edf
Merge branch 'main' into markdown
miteshashar Oct 31, 2022
7c5603f
Merge branch 'markdown' into markdown-tests
miteshashar Oct 31, 2022
b16a2e7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 31, 2022
f4adbd2
Merge branch 'markdown' into markdown-tests
miteshashar Oct 31, 2022
07b85f3
Merge branch 'main' into markdown
miteshashar Oct 31, 2022
54fff74
Corrected preview bug - send profile to markdown API. Switched markdo…
miteshashar Oct 31, 2022
b6c1cf1
Merge branch 'markdown' into markdown-tests
miteshashar Oct 31, 2022
3ea2cfa
Better class-based tests structure for markdown. #1490
miteshashar Nov 3, 2022
73c692d
Functionality to update expected output for markdown tests. #1490 Res…
miteshashar Nov 3, 2022
740ea5c
Generate markdown debug output.html file. Cleanup. #1490 Also small c…
miteshashar Nov 3, 2022
756e882
Commit missed in 740ea5cbec09047904f665665497a601f765426d Related to …
miteshashar Nov 4, 2022
88582c2
Merge branch 'markdown-tests' into markdown-1493
miteshashar Nov 7, 2022
c7f9bca
Updated expected_output for the test basic.html to accommodate for ch…
miteshashar Nov 7, 2022
c9a84b3
Tests for markdown-it-py plugins for ins/del/sup/sub/mark #1493
miteshashar Nov 7, 2022
2105f27
Merge branch 'markdown-tests' into markdown-1496
miteshashar Nov 8, 2022
09a22ec
Tests for embeddable fenced blocks for markmap, vega-lite and mermaid…
miteshashar Nov 8, 2022
c433779
Tests for embeddable fenced blocks for markmap, vega-lite and mermaid…
miteshashar Nov 8, 2022
e516388
Merge branch 'main' into markdown
miteshashar Nov 11, 2022
34561c6
Merge branch 'markdown' into markdown-tests
miteshashar Nov 11, 2022
211f167
Merge branch 'markdown-tests' into markdown-1493
miteshashar Nov 11, 2022
2ddec91
Merge branch 'markdown-tests' into markdown-1496
miteshashar Nov 11, 2022
d41ece8
Introduce prismjs to render fenced blocks with language-* output. #1515
miteshashar Nov 14, 2022
34d598b
Merge branch 'main' into markdown
jace Nov 14, 2022
ad26b61
Fix delete operation in labels and submission form page (#1516)
vidya-ram Nov 15, 2022
02ed3d7
Add explicit timeout for all requests (#1521)
jace Nov 15, 2022
a1eafaf
Merge branch 'main' into markdown
miteshashar Nov 15, 2022
9a664eb
Merge branch 'markdown' into markdown-tests
miteshashar Nov 15, 2022
4e64ca9
Merge branch 'markdown-tests' into markdown-1493
miteshashar Nov 15, 2022
93666ec
Merge branch 'markdown-tests' into markdown-1496
miteshashar Nov 15, 2022
ec8793f
Merge branch 'markdown-tests' into markdown-1515
miteshashar Nov 15, 2022
339accf
Merge branch 'main' into markdown
jace Nov 16, 2022
69885ba
Merge branch 'main' into markdown
miteshashar Nov 25, 2022
a6c1a29
Merge branch 'markdown-tests' into markdown
miteshashar Nov 25, 2022
78007fc
Merge branch 'markdown-1493' into markdown
miteshashar Nov 25, 2022
7c5c831
Disable the strikethrough token for markdown's basic profile
miteshashar Nov 25, 2022
7270dd2
Merge branch 'markdown-1496' into markdown
miteshashar Nov 25, 2022
823b187
Changes in profiles. Basic is now just commonmark. Document is GFM pl…
miteshashar Nov 25, 2022
d30adfc
Updated expected output of markdown tests, subsequent to changes in p…
miteshashar Nov 25, 2022
6aa886c
In markdown_cached_column, moved the custom composite class outside t…
miteshashar Nov 26, 2022
0f73efe
Removed comments now not required. Corrected docstring for markdown()
miteshashar Nov 26, 2022
51e5197
Use new markdown_cached_column for all markdown columns #1485
miteshashar Nov 26, 2022
b4c728f
Removed imports not required anymore #1485
miteshashar Nov 28, 2022
dc6b2dd
ToC plugin #1533
miteshashar Dec 3, 2022
dcf2bc6
Merge branch 'markdown-1515' into markdown
miteshashar Dec 3, 2022
be4c6cc
Merge branch 'main' into markdown
miteshashar Dec 5, 2022
f551782
Merge branch 'main' into markdown
miteshashar Dec 6, 2022
67fb0af
Improved UX for embeds #1496
miteshashar Dec 7, 2022
5f79df0
Since markdown embed content is now in a hidden container, it can be …
miteshashar Dec 7, 2022
b60f21c
Call initEmbed for submission guidelines expansion.
miteshashar Dec 7, 2022
b7940b0
Prism activation bug for SPA navigation solved #1515
miteshashar Dec 7, 2022
970e7fd
Updated expected output for embed tests to accommodate the latest mar…
miteshashar Dec 7, 2022
fd33abf
Added linkedin URL with colons to markdown test case in links.toml
miteshashar Dec 7, 2022
6a5ea02
Use a registry for profiles
jace Dec 8, 2022
3ac45d6
Rename constructor to 'create'
jace Dec 9, 2022
f2e1f51
Move linkify config into profile
jace Dec 9, 2022
8d323e8
Merge profiles into base file
jace Dec 9, 2022
b2cc08c
Added comments for markdown tests. Changed parametrization for markdo…
miteshashar Dec 9, 2022
269a1f5
Use a dataclass for markdown parser config
jace Dec 11, 2022
7da7909
Remove default_factory for MarkdownConfig
jace Dec 12, 2022
30ac358
Add 'default' to allowed values for MarkdownConfig.preset.
miteshashar Dec 12, 2022
5cb9953
Move MarkdownCase and MarkdownTestRegistry inside pytest_generate_tests
miteshashar Dec 12, 2022
b7836f8
Move render method into config class
jace Dec 12, 2022
4818030
Empty config is not missing config
jace Dec 12, 2022
762ea43
Use single quotes and constant var
jace Dec 12, 2022
c524611
Moved pytest_generate_tests and the data for markdown to the markdown…
miteshashar Dec 12, 2022
256bd72
Additional string quotes and char constants
jace Dec 12, 2022
0a64f00
Merge branch 'markdown-temp-tests' into markdown
miteshashar Dec 12, 2022
914605a
MarkdownPlugin registry
jace Dec 12, 2022
d8b89fb
Fix docstring
jace Dec 12, 2022
3158ff2
Move conftest further down to subfolder
jace Dec 12, 2022
d1c0a4f
Fix CSS for typeform scroll bug.
miteshashar Dec 12, 2022
3c99f32
Merge branch 'main' into markdown
miteshashar Dec 12, 2022
bb41905
Rename unified_diff_output to fail_with_diff
miteshashar Dec 13, 2022
b5fe3a2
Added debounceInterval in markmap.js. Removed unnecessary console sta…
miteshashar Dec 13, 2022
35385f2
Bind resizeMarkmapContainers callback to Markmap in markmap.js
miteshashar Dec 13, 2022
c60b1e2
In initEmbed, send parentContainer to addVegaSupport.
miteshashar Dec 13, 2022
d63c59b
Markdown-it-py plugin to add language-none class code fences without …
miteshashar Dec 13, 2022
9b25f07
Use the document profile for the Update.body column.
miteshashar Dec 13, 2022
cde4fc9
Reverse unnamed function correction made in comments.js.
miteshashar Dec 13, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,10 @@ deps-dev:
deps-test:
pip-compile --upgrade requirements_test.in

tests-data: tests-data-md
tests-data: tests-data-markdown

tests-data-md:
pytest -m update_markdown_data
tests-data-markdown:
pytest -v -m update_markdown_data

debug-markdown-tests:
pytest -v -m debug_markdown_output
5 changes: 3 additions & 2 deletions funnel/assets/js/project_header.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import Video from './utils/embedvideo';
import Analytics from './utils/analytics';
import Spa from './utils/spahelper';
import Form from './utils/formhelper';
import TypeformEmbed from './utils/typeform_embed';
import initEmbed from './utils/initembed';

const Ticketing = {
init(tickets) {
Expand Down Expand Up @@ -173,6 +173,7 @@ $(() => {
event.preventDefault();
$(this).addClass('mui--hide');
$(this).next('.js-htmltruncate-full').removeClass('mui--hide');
initEmbed($(this).next('.js-htmltruncate-full'));
});

// Adding the embed video player
Expand Down Expand Up @@ -224,6 +225,6 @@ $(() => {
});

// Include parent container
TypeformEmbed.init('#about .markdown');
initEmbed('#about .markdown');
};
});
2 changes: 1 addition & 1 deletion funnel/assets/js/submission_form.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ $(() => {
'Content-Type': 'application/x-www-form-urlencoded',
},
body: new URLSearchParams({
type: 'submission',
profile: 'document',
text: view.state.doc.toString(),
}).toString(),
});
Expand Down
10 changes: 6 additions & 4 deletions funnel/assets/js/utils/initembed.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@ import addVegaSupport from './vegaembed';
import TypeformEmbed from './typeform_embed';
import MarkmapEmbed from './markmap';
import MermaidEmbed from './mermaid';
import PrismEmbed from './prism';

export default function initEmbed(markdownElem = '') {
export default function initEmbed(parentContainer = '') {
addVegaSupport();
if (markdownElem) TypeformEmbed.init(markdownElem);
MarkmapEmbed.init();
MermaidEmbed.init();
if (parentContainer) TypeformEmbed.init(parentContainer);
MarkmapEmbed.init(parentContainer);
MermaidEmbed.init(parentContainer);
PrismEmbed.init(parentContainer);
}
44 changes: 36 additions & 8 deletions funnel/assets/js/utils/markmap.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,34 @@
const MarkmapEmbed = {
addMarkmap() {
$('.language-markmap').each(function embedMarkmap() {
$(this).addClass('embed-added');
$(this).find('code').addClass('markmap');
});
window.markmap.autoLoader.renderAll();
const self = this;
self.container
.find('.md-embed-markmap:not(.activating):not(.activated)')
.each(function embedMarkmap() {
$(this).find('.embed-loading').html('Loading mindmap…');
$(this).addClass('activating');
const current = $(this).find('.embed-container');
current
.addClass('markmap')
.append(
`<script type="text/template">${$(this)
.find('.embed-content')
.text()}</script>`
);
window.markmap.autoLoader.renderAllUnder(this);
$(this).addClass('activated').removeClass('activating');
});
},
resizeTimer: null,
resizeMarkmapContainers() {
if (this.resizeTimer) clearTimeout(this.resizeTimer);
this.resizeTimer = setTimeout(() => {
$('.md-embed-markmap.activated svg').each(function mmresized() {
const circles = $(this).find('circle');
const firstNode = circles[circles.length - 1];
firstNode.dispatchEvent(new Event('click'));
miteshashar marked this conversation as resolved.
Show resolved Hide resolved
firstNode.dispatchEvent(new Event('click'));
});
}, 500);
miteshashar marked this conversation as resolved.
Show resolved Hide resolved
},
loadMarkmap() {
const self = this;
Expand Down Expand Up @@ -41,6 +65,7 @@ const MarkmapEmbed = {
loadMarkmapScript();
} else {
self.addMarkmap();
window.addEventListener('resize', this.resizeMarkmapContainers);
miteshashar marked this conversation as resolved.
Show resolved Hide resolved
}
});
};
Expand All @@ -50,9 +75,12 @@ const MarkmapEmbed = {
self.addMarkmap();
}
},
init(containerDiv) {
this.containerDiv = containerDiv;
if ($('.language-markmap').length > 0) {
init(container) {
this.container = $(container || 'body');
if (
this.container.find('.md-embed-markmap:not(.activated):not(.activated)').length >
0
) {
this.loadMarkmap();
}
},
Expand Down
37 changes: 31 additions & 6 deletions funnel/assets/js/utils/mermaid.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,30 @@
const MermaidEmbed = {
addMermaid() {
$('.language-mermaid').each(function embedMarkmap() {
$(this).addClass('embed-added');
$(this).find('code').addClass('mermaid');
const self = this;
let idCount = $('.md-embed-mermaid.activating, .md-embed-mermaid.activated').length;
const idMarker = 'mermaid_elem_';
const instances = self.container.find(
'.md-embed-mermaid:not(.activating):not(.activated)'
);
instances.each(function embedMermaid() {
const root = $(this);
$(this).find('.embed-loading').html('Loading visualisation&mldr;');
root.addClass('activating');
const contentElem = root.find('.embed-content');
const containerElem = root.find('.embed-container');
const definition = contentElem.text();
let elemId = containerElem.attr('id');
if (!elemId) {
elemId = `${idMarker}${idCount}`;
do {
idCount += 1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this required since since .md-embed-mermaid.activating is considered while taking count?

} while ($(`#${idMarker}${idCount}`).length > 0);
}
window.mermaid.render(elemId, definition, (svg) => {
containerElem.html(svg);
root.addClass('activated').removeClass('activating');
});
});
window.mermaid.initialize({ startOnLoad: true });
},
loadMermaid() {
const self = this;
Expand All @@ -14,14 +34,19 @@ const MermaidEmbed = {
dataType: 'script',
cache: true,
}).done(() => {
window.mermaid.initialize({ startOnLoad: false });
self.addMermaid();
});
} else {
self.addMermaid();
}
},
init() {
if ($('.language-mermaid').length > 0) {
init(container) {
this.container = $(container || 'body');
if (
this.container.find('.md-embed-mermaid:not(.activating):not(.activated)').length >
0
) {
this.loadMermaid();
}
},
Expand Down
57 changes: 57 additions & 0 deletions funnel/assets/js/utils/prism.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
const PrismEmbed = {
activatePrism() {
$('code[class*=language-]:not(.activated):not(.activating)').each(
function activate() {
window.Prism.highlightElement(this);
}
);
},
hooked: false,
loadPrism() {
const CDN_CSS = 'https://unpkg.com/prismjs/themes/prism.min.css';
const CDN = [
'https://unpkg.com/prismjs/components/prism-core.min.js',
'https://unpkg.com/prismjs/plugins/autoloader/prism-autoloader.min.js',
miteshashar marked this conversation as resolved.
Show resolved Hide resolved
];
let asset = 0;
const loadPrismStyle = () => {
$('head').append($(`<link href="${CDN_CSS}" rel="stylesheet"></link>`));
};
const loadPrismScript = () => {
$.ajax({
url: CDN[asset],
dataType: 'script',
cache: true,
}).done(() => {
if (asset < CDN.length - 1) {
asset += 1;
loadPrismScript();
} else {
if (!this.hooked) {
window.Prism.hooks.add('before-sanity-check', (env) => {
if (env.element) $(env.element).addClass('activating');
});
window.Prism.hooks.add('complete', (env) => {
if (env.element)
$(env.element).addClass('activated').removeClass('activating');
});
this.hooked = true;
}
this.activatePrism();
}
});
};
if (!window.Prism) {
loadPrismStyle();
loadPrismScript();
} else this.activatePrism();
},
init(containerDiv) {
this.containerDiv = containerDiv;
if ($('code[class*=language-]:not(.activated):not(.activating)').length > 0) {
this.loadPrism();
}
},
};

export default PrismEmbed;
46 changes: 31 additions & 15 deletions funnel/assets/js/utils/vegaembed.js
Original file line number Diff line number Diff line change
@@ -1,20 +1,36 @@
/* global vegaEmbed */

function addVegaChart() {
$('.language-vega-lite').each(function embedVegaChart() {
vegaEmbed(this, JSON.parse($(this).find('code').text()), {
renderer: 'svg',
actions: {
source: false,
editor: false,
compiled: false,
},
function addVegaChart(parentElement) {
parentElement
.find('.md-embed-vega-lite:not(.activating):not(.activated)')
.each(async function embedVegaChart() {
const root = $(this);
root.find('.embed-loading').html('Loading visualisation&mldr;');
root.addClass('activating');
const embedded = await vegaEmbed(
root.find('.embed-container')[0],
JSON.parse(root.find('.embed-content').text()),
{
renderer: 'svg',
actions: {
source: false,
editor: false,
compiled: false,
},
}
);
embedded.view.runAfter(() => {
root.addClass('activated').removeClass('activating');
});
});
});
}

function addVegaSupport() {
if ($('.language-vega-lite').length > 0) {
function addVegaSupport(container) {
const parentElement = $(container || 'body');
if (
parentElement.find('.md-embed-vega-lite:not(.activating):not(.activated)').length >
0
) {
const vegaliteCDN = [
'https://cdn.jsdelivr.net/npm/vega@5',
'https://cdn.jsdelivr.net/npm/vega-lite@5',
Expand All @@ -32,15 +48,15 @@ function addVegaSupport() {
vegaliteUrl += 1;
loadVegaScript();
}
// Once all vega js is loaded, initialize vega visualization on all pre tags with class 'language-vega-lite'
// Once all vega js is loaded, initialize vega visualization on all pre tags with class 'md-embed-vega-lite'
if (vegaliteUrl === vegaliteCDN.length) {
addVegaChart();
addVegaChart(parentElement);
}
});
};
loadVegaScript();
} else {
addVegaChart();
addVegaChart(parentElement);
}
}
}
Expand Down
Loading