Skip to content

Commit

Permalink
example of pandoc + nbconvert alternative
Browse files Browse the repository at this point in the history
  • Loading branch information
oliverlambson committed Aug 22, 2024
1 parent 1c21afc commit 4e6d283
Show file tree
Hide file tree
Showing 9 changed files with 170 additions and 0 deletions.
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,6 @@ Inspired by [evidence.dev](https://github.com/evidence-dev/evidence),
(1) I wanted to use python for data analysis and charting,
(2) I didn't want a static site, and
(3) I didn't want to pay for deployment.

You could also achieve something similar with pandoc and nbconvert,
see [examples/\_alternatives](./examples/_alternatives/) for more.
3 changes: 3 additions & 0 deletions examples/_alternatives/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.venv
.ipynb_checkpoints
*.pdf
71 changes: 71 additions & 0 deletions examples/_alternatives/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Reports from markdown

Using existing tools, instead of [bored-charts](https://www.github.com/oliverlambson/bored-charts)

We can achieve markdown with python charts -> pdf via pandoc and nbconvert:

1. Convert markdown to jupyter notebook with pandoc
2. Execute the notebook and output it to pdf with nbconvert

Two examples are provided: (1) a full example with separated python analysis and execution of
them in markdown code cells, and (2) a minimal example with inline python code. I really
don't like the inline code, at that point all we've done is make a worse jupyter notebook.
Remember, the point of this is to actually do our analysis as code so that it's separate, clear, and maybe even tested.

## Running the examples

You're going to need [uv](<(https://docs.astral.sh/uv/getting-started/installation/)>) and
[pandoc](https://pandoc.org/installing.html) installed.

```sh
uv venv
uv pip install -r requirements.txt
source .venv/bin/activate
./build.sh examples/full/main.md # or ./build.sh examples/minimal/main.md
```

## Comparison to bored-charts

<table>
<tr>
<th></th>
<th>pros</th>
<th>cons</th>
</tr>
<tr>
<th>pandoc + nbconvert</th>
<td>
<ul>
<li>uses existing tools</li>
</ul>
</td>
<td>
<ul>
<li>no live reports, only statically built<sup>[1]</sup></li>
<li>heavy dependencies</li>
<li>encourages inline code</li>
</ul>
</td>
</tr>
<tr>
<th>bored-charts</th>
<td>
<ul>
<li>live reports</li>
<li>lightweight</li>
<li>extensible</li>
</ul>
</td>
<td>
<ul>
<li>some guy's side project</li>
<li>limited analysis logic in markdown<sup>[2]</sup></li>
</ul>
</td>
</tr>
</table>

---

1. We could use voila for live reports, but at that point we've really building a massive ugly baby
2. I think this is a pro
25 changes: 25 additions & 0 deletions examples/_alternatives/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env bash

set -euo pipefail

file="${1:-}"
if [ -z "$file" ]; then
echo "Usage: $0 <file.md>"
exit 1
fi
name="${file%.md}"
if [ "$file" != "$name.md" ]; then
echo "File must be markdown"
exit 1
fi

echo "Preprocessing to make python code blocks compatible with pandoc"
sed -e 's/^```py$/```code/' -e 's/^```python$/```code/' "$file" >"$file.tmp"

echo "Converting to jupyter notebook"
pandoc -f markdown -t ipynb "$file.tmp" -o "$name.ipynb"
rm "$file.tmp"

echo "Exporting to PDF"
jupyter nbconvert --no-input --execute --allow-chromium-download --to webpdf "$name.ipynb"
rm "$name.ipynb"
Empty file.
14 changes: 14 additions & 0 deletions examples/_alternatives/reports/full/analysis/figures.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import numpy as np
import matplotlib.pyplot as plt


def cosplot():
x = np.linspace(0, 10, 100)
y = np.cos(x)

fig, ax = plt.subplots()
ax.plot(x, y)
ax.set_title("Cosine Wave")
ax.set_xlabel("x")
ax.set_ylabel("cos(x)")
return fig
14 changes: 14 additions & 0 deletions examples/_alternatives/reports/full/main.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# report on cosine

```py
# we're importing from the analysis package which is available in the same directory
# as this markdown report
from analysis import figures
```

Here's cosine:

```py
figures.cosplot();
# note the trailing simicolon, without it the plot will be displayed twice
```
36 changes: 36 additions & 0 deletions examples/_alternatives/reports/minimal/main.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# report on sine

> I'm not a fan of this because it's essentially a worse jupyter notebook.
> If you're going to colocate your analysis and commentary, you should use a jupyter notebook.
>
> The reason I don't want to use jupyter notebooks is because I want to do my analysis as code and commentary as markdown, then tie them together really easily.
Simple hello world print:

```py
print("Hello, world!")
```

Import numpy and pyplot.

```py
import numpy as np
import matplotlib.pyplot as plt
```

Generate some data.

```py
x = np.linspace(0, 10, 100)
y = np.sin(x)
```

Plot it:

```py
plt.plot(x, y)
plt.title("Sine Wave")
plt.xlabel("x")
plt.ylabel("sin(x)")
plt.show()
```
4 changes: 4 additions & 0 deletions examples/_alternatives/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
jupyter
nbconvert[webpdf]
numpy
matplotlib

0 comments on commit 4e6d283

Please sign in to comment.