Skip to content

Commit

Permalink
chore: update blog
Browse files Browse the repository at this point in the history
  • Loading branch information
@anirudhsudhir
anirudhsudhir committed Apr 4, 2024
1 parent 9314b3c commit c2576f4
Show file tree
Hide file tree
Showing 9 changed files with 4,318 additions and 1,123 deletions.
41 changes: 0 additions & 41 deletions search.go.temp
View file

This file was deleted.

126 changes: 83 additions & 43 deletions site/content/posts/fireside_anna.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ There are several amazing SSGs out there, like [Hugo](https://gohugo.io/) and
[11ty](https://www.11ty.dev/). Building your own SSG is an amazing learning
experience. It also motivates one to maintain and improve their personal site.


> Presented and written by Adhesh, Anirudh, Aditya and Nathan
Building personal blogs from the ground up can be a *tedious process*. Some of us
Expand All @@ -38,12 +37,14 @@ is a lot of time spent fixing things rather than getting productive work done.

> A static site generator is an immensely useful application
![](https://raw.githubusercontent.com/acmpesuecc/anna/main/site/static/images/posts/fireside-anna/lighthouse.png)
<!-- ![Lighthouse scores of the anna-docs page](https://raw.githubusercontent.com/acmpesuecc/anna/main/site/static/images/posts/fireside-anna/lighthouse.png) -->
![Lighthouse scores of the anna-docs page](/static/images/posts/fireside-anna/lighthouse.png)

It can simplify the whole process: allowing you to spend time and energy
It can simplify the whole process: allowing you to spend time and energy
on quality content. Keep reading to find out how we designed anna `v1.0.0`

---

## Introduction

ACM-PESU ECC conducts the ACM Industrial Experience Program (AIEP), an annual program that spans six weeks.
Expand All @@ -53,12 +54,12 @@ ACM-PESU ECC conducts the ACM Industrial Experience Program (AIEP), an annual pr
> projects. It is an excellent opportunity to interact with like-minded
> individuals.
Our AIEP team consisted of [Adhesh](https://github.com/DedLad), [Aditya](https://github.com/bwaklog),
Our AIEP team consisted of [Adhesh](https://github.com/DedLad), [Aditya](https://github.com/bwaklog),
[Nathan](https://github.com/polarhive), and [Anirudh](https://github.com/anirudhsudhir).

Our mentors (cool ass senior names!) gave us some great ideas for a team of us
four freshers. We were puzzled whether to build a distributed Postgres clone or
a load balancer.
four freshers.
We were puzzled whether to build a distributed Postgres clone or a load balancer.

Deep discussions over a week got us to the topic of making
blog sites and how tiring it is to work with, which only gets worse as you
Expand All @@ -82,23 +83,23 @@ A static site generator in Go
## The small but big decision!

Anna is written in [Go](https://go.dev). We considered writing it in Rust, but
that came with a steep learning curve. Go is a powerful language and has
excellent concurrency support, which suited our requirements to build a
performant application.
that came with a steep learning curve.
Go is a powerful language and has excellent concurrency support, which suited our requirements to build a performant application.

### What's in a name?

Probably the first thing that us four came across when joining ACM and HSP was
the famous Saaru repository. [Saaru](https://github.com/anirudhRowjee/saaru),
one of the projects that inspired our ssg, is derived from a [Kannada](https://en.wikipedia.org/wiki/Kannada) word.
Probably the first thing that us four came across when joining ACM and HSP was the famous Saaru repository.
[Saaru](https://github.com/anirudhRowjee/saaru),
one of the projects that inspired our ssg, is derived from a [Kannada](https://en.wikipedia.org/wiki/Kannada) word.
Saaru is a thin lentil soup, usually served with rice.

> In Kannada, rice is referred to as 'anna'( ಅನ್ನ) pronounced <i>/ɐnːɐ/</i>
> In Kannada, rice is referred to as 'anna'(ಅನ್ನ) pronounced <i>/ɐnːɐ/</i>
This was just a playful stunt that we played. We plan on beating Saaru at
buildtimes, optimizing at runtime.
build times, optimizing at runtime.

---

## Genesis

We began the project in a unique manner, with each of us creating our own
Expand All @@ -111,6 +112,7 @@ HTML was injected into a layout.html file and served over a local web server.
Later, we implemented a front matter YAML parser to retrieve page metadata

---

## What made us develop this to a great extent?

- Beginner-friendly: An easy setup wizard, easy and ready to use layouts, and themes. We want the
Expand All @@ -127,6 +129,7 @@ toolchain or escaping dependency hell.
domain names. Now their anna sites live on [hegde.live] and [sudhir.live]

---

## Benchmarks! Can anna lift??

In simple terms, to beat Saaru's render time (P.S. we did!). Something you
Expand All @@ -137,36 +140,71 @@ Adhesh was pretty excited to pick up Go and implement profiling, shaving
milliseconds off-of build times, when he implemented parallel rendering using
goroutines.

> - Switched to a two goroutine system
> - The main goroutine runs the application and renders pages
> - The second goroutine runs the local web server
> - Eliminated locks and restarting of application on file modification
## Prototype

The initial prototype built by Adhesh consisted of a multi-goroutine system.
A new goroutine would be spawned to walk the required directories.
If the current path being walked was a file, the path would be passed to another function along with its current modification time.

The previous mod time of the file would then be retrieved from a map holding the mod times of all the files:

- If the given file was freshly created, its modification time would be added to the map.
- If there was no change in the mod time, no changes would be made.
- If there was a change between the current and previous mod times, another function would be called.

The new function checks if a child process is running:

- For the first render, when a process has not been created, a new process is created that runs anna ("go run main.go --serve")
- For successive renders, the existing process is killed and a new process is spawned once again that runs anna.

This prototype was not very efficient as it created and killed processes for every change.
It had multiple goroutines attempting to walk the directories at the same time.
It also used multiple mutual exclusion locks to prevent data races.
Integrating this into the project also proved to be challenging.

## Improved version

The live reload feature was improved by Anirudh.
The updated version utilised two goroutines.

The main goroutine used the earlier file walker, with one important change: it sequentially traversed the directory without spawning new goroutines.
For any modification to a file in the current traversal, a vanilla render of the entire site would be performed.
The goroutine would then sleep for a specified duration (currently 1 second) before attempting the next directory traversal.

The secondary goroutine ran a local web server that served the rendered/ directory.

This eliminated all locks and avoided the creation and destruction of any child processes.

---

## We cook! 🍳

Here are some screenshots out of our group chats, that demonstrate build
times, profiling et-al when having 1000s of markdown files or in this case
Here are some screenshots out of our group chats, that demonstrate build times, profiling et-al when having thousands of markdown files or in this case
just copy-pasting a single markdown file en-mass!

![anna-bench](https://raw.githubusercontent.com/acmpesuecc/anna/main/site/static/images/posts/fireside-anna/bench.png)
<!-- ![Hyperfine benchmarks comparing the render times of anna, Saaru and 11ty](https://raw.githubusercontent.com/acmpesuecc/anna/main/site/static/images/posts/fireside-anna/bench.png) -->
![Hyperfine benchmarks comparing the render times of anna, Saaru and 11ty](/static/images/posts/fireside-anna/bench.png)

> After about 2 weeks of training (*ahem*) coding, we had a (merge) bringing parallel rendering and profiling to the table
### profiling
### Profiling

Heres the CPU profile of anna, generated using pprof.
This profile was generated while rendering this site.

Here's an SVG showing how much time each sys-call / process takes and how each block adds-up to render / build times

![anna-profiling](https://raw.githubusercontent.com/acmpesuecc/anna/main/site/static/images/posts/fireside-anna/cpu_prof.svg)
![CPU profile of an anna render generated using pprof](https://raw.githubusercontent.com/acmpesuecc/anna/main/site/static/images/posts/fireside-anna/cpu_prof.svg)
<!-- ![CPU profile of an anna render generated using pprof](/static/images/posts/fireside-anna/cpu_prof.svg) -->

You may wanna zoom-in about 3-4x times to get to see how our ssg works

---

## A big rewrite (when we went for a TDD approach)

Starting off this project, we kept adding functionality without optimization.
We didn’t have a proper structure; PRs would keep breaking features and
overwriting functions written by fellow team-mates.
We didn’t have a proper structure; PRs would keep breaking features and overwriting functions written by fellow team-mates.

### A new proposed rendering system

Expand All @@ -191,7 +229,7 @@ pkg
└── parser_integration_test.go
```

Currently there are two separate types of files that have to be rendered. One set includes user-defined files such as `index.md`, `docs.md` and various posts. These are specific to a user.
Currently there are two separate types of files that have to be rendered. One set includes user-defined files such as `index.md`, `docs.md` and various posts. These are specific to a user.

The second set of files that are rendered include `tags.html`, `sub-tags.html` and `posts.html`
Now, the generator/engine has a method to render "anna specific" pages and another method to render "user defined" pages which include all the user pages and posts
Expand All @@ -202,32 +240,32 @@ Here's some of Anirudh's work written during week-2
> - Wrote unit and integration tests for the parser and engine package
> - Split the rendering system to make parallelisation easier by switching to a three method system.
> - Render "user defined" pages which include all markdown files and posts (This method has been parallelised)
> - Render tags and tag-subpages separately, which could be parallelised in the future
> - Render tags and tag-sub pages separately, which could be parallelised in the future
> - Wrote a benchmark for main.go that times the entire application
---

## To search or not to search? 🤔

> That is the question > Is our *static site* becoming and at what cost?
We were wondering if we’d need a search function on our site since Google and
any other web-crawler index our site anyway. If we needed to implement it, we
had a constraint: we cannot use an API. It had to be static and local to be
user-friendly to work with. Aditya and Anirudh implemented a JSON index generator
that uses "Deep Data Merge" to index posts on our site.
user-friendly to work with.
Aditya and Anirudh implemented a JSON index generator that uses "Deep Data Merge" to index posts on our site.

This index is built at runtime and works without any lag or noticeable delay
when searching across posts. We mean to re-write it using WASM if necessary
and if it costs us time when performing searches.
This index is built at runtime and works without any lag or noticeable delay when searching across posts.
We mean to re-write it using WASM if necessary and if it costs us time when performing searches.

![anna-search](https://raw.githubusercontent.com/acmpesuecc/anna/main/site/static/images/posts/fireside-anna/search.gif)
<!-- ![anna-search](https://raw.githubusercontent.com/acmpesuecc/anna/main/site/static/images/posts/fireside-anna/search.gif) -->
![Demonstration of the search feature in anna](/static/images/posts/fireside-anna/search.gif)

## JS integration as plugins

Aditya added a field to our frontmatter which lets you pick and add certain JS
based snippets to your site. This way, you get to add `highlight.js` support, analytics scripts
and donation page widgets; that you can source from the `static/scripts` folder and toggle as
needed per-markdown page.
based snippets to your site.
This way, you get to add `highlight.js` support, analytics scripts and donation page widgets; that you can source from the `static/scripts` folder and toggle as needed per-markdown page.

## Wizard

Expand All @@ -239,16 +277,16 @@ validates fields using regex checks so you don’t need to worry about relative
paths in baseURLs, canonical links, and sitemaps. After successfully completing
the setup, the wizard launches a live preview of your site in a new tab.

![anna-search](https://raw.githubusercontent.com/acmpesuecc/anna/main/site/static/images/posts/fireside-anna/wizard.gif)
<!-- ![anna-search](https://raw.githubusercontent.com/acmpesuecc/anna/main/site/static/images/posts/fireside-anna/wizard.gif) -->
![Demonstration of the GUI wizard in anna](/static/images/posts/fireside-anna/wizard.gif)

### Raw HTML

What if you'd want to add a contact form to your site? or embedd YouTube videos or iframes of your choosing?
What if you'd want to add a contact form to your site? or embed YouTube videos or iframes of your choosing?

Anna let's us do that! Although, the point of a static site generator is to
quickly get to writing and focusing on the content. You can still embedd js
elements and iframe as needed to showcase any interesting YouTube videos or
to just rickroll people!
quickly get to writing and focusing on the content.
You can still embed js elements and iframe as needed to showcase any interesting YouTube videos or to just rickroll people!

## Tags

Expand All @@ -257,7 +295,8 @@ nice sub-page on your site so readers can discover similar content or browser
by category.

---
#### changelog: showcasing important additions --- as the weeks proceeded

### changelog: showcasing important additions --- as the weeks proceeded

Nathan:

Expand Down Expand Up @@ -285,9 +324,10 @@ Aditya:
- feat: json generator implementation along with a site wide search bar by @bwaklog in #70

---

## Feedback? / Learnings

We are at week: 3/6 and have a lot of things in store and bugs to squash!
We are at week: 3/6 and have a lot of things in store and bugs to squash!

> Feel free to ask any questions / send feature requests you'd like to see?
Expand Down
Loading

0 comments on commit c2576f4

Please sign in to comment.