Question on the Rust implementation

[LorisFriedel]

Hello!

I was looking at the Jsonnet benchmarks again and I found myself wondering why your implementation @CertainLach is just destroying every other one, while being more user-friendly most of the time?

I can't read Rust (for now 😄) but that would be awesome if you could help me/us understand what are the main differences in your implementation compared to the other one and making the Rust implem such a killer!

Thanks again for your work!

[CertainLach]

I didn't looked too much at other implementations, so I don't sure.
I was just writing this interpreter as I can see it should look like (thus getting various spec incompatibilities in the process, i.e: #125), and it did work well.

There is still some things regarding optimizations I want to implement, and which will make jrsonnet even faster:

1. Proper TCO: replacing recursive operations with loops in primary evaluation function, allows it to have tailstrict which will not be able to overflow the stack: WIP: Proper tco optimization #121. cpp-jsonnet does have this, but due to interpreter is fully written in this style, it is very hard to implement optimizations in it. My PR, on the other side, only rewrites main interpreter loop, leaving everything else intact.

2. Use indices in locals: In jrsonnet, local variables are stored in (pseudocode)

struct Locals {
  parent: Option<Locals>,
  current: HashMap<String, Thunk>,
}

Where String - variable name, and Thunk - variable value.

But it is possible to rewrite source code to use indices:

local a = 1, b = 2, c = a + b; c
=>
local <0> = 1, <1> = 2, <2> = <0> + <1>; <2>

And then store this in

PersistentVec<Thunk>

This optimization is implemented by every jsonnet interpreter: go-jsonnet, cpp-jsonnet, sjsonnet... Except jrsonnet.

3. Optimized AST storage. In jrsonnet, AST cache locality is awful, every AST node is stored in separate allocation, and there is a lot of Rc<Expr> going around... Not sure why it doesn't make jrsonnet the worst, but whatever. In master branch I have implemented rowan-based parser, and I want to use it for everything in the future.

4. Proper GC. GC in Rust is hard, most of the implementations are not as performant as golang/java GCs/boehm, and jrsonnet uses port of python cycle collector, which isn't fast. I want to switch to immix/other gc algorithm, but this requires writing code in style similar to this: https://github.com/withoutboats/shifgrethor (This will even allow to use https://github.com/mmtk/mmtk-core!), and this needs some of the nightly features.

[LorisFriedel]

And I thought this couldn't get any better! Thank you for the detailed insights of what's coming next, I'm really happy to see this implementation thrive as it does, it's helping a lot in many projects!

I'm rooting for you @CertainLach 🥇