New STProver architecture

[rpanic]

We want to build a STProver that has few properties:

1. Be Tree-agnostic, as long as the tree implementation has a checkMembership() and computeRoot() functionality
2. Be able to process multiple batches of connected ST batches
   This means that we want to encode the applying of a set of ST-batches, while being able to mark any batch on whether it should progress the state root or not. The output of this Prover will be a to stateRoot
3. Batches and their contents have to be assertable easily.
4. (not sure if that is needed - might be for merging) There should be a single assertable hash with a list of all applied STs, that can be asserted either in the blockProof/settlement/etc

Since we can make the assumptions that we have some fixed order in which we process ST-batches, we can safely encode the batches themself in a hashlist.

The new ST Prover can be imagined as two paths running independently and only at the end merging together and proving their equality.

The first path is the emitting part, where there is a set of circuits that run in a fixed order, which emit state transitions incrementally. In our case this would be Runtimes via the State abstraction and the Protocol + Block Producing circuits via their hooks.
Each part of the execution emits a independent set of STs (called a Batch) $B_n = { ST_1, ST_2, ..., ST_m }$ and a boolean indicator on whether those STs should be applied or thrown away.
Setting the indicator to $true$ means that all ST preconditions will be checked and all ST updates will be applied. The indicator being $false$ means only the preconditions will be checked and updates applied, but the end result being thrown away in favor of the state before that particular batch.

The set of all Batches will continously aggregated into a hash list based commitment as { stsHash, applied } where stsHash is itself a hash list commitment of all STs in that batch.

The second path is the state modification part of the circuits, in our case the STProver. It will take a set of Batches and a beginning state $S_0$ and apply the batches one by one to arrive at a end state $S_n$ along with a commitment to all executed batches $c(B_0, ..., B_n)$.

The algorithm by which it does this is fairly straight forward.
Define the state of the algorithm as a struct of: { currentBatch: { stsHash: Field, stagedRoot: Field }, batchListHash: Field, finalizedRoot: Field }

First, the set of batches { sts: ST[], applied: Bool }[] will be flattened trivially into { st: ST, type: "continue" | "closeAndApply" | "closeAndThrowAway" }[]

Then, for each of those entries:

1. Apply the current ST onto the staged root and append the ST to the stsHash
2. If type == "closeAndApply":
   1. Add the currentBatch to the batchListHash
   2. set finalizedRoot to stagedRoot
   3. Reset currentBatch
3. If type == "closeAndThrowAway":
   1. Add the currentBatch to the batchListHash
   2. Reset current batch

Notice that for closeAndThrowAway, we don't update the finalizedRoot, but instead throw away the result. This ensures that all preconditions of the STs are still checked, but no updated are applied.

Merging the two paths:
It makes sense to choose a very long interval in which the two paths run independently, because it improves the proving efficiency. When choosing to merge, two things have to be enforced:

1. Assert that the batchListHash of both paths match
2. The new output is set to the ST-path's finalizedRoot output

When this happens at any point in the proof chain, we assert the validity, completeness and correctness of the STs and the equality of both paths.

Example:

In our example, we start at stateroot $S_1$, protocol hooks that progress from $S_1 \to S_2$ , and a failed runtime method $S_2 \to S_3$. Finally, we have some imaginary afterTx hooks that progress from $S_2 \to S_4$.

A the emitted batch list looks like this:

$$\begin{aligned} \{ batch: ST_a, applied: true \}\\\ \{ batch: ST_b, applied: false \}\\\ \{ batch: ST_c, applied: true \} \end{aligned}$$

The corresponding steps in the ST Prover would look like

$$\begin{aligned} \{ finalizedStateRoot: S_1, stagedRoot: S_2, batch: ST_a \}\\\ \{ finalizedStateRoot: S_2, stagedRoot: S_3, batch: ST_b \}\\\ \{ finalizedStateRoot: S_2, stagedRoot: S_4, batch: ST_c \}\\\ \rightarrow output: S_4 (S_1 \rightarrow S_2 \rightarrow S_4) \end{aligned}$$

As we can see, the third batch, is based off $S_2$, so all STs that were emitted in the runtime, have been proven to be correct, but the stateroot has been thrown away.