[wip] [tracking] Using reth and revm in our stack

[0xaatif]

We're playing with the idea of (also) using reth and revm in the same places we use jerigon.

flowchart TD
jerigon --execution traces--> zero-bin --reshape--> trace_decoder --trim/reshape-->  evm_arithmetization --prove--> plonky2
native-tracer --execution traces--> zero-bin
zeth["'zeth'"] --execution traces--> zero-bin

Loading

This issue charts a path for this integration.

fraktal-zkp/zeth

Folks on the team have seen the zeth prototype stack successfully prove transactions with our zk_evm.
This consists of:

• Changes to zero_bin to handle the new client: permalink
• Changes to revm to allow it to collect execution traces: comparison
• Changes to reth to plumb the traces: comparison
• Trivial change to revm-inspectors to juggle dependencies.

SMT support

We would want our zeth client to support using SMTs as the "world" storage.
I'm aware of two SMT "flavours": a "standard SMT", described in this post, and a "modified SMT" used in practice by Hermez.
It is NOT clear what the differences are, but it has been noted that they are not compatible.
It is NOT clear at this time which we want to target for this work.
SMTs also use a dense binary serialization format.

scroll-tech has a fork of reth, broken down in paradigmxyz/reth#11830, which abstracts over storage formats for their own zktrie structure.
We should be able to build off this work for our needs.
We may want to use their zktrie directly, but note that we'd need to pull in a go toolchain into our repo.
If not, revm's trait Host is the right level to target.

Strawman decision/workstream

• Validate native tracers with reth #790 validate using native tracers for reth. We need to write in a zero tracer for (efficient) witness extraction, and according to Arnau, there are some edge cases where native tracing is not appropriate.

  • We're not doing this on the advice of @Nashtare, in slack:

    Note that native tracers from unmodified clients have inherent limitations that make them unsuitable for production (unprovable blocks), hence the need even with Reth to use some kind of zero-tracer either what Francesco implemented or another approach.

• Repro of zeth demo #777

• Check in a test with the above into zk_evm CI. (medium, low risk (because we already do similar things in CI, right?))

  • Acceptance:
    1. assert that zeth produces a given witness file after a particular RPC call.
    2. assert that a given witness proves in zk_evm.

• Do we want to own/collaborate/drive/fund work on zeth?

  • [own it in 0xPolygonZero] Should we fork the above repo, or start from scratch?
    Note that fraktal-zkp/zeth does not list its license. paradigmxyz/reth and bluealloy/revm both list MIT licenses.
    • [start from scratch] in a fresh fork of revm
      • Forking reth looks like it'll be a necessity for the plumbing work - existing execution extension and other APIs are not sufficient to get the traces we need.
      • Forking will be inevitable was we'll want to upstream changes. It looks like reth is reasonably open to external contributions.
      • It looks like the evm_interpreter changes should be trivial to implement in terms of the public trait Database, so we may not need to fork it, but we will need to change the code in reth that runs the interpreter.
      • Have a current fork of reth, with changes as outlined above producing the same snapshot as zeth (medium, low risk - we're building off proven work).
        • Acceptance:
          • Repo does not need CI other than the snapshot testing.
      • Open a PR against reth with our changes. Likely to be mired in API discussion, so we'll need a viable plan to keep our branch current. (medium, low risk but long tail)
    • How do we do cross-repo testing?
      • Having each repo pull in the latest version of the other to do "current" testing is going to be painful - let's agree on a snapshot, and advance that in-step, decoupling the two repos.

• Our current SMT library is unsuitable for use in a client, and probably needs to be rewritten, or heavily refactored to be used in the way we'd need: smt_trie API should be suitable for trace_decoder #706 (medium, low risk, nonblocking).

• spike: start introducing an SMT into DatabaseState abstraction paradigmxyz/reth#11786, and note the key APIs that might need tweaking

Secondary objective to ramp up on reth/revm for future SP1/RISV/proving work.

[pgebheim]

Thanks Aatif,

@wborgeaud -- could you comment on the difference in the smts?

I think now is the time to cut to the more vanilla implementation.

[0xaatif]

Comment from @paulgoleary in stand-up today was that Hermez compatibility is NOT what we mean by "SMT support".
Does that not mean that the work looks a bit like adding Type 2 support?

• new client exposes (probably scroll's) SMT format (probably in binary).
  • Surely we don't need to do anything here if scroll are working on this, and we plan on treating their thing as "the most standard SMT"?
• new work in trace_decoder to parse and pre-process that format.
• new work in evm_arithmetization to prove off the new SMT format.

[paulgoleary]

Thanks Aatif.

It's been noted elsewhere but worth doc'ing here that it is not a goal of this work (afaik) to be compatible with the Hermez SMT. In fact, we should be as standard as possible.

A related point is that we expect that the REVM zkVM compilation path (e.g. Succinct RSP) to also be a viable proving stack for use in the CDK. This emphasizes the need to be as standard 'as possible' since one of the major benefits of the approach is the ability to compile the 'standard' REVM directly to a prover.

[Nashtare]

I'm aware of two SMT "flavours": a "standard SMT", described in this post, and a "modified SMT" used in practice by Hermez.
It is NOT clear what the differences are, but it has been noted that they are not compatible.
It is NOT clear at this time which we want to target for this work.

Because of time/resource constraints, we'd probably want to stick with Hermez SMT for now (we had actually targeted a more optimized version first, but then reverted for easier compatibility during transition Hermez -> Zero prover). Given that we also need to target cdk-erigon (which uses hermez'), we probably would want to stick to this one here (though there were some discussions about tweaking the internals, see point below on smt_trie).

Note that fraktal-zkp/zeth does not list its license.

They do not have a given LICENSE.md file but mention being MIT-licensed in their README.

Our current SMT library is unsuitable for use in a client, and probably needs to be rewritten, or heavily refactored to be used in the way we'd need

Agreed on this. You mention this being medium, would you have an idea of the time it would take for a full revamping? I mentioned internal tweaks above, because Hermez and our SMT / prover backend rely on Poseidon, while we initiated discussions some time ago on possibly using Poseidon2 which is much faster, while providing roughly the same security benefits. If we wanted to do this, we'd need to decide whether this comes with dropping fully Poseidon support, or through a more modular fashion, to then support both Hermez style and the more generic/efficient cdk stack that could leverage Poseidon2.

[temaniarpit27]

@0xaatif https://github.com/0xPolygonZero/zk_evm/pull/511/files

[temaniarpit27]

@0xaatif for first point - created 2 tasks #790, #791
Will be picking them up

[temaniarpit27]

Closed #790
Native tracer might fail and no need to handle this for now