Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] SVM: Integration of SolFuzz-Agave - v2.1.6 #32

Open
wants to merge 7 commits into
base: v2.1.6-joec
Choose a base branch
from

Conversation

buffalojoec
Copy link
Owner

NOTE: This change is intended for the new SVM repository, not the Agave monorepo.
Its draft inclusion here is only for review & collaboration purposes.

Problem

Both the Anza and Firedancer teams have been using SolFuzz-Agave
for crucial testing of validator components as well as BPF programs. It's also a
potentially very useful tool for other developers in each domain.

There are actually multiple problems being addressed by these changes - which
include only the addition of new libraries and no changes to Agave's hot
path.

First and foremost, the Firedancer team maintains SolFuzz-Agave standalone, and
whenever changes are made to Agave's execution layer, they must cherry-pick and
update the harness to be compatible with the new APIs. It would be much easier
if we maintained changes to SolFuzz-Agave alongside any API-breaking PRs to Agave.

Secondly, Anza's developer tooling team has been working on both migrating
builtins to BPF programs as well as rewrites of SPL Token, SPL Token-2022 and
more. We use SolFuzz-Agave and Mollusk
to test BPF programs in many different ways, including conformance testing
between two versions of the same program.

However, the same internal entrypoint into Agave's program runtime is
reimplemented across both tools. It would be much easier - and safer - if this
entrypoint lived in one place, and was instead used by both tools. This would
also mean that it could be used for other future tooling, which would be a huge
benefit.

https://github.com/buffalojoec/mollusk/blob/c01523016195c315870bd9e4bfa3a64a6cffa659/harness/src/lib.rs#L182-L218

The above problems are the motivation for both the integration of SolFuzz-Agave
itself, as well as the structure of the libraries included in this PR.

Including SolFuzz-Agave in Agave's SVM maintenance allows a much more stable API
for testing and fuzzing validator execution-layer components, which helps
Firedancer as well as any new validator clients who wish to benefit from the
same tooling. It also allows us to build more robust BPF program tooling, which
can help the broader ecosystem in obvious ways.

Summary of Changes

The overall change is straightforward: integrate SolFuzz-Agave into the SVM
stack by creating a new set of standalone crates that comprise a
reimplementation of the original SolFuzz-Agave.

I've refactored a lot of the original code to make it easier to maintain and
change, and I've included testing from the original repository to ensure maximum
compatibility.

NOTE: This change is only handling the instr (instruction) entrypoint. The
others (transaction, VM) will come later, and follow a similar pattern.

I've also decided to break the tooling up into multiple crates. They are
summarized as follows:

  • -fixture: The library for working with fixtures created by Firedancer's
    Protosol protobuf definitions. It
    offers conversions between prost-generated protobuf types and Agave SDK
    types.
  • -fixture-fs: An extension crate for working with fixtures (above) on the
    local filesytem (ie. load_from_blob_file).
  • -instr-entrypoint: The main entrypoint into Agave's program runtime. This
    library has nothing to do with fixtures, and can be used standalone to build
    tooling (ie. Mollusk).
  • -instr: The actual SolFuzz-Agave instruction harness itself, which depends
    on the above libraries to offer the same fuzzing and testing entrypoint as the
    original tool, including any custom setup or checks that are not imposed on
    the base entrypoint.

@buffalojoec buffalojoec force-pushed the sf-agave-integration-wip-2.1.6 branch from c1fa5c7 to 647d6b9 Compare January 22, 2025 06:02
@buffalojoec buffalojoec force-pushed the sf-agave-integration-wip-2.1.6 branch from 647d6b9 to 7ece70f Compare January 23, 2025 06:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants