Create inkscope-fuzzer.md

[feliam]

Project Abstract

Even though smart contracts in Polkadot are not currently securing substantial value relative to other chains, it is prudent to establish robust security frameworks proactively before significant funds are put at risk.

To address these challenges, we propose developing a custom WebAssembly fuzzer for smart contracts. Fuzzing is an effective way to find security bugs in software, it finds vulnerabilities often missed by static program analysis and manual code inspection. By testing bytecode builds at the pre-deployment stage, contract vulnerabilities can be discovered and patched earlier in the cycle. Language-agnostic fuzzing will cover all smart contract languages compilable to WASM, including ink!, with extensibility for any languages added in future.

Grant level

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $30,000, 3 approvals
• Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

• The application template has been copied and aptly renamed (project_name.md).
• I have read the application guidelines.
• Payment details have been provided (bank details via email or Polkadot (USDC & USDT) address in the application).
• I am aware that, in order to receive a grant, I (and the entity I represent) have to successfully complete a KYC/KYB check.
• The software delivered for this grant will be released under an open-source license specified in the application.
• The initial PR contains only one commit (squash and force-push if needed).
• The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
• I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[feliam]

I have read and hereby sign the Contributor License Agreement.

[keeganquigley]

Thanks @feliam for your application, I have a couple of initial questions:

• Our payment options have recently changed, would you be willing receive USDC on Asset Hub instead? If so can you please update the address?
• How will you plan to maintain the project after the grant is finished? Would you go for alternative funding such as the treasury or the futures fund?
• I noticed that you have already done some work in the Octopus repo. Is this a separate project? Can you specify in the application what has already been built, and what would be built as part of the grant? Since we only fund software that doesn't exist yet.

[feliam]

Hello @keeganquigley:

Thanks @feliam for your application, I have a couple of initial questions:

• Our payment options have recently changed, would you be willing receive USDC on Asset Hub instead? If so can you please update the address?

Sure. No issues there. Here you have our address for USDC on Asset Hub: 13mMCgidf4fdevVBCENS4U3pdQKLzcFQiZEMjuGTVrTJg1tE
Should I change the PR also?

• How will you plan to maintain the project after the grant is finished? Would you go for alternative funding such as the treasury or the futures fund?

We envision the project to be useful in itself immediately after this grant is finished. We are happy to push it further with alternative fundings like the treasury or the futures funds.

• I noticed that you have already done some work in the Octopus repo. Is this a separate project? Can you specify in the application what has already been built, and what would be built as part of the grant? Since we only fund software that doesn't exist yet.

The octopus fork was a simple showcase of our work. We plan to start the fuzzer from scratch as a fresh new repo, written in rust (it is already in the works). We plan though to seek inspiration from all the good sources like polkadot-sdk and ink.

o/

[keeganquigley]

Thanks for the changes @feliam this proposal looks really good. To answer your question, yes could you please update the Polkadot address into the application as well? Thanks!

[keeganquigley]

Thanks for the updates! I'm happy to go forward with it and will also share it with the rest of the committee to take a look.

In the meantime, we also just started requiring KYC/KYB checks for all potential grantees. Could you please provide the information outlined under this link? Let me know if you have any questions or issues!

[0xLucca]

Thanks for the updates! I'm happy to go forward with it and will also share it with the rest of the committee to take a look.

In the meantime, we also just started requiring KYC/KYB checks for all potential grantees. Could you please provide the information outlined under this link? Let me know if you have any questions or issues!

Hey @keeganquigley! We are happy to complete the KYC checks as required.

When trying to complete the form, we realized there were questions asking for specifics about our company, legal structure, registration number, etc. As we are currently just a group of developers without an official business entity, we do not have this information to provide.

Please let us know if we need to formally create a legal business entity before moving forward with the grant process. If so, we can certainly do that and provide the requested details. We just had not taken that step before applying as we were focused on the development work itself.

[semuelle]

Hi @0xLucca. You don't have to have a legal entity. However, that means that every beneficiary of the grant needs to perform KYC and the name and address of whoever issues the invoices in the end should be listed under "registered address" (feel free to email that to us if you prefer not to disclose it).

[feliam]

@semuelle We have both completed the KYC.
We would prefer to email the registered address, how should we proceed with that? thnks

[semuelle]

@feliam, just send an email with the project name, PR number, your name and address to [email protected]. Thanks!

[feliam]

@semuelle Mail sent!

[0xLucca]

If there's anything else needed from our end or if you have any further questions, please don't hesitate to let us know.
@semuelle @keeganquigley

[keeganquigley]

Thanks @0xLucca I will make sure to bring this application up with committee members during our grants sync next week.

[semuelle]

Hi @feliam & @0xLucca, since I'm not too familiar with fuzzers, could you explain how testing a fuzzer works? Do write unit tests for specific issues that should be caught? In this case, would you create a library of known ink! issues that you would test against? Or how do you measure its effectiveness?

[feliam]

@semuelle

Fuzzing is a bug-finding technique where randomly generated or mutated input is provided to a program while it is monitored for unexpected behavior. When applied to normal binary computer programs, finding a bug is somewhat similar to finding inputs that make the process crash, like a memory safety error, etc.

In the context of smart contracts the goal is to discover sequences of transactions that violate predefined properties rather than causing crashes. You do not write unittests for it but you need to write properties (or invariants) for it.
The technique needs the contract writer (or auditor) to write down specific properties that should never fail. The properties are encoded as normal contract methods and can be written in the original language, here ink!. Then the fuzzer is used to find a sequence of transactions that would violate those properties.

We aim to build a functional fuzzer framework that could run most of the ink-examples contracts. An initial measurement of its effectiveness can be done via code coverage, like reaching >85% of code coverage automatically. We will provide some example contracts with injected errors and prepared properties that can be broken with the fuzzer. E.g. a psp22 with an integer overflow that could make the amount of tokens held by owners bigger than the defined total supply.

[semuelle]

Hi @feliam, my question was more pointed towards how you test your project, not how an inkscope user would test their project. We usually ask for unit or integration tests for any software project, depending on what makes most sense, and I was curious whether your unit tests cover specific contract vulnerabilities or if it's something more general. If it's the latter, it would be great to have a collection of contracts (not just an example or two) to test against. Could you add that to the test spec or as a separate deliverable?

Also, is the goal to have these changes merged into FuzzingLabs/octopus? Or are you planning to maintain your own fork?

[feliam]

@semuelle Thank you for the clarification.

To test this project, we are taking a phased approach focused first on ensuring the core functionality is working properly first before expanding to additional integration and vulnerability testing. For this initial phase, our testing methodology includes both unit tests and coverage analysis:
Unit Tests
We are writing unit tests for each reimplemented host function from the pallet-contracts in the fuzzer's emulated runtime environment. These tests ensure key components like state handling and environment interactions are working as expected.
Coverage Analysis
To validate the fuzzer is exercising contracts effectively, we will be benchmarking against the ink open source examples (https://github.com/paritytech/ink-examples). The target is 85% code coverage to ensure our modifications can fuzz real-world contracts.
Minimal Example end to end fuzz campaign
Contribute a minimal example of how to write a contract invariant to help users integrate the fuzzer into their testing.

In a follow-on phase, not included in the scope of this project, we intend to expand testing to:
Validate against false positives using the actual Substrate runtime environment
Build a corpus of smart contracts with known vulnerabilities to benchmark the fuzzer’s effectiveness at finding bugs
Contribute an extensive, well documented and reusable example set to help users integrate property based fuzzing into their contract testing

We believe this incremental approach strikes the right balance between delivering core functionality quickly while prioritizing more advanced integration over time. Please let me know if you would like any additional details or have suggestions on the test plan.

Regarding your second question, I want to clarify there is no direct relationship between this project and Octopus. We previously demonstrated modifications to Octopus to enable WASM contract analysis, but that was simply a proof of concept to try some ideas. This current project is entirely separate and it is started from scratch. It does not involve any forking or changes to the Octopus codebase itself. Apologies if that was unclear earlier - to confirm, we are developing a standalone fuzzer for wasm smart contracts without any dependence on Octopus.

[keeganquigley]

Thanks for the explanations @feliam, in addition to the Parity example contracts, you might want to check out ink! Dev Hub, they have a few examples of complex contracts such as cross-contract calls, an NFT marketplace, and a DEX. They also house the aWASoMe repo which has a ton more examples/tools/resources. ink!athon has some as well.

If it is helpful, the CoinFabrik team has also published a number of ink! contract examples with known vulnerabilities, categorized by vulnerability class.

[keeganquigley]

Hi @feliam sorry for the delay. I spoke with one of Parity's security experts and have the following additional questions:

• Will the tool depend on the concrete version of pallet-contracts? Who will maintain the fuzzer in the future?
• Does the fuzzer need the source code of a contract to define properties, or can it extract them from wasm blobs? Since typically ink! contracts are distributed without source code.
• What will be the methodologies behind handling the state of the fuzzer? For example, the snapshot-based fuzzing proposed by ityfuzz.
• What are some of the challenges/limitations you might see? For example, what do you mean by saying that the fuzzer will work with most of the ink-examples contracts?

Thanks for any insights you can provide!

[feliam]

Hello @keeganquigley,

Will the tool depend on the concrete version of pallet-contracts? Who will maintain the fuzzer in the future?

We plan to create a fuzzer execution engine that is back compatible with all previous versions of pallet-contract up until 9e6298e7f773c61d384677e6eebfa8d7eee9d2ac

The fuzzer will implement all different host function versions(seals) that can coexist across pallet-contract versions. Note that, for example, there are currently 3 different versions of the call() host function (call, call_v1, call_v2). The fuzzer will implement all versions and link calls to the appropriate underlying host function implementation based on the contract being tested.

As the primary developers, we take responsibility for maintaining the fuzzer going forward. This will involve keeping it up-to-date, fixing issues, and adding new features. We hope to fund this maintenance through Polkadot treasury proposals that benefit the community. An actively maintained, open source fuzzer can improve smart contract security for multiple chains built on Substrate.

We welcome collaboration from other developers as well. Over time, we envision this becoming a community-driven project supported by multiple contributors.

Does the fuzzer need the source code of a contract to define properties, or can it extract them from wasm blobs? Since typically ink! contracts are distributed without source code.

TL;DR Contract authors can encode properties during development for more targeted testing. Also users can still fuzz opaque WASM to a limited degree for due diligence.

While we do not like the idea of trusting an opaque wasm blob without the associated and verified source code, the fuzzer will not strictly need the target contract source code to run.

The fuzzer can operate directly on WASM contract blobs without needing access to source code. However, encoding meaningful properties to check would be challenging without an understanding of the contract's intended behavior and almost impossible without an ABI.
Ideally, the contract developer would write properties and invariants directly in the original ink! source code, which the fuzzer could test from the compiled WASM.

If you are compelled to fuzz a third party contract where source is unavailable you will need to either reverse engineer it or otherwise obtain the ABI that the contract implements. Then write a shell contract that uses it and test the inner opaque contract.

What will be the methodologies behind handling the state of the fuzzer? For example, the snapshot-based fuzzing proposed by ityfuzz.

For the first iteration, we will try to focus on correctness and keep it as simple as possible.
We will keep the current emulated state in memory and copy on write when possible. Also, we will maintain a map from the sequence of transactions into reached states. This way we can minimize the re-execution of transactions with known results. In short, if the fuzzer has already run a certain prologue of transactions it will not re-execute it in a new try that shares
the same initial sequence of transaction.

What are some of the challenges/limitations you might see? For example, what do you mean by saying that the fuzzer will work with most of the ink-examples contracts?

In this phase, we will focus on core host functionality required for general contract operation. Precise gas tracking and more exotic interfaces like chain extensions and call runtime will be deferred. This scoped approach allows faster initial development, but limits compatibility with contracts that expect full-fidelity emulation. Code paths depending on precise gas usage or runtime APIs will likely behave differently than on a live chain. We recognize this prevents fuzzing certain targets effectively. But it is a reasonable compromise for an initial working release.
By acknowledging limitations upfront and conveying concrete plans to address them incrementally, we aim for transparency without overstating initial capabilities.

Let us know if you need further clarifications
o/

[github-actions]

Congratulations and welcome to the Web3 Foundation Grants Program! Please refer to our Milestone Delivery repository for instructions on how to submit milestones and invoices, our FAQ for frequently asked questions and the support section of our README for more ways to find answers to your questions.

Before you start, take a moment to read through our announcement guidelines for all communications related to the grant or make them known to the right person in your organisation. In particular, please don't announce the grant publicly before at least the first milestone of your project has been approved. At that point or shortly before, you can get in touch with us at [email protected] and we'll be happy to collaborate on an announcement about the work you’re doing.

Lastly, please remember to let us know in case you run into any delays or deviate from the deliverables in your application. You can either leave a comment here or directly request to amend your application via PR. We wish you luck with your project! 🚀