Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add simple json report generation #195

Open
wants to merge 55 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
cbde4a3
simple json report generation
tuturu-tech Aug 4, 2023
095fb73
fix json template, change htmlReportPath to corpus path
tuturu-tech Aug 4, 2023
18433cc
added html and json file path configs, updated report generation
tuturu-tech Aug 4, 2023
8ec0007
fix lint error
tuturu-tech Aug 4, 2023
194a782
Update to go-ethereum 1.12.0 (#199)
Xenomega Aug 11, 2023
2dddaf1
Bump golang.org/x/net from 0.12.0 to 0.14.0 (#196)
dependabot[bot] Aug 11, 2023
ab00c59
`console.log` go brrr (#193)
anishnaik Aug 15, 2023
3172b53
Added denomination parsing to AST value extraction (#202)
Xenomega Aug 18, 2023
b911551
Update README.md (#184)
montyly Aug 22, 2023
931bba0
Multi-channel and multi-error log support (#180)
anishnaik Aug 22, 2023
aa82533
Update version (#210)
anishnaik Aug 23, 2023
24ebcc4
Support NoColor and --no-color (#222)
Exca-DK Sep 25, 2023
e370832
Update CONTRIBUTING.md (#185)
montyly Sep 25, 2023
f2c956e
Bump github.com/rs/zerolog from 1.29.0 to 1.30.0 (#186)
dependabot[bot] Sep 25, 2023
def06b2
Bump github.com/google/uuid from 1.3.0 to 1.3.1 (#215)
dependabot[bot] Sep 25, 2023
4b2dacf
Bump golang.org/x/crypto from 0.12.0 to 0.13.0 (#226)
dependabot[bot] Sep 25, 2023
b4e7547
Bump github.com/rs/zerolog from 1.30.0 to 1.31.0 (#236)
dependabot[bot] Nov 8, 2023
f4f6605
add setuptools to pip install line (#252)
anishnaik Nov 8, 2023
914a144
Bump actions/checkout from 3 to 4 (#228)
dependabot[bot] Nov 8, 2023
ae4c36e
Bump golang.org/x/crypto from 0.13.0 to 0.14.0 (#240)
dependabot[bot] Nov 8, 2023
1f8afda
Fix `testAllContracts` behavior (#253)
anishnaik Nov 8, 2023
8cc3957
Open non-zero coverage reports in corpus by default (#243)
damilolaedwards Nov 8, 2023
9ad9ff2
Fix `warp` cheatcode (#255)
anishnaik Nov 9, 2023
7065f74
Explicitly reset the state trie's cache to prevent mem leak (#290)
anishnaik Feb 13, 2024
7f0c072
Bump github.com/spf13/cobra from 1.7.0 to 1.8.0 (#257)
dependabot[bot] Feb 13, 2024
6313c92
Bump github.com/google/uuid from 1.3.1 to 1.6.0 (#288)
dependabot[bot] Feb 13, 2024
b166363
Bump golang.org/x/net from 0.14.0 to 0.21.0 (#291)
dependabot[bot] Feb 13, 2024
38ed335
Fix the copy length based panic in the parseBytes32 cheatcode (#293)
Xenomega Feb 13, 2024
6c13d2a
Enable all testing modes by default, update property mode testing, im…
anishnaik Feb 20, 2024
57cc20c
fix OOB panic (#303)
anishnaik Feb 21, 2024
6692150
Added a unique exit code for failed tests, added generic way to bubbl…
Xenomega Feb 26, 2024
3320811
Fix corpus call method resolution bug, improve startup logging (#308)
Xenomega Feb 28, 2024
c4b9d98
Add shrinking limits (#297)
Xenomega Feb 28, 2024
4a0f0c1
update version (#310)
anishnaik Feb 28, 2024
3f6a58d
Feat: snapshot and revertTo cheatcodes (#276)
damilolaedwards Mar 25, 2024
75ac86e
fix: attach execution trace to reverting properties (#335)
0xalpharush Mar 25, 2024
c3127e4
fix: use signature in traces to handle overloaded function names (#336)
0xalpharush Mar 25, 2024
e50b36d
feat: add execution trace for failed target contract deployments (#337)
0xalpharush Mar 27, 2024
7fdab25
Output non-fuzzing failure related error messages (#312)
Xenomega Mar 27, 2024
fa498d1
Update README to reflect current CLI and config options (#343)
elopez Apr 3, 2024
c755683
Bump golang.org/x/crypto from 0.19.0 to 0.22.0 (#345)
dependabot[bot] Apr 8, 2024
f46c4e1
Bump google.golang.org/protobuf from 1.28.1 to 1.33.0 (#332)
dependabot[bot] Apr 8, 2024
42a16b0
Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#314)
dependabot[bot] Apr 8, 2024
e58ae6e
fix two outdated comments (#347)
konnov Apr 10, 2024
74dbf0e
ci: automated release builds (#342)
elopez Apr 10, 2024
68c5be2
`mdbook` for medusa (#348)
anishnaik May 3, 2024
69a9979
fix: use the shrunken abi values as the msg's data (#374)
0xalpharush Jun 24, 2024
5e5d0e0
chore: fix typos (#387)
0xalpharush Jul 2, 2024
827dbed
feat: call constant methods occasionally and allow assertion testing …
0xalpharush Jul 2, 2024
8320d6e
fix: re-encode calldata in mutator and refactor (#380)
0xalpharush Jul 2, 2024
3259c27
excluding compilation platform fallsback to default (#362)
0xalpharush Jul 2, 2024
3cbe484
feat: display test cases discovered by the fuzzer (#382)
0xalpharush Jul 2, 2024
f37287f
fix json template, change htmlReportPath to corpus path
tuturu-tech Aug 4, 2023
ed6627f
added html and json file path configs, updated report generation
tuturu-tech Aug 4, 2023
4fb3645
Remove duplicate member values passed to ProjectConfig
s4nsec Jul 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 76 additions & 18 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ on:
push:
branches:
- master
tags:
- "v*"
pull_request:
branches:
- master
Expand All @@ -22,13 +24,16 @@ jobs:
needs: [lint, test]
strategy:
matrix:
environment: [ubuntu-latest, macos-latest, windows-latest]
environment: [ubuntu-latest, macos-12, macos-14, windows-latest]
permissions:
contents: read
id-token: write

runs-on: ${{ matrix.environment }}
timeout-minutes: 10

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Speed up Go (Windows)
if: runner.os == 'Windows'
Expand All @@ -44,41 +49,85 @@ jobs:
printf 'TEMP=%s\\tmpdir\n' "$DIR" | tee -a "$GITHUB_ENV"
go env

- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: "^1.18.1"
# disable caching during release (tag) builds
cache: ${{ !startsWith(github.ref, 'refs/tags/') }}

- name: Build (Linux and macOS)
if: runner.os == 'Linux' || runner.os == 'macOS'
run: go build -o medusa -v .

- name: Compress (Linux and macOS)
if: runner.os == 'Linux' || runner.os == 'macOS'
run: tar -czvf medusa.tar.gz medusa
run: tar -czvf medusa-${{ runner.os }}-${{ runner.arch }}.tar.gz medusa

- name: Build (Windows)
if: runner.os == 'Windows'
run: go build -o medusa.exe -v .

- name: Compress (Windows)
if: runner.os == 'Windows'
run: tar -czvf medusa.tar.gz medusa.exe
run: tar -czvf medusa-${{ runner.os }}-${{ runner.arch }}.tar.gz medusa.exe

- name: Upload artifact on merge to master
if: github.ref == 'refs/heads/master'
uses: actions/upload-artifact@v3
- name: Rename for release
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
shell: bash
run: |
[ ! -f medusa-Linux-X64.tar.gz ] || mv medusa-Linux-X64.tar.gz medusa-linux-x64.tar.gz
[ ! -f medusa-macOS-X64.tar.gz ] || mv medusa-macOS-X64.tar.gz medusa-mac-x64.tar.gz
[ ! -f medusa-macOS-ARM64.tar.gz ] || mv medusa-macOS-ARM64.tar.gz medusa-mac-arm64.tar.gz
[ ! -f medusa-Windows-X64.tar.gz ] || mv medusa-Windows-X64.tar.gz medusa-win-x64.tar.gz

- name: Sign artifact
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
uses: sigstore/[email protected]
with:
inputs: ./medusa-*.tar.gz

- name: Upload artifact
if: github.ref == 'refs/heads/master' || (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/'))
uses: actions/upload-artifact@v4
with:
name: medusa-${{ runner.os }}
path: medusa.tar.gz
name: medusa-${{ runner.os }}-${{ runner.arch }}
path: |
./medusa-*.tar.gz
./medusa-*.tar.gz.sigstore

release:
needs: [build]
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
permissions:
contents: write

runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Download binaries
uses: actions/download-artifact@v4
with:
pattern: medusa-*
merge-multiple: true

- name: Create GitHub release and upload binaries
uses: softprops/action-gh-release@9d7c94cfd0a1f3ed45544c887983e9fa900f0564 # v2.0.4
with:
draft: true
name: "${{ github.ref_name }}"
files: |
./medusa-*.tar.gz
./medusa-*.tar.gz.sigstore

lint:
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: "^1.18.1"

Expand Down Expand Up @@ -110,13 +159,17 @@ jobs:
test:
strategy:
matrix:
environment: [ubuntu-latest, macos-latest, windows-latest]
environment: [ubuntu-latest, macos-12, macos-14, windows-latest]

runs-on: ${{ matrix.environment }}
timeout-minutes: 20

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Speed up Go, Python, Node (Windows)
if: runner.os == 'Windows'
Expand All @@ -142,11 +195,11 @@ jobs:
npm config set cache "$DIR\\npm-cache" --global
echo "::endgroup::"

- uses: actions/setup-go@v4
- uses: actions/setup-go@v5
with:
go-version: "^1.18.1"

- uses: actions/setup-node@v3
- uses: actions/setup-node@v4
with:
node-version: 18.15

Expand All @@ -165,9 +218,14 @@ jobs:
run: go test ./...

all-checks:
needs: [lint, test, build]
if: always()
needs: [lint, test, build, release]

runs-on: ubuntu-latest

steps:
- run: true
- name: Decide whether the needed jobs succeeded or failed
uses: re-actors/alls-green@05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe # v1.2.2
with:
allowed-skips: release
jobs: ${{ toJSON(needs) }}
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,7 @@
*node_modules/

# Medusa binary
medusa
medusa

# Medusa docs
docs/book
41 changes: 41 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,47 @@ If any of these requirements are violated, you should expect your pull request t

Pull request reviewers have a responsibility to uphold these standards. Even if a pull request is compliant with these requirements, a reviewer which identifies an opportunity to document some caveat (such as a `// TODO: ` comment) should request it be added prior to pull request approval.

### Linters

Several linters and security checkers are run on the PRs.

#### Go

To install

- `go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest`

To run

- `go fmt ./...`
- `golangci-lint run --timeout 5m0s`

#### Markdown/Json/Yaml

To install

- `npm install -g prettier`
- `npm install -g [email protected]`

To run

- `prettier '**.json' '**/*.md' '**/*.yml' '!(pkg)'`
- `markdown-link-check --config .github/workflows/resources/markdown_link_check.json ./*.md`

To format (overwrite files)

- `prettier '**.json' '**/*.md' '**/*.yml' '!(pkg)' -w`

#### Github action

To install

- `go install github.com/rhysd/actionlint/cmd/actionlint@latest`

To run

- `actionlint`

### Cross-platform considerations

- Ensure file/directory names do not exceed 32 characters in length to minimize filepath length issues on Windows. File/directory names should be shorter than this where possible.
Expand Down
91 changes: 7 additions & 84 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,95 +17,18 @@ It provides parallelized fuzz testing of smart contracts through CLI, or its Go
- ✔️**Extensible low-level testing API** through events and hooks provided throughout the fuzzer, workers, and test chains.
- ❌ **Extensible high-level testing API** allowing for the addition of per-contract or global post call/event property tests with minimal effort.

## Installation
## Documentation

### Precompiled binaries
To learn more about how to install and use `medusa`, please refer to our [documentation](./docs/src/SUMMARY.md).

To use `medusa`, first ensure you have [crytic-compile](https://github.com/crytic/crytic-compile) and a suitable compilation framework (e.g. `solc`, `hardhat`) installed on your machine.
For a better viewing experience, we recommend you install [mdbook](https://rust-lang.github.io/mdBook/guide/installation.html)
and then running the following steps from medusa's source directory:

You can then fetch the latest binaries for your platform from our [GitHub Releases](https://github.com/crytic/medusa/releases) page.

### Building from source

#### Requirements

- You must have at least go 1.18 installed.
- [Windows only] The `go-ethereum` dependency may require [TDM-GCC](https://jmeubank.github.io/tdm-gcc/) to build.

#### Steps

- Clone the repository, then execute `go build` in the repository root.
- Go will automatically fetch all dependencies and build a binary for you in the same folder when completed.

## Usage

Although we recommend users run `medusa` in a configuration file driven format for more customizability, you can also run `medusa` through the CLI directly.
We provide instructions for both below.

We recommend you familiarize yourself with writing [assertion](https://github.com/crytic/building-secure-contracts/blob/master/program-analysis/echidna/basic/assertion-checking.md) and [property](https://github.com/crytic/building-secure-contracts/blob/master/program-analysis/echidna/introduction/how-to-test-a-property.md) tests for Echidna. `medusa` supports Echidna-like property testing with config-defined function prefixes (default: `fuzz_`) and assertion testing using Solidity `assert(...)` statements.

### Command-line only

You can use the following command to run `medusa` against a contract:

```console
medusa fuzz --target contract.sol --deployment-order ContractName
```

Where:

- `--target` specifies the path `crytic-compile` should use to compile contracts
- `--deployment-order` specifies comma-separated names of contracts to be deployed for testing.

**Note:** Check out the [command-line interface](https://github.com/crytic/medusa/wiki/Command-Line-Interface) wiki page, or run `medusa --help` for more information.

### Configuration file driven

The preferred method to use medusa is to enter your project directory (hardhat directory, or directory with your contracts),
then execute the following command:

```console
medusa init
```

This will create a `medusa.json` in your current folder. There are two required fields that should be set correctly:

- Set your `"target"` under `"compilation"` to point to the file/directory which `crytic-compile` should use to build your contracts.
- Put the names of any contracts you wish to deploy and run tests against in the `"deploymentOrder"` field. This must be non-empty.

After you have a configuration in place, you can execute:

```console
medusa fuzz
```bash
cd docs
mdbook serve
```

This will use the `medusa.json` configuration in the current directory and begin the fuzzing campaign.

**Note:** Check out the [project configuration](https://github.com/crytic/medusa/wiki/Project-Configuration) wiki page, or run `medusa --help` for more information.

## Running Unit Tests

First, install [crytic-compile](https://github.com/crytic/crytic-compile), [solc-select](https://github.com/crytic/solc-select), and ensure you have `solc` (version >=0.8.7), and `hardhat` available on your system.

- From the root of the repository, invoke `go test -v ./...` on through command-line to run tests from all packages at or below the root.
- Or enter each package directory to run `go test -v .` to test the immediate package.
- Note: the `-v` parameter provides verbose output.
- Otherwise, use an IDE like [GoLand](https://www.jetbrains.com/go/) to visualize the tests and logically separate output.

## FAQs

**Why create `medusa` if Echidna is already working just fine?**

With `medusa`, we are exploring a different EVM implementation and language for our smart contract fuzzer. We believe that
experimenting with a new fuzzer provides us with the following benefits:

- Since `medusa` is written in Go, we believe that this will **lower the barrier of entry for external contributions**.
We have taken great care in thoroughly commenting our code so that it is easy for new contributors to get up-to-speed and start contributing!
- The use of Go allows us to build an API to hook into the various parts of the fuzzer to build custom testing methodologies. See the [API Overview (WIP)](<https://github.com/crytic/medusa/wiki/API-Overview-(WIP)>) section in the Wiki for more details.
- Our forked version of go-ethereum, [`medusa-geth`](https://github.com/crytic/medusa-geth), exhibits behavior that is closer to that of the EVM in production environments.
- We can take the lessons we learned while developing Echidna to create a fuzzer that is just as feature-rich but with additional capabilities to
create powerful and unique testing methodologies.

## Contributing

For information about how to contribute to this project, check out the [CONTRIBUTING](./CONTRIBUTING.md) guidelines.
Expand Down
27 changes: 25 additions & 2 deletions chain/cheat_code_contract.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,30 @@ type cheatCodeRawReturnData struct {
Err error
}

// getCheatCodeProviders obtains a cheatCodeTracer (used to power cheat code analysis) and associated CheatCodeContract
// objects linked to the tracer (providing on-chain callable methods as an entry point). These objects are attached to
// the TestChain to enable cheat code functionality.
// Returns the tracer and associated pre-compile contracts, or an error, if one occurred.
func getCheatCodeProviders() (*cheatCodeTracer, []*CheatCodeContract, error) {
// Create a cheat code tracer and attach it to the chain.
tracer := newCheatCodeTracer()

// Obtain our standard cheat code pre-compile
stdCheatCodeContract, err := getStandardCheatCodeContract(tracer)
if err != nil {
return nil, nil, err
}

// Obtain the console.log pre-compile
consoleCheatCodeContract, err := getConsoleLogCheatCodeContract(tracer)
if err != nil {
return nil, nil, err
}

// Return the tracer and precompiles
return tracer, []*CheatCodeContract{stdCheatCodeContract, consoleCheatCodeContract}, nil
}

// newCheatCodeContract returns a new precompiledContract which uses the attached cheatCodeTracer for execution
// context.
func newCheatCodeContract(tracer *cheatCodeTracer, address common.Address, name string) *CheatCodeContract {
Expand Down Expand Up @@ -98,7 +122,7 @@ func (c *CheatCodeContract) Abi() *abi.ABI {
}

// addMethod adds a new method to the precompiled contract.
// Returns an error if one occurred.
// Throws a panic if either the name is the empty string or the handler is nil.
func (c *CheatCodeContract) addMethod(name string, inputs abi.Arguments, outputs abi.Arguments, handler cheatCodeMethodHandler) {
// Verify a method name was provided
if name == "" {
Expand All @@ -117,7 +141,6 @@ func (c *CheatCodeContract) addMethod(name string, inputs abi.Arguments, outputs
method: method,
handler: handler,
}

// Add the method to the ABI.
// Note: Normally the key here should be the method name, not sig. But cheat code contracts have duplicate
// method names with different parameter types, so we use this so they don't override.
Expand Down
Loading