EVM parsing functions with ABI #421

pro-wh · 2023-05-19T00:04:45Z

This PR has just the basics for some code to parse Ethereum ABI format things, including tx input data, tx result data, and log data. Later PRs build upon this

pro-wh · 2023-05-19T00:05:42Z

=== RUN   TestEVMParseData
method approve
arg address _spender = 0x250d48C5E78f1E85F7AB07FEC61E93ba703aE668
arg uint256 _value = 4000000000000000000
--- PASS: TestEVMParseData (0.00s)
=== RUN   TestEVMParseEvent
event Approval
arg address indexed _owner = 0x0000000000000000000000000ecf5262e5b864e1612875f8fc18f151315b5e91
arg address indexed _spender = 0x000000000000000000000000250d48c5e78f1e85f7ab07fec61e93ba703ae668
arg uint256 _value = 4000000000000000000
--- PASS: TestEVMParseEvent (0.00s)
PASS

mitjat

Super neat! IMO this needs some JSON compatibility tests (see thread inline) and fmt.Printf replaced with asserts ... and it's ready.

I'm assuming you were planning to integrate these funcs into the analyzer in a separate PR?

api/logic/evm.go

mitjat · 2023-05-24T02:56:14Z

api/logic/evm.go

+	ethCommon "github.com/ethereum/go-ethereum/common"
+)
+
+func EVMParseData(data []byte, contractABI *abi.ABI) (*abi.Method, []interface{}, error) {


It's not clear what Go types will be used in the elements of []interface{}.

Looking at the ethereum-go tests, it's "whatever makes sense" https://github.com/ethereum/go-ethereum/blob/9231770811cda0473a7fa4e2bccc95bf62aae634/accounts/abi/abi_test.go#L810-L813 but it's not clearly documented. Going more into the weeds, it is one of the many possible return values of toGoType().

Perhaps we don't need to care much, as long as serializes cleanly into JSON (which is the format we use in the DB, and in our API).
Please test at least with large integers; ethereum-go uses *big.Int, which is reasonable, but does not support pgx-compatible serialization. We'll be converting everything into JSON manually before throwing it at pgx, so we're probably fine?
Anyway, it would be good to have some EVM parsing tests not just into Go-native types, but into JSON. If you can find/generate example inputs for wacky types (e.g. multi-dimensional array of tuples of (bigint, address)), all the better.

😬 yeah should check json

yeah it's *big.Int and it tries to put it into json as just the digits 😱

alright tests are added

pro-wh · 2023-05-24T18:54:49Z

we don't have the ABIs being collected yet, so I don't have a place to integrate this yet

mitjat · 2023-05-25T02:27:05Z

we don't have the ABIs being collected yet, so I don't have a place to integrate this yet

You can integrate with the existing ERC-20 parsing; it will make it easier to then later generalize from ERC-20 to [any contract whose ABI we have].

pro-wh · 2023-05-25T18:57:58Z

we have the ERC-20 ABI, but we don't have a piece of data telling us which contract to use which ABI for

pro-wh · 2023-05-25T21:14:14Z

struct O {
    uint16 n;
    string s;
}

interface Varied {
    function test(
        int8 i8,
        uint8 u8,
        int256 i,
        uint256 u,
        bool b,
        // not supported in go-ethereum
        // fixed128x18 f18,
        // ufixed128x18 uf18,
        bytes32 b32,
        address a,
        function (uint16) external returns (uint16) f,
        uint16[2] calldata xy,
        bytes calldata buf,
        string calldata s,
        uint16[] calldata l,
        O calldata o
    ) external;
}

0xa85acffaffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000000000000000000000000000000000000000000000000000000001ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000010101010101010101010101010101010101010101010101010101010101010101000000000000000000000000010101010101010101010101010101010101010101010101010101010101010101010101010101010202020200000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000001c00000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000024000000000000000000000000000000000000000000000000000000000000002800000000000000000000000000000000000000000000000000000000000000001010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000016100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000016100000000000000000000000000000000000000000000000000000000000000

[-1,1,-1,1,true,[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],"0x0101010101010101010101010101010101010101",[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2],[1,1],"AQ==","a",[1],{"n":1,"s":"a"}]

eth (solidity) -> go (go-ethereum) -> JSON

pro-wh · 2023-05-25T21:16:47Z

current complaints:

large integers are in json as numbers, we want strings
function is array of numbers, we probably want base64 bytes, or maybe hex
bytes32 (and similar sizes I assume) is array of numbers, we probably want base64 bytes, or maybe hex

pro-wh · 2023-05-25T23:44:28Z

[-1,1,"-1","1",true,"AQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQE=","0x0101010101010101010101010101010101010101","AQEBAQEBAQEBAQEBAQEBAQEBAQECAgIC",[1,1],"AQ==","a",[1],{"n":1,"s":"a"}]

custom converting

possibly weird philosophically: uint8[] and uint8[n] I believe will also be base64

mitjat

Thank you! Largely LGTM for the current changes.

Also, to revisit the deeper-integration conversation (stupid github for preventing PR-level threads):

I'm assuming you were planning to integrate these funcs into the analyzer in a separate PR?

we don't have the ABIs being collected yet, so I don't have a place to integrate this yet

You can integrate with the existing ERC-20 parsing; it will make it easier to then later generalize from ERC-20 to [any contract whose ABI we have].

we have the ERC-20 ABI, but we don't have a piece of data telling us which contract to use which ABI for

We do have token_type in chain.evm_tokens. Though doing that properly will require a new analyzer that will parse events and txs post-hoc, as ABIs become available, right? Created #872 to track this.

What you can do short-term though is replace the slicing here. We'll still only heuristically decide the event is an ERC20 Transfer, but then we'll parse it with the ABI. Kinda silly, but might be a worthy early use case for the new code, and a way of testing it out gradually.

api/logic/evm_test.go

pro-wh · 2023-06-22T21:19:09Z

note for later: see BoundContract.UnpackLog for how to parse the topics. filter .Inputs by their .Indexed, then use abi.ParseTopics. or probably abi.ParseTopicsIntoMap for our use case

mitjat

Thank you!
I'll leave it to you to decide how to show bytes and bytes1..bytes32 to the end users. I'd go for hex, but I can see how it's a matter of opinion. I'm quite convinced the final format is mostly the domain of this code though, not the frontend.

api/logic/evm_test.go

pro-wh · 2023-10-16T22:04:10Z

oh update: uint8[] and uint8[n] don't become hex/base64. they're proper arrays of numbers

pro-wh · 2023-10-16T22:21:43Z

ok bytes and bytesn are now like "0x1234" strings

pro-wh requested review from aefhm, Andrew7234 and mitjat as code owners May 19, 2023 00:04

mitjat reviewed May 24, 2023

View reviewed changes

pro-wh force-pushed the pro-wh/feature/abi branch 2 times, most recently from 5fd593b to 39de649 Compare May 24, 2023 23:47

pro-wh force-pushed the pro-wh/feature/abi branch 2 times, most recently from 5496614 to e90ef11 Compare May 26, 2023 22:57

mitjat approved these changes Jun 2, 2023

View reviewed changes

api/logic/evm_test.go Outdated Show resolved Hide resolved

api/logic/evm_test.go Outdated Show resolved Hide resolved

pro-wh mentioned this pull request Jun 22, 2023

ERC-721 support #447

Merged

pro-wh force-pushed the pro-wh/feature/abi branch from e90ef11 to 61205ba Compare June 29, 2023 17:55

pro-wh requested a review from ptrus as a code owner June 29, 2023 17:55

pro-wh force-pushed the pro-wh/feature/abi branch from 61205ba to 098fdc7 Compare August 8, 2023 22:41

pro-wh force-pushed the pro-wh/feature/abi branch 2 times, most recently from b58ee98 to 340b0b2 Compare September 21, 2023 21:00

pro-wh mentioned this pull request Sep 21, 2023

runtime: use abiparse in VisitEVMEvent #521

Merged

pro-wh force-pushed the pro-wh/feature/abi branch from 340b0b2 to 59d0d44 Compare September 21, 2023 21:20

mitjat approved these changes Sep 27, 2023

View reviewed changes

api/logic/evm_test.go Outdated Show resolved Hide resolved

api/logic/evm_test.go Outdated Show resolved Hide resolved

api/logic/evm_test.go Outdated Show resolved Hide resolved

pro-wh added 2 commits October 16, 2023 12:22

EVM parsing functions with ABI

d0b30c2

evm: adjust unpacked types for json

dd6a5f5

pro-wh force-pushed the pro-wh/feature/abi branch 2 times, most recently from f4550f5 to 36552bd Compare October 16, 2023 21:26

pro-wh force-pushed the pro-wh/feature/abi branch 2 times, most recently from a88b09f to b91c364 Compare October 16, 2023 22:03

pro-wh added 3 commits October 16, 2023 15:19

evm: byte arrays as hex

100858e

evm: add uint8 array tests

beac905

logic: junk up our docstring

f361f74

pro-wh force-pushed the pro-wh/feature/abi branch from b91c364 to f361f74 Compare October 16, 2023 22:21

pro-wh merged commit 9e33fae into main Oct 16, 2023

pro-wh deleted the pro-wh/feature/abi branch October 16, 2023 22:38

csillag mentioned this pull request Dec 30, 2024

backfilling analyzer that parses EVM data once the ABI becomes available #872

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EVM parsing functions with ABI #421

EVM parsing functions with ABI #421

pro-wh commented May 19, 2023 •

edited

Loading

pro-wh commented May 19, 2023

mitjat left a comment

mitjat May 24, 2023

pro-wh May 24, 2023

pro-wh May 24, 2023

pro-wh Sep 19, 2023

pro-wh commented May 24, 2023

mitjat commented May 25, 2023

pro-wh commented May 25, 2023

pro-wh commented May 25, 2023

pro-wh commented May 25, 2023

pro-wh commented May 25, 2023 •

edited

Loading

mitjat left a comment •

edited by csillag

Loading

pro-wh commented Jun 22, 2023 •

edited

Loading

mitjat left a comment

pro-wh commented Oct 16, 2023

pro-wh commented Oct 16, 2023 •

edited

Loading

EVM parsing functions with ABI #421

EVM parsing functions with ABI #421

Conversation

pro-wh commented May 19, 2023 • edited Loading

pro-wh commented May 19, 2023

mitjat left a comment

Choose a reason for hiding this comment

mitjat May 24, 2023

Choose a reason for hiding this comment

pro-wh May 24, 2023

Choose a reason for hiding this comment

pro-wh May 24, 2023

Choose a reason for hiding this comment

pro-wh Sep 19, 2023

Choose a reason for hiding this comment

pro-wh commented May 24, 2023

mitjat commented May 25, 2023

pro-wh commented May 25, 2023

pro-wh commented May 25, 2023

pro-wh commented May 25, 2023

pro-wh commented May 25, 2023 • edited Loading

mitjat left a comment • edited by csillag Loading

Choose a reason for hiding this comment

pro-wh commented Jun 22, 2023 • edited Loading

mitjat left a comment

Choose a reason for hiding this comment

pro-wh commented Oct 16, 2023

pro-wh commented Oct 16, 2023 • edited Loading

pro-wh commented May 19, 2023 •

edited

Loading

pro-wh commented May 25, 2023 •

edited

Loading

mitjat left a comment •

edited by csillag

Loading

pro-wh commented Jun 22, 2023 •

edited

Loading

pro-wh commented Oct 16, 2023 •

edited

Loading