quai_getLogs is not working

[jdowning100]

Steps to reproduce:

Find logs in the explorer, for example: https://quaiscan.io/address/0x0035d6329028c0B54E78B9a52f1305aa98ccCbCD?tab=logs&shard=undefined

Copy topic 0

Call quai_getLogs in Postman on rpc.quai.network/cyprus1:

{
"jsonrpc": "2.0",
"id": 1,
"method": "quai_getLogs",
"params": [{
"fromBlock": "0x2F1E8", // starting block number
"toBlock": "latest",
"address": "0x0035d6329028c0B54E78B9a52f1305aa98ccCbCD",
"topics": [
"0x0d3648bd0f6ba80134a33ba9275ac585d9d315f0ad8355cddefde31afa28d0e9"
]
}]
}

Change "fromBlock" to the block the transaction was included in (or several blocks before)
Expected result is the logs returned, but currently the return is empty. However this works sometimes with the QFAN token events. We are likely not indexing bloom filters correctly.

[Djadih]

always got log for immediate transfer
after some time stop getting logs, return empty list no errors
maybe reorgs? secondary vs primary bloom filter?
not reorgs, sometimes available after 10k blocks, sometimes expire after not even 1000
no pattern with prime, coincidences, no correspondence to bloom filter depths

[Djadih]

https://quaiscan.io/tx/0x00300069ad67a98b868bf4bcc1c85cb8028053b7836650141f58b9bcee274762

[Djadih]

Was able to find the following tx
https://quaiscan.io/tx/0x0051003d8ae1c0bf118451f872ff6e852f56d407ff9741f98a62fcd381c32bdb?tab=logs&shard=

With query:
{
"jsonrpc": "2.0",
"id": 1,
"method": "quai_getLogs",
"params": [{
"fromBlock": "0xfde80", // starting block number
"toBlock": "latest",
"address": "0x004A2e19E838218544eE571a900f62Cb050B39CE",
"topics": [
"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
"0x000000000000000000000000002be1285e2dd45f425ce77a1a39d5c587750dc2",
"0x0000000000000000000000000014559244bdd2552106f28c35940acfd7b58a5d"
]
}]
}

[Djadih]

1. GetLogs API call (form PublicFilterAPI) parses the given arguments (like rpc.LatestBlockNumber, finds the latest header, pulls the number, etc.) and creates a "range filter". The range filter has a "filters" variable which is a [][][]byte, basically this is an array of filters, where each filter is an array of arguments, where each argument is a byte array. So it could be [][]common.Hash for example when it comes to the addresses field.
2. The filter looks first at the indexed logs, based on the given start and stop block numbers (as if it were a continuous array). The main output is a channel of block numbers that are confirmed to have a match.
3. The indexed logs process starts a matcher, which is basically a channel with some processing around it. It runs go routines in parallel to find partial matches. Here it creates a channel of "*partialMatches" type of the same size as the above.
4. The matcher.run assembles a sub filtering pipeline which is all daisy chained based on sub matching results.
5. The matcher does the actual partial matching, and based on that does more thorough matching.
6. The first level it takes is [][]bloomIndexes, where the top level index is describing the position in the query. For example, addresses, then topics. The second level being that a query could contain multiple addresses or multiple topics. The way subMatch works is it takes all the results of the first level (eg addresses), ORs the results together. Then it ANDs the results with the next level (eg topics) to produce a final bloom filter. It short circuits if the first level has only zeros.

Notes:

• There are actually 3 hashes used to generate and retrieve the bit fields. Its achieved by doing some shifts and taking the keccak 3 times.

[kiltsonfire]

#2356