From 018f143d0ea9a691aa663f48a9d61232e1b76f6a Mon Sep 17 00:00:00 2001 From: tjayrush Date: Sun, 3 Sep 2023 02:11:56 -0400 Subject: [PATCH 01/33] Removes unused files --- docs/content/readmes/accounts-abis.md | 62 ---------- docs/content/readmes/accounts-export.md | 99 ---------------- docs/content/readmes/accounts-list.md | 56 --------- docs/content/readmes/accounts-monitors.md | 57 --------- docs/content/readmes/accounts-names.md | 57 --------- docs/content/readmes/admin-chunks.md | 64 ----------- docs/content/readmes/admin-config.md | 37 ------ docs/content/readmes/admin-daemon.md | 50 -------- docs/content/readmes/admin-init.md | 55 --------- docs/content/readmes/admin-scrape.md | 108 ------------------ docs/content/readmes/admin-status.md | 38 ------ docs/content/readmes/chaindata-blocks.md | 71 ------------ docs/content/readmes/chaindata-logs.md | 45 -------- docs/content/readmes/chaindata-receipts.md | 47 -------- docs/content/readmes/chaindata-traces.md | 53 --------- .../content/readmes/chaindata-transactions.md | 62 ---------- docs/content/readmes/chaindata-when.md | 57 --------- docs/content/readmes/chainstate-state.md | 56 --------- docs/content/readmes/chainstate-tokens.md | 58 ---------- docs/content/readmes/other-explore.md | 32 ------ docs/content/readmes/other-slurp.md | 52 --------- 21 files changed, 1216 deletions(-) delete mode 100644 docs/content/readmes/accounts-abis.md delete mode 100644 docs/content/readmes/accounts-export.md delete mode 100644 docs/content/readmes/accounts-list.md delete mode 100644 docs/content/readmes/accounts-monitors.md delete mode 100644 docs/content/readmes/accounts-names.md delete mode 100644 docs/content/readmes/admin-chunks.md delete mode 100644 docs/content/readmes/admin-config.md delete mode 100644 docs/content/readmes/admin-daemon.md delete mode 100644 docs/content/readmes/admin-init.md delete mode 100644 docs/content/readmes/admin-scrape.md delete mode 100644 docs/content/readmes/admin-status.md delete mode 100644 docs/content/readmes/chaindata-blocks.md delete mode 100644 docs/content/readmes/chaindata-logs.md delete mode 100644 docs/content/readmes/chaindata-receipts.md delete mode 100644 docs/content/readmes/chaindata-traces.md delete mode 100644 docs/content/readmes/chaindata-transactions.md delete mode 100644 docs/content/readmes/chaindata-when.md delete mode 100644 docs/content/readmes/chainstate-state.md delete mode 100644 docs/content/readmes/chainstate-tokens.md delete mode 100644 docs/content/readmes/other-explore.md delete mode 100644 docs/content/readmes/other-slurp.md diff --git a/docs/content/readmes/accounts-abis.md b/docs/content/readmes/accounts-abis.md deleted file mode 100644 index 938e78e0c0..0000000000 --- a/docs/content/readmes/accounts-abis.md +++ /dev/null @@ -1,62 +0,0 @@ -## chifra abis - - -`chifra abis` retrieves ABI files for the given address(es). It searches for the ABI in this order: -the current local folder, the TrueBlocks cache, Etherscan, or (in the -future) ENS and Sourcify. - -While this tool may be used from the command line and the API, its primary purpose is in support of -tools such as `chifra export` to support the `--articulate` option. - -The `--known` option prints a list of semi-standard function signatures such as the ERC20 standard, -ERC 721 standard, various functions from OpenZeppelin, various Uniswap functions, etc. As an -optimization, the `known` signatures are searched first during articulation. - -The `--sol` option converts the provided Solidity file into an ABI json file. The results are -dropped into the current working folder. - -The `--find` option is experimental. Please see the notes below for more information. - -```[plaintext] -Purpose: - Fetches the ABI for a smart contract. - -Usage: - chifra abis [flags]
[address...] - -Arguments: - addrs - a list of one or more smart contracts whose ABIs to display (required) - -Flags: - -k, --known load common 'known' ABIs from cache - -f, --find strings search for function or event declarations given a four- or 32-byte code(s) - -n, --hint strings for the --find option only, provide hints to speed up the search - -e, --encode string generate the 32-byte encoding for a given cannonical function or event signature - -C, --clean remove an abi file for an address or all zero-length files if no address is given - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - Search for either four byte signatures or event signatures with the --find option. -``` - -Data models produced by this tool: - -- [abi](/data-model/accounts/#abi) -- [function](/data-model/other/#function) -- [parameter](/data-model/other/#parameter) - -Links: - -- [api docs](/api/#operation/accounts-abis) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/abis) - - -### notes - -The `chifra abis --find` option scans the cross product of two sets. The first set contains more than 100,000 function and event -names. The second set contains approximately 700 function signatures. The cross product of these two sets creates 70,000,000 -combinations of name(signature) each of which is hashed to create either a four-byte or a 32-byte hash. Very infrequently, -the tool will find matches for an otherwise unknown signatures. - diff --git a/docs/content/readmes/accounts-export.md b/docs/content/readmes/accounts-export.md deleted file mode 100644 index 224cb0fa1e..0000000000 --- a/docs/content/readmes/accounts-export.md +++ /dev/null @@ -1,99 +0,0 @@ -## chifra export - - -The `chifra export` tools provides a major part of the functionality of the TrueBlocks system. Using -the index of appearances created with `chifra scrape` and the list of transaction identifiers -created with `chifra list`, `chifra export` completes the actual extraction of an address's transactional -history from the node. - -You may use `topics`, `fourbyte` values at the start of a transaction's input data, and/or a log's -`source address` or `emitter` to filter the results. - -You may also choose which portions of the Ethereum data structures (`--transactions`, `--logs`, -`--traces`, etc.) as you wish. - -By default, the results of the extraction are delivered to your console, however, you may export -the results to any database (with a little bit of work). The format of the data, its content and -its destination are up to you. - -```[plaintext] -Purpose: - Export full detail of transactions for one or more addresses. - -Usage: - chifra export [flags]
[address...] [topics...] [fourbytes...] - -Arguments: - addrs - one or more addresses (0x...) to export (required) - topics - filter by one or more log topics (only for --logs option) - fourbytes - filter by one or more fourbytes (only for transactions and trace options) - -Flags: - -p, --appearances export a list of appearances - -r, --receipts export receipts instead of transactional data - -l, --logs export logs instead of transactional data - -t, --traces export traces instead of transactional data - -n, --neighbors export the neighbors of the given address - -C, --accounting attach accounting records to the exported data (applies to transactions export only) - -A, --statements for the accounting options only, export only statements - -b, --balances traverse the transaction history and show each change in ETH balances - -a, --articulate articulate transactions, traces, logs, and outputs - -R, --cache_traces force the transaction's traces into the cache - -U, --count only available for --appearances mode, if present, return only the number of records - -c, --first_record uint the first record to process - -e, --max_records uint the maximum number of records to process (default 250) - -N, --relevant for log and accounting export only, export only logs relevant to one of the given export addresses - -m, --emitter strings for log export only, export only logs if emitted by one of these address(es) - -B, --topic strings for log export only, export only logs with this topic(s) - -P, --asset strings for the accounting options only, export statements only for this asset - -f, --flow string for the accounting options only, export statements with incoming, outgoing, or zero value - One of [ in | out | zero ] - -y, --factory for --traces only, report addresses created by (or self-destructed by) the given address(es) - -u, --unripe export transactions labeled upripe (i.e. less than 28 blocks old) - -E, --reversed produce results in reverse chronological order - -z, --no_zero for the --count option only, suppress the display of zero appearance accounts - -F, --first_block uint first block to process (inclusive) - -L, --last_block uint last block to process (inclusive) - -H, --ether specify value in ether - -o, --cache force the results of the query into the cache - -D, --decache removes related items from the cache - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - Articulating the export means turn the EVM's byte data into human-readable text (if possible). - - For the --logs option, you may optionally specify one or more --emitter, one or more --topics, or both. - - The --logs option is significantly faster if you provide an --emitter or a --topic. - - Neighbors include every address that appears in any transaction in which the export address also appears. - - If present, --first_/--last_block are applied, followed by user-supplied filters such as asset or topic, followed by --first_/--max_record if present. - - The --first_record and --max_record options are zero-based (as are the block options). - - The _block and _record filters are ignored when used with the --count option. - - If the --reversed option is present, the appearance list is reversed prior to all processing (including filtering). - - The --decache option will remove all cache items (blocks, transactions, traces, etc.) for the given address(es). -``` - -Data models produced by this tool: - -- [appearance](/data-model/accounts/#appearance) -- [reconciliation](/data-model/accounts/#reconciliation) -- [monitor](/data-model/accounts/#monitor) -- [appearancecount](/data-model/accounts/#appearancecount) -- [statement](/data-model/accounts/#statement) -- [transaction](/data-model/chaindata/#transaction) -- [transfer](/data-model/chaindata/#transfer) -- [receipt](/data-model/chaindata/#receipt) -- [log](/data-model/chaindata/#log) -- [trace](/data-model/chaindata/#trace) -- [traceaction](/data-model/chaindata/#traceaction) -- [traceresult](/data-model/chaindata/#traceresult) -- [token](/data-model/chainstate/#token) -- [function](/data-model/other/#function) -- [parameter](/data-model/other/#parameter) - -Links: - -- [api docs](/api/#operation/accounts-export) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/export) - diff --git a/docs/content/readmes/accounts-list.md b/docs/content/readmes/accounts-list.md deleted file mode 100644 index 784230c074..0000000000 --- a/docs/content/readmes/accounts-list.md +++ /dev/null @@ -1,56 +0,0 @@ -## chifra list - - -`chifra list` takes one or more addresses, queries the index of appearances, and builds TrueBlocks -monitors. A TrueBlocks monitor is a file that contains blockNumber.transactionId pairs (transaction -identifiers) representing the history of the address. - -Because TrueBlocks only extracts data from the Ethereum node when it's requested, the first time -you list an address it takes about a minute. Subsequent queries are much faster because TrueBlocks -caches the results. - -Note that `chifra list` only queries the index, it does not extract the full transactional details. -You may use `chifra export` for that. - -```[plaintext] -Purpose: - List every appearance of an address anywhere on the chain. - -Usage: - chifra list [flags]
[address...] - -Arguments: - addrs - one or more addresses (0x...) to list (required) - -Flags: - -U, --count display only the count of records for each monitor - -z, --no_zero for the --count option only, suppress the display of zero appearance accounts - -b, --bounds report first and last block this address appears - -u, --unripe list transactions labeled upripe (i.e. less than 28 blocks old) - -s, --silent freshen the monitor only (no reporting) - -c, --first_record uint the first record to process - -e, --max_records uint the maximum number of records to process (default 250) - -E, --reversed produce results in reverse chronological order - -F, --first_block uint first block to export (inclusive, ignored when freshening) - -L, --last_block uint last block to export (inclusive, ignored when freshening) - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - No other options are permitted when --silent is selected. -``` - -Data models produced by this tool: - -- [appearance](/data-model/accounts/#appearance) -- [monitor](/data-model/accounts/#monitor) -- [appearancecount](/data-model/accounts/#appearancecount) -- [bounds](/data-model/accounts/#bounds) - -Links: - -- [api docs](/api/#operation/accounts-list) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/list) - diff --git a/docs/content/readmes/accounts-monitors.md b/docs/content/readmes/accounts-monitors.md deleted file mode 100644 index c803265e2a..0000000000 --- a/docs/content/readmes/accounts-monitors.md +++ /dev/null @@ -1,57 +0,0 @@ -## chifra monitors - - -`chifra monitors`` has two purposes: (1) to `--watch` a set of addresses. This function is in its early -stages and will be better explained elsewhere. Please see an example of what one may do with -`chifra monitors --watch`, and (2) allows one to manage existing monitored addresses. - -A "monitor" is simply a file on a hard drive that represents the transactional history of a given -Ethereum address. Monitors are very small, being only the `` pair representing each -appearance of an address. Monitor files are only created when a user expresses interest in a -particular address. In this way, TrueBlock is able to continue to work on small desktop or even -laptop computers. (See `chifra list`.) - -You may use the `--delete` command to delete (or `--undelete` if already deleted) an address. The -monitor is not removed from your computer if you delete it. It is just marked as being deleted -making it invisible to the TrueBlocks explorer. - -Use the `--remove` command to permanently remove a monitor from your computer. This is an -irreversible operation and requires the monitor to have been previously deleted. - -```[plaintext] -Purpose: - Add, remove, clean, and list address monitors. - -Usage: - chifra monitors [flags]
[address...] - -Arguments: - addrs - one or more addresses (0x...) to process - -Flags: - --delete delete a monitor, but do not remove it - --undelete undelete a previously deleted monitor - --remove remove a previously deleted monitor - -C, --clean clean (i.e. remove duplicate appearances) from monitors - -l, --list list monitors in the cache (--verbose for more detail) - -w, --watch continually scan for new blocks and extract data for monitored addresses - -s, --sleep float seconds to sleep between monitor passes (default 14) - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all monitors will be cleaned. -``` - -Data models produced by this tool: - -- [monitor](/data-model/accounts/#monitor) -- [monitorclean](/data-model/admin/#monitorclean) - -Links: - -- [api docs](/api/#operation/accounts-monitors) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/monitors) - diff --git a/docs/content/readmes/accounts-names.md b/docs/content/readmes/accounts-names.md deleted file mode 100644 index 80fb2d7bd4..0000000000 --- a/docs/content/readmes/accounts-names.md +++ /dev/null @@ -1,57 +0,0 @@ -## chifra names - - -`chifra names` is a surprisingly useful tool. It allows one to associate textual names with Ethereum -addresses. One may ask why this is necessary given that ENS exists. The answer is a single -word: "privacy". ENS names are public. In many cases, users desire to keep personal addresses -private. Try to do this on a website. - -Like `chifra abis`, this tool is useful from the command line but is primarily used in support of -other tools, especially `chifra export` where naming addresses becomes the single best way to -turn unintelligible blockchain data into understandable information. - -The various options allow you to search and filter the results. The `tags` option is used primarily -by the TrueBlocks explorer. - -You may use the TrueBlocks explorer to manage (add, edit, delete) address-name associations. - -```[plaintext] -Purpose: - Query addresses or names of well known accounts. - -Usage: - chifra names [flags] [term...] - -Arguments: - terms - a space separated list of one or more search terms (required) - -Flags: - -e, --expand expand search to include all fields (search name, address, and symbol otherwise) - -m, --match_case do case-sensitive search - -a, --all include all (including custom) names in the search - -c, --custom include only custom named accounts in the search - -p, --prefund include prefund accounts in the search - -s, --addr display only addresses in the results (useful for scripting, assumes --no_header) - -g, --tags export the list of tags and subtags only - -C, --clean clean the data (addrs to lower case, sort by addr) - -r, --regular only available with --clean, cleans regular names database - -d, --dry_run only available with --clean or --autoname, outputs changes to stdout instead of updating databases - -A, --autoname string an address assumed to be a token, added automatically to names database if true - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - The tool will accept up to three terms, each of which must match against any field in the database. - - The --match_case option enables case sensitive matching. -``` - -Data models produced by this tool: - -- [name](/data-model/accounts/#name) - -Links: - -- [api docs](/api/#operation/accounts-names) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/names) - diff --git a/docs/content/readmes/admin-chunks.md b/docs/content/readmes/admin-chunks.md deleted file mode 100644 index c778bff2e7..0000000000 --- a/docs/content/readmes/admin-chunks.md +++ /dev/null @@ -1,64 +0,0 @@ -## chifra chunks - - -The chifra chunks routine provides tools for interacting with, checking the validity of, cleaning up, -and analyizing the Unchained Index. It provides options to list pins, the Manifest, summary data -on the index, Bloom filters, addresses, and appearances. While still in its early stages, this -tool will eventually allow users to clean their local index, clean their remote index, study -the indexes, etc. Stay tuned. - -```[plaintext] -Purpose: - Manage, investigate, and display the Unchained Index. - -Usage: - chifra chunks [flags] [blocks...] [address...] - -Arguments: - mode - the type of data to process (required) - One of [ manifest | index | blooms | addresses | appearances | stats ] - blocks - an optional list of blocks to intersect with chunk ranges - -Flags: - -c, --check check the manifest, index, or blooms for internal consistency - -i, --pin pin the manifest or each index chunk and bloom - -p, --publish publish the manifest to the Unchained Index smart contract - -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract - -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk - -F, --first_block uint first block to process (inclusive) - -L, --last_block uint last block to process (inclusive) - -m, --max_addrs uint the max number of addresses to process in a given chunk - -d, --deep if true, dig more deeply during checking (manifest only) - -s, --sleep float for --remote pinning only, seconds to sleep between API calls - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - Mode determines which type of data to display or process. - - Certain options are only available in certain modes. - - If blocks are provided, only chunks intersecting with those blocks are displayed. - - The --truncate option updates the manifest and removes local data, but does not alter remote pins. - - The --belongs option is only available in the index mode. - - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - - The --pin option requires a locally running IPFS node or a pinning service API key. - - The --publish option requires a private key. -``` - -Data models produced by this tool: - -- [appearance](/data-model/accounts/#appearance) -- [manifest](/data-model/admin/#manifest) -- [chunkrecord](/data-model/admin/#chunkrecord) -- [chunkindex](/data-model/admin/#chunkindex) -- [chunkbloom](/data-model/admin/#chunkbloom) -- [chunkaddress](/data-model/admin/#chunkaddress) -- [chunkstats](/data-model/admin/#chunkstats) -- [reportcheck](/data-model/admin/#reportcheck) -- [chunkpinreport](/data-model/admin/#chunkpinreport) - -Links: - -- [api docs](/api/#operation/admin-chunks) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/chunks) - diff --git a/docs/content/readmes/admin-config.md b/docs/content/readmes/admin-config.md deleted file mode 100644 index 570bfe3152..0000000000 --- a/docs/content/readmes/admin-config.md +++ /dev/null @@ -1,37 +0,0 @@ -## chifra config - - -The `chifra config` program allows you to manage the various TrueBlocks caches. You may list all of the -caches, some of the cache, or even individual caches either in terse or full detail. The cache of -interest is specified with the `modes` option. - -TrueBlocks maintains caches for the index of address appearances, named addresses, abi files, as -well as other data including blockchain data, and address monitors. - -```[plaintext] -Purpose: - Report on and edit the configuration of the TrueBlocks system. - -Usage: - chifra config [flags] - -Arguments: - mode - either show or edit the configuration - One of [ show | edit ] - -Flags: - -a, --paths show the configuration paths for the system - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen -``` - -Data models produced by this tool: - -- [chain](/data-model/admin/#chain) - -Links: - -- [api docs](/api/#operation/admin-config) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/config) - diff --git a/docs/content/readmes/admin-daemon.md b/docs/content/readmes/admin-daemon.md deleted file mode 100644 index 1cf7439aec..0000000000 --- a/docs/content/readmes/admin-daemon.md +++ /dev/null @@ -1,50 +0,0 @@ -## chifra daemon - - -`chifra daemon` manages chifra's long-running processes include its JSON API server. Each of the -`chifra` commands along with all of its options, are provided not only by the command line, but -also the API server. We call this process the `flame` server, which is written in Go. - -In the future, this daemon may also manage other long-running processes. - -Another way to get help to run `chifra --help` or `chifra --help` on your command line. -See below for an example of converting command line options to a call to the API. There's a -one-to-one correspondence between the command line tools and options and the API routes and -their options. - -```[plaintext] -Purpose: - Initalize and control long-running processes such as the API and the scrapers. - -Usage: - chifra daemon [flags] - -Aliases: - daemon, serve - -Flags: - -p, --port string specify the server's port (default ":8080") - -a, --api string instruct the node to start the API server - One of [ off | on ] (default "on") - -s, --scrape string start the scraper, initialize it with either just blooms or entire index, generate for new blocks - One of [ off | blooms | index ] - -m, --monitor instruct the node to start the monitors tool - -g, --grpc run gRPC server to serve names - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - To start API open terminal window and run chifra daemon. - - See the API documentation (https://trueblocks.io/api) for more information. -``` - -Data models produced by this tool: - -- none - -Links: - -- no api for this command -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/daemon) - diff --git a/docs/content/readmes/admin-init.md b/docs/content/readmes/admin-init.md deleted file mode 100644 index 242991aa2a..0000000000 --- a/docs/content/readmes/admin-init.md +++ /dev/null @@ -1,55 +0,0 @@ -## chifra init - - -When invoked, `chifra init` reads a value from a smart contract called **The Unchained Index** -(0x0c316b7042b419d07d343f2f4f5bd54ff731183d). - -This value (`manifestHashMap`) is an IPFS hash pointing to a pinned file (called the Manifest) that -contains a large collection of other IPFS hashes. These other hashes point to each of the Bloom -filter and Index Chunk. TrueBlocks periodically publishes the Manifest's hash to the smart contract. -This makes the index available for our software to use and impossible for us to withhold. Both of -these aspects of the manifest are by design. - -If you stop `chifra init` before it finishes, it will pick up again where it left off the next -time you run it. - -Certain parts of the system (`chifra list` and `chifra export` for example) if you have not -previously run `chifra init` or `chifra scrape`. You will be warned by the system until it's -satified. - -If you run `chifra init` and allow it to complete, the next time you run `chifra scrape`, it will -start where `init` finished. This means that only the blooms will be stored on your hard drive. -Subsequent scraping will produce both chunks and blooms, although you can, if you wish delete -chunks that are not being used. You may periodically run `chifra init` if you prefer not to scrape. - -```[plaintext] -Purpose: - Initialize the TrueBlocks system by downloading the Unchained Index from IPFS. - -Usage: - chifra init [flags] - -Flags: - -a, --all in addition to Bloom filters, download full index chunks (recommended) - -d, --dry_run display the results of the download without actually downloading - -F, --first_block uint do not download any chunks earlier than this block - -s, --sleep float seconds to sleep between downloads - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - If run with no options, this tool will download or freshen only the Bloom filters. - - The --first_block option will fall back to the start of the containing chunk. - - You may re-run the tool as often as you wish. It will repair or freshen the index. -``` - -Data models produced by this tool: - -- [manifest](/data-model/admin/#manifest) -- [chunkrecord](/data-model/admin/#chunkrecord) - -Links: - -- [api docs](/api/#operation/admin-init) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/init) - diff --git a/docs/content/readmes/admin-scrape.md b/docs/content/readmes/admin-scrape.md deleted file mode 100644 index f6f21662f0..0000000000 --- a/docs/content/readmes/admin-scrape.md +++ /dev/null @@ -1,108 +0,0 @@ -## chifra scrape - - -The `chifra scrape` application creates TrueBlocks' chunked index of address appearances -- the -fundamental data structure of the entire system. It also, optionally, pins each chunk of the index -to IPFS. - -`chifra scrape` is a long running process, therefore we advise you run it as a service or in terminal -multiplexer such as `tmux`. You may start and stop `chifra scrape` as needed, but doing so means the -scraper will not be keeping up with the front of the blockchain. The next time it starts, it will -have to catch up to the chain, a process that may take several hours depending on how long ago it -was last run. See the section below and the "Papers" section of our website for more information -on how the scraping process works and prerequisites for it proper operation. - -You may adjust the speed of the index creation with the `--sleep` and `--block_cnt` options. On -some machines, or when running against some EVM node software, the scraper may overburden the -hardware. Slowing things down will ensure proper operation. Finally, you may optionally `--pin` -each new chunk to IPFS which naturally shards the database among all users. By default, pinning -is against a locally running IPFS node, but the `--remote` option allows pinning to an IPFS -pinning service such as Pinata or Estuary. - -```[plaintext] -Purpose: - Scan the chain and update the TrueBlocks index of appearances. - -Usage: - chifra scrape [flags] - -Flags: - -n, --block_cnt uint maximum number of blocks to process per pass (default 2000) - -i, --pin pin new chunks (requires locally-running IPFS daemon or --remote) - -r, --remote pin new chunks to the gateway (requires pinning service keys) - -s, --sleep float seconds to sleep between scraper passes (default 14) - -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) - -v, --verbose enable verbose output - -h, --help display this help screen -``` - -Data models produced by this tool: - -- [manifest](/data-model/admin/#manifest) -- [chunkrecord](/data-model/admin/#chunkrecord) - -Links: - -- [api docs](/api/#operation/admin-scrape) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/scrape) - -### configuration - -Each of the following additional configurable command line options are available. - -**Configuration file:** `$CONFIG/$CHAIN/blockScrape.toml` -**Configuration group:** `[settings]` - -| Item | Type | Default | Description / Default | -| ------------------ | ------------ | ------------ | --------- | -| apps_per_chunk | uint64 | 200000 | the number of appearances to build into a chunk before consolidating it | -| snap_to_grid | uint64 | 100000 | an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index | -| first_snap | uint64 | 0 | the first block at which snap_to_grid is enabled | -| unripe_dist | uint64 | 28 | the distance (in blocks) from the front of the chain under which (inclusive) a block is considered unripe | -| channel_count | uint64 | 20 | number of concurrent processing channels | -| allow_missing | bool | true | do not report errors for blockchains that contain blocks with zero addresses | - - -These items may be set in three ways, each overridding the preceeding method: - --- in the above configuration file under the `[settings]` group, --- in the environment by exporting the configuration item as UPPER_CASE, without underbars, and prepended with TB_SETTINGS_, or --- on the command line using the configuration item with leading dashes (i.e., `--name`). - - -### further information - -Each time `chifra scrape` runs, it begins at the last block it completed processing (plus one). With -each pass, the scraper descends as deeply as is possible into each block's data. (This is why -TrueBlocks requires a `--tracing` node.) As the scraper encounters appearances of address in the -block's data, it adds those appearance to a growing index. Periodically (after processing the the -block that contains the 2,000,000th appearance), the system consolidates an **index chunk**. - -An **index chunk** is a portion of the index containing approximately 2,000,000 records (although, -this number is adjustable for different chains). As part of the consolidation, the scraper creates -a Bloom filter representing the set membership in the associated index portion. The Bloom filters -are an order of magnitude smaller than the index chunks. The system then pushes both the index -chunk and the Bloom filter to IPFS. In this way, TrueBlocks creates an immutable, uncapturable -index of appearances that can be used not only by TrueBlocks, but any member of the community who -needs it. (Hint: We all need it.) - -Users of the [TrueBlocks Explorer](https://github.com/TrueBlocks/trueblocks-explorer) (or any other -software) may subsequently download the Bloom filters, query them to determine which **index -chunks** need to be downloaded, and thereby build a historical list of transactions for a given -address. This is accomplished while imposing a minimum amount of resource requirement on the end -user's machine. - -Recently, we enabled the ability for the end user to pin these downloaded index chunks and blooms -on their own machines. The user needs the data for the software to operate--sharing requires -minimal effort and makes the data available to other people. Everyone is better off. A -naturally-occuring network effect. - -### prerequisites - -`chifra scrape` works with any EVM-based blockchain, but does not currently work without a "tracing, -archive" RPC endpoint. The Erigon blockchain node, given its minimal disc footprint for an archive -node and its support of the required `trace_` endpoint routines, is highly recommended. - -Please [see this article](https://trueblocks.io/blog/a-long-winded-explanation-of-trueblocks/) for -more information about running the scraper and building and sharing the index of appearances. - diff --git a/docs/content/readmes/admin-status.md b/docs/content/readmes/admin-status.md deleted file mode 100644 index 5cda5687b9..0000000000 --- a/docs/content/readmes/admin-status.md +++ /dev/null @@ -1,38 +0,0 @@ -## chifra status - - -```[plaintext] -Purpose: - Report on the state of the internal binary caches. - -Usage: - chifra status [mode...] [flags] - -Arguments: - modes - the (optional) name of the binary cache to report on, terse otherwise - One or more of [ index | blooms | blocks | transactions | traces | logs | statements | results | state | tokens | monitors | names | abis | slurps | staging | unripe | maps | some | all ] - -Flags: - -c, --first_record uint the first record to process - -e, --max_records uint the maximum number of records to process (default 10000) - -a, --chains include a list of chain configurations in the output - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - The some mode includes index, monitors, names, slurps, and abis. - - If no mode is supplied, a terse report is generated. -``` - -Data models produced by this tool: - -- [status](/data-model/admin/#status) -- [cacheitem](/data-model/admin/#cacheitem) -- [chain](/data-model/admin/#chain) - -Links: - -- [api docs](/api/#operation/admin-status) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/status) - diff --git a/docs/content/readmes/chaindata-blocks.md b/docs/content/readmes/chaindata-blocks.md deleted file mode 100644 index 97ac26c208..0000000000 --- a/docs/content/readmes/chaindata-blocks.md +++ /dev/null @@ -1,71 +0,0 @@ -## chifra blocks - - -The `chifra blocks` tool retrieves block data from your Ethereum node or, if previously cached, from the -TrueBlocks cache. You may specify multiple blocks per invocation. - -By default, `chifra blocks` queries the full transactional details of the block (including receipts). -You may optionally retrieve only the transaction hashes in the block (which is significantly faster). -Additionally, you may also use this tool to retrieve uncle blocks at a give height. - -Another useful feature of `chifra blocks` is the ability to extract address appearances from a block. -TrueBlocks uses a similar feature internally to build its index of appearances. This type of data -is very insightful when studying end user behavior and chain-wide adoption analysis. - -```[plaintext] -Purpose: - Retrieve one or more blocks from the chain or local cache. - -Usage: - chifra blocks [flags] [block...] - -Arguments: - blocks - a space-separated list of one or more block identifiers (required) - -Flags: - -e, --hashes display only transaction hashes, default is to display full transaction detail - -c, --uncles display uncle blocks (if any) instead of the requested block - -t, --traces export the traces from the block as opposed to the block data - -u, --uniq display a list of uniq address appearances per transaction - -f, --flow string for the --uniq option only, export only from or to (including trace from or to) - One of [ from | to | reward ] - -l, --logs display only the logs found in the block(s) - -m, --emitter strings for the --logs option only, filter logs to show only those logs emitted by the given address(es) - -B, --topic strings for the --logs option only, filter logs to show only those with this topic(s) - -a, --articulate for the --logs option only, articulate the retrieved data if ABIs can be found - -r, --big_range uint for the --logs option only, allow for block ranges larger than 500 (default 500) - -U, --count display the number of the lists of appearances for --addrs or --uniq - -w, --raw report JSON data from the source with minimal processing - -o, --cache force the results of the query into the cache - -D, --decache removes related items from the cache - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - Blocks is a space-separated list of values, a start-end range, a special, or any combination. - - Blocks may be specified as either numbers or hashes. - - Special blocks are detailed under chifra when --list. - - With the --logs option, optionally specify one or more --emitter, one or more --topics, either or both. - - The --logs option is significantly faster if you provide an --emitter and/or a --topic. - - Multiple topics match on topic0, topic1, and so on, not on different topic0's. - - For the --logs option, large block ranges may crash the node, use --big_range to specify a larger range. - - The --decache option removes the block(s), all transactions in those block(s), and all traces in those transactions from the cache. -``` - -Data models produced by this tool: - -- [appearance](/data-model/accounts/#appearance) -- [block](/data-model/chaindata/#block) -- [log](/data-model/chaindata/#log) -- [logfilter](/data-model/chaindata/#logfilter) -- [trace](/data-model/chaindata/#trace) -- [traceaction](/data-model/chaindata/#traceaction) -- [traceresult](/data-model/chaindata/#traceresult) -- [blockcount](/data-model/chaindata/#blockcount) - -Links: - -- [api docs](/api/#operation/chaindata-blocks) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/blocks) - diff --git a/docs/content/readmes/chaindata-logs.md b/docs/content/readmes/chaindata-logs.md deleted file mode 100644 index 29a87087b9..0000000000 --- a/docs/content/readmes/chaindata-logs.md +++ /dev/null @@ -1,45 +0,0 @@ -## chifra logs - - -`chifra logs` returns the given transaction's logs. You may specify multiple transaction identifiers -per invocation. - -The `--articulate` option fetches the ABI from each encountered smart contract to better describe -the reported data. The `--topic` and `--source` options allow you to filter your results. - -```[plaintext] -Purpose: - Retrieve logs for the given transaction(s). - -Usage: - chifra logs [flags] [tx_id...] - -Arguments: - transactions - a space-separated list of one or more transaction identifiers (required) - -Flags: - -a, --articulate articulate the retrieved data if ABIs can be found - -w, --raw report JSON data from the source with minimal processing - -o, --cache force the results of the query into the cache - -D, --decache removes related items from the cache - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - The transactions list may be one or more transaction hashes, blockNumber.transactionID pairs, or a blockHash.transactionID pairs. - - This tool checks for valid input syntax, but does not check that the transaction requested actually exists. - - If the queried node does not store historical state, the results for most older transactions are undefined. - - If you specify a 32-byte hash, it will be assumed to be a transaction hash, if it is not, the hash will be used as a topic. -``` - -Data models produced by this tool: - -- [log](/data-model/chaindata/#log) -- [logfilter](/data-model/chaindata/#logfilter) - -Links: - -- [api docs](/api/#operation/chaindata-logs) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/logs) - diff --git a/docs/content/readmes/chaindata-receipts.md b/docs/content/readmes/chaindata-receipts.md deleted file mode 100644 index 3e53c315e2..0000000000 --- a/docs/content/readmes/chaindata-receipts.md +++ /dev/null @@ -1,47 +0,0 @@ -## chifra receipts - - -`chifra receipts` returns the given transaction's receipt. You may specify multiple transaction identifiers -per invocation. - -The `--articulate` option fetches the ABI from each encountered smart contract (including those -encountered in a trace--if the `--trace` option is enabled) to better describe the reported data. - -Generally speaking, this tool is less useful than others as you may report the same data using -`chifra transactions` and more focused data using `chifra logs`. It is included here for -completeness, as the `receipt` is a fundamental data structure in Ethereum. - -```[plaintext] -Purpose: - Retrieve receipts for the given transaction(s). - -Usage: - chifra receipts [flags] [tx_id...] - -Arguments: - transactions - a space-separated list of one or more transaction identifiers (required) - -Flags: - -a, --articulate articulate the retrieved data if ABIs can be found - -w, --raw report JSON data from the source with minimal processing - -o, --cache force the results of the query into the cache - -D, --decache removes related items from the cache - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - The transactions list may be one or more transaction hashes, blockNumber.transactionID pairs, or a blockHash.transactionID pairs. - - This tool checks for valid input syntax, but does not check that the transaction requested actually exists. - - If the queried node does not store historical state, the results for most older transactions are undefined. -``` - -Data models produced by this tool: - -- [receipt](/data-model/chaindata/#receipt) - -Links: - -- [api docs](/api/#operation/chaindata-receipts) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/receipts) - diff --git a/docs/content/readmes/chaindata-traces.md b/docs/content/readmes/chaindata-traces.md deleted file mode 100644 index 11da772825..0000000000 --- a/docs/content/readmes/chaindata-traces.md +++ /dev/null @@ -1,53 +0,0 @@ -## chifra traces - - -The `chifra traces` tool retrieves a transaction's traces. You may specify multiple transaction -identifiers per invocation. - -The `--articulate` option fetches the ABI from each encountered smart contract to better describe -the reported data. - -The `--filter` option calls your node's `trace_filter` routine (if available) using a bang-separated -string of the same values used by `trace_fitler`. - -```[plaintext] -Purpose: - Retrieve traces for the given transaction(s). - -Usage: - chifra traces [flags] [tx_id...] - -Arguments: - transactions - a space-separated list of one or more transaction identifiers (required) - -Flags: - -a, --articulate articulate the retrieved data if ABIs can be found - -f, --filter string call the node's trace_filter routine with bang-separated filter - -U, --count show the number of traces for the transaction only (fast) - -w, --raw report JSON data from the source with minimal processing - -o, --cache force the results of the query into the cache - -D, --decache removes related items from the cache - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - The transactions list may be one or more transaction hashes, blockNumber.transactionID pairs, or a blockHash.transactionID pairs. - - This tool checks for valid input syntax, but does not check that the transaction requested actually exists. - - If the queried node does not store historical state, the results for most older transactions are undefined. - - A bang separated filter has the following fields (at least one of which is required) and is separated with a bang (!): fromBlk, toBlk, fromAddr, toAddr, after, count. -``` - -Data models produced by this tool: - -- [trace](/data-model/chaindata/#trace) -- [traceaction](/data-model/chaindata/#traceaction) -- [traceresult](/data-model/chaindata/#traceresult) -- [tracecount](/data-model/chaindata/#tracecount) -- [tracefilter](/data-model/chaindata/#tracefilter) - -Links: - -- [api docs](/api/#operation/chaindata-traces) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/traces) - diff --git a/docs/content/readmes/chaindata-transactions.md b/docs/content/readmes/chaindata-transactions.md deleted file mode 100644 index edfa5c5d14..0000000000 --- a/docs/content/readmes/chaindata-transactions.md +++ /dev/null @@ -1,62 +0,0 @@ -## chifra transactions - - -The `chifra transactions` tool retrieves transactions directly from the Ethereum node (using the `--raw` -option) or from the TrueBlocks cache (if present). You may specify multiple transaction identifiers -per invocation. Unlike the Ethereum RPC, the reported transactions include the transaction's receipt -and generated logs. - -The `--articulate` option fetches the ABI from each encountered smart contract (including those -encountered in a trace--if the `--trace` option is enabled) to better describe the reported data. - -The `--trace` option attaches an array transaction traces to the output (if the node you're querying -has --tracing enabled), while the `--uniq` option displays a list of uniq address appearances -instead of the underlying data (including uniq addresses in traces if enabled). - -```[plaintext] -Purpose: - Retrieve one or more transactions from the chain or local cache. - -Usage: - chifra transactions [flags] [tx_id...] - -Arguments: - transactions - a space-separated list of one or more transaction identifiers (required) - -Flags: - -a, --articulate articulate the retrieved data if ABIs can be found - -t, --traces include the transaction's traces in the results - -u, --uniq display a list of uniq addresses found in the transaction - -f, --flow string for the uniq option only, export only from or to (including trace from or to) - One of [ from | to ] - -l, --logs display only the logs found in the transaction(s) - -m, --emitter strings for the --logs option only, filter logs to show only those logs emitted by the given address(es) - -B, --topic strings for the --logs option only, filter logs to show only those with this topic(s) - -A, --account_for string reconcile the transaction as per the provided address - -H, --ether specify value in ether - -w, --raw report JSON data from the source with minimal processing - -o, --cache force the results of the query into the cache - -D, --decache removes related items from the cache - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - The transactions list may be one or more transaction hashes, blockNumber.transactionID pairs, or a blockHash.transactionID pairs. - - This tool checks for valid input syntax, but does not check that the transaction requested actually exists. - - If the queried node does not store historical state, the results for most older transactions are undefined. - - The --traces option, when used with --account_for, will descend into traces to complete reconciliations. - - The --decache option removes the all transaction(s) and all traces in those transactions from the cache. -``` - -Data models produced by this tool: - -- [statement](/data-model/accounts/#statement) -- [transaction](/data-model/chaindata/#transaction) -- [transfer](/data-model/chaindata/#transfer) - -Links: - -- [api docs](/api/#operation/chaindata-transactions) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/transactions) - diff --git a/docs/content/readmes/chaindata-when.md b/docs/content/readmes/chaindata-when.md deleted file mode 100644 index 1cf11c1781..0000000000 --- a/docs/content/readmes/chaindata-when.md +++ /dev/null @@ -1,57 +0,0 @@ -## chifra when - - -The `chifra when` tool answers one of two questions: (1) "At what date and time did a given block -occur?" or (2) "What block occurred at or before a given date and time?" - -In the first case, supply a block number or hash and the date and time of that block are displayed. -In the later case, supply a date (and optionally a time) and the block number that occurred at or -just prior to that date is displayed. - -The values for `date` and `time` are specified in JSON format. `hour`/`minute`/`second` are -optional, and if omitted, default to zero in each case. Block numbers may be specified as either -integers or hexadecimal number or block hashes. You may specify any number of dates and/or blocks -per invocation. - -```[plaintext] -Purpose: - Find block(s) based on date, blockNum, timestamp, or 'special'. - -Usage: - chifra when [flags] < block | date > [ block... | date... ] - -Arguments: - blocks - one or more dates, block numbers, hashes, or special named blocks (see notes) - -Flags: - -l, --list export a list of the 'special' blocks - -t, --timestamps display or process timestamps - -U, --count with --timestamps only, returns the number of timestamps in the cache - -r, --repair with --timestamps only, repairs block(s) in the block range by re-querying from the chain - -c, --check with --timestamps only, checks the validity of the timestamp data - -u, --update with --timestamps only, bring the timestamp database forward to the latest block - -d, --deep with --timestamps --check only, verifies timestamps from on chain (slow) - -o, --cache force the results of the query into the cache - -D, --decache removes related items from the cache - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - The block list may contain any combination of number, hash, date, special named blocks. - - Block numbers, timestamps, or dates in the future are estimated with 13 second blocks. - - Dates must be formatted in JSON format: YYYY-MM-DD[THH[:MM[:SS]]]. -``` - -Data models produced by this tool: - -- [block](/data-model/chaindata/#block) -- [namedblock](/data-model/chaindata/#namedblock) -- [timestamp](/data-model/chaindata/#timestamp) -- [timestampcount](/data-model/chaindata/#timestampcount) - -Links: - -- [api docs](/api/#operation/chaindata-when) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/when) - diff --git a/docs/content/readmes/chainstate-state.md b/docs/content/readmes/chainstate-state.md deleted file mode 100644 index d5dea0082f..0000000000 --- a/docs/content/readmes/chainstate-state.md +++ /dev/null @@ -1,56 +0,0 @@ -## chifra state - - -The `chifra state` tool retrieves the balance of an address (or list of addresses) at the given block -(or blocks). Specify multiple addresses and/or multiple blocks if you wish, but you must specify -at least one address. If no block is specified, the latest block is reported. - -You may also query to see if an address is a smart contract as well as retrieve a contract's -byte code. - -```[plaintext] -Purpose: - Retrieve account balance(s) for one or more addresses at given block(s). - -Usage: - chifra state [flags]
[address...] [block...] - -Arguments: - addrs - one or more addresses (0x...) from which to retrieve balances (required) - blocks - an optional list of one or more blocks at which to report balances, defaults to 'latest' - -Flags: - -p, --parts strings control which state to export - One or more of [ none | some | all | balance | nonce | code | proxy | deployed | accttype ] - -c, --changes only report a balance when it changes from one block to the next - -z, --no_zero suppress the display of zero balance accounts - -l, --call string call a smart contract with a solidity syntax, a four-byte and parameters, or encoded call data - -a, --articulate for the --call option only, articulate the retrieved data if ABIs can be found - -r, --proxy_for string for the --call option only, redirects calls to this implementation - -H, --ether specify value in ether - -o, --cache force the results of the query into the cache - -D, --decache removes related items from the cache - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - Blocks is a space-separated list of values, a start-end range, a special, or any combination. - - If the queried node does not store historical state, the results are undefined. - - Special blocks are detailed under chifra when --list. - - Balance is the default mode. To select a single mode use none first, followed by that mode. - - Valid parameters for --call include Solidity-like syntax: balanceOf(0x316b...183d), a four-byte followed by parameters: 0x70a08231(0x316b...183d), or encoded input data. - - You may specify multiple modes on a single line. -``` - -Data models produced by this tool: - -- [callresult](/data-model/chainstate/#callresult) -- [ethcall](/data-model/chainstate/#ethcall) - -Links: - -- [api docs](/api/#operation/chainstate-state) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/state) - diff --git a/docs/content/readmes/chainstate-tokens.md b/docs/content/readmes/chainstate-tokens.md deleted file mode 100644 index 13089d9cd2..0000000000 --- a/docs/content/readmes/chainstate-tokens.md +++ /dev/null @@ -1,58 +0,0 @@ -## chifra tokens - - -Given the address of an ERC20 token contract, the `chifra tokens` tool reports token balances for one or -more additional addresses. Alternatively, the tool can report the token balances for multiple ERC20 -tokens for a single addresses. - -In normal operation the **first item** in the `address_list` is assumed to be an ERC20 token -contract whose balances are being queried, whereas the remainder of the list is assumed to be -addresses on which to report. - -In `--byAcct` mode, **all addresses** in the `address_list` are assumed to be ERC20 token contracts, -except the final one which is the account whose token balances are reported. - -You may optionally specify one or more blocks at which to report. If no block is specified, the -latest block is assumed. You may also optionally specify which parts of the token data to extract. - -```[plaintext] -Purpose: - Retrieve token balance(s) for one or more addresses at given block(s). - -Usage: - chifra tokens [flags]
[address...] [block...] - -Arguments: - addrs - two or more addresses (0x...), the first is an ERC20 token, balances for the rest are reported (required) - blocks - an optional list of one or more blocks at which to report balances, defaults to 'latest' - -Flags: - -p, --parts strings which parts of the token information to retrieve - One or more of [ name | symbol | decimals | totalSupply | version | all ] - -b, --by_acct consider each address an ERC20 token except the last, whose balance is reported for each token - -c, --changes only report a balance when it changes from one block to the next - -z, --no_zero suppress the display of zero balance accounts - -o, --cache force the results of the query into the cache - -D, --decache removes related items from the cache - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - Blocks is a space-separated list of values, a start-end range, a special, or any combination. - - If the token contract(s) from which you request balances are not ERC20 compliant, the results are undefined. - - If the queried node does not store historical state, the results are undefined. - - Special blocks are detailed under chifra when --list. - - If the --parts option is not empty, all addresses are considered tokens and each token's attributes are presented. -``` - -Data models produced by this tool: - -- [token](/data-model/chainstate/#token) - -Links: - -- [api docs](/api/#operation/chainstate-tokens) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/tokens) - diff --git a/docs/content/readmes/other-explore.md b/docs/content/readmes/other-explore.md deleted file mode 100644 index 79e738394b..0000000000 --- a/docs/content/readmes/other-explore.md +++ /dev/null @@ -1,32 +0,0 @@ -## chifra explore - - -`chifra explore` opens Etherscan (and other explorers -- including our own) to the block identifier, -transaction identifier, or address you specify. It's a handy (configurable) way to open an explorer -from the command line, nothing more. - -```[plaintext] -Purpose: - Open a local or remote explorer for one or more addresses, blocks, or transactions. - -Usage: - chifra explore [flags] [term...] - -Arguments: - terms - one or more address, name, block, or transaction identifier - -Flags: - -l, --local open the local TrueBlocks explorer - -g, --google search google excluding popular blockchain explorers - -h, --help display this help screen -``` - -Data models produced by this tool: - -- none - -Links: - -- no api for this command -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/explore) - diff --git a/docs/content/readmes/other-slurp.md b/docs/content/readmes/other-slurp.md deleted file mode 100644 index 93b9e0a27b..0000000000 --- a/docs/content/readmes/other-slurp.md +++ /dev/null @@ -1,52 +0,0 @@ -## chifra slurp - - -`chifra slurp` is the first tool we built in the Ethereum space. It even has its [own website](http://ethslurp.com). - -While it's useful, it has two shortcomings. First, it is fully centralized, pulling its data from -[http://etherscan.io](http://etherscan.io). Second, is that it does not report every transaction -for a given account. This is actually a shortcoming with Etherscan. It's too complicated to explain -here, but see our blog. - -While `chifra slurp` has its shortcomings, it does provides some nice features. You may use it to pull -any transaction initiated by an EOA for example or to explore mining rewards. Visit the above -referenced website for more information. - -```[plaintext] -Purpose: - Fetch data from Etherscan for any address. - -Usage: - chifra slurp [flags]
[address...] [block...] - -Arguments: - addrs - one or more addresses to slurp from Etherscan (required) - blocks - an optional range of blocks to slurp - -Flags: - -t, --types strings which types of transactions to request - One or more of [ ext | int | token | nfts | 1155 | miner | uncles | all ] - -p, --appearances show only the blocknumber.tx_id appearances of the exported transactions - -P, --per_page uint the number of records to request on each page (default 5000) - -s, --sleep float seconds to sleep between requests (default 0.25) - -w, --raw report JSON data from the source with minimal processing - -o, --cache force the results of the query into the cache - -D, --decache removes related items from the cache - -x, --fmt string export format, one of [none|json*|txt|csv] - -v, --verbose enable verbose output - -h, --help display this help screen - -Notes: - - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - Portions of this software are Powered by Etherscan.io APIs. -``` - -Data models produced by this tool: - -- [slurp](/data-model/other/#slurp) - -Links: - -- [api docs](/api/#operation/other-slurp) -- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/slurp) - From 8f17b9da5434b1c6c07182ff34f8beb92121a22f Mon Sep 17 00:00:00 2001 From: tjayrush Date: Sun, 3 Sep 2023 10:39:12 -0400 Subject: [PATCH 02/33] Adds hidden --run_count option to the two scrapers (scrape, monitor --watch) --- CHANGES.md | 16 ++++ docs/content/api/openapi.yaml | 9 --- docs/content/chifra/accounts.md | 3 +- docs/readmes/accounts-monitors.md | 3 +- sdk/python/src/_monitors.py | 1 - sdk/typescript/src/paths/monitors.ts | 1 - src/apps/chifra/cmd/monitors.go | 7 +- src/apps/chifra/cmd/scrape.go | 2 + .../chifra/internal/chunks/handle_check.go | 4 +- src/apps/chifra/internal/monitors/README.md | 3 +- .../chifra/internal/monitors/handle_watch.go | 8 +- src/apps/chifra/internal/monitors/options.go | 9 ++- src/apps/chifra/internal/monitors/validate.go | 8 +- .../chifra/internal/scrape/handle_scrape.go | 12 ++- src/apps/chifra/internal/scrape/options.go | 5 ++ ...handle_scrape_blaze.go => scrape_batch.go} | 4 +- ...{handle_blaze_blaze.go => scrape_blaze.go} | 31 +++---- .../internal/scrape/scrape_consolidate.go | 10 +-- .../chifra/internal/scrape/scrape_manager.go | 4 +- .../chifra/internal/scrape/scrape_prepare.go | 8 +- src/apps/chifra/internal/scrape/validate.go | 4 + .../chifra/pkg/config/scrapeCfg/scrapeCfg.go | 5 -- src/apps/chifra/pkg/types/types_namedblock.go | 2 +- src/cmd-line-options.csv | 5 +- src/dev_tools/makeClass/results/report.csv | 6 +- .../testRunner/testCases/apps/acctExport.csv | 4 +- .../acctExport/acctExport_caps_allowed_m.txt | 4 +- .../acctExport_caps_disallowed_1_m.txt | 4 +- .../acctExport_caps_disallowed_2_m.txt | 4 +- .../acctExport_caps_disallowed_3_m.txt | 4 +- .../acctExport_caps_disallowed_4_m.txt | 4 +- .../acctExport_monitors_decache.txt | 4 +- .../acctExport_monitors_decache_again.txt | 4 +- .../acctExport_monitors_delete_1_fail.txt | 4 +- .../acctExport_monitors_err_batch_size2.txt | 6 +- .../acctExport_monitors_err_not_both1.txt | 4 +- .../acctExport_monitors_err_not_both2.txt | 4 +- .../acctExport_monitors_err_run_once.txt | 10 +-- .../acctExport_monitors_err_sleep.txt | 4 +- .../acctExport_monitors_err_watch.txt | 4 +- .../acctExport/acctExport_monitors_help.txt | 4 +- .../acctExport_monitors_long_help.txt | 4 +- .../acctExport_monitors_no_params.txt | 4 +- .../acctExport_monitors_remove_fail.txt | 4 +- .../acctExport_monitors_undelete2.txt | 4 +- .../acctExport_monitors_undelete_fail.txt | 4 +- .../acctExport/acctExport_monitors_watch.txt | 4 +- .../acctExport_monitors_err_batch_size2.txt | 2 +- .../acctExport_monitors_err_run_once.txt | 4 +- .../api_tests/acctExport_monitors_watch.txt | 2 +- .../blockScrape/blockScrape_by_file_bad.txt | 1 + .../blockScrape/blockScrape_caps_allowed.txt | 1 + .../blockScrape_caps_disallowed_1.txt | 1 + .../blockScrape_caps_disallowed_10.txt | 1 + .../blockScrape_caps_disallowed_3.txt | 1 + .../blockScrape_caps_disallowed_4.txt | 1 + .../blockScrape_caps_disallowed_5.txt | 1 + .../blockScrape_caps_disallowed_6.txt | 1 + .../blockScrape_caps_disallowed_7.txt | 1 + .../blockScrape_caps_disallowed_8.txt | 1 + .../blockScrape_caps_disallowed_9.txt | 1 + .../apps/blockScrape/blockScrape_help.txt | 1 + .../blockScrape/blockScrape_help_long.txt | 1 + .../apps/blockScrape/blockScrape_invalid.txt | 1 + .../apps/blockScrape/blockScrape_listpins.txt | 1 + .../blockScrape/blockScrape_mode_01_fail.txt | 1 + .../blockScrape_nAProcs_invalid.txt | 1 + .../blockScrape_nBProcs_invalid.txt | 1 + .../apps/blockScrape/blockScrape_nBlocks.txt | 1 + .../blockScrape_nBlocks_invalid.txt | 1 + .../blockScrape/blockScrape_real_run2.txt | 1 + test/gold/apps/chifra/chifra_help_rm.txt | 4 +- test/gold/apps/chifra/chifra_help_scrape.txt | 1 + .../whenBlock_combined_options_1.txt | 4 +- .../whenBlock_long_verbose_valid_block.txt | 4 +- .../whenBlock/whenBlock_mixed_options.txt | 80 +++++++++---------- .../whenBlock_special_block_daofork.txt | 4 +- .../whenBlock_special_block_daofund.txt | 4 +- .../whenBlock_special_block_daohack.txt | 4 +- .../whenBlock_special_block_first.txt | 4 +- .../whenBlock_special_block_multi1.txt | 8 +- .../whenBlock_special_block_stateclear.txt | 4 +- .../whenBlock_special_block_tangerine.txt | 4 +- 83 files changed, 229 insertions(+), 191 deletions(-) rename src/apps/chifra/internal/scrape/{handle_scrape_blaze.go => scrape_batch.go} (95%) rename src/apps/chifra/internal/scrape/{handle_blaze_blaze.go => scrape_blaze.go} (80%) diff --git a/CHANGES.md b/CHANGES.md index 261eb4ec37..77f55c1698 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -4,6 +4,22 @@ This file details changes made to TrueBlocks over time. See the [migration notes](./MIGRATIONS.md) for any required actions you must take to stay up to date. +## v1.1.0 (2023/09/05) + +Added `--watch_list`, `--commands`, `--batch_size`, and `--run_count` options to `chifra monitors`. +Added `--run_count` to `chifra scrape` (for debugging purposes). +Changed names of some rarely used special blocks in `chifra when`. Breaking, but minimal impact. +Removes a number of previously deprecated options. `chifra abis --sol`, `chifra names --named`, `chifra names --to_custom`, `chifra transactions --trace`, and `chifra --log_level` for all commands. +Better error reporting when running against non-tracing nodes. +Fixed an issue with Content-Type in the server. +Fixed an issue where user could hit cntl+c during caching and corrupt the database. +Fixed an issue where scraper missed some smart contract addresses created during out of gas transactions. +Better error handling from the RPC. +Added a verbose mode to `chifra when` to include more specials and a description for each special block. +Now disallows running `chifra scrape` if the node is not a tracing archive node. +Near complete re-write of block scraper to fix few bugs and prepare for writing v1.0.0 of the spec. +Add run_count to both the scraper and the monitor --watch code, hides both because they are intended for debugging only. + ## v1.0.0 (2023/08/20) **OUR FIRST OFFICIAL RELEASE!** diff --git a/docs/content/api/openapi.yaml b/docs/content/api/openapi.yaml index 62a5aa98a4..957d4a27d8 100644 --- a/docs/content/api/openapi.yaml +++ b/docs/content/api/openapi.yaml @@ -622,15 +622,6 @@ paths: schema: type: number format: uint64 - - name: runOnce - description: > - available with --watch option only, only run the monitor --watch commands once then quit - required: false - style: form - in: query - explode: true - schema: - type: boolean - name: sleep description: > available with --watch option only, the number of seconds to sleep between runs diff --git a/docs/content/chifra/accounts.md b/docs/content/chifra/accounts.md index f2bbabf65c..a1b3b53a55 100644 --- a/docs/content/chifra/accounts.md +++ b/docs/content/chifra/accounts.md @@ -243,7 +243,6 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -251,7 +250,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/docs/readmes/accounts-monitors.md b/docs/readmes/accounts-monitors.md index 07916c4da4..bbb29ce36c 100644 --- a/docs/readmes/accounts-monitors.md +++ b/docs/readmes/accounts-monitors.md @@ -63,7 +63,6 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -71,7 +70,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/sdk/python/src/_monitors.py b/sdk/python/src/_monitors.py index 018c9217bb..a6af4267a3 100644 --- a/sdk/python/src/_monitors.py +++ b/sdk/python/src/_monitors.py @@ -16,7 +16,6 @@ "watchlist": {"hotkey": "-a", "type": "flag"}, "commands": {"hotkey": "-c", "type": "flag"}, "batchSize": {"hotkey": "-b", "type": "flag"}, - "runOnce": {"hotkey": "-r", "type": "switch"}, "sleep": {"hotkey": "-s", "type": "flag"}, "fmt": {"hotkey": "-x", "type": "flag"}, "verbose:": {"hotkey": "-v", "type": "switch"}, diff --git a/sdk/typescript/src/paths/monitors.ts b/sdk/typescript/src/paths/monitors.ts index 39d370c427..4d77d22ac3 100644 --- a/sdk/typescript/src/paths/monitors.ts +++ b/sdk/typescript/src/paths/monitors.ts @@ -18,7 +18,6 @@ export function getMonitors( watchlist?: string, commands?: string, batchSize?: uint64, - runOnce?: boolean, sleep?: double, chain: string, noHeader?: boolean, diff --git a/src/apps/chifra/cmd/monitors.go b/src/apps/chifra/cmd/monitors.go index 436417548b..592bb4b789 100644 --- a/src/apps/chifra/cmd/monitors.go +++ b/src/apps/chifra/cmd/monitors.go @@ -49,7 +49,7 @@ const longMonitors = `Purpose: const notesMonitors = ` Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).` @@ -70,8 +70,11 @@ func init() { monitorsCmd.Flags().StringVarP(&monitorsPkg.GetOptions().Watchlist, "watchlist", "a", "", "available with --watch option only, a file containing the addresses to watch") monitorsCmd.Flags().StringVarP(&monitorsPkg.GetOptions().Commands, "commands", "c", "", "available with --watch option only, the file containing the list of commands to apply to each watched address") monitorsCmd.Flags().Uint64VarP(&monitorsPkg.GetOptions().BatchSize, "batch_size", "b", 8, "available with --watch option only, the number of monitors to process in each batch") - monitorsCmd.Flags().BoolVarP(&monitorsPkg.GetOptions().RunOnce, "run_once", "r", false, "available with --watch option only, only run the monitor --watch commands once then quit") + monitorsCmd.Flags().Uint64VarP(&monitorsPkg.GetOptions().RunCount, "run_count", "u", 0, "available with --watch option only, run the monitor this many times, then quit (hidden)") monitorsCmd.Flags().Float64VarP(&monitorsPkg.GetOptions().Sleep, "sleep", "s", 14, "available with --watch option only, the number of seconds to sleep between runs") + if os.Getenv("TEST_MODE") != "true" { + monitorsCmd.Flags().MarkHidden("run_count") + } globals.InitGlobals(monitorsCmd, &monitorsPkg.GetOptions().Globals, capabilities) monitorsCmd.SetUsageTemplate(UsageWithNotes(notesMonitors)) diff --git a/src/apps/chifra/cmd/scrape.go b/src/apps/chifra/cmd/scrape.go index 204becc965..7b3e727f29 100644 --- a/src/apps/chifra/cmd/scrape.go +++ b/src/apps/chifra/cmd/scrape.go @@ -62,6 +62,7 @@ func init() { scrapeCmd.Flags().BoolVarP(&scrapePkg.GetOptions().Remote, "remote", "r", false, "pin new chunks to the gateway (requires pinning service keys)") scrapeCmd.Flags().Float64VarP(&scrapePkg.GetOptions().Sleep, "sleep", "s", 14, "seconds to sleep between scraper passes") scrapeCmd.Flags().Uint64VarP(&scrapePkg.GetOptions().StartBlock, "start_block", "l", 0, "first block to visit when scraping (snapped back to most recent snap_to_grid mark)") + scrapeCmd.Flags().Uint64VarP(&scrapePkg.GetOptions().RunCount, "run_count", "u", 0, "run the scraper this many times, then quit (hidden)") scrapeCmd.Flags().Uint64VarP(&scrapePkg.GetOptions().Settings.Apps_per_chunk, "apps_per_chunk", "", 200000, "the number of appearances to build into a chunk before consolidating it (hidden)") scrapeCmd.Flags().Uint64VarP(&scrapePkg.GetOptions().Settings.Snap_to_grid, "snap_to_grid", "", 100000, "an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden)") scrapeCmd.Flags().Uint64VarP(&scrapePkg.GetOptions().Settings.First_snap, "first_snap", "", 0, "the first block at which snap_to_grid is enabled (hidden)") @@ -69,6 +70,7 @@ func init() { scrapeCmd.Flags().Uint64VarP(&scrapePkg.GetOptions().Settings.Channel_count, "channel_count", "", 20, "number of concurrent processing channels (hidden)") scrapeCmd.Flags().BoolVarP(&scrapePkg.GetOptions().Settings.Allow_missing, "allow_missing", "", false, "do not report errors for blockchains that contain blocks with zero addresses (hidden)") if os.Getenv("TEST_MODE") != "true" { + scrapeCmd.Flags().MarkHidden("run_count") scrapeCmd.Flags().MarkHidden("apps_per_chunk") scrapeCmd.Flags().MarkHidden("snap_to_grid") scrapeCmd.Flags().MarkHidden("first_snap") diff --git a/src/apps/chifra/internal/chunks/handle_check.go b/src/apps/chifra/internal/chunks/handle_check.go index a3a9aec530..9f4c9f233e 100644 --- a/src/apps/chifra/internal/chunks/handle_check.go +++ b/src/apps/chifra/internal/chunks/handle_check.go @@ -111,7 +111,9 @@ func (opts *ChunksOptions) HandleCheck(blockNums []uint64) error { reports := []simpleReportCheck{} - allowMissing := scrapeCfg.AllowMissing(chain) + s, _ := scrapeCfg.GetSettings(chain, "blockScrape.toml", nil) + allowMissing := s.Allow_missing + seq := simpleReportCheck{Reason: "Filenames sequential"} if err := opts.CheckSequential(fileNames, cacheArray, remoteArray, allowMissing, &seq); err != nil { return err diff --git a/src/apps/chifra/internal/monitors/README.md b/src/apps/chifra/internal/monitors/README.md index 5378165537..ee59c5f1c9 100644 --- a/src/apps/chifra/internal/monitors/README.md +++ b/src/apps/chifra/internal/monitors/README.md @@ -63,7 +63,6 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -71,7 +70,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/src/apps/chifra/internal/monitors/handle_watch.go b/src/apps/chifra/internal/monitors/handle_watch.go index 40ce98b26c..ce2129049b 100644 --- a/src/apps/chifra/internal/monitors/handle_watch.go +++ b/src/apps/chifra/internal/monitors/handle_watch.go @@ -46,6 +46,7 @@ func (opts *MonitorsOptions) RunMonitorScraper(wg *sync.WaitGroup, s *Scraper) { s.ChangeState(true, tmpPath) + runCount := uint64(0) for { if !s.Running { s.Pause() @@ -61,11 +62,16 @@ func (opts *MonitorsOptions) RunMonitorScraper(wg *sync.WaitGroup, s *Scraper) { logger.Error(err) return } else { - if canceled || opts.RunOnce { + if canceled { return } } + runCount++ + if opts.RunCount == 0 || runCount >= opts.RunCount { + return + } + sleep := opts.Sleep if sleep > 0 { ms := time.Duration(sleep*1000) * time.Millisecond diff --git a/src/apps/chifra/internal/monitors/options.go b/src/apps/chifra/internal/monitors/options.go index 03e9047f02..4c5db06621 100644 --- a/src/apps/chifra/internal/monitors/options.go +++ b/src/apps/chifra/internal/monitors/options.go @@ -32,7 +32,7 @@ type MonitorsOptions struct { Watchlist string `json:"watchlist,omitempty"` // Available with --watch option only, a file containing the addresses to watch Commands string `json:"commands,omitempty"` // Available with --watch option only, the file containing the list of commands to apply to each watched address BatchSize uint64 `json:"batchSize,omitempty"` // Available with --watch option only, the number of monitors to process in each batch - RunOnce bool `json:"runOnce,omitempty"` // Available with --watch option only, only run the monitor --watch commands once then quit + RunCount uint64 `json:"runCount,omitempty"` // Available with --watch option only, run the monitor this many times, then quit Sleep float64 `json:"sleep,omitempty"` // Available with --watch option only, the number of seconds to sleep between runs Globals globals.GlobalOptions `json:"globals,omitempty"` // The global options Conn *rpc.Connection `json:"conn,omitempty"` // The connection to the RPC server @@ -57,7 +57,7 @@ func (opts *MonitorsOptions) testLog() { logger.TestLog(len(opts.Watchlist) > 0, "Watchlist: ", opts.Watchlist) logger.TestLog(len(opts.Commands) > 0, "Commands: ", opts.Commands) logger.TestLog(opts.BatchSize != 8, "BatchSize: ", opts.BatchSize) - logger.TestLog(opts.RunOnce, "RunOnce: ", opts.RunOnce) + logger.TestLog(opts.RunCount != 0, "RunCount: ", opts.RunCount) logger.TestLog(opts.Sleep != float64(14), "Sleep: ", opts.Sleep) opts.Conn.TestLog(opts.getCaches()) opts.Globals.TestLog() @@ -74,6 +74,7 @@ func monitorsFinishParseApi(w http.ResponseWriter, r *http.Request) *MonitorsOpt copy := defaultMonitorsOptions opts := © opts.BatchSize = 8 + opts.RunCount = 0 opts.Sleep = 14 for key, value := range r.URL.Query() { switch key { @@ -100,8 +101,8 @@ func monitorsFinishParseApi(w http.ResponseWriter, r *http.Request) *MonitorsOpt opts.Commands = value[0] case "batchSize": opts.BatchSize = globals.ToUint64(value[0]) - case "runOnce": - opts.RunOnce = true + case "runCount": + opts.RunCount = globals.ToUint64(value[0]) case "sleep": opts.Sleep = globals.ToFloat64(value[0]) default: diff --git a/src/apps/chifra/internal/monitors/validate.go b/src/apps/chifra/internal/monitors/validate.go index 9c819f59de..69eca3cbe1 100644 --- a/src/apps/chifra/internal/monitors/validate.go +++ b/src/apps/chifra/internal/monitors/validate.go @@ -72,18 +72,18 @@ func (opts *MonitorsOptions) validateMonitors() error { } if opts.BatchSize < 1 { - return validate.Usage("The {0} option must be greater than zero.", "--batch-size") + return validate.Usage("The {0} option must be greater than zero.", "--batch_size") } } else { if opts.BatchSize != 8 { - return validate.Usage("The {0} option is not available{1}.", "--batch-size", " without --watch") + return validate.Usage("The {0} option is not available{1}.", "--batch_size", " without --watch") } else { opts.BatchSize = 0 } - if opts.RunOnce { - return validate.Usage("The {0} option is not available{1}.", "--run-once", " without --watch") + if opts.RunCount > 0 { + return validate.Usage("The {0} option is not available{1}.", "--run_count", " without --watch") } if opts.Sleep != 14 { diff --git a/src/apps/chifra/internal/scrape/handle_scrape.go b/src/apps/chifra/internal/scrape/handle_scrape.go index e8ead86996..dd525e2e8d 100644 --- a/src/apps/chifra/internal/scrape/handle_scrape.go +++ b/src/apps/chifra/internal/scrape/handle_scrape.go @@ -37,7 +37,7 @@ func (opts *ScrapeOptions) HandleScrape() error { } // Clean the temporary files and makes sure block zero has been processed - if ok, err := opts.Prepare(); !ok || err != nil { + if ok, err := blazeMan.Prepare(); !ok || err != nil { return err } @@ -45,6 +45,7 @@ func (opts *ScrapeOptions) HandleScrape() error { unripePath := filepath.Join(config.GetPathToIndex(chain), "unripe") // The forever loop. Loop until the user hits Cntl+C or the server tells us to stop. + runCount := uint64(0) for { if blazeMan.meta, err = opts.Conn.GetMetaData(testMode); err != nil { logger.Error(fmt.Sprintf("Error fetching meta data: %s. Sleeping...", err)) @@ -89,7 +90,7 @@ func (opts *ScrapeOptions) HandleScrape() error { // Here we do the actual scrape for this round. If anything goes wrong, the // function will have cleaned up (i.e. remove the unstaged ripe blocks). Note // that we don't quit, instead we sleep and we retry continually. - if err := blazeMan.HandleScrapeBlaze(); err != nil { + if err := blazeMan.ScrapeBatch(); err != nil { logger.Error(colors.BrightRed, err, colors.Off) goto PAUSE } @@ -110,7 +111,12 @@ func (opts *ScrapeOptions) HandleScrape() error { return err } - blazeMan.Pause() + runCount++ + if opts.RunCount == 0 || runCount >= opts.RunCount { + break + } + + blazeMan.pause() } return nil diff --git a/src/apps/chifra/internal/scrape/options.go b/src/apps/chifra/internal/scrape/options.go index 8006bf95f4..83aa60e613 100644 --- a/src/apps/chifra/internal/scrape/options.go +++ b/src/apps/chifra/internal/scrape/options.go @@ -26,6 +26,7 @@ type ScrapeOptions struct { Remote bool `json:"remote,omitempty"` // Pin new chunks to the gateway (requires pinning service keys) Sleep float64 `json:"sleep,omitempty"` // Seconds to sleep between scraper passes StartBlock uint64 `json:"startBlock,omitempty"` // First block to visit when scraping (snapped back to most recent snap_to_grid mark) + RunCount uint64 `json:"runCount,omitempty"` // Run the scraper this many times, then quit Settings scrapeCfg.ScrapeSettings `json:"settings,omitempty"` // Configuration items for the scrape Globals globals.GlobalOptions `json:"globals,omitempty"` // The global options Conn *rpc.Connection `json:"conn,omitempty"` // The connection to the RPC server @@ -45,6 +46,7 @@ func (opts *ScrapeOptions) testLog() { logger.TestLog(opts.Remote, "Remote: ", opts.Remote) logger.TestLog(opts.Sleep != float64(14), "Sleep: ", opts.Sleep) logger.TestLog(opts.StartBlock != 0, "StartBlock: ", opts.StartBlock) + logger.TestLog(opts.RunCount != 0, "RunCount: ", opts.RunCount) opts.Settings.TestLog(opts.Globals.Chain, opts.Globals.TestMode) opts.Conn.TestLog(opts.getCaches()) opts.Globals.TestLog() @@ -63,6 +65,7 @@ func scrapeFinishParseApi(w http.ResponseWriter, r *http.Request) *ScrapeOptions opts.BlockCnt = 2000 opts.Sleep = 14 opts.StartBlock = 0 + opts.RunCount = 0 opts.Settings.Apps_per_chunk = 200000 opts.Settings.Snap_to_grid = 100000 opts.Settings.First_snap = 0 @@ -80,6 +83,8 @@ func scrapeFinishParseApi(w http.ResponseWriter, r *http.Request) *ScrapeOptions opts.Sleep = globals.ToFloat64(value[0]) case "startBlock": opts.StartBlock = globals.ToUint64(value[0]) + case "runCount": + opts.RunCount = globals.ToUint64(value[0]) case "appsPerChunk": opts.Settings.Apps_per_chunk = globals.ToUint64(value[0]) case "snapToGrid": diff --git a/src/apps/chifra/internal/scrape/handle_scrape_blaze.go b/src/apps/chifra/internal/scrape/scrape_batch.go similarity index 95% rename from src/apps/chifra/internal/scrape/handle_scrape_blaze.go rename to src/apps/chifra/internal/scrape/scrape_batch.go index f90613689c..2cdfa5f31b 100644 --- a/src/apps/chifra/internal/scrape/handle_scrape_blaze.go +++ b/src/apps/chifra/internal/scrape/scrape_batch.go @@ -19,9 +19,9 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" ) -// HandleScrapeBlaze is called each time around the forever loop prior to calling into +// ScrapeBatch is called each time around the forever loop prior to calling into // Blaze to actually scrape the blocks. -func (bm *BlazeManager) HandleScrapeBlaze() error { +func (bm *BlazeManager) ScrapeBatch() error { chain := bm.chain // Do the actual scrape, wait until it finishes, clean up and return on failure diff --git a/src/apps/chifra/internal/scrape/handle_blaze_blaze.go b/src/apps/chifra/internal/scrape/scrape_blaze.go similarity index 80% rename from src/apps/chifra/internal/scrape/handle_blaze_blaze.go rename to src/apps/chifra/internal/scrape/scrape_blaze.go index d147470f30..8554539c77 100644 --- a/src/apps/chifra/internal/scrape/handle_blaze_blaze.go +++ b/src/apps/chifra/internal/scrape/scrape_blaze.go @@ -20,21 +20,15 @@ import ( // HandleBlaze does the actual scraping, walking through block_cnt blocks and querying traces and logs // and then extracting addresses and timestamps from those data structures. func (bm *BlazeManager) HandleBlaze() (ok bool, err error) { + nChannels := int(bm.opts.Settings.Channel_count) blocks := []base.Blknum{} start := bm.StartBlock() end := bm.StartBlock() + bm.BlockCount() - for block := start; block < end; block++ { blocks = append(blocks, block) } - return bm.HandleBlaze1(blocks) -} - -func (bm *BlazeManager) HandleBlaze1(blocks []base.Blknum) (ok bool, err error) { - nChannels := int(bm.opts.Settings.Channel_count) - // We need three pipelines...we shove into blocks, blocks shoves into appearances and timestamps blockChannel := make(chan base.Blknum) appearanceChannel := make(chan scrapedData) @@ -46,7 +40,7 @@ func (bm *BlazeManager) HandleBlaze1(blocks []base.Blknum) (ok bool, err error) blockWg.Add(nChannels) for i := 0; i < nChannels; i++ { go func() { - _ = bm.BlazeProcessBlocks(blockChannel, &blockWg, appearanceChannel, tsChannel) + _ = bm.ProcessBlocks(blockChannel, &blockWg, appearanceChannel, tsChannel) }() } @@ -54,7 +48,7 @@ func (bm *BlazeManager) HandleBlaze1(blocks []base.Blknum) (ok bool, err error) appWg.Add(nChannels) for i := 0; i < nChannels; i++ { go func() { - _ = bm.BlazeProcessAppearances(appearanceChannel, &appWg) + _ = bm.ProcessAppearances(appearanceChannel, &appWg) }() } @@ -62,7 +56,7 @@ func (bm *BlazeManager) HandleBlaze1(blocks []base.Blknum) (ok bool, err error) tsWg.Add(nChannels) for i := 0; i < nChannels; i++ { go func() { - _ = bm.BlazeProcessTimestamps(tsChannel, &tsWg) + _ = bm.ProcessTimestamps(tsChannel, &tsWg) }() } @@ -84,9 +78,9 @@ func (bm *BlazeManager) HandleBlaze1(blocks []base.Blknum) (ok bool, err error) return true, nil } -// BlazeProcessBlocks processes the block channel and for each block query the node for both +// ProcessBlocks processes the block channel and for each block query the node for both // traces and logs. Send results down appearanceChannel. -func (bm *BlazeManager) BlazeProcessBlocks(blockChannel chan base.Blknum, blockWg *sync.WaitGroup, appearanceChannel chan scrapedData, tsChannel chan tslib.TimestampRecord) (err error) { +func (bm *BlazeManager) ProcessBlocks(blockChannel chan base.Blknum, blockWg *sync.WaitGroup, appearanceChannel chan scrapedData, tsChannel chan tslib.TimestampRecord) (err error) { defer blockWg.Done() for bn := range blockChannel { @@ -122,8 +116,8 @@ func (bm *BlazeManager) BlazeProcessBlocks(blockChannel chan base.Blknum, blockW var blazeMutex sync.Mutex -// BlazeProcessAppearances processes scrapedData objects shoved down the appearanceChannel -func (bm *BlazeManager) BlazeProcessAppearances(appearanceChannel chan scrapedData, appWg *sync.WaitGroup) (err error) { +// ProcessAppearances processes scrapedData objects shoved down the appearanceChannel +func (bm *BlazeManager) ProcessAppearances(appearanceChannel chan scrapedData, appWg *sync.WaitGroup) (err error) { defer appWg.Done() for sData := range appearanceChannel { @@ -139,7 +133,7 @@ func (bm *BlazeManager) BlazeProcessAppearances(appearanceChannel chan scrapedDa return err } - err = bm.WriteAppearancesBlaze(sData.bn, addrMap) + err = bm.WriteAppearances(sData.bn, addrMap) if err != nil { return err } @@ -148,8 +142,8 @@ func (bm *BlazeManager) BlazeProcessAppearances(appearanceChannel chan scrapedDa return } -// BlazeProcessTimestamps processes timestamp data (currently by printing to a temporary file) -func (bm *BlazeManager) BlazeProcessTimestamps(tsChannel chan tslib.TimestampRecord, tsWg *sync.WaitGroup) (err error) { +// ProcessTimestamps processes timestamp data (currently by printing to a temporary file) +func (bm *BlazeManager) ProcessTimestamps(tsChannel chan tslib.TimestampRecord, tsWg *sync.WaitGroup) (err error) { defer tsWg.Done() for ts := range tsChannel { @@ -163,7 +157,8 @@ func (bm *BlazeManager) BlazeProcessTimestamps(tsChannel chan tslib.TimestampRec var writeMutex sync.Mutex -func (bm *BlazeManager) WriteAppearancesBlaze(bn base.Blknum, addrMap index.AddressBooleanMap) (err error) { +// WriteAppearances writes the appearance for a chunk to a file +func (bm *BlazeManager) WriteAppearances(bn base.Blknum, addrMap index.AddressBooleanMap) (err error) { if len(addrMap) > 0 { appearanceArray := make([]string, 0, len(addrMap)) for record := range addrMap { diff --git a/src/apps/chifra/internal/scrape/scrape_consolidate.go b/src/apps/chifra/internal/scrape/scrape_consolidate.go index 779be4d836..ba9d0ed3ad 100644 --- a/src/apps/chifra/internal/scrape/scrape_consolidate.go +++ b/src/apps/chifra/internal/scrape/scrape_consolidate.go @@ -10,7 +10,6 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config/scrapeCfg" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/file" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/index" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" @@ -24,8 +23,6 @@ const asciiAppearanceSize = 59 func (bm *BlazeManager) Consolidate() (bool, error) { chain := bm.chain - bm.syncedReporting(bm.StartBlock()+bm.BlockCount(), true /* force */) - // Get a sorted list of files in the ripe folder ripeFolder := filepath.Join(config.GetPathToIndex(chain), "ripe") ripeFileList, err := os.ReadDir(ripeFolder) @@ -60,7 +57,7 @@ func (bm *BlazeManager) Consolidate() (bool, error) { unripeDist := bm.opts.Settings.Unripe_dist if uint64(ripeCnt) < (bm.BlockCount() - unripeDist) { // Then, if they are not at least sequential, clean up and try again... - allowMissing := scrapeCfg.AllowMissing(chain) + allowMissing := bm.opts.Settings.Allow_missing if err := isListSequential(chain, ripeFileList, allowMissing); err != nil { _ = index.CleanTemporaryFolders(config.GetPathToCache(chain), false) return true, err @@ -161,15 +158,12 @@ func (bm *BlazeManager) Consolidate() (bool, error) { os.Remove(fileName) // cleans up by replacing the previous stage return true, err } - // logger.Info(colors.Red, "fileName:", fileName, colors.Off) - // logger.Info(colors.Red, "curRange:", curRange, colors.Off) } stageFn, _ = file.LatestFileInFolder(stageFolder) // it may not exist... nAppsNow := int(file.FileSize(stageFn) / asciiAppearanceSize) - bm.Report(nAppsThen, nAppsNow) + bm.report(nAppsThen, nAppsNow) - // logger.Info("Removing backup file as it's not needed.") os.Remove(backupFn) // commits the change return true, err diff --git a/src/apps/chifra/internal/scrape/scrape_manager.go b/src/apps/chifra/internal/scrape/scrape_manager.go index f7dc1f4064..d217527a27 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager.go +++ b/src/apps/chifra/internal/scrape/scrape_manager.go @@ -38,7 +38,7 @@ func (bm *BlazeManager) BlockCount() base.Blknum { } // Report prints out a report of the progress of the scraper. -func (bm *BlazeManager) Report(nAppsThen, nAppsNow int) { +func (bm *BlazeManager) report(nAppsThen, nAppsNow int) { settings := bm.opts.Settings msg := "Block={%d} have {%d} appearances of {%d} ({%0.1f%%}). Need {%d} more. Added {%d} records ({%0.2f} apps/blk)." @@ -56,7 +56,7 @@ func (bm *BlazeManager) Report(nAppsThen, nAppsNow int) { } // Pause goes to sleep for a period of time based on the settings. -func (bm *BlazeManager) Pause() { +func (bm *BlazeManager) pause() { // we always pause at least a quarter of a second to allow the node to 'rest' time.Sleep(250 * time.Millisecond) isDefaultSleep := bm.opts.Sleep >= 13 && bm.opts.Sleep <= 14 diff --git a/src/apps/chifra/internal/scrape/scrape_prepare.go b/src/apps/chifra/internal/scrape/scrape_prepare.go index 969b1378ee..b4d62f3fb9 100644 --- a/src/apps/chifra/internal/scrape/scrape_prepare.go +++ b/src/apps/chifra/internal/scrape/scrape_prepare.go @@ -19,8 +19,8 @@ import ( // forever loop. Returns true if processing should continue, false otherwise. // The routine cleans the temporary folders (if any) and then makes sure the zero // block (reads the allocation file, if present) is processed. -func (opts *ScrapeOptions) Prepare() (ok bool, err error) { - chain := opts.Globals.Chain +func (bm *BlazeManager) Prepare() (ok bool, err error) { + chain := bm.chain // We always clean the temporary folders (other than staging) when starting _ = index.CleanTemporaryFolders(config.GetPathToIndex(chain), false) @@ -50,13 +50,13 @@ func (opts *ScrapeOptions) Prepare() (ok bool, err error) { array := []tslib.TimestampRecord{} array = append(array, tslib.TimestampRecord{ Bn: uint32(0), - Ts: uint32(opts.Conn.GetBlockTimestamp(0)), + Ts: uint32(bm.opts.Conn.GetBlockTimestamp(0)), }) _ = tslib.Append(chain, array) logger.Info("Writing block zero allocations for", len(prefunds), "prefunds, nAddresses:", len(appMap)) indexPath := index.ToIndexPath(bloomPath) - if report, err := index.WriteChunk(chain, indexPath, appMap, len(prefunds), opts.Pin, opts.Remote); err != nil { + if report, err := index.WriteChunk(chain, indexPath, appMap, len(prefunds), bm.opts.Pin, bm.opts.Remote); err != nil { return false, err } else if report == nil { logger.Fatal("Should not happen, write chunk returned empty report") diff --git a/src/apps/chifra/internal/scrape/validate.go b/src/apps/chifra/internal/scrape/validate.go index 8ca0fd8ce1..6db468018b 100644 --- a/src/apps/chifra/internal/scrape/validate.go +++ b/src/apps/chifra/internal/scrape/validate.go @@ -35,6 +35,10 @@ func (opts *ScrapeOptions) validateScrape() error { return validate.Usage("{0} requires tracing, err: {1}", "chifra scrape", rpc.ErrTraceBlockMissing) } + if !opts.Conn.IsNodeArchive() { + return validate.Usage("{0} requires {1}.", "chifra scrape", "an archive node") + } + if opts.Sleep < .25 { return validate.Usage("The {0} option ({1}) must {2}.", "--sleep", fmt.Sprintf("%f", opts.Sleep), "be at least .25") } diff --git a/src/apps/chifra/pkg/config/scrapeCfg/scrapeCfg.go b/src/apps/chifra/pkg/config/scrapeCfg/scrapeCfg.go index ba4ebb5dfc..99f5223284 100644 --- a/src/apps/chifra/pkg/config/scrapeCfg/scrapeCfg.go +++ b/src/apps/chifra/pkg/config/scrapeCfg/scrapeCfg.go @@ -184,11 +184,6 @@ func (s *ScrapeSettings) overlay(chain string, overlay ScrapeSettings) { // EXISTING_CODE // -func AllowMissing(chain string) bool { - s, _ := GetSettings(chain, "blockScrape.toml", nil) - return s.Allow_missing -} - func toEnvStr(name string) string { return "TB_SETTINGS_" + strings.ToUpper(strings.Replace(name, "_", "", -1)) } diff --git a/src/apps/chifra/pkg/types/types_namedblock.go b/src/apps/chifra/pkg/types/types_namedblock.go index 8234d1a520..d3de571866 100644 --- a/src/apps/chifra/pkg/types/types_namedblock.go +++ b/src/apps/chifra/pkg/types/types_namedblock.go @@ -81,7 +81,7 @@ func (s *SimpleNamedBlock) Model(verbose bool, format string, extraOptions map[s if verbose { model["component"] = s.Component - order = append([]string{"component"}, order...) + order = append(order, "component") model["description"] = s.Description order = append(order, "description") diff --git a/src/cmd-line-options.csv b/src/cmd-line-options.csv index cc85c7b650..0efccb2657 100644 --- a/src/cmd-line-options.csv +++ b/src/cmd-line-options.csv @@ -50,11 +50,11 @@ num,group,tags,api_route,tool,longName,hotKey,def_val,is_required,is_customizabl 11288,apps,Accounts,monitors,acctExport,watchlist,a,,false,false,true,true,gocmd,flag,,available with --watch option only, a file containing the addresses to watch 11292,apps,Accounts,monitors,acctExport,commands,c,,false,false,true,true,gocmd,flag,,available with --watch option only, the file containing the list of commands to apply to each watched address 11294,apps,Accounts,monitors,acctExport,batch_size,b,8,false,false,true,true,gocmd,flag,,available with --watch option only, the number of monitors to process in each batch -11296,apps,Accounts,monitors,acctExport,run_once,r,,false,false,true,true,gocmd,switch,,available with --watch option only, only run the monitor --watch commands once then quit +11296,apps,Accounts,monitors,acctExport,run_count,u,0,false,false,false,false,gocmd,flag,,available with --watch option only, run the monitor this many times, then quit 11298,apps,Accounts,monitors,acctExport,sleep,s,14,false,false,true,true,gocmd,flag,,available with --watch option only, the number of seconds to sleep between runs 11094,apps,Accounts,monitors,acctExport,,,,false,false,true,true,--,description,,Add, remove, clean, and list address monitors. 11095,apps,Accounts,monitors,acctExport,n1,,,false,false,false,false,--,note,,An `address` must be either an ENS name or start with '0x' and be forty-two characters long. -11096,apps,Accounts,monitors,acctExport,n2,,,false,false,false,false,--,note,,If no address is presented to the --clean command, all existing monitors are be cleaned. +11096,apps,Accounts,monitors,acctExport,n2,,,false,false,false,false,--,note,,If no address is presented to the --clean command, all existing monitors will be cleaned. 11096,apps,Accounts,monitors,acctExport,n3,,,false,false,false,false,--,note,,The --watch option requires two additional parameters to be specified: `--watchlist` and `--commands`. 11097,apps,Accounts,monitors,acctExport,n4,,,false,false,false,false,--,note,,Addresses provided on the command line are ignored in `--watch` mode. 11098,apps,Accounts,monitors,acctExport,n5,,,false,false,false,false,--,note,,Providing the value `existing` to the `--watchlist` monitors all existing monitor files (see --list). @@ -223,6 +223,7 @@ num,group,tags,api_route,tool,longName,hotKey,def_val,is_required,is_customizabl 12132,apps,Admin,scrape,blockScrape,remote,r,,false,false,true,true,gocmd,switch,,pin new chunks to the gateway (requires pinning service keys) 12115,apps,Admin,scrape,blockScrape,sleep,s,14,false,false,true,true,gocmd,flag,,seconds to sleep between scraper passes 12185,apps,Admin,scrape,blockScrape,start_block,l,0,false,false,true,true,gocmd,flag,,first block to visit when scraping (snapped back to most recent snap_to_grid mark) +12187,apps,Admin,scrape,blockScrape,run_count,u,0,false,false,false,false,gocmd,flag,,run the scraper this many times, then quit 12160,apps,Admin,scrape,blockScrape,apps_per_chunk,,200000,false,false,false,false,config,flag,,the number of appearances to build into a chunk before consolidating it 12170,apps,Admin,scrape,blockScrape,snap_to_grid,,100000,false,false,false,false,config,flag,,an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index 12175,apps,Admin,scrape,blockScrape,first_snap,,0,false,false,false,false,config,flag,,the first block at which snap_to_grid is enabled diff --git a/src/dev_tools/makeClass/results/report.csv b/src/dev_tools/makeClass/results/report.csv index 799b099f6a..c4041b7b74 100644 --- a/src/dev_tools/makeClass/results/report.csv +++ b/src/dev_tools/makeClass/results/report.csv @@ -447,9 +447,9 @@ monitors,remove,cmds monitors,remove,python monitors,remove,readme monitors,remove,typescript -monitors,runOnce,api -monitors,runOnce,python -monitors,runOnce,typescript +monitors,runCount,api +monitors,runCount,python +monitors,runCount,typescript monitors,sleep,api monitors,sleep,cmds monitors,sleep,python diff --git a/src/dev_tools/testRunner/testCases/apps/acctExport.csv b/src/dev_tools/testRunner/testCases/apps/acctExport.csv index ed33ad46f1..e24cded770 100644 --- a/src/dev_tools/testRunner/testCases/apps/acctExport.csv +++ b/src/dev_tools/testRunner/testCases/apps/acctExport.csv @@ -77,9 +77,9 @@ on ,both ,fast ,monitors ,apps/acctExport ,monitors_err_not_both1 ,y on ,both ,fast ,monitors ,apps/acctExport ,monitors_err_not_both2 ,y ,addrs = 0xf503017d7baf7fbc0fff7492b751025c6a78179b & fmt = json & delete & undelete on ,both ,fast ,monitors ,apps/acctExport ,monitors_err_sleep ,y ,addrs = 0xf503017d7baf7fbc0fff7492b751025c6a78179b & fmt = json & sleep on ,both ,fast ,monitors ,apps/acctExport ,monitors_err_batch_size2 ,y ,addrs = 0xf503017d7baf7fbc0fff7492b751025c6a78179b & fmt = json & batch_size = 1 -on ,both ,fast ,monitors ,apps/acctExport ,monitors_err_run_once ,y ,addrs = 0xf503017d7baf7fbc0fff7492b751025c6a78179b & fmt = json & run_once +on ,both ,fast ,monitors ,apps/acctExport ,monitors_err_run_once ,y ,addrs = 0xf503017d7baf7fbc0fff7492b751025c6a78179b & fmt = json & run_count = 1 on ,both ,fast ,monitors ,apps/acctExport ,monitors_err_watch ,y ,watch & batch_size = 0 & fmt = json -local ,both ,fast ,monitors ,apps/acctExport ,monitors_watch ,y ,watch & commands = ./command.fil & watchlist = ./watches.txt & run_once & fmt = json +local ,both ,fast ,monitors ,apps/acctExport ,monitors_watch ,y ,watch & commands = ./command.fil & watchlist = ./watches.txt & run_count = 1 & fmt = json on ,both ,fast ,list ,apps/acctExport ,list_prepare_1 ,y ,addrs = 0x001d14804b399c6ef80e64576f657660804fec0b & fmt = json & last_block = 1501460 on ,both ,fast ,monitors ,apps/acctExport ,monitors_clean ,y ,addrs = 0x001d14804b399c6ef80e64576f657660804fec0b & clean diff --git a/test/gold/apps/acctExport/acctExport_caps_allowed_m.txt b/test/gold/apps/acctExport/acctExport_caps_allowed_m.txt index bcbbf8b664..087beeeaca 100644 --- a/test/gold/apps/acctExport/acctExport_caps_allowed_m.txt +++ b/test/gold/apps/acctExport/acctExport_caps_allowed_m.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_caps_disallowed_1_m.txt b/test/gold/apps/acctExport/acctExport_caps_disallowed_1_m.txt index 4108f7bdcc..86b64a307f 100644 --- a/test/gold/apps/acctExport/acctExport_caps_disallowed_1_m.txt +++ b/test/gold/apps/acctExport/acctExport_caps_disallowed_1_m.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_caps_disallowed_2_m.txt b/test/gold/apps/acctExport/acctExport_caps_disallowed_2_m.txt index 1c125d966c..a34e618b44 100644 --- a/test/gold/apps/acctExport/acctExport_caps_disallowed_2_m.txt +++ b/test/gold/apps/acctExport/acctExport_caps_disallowed_2_m.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_caps_disallowed_3_m.txt b/test/gold/apps/acctExport/acctExport_caps_disallowed_3_m.txt index 07eb06c33b..8fae8c57d7 100644 --- a/test/gold/apps/acctExport/acctExport_caps_disallowed_3_m.txt +++ b/test/gold/apps/acctExport/acctExport_caps_disallowed_3_m.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_caps_disallowed_4_m.txt b/test/gold/apps/acctExport/acctExport_caps_disallowed_4_m.txt index b59473a2e9..270843cb81 100644 --- a/test/gold/apps/acctExport/acctExport_caps_disallowed_4_m.txt +++ b/test/gold/apps/acctExport/acctExport_caps_disallowed_4_m.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_decache.txt b/test/gold/apps/acctExport/acctExport_monitors_decache.txt index 9316815ad0..8c6057d8f3 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_decache.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_decache.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_decache_again.txt b/test/gold/apps/acctExport/acctExport_monitors_decache_again.txt index 9316815ad0..8c6057d8f3 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_decache_again.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_decache_again.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_delete_1_fail.txt b/test/gold/apps/acctExport/acctExport_monitors_delete_1_fail.txt index a3e73d28db..ecf4501e94 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_delete_1_fail.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_delete_1_fail.txt @@ -19,7 +19,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -27,7 +27,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_err_batch_size2.txt b/test/gold/apps/acctExport/acctExport_monitors_err_batch_size2.txt index ab694aadef..359b9cbcee 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_err_batch_size2.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_err_batch_size2.txt @@ -2,7 +2,7 @@ chifra monitors 0xf503017d7baf7fbc0fff7492b751025c6a78179b --fmt json --batch_s TEST[DATE|TIME] Addrs: [0xf503017d7baf7fbc0fff7492b751025c6a78179b] TEST[DATE|TIME] BatchSize: 1 TEST[DATE|TIME] Format: json -Error: The --batch-size option is not available without --watch. +Error: The --batch_size option is not available without --watch. Usage: chifra monitors [flags]
[address...] @@ -19,7 +19,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -27,7 +27,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_err_not_both1.txt b/test/gold/apps/acctExport/acctExport_monitors_err_not_both1.txt index 9e2f2b992e..5296ee9ef9 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_err_not_both1.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_err_not_both1.txt @@ -20,7 +20,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -28,7 +28,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_err_not_both2.txt b/test/gold/apps/acctExport/acctExport_monitors_err_not_both2.txt index 7c60a80a66..97dc2b5860 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_err_not_both2.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_err_not_both2.txt @@ -20,7 +20,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -28,7 +28,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_err_run_once.txt b/test/gold/apps/acctExport/acctExport_monitors_err_run_once.txt index 72f995aca3..f83fd21aec 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_err_run_once.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_err_run_once.txt @@ -1,8 +1,8 @@ -chifra monitors 0xf503017d7baf7fbc0fff7492b751025c6a78179b --fmt json --run_once +chifra monitors 0xf503017d7baf7fbc0fff7492b751025c6a78179b --fmt json --run_count 1 TEST[DATE|TIME] Addrs: [0xf503017d7baf7fbc0fff7492b751025c6a78179b] -TEST[DATE|TIME] RunOnce: true +TEST[DATE|TIME] RunCount: 1 TEST[DATE|TIME] Format: json -Error: The --run-once option is not available without --watch. +Error: The --run_count option is not available without --watch. Usage: chifra monitors [flags]
[address...] @@ -19,7 +19,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -27,7 +27,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_err_sleep.txt b/test/gold/apps/acctExport/acctExport_monitors_err_sleep.txt index 85776d3a66..2900228d7f 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_err_sleep.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_err_sleep.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_err_watch.txt b/test/gold/apps/acctExport/acctExport_monitors_err_watch.txt index 62aef639ab..f603369cbd 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_err_watch.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_err_watch.txt @@ -19,7 +19,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -27,7 +27,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_help.txt b/test/gold/apps/acctExport/acctExport_monitors_help.txt index 831bd90b20..bc4a945b20 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_help.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_help.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_long_help.txt b/test/gold/apps/acctExport/acctExport_monitors_long_help.txt index 0867f532da..d227a75c0b 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_long_help.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_long_help.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_no_params.txt b/test/gold/apps/acctExport/acctExport_monitors_no_params.txt index 8c8df94fbf..3edd3535ff 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_no_params.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_no_params.txt @@ -17,7 +17,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -25,7 +25,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_remove_fail.txt b/test/gold/apps/acctExport/acctExport_monitors_remove_fail.txt index f0b2f8b4c1..7d784762b7 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_remove_fail.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_remove_fail.txt @@ -19,7 +19,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -27,7 +27,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_undelete2.txt b/test/gold/apps/acctExport/acctExport_monitors_undelete2.txt index 215be6719d..362291cf3b 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_undelete2.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_undelete2.txt @@ -19,7 +19,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -27,7 +27,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_undelete_fail.txt b/test/gold/apps/acctExport/acctExport_monitors_undelete_fail.txt index 046185fce4..cd099e2b93 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_undelete_fail.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_undelete_fail.txt @@ -19,7 +19,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -27,7 +27,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/acctExport/acctExport_monitors_watch.txt b/test/gold/apps/acctExport/acctExport_monitors_watch.txt index 2cad272115..65295e2e27 100644 --- a/test/gold/apps/acctExport/acctExport_monitors_watch.txt +++ b/test/gold/apps/acctExport/acctExport_monitors_watch.txt @@ -1,8 +1,8 @@ -chifra monitors --watch --commands ./command.fil --watchlist ./watches.txt --run_once --fmt json +chifra monitors --watch --commands ./command.fil --watchlist ./watches.txt --run_count --fmt json TEST[DATE|TIME] Watch: true TEST[DATE|TIME] Watchlist: ./watches.txt TEST[DATE|TIME] Commands: ./command.fil -TEST[DATE|TIME] RunOnce: true +TEST[DATE|TIME] RunCount: 1 TEST[DATE|TIME] Format: json INFO[DATE|TIME] Reading address list from ./watches.txt 0-6 of 6: chifra export --freshen 0x0274d600fa9e2d9b18911a97f75bf7b3e61167d5 0x03836b0e02b4a613ba1d15834e6d77f409099d8f 0x0f6a7da303525288343a00a53e0557cc93920a6f 0x0c28a260c6b2969349eeaa3ba5253b616c1d9ac0 0x0ed97e3f247d915d8a2efb6a44f5acc4c6b71aae 0x02f2b09b33fdbd406ead954a31f98bd29a2a3492 diff --git a/test/gold/apps/acctExport/api_tests/acctExport_monitors_err_batch_size2.txt b/test/gold/apps/acctExport/api_tests/acctExport_monitors_err_batch_size2.txt index a0ee5db153..a52ba50515 100644 --- a/test/gold/apps/acctExport/api_tests/acctExport_monitors_err_batch_size2.txt +++ b/test/gold/apps/acctExport/api_tests/acctExport_monitors_err_batch_size2.txt @@ -1,6 +1,6 @@ monitors?addrs=0xf503017d7baf7fbc0fff7492b751025c6a78179b&fmt=json&batchSize=1 { "errors": [ - "The --batch-size option is not available without --watch." + "The --batch_size option is not available without --watch." ] } diff --git a/test/gold/apps/acctExport/api_tests/acctExport_monitors_err_run_once.txt b/test/gold/apps/acctExport/api_tests/acctExport_monitors_err_run_once.txt index 3c021f1e2f..c23a93e44a 100644 --- a/test/gold/apps/acctExport/api_tests/acctExport_monitors_err_run_once.txt +++ b/test/gold/apps/acctExport/api_tests/acctExport_monitors_err_run_once.txt @@ -1,6 +1,6 @@ -monitors?addrs=0xf503017d7baf7fbc0fff7492b751025c6a78179b&fmt=json&runOnce +monitors?addrs=0xf503017d7baf7fbc0fff7492b751025c6a78179b&fmt=json&runCount=1 { "errors": [ - "The --run-once option is not available without --watch." + "The --run_count option is not available without --watch." ] } diff --git a/test/gold/apps/acctExport/api_tests/acctExport_monitors_watch.txt b/test/gold/apps/acctExport/api_tests/acctExport_monitors_watch.txt index e08ea3cadc..e4a21e5008 100644 --- a/test/gold/apps/acctExport/api_tests/acctExport_monitors_watch.txt +++ b/test/gold/apps/acctExport/api_tests/acctExport_monitors_watch.txt @@ -1,4 +1,4 @@ -monitors?watch&commands=./command.fil&watchlist=./watches.txt&runOnce&fmt=json +monitors?watch&commands=./command.fil&watchlist=./watches.txt&runCount=1&fmt=json { "errors": [ "The --watch options is not available from the API" diff --git a/test/gold/apps/blockScrape/blockScrape_by_file_bad.txt b/test/gold/apps/blockScrape/blockScrape_by_file_bad.txt index 60af34af7c..4c3c61c4ad 100644 --- a/test/gold/apps/blockScrape/blockScrape_by_file_bad.txt +++ b/test/gold/apps/blockScrape/blockScrape_by_file_bad.txt @@ -9,6 +9,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_caps_allowed.txt b/test/gold/apps/blockScrape/blockScrape_caps_allowed.txt index 6783e6645e..f2a4737874 100644 --- a/test/gold/apps/blockScrape/blockScrape_caps_allowed.txt +++ b/test/gold/apps/blockScrape/blockScrape_caps_allowed.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_1.txt b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_1.txt index f3eca432f3..c3a546a905 100644 --- a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_1.txt +++ b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_1.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_10.txt b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_10.txt index 6a002f00ae..47e750b700 100644 --- a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_10.txt +++ b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_10.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_3.txt b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_3.txt index 397914c844..20d37fa221 100644 --- a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_3.txt +++ b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_3.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_4.txt b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_4.txt index 9b00d04ccf..8be6e32eab 100644 --- a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_4.txt +++ b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_4.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_5.txt b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_5.txt index 0ff9aded76..d45ce4b3bb 100644 --- a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_5.txt +++ b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_5.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_6.txt b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_6.txt index b9660b443f..e5d288e44f 100644 --- a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_6.txt +++ b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_6.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_7.txt b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_7.txt index 5098d60039..684d602d82 100644 --- a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_7.txt +++ b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_7.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_8.txt b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_8.txt index a53c5be0ad..7ecf034d16 100644 --- a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_8.txt +++ b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_8.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_9.txt b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_9.txt index e85013bd3a..35f0233ecb 100644 --- a/test/gold/apps/blockScrape/blockScrape_caps_disallowed_9.txt +++ b/test/gold/apps/blockScrape/blockScrape_caps_disallowed_9.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_help.txt b/test/gold/apps/blockScrape/blockScrape_help.txt index e856038a3b..7bfb9343b6 100644 --- a/test/gold/apps/blockScrape/blockScrape_help.txt +++ b/test/gold/apps/blockScrape/blockScrape_help.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_help_long.txt b/test/gold/apps/blockScrape/blockScrape_help_long.txt index 2a41400e4d..748d980b7c 100644 --- a/test/gold/apps/blockScrape/blockScrape_help_long.txt +++ b/test/gold/apps/blockScrape/blockScrape_help_long.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_invalid.txt b/test/gold/apps/blockScrape/blockScrape_invalid.txt index c2366e5308..2ae7eb5a3d 100644 --- a/test/gold/apps/blockScrape/blockScrape_invalid.txt +++ b/test/gold/apps/blockScrape/blockScrape_invalid.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_listpins.txt b/test/gold/apps/blockScrape/blockScrape_listpins.txt index 1f4d1a7318..b67d9be46a 100644 --- a/test/gold/apps/blockScrape/blockScrape_listpins.txt +++ b/test/gold/apps/blockScrape/blockScrape_listpins.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_mode_01_fail.txt b/test/gold/apps/blockScrape/blockScrape_mode_01_fail.txt index 7a2b64e743..66b5c3cdc1 100644 --- a/test/gold/apps/blockScrape/blockScrape_mode_01_fail.txt +++ b/test/gold/apps/blockScrape/blockScrape_mode_01_fail.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_nAProcs_invalid.txt b/test/gold/apps/blockScrape/blockScrape_nAProcs_invalid.txt index ba80013097..bedaf6dac7 100644 --- a/test/gold/apps/blockScrape/blockScrape_nAProcs_invalid.txt +++ b/test/gold/apps/blockScrape/blockScrape_nAProcs_invalid.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_nBProcs_invalid.txt b/test/gold/apps/blockScrape/blockScrape_nBProcs_invalid.txt index ba80013097..bedaf6dac7 100644 --- a/test/gold/apps/blockScrape/blockScrape_nBProcs_invalid.txt +++ b/test/gold/apps/blockScrape/blockScrape_nBProcs_invalid.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_nBlocks.txt b/test/gold/apps/blockScrape/blockScrape_nBlocks.txt index 68ddea0867..f79bf4a9ea 100644 --- a/test/gold/apps/blockScrape/blockScrape_nBlocks.txt +++ b/test/gold/apps/blockScrape/blockScrape_nBlocks.txt @@ -12,6 +12,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_nBlocks_invalid.txt b/test/gold/apps/blockScrape/blockScrape_nBlocks_invalid.txt index aaf35726c9..07744574e5 100644 --- a/test/gold/apps/blockScrape/blockScrape_nBlocks_invalid.txt +++ b/test/gold/apps/blockScrape/blockScrape_nBlocks_invalid.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/blockScrape/blockScrape_real_run2.txt b/test/gold/apps/blockScrape/blockScrape_real_run2.txt index b902fd982d..7868116ea7 100644 --- a/test/gold/apps/blockScrape/blockScrape_real_run2.txt +++ b/test/gold/apps/blockScrape/blockScrape_real_run2.txt @@ -12,6 +12,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/apps/chifra/chifra_help_rm.txt b/test/gold/apps/chifra/chifra_help_rm.txt index 101d45c165..14992538db 100644 --- a/test/gold/apps/chifra/chifra_help_rm.txt +++ b/test/gold/apps/chifra/chifra_help_rm.txt @@ -18,7 +18,7 @@ Flags: -a, --watchlist string available with --watch option only, a file containing the addresses to watch -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) - -r, --run_once available with --watch option only, only run the monitor --watch commands once then quit + -u, --run_count uint available with --watch option only, run the monitor this many times, then quit (hidden) -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) -x, --fmt string export format, one of [none|json*|txt|csv] -v, --verbose enable verbose output @@ -26,7 +26,7 @@ Flags: Notes: - An address must be either an ENS name or start with '0x' and be forty-two characters long. - - If no address is presented to the --clean command, all existing monitors are be cleaned. + - If no address is presented to the --clean command, all existing monitors will be cleaned. - The --watch option requires two additional parameters to be specified: --watchlist and --commands. - Addresses provided on the command line are ignored in --watch mode. - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). diff --git a/test/gold/apps/chifra/chifra_help_scrape.txt b/test/gold/apps/chifra/chifra_help_scrape.txt index bb4e379b5a..ff3aefb628 100644 --- a/test/gold/apps/chifra/chifra_help_scrape.txt +++ b/test/gold/apps/chifra/chifra_help_scrape.txt @@ -11,6 +11,7 @@ Flags: -r, --remote pin new chunks to the gateway (requires pinning service keys) -s, --sleep float seconds to sleep between scraper passes (default 14) -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -u, --run_count uint run the scraper this many times, then quit (hidden) --apps_per_chunk uint the number of appearances to build into a chunk before consolidating it (hidden) (default 200000) --snap_to_grid uint an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index (hidden) (default 100000) --first_snap uint the first block at which snap_to_grid is enabled (hidden) diff --git a/test/gold/tools/whenBlock/whenBlock_combined_options_1.txt b/test/gold/tools/whenBlock/whenBlock_combined_options_1.txt index 8602da8984..f6ae075c34 100644 --- a/test/gold/tools/whenBlock/whenBlock_combined_options_1.txt +++ b/test/gold/tools/whenBlock/whenBlock_combined_options_1.txt @@ -3,5 +3,5 @@ TEST[DATE|TIME] Blocks: [10000] TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Caps: cache,decache TEST[DATE|TIME] Format: txt -component blockNumber timestamp date name description - 10000 1438334627 2015-07-31 09:23:47 UTC +blockNumber timestamp date name component description +10000 1438334627 2015-07-31 09:23:47 UTC diff --git a/test/gold/tools/whenBlock/whenBlock_long_verbose_valid_block.txt b/test/gold/tools/whenBlock/whenBlock_long_verbose_valid_block.txt index f38d726f8c..3550be0e57 100644 --- a/test/gold/tools/whenBlock/whenBlock_long_verbose_valid_block.txt +++ b/test/gold/tools/whenBlock/whenBlock_long_verbose_valid_block.txt @@ -3,5 +3,5 @@ TEST[DATE|TIME] Blocks: [1000] TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Caps: cache,decache TEST[DATE|TIME] Format: txt -component blockNumber timestamp date name description - 1000 1438272138 2015-07-30 16:02:18 UTC +blockNumber timestamp date name component description +1000 1438272138 2015-07-30 16:02:18 UTC diff --git a/test/gold/tools/whenBlock/whenBlock_mixed_options.txt b/test/gold/tools/whenBlock/whenBlock_mixed_options.txt index 04157f6197..0aabe4412e 100644 --- a/test/gold/tools/whenBlock/whenBlock_mixed_options.txt +++ b/test/gold/tools/whenBlock/whenBlock_mixed_options.txt @@ -3,43 +3,43 @@ TEST[DATE|TIME] List: true TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Caps: cache,decache TEST[DATE|TIME] Format: txt -component blockNumber timestamp date name description -beacon 1 1606824028 2020-12-01 12:00:28 UTC beacon_genesis The Beacon Chain needed 16384 deposits of 32 staked ETH to ship securely. This happened on November 27, meaning the Beacon Chain started producing blocks on December 1, 2020. This is an important first step in achieving the Ethereum vision. -beacon 74240 1635332183 2021-10-27 10:56:23 UTC altair The Altair upgrade was the first scheduled upgrade for the Beacon Chain. It added support for sync committees—enabling light clients, and increased validator inactivity and slashing penalties as development progressed towards The Merge. -beacon 144896 1662464087 2022-09-06 11:34:47 UTC bellatrix The Bellatrix upgrade was the second scheduled upgrade for the Beacon Chain, preparing the chain for The Merge. It brings validator penalties to their full values for inactivity and slashable offenses. Bellatrix also includes an update to the fork choice rules to prepare the chain for The Merge and the transition from the last proof-of-work block to the first proof-of-stake block. This includes making consensus clients aware of the terminal total difficulty of 58750000000000000000000. -beacon 194048 1681338455 2023-04-12 22:27:35 UTC capella The Capella upgrade was the third major upgrade to the consensus layer (Beacon Chain) and enabled staking withdrawals. Capella occurred synchronously with the execution layer upgrade, Shanghai, and enabled staking withdrawal functionality. This consensus layer upgrade brought the ability for stakers who did not provide withdrawal credentials with their initial deposit to do so, thereby enabling withdrawals. The upgrade also provided automatic account sweeping functionality, which continuously processes validator accounts for any available rewards payments or full withdrawals. -execution 0 1438269975 2015-07-30 15:26:15 UTC frontier Frontier was a live, but barebone implementation of the Ethereum project. It followed the successful Olympic testing phase. It was intended for technical users, specifically developers. Blocks had a gas limit of 5,000. This ‘thawing’ period enabled miners to start their operations and for early adopters to install their clients without having to ‘rush’. -execution 46147 1438918233 2015-08-07 03:30:33 UTC firstTrans The block containing the first transaction on the Ethereum Mainnet blockchain. -execution 50111 1438984075 2015-08-07 21:47:55 UTC firstContract The block containing the first contract on the Ethereum Mainnet blockchain. -execution 200000 1441661589 2015-09-07 21:33:09 UTC iceage The frontier thawing fork lifted the 5,000 gas limit per block and set the default gas price to 51 gwei. This allowed for transactions – transactions require 21,000 gas. The difficulty bomb was introduced to ensure a future hard-fork to proof-of-stake. -execution 543626 1447577992 2015-11-15 08:59:52 UTC devcon1 The first Ethereum developer conference. -execution 1150000 1457981393 2016-03-14 18:49:53 UTC homestead The Homestead fork that looked to the future. It included several protocol changes and a networking change that gave Ethereum the ability to do further network upgrades. -execution 1428756 1461980573 2016-04-30 01:42:53 UTC daofund The first block people were able to fund the infamous DAO. -execution 1718497 1466134488 2016-06-17 03:34:48 UTC daohack The first blocks of the DAO hack. -execution 1920000 1469020840 2016-07-20 13:20:40 UTC daofork The DAO fork was in response to the 2016 DAO attack where an insecure DAO contract was drained of over 3.6 million ETH in a hack. The fork moved the funds from the faulty contract to a new contract with a single function: withdraw. Anyone who lost funds could withdraw 1 ETH for every 100 DAO tokens in their wallets. This course of action was voted on by the Ethereum community. Any ETH holder was able to vote via a transaction on a voting platform. The decision to fork reached over 85% of the votes. Some miners refused to fork because the DAO incident wasn't a defect in the protocol. They went on to form Ethereum Classic. -execution 2286910 1474275599 2016-09-19 08:59:59 UTC devcon2 The second Ethereum developer conference. -execution 2463000 1476796771 2016-10-18 13:19:31 UTC tangerine_whistle The Tangerine Whistle fork was the first response to the denial of service (DoS) attacks on the network (September/October 2016) including: addressing urgent network health issues concerning underpriced operation codes. -execution 2675000 1479831344 2016-11-22 16:15:44 UTC spurious_dragon The Spurious Dragon fork was the second response to the denial of service (DoS) attacks on the network (September/October 2016) including: tuning opcode pricing to prevent future attacks on the network. enabling “debloat” of the blockchain state. adding replay attack protection. -execution 2717576 1480448309 2016-11-29 19:38:29 UTC stateclear The block of the state clearing soft fork. -execution 3265360 1488272400 2017-02-28 09:00:00 UTC eea The establishment of the Ethereum Enterprise Association. -execution 3327417 1489165544 2017-03-10 17:05:44 UTC ens2 The second ENS deployment of ENS. -execution 4041179 1500417203 2017-07-18 22:33:23 UTC parityhack1 The block of the first Parity hack. -execution 4370000 1508131331 2017-10-16 05:22:11 UTC byzantium The Byzantium fork: Reduced block mining rewards from 5 to 3 ETH. Delayed the difficulty bomb by a year. Added ability to make non-state-changing calls to other contracts. Added certain cryptography methods to allow for layer 2 scaling. -execution 4469339 1509526798 2017-11-01 08:59:58 UTC devcon3 The third Ethereum developer conference. -execution 4501969 1509981921 2017-11-06 15:25:21 UTC parityhack2 The block of the second Parity hack. -execution 4605167 1511415679 2017-11-23 05:41:19 UTC kitties The block of the CryptoKitties contract. -execution 4620855 1511634257 2017-11-25 18:24:17 UTC makerdao The block when MakerDAO was deployed. -execution 6610517 1540890000 2018-10-30 09:00:00 UTC devcon4 The fourth Ethereum developer conference. -execution 6627917 1541137042 2018-11-02 05:37:22 UTC uniswap The block when Uniswap was deployed. -execution 7280000 1551383524 2019-02-28 19:52:04 UTC constantinople The Constantinople fork: Ensured the blockchain didn't freeze before proof-of-stake was implemented. Optimised the gas cost of certain actions in the EVM. Added the ability to interact with addresses that haven't been created yet. -execution 8700401 1570525179 2019-10-08 08:59:39 UTC devcon5 The fifth Ethereum developer conference. -execution 8928158 1573672677 2019-11-13 19:17:57 UTC mcdai The block when Multi-Collateral Dai was deployed. -execution 9069000 1575764709 2019-12-08 00:25:09 UTC istanbul The Istanbul fork: Optimised the gas cost of certain actions in the EVM. Improved denial-of-service attack resilience. Made Layer 2 scaling solutions based on SNARKs and STARKs more performant. Enabled Ethereum and Zcash to interoperate. Allowed contracts to introduce more creative functions. -execution 9200000 1577953849 2020-01-02 08:30:49 UTC muir_glacier The Muir Glacier fork introduced a delay to the difficulty bomb. Increases in block difficulty of the proof-of-work consensus mechanism threatened to degrade the usability of Ethereum by increasing wait times for sending transactions and using dapps. -execution 11052984 1602667372 2020-10-14 09:22:52 UTC deposit_contract The staking deposit contract introduced staking to the Ethereum ecosystem. Although a Mainnet contract, it had a direct impact on the timeline for launching the Beacon Chain, an important Ethereum upgrade. -execution 12244000 1618481223 2021-04-15 10:07:03 UTC berlin The Berlin upgrade optimized gas cost for certain EVM actions, and increases support for multiple transaction types. -execution 12965000 1628166822 2021-08-05 12:33:42 UTC london The London upgrade introduced EIP-1559 , which reformed the transaction fee market, along with changes to how gas refunds are handled and the Ice Age schedule. -execution 13773000 1639079723 2021-12-09 19:55:23 UTC arrow_glacier The Arrow Glacier network upgrade pushed back the difficulty bomb by several months. This is the only change introduced in this upgrade, and is similar in nature to the Muir Glacier upgrade. Similar changes have been performed on the Byzantium, Constantinople and London network upgrades. -execution 15050000 1656586444 2022-06-30 10:54:04 UTC gray_glacier The Gray Glacier network upgrade pushed back the difficulty bomb by three months. This is the only change introduced in this upgrade, and is similar in nature to the Arrow Glacier and Muir Glacier upgrades. Similar changes have been performed on the Byzantium, Constantinople and London network upgrades. -execution 15537393 1663224162 2022-09-15 06:42:42 UTC paris_merge The Paris upgrade was triggered by the proof-of-work blockchain passing a terminal total difficulty of 58750000000000000000000. This happened at block 15537393 on 15th September 2022, triggering the Paris upgrade the next block. Paris was The Merge transition - its major feature was switching off the proof-of-work mining algorithm and associated consensus logic and switching on proof-of-stake instead. Paris itself was an upgrade to the execution clients (equivalent to Bellatrix on the consensus layer) that enabled them to take instruction from their connected consensus clients. This required a new set of internal API methods, collectively known as the Engine API, to be activated. This was arguably the most significant upgrade in Ethereum history since Homestead! -execution 17034870 1681338479 2023-04-12 22:27:59 UTC shanghai The Shanghai upgrade brought staking withdrawals to the execution layer. In tandem with the Capella upgrade, this enabled blocks to accept withdrawal operations, which allows stakers to withdraw their ETH from the Beacon Chain to the execution layer. +blockNumber timestamp date name component description +1 1606824028 2020-12-01 12:00:28 UTC beacon_genesis beacon The Beacon Chain needed 16384 deposits of 32 staked ETH to ship securely. This happened on November 27, meaning the Beacon Chain started producing blocks on December 1, 2020. This is an important first step in achieving the Ethereum vision. +74240 1635332183 2021-10-27 10:56:23 UTC altair beacon The Altair upgrade was the first scheduled upgrade for the Beacon Chain. It added support for sync committees—enabling light clients, and increased validator inactivity and slashing penalties as development progressed towards The Merge. +144896 1662464087 2022-09-06 11:34:47 UTC bellatrix beacon The Bellatrix upgrade was the second scheduled upgrade for the Beacon Chain, preparing the chain for The Merge. It brings validator penalties to their full values for inactivity and slashable offenses. Bellatrix also includes an update to the fork choice rules to prepare the chain for The Merge and the transition from the last proof-of-work block to the first proof-of-stake block. This includes making consensus clients aware of the terminal total difficulty of 58750000000000000000000. +194048 1681338455 2023-04-12 22:27:35 UTC capella beacon The Capella upgrade was the third major upgrade to the consensus layer (Beacon Chain) and enabled staking withdrawals. Capella occurred synchronously with the execution layer upgrade, Shanghai, and enabled staking withdrawal functionality. This consensus layer upgrade brought the ability for stakers who did not provide withdrawal credentials with their initial deposit to do so, thereby enabling withdrawals. The upgrade also provided automatic account sweeping functionality, which continuously processes validator accounts for any available rewards payments or full withdrawals. +0 1438269975 2015-07-30 15:26:15 UTC frontier execution Frontier was a live, but barebone implementation of the Ethereum project. It followed the successful Olympic testing phase. It was intended for technical users, specifically developers. Blocks had a gas limit of 5,000. This ‘thawing’ period enabled miners to start their operations and for early adopters to install their clients without having to ‘rush’. +46147 1438918233 2015-08-07 03:30:33 UTC firstTrans execution The block containing the first transaction on the Ethereum Mainnet blockchain. +50111 1438984075 2015-08-07 21:47:55 UTC firstContract execution The block containing the first contract on the Ethereum Mainnet blockchain. +200000 1441661589 2015-09-07 21:33:09 UTC iceage execution The frontier thawing fork lifted the 5,000 gas limit per block and set the default gas price to 51 gwei. This allowed for transactions – transactions require 21,000 gas. The difficulty bomb was introduced to ensure a future hard-fork to proof-of-stake. +543626 1447577992 2015-11-15 08:59:52 UTC devcon1 execution The first Ethereum developer conference. +1150000 1457981393 2016-03-14 18:49:53 UTC homestead execution The Homestead fork that looked to the future. It included several protocol changes and a networking change that gave Ethereum the ability to do further network upgrades. +1428756 1461980573 2016-04-30 01:42:53 UTC daofund execution The first block people were able to fund the infamous DAO. +1718497 1466134488 2016-06-17 03:34:48 UTC daohack execution The first blocks of the DAO hack. +1920000 1469020840 2016-07-20 13:20:40 UTC daofork execution The DAO fork was in response to the 2016 DAO attack where an insecure DAO contract was drained of over 3.6 million ETH in a hack. The fork moved the funds from the faulty contract to a new contract with a single function: withdraw. Anyone who lost funds could withdraw 1 ETH for every 100 DAO tokens in their wallets. This course of action was voted on by the Ethereum community. Any ETH holder was able to vote via a transaction on a voting platform. The decision to fork reached over 85% of the votes. Some miners refused to fork because the DAO incident wasn't a defect in the protocol. They went on to form Ethereum Classic. +2286910 1474275599 2016-09-19 08:59:59 UTC devcon2 execution The second Ethereum developer conference. +2463000 1476796771 2016-10-18 13:19:31 UTC tangerine_whistle execution The Tangerine Whistle fork was the first response to the denial of service (DoS) attacks on the network (September/October 2016) including: addressing urgent network health issues concerning underpriced operation codes. +2675000 1479831344 2016-11-22 16:15:44 UTC spurious_dragon execution The Spurious Dragon fork was the second response to the denial of service (DoS) attacks on the network (September/October 2016) including: tuning opcode pricing to prevent future attacks on the network. enabling “debloat” of the blockchain state. adding replay attack protection. +2717576 1480448309 2016-11-29 19:38:29 UTC stateclear execution The block of the state clearing soft fork. +3265360 1488272400 2017-02-28 09:00:00 UTC eea execution The establishment of the Ethereum Enterprise Association. +3327417 1489165544 2017-03-10 17:05:44 UTC ens2 execution The second ENS deployment of ENS. +4041179 1500417203 2017-07-18 22:33:23 UTC parityhack1 execution The block of the first Parity hack. +4370000 1508131331 2017-10-16 05:22:11 UTC byzantium execution The Byzantium fork: Reduced block mining rewards from 5 to 3 ETH. Delayed the difficulty bomb by a year. Added ability to make non-state-changing calls to other contracts. Added certain cryptography methods to allow for layer 2 scaling. +4469339 1509526798 2017-11-01 08:59:58 UTC devcon3 execution The third Ethereum developer conference. +4501969 1509981921 2017-11-06 15:25:21 UTC parityhack2 execution The block of the second Parity hack. +4605167 1511415679 2017-11-23 05:41:19 UTC kitties execution The block of the CryptoKitties contract. +4620855 1511634257 2017-11-25 18:24:17 UTC makerdao execution The block when MakerDAO was deployed. +6610517 1540890000 2018-10-30 09:00:00 UTC devcon4 execution The fourth Ethereum developer conference. +6627917 1541137042 2018-11-02 05:37:22 UTC uniswap execution The block when Uniswap was deployed. +7280000 1551383524 2019-02-28 19:52:04 UTC constantinople execution The Constantinople fork: Ensured the blockchain didn't freeze before proof-of-stake was implemented. Optimised the gas cost of certain actions in the EVM. Added the ability to interact with addresses that haven't been created yet. +8700401 1570525179 2019-10-08 08:59:39 UTC devcon5 execution The fifth Ethereum developer conference. +8928158 1573672677 2019-11-13 19:17:57 UTC mcdai execution The block when Multi-Collateral Dai was deployed. +9069000 1575764709 2019-12-08 00:25:09 UTC istanbul execution The Istanbul fork: Optimised the gas cost of certain actions in the EVM. Improved denial-of-service attack resilience. Made Layer 2 scaling solutions based on SNARKs and STARKs more performant. Enabled Ethereum and Zcash to interoperate. Allowed contracts to introduce more creative functions. +9200000 1577953849 2020-01-02 08:30:49 UTC muir_glacier execution The Muir Glacier fork introduced a delay to the difficulty bomb. Increases in block difficulty of the proof-of-work consensus mechanism threatened to degrade the usability of Ethereum by increasing wait times for sending transactions and using dapps. +11052984 1602667372 2020-10-14 09:22:52 UTC deposit_contract execution The staking deposit contract introduced staking to the Ethereum ecosystem. Although a Mainnet contract, it had a direct impact on the timeline for launching the Beacon Chain, an important Ethereum upgrade. +12244000 1618481223 2021-04-15 10:07:03 UTC berlin execution The Berlin upgrade optimized gas cost for certain EVM actions, and increases support for multiple transaction types. +12965000 1628166822 2021-08-05 12:33:42 UTC london execution The London upgrade introduced EIP-1559 , which reformed the transaction fee market, along with changes to how gas refunds are handled and the Ice Age schedule. +13773000 1639079723 2021-12-09 19:55:23 UTC arrow_glacier execution The Arrow Glacier network upgrade pushed back the difficulty bomb by several months. This is the only change introduced in this upgrade, and is similar in nature to the Muir Glacier upgrade. Similar changes have been performed on the Byzantium, Constantinople and London network upgrades. +15050000 1656586444 2022-06-30 10:54:04 UTC gray_glacier execution The Gray Glacier network upgrade pushed back the difficulty bomb by three months. This is the only change introduced in this upgrade, and is similar in nature to the Arrow Glacier and Muir Glacier upgrades. Similar changes have been performed on the Byzantium, Constantinople and London network upgrades. +15537393 1663224162 2022-09-15 06:42:42 UTC paris_merge execution The Paris upgrade was triggered by the proof-of-work blockchain passing a terminal total difficulty of 58750000000000000000000. This happened at block 15537393 on 15th September 2022, triggering the Paris upgrade the next block. Paris was The Merge transition - its major feature was switching off the proof-of-work mining algorithm and associated consensus logic and switching on proof-of-stake instead. Paris itself was an upgrade to the execution clients (equivalent to Bellatrix on the consensus layer) that enabled them to take instruction from their connected consensus clients. This required a new set of internal API methods, collectively known as the Engine API, to be activated. This was arguably the most significant upgrade in Ethereum history since Homestead! +17034870 1681338479 2023-04-12 22:27:59 UTC shanghai execution The Shanghai upgrade brought staking withdrawals to the execution layer. In tandem with the Capella upgrade, this enabled blocks to accept withdrawal operations, which allows stakers to withdraw their ETH from the Beacon Chain to the execution layer. diff --git a/test/gold/tools/whenBlock/whenBlock_special_block_daofork.txt b/test/gold/tools/whenBlock/whenBlock_special_block_daofork.txt index 5c36e968a4..00d239eac1 100644 --- a/test/gold/tools/whenBlock/whenBlock_special_block_daofork.txt +++ b/test/gold/tools/whenBlock/whenBlock_special_block_daofork.txt @@ -3,5 +3,5 @@ TEST[DATE|TIME] Blocks: [daofork] TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Caps: cache,decache TEST[DATE|TIME] Format: txt -component blockNumber timestamp date name description -execution 1920000 1469020840 2016-07-20 13:20:40 UTC daofork The DAO fork was in response to the 2016 DAO attack where an insecure DAO contract was drained of over 3.6 million ETH in a hack. The fork moved the funds from the faulty contract to a new contract with a single function: withdraw. Anyone who lost funds could withdraw 1 ETH for every 100 DAO tokens in their wallets. This course of action was voted on by the Ethereum community. Any ETH holder was able to vote via a transaction on a voting platform. The decision to fork reached over 85% of the votes. Some miners refused to fork because the DAO incident wasn't a defect in the protocol. They went on to form Ethereum Classic. +blockNumber timestamp date name component description +1920000 1469020840 2016-07-20 13:20:40 UTC daofork execution The DAO fork was in response to the 2016 DAO attack where an insecure DAO contract was drained of over 3.6 million ETH in a hack. The fork moved the funds from the faulty contract to a new contract with a single function: withdraw. Anyone who lost funds could withdraw 1 ETH for every 100 DAO tokens in their wallets. This course of action was voted on by the Ethereum community. Any ETH holder was able to vote via a transaction on a voting platform. The decision to fork reached over 85% of the votes. Some miners refused to fork because the DAO incident wasn't a defect in the protocol. They went on to form Ethereum Classic. diff --git a/test/gold/tools/whenBlock/whenBlock_special_block_daofund.txt b/test/gold/tools/whenBlock/whenBlock_special_block_daofund.txt index 10eebd26d6..abd54c9133 100644 --- a/test/gold/tools/whenBlock/whenBlock_special_block_daofund.txt +++ b/test/gold/tools/whenBlock/whenBlock_special_block_daofund.txt @@ -3,5 +3,5 @@ TEST[DATE|TIME] Blocks: [daofund] TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Caps: cache,decache TEST[DATE|TIME] Format: txt -component blockNumber timestamp date name description -execution 1428756 1461980573 2016-04-30 01:42:53 UTC daofund The first block people were able to fund the infamous DAO. +blockNumber timestamp date name component description +1428756 1461980573 2016-04-30 01:42:53 UTC daofund execution The first block people were able to fund the infamous DAO. diff --git a/test/gold/tools/whenBlock/whenBlock_special_block_daohack.txt b/test/gold/tools/whenBlock/whenBlock_special_block_daohack.txt index 8f0a4cbaba..e33d612931 100644 --- a/test/gold/tools/whenBlock/whenBlock_special_block_daohack.txt +++ b/test/gold/tools/whenBlock/whenBlock_special_block_daohack.txt @@ -3,5 +3,5 @@ TEST[DATE|TIME] Blocks: [daohack] TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Caps: cache,decache TEST[DATE|TIME] Format: txt -component blockNumber timestamp date name description -execution 1718497 1466134488 2016-06-17 03:34:48 UTC daohack The first blocks of the DAO hack. +blockNumber timestamp date name component description +1718497 1466134488 2016-06-17 03:34:48 UTC daohack execution The first blocks of the DAO hack. diff --git a/test/gold/tools/whenBlock/whenBlock_special_block_first.txt b/test/gold/tools/whenBlock/whenBlock_special_block_first.txt index 3da9e89fef..2dc5f56d8f 100644 --- a/test/gold/tools/whenBlock/whenBlock_special_block_first.txt +++ b/test/gold/tools/whenBlock/whenBlock_special_block_first.txt @@ -3,5 +3,5 @@ TEST[DATE|TIME] Blocks: [frontier] TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Caps: cache,decache TEST[DATE|TIME] Format: txt -component blockNumber timestamp date name description -execution 0 1438269975 2015-07-30 15:26:15 UTC frontier Frontier was a live, but barebone implementation of the Ethereum project. It followed the successful Olympic testing phase. It was intended for technical users, specifically developers. Blocks had a gas limit of 5,000. This ‘thawing’ period enabled miners to start their operations and for early adopters to install their clients without having to ‘rush’. +blockNumber timestamp date name component description +0 1438269975 2015-07-30 15:26:15 UTC frontier execution Frontier was a live, but barebone implementation of the Ethereum project. It followed the successful Olympic testing phase. It was intended for technical users, specifically developers. Blocks had a gas limit of 5,000. This ‘thawing’ period enabled miners to start their operations and for early adopters to install their clients without having to ‘rush’. diff --git a/test/gold/tools/whenBlock/whenBlock_special_block_multi1.txt b/test/gold/tools/whenBlock/whenBlock_special_block_multi1.txt index 91f60731c7..1986feb504 100644 --- a/test/gold/tools/whenBlock/whenBlock_special_block_multi1.txt +++ b/test/gold/tools/whenBlock/whenBlock_special_block_multi1.txt @@ -3,7 +3,7 @@ TEST[DATE|TIME] Blocks: [stateclear iceage 4001001] TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Caps: cache,decache TEST[DATE|TIME] Format: txt -component blockNumber timestamp date name description -execution 2717576 1480448309 2016-11-29 19:38:29 UTC stateclear The block of the state clearing soft fork. -execution 200000 1441661589 2015-09-07 21:33:09 UTC iceage The frontier thawing fork lifted the 5,000 gas limit per block and set the default gas price to 51 gwei. This allowed for transactions – transactions require 21,000 gas. The difficulty bomb was introduced to ensure a future hard-fork to proof-of-stake. - 4001001 1499651592 2017-07-10 01:53:12 UTC +blockNumber timestamp date name component description +2717576 1480448309 2016-11-29 19:38:29 UTC stateclear execution The block of the state clearing soft fork. +200000 1441661589 2015-09-07 21:33:09 UTC iceage execution The frontier thawing fork lifted the 5,000 gas limit per block and set the default gas price to 51 gwei. This allowed for transactions – transactions require 21,000 gas. The difficulty bomb was introduced to ensure a future hard-fork to proof-of-stake. +4001001 1499651592 2017-07-10 01:53:12 UTC diff --git a/test/gold/tools/whenBlock/whenBlock_special_block_stateclear.txt b/test/gold/tools/whenBlock/whenBlock_special_block_stateclear.txt index 5c1b380e95..ddcbca4c35 100644 --- a/test/gold/tools/whenBlock/whenBlock_special_block_stateclear.txt +++ b/test/gold/tools/whenBlock/whenBlock_special_block_stateclear.txt @@ -3,5 +3,5 @@ TEST[DATE|TIME] Blocks: [stateclear] TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Caps: cache,decache TEST[DATE|TIME] Format: txt -component blockNumber timestamp date name description -execution 2717576 1480448309 2016-11-29 19:38:29 UTC stateclear The block of the state clearing soft fork. +blockNumber timestamp date name component description +2717576 1480448309 2016-11-29 19:38:29 UTC stateclear execution The block of the state clearing soft fork. diff --git a/test/gold/tools/whenBlock/whenBlock_special_block_tangerine.txt b/test/gold/tools/whenBlock/whenBlock_special_block_tangerine.txt index 2665254ef4..f6e7908c69 100644 --- a/test/gold/tools/whenBlock/whenBlock_special_block_tangerine.txt +++ b/test/gold/tools/whenBlock/whenBlock_special_block_tangerine.txt @@ -3,5 +3,5 @@ TEST[DATE|TIME] Blocks: [tangerine_whistle] TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Caps: cache,decache TEST[DATE|TIME] Format: txt -component blockNumber timestamp date name description -execution 2463000 1476796771 2016-10-18 13:19:31 UTC tangerine_whistle The Tangerine Whistle fork was the first response to the denial of service (DoS) attacks on the network (September/October 2016) including: addressing urgent network health issues concerning underpriced operation codes. +blockNumber timestamp date name component description +2463000 1476796771 2016-10-18 13:19:31 UTC tangerine_whistle execution The Tangerine Whistle fork was the first response to the denial of service (DoS) attacks on the network (September/October 2016) including: addressing urgent network health issues concerning underpriced operation codes. From f83167eafe5a3bb8e1047ecc8c1b7e018805a23a Mon Sep 17 00:00:00 2001 From: tjayrush Date: Sun, 3 Sep 2023 15:26:04 -0400 Subject: [PATCH 03/33] Takes possesion of startBlock, blockCnt, and ripeBlock into blazeManager to simplify source of truth. --- .../chifra/internal/scrape/handle_scrape.go | 82 +++++++++---------- .../chifra/internal/scrape/scrape_manager.go | 8 +- 2 files changed, 45 insertions(+), 45 deletions(-) diff --git a/src/apps/chifra/internal/scrape/handle_scrape.go b/src/apps/chifra/internal/scrape/handle_scrape.go index dd525e2e8d..83e6bd76e6 100644 --- a/src/apps/chifra/internal/scrape/handle_scrape.go +++ b/src/apps/chifra/internal/scrape/handle_scrape.go @@ -16,87 +16,84 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" ) -// HandleScrape enters a forever loop and continually scrapes either BlockCnt blocks -// or until the chain is caught up. It pauses for Sleep --sleep seconds between each scrape. +// HandleScrape enters a forever loop and continually scrapes --block_cnt blocks +// (or less if close to the head). The forever loop pauses each round for +// --sleep seconds (or, if not close to the head, for .25 seconds). func (opts *ScrapeOptions) HandleScrape() error { var err error chain := opts.Globals.Chain testMode := opts.Globals.TestMode - origBlockCnt := opts.BlockCnt - blazeMan := BlazeManager{ + bm := BlazeManager{ chain: chain, - timestamps: make([]tslib.TimestampRecord, 0, origBlockCnt), - processedMap: make(map[base.Blknum]bool, origBlockCnt), + timestamps: make([]tslib.TimestampRecord, 0, opts.BlockCnt), + processedMap: make(map[base.Blknum]bool, opts.BlockCnt), nProcessed: 0, opts: opts, } - blazeMan.meta, err = opts.Conn.GetMetaData(testMode) + bm.meta, err = opts.Conn.GetMetaData(testMode) if err != nil { return err } // Clean the temporary files and makes sure block zero has been processed - if ok, err := blazeMan.Prepare(); !ok || err != nil { + if ok, err := bm.Prepare(); !ok || err != nil { return err } - ripeBlock := base.Blknum(0) - unripePath := filepath.Join(config.GetPathToIndex(chain), "unripe") - - // The forever loop. Loop until the user hits Cntl+C or the server tells us to stop. runCount := uint64(0) + // The forever loop. Loop until the user hits Cntl+C, until runCount runs + // out, or until the server tells us to stop. for { - if blazeMan.meta, err = opts.Conn.GetMetaData(testMode); err != nil { + if bm.meta, err = opts.Conn.GetMetaData(testMode); err != nil { logger.Error(fmt.Sprintf("Error fetching meta data: %s. Sleeping...", err)) goto PAUSE } - if blazeMan.meta.NextIndexHeight() > blazeMan.meta.ChainHeight() { - // If the user is re-syncing the chain, the index may be ahead of the chain, - // so we go to sleep and try again later. + if bm.meta.NextIndexHeight() > bm.meta.ChainHeight() { + // The user may have restarted his node's sync (that is, started over). + // In this case, the index may be ahead of the chain, so we go to sleep + // and try again later. msg := fmt.Sprintf("The index (%d) is ahead of the chain (%d).", - blazeMan.meta.NextIndexHeight(), - blazeMan.meta.ChainHeight(), + bm.meta.NextIndexHeight(), + bm.meta.ChainHeight(), ) logger.Error(msg) goto PAUSE } - opts.StartBlock = blazeMan.meta.NextIndexHeight() - opts.BlockCnt = origBlockCnt - if (blazeMan.StartBlock() + blazeMan.BlockCount()) > blazeMan.meta.ChainHeight() { - opts.BlockCnt = (blazeMan.meta.Latest - blazeMan.StartBlock()) + bm = BlazeManager{ + chain: chain, + opts: opts, + nProcessed: 0, + timestamps: make([]tslib.TimestampRecord, 0, opts.BlockCnt), + processedMap: make(map[base.Blknum]bool, opts.BlockCnt), + meta: bm.meta, } - // The 'ripeBlock' is the head of the chain unless the chain is further along - // than 'UnripeDist.' If it is, the `ripeBlock` is 'UnripeDist' behind the - // head (i.e., 28 blocks usually - six minutes) - ripeBlock = blazeMan.meta.Latest - if ripeBlock > opts.Settings.Unripe_dist { - ripeBlock = blazeMan.meta.Latest - opts.Settings.Unripe_dist + // Adjust startBlock, blockCount, and ripeBlock for this round + bm.startBlock = bm.meta.NextIndexHeight() + bm.blockCount = opts.BlockCnt + if (bm.StartBlock() + bm.BlockCount()) > bm.meta.ChainHeight() { + bm.blockCount = (bm.meta.Latest - bm.StartBlock()) } - blazeMan = BlazeManager{ - chain: chain, - opts: opts, - nProcessed: 0, - ripeBlock: ripeBlock, - timestamps: make([]tslib.TimestampRecord, 0, origBlockCnt), - processedMap: make(map[base.Blknum]bool, origBlockCnt), - meta: blazeMan.meta, + // Keep ripeBlock at least Unripe_dist behind the head (i.e., 28 blocks usually - six minutes) + bm.ripeBlock = bm.meta.ChainHeight() + if bm.ripeBlock > opts.Settings.Unripe_dist { + bm.ripeBlock = bm.meta.ChainHeight() - opts.Settings.Unripe_dist } - // Here we do the actual scrape for this round. If anything goes wrong, the - // function will have cleaned up (i.e. remove the unstaged ripe blocks). Note - // that we don't quit, instead we sleep and we retry continually. - if err := blazeMan.ScrapeBatch(); err != nil { + // Here we do the actual scrape for this round. If anything goes wrong, the called + // function cleans up (i.e. remove the unstaged ripe blocks). Note that even on error, + // we don't quit. Instead we retry in the hopes that whatever happened corrects itself. + if err := bm.ScrapeBatch(); err != nil { logger.Error(colors.BrightRed, err, colors.Off) goto PAUSE } // Try to create chunks... - if ok, err := blazeMan.Consolidate(); !ok || err != nil { + if ok, err := bm.Consolidate(); !ok || err != nil { logger.Error(err) if !ok { break @@ -107,6 +104,7 @@ func (opts *ScrapeOptions) HandleScrape() error { PAUSE: // The chain frequently re-orgs. Before sleeping, we remove any unripe files so they // are re-queried in the next round. This is the reason for the unripePath. + unripePath := filepath.Join(config.GetPathToIndex(chain), "unripe") if err = os.RemoveAll(unripePath); err != nil { return err } @@ -116,7 +114,7 @@ func (opts *ScrapeOptions) HandleScrape() error { break } - blazeMan.pause() + bm.pause() } return nil diff --git a/src/apps/chifra/internal/scrape/scrape_manager.go b/src/apps/chifra/internal/scrape/scrape_manager.go index d217527a27..b783319d36 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager.go +++ b/src/apps/chifra/internal/scrape/scrape_manager.go @@ -19,22 +19,24 @@ import ( // if every block was visited or not. type BlazeManager struct { chain string - ripeBlock base.Blknum timestamps []tslib.TimestampRecord processedMap map[base.Blknum]bool nProcessed uint64 opts *ScrapeOptions meta *rpc.MetaData + startBlock base.Blknum + blockCount base.Blknum + ripeBlock base.Blknum } // StartBlock returns the start block for the current pass of the scraper. func (bm *BlazeManager) StartBlock() base.Blknum { - return bm.opts.StartBlock + return bm.startBlock } // BlockCount returns the number of blocks to process for this pass of the scraper. func (bm *BlazeManager) BlockCount() base.Blknum { - return bm.opts.BlockCnt + return bm.blockCount } // Report prints out a report of the progress of the scraper. From dca118619ea17590b609274e79a6b2401d29ad23 Mon Sep 17 00:00:00 2001 From: tjayrush Date: Sun, 3 Sep 2023 16:12:56 -0400 Subject: [PATCH 04/33] And he was pleased. --- .../chifra/internal/scrape/handle_scrape.go | 74 +++++++++++-------- .../chifra/internal/scrape/scrape_batch.go | 10 +-- 2 files changed, 46 insertions(+), 38 deletions(-) diff --git a/src/apps/chifra/internal/scrape/handle_scrape.go b/src/apps/chifra/internal/scrape/handle_scrape.go index 83e6bd76e6..e34db886b9 100644 --- a/src/apps/chifra/internal/scrape/handle_scrape.go +++ b/src/apps/chifra/internal/scrape/handle_scrape.go @@ -5,7 +5,6 @@ package scrapePkg // be found in the LICENSE file. import ( - "fmt" "os" "path/filepath" @@ -14,6 +13,8 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" + "github.com/pkg/errors" ) // HandleScrape enters a forever loop and continually scrapes --block_cnt blocks @@ -31,10 +32,10 @@ func (opts *ScrapeOptions) HandleScrape() error { nProcessed: 0, opts: opts, } - bm.meta, err = opts.Conn.GetMetaData(testMode) - if err != nil { - return err - } + // bm.meta, err = opts.Conn.GetMetaData(testMode) + // if err != nil { + // return err + // } // Clean the temporary files and makes sure block zero has been processed if ok, err := bm.Prepare(); !ok || err != nil { @@ -42,26 +43,36 @@ func (opts *ScrapeOptions) HandleScrape() error { } runCount := uint64(0) - // The forever loop. Loop until the user hits Cntl+C, until runCount runs - // out, or until the server tells us to stop. + // Loop until the user hits Cntl+C, until runCount runs out, or until + // the server tells us to stop. for { + // Fetch the meta data which tells us how far along the index is. if bm.meta, err = opts.Conn.GetMetaData(testMode); err != nil { - logger.Error(fmt.Sprintf("Error fetching meta data: %s. Sleeping...", err)) + var ErrFetchingMeta = errors.Errorf("error fetching meta data: %s.", err) + logger.Error(ErrFetchingMeta) + goto PAUSE + } + + // We're may be too close to the start of the chain to have ripe blocks. + // Report no error but try again soon. + if bm.meta.ChainHeight() < opts.Settings.Unripe_dist { goto PAUSE } + // The user may have restarted his node's sync (that is, started over). + // In this case, the index may be ahead of the chain, if so we go to + // sleep and try again later in the hopes that the chain catches up. if bm.meta.NextIndexHeight() > bm.meta.ChainHeight() { - // The user may have restarted his node's sync (that is, started over). - // In this case, the index may be ahead of the chain, so we go to sleep - // and try again later. - msg := fmt.Sprintf("The index (%d) is ahead of the chain (%d).", + var ErrIndexAhead = errors.Errorf( + "index (%d) is ahead of chain (%d)", bm.meta.NextIndexHeight(), bm.meta.ChainHeight(), ) - logger.Error(msg) + logger.Error(ErrIndexAhead) goto PAUSE } + // Let's start a new round... bm = BlazeManager{ chain: chain, opts: opts, @@ -71,30 +82,26 @@ func (opts *ScrapeOptions) HandleScrape() error { meta: bm.meta, } - // Adjust startBlock, blockCount, and ripeBlock for this round + // Order dependant, be careful! + // first block to scrape (one past end of previous round). bm.startBlock = bm.meta.NextIndexHeight() - bm.blockCount = opts.BlockCnt - if (bm.StartBlock() + bm.BlockCount()) > bm.meta.ChainHeight() { - bm.blockCount = (bm.meta.Latest - bm.StartBlock()) - } + // user supplied, but not so many to pass the chain tip. + bm.blockCount = utils.Min(opts.BlockCnt, bm.meta.ChainHeight()-bm.StartBlock()+1) + // Unripe_dist behind the chain tip. + bm.ripeBlock = bm.meta.ChainHeight() - opts.Settings.Unripe_dist - // Keep ripeBlock at least Unripe_dist behind the head (i.e., 28 blocks usually - six minutes) - bm.ripeBlock = bm.meta.ChainHeight() - if bm.ripeBlock > opts.Settings.Unripe_dist { - bm.ripeBlock = bm.meta.ChainHeight() - opts.Settings.Unripe_dist - } - - // Here we do the actual scrape for this round. If anything goes wrong, the called - // function cleans up (i.e. remove the unstaged ripe blocks). Note that even on error, - // we don't quit. Instead we retry in the hopes that whatever happened corrects itself. - if err := bm.ScrapeBatch(); err != nil { + // Scrape this round. Only quit on catostrophic errors. Report and sleep otherwise. + if err, ok := bm.ScrapeBatch(); !ok || err != nil { logger.Error(colors.BrightRed, err, colors.Off) + if !ok { + break + } goto PAUSE } - // Try to create chunks... + // Consilidate a chunk (if possible). Only quit on catostrophic errors. Report and sleep otherwise. if ok, err := bm.Consolidate(); !ok || err != nil { - logger.Error(err) + logger.Error(colors.BrightRed, err, colors.Off) if !ok { break } @@ -102,8 +109,9 @@ func (opts *ScrapeOptions) HandleScrape() error { } PAUSE: - // The chain frequently re-orgs. Before sleeping, we remove any unripe files so they - // are re-queried in the next round. This is the reason for the unripePath. + // If we've gotten this far, we want to clean up the unripe files (we no longer need them). + // The chain frequently re-orgs, so we want to re-qeury these next round. This is why + // we have an unripePath. unripePath := filepath.Join(config.GetPathToIndex(chain), "unripe") if err = os.RemoveAll(unripePath); err != nil { return err @@ -114,8 +122,10 @@ func (opts *ScrapeOptions) HandleScrape() error { break } + // sleep for a bit (there's no new blocks anyway if we're caught up). bm.pause() } + // We've left the loop and we're done. return nil } diff --git a/src/apps/chifra/internal/scrape/scrape_batch.go b/src/apps/chifra/internal/scrape/scrape_batch.go index 2cdfa5f31b..d8dc66028d 100644 --- a/src/apps/chifra/internal/scrape/scrape_batch.go +++ b/src/apps/chifra/internal/scrape/scrape_batch.go @@ -21,13 +21,13 @@ import ( // ScrapeBatch is called each time around the forever loop prior to calling into // Blaze to actually scrape the blocks. -func (bm *BlazeManager) ScrapeBatch() error { +func (bm *BlazeManager) ScrapeBatch() (error, bool) { chain := bm.chain // Do the actual scrape, wait until it finishes, clean up and return on failure if _, err := bm.HandleBlaze(); err != nil { _ = index.CleanTemporaryFolders(config.GetPathToIndex(chain), false) - return err + return err, true } start := bm.StartBlock() @@ -39,13 +39,11 @@ func (bm *BlazeManager) ScrapeBatch() error { // error, so clean up, report the error and return. The loop will repeat. _ = index.CleanTemporaryFolders(config.GetPathToIndex(chain), false) msg := fmt.Sprintf("A block %d was not processed%s", bn, strings.Repeat(" ", 50)) - return errors.New(msg) + return errors.New(msg), true } } - _ = bm.WriteTimestamps(end) - - return nil + return bm.WriteTimestamps(end), true } // TODO: Protect against overwriting files on disc From a9f2924af4925f5fed7500a318e22b8ab2a39b6d Mon Sep 17 00:00:00 2001 From: tjayrush Date: Sun, 3 Sep 2023 19:05:38 -0400 Subject: [PATCH 05/33] Further cleaning --- .../chifra/internal/monitors/handle_watch.go | 2 +- .../chifra/internal/scrape/handle_scrape.go | 38 +++++----- .../chifra/internal/scrape/save_timestamps.go | 74 +++++++++++++++++++ .../chifra/internal/scrape/scrape_batch.go | 65 +--------------- .../chifra/internal/scrape/scrape_blaze.go | 22 ++++-- .../internal/scrape/scrape_consolidate.go | 5 +- .../chifra/internal/scrape/scrape_manager.go | 39 ++++------ .../chifra/internal/scrape/scrape_prepare.go | 8 +- src/apps/chifra/pkg/rpc/client.go | 4 +- src/apps/chifra/pkg/rpc/get_meta.go | 5 ++ 10 files changed, 138 insertions(+), 124 deletions(-) create mode 100644 src/apps/chifra/internal/scrape/save_timestamps.go diff --git a/src/apps/chifra/internal/monitors/handle_watch.go b/src/apps/chifra/internal/monitors/handle_watch.go index ce2129049b..e0d2c3a079 100644 --- a/src/apps/chifra/internal/monitors/handle_watch.go +++ b/src/apps/chifra/internal/monitors/handle_watch.go @@ -68,7 +68,7 @@ func (opts *MonitorsOptions) RunMonitorScraper(wg *sync.WaitGroup, s *Scraper) { } runCount++ - if opts.RunCount == 0 || runCount >= opts.RunCount { + if opts.RunCount != 0 && runCount >= opts.RunCount { return } diff --git a/src/apps/chifra/internal/scrape/handle_scrape.go b/src/apps/chifra/internal/scrape/handle_scrape.go index e34db886b9..b1b54010cd 100644 --- a/src/apps/chifra/internal/scrape/handle_scrape.go +++ b/src/apps/chifra/internal/scrape/handle_scrape.go @@ -5,6 +5,7 @@ package scrapePkg // be found in the LICENSE file. import ( + "fmt" "os" "path/filepath" @@ -14,7 +15,6 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" - "github.com/pkg/errors" ) // HandleScrape enters a forever loop and continually scrapes --block_cnt blocks @@ -25,20 +25,8 @@ func (opts *ScrapeOptions) HandleScrape() error { chain := opts.Globals.Chain testMode := opts.Globals.TestMode - bm := BlazeManager{ - chain: chain, - timestamps: make([]tslib.TimestampRecord, 0, opts.BlockCnt), - processedMap: make(map[base.Blknum]bool, opts.BlockCnt), - nProcessed: 0, - opts: opts, - } - // bm.meta, err = opts.Conn.GetMetaData(testMode) - // if err != nil { - // return err - // } - // Clean the temporary files and makes sure block zero has been processed - if ok, err := bm.Prepare(); !ok || err != nil { + if ok, err := opts.Prepare(); !ok || err != nil { return err } @@ -46,24 +34,31 @@ func (opts *ScrapeOptions) HandleScrape() error { // Loop until the user hits Cntl+C, until runCount runs out, or until // the server tells us to stop. for { + // We create a new manager for each loop... + bm := BlazeManager{ + chain: chain, + } + // Fetch the meta data which tells us how far along the index is. if bm.meta, err = opts.Conn.GetMetaData(testMode); err != nil { - var ErrFetchingMeta = errors.Errorf("error fetching meta data: %s.", err) + var ErrFetchingMeta = fmt.Errorf("error fetching meta data: %s", err) logger.Error(ErrFetchingMeta) goto PAUSE } + // logger.Info(colors.Green+"meta data fetched"+colors.Off, bm.meta) // We're may be too close to the start of the chain to have ripe blocks. // Report no error but try again soon. if bm.meta.ChainHeight() < opts.Settings.Unripe_dist { goto PAUSE } + // logger.Info(colors.Green+"ripe block found"+colors.Off, bm.meta.ChainHeight()-opts.Settings.Unripe_dist) // The user may have restarted his node's sync (that is, started over). // In this case, the index may be ahead of the chain, if so we go to // sleep and try again later in the hopes that the chain catches up. if bm.meta.NextIndexHeight() > bm.meta.ChainHeight() { - var ErrIndexAhead = errors.Errorf( + var ErrIndexAhead = fmt.Errorf( "index (%d) is ahead of chain (%d)", bm.meta.NextIndexHeight(), bm.meta.ChainHeight(), @@ -71,6 +66,7 @@ func (opts *ScrapeOptions) HandleScrape() error { logger.Error(ErrIndexAhead) goto PAUSE } + // logger.Info(colors.Green+"index is not ahead of chain"+colors.Off, bm.meta.NextIndexHeight()) // Let's start a new round... bm = BlazeManager{ @@ -80,6 +76,7 @@ func (opts *ScrapeOptions) HandleScrape() error { timestamps: make([]tslib.TimestampRecord, 0, opts.BlockCnt), processedMap: make(map[base.Blknum]bool, opts.BlockCnt), meta: bm.meta, + nChannels: int(opts.Settings.Channel_count), } // Order dependant, be careful! @@ -89,6 +86,7 @@ func (opts *ScrapeOptions) HandleScrape() error { bm.blockCount = utils.Min(opts.BlockCnt, bm.meta.ChainHeight()-bm.StartBlock()+1) // Unripe_dist behind the chain tip. bm.ripeBlock = bm.meta.ChainHeight() - opts.Settings.Unripe_dist + // logger.Info(colors.Green, bm.startBlock, bm.blockCount, bm.ripeBlock) // Scrape this round. Only quit on catostrophic errors. Report and sleep otherwise. if err, ok := bm.ScrapeBatch(); !ok || err != nil { @@ -98,6 +96,7 @@ func (opts *ScrapeOptions) HandleScrape() error { } goto PAUSE } + // logger.Info(colors.Green+"scrape batch complete"+colors.Off, bm.nProcessed) // Consilidate a chunk (if possible). Only quit on catostrophic errors. Report and sleep otherwise. if ok, err := bm.Consolidate(); !ok || err != nil { @@ -107,6 +106,7 @@ func (opts *ScrapeOptions) HandleScrape() error { } goto PAUSE } + // logger.Info(colors.Green+"consolidate complete"+colors.Off, bm.nProcessed) PAUSE: // If we've gotten this far, we want to clean up the unripe files (we no longer need them). @@ -114,16 +114,18 @@ func (opts *ScrapeOptions) HandleScrape() error { // we have an unripePath. unripePath := filepath.Join(config.GetPathToIndex(chain), "unripe") if err = os.RemoveAll(unripePath); err != nil { + logger.Error(colors.BrightRed, err, colors.Off) return err } runCount++ - if opts.RunCount == 0 || runCount >= opts.RunCount { + if opts.RunCount != 0 && runCount >= opts.RunCount { + logger.Info("run count reached") break } // sleep for a bit (there's no new blocks anyway if we're caught up). - bm.pause() + opts.pause(bm.meta.ChainHeight() - bm.meta.StageHeight()) } // We've left the loop and we're done. diff --git a/src/apps/chifra/internal/scrape/save_timestamps.go b/src/apps/chifra/internal/scrape/save_timestamps.go new file mode 100644 index 0000000000..e7b8bd8804 --- /dev/null +++ b/src/apps/chifra/internal/scrape/save_timestamps.go @@ -0,0 +1,74 @@ +package scrapePkg + +// Copyright 2021 The TrueBlocks Authors. All rights reserved. +// Use of this source code is governed by a license that can +// be found in the LICENSE file. + +import ( + "encoding/binary" + "fmt" + "os" + "sort" + + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/rpc" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" +) + +// TODO: Protect against overwriting files on disc + +func (bm *BlazeManager) SaveTimestamps(endPoint uint64) error { + chain := bm.chain + conn := rpc.TempConnection(chain) + + sort.Slice(bm.timestamps, func(i, j int) bool { + return bm.timestamps[i].Bn < bm.timestamps[j].Bn + }) + + // Assume that the existing timestamps file always contains valid timestamps in a valid order so we can only append + tsPath := config.GetPathToIndex(chain) + "ts.bin" + fp, err := os.OpenFile(tsPath, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0644) + if err != nil { + return err + } + + defer func() { + tslib.DeCache(chain) + fp.Close() + // sigintTrap.Disable(trapCh) + // writeMutex.Unlock() + }() + + nTs, _ := tslib.NTimestamps(chain) + + cnt := 0 + for bn := nTs; bn < endPoint; bn++ { + // Append to the timestamps file all the new timestamps but as we do that make sure we're + // not skipping anything at the front, in the middle, or at the end of the list + ts := tslib.TimestampRecord{} + if cnt >= len(bm.timestamps) { + ts = tslib.TimestampRecord{ + Bn: uint32(bn), + Ts: uint32(conn.GetBlockTimestamp(bn)), + } + } else { + ts = bm.timestamps[cnt] + if bm.timestamps[cnt].Bn != uint32(bn) { + ts = tslib.TimestampRecord{ + Bn: uint32(bn), + Ts: uint32(conn.GetBlockTimestamp(bn)), + } + cnt-- // set it back + } + } + + logger.Progress((bn%13) == 0, fmt.Sprintf("Checking or updating timestamps %-04d of %-04d (%d remaining)%s", bn, endPoint, endPoint-bn, spaces)) + if err = binary.Write(fp, binary.LittleEndian, &ts); err != nil { + return err + } + + cnt++ + } + return nil +} diff --git a/src/apps/chifra/internal/scrape/scrape_batch.go b/src/apps/chifra/internal/scrape/scrape_batch.go index d8dc66028d..4c21037136 100644 --- a/src/apps/chifra/internal/scrape/scrape_batch.go +++ b/src/apps/chifra/internal/scrape/scrape_batch.go @@ -5,18 +5,12 @@ package scrapePkg // be found in the LICENSE file. import ( - "encoding/binary" "errors" "fmt" - "os" - "sort" "strings" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/index" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/rpc" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" ) // ScrapeBatch is called each time around the forever loop prior to calling into @@ -43,62 +37,5 @@ func (bm *BlazeManager) ScrapeBatch() (error, bool) { } } - return bm.WriteTimestamps(end), true -} - -// TODO: Protect against overwriting files on disc - -func (bm *BlazeManager) WriteTimestamps(endPoint uint64) error { - chain := bm.chain - conn := rpc.TempConnection(chain) - - sort.Slice(bm.timestamps, func(i, j int) bool { - return bm.timestamps[i].Bn < bm.timestamps[j].Bn - }) - - // Assume that the existing timestamps file always contains valid timestamps in a valid order so we can only append - tsPath := config.GetPathToIndex(chain) + "ts.bin" - fp, err := os.OpenFile(tsPath, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0644) - if err != nil { - return err - } - - defer func() { - tslib.DeCache(chain) - fp.Close() - // sigintTrap.Disable(trapCh) - // writeMutex.Unlock() - }() - - nTs, _ := tslib.NTimestamps(chain) - - cnt := 0 - for bn := nTs; bn < endPoint; bn++ { - // Append to the timestamps file all the new timestamps but as we do that make sure we're - // not skipping anything at the front, in the middle, or at the end of the list - ts := tslib.TimestampRecord{} - if cnt >= len(bm.timestamps) { - ts = tslib.TimestampRecord{ - Bn: uint32(bn), - Ts: uint32(conn.GetBlockTimestamp(bn)), - } - } else { - ts = bm.timestamps[cnt] - if bm.timestamps[cnt].Bn != uint32(bn) { - ts = tslib.TimestampRecord{ - Bn: uint32(bn), - Ts: uint32(conn.GetBlockTimestamp(bn)), - } - cnt-- // set it back - } - } - - logger.Progress((bn%13) == 0, fmt.Sprintf("Checking or updating timestamps %-04d of %-04d (%d remaining)%s", bn, endPoint, endPoint-bn, spaces)) - if err = binary.Write(fp, binary.LittleEndian, &ts); err != nil { - return err - } - - cnt++ - } - return nil + return bm.SaveTimestamps(end), true } diff --git a/src/apps/chifra/internal/scrape/scrape_blaze.go b/src/apps/chifra/internal/scrape/scrape_blaze.go index 8554539c77..950f0fecb2 100644 --- a/src/apps/chifra/internal/scrape/scrape_blaze.go +++ b/src/apps/chifra/internal/scrape/scrape_blaze.go @@ -14,13 +14,13 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/rpc" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/types" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" ) // HandleBlaze does the actual scraping, walking through block_cnt blocks and querying traces and logs // and then extracting addresses and timestamps from those data structures. func (bm *BlazeManager) HandleBlaze() (ok bool, err error) { - nChannels := int(bm.opts.Settings.Channel_count) blocks := []base.Blknum{} start := bm.StartBlock() @@ -37,24 +37,24 @@ func (bm *BlazeManager) HandleBlaze() (ok bool, err error) { // TODO: The go routines below may fail. Question -- how does one respond to an error inside a go routine? blockWg := sync.WaitGroup{} - blockWg.Add(nChannels) - for i := 0; i < nChannels; i++ { + blockWg.Add(bm.nChannels) + for i := 0; i < bm.nChannels; i++ { go func() { _ = bm.ProcessBlocks(blockChannel, &blockWg, appearanceChannel, tsChannel) }() } appWg := sync.WaitGroup{} - appWg.Add(nChannels) - for i := 0; i < nChannels; i++ { + appWg.Add(bm.nChannels) + for i := 0; i < bm.nChannels; i++ { go func() { _ = bm.ProcessAppearances(appearanceChannel, &appWg) }() } tsWg := sync.WaitGroup{} - tsWg.Add(nChannels) - for i := 0; i < nChannels; i++ { + tsWg.Add(bm.nChannels) + for i := 0; i < bm.nChannels; i++ { go func() { _ = bm.ProcessTimestamps(tsChannel, &tsWg) }() @@ -216,3 +216,11 @@ func (bm *BlazeManager) syncedReporting(bn base.Blknum, force bool) { logger.Progress(true, msg) } } + +// scrapedData combines the extracted block data, trace data, and log data into a +// structure that is passed through to the AddressChannel for further processing. +type scrapedData struct { + bn base.Blknum + traces []types.SimpleTrace + receipts []types.SimpleReceipt +} diff --git a/src/apps/chifra/internal/scrape/scrape_consolidate.go b/src/apps/chifra/internal/scrape/scrape_consolidate.go index ba9d0ed3ad..a0dd4f941e 100644 --- a/src/apps/chifra/internal/scrape/scrape_consolidate.go +++ b/src/apps/chifra/internal/scrape/scrape_consolidate.go @@ -162,7 +162,7 @@ func (bm *BlazeManager) Consolidate() (bool, error) { stageFn, _ = file.LatestFileInFolder(stageFolder) // it may not exist... nAppsNow := int(file.FileSize(stageFn) / asciiAppearanceSize) - bm.report(nAppsThen, nAppsNow) + bm.report(int(bm.opts.Settings.Apps_per_chunk), nAppsThen, nAppsNow) os.Remove(backupFn) // commits the change @@ -175,8 +175,7 @@ func isListSequential(chain string, ripeFileList []os.DirEntry, allowMissing boo fileRange := base.RangeFromFilename(file.Name()) if prev != base.NotARange && prev != fileRange { if !prev.Preceeds(fileRange, !allowMissing) { - msg := fmt.Sprintf("Ripe files are not sequential (%s ==> %s)", prev, fileRange) - return errors.New(msg) + return fmt.Errorf("ripe files are not sequential (%s ==> %s)", prev, fileRange) } } prev = fileRange diff --git a/src/apps/chifra/internal/scrape/scrape_manager.go b/src/apps/chifra/internal/scrape/scrape_manager.go index b783319d36..92f1885c45 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager.go +++ b/src/apps/chifra/internal/scrape/scrape_manager.go @@ -10,7 +10,6 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/rpc" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/types" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" ) @@ -27,6 +26,7 @@ type BlazeManager struct { startBlock base.Blknum blockCount base.Blknum ripeBlock base.Blknum + nChannels int } // StartBlock returns the start block for the current pass of the scraper. @@ -40,46 +40,35 @@ func (bm *BlazeManager) BlockCount() base.Blknum { } // Report prints out a report of the progress of the scraper. -func (bm *BlazeManager) report(nAppsThen, nAppsNow int) { - settings := bm.opts.Settings - - msg := "Block={%d} have {%d} appearances of {%d} ({%0.1f%%}). Need {%d} more. Added {%d} records ({%0.2f} apps/blk)." - need := settings.Apps_per_chunk - utils.Min(settings.Apps_per_chunk, uint64(nAppsNow)) +func (bm *BlazeManager) report(perChunk, nAppsThen, nAppsNow int) { + need := perChunk - utils.Min(perChunk, nAppsNow) seen := nAppsNow if nAppsThen < nAppsNow { seen = nAppsNow - nAppsThen } - pct := float64(nAppsNow) / float64(settings.Apps_per_chunk) + pct := float64(nAppsNow) / float64(perChunk) pBlk := float64(seen) / float64(bm.BlockCount()) height := bm.StartBlock() + bm.BlockCount() - 1 - msg = strings.Replace(msg, "{", colors.Green, -1) + + const templ = `Block={%d} have {%d} appearances of {%d} ({%0.1f%%}). Need {%d} more. Added {%d} records ({%0.2f} apps/blk).` + msg := strings.Replace(templ, "{", colors.Green, -1) msg = strings.Replace(msg, "}", colors.Off, -1) - logger.Info(fmt.Sprintf(msg, height, nAppsNow, settings.Apps_per_chunk, pct*100, need, seen, pBlk)) + logger.Info(fmt.Sprintf(msg, height, nAppsNow, perChunk, pct*100, need, seen, pBlk)) } // Pause goes to sleep for a period of time based on the settings. -func (bm *BlazeManager) pause() { +func (opts *ScrapeOptions) pause(dist uint64) { // we always pause at least a quarter of a second to allow the node to 'rest' time.Sleep(250 * time.Millisecond) - isDefaultSleep := bm.opts.Sleep >= 13 && bm.opts.Sleep <= 14 - distanceFromHead := bm.meta.Latest - bm.meta.Staging - shouldSleep := !isDefaultSleep || distanceFromHead <= (2*bm.opts.Settings.Unripe_dist) + isDefaultSleep := opts.Sleep >= 13 && opts.Sleep <= 14 + // distanceFromHead := meta.ChainHeight() - meta.Staging + shouldSleep := !isDefaultSleep || dist <= (2*opts.Settings.Unripe_dist) if shouldSleep { - sleep := bm.opts.Sleep // this value may change elsewhere allow us to break out of sleeping???? - if sleep > 1 { - logger.Info("Sleeping for", sleep, "seconds -", distanceFromHead, "away from head.") - } + sleep := opts.Sleep // this value may change elsewhere allow us to break out of sleeping???? + logger.Progress(sleep > 1, "Sleeping for", sleep, "seconds -", dist, "away from head.") halfSecs := (sleep * 2) - 1 // we already slept one quarter of a second for i := 0; i < int(halfSecs); i++ { time.Sleep(time.Duration(500) * time.Millisecond) } } } - -// scrapedData combines the extracted block data, trace data, and log data into a -// structure that is passed through to the AddressChannel for further processing. -type scrapedData struct { - bn base.Blknum - traces []types.SimpleTrace - receipts []types.SimpleReceipt -} diff --git a/src/apps/chifra/internal/scrape/scrape_prepare.go b/src/apps/chifra/internal/scrape/scrape_prepare.go index b4d62f3fb9..969b1378ee 100644 --- a/src/apps/chifra/internal/scrape/scrape_prepare.go +++ b/src/apps/chifra/internal/scrape/scrape_prepare.go @@ -19,8 +19,8 @@ import ( // forever loop. Returns true if processing should continue, false otherwise. // The routine cleans the temporary folders (if any) and then makes sure the zero // block (reads the allocation file, if present) is processed. -func (bm *BlazeManager) Prepare() (ok bool, err error) { - chain := bm.chain +func (opts *ScrapeOptions) Prepare() (ok bool, err error) { + chain := opts.Globals.Chain // We always clean the temporary folders (other than staging) when starting _ = index.CleanTemporaryFolders(config.GetPathToIndex(chain), false) @@ -50,13 +50,13 @@ func (bm *BlazeManager) Prepare() (ok bool, err error) { array := []tslib.TimestampRecord{} array = append(array, tslib.TimestampRecord{ Bn: uint32(0), - Ts: uint32(bm.opts.Conn.GetBlockTimestamp(0)), + Ts: uint32(opts.Conn.GetBlockTimestamp(0)), }) _ = tslib.Append(chain, array) logger.Info("Writing block zero allocations for", len(prefunds), "prefunds, nAddresses:", len(appMap)) indexPath := index.ToIndexPath(bloomPath) - if report, err := index.WriteChunk(chain, indexPath, appMap, len(prefunds), bm.opts.Pin, bm.opts.Remote); err != nil { + if report, err := index.WriteChunk(chain, indexPath, appMap, len(prefunds), opts.Pin, opts.Remote); err != nil { return false, err } else if report == nil { logger.Fatal("Should not happen, write chunk returned empty report") diff --git a/src/apps/chifra/pkg/rpc/client.go b/src/apps/chifra/pkg/rpc/client.go index 594acc200e..3651989e5b 100644 --- a/src/apps/chifra/pkg/rpc/client.go +++ b/src/apps/chifra/pkg/rpc/client.go @@ -53,14 +53,14 @@ func (conn *Connection) GetClientIDs() (uint64, uint64, error) { // GetLatestBlockNumber returns the block number at the front of the chain (i.e. latest) func (conn *Connection) GetLatestBlockNumber() uint64 { if ec, err := conn.getClient(); err != nil { - logger.Error("Could not connect to RPC client: %w", err) + logger.Error("Could not connect to RPC client", err) return 0 } else { defer ec.Close() r, err := ec.BlockNumber(context.Background()) if err != nil { - logger.Error("Could not connect to RPC client: %w", err) + logger.Error("Could not connect to RPC client", err) return 0 } diff --git a/src/apps/chifra/pkg/rpc/get_meta.go b/src/apps/chifra/pkg/rpc/get_meta.go index 03b4861a19..69df0123a3 100644 --- a/src/apps/chifra/pkg/rpc/get_meta.go +++ b/src/apps/chifra/pkg/rpc/get_meta.go @@ -105,3 +105,8 @@ func (m *MetaData) NextIndexHeight() base.Blknum { func (m *MetaData) ChainHeight() base.Blknum { return m.Latest } + +// StageHieght returns the highest block that's been staged +func (m *MetaData) StageHeight() base.Blknum { + return m.Staging +} From a23a1e9f9dc47e62eb804b210cfe463e53e54557 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Mon, 4 Sep 2023 00:32:57 -0400 Subject: [PATCH 06/33] Working version --- src/apps/chifra/internal/scrape/save_timestamps.go | 2 +- src/apps/chifra/pkg/tslib/append.go | 2 +- src/apps/chifra/pkg/tslib/repair.go | 2 +- src/apps/chifra/pkg/tslib/truncate.go | 2 +- src/apps/chifra/pkg/tslib/tsdb.go | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/apps/chifra/internal/scrape/save_timestamps.go b/src/apps/chifra/internal/scrape/save_timestamps.go index e7b8bd8804..bd6c6c5e04 100644 --- a/src/apps/chifra/internal/scrape/save_timestamps.go +++ b/src/apps/chifra/internal/scrape/save_timestamps.go @@ -34,7 +34,7 @@ func (bm *BlazeManager) SaveTimestamps(endPoint uint64) error { } defer func() { - tslib.DeCache(chain) + tslib.ClearCache(chain) fp.Close() // sigintTrap.Disable(trapCh) // writeMutex.Unlock() diff --git a/src/apps/chifra/pkg/tslib/append.go b/src/apps/chifra/pkg/tslib/append.go index 86239aa18d..da8325ad34 100644 --- a/src/apps/chifra/pkg/tslib/append.go +++ b/src/apps/chifra/pkg/tslib/append.go @@ -13,7 +13,7 @@ func Append(chain string, tsArray []TimestampRecord) error { tsFn := config.GetPathToIndex(chain) + "ts.bin" tmpPath := filepath.Join(config.GetPathToCache(chain), "tmp") if backupFn, err := file.MakeBackup(tmpPath, tsFn); err == nil { - DeCache(chain) + ClearCache(chain) defer func() { if file.FileExists(backupFn) { _ = os.Rename(backupFn, tsFn) diff --git a/src/apps/chifra/pkg/tslib/repair.go b/src/apps/chifra/pkg/tslib/repair.go index 713099c9d0..ae2fbe841c 100644 --- a/src/apps/chifra/pkg/tslib/repair.go +++ b/src/apps/chifra/pkg/tslib/repair.go @@ -29,7 +29,7 @@ func Repair(chain string, bn uint64) error { tmpPath := filepath.Join(config.GetPathToCache(chain), "tmp") if backupFn, err := file.MakeBackup(tmpPath, tsFn); err == nil { defer func() { - DeCache(chain) + ClearCache(chain) if file.FileExists(backupFn) { // If the backup file exists, something failed, so we replace the original file. _ = os.Rename(backupFn, tsFn) diff --git a/src/apps/chifra/pkg/tslib/truncate.go b/src/apps/chifra/pkg/tslib/truncate.go index 39f329f3c1..b136fa7ee2 100644 --- a/src/apps/chifra/pkg/tslib/truncate.go +++ b/src/apps/chifra/pkg/tslib/truncate.go @@ -34,7 +34,7 @@ func Truncate(chain string, maxBn uint64) error { tmpPath := filepath.Join(config.GetPathToCache(chain), "tmp") if backupFn, err := file.MakeBackup(tmpPath, tsFn); err == nil { defer func() { - DeCache(chain) + ClearCache(chain) if file.FileExists(backupFn) { // If the backup file exists, something failed, so we replace the original file. _ = os.Rename(backupFn, tsFn) diff --git a/src/apps/chifra/pkg/tslib/tsdb.go b/src/apps/chifra/pkg/tslib/tsdb.go index 516432e138..d9e83a0fdc 100644 --- a/src/apps/chifra/pkg/tslib/tsdb.go +++ b/src/apps/chifra/pkg/tslib/tsdb.go @@ -121,7 +121,7 @@ func FromTs(chain string, ts base.Timestamp) (*TimestampRecord, error) { return &perChainTimestamps[chain].memory[index], nil } -func DeCache(chain string) { +func ClearCache(chain string) { perChainTimestamps[chain] = TimestampDatabase{ loaded: false, count: 0, From bfde41c09d9abd1567e6091320d5af657af2be8d Mon Sep 17 00:00:00 2001 From: tjayrush Date: Mon, 4 Sep 2023 02:22:47 -0400 Subject: [PATCH 07/33] This likely works. Needs testing. Best version yet. --- .../chifra/internal/scrape/handle_scrape.go | 23 +- .../chifra/internal/scrape/save_timestamps.go | 33 +-- .../chifra/internal/scrape/scrape_batch.go | 38 ++-- .../chifra/internal/scrape/scrape_blaze.go | 11 +- .../internal/scrape/scrape_consolidate.go | 197 +++++++++--------- .../chifra/internal/scrape/scrape_manager.go | 5 + 6 files changed, 158 insertions(+), 149 deletions(-) diff --git a/src/apps/chifra/internal/scrape/handle_scrape.go b/src/apps/chifra/internal/scrape/handle_scrape.go index b1b54010cd..e33660f194 100644 --- a/src/apps/chifra/internal/scrape/handle_scrape.go +++ b/src/apps/chifra/internal/scrape/handle_scrape.go @@ -21,7 +21,10 @@ import ( // (or less if close to the head). The forever loop pauses each round for // --sleep seconds (or, if not close to the head, for .25 seconds). func (opts *ScrapeOptions) HandleScrape() error { + var blocks = make([]base.Blknum, 0, opts.BlockCnt) var err error + var ok bool + chain := opts.Globals.Chain testMode := opts.Globals.TestMode @@ -42,17 +45,15 @@ func (opts *ScrapeOptions) HandleScrape() error { // Fetch the meta data which tells us how far along the index is. if bm.meta, err = opts.Conn.GetMetaData(testMode); err != nil { var ErrFetchingMeta = fmt.Errorf("error fetching meta data: %s", err) - logger.Error(ErrFetchingMeta) + logger.Error(colors.BrightRed, ErrFetchingMeta, colors.Off) goto PAUSE } - // logger.Info(colors.Green+"meta data fetched"+colors.Off, bm.meta) // We're may be too close to the start of the chain to have ripe blocks. // Report no error but try again soon. if bm.meta.ChainHeight() < opts.Settings.Unripe_dist { goto PAUSE } - // logger.Info(colors.Green+"ripe block found"+colors.Off, bm.meta.ChainHeight()-opts.Settings.Unripe_dist) // The user may have restarted his node's sync (that is, started over). // In this case, the index may be ahead of the chain, if so we go to @@ -63,10 +64,9 @@ func (opts *ScrapeOptions) HandleScrape() error { bm.meta.NextIndexHeight(), bm.meta.ChainHeight(), ) - logger.Error(ErrIndexAhead) + logger.Error(colors.BrightRed, ErrIndexAhead, colors.Off) goto PAUSE } - // logger.Info(colors.Green+"index is not ahead of chain"+colors.Off, bm.meta.NextIndexHeight()) // Let's start a new round... bm = BlazeManager{ @@ -86,27 +86,30 @@ func (opts *ScrapeOptions) HandleScrape() error { bm.blockCount = utils.Min(opts.BlockCnt, bm.meta.ChainHeight()-bm.StartBlock()+1) // Unripe_dist behind the chain tip. bm.ripeBlock = bm.meta.ChainHeight() - opts.Settings.Unripe_dist - // logger.Info(colors.Green, bm.startBlock, bm.blockCount, bm.ripeBlock) + + // These are the blocks we're going to process this round + blocks = make([]base.Blknum, 0, bm.BlockCount()) + for block := bm.StartBlock(); block < bm.EndBlock(); block++ { + blocks = append(blocks, block) + } // Scrape this round. Only quit on catostrophic errors. Report and sleep otherwise. - if err, ok := bm.ScrapeBatch(); !ok || err != nil { + if err, ok = bm.ScrapeBatch(blocks); !ok || err != nil { logger.Error(colors.BrightRed, err, colors.Off) if !ok { break } goto PAUSE } - // logger.Info(colors.Green+"scrape batch complete"+colors.Off, bm.nProcessed) // Consilidate a chunk (if possible). Only quit on catostrophic errors. Report and sleep otherwise. - if ok, err := bm.Consolidate(); !ok || err != nil { + if err, ok = bm.Consolidate(blocks); !ok || err != nil { logger.Error(colors.BrightRed, err, colors.Off) if !ok { break } goto PAUSE } - // logger.Info(colors.Green+"consolidate complete"+colors.Off, bm.nProcessed) PAUSE: // If we've gotten this far, we want to clean up the unripe files (we no longer need them). diff --git a/src/apps/chifra/internal/scrape/save_timestamps.go b/src/apps/chifra/internal/scrape/save_timestamps.go index bd6c6c5e04..952e85ee70 100644 --- a/src/apps/chifra/internal/scrape/save_timestamps.go +++ b/src/apps/chifra/internal/scrape/save_timestamps.go @@ -10,23 +10,23 @@ import ( "os" "sort" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/rpc" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" ) // TODO: Protect against overwriting files on disc -func (bm *BlazeManager) SaveTimestamps(endPoint uint64) error { +func (bm *BlazeManager) WriteTimestamps(blocks []base.Blknum) error { chain := bm.chain - conn := rpc.TempConnection(chain) sort.Slice(bm.timestamps, func(i, j int) bool { return bm.timestamps[i].Bn < bm.timestamps[j].Bn }) - // Assume that the existing timestamps file always contains valid timestamps in a valid order so we can only append + // Assume that the existing timestamps file always contains valid timestamps in + // a valid order so we can only append as we go (which is very fast) tsPath := config.GetPathToIndex(chain) + "ts.bin" fp, err := os.OpenFile(tsPath, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0644) if err != nil { @@ -36,39 +36,42 @@ func (bm *BlazeManager) SaveTimestamps(endPoint uint64) error { defer func() { tslib.ClearCache(chain) fp.Close() - // sigintTrap.Disable(trapCh) - // writeMutex.Unlock() }() - nTs, _ := tslib.NTimestamps(chain) - cnt := 0 - for bn := nTs; bn < endPoint; bn++ { + for _, block := range blocks { // Append to the timestamps file all the new timestamps but as we do that make sure we're // not skipping anything at the front, in the middle, or at the end of the list ts := tslib.TimestampRecord{} if cnt >= len(bm.timestamps) { ts = tslib.TimestampRecord{ - Bn: uint32(bn), - Ts: uint32(conn.GetBlockTimestamp(bn)), + Bn: uint32(block), + Ts: uint32(bm.opts.Conn.GetBlockTimestamp(block)), } } else { ts = bm.timestamps[cnt] - if bm.timestamps[cnt].Bn != uint32(bn) { + if bm.timestamps[cnt].Bn != uint32(block) { ts = tslib.TimestampRecord{ - Bn: uint32(bn), - Ts: uint32(conn.GetBlockTimestamp(bn)), + Bn: uint32(block), + Ts: uint32(bm.opts.Conn.GetBlockTimestamp(block)), } cnt-- // set it back } } - logger.Progress((bn%13) == 0, fmt.Sprintf("Checking or updating timestamps %-04d of %-04d (%d remaining)%s", bn, endPoint, endPoint-bn, spaces)) + msg := fmt.Sprintf("Updating timestamps %-04d of %-04d (%-04d remaining)", + block, + blocks[len(blocks)-1], + blocks[len(blocks)-1]-block, + ) + logger.Progress((block%13) == 0, msg) + if err = binary.Write(fp, binary.LittleEndian, &ts); err != nil { return err } cnt++ } + return nil } diff --git a/src/apps/chifra/internal/scrape/scrape_batch.go b/src/apps/chifra/internal/scrape/scrape_batch.go index 4c21037136..1859f3e137 100644 --- a/src/apps/chifra/internal/scrape/scrape_batch.go +++ b/src/apps/chifra/internal/scrape/scrape_batch.go @@ -5,37 +5,33 @@ package scrapePkg // be found in the LICENSE file. import ( - "errors" "fmt" - "strings" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/index" ) -// ScrapeBatch is called each time around the forever loop prior to calling into -// Blaze to actually scrape the blocks. -func (bm *BlazeManager) ScrapeBatch() (error, bool) { - chain := bm.chain +// ScrapeBatch is called each time around the forever loop. It calls into +// HandleBlaze and writes the timestamps if there's no error. +func (bm *BlazeManager) ScrapeBatch(blocks []base.Blknum) (error, bool) { + indexPath := config.GetPathToIndex(bm.chain) - // Do the actual scrape, wait until it finishes, clean up and return on failure - if _, err := bm.HandleBlaze(); err != nil { - _ = index.CleanTemporaryFolders(config.GetPathToIndex(chain), false) - return err, true + if err, ok := bm.HandleBlaze(blocks); !ok || err != nil { + _ = index.CleanTemporaryFolders(indexPath, false) + return err, ok } - start := bm.StartBlock() - end := bm.StartBlock() + bm.BlockCount() - - for bn := start; bn < end; bn++ { - if !bm.processedMap[bn] { - // At least one block was not processed. This would only happen in the event of an - // error, so clean up, report the error and return. The loop will repeat. - _ = index.CleanTemporaryFolders(config.GetPathToIndex(chain), false) - msg := fmt.Sprintf("A block %d was not processed%s", bn, strings.Repeat(" ", 50)) - return errors.New(msg), true + // Check to see if we missed any blocks... + for _, block := range blocks { + if !bm.processedMap[block] { + // We missed a block. We need to clean up and continue + // next time around the loop. This may happen if the + // node returns an error for example. + _ = index.CleanTemporaryFolders(indexPath, false) + return fmt.Errorf("a block (%d) was not processed", block), true } } - return bm.SaveTimestamps(end), true + return bm.WriteTimestamps(blocks), true } diff --git a/src/apps/chifra/internal/scrape/scrape_blaze.go b/src/apps/chifra/internal/scrape/scrape_blaze.go index 950f0fecb2..c02d5b7272 100644 --- a/src/apps/chifra/internal/scrape/scrape_blaze.go +++ b/src/apps/chifra/internal/scrape/scrape_blaze.go @@ -20,14 +20,7 @@ import ( // HandleBlaze does the actual scraping, walking through block_cnt blocks and querying traces and logs // and then extracting addresses and timestamps from those data structures. -func (bm *BlazeManager) HandleBlaze() (ok bool, err error) { - blocks := []base.Blknum{} - - start := bm.StartBlock() - end := bm.StartBlock() + bm.BlockCount() - for block := start; block < end; block++ { - blocks = append(blocks, block) - } +func (bm *BlazeManager) HandleBlaze(blocks []base.Blknum) (err error, ok bool) { // We need three pipelines...we shove into blocks, blocks shoves into appearances and timestamps blockChannel := make(chan base.Blknum) @@ -75,7 +68,7 @@ func (bm *BlazeManager) HandleBlaze() (ok bool, err error) { close(tsChannel) tsWg.Wait() - return true, nil + return nil, true } // ProcessBlocks processes the block channel and for each block query the node for both diff --git a/src/apps/chifra/internal/scrape/scrape_consolidate.go b/src/apps/chifra/internal/scrape/scrape_consolidate.go index a0dd4f941e..08f3dd53e0 100644 --- a/src/apps/chifra/internal/scrape/scrape_consolidate.go +++ b/src/apps/chifra/internal/scrape/scrape_consolidate.go @@ -5,7 +5,6 @@ import ( "fmt" "os" "path/filepath" - "strconv" "strings" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" @@ -13,118 +12,129 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/file" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/index" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/rpc" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" ) const asciiAppearanceSize = 59 // Consolidate calls into the block scraper to (a) call Blaze and (b) consolidate if applicable -func (bm *BlazeManager) Consolidate() (bool, error) { +func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { chain := bm.chain + settings := bm.opts.Settings + testMode := bm.opts.Globals.TestMode - // Get a sorted list of files in the ripe folder ripeFolder := filepath.Join(config.GetPathToIndex(chain), "ripe") - ripeFileList, err := os.ReadDir(ripeFolder) + stageFolder := filepath.Join(config.GetPathToIndex(chain), "staging") + + ripeFiles, err := os.ReadDir(ripeFolder) if err != nil { - return true, err + return err, true } - stageFolder := filepath.Join(config.GetPathToIndex(chain), "staging") - if len(ripeFileList) == 0 { - // On active chains, this most likely never happens, but on some less used or private chains, this is a frequent occurrence. - // return a message, but don't do anything about it. - msg := fmt.Sprintf("No new blocks at block %d (%d away from head)%s", bm.meta.Latest, (bm.meta.Latest - bm.meta.Ripe), spaces) - logger.Info(msg) - - // we need to move the file to the end of the scraped range so we show progress - stageFn, _ := file.LatestFileInFolder(stageFolder) // it may not exist... - stageRange := base.RangeFromFilename(stageFn) - end := bm.StartBlock() + bm.BlockCount() - newRangeLast := utils.Min(bm.ripeBlock, end-1) - if stageRange.Last < newRangeLast { - newRange := base.FileRange{First: stageRange.First, Last: newRangeLast} - newFilename := filepath.Join(stageFolder, newRange.String()+".txt") - _ = os.Rename(stageFn, newFilename) - _ = os.Remove(stageFn) // seems redundant, but may not be on some operating systems + if len(ripeFiles) == 0 { + // There are no new ripeFiles, but we've processed all blocks in the block + // list. We want to rename the stage file (if it exists) to reflect the + // new stage head. If the file doesn't exist, just touch it. The scraper + // uses the stage file's name to know where to start the next loop. + curStage, _ := file.LatestFileInFolder(stageFolder) // it may not exist... + curRange := base.RangeFromFilename(curStage) + + // All blocks in the list have been processed, so we know where the stage ends. + newRange := base.FileRange{First: curRange.First, Last: blocks[len(blocks)-1]} + newStage := filepath.Join(stageFolder, newRange.String()+".txt") + + if file.FileExists(curStage) { + _ = os.Rename(curStage, newStage) + _ = os.Remove(curStage) // seems redundant, but may not be on some operating systems + } else { + file.Touch(newStage) } - return true, nil + + // Let the user know something happened... + return fmt.Errorf("no new ripe blocks found at %d (%d away from head)", + bm.meta.ChainHeight(), + bm.meta.ChainHeight() - bm.meta.Ripe, + ), true } - // Check to see if we got as many ripe files as we were expecting. In the case when AllowMissing is true, we - // can't really know, but if AllowMissing is false, then the number of files should be the same as the range width - ripeCnt := len(ripeFileList) - unripeDist := bm.opts.Settings.Unripe_dist - if uint64(ripeCnt) < (bm.BlockCount() - unripeDist) { - // Then, if they are not at least sequential, clean up and try again... - allowMissing := bm.opts.Settings.Allow_missing - if err := isListSequential(chain, ripeFileList, allowMissing); err != nil { + // If the number of ripe files is less than we expected... + if uint64(len(ripeFiles)) < (bm.BlockCount() - settings.Unripe_dist) { + // ...at least make sure they're sequential (if the chain so desires). + if err := bm.isListSequential(chain, ripeFiles); err != nil { _ = index.CleanTemporaryFolders(config.GetPathToCache(chain), false) - return true, err + return err, true } } stageFn, _ := file.LatestFileInFolder(stageFolder) // it may not exist... - nAppsThen := int(file.FileSize(stageFn) / asciiAppearanceSize) - - // ripeRange := rangeFromFileList(ripeFileList) stageRange := base.RangeFromFilename(stageFn) + nAppsThen := int(file.FileSize(stageFn) / asciiAppearanceSize) - curRange := base.FileRange{First: bm.StartBlock(), Last: bm.StartBlock() + bm.BlockCount() - 1} + var curRange base.FileRange if file.FileExists(stageFn) { curRange = stageRange + } else { + curRange = base.FileRange{First: blocks[0], Last: blocks[len(blocks)-1]} } - // Note, this file may be empty or non-existant - tmpPath := filepath.Join(config.GetPathToCache(chain) + "tmp") - backupFn, err := file.MakeBackup(tmpPath, stageFn) + // Note, the stage may be empty or non-existant + backupFn, err := file.MakeBackup(filepath.Join(config.GetPathToCache(chain)+"tmp"), stageFn) if err != nil { - return true, errors.New("Could not create backup file: " + err.Error()) + return errors.New("Could not create backup file: " + err.Error()), true } - - // logger.Info("Created backup file for stage") defer func() { + // If the backup file exists, the function did not complete. In that case, we replace the original file. if backupFn != "" && file.FileExists(backupFn) { - // If the backup file exists, something failed, so we replace the original file. - // logger.Info("Replacing backed up staging file") _ = os.Rename(backupFn, stageFn) _ = os.Remove(backupFn) // seems redundant, but may not be on some operating systems } }() - appearances := file.AsciiFileToLines(stageFn) - os.Remove(stageFn) // we have a backup copy, so it's not so bad to delete it here - for _, ripeFile := range ripeFileList { - ripePath := filepath.Join(ripeFolder, ripeFile.Name()) - appearances = append(appearances, file.AsciiFileToLines(ripePath)...) - os.Remove(ripePath) // if we fail halfway through, this will get noticed next time around and cleaned up - curCount := uint64(len(appearances)) + // Note: + // if we fail at any point past this, the backup file will replace the original file - ripeRange := base.RangeFromFilename(ripePath) + // Start by copying in the existing appearances on the stage... + appearances := file.AsciiFileToLines(stageFn) + os.Remove(stageFn) // We have a backup copy, so it's okay to remove this here... + + // Spin through each ripe file and add it to the appearances array until we have either (a) a snap + // point or (b) enough appearances to overtop. In either case, we consolidate the appearances into + // a chunk, create its bloom, and write it to disk. Note that each time we copy one of the ripe + // files, we remove it. + for _, ripeFile := range ripeFiles { + thisFile := filepath.Join(ripeFolder, ripeFile.Name()) + ripeRange := base.RangeFromFilename(thisFile) curRange.Last = ripeRange.Last - isSnap := (curRange.Last >= bm.opts.Settings.First_snap && (curRange.Last%bm.opts.Settings.Snap_to_grid) == 0) - isOvertop := (curCount >= uint64(bm.opts.Settings.Apps_per_chunk)) + // append the appearances from this file + appearances = append(appearances, file.AsciiFileToLines(thisFile)...) + os.Remove(thisFile) // if we fail halfway through, this will get noticed next time around and cleaned up + curCount := uint64(len(appearances)) + + isSnap := (curRange.Last >= settings.First_snap && (curRange.Last%settings.Snap_to_grid) == 0) + isOvertop := curCount >= settings.Apps_per_chunk if isSnap || isOvertop { - // we're consolidating... + // We've found a chunk. We can consolidate it. A chunk is a relational table relating + // a table of addresses to where they appear in a table of appearances. We store, for + // each address, its appearances. appMap := make(index.AddressAppearanceMap, len(appearances)) for _, line := range appearances { parts := strings.Split(line, "\t") - if len(parts) == 3 { - addr := strings.ToLower(parts[0]) - bn, _ := strconv.ParseUint(parts[1], 10, 32) - txid, _ := strconv.ParseUint(parts[2], 10, 32) - appMap[addr] = append(appMap[addr], index.AppearanceRecord{ - BlockNumber: uint32(bn), - TransactionId: uint32(txid), - }) + if len(parts) != 3 { + // protect ourselves -- TODO: should we report this? + continue } + addr := strings.ToLower(parts[0]) + appMap[addr] = append(appMap[addr], index.AppearanceRecord{ + BlockNumber: uint32(utils.MustParseUint(parts[1])), + TransactionId: uint32(utils.MustParseUint(parts[2])), + }) } indexPath := config.GetPathToIndex(chain) + "finalized/" + curRange.String() + ".bin" if report, err := index.WriteChunk(chain, indexPath, appMap, len(appearances), bm.opts.Pin, bm.opts.Remote); err != nil { - return false, err + return err, false } else if report == nil { logger.Fatal("Should not happen, write chunk returned empty report") } else { @@ -132,46 +142,49 @@ func (bm *BlazeManager) Consolidate() (bool, error) { report.Report() } - curRange.First = curRange.Last + 1 - appearances = []string{} + // There may be more than one chunk in the current stage, so continue until we're finished. + appearances = []string{} // clear } } + // There are almost certainly appearances left over that were not consolidated. Write these back to the stage. if len(appearances) > 0 { - lineLast := appearances[len(appearances)-1] - parts := strings.Split(lineLast, "\t") - Last := uint64(0) - if len(parts) > 1 { - Last, _ = strconv.ParseUint(parts[1], 10, 32) - Last = utils.Max(utils.Min(bm.ripeBlock, bm.StartBlock()+bm.BlockCount()-1), Last) - } else { - return true, errors.New("Cannot find last block number at lineLast in consolidate: " + lineLast) - } + lastLine := appearances[len(appearances)-1] + parts := strings.Split(lastLine, "\t") + if len(parts) < 2 { + // note that we don't remove the backup file + return fmt.Errorf("cannot find block number at last line (%s)", lastLine), true - conn := rpc.TempConnection(chain) - m, _ := conn.GetMetaData(bm.opts.Globals.TestMode) - rng := base.FileRange{First: m.Finalized + 1, Last: Last} - f := fmt.Sprintf("%s.txt", rng) - fileName := filepath.Join(config.GetPathToIndex(chain), "staging", f) - err = file.LinesToAsciiFile(fileName, appearances) - if err != nil { - os.Remove(fileName) // cleans up by replacing the previous stage - return true, err + } else { + last := blocks[len(blocks)-1] + meta, _ := bm.opts.Conn.GetMetaData(testMode) // the meta may have changed when we wrote the chunks + newStageRng := base.FileRange{First: meta.Finalized + 1, Last: last} + newStageFn := filepath.Join(config.GetPathToIndex(chain), "staging", fmt.Sprintf("%s.txt", newStageRng)) + err = file.LinesToAsciiFile(newStageFn, appearances) + if err != nil { + os.Remove(newStageFn) // remove it if it was created, note that we don't remove the backup file + return err, true + } } } + // Let the user know what happened... stageFn, _ = file.LatestFileInFolder(stageFolder) // it may not exist... nAppsNow := int(file.FileSize(stageFn) / asciiAppearanceSize) - bm.report(int(bm.opts.Settings.Apps_per_chunk), nAppsThen, nAppsNow) + bm.report(int(settings.Apps_per_chunk), nAppsThen, nAppsNow) - os.Remove(backupFn) // commits the change + // Commit the change by deleting the backup file (if the backup file exists + // when this function exits, it will replace the original file - thereby + // starting over). + os.Remove(backupFn) - return true, err + return err, true } -func isListSequential(chain string, ripeFileList []os.DirEntry, allowMissing bool) error { +func (bm *BlazeManager) isListSequential(chain string, ripeFiles []os.DirEntry) error { prev := base.NotARange - for _, file := range ripeFileList { + allowMissing := bm.opts.Settings.Allow_missing + for _, file := range ripeFiles { fileRange := base.RangeFromFilename(file.Name()) if prev != base.NotARange && prev != fileRange { if !prev.Preceeds(fileRange, !allowMissing) { @@ -182,7 +195,3 @@ func isListSequential(chain string, ripeFileList []os.DirEntry, allowMissing boo } return nil } - -var spaces = strings.Repeat(" ", 40) - -// TODO: chifra scrape misreports appearances per block when consolidating #2291 (closed, but copied here as a TODO) diff --git a/src/apps/chifra/internal/scrape/scrape_manager.go b/src/apps/chifra/internal/scrape/scrape_manager.go index 92f1885c45..57f581b7dc 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager.go +++ b/src/apps/chifra/internal/scrape/scrape_manager.go @@ -39,6 +39,11 @@ func (bm *BlazeManager) BlockCount() base.Blknum { return bm.blockCount } +// EndBlock returns the last block to process for this pass of the scraper. +func (bm *BlazeManager) EndBlock() base.Blknum { + return bm.startBlock + bm.blockCount +} + // Report prints out a report of the progress of the scraper. func (bm *BlazeManager) report(perChunk, nAppsThen, nAppsNow int) { need := perChunk - utils.Min(perChunk, nAppsNow) From 407a104b3d7db7a46a7f39304e1b270475377f37 Mon Sep 17 00:00:00 2001 From: tjayrush Date: Mon, 4 Sep 2023 12:59:23 -0400 Subject: [PATCH 08/33] Cleaning --- src/apps/chifra/internal/monitors/handle_watch.go | 2 +- src/apps/chifra/internal/scrape/handle_scrape.go | 3 ++- src/apps/chifra/internal/scrape/scrape_batch.go | 1 + src/apps/chifra/internal/scrape/scrape_manager.go | 1 - 4 files changed, 4 insertions(+), 3 deletions(-) diff --git a/src/apps/chifra/internal/monitors/handle_watch.go b/src/apps/chifra/internal/monitors/handle_watch.go index e0d2c3a079..dc0090eb09 100644 --- a/src/apps/chifra/internal/monitors/handle_watch.go +++ b/src/apps/chifra/internal/monitors/handle_watch.go @@ -161,7 +161,7 @@ func (opts *MonitorsOptions) Refresh(monitors []monitor.Monitor) (bool, error) { continue } - fmt.Printf("Processing item %d in batch %d: %d %d\n", j, i, countsBefore[j], countAfter) + logger.Info("Processing item %d in batch %d: %d %d\n", j, i, countsBefore[j], countAfter) for _, cmd := range theCmds { countBefore := countsBefore[j] diff --git a/src/apps/chifra/internal/scrape/handle_scrape.go b/src/apps/chifra/internal/scrape/handle_scrape.go index e33660f194..927895cc34 100644 --- a/src/apps/chifra/internal/scrape/handle_scrape.go +++ b/src/apps/chifra/internal/scrape/handle_scrape.go @@ -37,7 +37,7 @@ func (opts *ScrapeOptions) HandleScrape() error { // Loop until the user hits Cntl+C, until runCount runs out, or until // the server tells us to stop. for { - // We create a new manager for each loop... + // We create a new manager for each loop...we will populate it in a minute... bm := BlazeManager{ chain: chain, } @@ -134,3 +134,4 @@ func (opts *ScrapeOptions) HandleScrape() error { // We've left the loop and we're done. return nil } + diff --git a/src/apps/chifra/internal/scrape/scrape_batch.go b/src/apps/chifra/internal/scrape/scrape_batch.go index 1859f3e137..92da2cca10 100644 --- a/src/apps/chifra/internal/scrape/scrape_batch.go +++ b/src/apps/chifra/internal/scrape/scrape_batch.go @@ -35,3 +35,4 @@ func (bm *BlazeManager) ScrapeBatch(blocks []base.Blknum) (error, bool) { return bm.WriteTimestamps(blocks), true } + diff --git a/src/apps/chifra/internal/scrape/scrape_manager.go b/src/apps/chifra/internal/scrape/scrape_manager.go index 57f581b7dc..491c449d57 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager.go +++ b/src/apps/chifra/internal/scrape/scrape_manager.go @@ -66,7 +66,6 @@ func (opts *ScrapeOptions) pause(dist uint64) { // we always pause at least a quarter of a second to allow the node to 'rest' time.Sleep(250 * time.Millisecond) isDefaultSleep := opts.Sleep >= 13 && opts.Sleep <= 14 - // distanceFromHead := meta.ChainHeight() - meta.Staging shouldSleep := !isDefaultSleep || dist <= (2*opts.Settings.Unripe_dist) if shouldSleep { sleep := opts.Sleep // this value may change elsewhere allow us to break out of sleeping???? From 12e8cb9a2bc2a83f02ba20ef9293f0860a32e469 Mon Sep 17 00:00:00 2001 From: tjayrush Date: Tue, 5 Sep 2023 20:47:31 -0400 Subject: [PATCH 09/33] Are we done? --- .../chifra/internal/chunks/handle_truncate.go | 5 +- src/apps/chifra/internal/init/handle_init.go | 2 +- .../chifra/internal/scrape/handle_scrape.go | 94 +++++-- .../chifra/internal/scrape/scrape_batch.go | 20 +- .../chifra/internal/scrape/scrape_blaze.go | 27 +- .../internal/scrape/scrape_consolidate.go | 257 ++++++++---------- .../chifra/internal/scrape/scrape_manager.go | 87 +++--- .../internal/scrape/scrape_manager_utils.go | 44 +++ .../chifra/internal/scrape/scrape_prepare.go | 2 +- src/apps/chifra/pkg/file/asciifiles.go | 2 +- src/apps/chifra/pkg/file/count.go | 7 +- src/apps/chifra/pkg/index/establish.go | 19 +- 12 files changed, 336 insertions(+), 230 deletions(-) create mode 100644 src/apps/chifra/internal/scrape/scrape_manager_utils.go diff --git a/src/apps/chifra/internal/chunks/handle_truncate.go b/src/apps/chifra/internal/chunks/handle_truncate.go index 9964f92146..c69cbbcf1a 100644 --- a/src/apps/chifra/internal/chunks/handle_truncate.go +++ b/src/apps/chifra/internal/chunks/handle_truncate.go @@ -22,6 +22,8 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/walk" ) +// TODO: We need to make sure, when we truncate, that we truncate the corresponding maps files as well. + func (opts *ChunksOptions) HandleTruncate(blockNums []uint64) error { chain := opts.Globals.Chain @@ -34,8 +36,7 @@ func (opts *ChunksOptions) HandleTruncate(blockNums []uint64) error { return nil } - indexPath := config.GetPathToIndex(chain) - _ = index.CleanTemporaryFolders(indexPath, true) + _ = index.CleanEphemeralIndexFolders(chain) ctx, cancel := context.WithCancel(context.Background()) fetchData := func(modelChan chan types.Modeler[types.RawModeler], errorChan chan error) { diff --git a/src/apps/chifra/internal/init/handle_init.go b/src/apps/chifra/internal/init/handle_init.go index d680068544..ac47dd35b0 100644 --- a/src/apps/chifra/internal/init/handle_init.go +++ b/src/apps/chifra/internal/init/handle_init.go @@ -30,7 +30,7 @@ func (opts *InitOptions) HandleInit() error { // TODO: BOGUS - IF THE SCRAPER IS RUNNING, THIS WILL CAUSE PROBLEMS // Make sure that the temporary scraper folders are empty, so that, when the // scraper starts, it starts on the correct block. - _ = index.CleanTemporaryFolders(config.GetPathToIndex(chain), true) + _ = index.CleanTempIndexFolders(chain, []string{"ripe", "unripe", "staging"}) remoteManifest, err := manifest.ReadManifest(chain, manifest.FromContract) if err != nil { diff --git a/src/apps/chifra/internal/scrape/handle_scrape.go b/src/apps/chifra/internal/scrape/handle_scrape.go index 927895cc34..c69fa1198c 100644 --- a/src/apps/chifra/internal/scrape/handle_scrape.go +++ b/src/apps/chifra/internal/scrape/handle_scrape.go @@ -7,11 +7,12 @@ package scrapePkg import ( "fmt" "os" - "path/filepath" + "strings" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/colors" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/file" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/index" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" @@ -45,26 +46,27 @@ func (opts *ScrapeOptions) HandleScrape() error { // Fetch the meta data which tells us how far along the index is. if bm.meta, err = opts.Conn.GetMetaData(testMode); err != nil { var ErrFetchingMeta = fmt.Errorf("error fetching meta data: %s", err) - logger.Error(colors.BrightRed, ErrFetchingMeta, colors.Off) + logger.Error(colors.BrightRed+ErrFetchingMeta.Error(), colors.Off) goto PAUSE } - // We're may be too close to the start of the chain to have ripe blocks. - // Report no error but try again soon. + // This only happens if the chain and the index scraper are both started at the + // same time (rarely). This protects against the case where the chain has no ripe blocks. + // Report no error and sleep for a while. if bm.meta.ChainHeight() < opts.Settings.Unripe_dist { goto PAUSE } - // The user may have restarted his node's sync (that is, started over). - // In this case, the index may be ahead of the chain, if so we go to - // sleep and try again later in the hopes that the chain catches up. - if bm.meta.NextIndexHeight() > bm.meta.ChainHeight() { + // Another rare case, but here the user has reset his/her node but not removed + // the index. In this case, the index is ahead of the chain. We go to sleep and + // try again later in the hopes that the chain catches up. + if bm.meta.NextIndexHeight() > bm.meta.ChainHeight()+1 { var ErrIndexAhead = fmt.Errorf( "index (%d) is ahead of chain (%d)", bm.meta.NextIndexHeight(), bm.meta.ChainHeight(), ) - logger.Error(colors.BrightRed, ErrIndexAhead, colors.Off) + logger.Error(colors.BrightRed+ErrIndexAhead.Error(), colors.Off) goto PAUSE } @@ -72,7 +74,8 @@ func (opts *ScrapeOptions) HandleScrape() error { bm = BlazeManager{ chain: chain, opts: opts, - nProcessed: 0, + nRipe: 0, + nUnripe: 0, timestamps: make([]tslib.TimestampRecord, 0, opts.BlockCnt), processedMap: make(map[base.Blknum]bool, opts.BlockCnt), meta: bm.meta, @@ -93,45 +96,82 @@ func (opts *ScrapeOptions) HandleScrape() error { blocks = append(blocks, block) } + if len(blocks) == 0 { + logger.Info("no blocks to scrape") + goto PAUSE + } + + if opts.Globals.Verbose { + logger.Info("chain head: ", bm.meta.ChainHeight()) + logger.Info("opts.BlockCnt: ", opts.BlockCnt) + logger.Info("opts.ChunkSize: ", opts.Settings.Apps_per_chunk) + logger.Info("block count: ", bm.blockCount) + logger.Info("ripe block: ", bm.ripeBlock) + logger.Info("start block: ", bm.startBlock) + logger.Info("len(blocks): ", len(blocks)) + if len(blocks) > 0 { + logger.Info("blocks[0]: ", blocks[0]) + logger.Info("blocks[len(blocks)-1]:", blocks[len(blocks)-1]) + } + } + // Scrape this round. Only quit on catostrophic errors. Report and sleep otherwise. if err, ok = bm.ScrapeBatch(blocks); !ok || err != nil { - logger.Error(colors.BrightRed, err, colors.Off) + if err != nil { + logger.Error(colors.BrightRed+err.Error(), colors.Off) + } if !ok { break } goto PAUSE } - // Consilidate a chunk (if possible). Only quit on catostrophic errors. Report and sleep otherwise. - if err, ok = bm.Consolidate(blocks); !ok || err != nil { - logger.Error(colors.BrightRed, err, colors.Off) - if !ok { - break - } + if bm.nRipe == 0 { + logger.Info(colors.Green+"no ripe files to consolidate", spaces, colors.Off) goto PAUSE - } - PAUSE: - // If we've gotten this far, we want to clean up the unripe files (we no longer need them). - // The chain frequently re-orgs, so we want to re-qeury these next round. This is why - // we have an unripePath. - unripePath := filepath.Join(config.GetPathToIndex(chain), "unripe") - if err = os.RemoveAll(unripePath); err != nil { - logger.Error(colors.BrightRed, err, colors.Off) - return err + } else { + // Consilidate a chunk (if possible). Only quit on catostrophic errors. Report and sleep otherwise. + if err, ok = bm.Consolidate(blocks); !ok || err != nil { + if err != nil { + logger.Error(colors.BrightRed+err.Error(), colors.Off) + } + if !ok { + break + } + goto PAUSE + } } + PAUSE: runCount++ if opts.RunCount != 0 && runCount >= opts.RunCount { + // No reason to clean up here. Next round will do so and user can use these files in the meantime. logger.Info("run count reached") break } // sleep for a bit (there's no new blocks anyway if we're caught up). opts.pause(bm.meta.ChainHeight() - bm.meta.StageHeight()) + + // defensive programming - just double checking our own understanding... + count := file.NFilesInFolder(bm.RipeFolder()) + if count != 0 { + _ = index.CleanEphemeralIndexFolders(chain) + err := fmt.Errorf("%d unexpected ripe files in %s", count, bm.RipeFolder()) + logger.Error(colors.BrightRed+err.Error(), colors.Off) + } + + // We want to clean up the unripe files. The chain may have (it frequently does) + // re-orged. We want to re-qeury these next round. This is why we have an unripePath. + if err = os.RemoveAll(bm.UnripeFolder()); err != nil { + logger.Error(colors.BrightRed, err, colors.Off) + return err + } } // We've left the loop and we're done. return nil } +var spaces = strings.Repeat(" ", 50) diff --git a/src/apps/chifra/internal/scrape/scrape_batch.go b/src/apps/chifra/internal/scrape/scrape_batch.go index 92da2cca10..d9377a5c8b 100644 --- a/src/apps/chifra/internal/scrape/scrape_batch.go +++ b/src/apps/chifra/internal/scrape/scrape_batch.go @@ -8,17 +8,16 @@ import ( "fmt" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/index" ) // ScrapeBatch is called each time around the forever loop. It calls into // HandleBlaze and writes the timestamps if there's no error. func (bm *BlazeManager) ScrapeBatch(blocks []base.Blknum) (error, bool) { - indexPath := config.GetPathToIndex(bm.chain) + chain := bm.chain if err, ok := bm.HandleBlaze(blocks); !ok || err != nil { - _ = index.CleanTemporaryFolders(indexPath, false) + _ = index.CleanEphemeralIndexFolders(chain) return err, ok } @@ -28,11 +27,22 @@ func (bm *BlazeManager) ScrapeBatch(blocks []base.Blknum) (error, bool) { // We missed a block. We need to clean up and continue // next time around the loop. This may happen if the // node returns an error for example. - _ = index.CleanTemporaryFolders(indexPath, false) + _ = index.CleanEphemeralIndexFolders(chain) return fmt.Errorf("a block (%d) was not processed", block), true } } + // defensive programming... + if len(blocks) != len(bm.processedMap) || len(blocks) != bm.nProcessed() { + _ = index.CleanEphemeralIndexFolders(chain) + return fmt.Errorf(`check failed len(blocks): %d len(map): %d nRipe: %d nUnripe: %d nProcessed: %d`, + len(blocks), + len(bm.processedMap), + bm.nRipe, + bm.nUnripe, + bm.nProcessed(), + ), true + } + return bm.WriteTimestamps(blocks), true } - diff --git a/src/apps/chifra/internal/scrape/scrape_blaze.go b/src/apps/chifra/internal/scrape/scrape_blaze.go index c02d5b7272..045c06422c 100644 --- a/src/apps/chifra/internal/scrape/scrape_blaze.go +++ b/src/apps/chifra/internal/scrape/scrape_blaze.go @@ -150,8 +150,19 @@ func (bm *BlazeManager) ProcessTimestamps(tsChannel chan tslib.TimestampRecord, var writeMutex sync.Mutex +// TODO: The original intent of creating files was so that we could start over where we left off +// if we failed. But this isn't how it works. We cleanup any temp files if we fail, which means +// we write these files and if we fail, we remove them. If we don't fail, we've written them out, +// but only to re-read them and delete them in this round. Would could have easily just kept them +// in an in-memory cache. This would also allow us to not have to stringify the data and just store +// pointers to structs in memory. We wouldn't have to keep a seperate timestamps database nor a +// processedMap (the pointer would serve that purpose). + // WriteAppearances writes the appearance for a chunk to a file func (bm *BlazeManager) WriteAppearances(bn base.Blknum, addrMap index.AddressBooleanMap) (err error) { + ripePath := config.GetPathToIndex(bm.chain) + "ripe/" + unripePath := config.GetPathToIndex(bm.chain) + "unripe/" + if len(addrMap) > 0 { appearanceArray := make([]string, 0, len(addrMap)) for record := range addrMap { @@ -160,15 +171,15 @@ func (bm *BlazeManager) WriteAppearances(bn base.Blknum, addrMap index.AddressBo sort.Strings(appearanceArray) blockNumStr := utils.PadNum(int(bn), 9) - fileName := config.GetPathToIndex(bm.chain) + "ripe/" + blockNumStr + ".txt" + fileName := ripePath + blockNumStr + ".txt" if bn > bm.ripeBlock { - fileName = config.GetPathToIndex(bm.chain) + "unripe/" + blockNumStr + ".txt" + fileName = unripePath + blockNumStr + ".txt" } toWrite := []byte(strings.Join(appearanceArray[:], "\n") + "\n") err = os.WriteFile(fileName, toWrite, 0744) // Uses os.O_WRONLY|os.O_CREATE|os.O_TRUNC if err != nil { - fmt.Println("call1 to WriteFile returned error", err) + fmt.Println("WriteFile returned error", err) return err } } @@ -177,7 +188,11 @@ func (bm *BlazeManager) WriteAppearances(bn base.Blknum, addrMap index.AddressBo writeMutex.Lock() bm.processedMap[bn] = true writeMutex.Unlock() - bm.nProcessed++ + if bn > bm.ripeBlock { + bm.nUnripe++ + } else { + bm.nRipe++ + } return } @@ -195,13 +210,13 @@ func (bm *BlazeManager) syncedReporting(bn base.Blknum, force bool) { defer atomic.StoreUint32(&locker, 0) // Only report once in a while (17 blocks) - if bm.nProcessed%17 == 0 || force { + if bm.nProcessed()%17 == 0 || force { dist := uint64(0) if bm.ripeBlock > bn { dist = (bm.ripeBlock - bn) } msg := fmt.Sprintf("Scraping %-04d of %-04d at block %d of %d (%d blocks from head)", - bm.nProcessed, + bm.nProcessed(), bm.BlockCount(), bn, bm.ripeBlock, diff --git a/src/apps/chifra/internal/scrape/scrape_consolidate.go b/src/apps/chifra/internal/scrape/scrape_consolidate.go index 08f3dd53e0..f94eda6752 100644 --- a/src/apps/chifra/internal/scrape/scrape_consolidate.go +++ b/src/apps/chifra/internal/scrape/scrape_consolidate.go @@ -1,10 +1,12 @@ package scrapePkg import ( + "context" "errors" "fmt" "os" "path/filepath" + "sort" "strings" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" @@ -12,6 +14,7 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/file" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/index" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/sigintTrap" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" ) @@ -19,121 +22,80 @@ const asciiAppearanceSize = 59 // Consolidate calls into the block scraper to (a) call Blaze and (b) consolidate if applicable func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { + var err error chain := bm.chain - settings := bm.opts.Settings - testMode := bm.opts.Globals.TestMode - ripeFolder := filepath.Join(config.GetPathToIndex(chain), "ripe") - stageFolder := filepath.Join(config.GetPathToIndex(chain), "staging") + indexPath := config.GetPathToIndex(chain) - ripeFiles, err := os.ReadDir(ripeFolder) - if err != nil { - return err, true - } - - if len(ripeFiles) == 0 { - // There are no new ripeFiles, but we've processed all blocks in the block - // list. We want to rename the stage file (if it exists) to reflect the - // new stage head. If the file doesn't exist, just touch it. The scraper - // uses the stage file's name to know where to start the next loop. - curStage, _ := file.LatestFileInFolder(stageFolder) // it may not exist... - curRange := base.RangeFromFilename(curStage) - - // All blocks in the list have been processed, so we know where the stage ends. - newRange := base.FileRange{First: curRange.First, Last: blocks[len(blocks)-1]} - newStage := filepath.Join(stageFolder, newRange.String()+".txt") - - if file.FileExists(curStage) { - _ = os.Rename(curStage, newStage) - _ = os.Remove(curStage) // seems redundant, but may not be on some operating systems - } else { - file.Touch(newStage) - } - - // Let the user know something happened... - return fmt.Errorf("no new ripe blocks found at %d (%d away from head)", - bm.meta.ChainHeight(), - bm.meta.ChainHeight() - bm.meta.Ripe, - ), true - } - - // If the number of ripe files is less than we expected... - if uint64(len(ripeFiles)) < (bm.BlockCount() - settings.Unripe_dist) { - // ...at least make sure they're sequential (if the chain so desires). - if err := bm.isListSequential(chain, ripeFiles); err != nil { - _ = index.CleanTemporaryFolders(config.GetPathToCache(chain), false) - return err, true + backupFn := "" + stageFn, _ := file.LatestFileInFolder(bm.StageFolder()) // it may not exist... + if file.FileExists(stageFn) { + backupFn, err = file.MakeBackup(filepath.Join(config.GetPathToCache(chain)+"tmp"), stageFn) + if err != nil { + return errors.New("Could not create backup file: " + err.Error()), true } + defer func() { + // If the backup file exists, the function did not complete. In that case, we replace the original file. + if backupFn != "" && file.FileExists(backupFn) { + _ = os.Rename(backupFn, stageFn) + _ = os.Remove(backupFn) // seems redundant, but may not be on some operating systems + } + }() } + countThen := file.FileSize(stageFn) / asciiAppearanceSize - stageFn, _ := file.LatestFileInFolder(stageFolder) // it may not exist... - stageRange := base.RangeFromFilename(stageFn) - nAppsThen := int(file.FileSize(stageFn) / asciiAppearanceSize) + // After this point if we fail the backup file will replace the original file, so + // we can safely remove these the stage file (and ripe files) and it will get replaced - var curRange base.FileRange - if file.FileExists(stageFn) { - curRange = stageRange - } else { - curRange = base.FileRange{First: blocks[0], Last: blocks[len(blocks)-1]} + // Make sure the user can't interrupt us... + ctx, cancel := context.WithCancel(context.Background()) + cleanOnQuit := func() { + logger.Warn(sigintTrap.TrapMessage) } - - // Note, the stage may be empty or non-existant - backupFn, err := file.MakeBackup(filepath.Join(config.GetPathToCache(chain)+"tmp"), stageFn) - if err != nil { - return errors.New("Could not create backup file: " + err.Error()), true + trapChannel := sigintTrap.Enable(ctx, cancel, cleanOnQuit) + defer sigintTrap.Disable(trapChannel) + + // Load the stage into the map... + exists := file.FileExists(stageFn) // order matters + appMap, chunkRange, nAppearances := bm.AsciiFileToAppearanceMap(stageFn) + if !exists { + // Brand new stage. + chunkRange = base.FileRange{First: bm.meta.Finalized + 1, Last: blocks[0]} } - defer func() { - // If the backup file exists, the function did not complete. In that case, we replace the original file. - if backupFn != "" && file.FileExists(backupFn) { - _ = os.Rename(backupFn, stageFn) - _ = os.Remove(backupFn) // seems redundant, but may not be on some operating systems - } - }() - // Note: - // if we fail at any point past this, the backup file will replace the original file - - // Start by copying in the existing appearances on the stage... - appearances := file.AsciiFileToLines(stageFn) - os.Remove(stageFn) // We have a backup copy, so it's okay to remove this here... - - // Spin through each ripe file and add it to the appearances array until we have either (a) a snap - // point or (b) enough appearances to overtop. In either case, we consolidate the appearances into - // a chunk, create its bloom, and write it to disk. Note that each time we copy one of the ripe - // files, we remove it. - for _, ripeFile := range ripeFiles { - thisFile := filepath.Join(ripeFolder, ripeFile.Name()) - ripeRange := base.RangeFromFilename(thisFile) - curRange.Last = ripeRange.Last + // For each block... + for _, block := range blocks { + fn := fmt.Sprintf("%09d.txt", block) + if file.FileExists(filepath.Join(bm.UnripeFolder(), fn)) { + continue // skip unripe files + } - // append the appearances from this file - appearances = append(appearances, file.AsciiFileToLines(thisFile)...) - os.Remove(thisFile) // if we fail halfway through, this will get noticed next time around and cleaned up - curCount := uint64(len(appearances)) + ripeFn := filepath.Join(bm.RipeFolder(), fn) + if !file.FileExists(ripeFn) { + msg := fmt.Sprintf("ripe file not found for block %d", block) + spaces + if !bm.AllowMissing() { + _ = index.CleanEphemeralIndexFolders(chain) + return errors.New(msg), true + } else { + logger.Warn(msg) + } + } - isSnap := (curRange.Last >= settings.First_snap && (curRange.Last%settings.Snap_to_grid) == 0) - isOvertop := curCount >= settings.Apps_per_chunk + // Read in the ripe file, add it to the appMap and... + thisMap, _, thisCount := bm.AsciiFileToAppearanceMap(ripeFn) + nAppearances += thisCount + for addr, apps := range thisMap { + appMap[addr] = append(appMap[addr], apps...) + } + chunkRange.Last = block + // ...decide if we need to consolidate... + isSnap := bm.IsSnap(chunkRange.Last) // Have we hit a snap point? + isOvertop := nAppearances >= int(bm.PerChunk()) // Does this block overtop a chunk? if isSnap || isOvertop { - // We've found a chunk. We can consolidate it. A chunk is a relational table relating - // a table of addresses to where they appear in a table of appearances. We store, for - // each address, its appearances. - appMap := make(index.AddressAppearanceMap, len(appearances)) - for _, line := range appearances { - parts := strings.Split(line, "\t") - if len(parts) != 3 { - // protect ourselves -- TODO: should we report this? - continue - } - addr := strings.ToLower(parts[0]) - appMap[addr] = append(appMap[addr], index.AppearanceRecord{ - BlockNumber: uint32(utils.MustParseUint(parts[1])), - TransactionId: uint32(utils.MustParseUint(parts[2])), - }) - } - - indexPath := config.GetPathToIndex(chain) + "finalized/" + curRange.String() + ".bin" - if report, err := index.WriteChunk(chain, indexPath, appMap, len(appearances), bm.opts.Pin, bm.opts.Remote); err != nil { + // Make a chunk - i.e., consolidate + chunkPath := indexPath + "finalized/" + chunkRange.String() + ".bin" + if report, err := index.WriteChunk(chain, chunkPath, appMap, nAppearances, bm.opts.Pin, bm.opts.Remote); err != nil { return err, false } else if report == nil { logger.Fatal("Should not happen, write chunk returned empty report") @@ -141,57 +103,74 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { report.Snapped = isSnap report.Report() } - - // There may be more than one chunk in the current stage, so continue until we're finished. - appearances = []string{} // clear + // reset for next chunk + bm.meta, _ = bm.opts.Conn.GetMetaData(bm.IsTestMode()) + appMap = make(index.AddressAppearanceMap, 0) + chunkRange.First = chunkRange.Last + 1 + chunkRange.Last = chunkRange.Last + 1 + nAppearances = 0 } } - // There are almost certainly appearances left over that were not consolidated. Write these back to the stage. - if len(appearances) > 0 { - lastLine := appearances[len(appearances)-1] - parts := strings.Split(lastLine, "\t") - if len(parts) < 2 { - // note that we don't remove the backup file - return fmt.Errorf("cannot find block number at last line (%s)", lastLine), true - - } else { - last := blocks[len(blocks)-1] - meta, _ := bm.opts.Conn.GetMetaData(testMode) // the meta may have changed when we wrote the chunks - newStageRng := base.FileRange{First: meta.Finalized + 1, Last: last} - newStageFn := filepath.Join(config.GetPathToIndex(chain), "staging", fmt.Sprintf("%s.txt", newStageRng)) - err = file.LinesToAsciiFile(newStageFn, appearances) - if err != nil { - os.Remove(newStageFn) // remove it if it was created, note that we don't remove the backup file - return err, true + if len(appMap) > 0 { // are there any appearances in this block range? + newRange := base.FileRange{First: bm.meta.Finalized + 1, Last: 0} + + // We need an array because we're going to write it back to disc + appearances := make([]string, 0, nAppearances) + for addr, apps := range appMap { + for _, app := range apps { + record := fmt.Sprintf("%s\t%09d\t%05d", addr, app.BlockNumber, app.TransactionId) + appearances = append(appearances, record) + newRange.Last = utils.Max(newRange.Last, uint64(app.BlockNumber)) } } + // The stage needs to be sorted because the end user queries it and we want the search to be fast + sort.Strings(appearances) + + newStageFn := filepath.Join(bm.StageFolder(), fmt.Sprintf("%s.txt", newRange)) + if err := file.LinesToAsciiFile(newStageFn, appearances); err != nil { + os.Remove(newStageFn) + return err, true + } } // Let the user know what happened... - stageFn, _ = file.LatestFileInFolder(stageFolder) // it may not exist... - nAppsNow := int(file.FileSize(stageFn) / asciiAppearanceSize) - bm.report(int(settings.Apps_per_chunk), nAppsThen, nAppsNow) + stageFn, _ = file.LatestFileInFolder(bm.StageFolder()) // it may not exist... + countNow := file.FileSize(stageFn) / asciiAppearanceSize + bm.report(int64(bm.PerChunk()), countThen, countNow) - // Commit the change by deleting the backup file (if the backup file exists - // when this function exits, it will replace the original file - thereby - // starting over). + // Commit the change by deleting the backup file. os.Remove(backupFn) - return err, true + return nil, true } -func (bm *BlazeManager) isListSequential(chain string, ripeFiles []os.DirEntry) error { - prev := base.NotARange - allowMissing := bm.opts.Settings.Allow_missing - for _, file := range ripeFiles { - fileRange := base.RangeFromFilename(file.Name()) - if prev != base.NotARange && prev != fileRange { - if !prev.Preceeds(fileRange, !allowMissing) { - return fmt.Errorf("ripe files are not sequential (%s ==> %s)", prev, fileRange) - } +// AsciiFileToAppearanceMap reads the appearances from the stage file and returns them as a map +func (bm *BlazeManager) AsciiFileToAppearanceMap(fn string) (index.AddressAppearanceMap, base.FileRange, int) { + appearances := file.AsciiFileToLines(fn) + os.Remove(fn) // It's okay to remove this. If it fails, we'll just start over. + + appMap := make(index.AddressAppearanceMap, len(appearances)) + fileRange := base.FileRange{First: utils.NOPOS, Last: 0} + + if len(appearances) == 0 { + return appMap, base.FileRange{First: 0, Last: 0}, 0 + } + + for _, line := range appearances { + line := line + parts := strings.Split(line, "\t") + if len(parts) == 3 { // shouldn't be needed, but just in case... + addr := strings.ToLower(parts[0]) + bn := utils.MustParseUint(strings.TrimLeft(parts[1], "0")) + txid := utils.MustParseUint(strings.TrimLeft(parts[2], "0")) + fileRange.First = utils.Min(fileRange.First, bn) + fileRange.Last = utils.Max(fileRange.Last, bn) + appMap[addr] = append(appMap[addr], index.AppearanceRecord{ + BlockNumber: uint32(bn), + TransactionId: uint32(txid), + }) } - prev = fileRange } - return nil + return appMap, fileRange, len(appearances) } diff --git a/src/apps/chifra/internal/scrape/scrape_manager.go b/src/apps/chifra/internal/scrape/scrape_manager.go index 491c449d57..de41b49d68 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager.go +++ b/src/apps/chifra/internal/scrape/scrape_manager.go @@ -1,16 +1,12 @@ package scrapePkg import ( - "fmt" - "strings" - "time" + "path/filepath" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/colors" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/rpc" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/tslib" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" ) // BlazeManager manages the scraper by keeping track of the progress of the scrape and @@ -20,12 +16,13 @@ type BlazeManager struct { chain string timestamps []tslib.TimestampRecord processedMap map[base.Blknum]bool - nProcessed uint64 opts *ScrapeOptions meta *rpc.MetaData startBlock base.Blknum blockCount base.Blknum ripeBlock base.Blknum + nRipe int + nUnripe int nChannels int } @@ -44,35 +41,55 @@ func (bm *BlazeManager) EndBlock() base.Blknum { return bm.startBlock + bm.blockCount } -// Report prints out a report of the progress of the scraper. -func (bm *BlazeManager) report(perChunk, nAppsThen, nAppsNow int) { - need := perChunk - utils.Min(perChunk, nAppsNow) - seen := nAppsNow - if nAppsThen < nAppsNow { - seen = nAppsNow - nAppsThen - } - pct := float64(nAppsNow) / float64(perChunk) - pBlk := float64(seen) / float64(bm.BlockCount()) - height := bm.StartBlock() + bm.BlockCount() - 1 - - const templ = `Block={%d} have {%d} appearances of {%d} ({%0.1f%%}). Need {%d} more. Added {%d} records ({%0.2f} apps/blk).` - msg := strings.Replace(templ, "{", colors.Green, -1) - msg = strings.Replace(msg, "}", colors.Off, -1) - logger.Info(fmt.Sprintf(msg, height, nAppsNow, perChunk, pct*100, need, seen, pBlk)) +// nProcessed returns the number of blocks processed so far (i.e., ripe + unripe). +func (bm *BlazeManager) nProcessed() int { + return bm.nRipe + bm.nUnripe +} + +// IsTestMode returns true if the scraper is running in test mode. +func (bm *BlazeManager) IsTestMode() bool { + return bm.opts.Globals.TestMode } -// Pause goes to sleep for a period of time based on the settings. -func (opts *ScrapeOptions) pause(dist uint64) { - // we always pause at least a quarter of a second to allow the node to 'rest' - time.Sleep(250 * time.Millisecond) - isDefaultSleep := opts.Sleep >= 13 && opts.Sleep <= 14 - shouldSleep := !isDefaultSleep || dist <= (2*opts.Settings.Unripe_dist) - if shouldSleep { - sleep := opts.Sleep // this value may change elsewhere allow us to break out of sleeping???? - logger.Progress(sleep > 1, "Sleeping for", sleep, "seconds -", dist, "away from head.") - halfSecs := (sleep * 2) - 1 // we already slept one quarter of a second - for i := 0; i < int(halfSecs); i++ { - time.Sleep(time.Duration(500) * time.Millisecond) - } +// PerChunk returns the number of blocks to process per chunk. +func (bm *BlazeManager) AllowMissing() bool { + if bm.chain != "mainnet" { + return true } + return bm.opts.Settings.Allow_missing +} + +// PerChunk returns the number of blocks to process per chunk. +func (bm *BlazeManager) PerChunk() base.Blknum { + return bm.opts.Settings.Apps_per_chunk +} + +// FirstSnap returns the first block to process. +func (bm *BlazeManager) FirstSnap() base.Blknum { + return bm.opts.Settings.First_snap +} + +// SnapTo returns the number of blocks to process per chunk. +func (bm *BlazeManager) SnapTo() base.Blknum { + return bm.opts.Settings.Snap_to_grid +} + +// IsSnap returns true if the block is a snap point. +func (bm *BlazeManager) IsSnap(block base.Blknum) bool { + return block >= bm.FirstSnap() && (block%bm.SnapTo()) == 0 +} + +// StageFolder returns the folder where the stage file is stored. +func (bm *BlazeManager) StageFolder() string { + return filepath.Join(config.GetPathToIndex(bm.chain), "staging") +} + +// RipeFolder returns the folder where the stage file is stored. +func (bm *BlazeManager) RipeFolder() string { + return filepath.Join(config.GetPathToIndex(bm.chain), "ripe") +} + +// UnripeFolder returns the folder where the stage file is stored. +func (bm *BlazeManager) UnripeFolder() string { + return filepath.Join(config.GetPathToIndex(bm.chain), "unripe") } diff --git a/src/apps/chifra/internal/scrape/scrape_manager_utils.go b/src/apps/chifra/internal/scrape/scrape_manager_utils.go new file mode 100644 index 0000000000..b548fafb9c --- /dev/null +++ b/src/apps/chifra/internal/scrape/scrape_manager_utils.go @@ -0,0 +1,44 @@ +package scrapePkg + +import ( + "fmt" + "strings" + "time" + + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/colors" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" +) + +// Report prints out a report of the progress of the scraper. +func (bm *BlazeManager) report(perChunk, nAppsThen, nAppsNow int64) { + need := perChunk - utils.Min(perChunk, nAppsNow) + seen := nAppsNow + if nAppsThen < nAppsNow { + seen = nAppsNow - nAppsThen + } + pct := float64(nAppsNow) / float64(perChunk) + pBlk := float64(seen) / float64(bm.BlockCount()) + height := bm.StartBlock() + bm.BlockCount() - 1 + + const templ = `Block={%d} have {%d} appearances of {%d} ({%0.1f%%}). Need {%d} more. Added {%d} records ({%0.2f} apps/blk).` + msg := strings.Replace(templ, "{", colors.Green, -1) + msg = strings.Replace(msg, "}", colors.Off, -1) + logger.Info(fmt.Sprintf(msg, height, nAppsNow, perChunk, pct*100, need, seen, pBlk)) +} + +// Pause goes to sleep for a period of time based on the settings. +func (opts *ScrapeOptions) pause(dist uint64) { + // we always pause at least a quarter of a second to allow the node to 'rest' + time.Sleep(250 * time.Millisecond) + isDefaultSleep := opts.Sleep >= 13 && opts.Sleep <= 14 + shouldSleep := !isDefaultSleep || dist <= (2*opts.Settings.Unripe_dist) + if shouldSleep { + sleep := opts.Sleep // this value may change elsewhere allow us to break out of sleeping???? + logger.Progress(sleep > 1, "Sleeping for", sleep, "seconds -", dist, "away from head.") + halfSecs := (sleep * 2) - 1 // we already slept one quarter of a second + for i := 0; i < int(halfSecs); i++ { + time.Sleep(time.Duration(500) * time.Millisecond) + } + } +} diff --git a/src/apps/chifra/internal/scrape/scrape_prepare.go b/src/apps/chifra/internal/scrape/scrape_prepare.go index 969b1378ee..e9230301fe 100644 --- a/src/apps/chifra/internal/scrape/scrape_prepare.go +++ b/src/apps/chifra/internal/scrape/scrape_prepare.go @@ -23,7 +23,7 @@ func (opts *ScrapeOptions) Prepare() (ok bool, err error) { chain := opts.Globals.Chain // We always clean the temporary folders (other than staging) when starting - _ = index.CleanTemporaryFolders(config.GetPathToIndex(chain), false) + _ = index.CleanEphemeralIndexFolders(chain) // If the file already exists, we're done. bloomPath := config.GetPathToIndex(chain) + "blooms/000000000-000000000.bloom" diff --git a/src/apps/chifra/pkg/file/asciifiles.go b/src/apps/chifra/pkg/file/asciifiles.go index a2fe8393fc..0c613393f3 100644 --- a/src/apps/chifra/pkg/file/asciifiles.go +++ b/src/apps/chifra/pkg/file/asciifiles.go @@ -74,7 +74,7 @@ func LinesToAsciiFile(filename string, value []string) error { } defer file.Close() - lines := strings.Join(value, "\n") + "\n" + lines := strings.Join(value[:], "\n") + "\n" _, err = io.WriteString(file, lines) if err != nil { return err diff --git a/src/apps/chifra/pkg/file/count.go b/src/apps/chifra/pkg/file/count.go index 7b1b1db779..13fe49b2df 100644 --- a/src/apps/chifra/pkg/file/count.go +++ b/src/apps/chifra/pkg/file/count.go @@ -2,14 +2,9 @@ package file import ( "os" - - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" ) func NFilesInFolder(path string) int { - files, err := os.ReadDir(path) - if err != nil { - logger.Fatal(err) - } + files, _ := os.ReadDir(path) return len(files) } diff --git a/src/apps/chifra/pkg/index/establish.go b/src/apps/chifra/pkg/index/establish.go index efee1203c5..8d870dbc5f 100644 --- a/src/apps/chifra/pkg/index/establish.go +++ b/src/apps/chifra/pkg/index/establish.go @@ -70,19 +70,24 @@ func EstablishIndexChunk(chain string, fileRange base.FileRange) (bool, error) { return file.FileExists(fileName), nil } -// CleanTemporaryFolders removes any files that may be partial or incomplete -func CleanTemporaryFolders(indexPath string, incStaging bool) error { - folders := []string{"ripe", "unripe", "maps", "staging"} - if !incStaging { - folders = folders[:len(folders)-2] - } +// CleanEphemeralIndexFolders removes files in ripe and unripe +func CleanEphemeralIndexFolders(chain string) error { + return CleanTempIndexFolders(chain, []string{"ripe", "unripe"}) +} + +// CleanTempIndexFolders removes any files that may be partial or incomplete +func CleanTempIndexFolders(chain string, subFolders []string) error { + indexPath := config.GetPathToIndex(chain) - for _, f := range folders { + for _, f := range subFolders { folder := filepath.Join(indexPath, f) + // We want to remove whatever is there... err := os.RemoveAll(folder) if err != nil { return err } + // ...but put it back + file.EstablishFolder(folder) } return nil From f198b656af8f91203153794d466a737e4876c01f Mon Sep 17 00:00:00 2001 From: tjayrush Date: Tue, 5 Sep 2023 20:52:11 -0400 Subject: [PATCH 10/33] Fixing lints --- src/apps/chifra/internal/monitors/handle_watch.go | 2 +- src/apps/chifra/pkg/index/establish.go | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/apps/chifra/internal/monitors/handle_watch.go b/src/apps/chifra/internal/monitors/handle_watch.go index dc0090eb09..3c7a573969 100644 --- a/src/apps/chifra/internal/monitors/handle_watch.go +++ b/src/apps/chifra/internal/monitors/handle_watch.go @@ -161,7 +161,7 @@ func (opts *MonitorsOptions) Refresh(monitors []monitor.Monitor) (bool, error) { continue } - logger.Info("Processing item %d in batch %d: %d %d\n", j, i, countsBefore[j], countAfter) + logger.Info(fmt.Printf("Processing item %d in batch %d: %d %d\n", j, i, countsBefore[j], countAfter)) for _, cmd := range theCmds { countBefore := countsBefore[j] diff --git a/src/apps/chifra/pkg/index/establish.go b/src/apps/chifra/pkg/index/establish.go index 8d870dbc5f..2c35eee357 100644 --- a/src/apps/chifra/pkg/index/establish.go +++ b/src/apps/chifra/pkg/index/establish.go @@ -87,7 +87,7 @@ func CleanTempIndexFolders(chain string, subFolders []string) error { return err } // ...but put it back - file.EstablishFolder(folder) + _ = file.EstablishFolder(folder) } return nil From f12517f3e311a05b0352a7d3bdd78179da4977c9 Mon Sep 17 00:00:00 2001 From: tjayrush Date: Tue, 5 Sep 2023 21:03:10 -0400 Subject: [PATCH 11/33] Merging in hot fix on master for port usage --- src/apps/chifra/internal/chunks/handle_truncate.go | 2 +- src/apps/chifra/internal/scrape/scrape_manager.go | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/src/apps/chifra/internal/chunks/handle_truncate.go b/src/apps/chifra/internal/chunks/handle_truncate.go index c69cbbcf1a..01c112abbf 100644 --- a/src/apps/chifra/internal/chunks/handle_truncate.go +++ b/src/apps/chifra/internal/chunks/handle_truncate.go @@ -36,7 +36,7 @@ func (opts *ChunksOptions) HandleTruncate(blockNums []uint64) error { return nil } - _ = index.CleanEphemeralIndexFolders(chain) + _ = index.CleanTempIndexFolders(chain, []string{"ripe", "unripe", "staging"}) ctx, cancel := context.WithCancel(context.Background()) fetchData := func(modelChan chan types.Modeler[types.RawModeler], errorChan chan error) { diff --git a/src/apps/chifra/internal/scrape/scrape_manager.go b/src/apps/chifra/internal/scrape/scrape_manager.go index de41b49d68..46c06d9b21 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager.go +++ b/src/apps/chifra/internal/scrape/scrape_manager.go @@ -51,7 +51,10 @@ func (bm *BlazeManager) IsTestMode() bool { return bm.opts.Globals.TestMode } -// PerChunk returns the number of blocks to process per chunk. +// AllowMissing returns true for all chains but mainnet and the value +// of the config item on mainnet (false by default). The scraper will +// halt if AllowMissing is false and a block with zero appearances is +// encountered. func (bm *BlazeManager) AllowMissing() bool { if bm.chain != "mainnet" { return true From d025fe4d358e9130e84bb6ead18cbad8b45f5f80 Mon Sep 17 00:00:00 2001 From: tjayrush Date: Tue, 5 Sep 2023 21:49:30 -0400 Subject: [PATCH 12/33] Updating --- docs/content/readmes/accounts-abis.md | 62 ++++++++++ docs/content/readmes/accounts-export.md | 98 ++++++++++++++++ docs/content/readmes/accounts-list.md | 56 +++++++++ docs/content/readmes/accounts-monitors.md | 88 ++++++++++++++ docs/content/readmes/accounts-names.md | 57 +++++++++ docs/content/readmes/admin-chunks.md | 64 +++++++++++ docs/content/readmes/admin-config.md | 37 ++++++ docs/content/readmes/admin-daemon.md | 50 ++++++++ docs/content/readmes/admin-init.md | 55 +++++++++ docs/content/readmes/admin-scrape.md | 108 ++++++++++++++++++ docs/content/readmes/admin-status.md | 38 ++++++ docs/content/readmes/chaindata-blocks.md | 71 ++++++++++++ docs/content/readmes/chaindata-logs.md | 45 ++++++++ docs/content/readmes/chaindata-receipts.md | 47 ++++++++ docs/content/readmes/chaindata-traces.md | 53 +++++++++ .../content/readmes/chaindata-transactions.md | 62 ++++++++++ docs/content/readmes/chaindata-when.md | 57 +++++++++ docs/content/readmes/chainstate-state.md | 56 +++++++++ docs/content/readmes/chainstate-tokens.md | 58 ++++++++++ docs/content/readmes/other-explore.md | 32 ++++++ docs/content/readmes/other-slurp.md | 52 +++++++++ docs/content/tutorials/index.md | 10 ++ .../chifra/internal/scrape/save_timestamps.go | 2 +- 23 files changed, 1257 insertions(+), 1 deletion(-) create mode 100644 docs/content/readmes/accounts-abis.md create mode 100644 docs/content/readmes/accounts-export.md create mode 100644 docs/content/readmes/accounts-list.md create mode 100644 docs/content/readmes/accounts-monitors.md create mode 100644 docs/content/readmes/accounts-names.md create mode 100644 docs/content/readmes/admin-chunks.md create mode 100644 docs/content/readmes/admin-config.md create mode 100644 docs/content/readmes/admin-daemon.md create mode 100644 docs/content/readmes/admin-init.md create mode 100644 docs/content/readmes/admin-scrape.md create mode 100644 docs/content/readmes/admin-status.md create mode 100644 docs/content/readmes/chaindata-blocks.md create mode 100644 docs/content/readmes/chaindata-logs.md create mode 100644 docs/content/readmes/chaindata-receipts.md create mode 100644 docs/content/readmes/chaindata-traces.md create mode 100644 docs/content/readmes/chaindata-transactions.md create mode 100644 docs/content/readmes/chaindata-when.md create mode 100644 docs/content/readmes/chainstate-state.md create mode 100644 docs/content/readmes/chainstate-tokens.md create mode 100644 docs/content/readmes/other-explore.md create mode 100644 docs/content/readmes/other-slurp.md create mode 100644 docs/content/tutorials/index.md diff --git a/docs/content/readmes/accounts-abis.md b/docs/content/readmes/accounts-abis.md new file mode 100644 index 0000000000..938e78e0c0 --- /dev/null +++ b/docs/content/readmes/accounts-abis.md @@ -0,0 +1,62 @@ +## chifra abis + + +`chifra abis` retrieves ABI files for the given address(es). It searches for the ABI in this order: +the current local folder, the TrueBlocks cache, Etherscan, or (in the +future) ENS and Sourcify. + +While this tool may be used from the command line and the API, its primary purpose is in support of +tools such as `chifra export` to support the `--articulate` option. + +The `--known` option prints a list of semi-standard function signatures such as the ERC20 standard, +ERC 721 standard, various functions from OpenZeppelin, various Uniswap functions, etc. As an +optimization, the `known` signatures are searched first during articulation. + +The `--sol` option converts the provided Solidity file into an ABI json file. The results are +dropped into the current working folder. + +The `--find` option is experimental. Please see the notes below for more information. + +```[plaintext] +Purpose: + Fetches the ABI for a smart contract. + +Usage: + chifra abis [flags]
[address...] + +Arguments: + addrs - a list of one or more smart contracts whose ABIs to display (required) + +Flags: + -k, --known load common 'known' ABIs from cache + -f, --find strings search for function or event declarations given a four- or 32-byte code(s) + -n, --hint strings for the --find option only, provide hints to speed up the search + -e, --encode string generate the 32-byte encoding for a given cannonical function or event signature + -C, --clean remove an abi file for an address or all zero-length files if no address is given + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - Search for either four byte signatures or event signatures with the --find option. +``` + +Data models produced by this tool: + +- [abi](/data-model/accounts/#abi) +- [function](/data-model/other/#function) +- [parameter](/data-model/other/#parameter) + +Links: + +- [api docs](/api/#operation/accounts-abis) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/abis) + + +### notes + +The `chifra abis --find` option scans the cross product of two sets. The first set contains more than 100,000 function and event +names. The second set contains approximately 700 function signatures. The cross product of these two sets creates 70,000,000 +combinations of name(signature) each of which is hashed to create either a four-byte or a 32-byte hash. Very infrequently, +the tool will find matches for an otherwise unknown signatures. + diff --git a/docs/content/readmes/accounts-export.md b/docs/content/readmes/accounts-export.md new file mode 100644 index 0000000000..73c9735f0a --- /dev/null +++ b/docs/content/readmes/accounts-export.md @@ -0,0 +1,98 @@ +## chifra export + + +The `chifra export` tools provides a major part of the functionality of the TrueBlocks system. Using +the index of appearances created with `chifra scrape` and the list of transaction identifiers +created with `chifra list`, `chifra export` completes the actual extraction of an address's transactional +history from the node. + +You may use `topics`, `fourbyte` values at the start of a transaction's input data, and/or a log's +`source address` or `emitter` to filter the results. + +You may also choose which portions of the Ethereum data structures (`--transactions`, `--logs`, +`--traces`, etc.) as you wish. + +By default, the results of the extraction are delivered to your console, however, you may export +the results to any database (with a little bit of work). The format of the data, its content and +its destination are up to you. + +```[plaintext] +Purpose: + Export full detail of transactions for one or more addresses. + +Usage: + chifra export [flags]
[address...] [topics...] [fourbytes...] + +Arguments: + addrs - one or more addresses (0x...) to export (required) + topics - filter by one or more log topics (only for --logs option) + fourbytes - filter by one or more fourbytes (only for transactions and trace options) + +Flags: + -p, --appearances export a list of appearances + -r, --receipts export receipts instead of transactional data + -l, --logs export logs instead of transactional data + -t, --traces export traces instead of transactional data + -n, --neighbors export the neighbors of the given address + -C, --accounting attach accounting records to the exported data (applies to transactions export only) + -A, --statements for the accounting options only, export only statements + -b, --balances traverse the transaction history and show each change in ETH balances + -a, --articulate articulate transactions, traces, logs, and outputs + -R, --cache_traces force the transaction's traces into the cache + -U, --count only available for --appearances mode, if present, return only the number of records + -c, --first_record uint the first record to process + -e, --max_records uint the maximum number of records to process (default 250) + -N, --relevant for log and accounting export only, export only logs relevant to one of the given export addresses + -m, --emitter strings for log export only, export only logs if emitted by one of these address(es) + -B, --topic strings for log export only, export only logs with this topic(s) + -P, --asset strings for the accounting options only, export statements only for this asset + -f, --flow string for the accounting options only, export statements with incoming, outgoing, or zero value + One of [ in | out | zero ] + -y, --factory for --traces only, report addresses created by (or self-destructed by) the given address(es) + -u, --unripe export transactions labeled upripe (i.e. less than 28 blocks old) + -E, --reversed produce results in reverse chronological order + -z, --no_zero for the --count option only, suppress the display of zero appearance accounts + -F, --first_block uint first block to process (inclusive) + -L, --last_block uint last block to process (inclusive) + -H, --ether specify value in ether + -o, --cache force the results of the query into the cache + -D, --decache removes related items from the cache + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - An address must be either an ENS name or start with '0x' and be forty-two characters long. + - Articulating the export means turn the EVM's byte data into human-readable text (if possible). + - For the --logs option, you may optionally specify one or more --emitter, one or more --topics, or both. + - The --logs option is significantly faster if you provide an --emitter or a --topic. + - Neighbors include every address that appears in any transaction in which the export address also appears. + - If present, --first_/--last_block are applied, followed by user-supplied filters such as asset or topic, followed by --first_/--max_record if present. + - The --first_record and --max_record options are zero-based (as are the block options). + - The _block and _record filters are ignored when used with the --count option. + - If the --reversed option is present, the appearance list is reversed prior to all processing (including filtering). + - The --decache option will remove all cache items (blocks, transactions, traces, etc.) for the given address(es). +``` + +Data models produced by this tool: + +- [appearance](/data-model/accounts/#appearance) +- [monitor](/data-model/accounts/#monitor) +- [appearancecount](/data-model/accounts/#appearancecount) +- [statement](/data-model/accounts/#statement) +- [transaction](/data-model/chaindata/#transaction) +- [transfer](/data-model/chaindata/#transfer) +- [receipt](/data-model/chaindata/#receipt) +- [log](/data-model/chaindata/#log) +- [trace](/data-model/chaindata/#trace) +- [traceaction](/data-model/chaindata/#traceaction) +- [traceresult](/data-model/chaindata/#traceresult) +- [token](/data-model/chainstate/#token) +- [function](/data-model/other/#function) +- [parameter](/data-model/other/#parameter) + +Links: + +- [api docs](/api/#operation/accounts-export) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/export) + diff --git a/docs/content/readmes/accounts-list.md b/docs/content/readmes/accounts-list.md new file mode 100644 index 0000000000..784230c074 --- /dev/null +++ b/docs/content/readmes/accounts-list.md @@ -0,0 +1,56 @@ +## chifra list + + +`chifra list` takes one or more addresses, queries the index of appearances, and builds TrueBlocks +monitors. A TrueBlocks monitor is a file that contains blockNumber.transactionId pairs (transaction +identifiers) representing the history of the address. + +Because TrueBlocks only extracts data from the Ethereum node when it's requested, the first time +you list an address it takes about a minute. Subsequent queries are much faster because TrueBlocks +caches the results. + +Note that `chifra list` only queries the index, it does not extract the full transactional details. +You may use `chifra export` for that. + +```[plaintext] +Purpose: + List every appearance of an address anywhere on the chain. + +Usage: + chifra list [flags]
[address...] + +Arguments: + addrs - one or more addresses (0x...) to list (required) + +Flags: + -U, --count display only the count of records for each monitor + -z, --no_zero for the --count option only, suppress the display of zero appearance accounts + -b, --bounds report first and last block this address appears + -u, --unripe list transactions labeled upripe (i.e. less than 28 blocks old) + -s, --silent freshen the monitor only (no reporting) + -c, --first_record uint the first record to process + -e, --max_records uint the maximum number of records to process (default 250) + -E, --reversed produce results in reverse chronological order + -F, --first_block uint first block to export (inclusive, ignored when freshening) + -L, --last_block uint last block to export (inclusive, ignored when freshening) + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - An address must be either an ENS name or start with '0x' and be forty-two characters long. + - No other options are permitted when --silent is selected. +``` + +Data models produced by this tool: + +- [appearance](/data-model/accounts/#appearance) +- [monitor](/data-model/accounts/#monitor) +- [appearancecount](/data-model/accounts/#appearancecount) +- [bounds](/data-model/accounts/#bounds) + +Links: + +- [api docs](/api/#operation/accounts-list) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/list) + diff --git a/docs/content/readmes/accounts-monitors.md b/docs/content/readmes/accounts-monitors.md new file mode 100644 index 0000000000..bbb29ce36c --- /dev/null +++ b/docs/content/readmes/accounts-monitors.md @@ -0,0 +1,88 @@ +## chifra monitors + + +`chifra monitors` has two purposes: (1) to display information about the current set of monitors, and (2) +to `--watch` a set of addresses. The `--watch` function allows one to "follow" an address (or set +of addresses) and keep an off-chain database fresh. + +### Crud commands + +`chifra list` creates a new monitor. See that tool's help file for more information. + +The `chifra monitors --delete` command deletes (or `--undelete` if already deleted) an address but does not remove it from your hard drive. The monitor is marked as being deleted, making it invisible to other tools. + +Use the `--remove` command to permanently remove a monitor from your computer. This is an +irreversible operation and requires the monitor to have been previously deleted. + +The `--decache` option will remove not only the monitor but all of the cached data associated with +the monitor (for example, transactions or traces). This is an irreversible operation (except +for the fact that the cache can be easily re-created with `chifra list
`). The monitor need not have been previously deleted. + +### Watching addresses + +The `--watch` command is special. It starts a long-running process that continually reads the blockchain looking for appearances of the addresses it is instructed to watch. It command requires two additional parameters: `--watchlist ` and `--commands `. The `--watchlist` file is simply a list of addresses or ENS names, one per line: + +``` +0x5e349eca2dc61abcd9dd99ce94d04136151a09ee +trueblocks.eth +0x855b26bc8ebabcdbefe82ee5e9d40d20a1a4c11f +etc. +``` + +You may monitor as many addresses as you wish, however, if the commands you specify take longer than the `--sleep` amount you specify (14 seconds by default), the results are undefined. (Adjust `--sleep` if necessary.) + +The `--commands` file may contain a list of any valid `chifra` command that operates on addresses. (Currently `export`, `list`, `state`, `tokens`.) Each command in the `--commands` file is executed once for each address in the `--watchlist` file. The `--commands` file may contain any number of commands, one per line with the above proviso. For example: + +``` +chifra list [{ADDRESS}] +chifra export --logs [{ADDRESS}] +etc. +``` + +The `[{ADDRESS}]` token is a stand-in for all addresses in the `--watchlist`. Addresses are processed in groups of `batch_size` (default 8). + +Invalid commands or invalid addresses are ignored. If a command fails, the process continues with the next command. If a command fails for a particular address, the process continues with the next address. A warning is generated. + +```[plaintext] +Purpose: + Add, remove, clean, and list address monitors. + +Usage: + chifra monitors [flags]
[address...] + +Arguments: + addrs - one or more addresses (0x...) to process + +Flags: + --delete delete a monitor, but do not remove it + --undelete undelete a previously deleted monitor + --remove remove a previously deleted monitor + -C, --clean clean (i.e. remove duplicate appearances) from monitors + -l, --list list monitors in the cache (--verbose for more detail) + -w, --watch continually scan for new blocks and extract data as per the command file + -a, --watchlist string available with --watch option only, a file containing the addresses to watch + -c, --commands string available with --watch option only, the file containing the list of commands to apply to each watched address + -b, --batch_size uint available with --watch option only, the number of monitors to process in each batch (default 8) + -s, --sleep float available with --watch option only, the number of seconds to sleep between runs (default 14) + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - An address must be either an ENS name or start with '0x' and be forty-two characters long. + - If no address is presented to the --clean command, all existing monitors will be cleaned. + - The --watch option requires two additional parameters to be specified: --watchlist and --commands. + - Addresses provided on the command line are ignored in --watch mode. + - Providing the value existing to the --watchlist monitors all existing monitor files (see --list). +``` + +Data models produced by this tool: + +- [monitor](/data-model/accounts/#monitor) +- [monitorclean](/data-model/admin/#monitorclean) + +Links: + +- [api docs](/api/#operation/accounts-monitors) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/monitors) + diff --git a/docs/content/readmes/accounts-names.md b/docs/content/readmes/accounts-names.md new file mode 100644 index 0000000000..80fb2d7bd4 --- /dev/null +++ b/docs/content/readmes/accounts-names.md @@ -0,0 +1,57 @@ +## chifra names + + +`chifra names` is a surprisingly useful tool. It allows one to associate textual names with Ethereum +addresses. One may ask why this is necessary given that ENS exists. The answer is a single +word: "privacy". ENS names are public. In many cases, users desire to keep personal addresses +private. Try to do this on a website. + +Like `chifra abis`, this tool is useful from the command line but is primarily used in support of +other tools, especially `chifra export` where naming addresses becomes the single best way to +turn unintelligible blockchain data into understandable information. + +The various options allow you to search and filter the results. The `tags` option is used primarily +by the TrueBlocks explorer. + +You may use the TrueBlocks explorer to manage (add, edit, delete) address-name associations. + +```[plaintext] +Purpose: + Query addresses or names of well known accounts. + +Usage: + chifra names [flags] [term...] + +Arguments: + terms - a space separated list of one or more search terms (required) + +Flags: + -e, --expand expand search to include all fields (search name, address, and symbol otherwise) + -m, --match_case do case-sensitive search + -a, --all include all (including custom) names in the search + -c, --custom include only custom named accounts in the search + -p, --prefund include prefund accounts in the search + -s, --addr display only addresses in the results (useful for scripting, assumes --no_header) + -g, --tags export the list of tags and subtags only + -C, --clean clean the data (addrs to lower case, sort by addr) + -r, --regular only available with --clean, cleans regular names database + -d, --dry_run only available with --clean or --autoname, outputs changes to stdout instead of updating databases + -A, --autoname string an address assumed to be a token, added automatically to names database if true + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - The tool will accept up to three terms, each of which must match against any field in the database. + - The --match_case option enables case sensitive matching. +``` + +Data models produced by this tool: + +- [name](/data-model/accounts/#name) + +Links: + +- [api docs](/api/#operation/accounts-names) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/names) + diff --git a/docs/content/readmes/admin-chunks.md b/docs/content/readmes/admin-chunks.md new file mode 100644 index 0000000000..c778bff2e7 --- /dev/null +++ b/docs/content/readmes/admin-chunks.md @@ -0,0 +1,64 @@ +## chifra chunks + + +The chifra chunks routine provides tools for interacting with, checking the validity of, cleaning up, +and analyizing the Unchained Index. It provides options to list pins, the Manifest, summary data +on the index, Bloom filters, addresses, and appearances. While still in its early stages, this +tool will eventually allow users to clean their local index, clean their remote index, study +the indexes, etc. Stay tuned. + +```[plaintext] +Purpose: + Manage, investigate, and display the Unchained Index. + +Usage: + chifra chunks [flags] [blocks...] [address...] + +Arguments: + mode - the type of data to process (required) + One of [ manifest | index | blooms | addresses | appearances | stats ] + blocks - an optional list of blocks to intersect with chunk ranges + +Flags: + -c, --check check the manifest, index, or blooms for internal consistency + -i, --pin pin the manifest or each index chunk and bloom + -p, --publish publish the manifest to the Unchained Index smart contract + -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract + -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk + -F, --first_block uint first block to process (inclusive) + -L, --last_block uint last block to process (inclusive) + -m, --max_addrs uint the max number of addresses to process in a given chunk + -d, --deep if true, dig more deeply during checking (manifest only) + -s, --sleep float for --remote pinning only, seconds to sleep between API calls + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - Mode determines which type of data to display or process. + - Certain options are only available in certain modes. + - If blocks are provided, only chunks intersecting with those blocks are displayed. + - The --truncate option updates the manifest and removes local data, but does not alter remote pins. + - The --belongs option is only available in the index mode. + - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. + - The --pin option requires a locally running IPFS node or a pinning service API key. + - The --publish option requires a private key. +``` + +Data models produced by this tool: + +- [appearance](/data-model/accounts/#appearance) +- [manifest](/data-model/admin/#manifest) +- [chunkrecord](/data-model/admin/#chunkrecord) +- [chunkindex](/data-model/admin/#chunkindex) +- [chunkbloom](/data-model/admin/#chunkbloom) +- [chunkaddress](/data-model/admin/#chunkaddress) +- [chunkstats](/data-model/admin/#chunkstats) +- [reportcheck](/data-model/admin/#reportcheck) +- [chunkpinreport](/data-model/admin/#chunkpinreport) + +Links: + +- [api docs](/api/#operation/admin-chunks) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/chunks) + diff --git a/docs/content/readmes/admin-config.md b/docs/content/readmes/admin-config.md new file mode 100644 index 0000000000..570bfe3152 --- /dev/null +++ b/docs/content/readmes/admin-config.md @@ -0,0 +1,37 @@ +## chifra config + + +The `chifra config` program allows you to manage the various TrueBlocks caches. You may list all of the +caches, some of the cache, or even individual caches either in terse or full detail. The cache of +interest is specified with the `modes` option. + +TrueBlocks maintains caches for the index of address appearances, named addresses, abi files, as +well as other data including blockchain data, and address monitors. + +```[plaintext] +Purpose: + Report on and edit the configuration of the TrueBlocks system. + +Usage: + chifra config [flags] + +Arguments: + mode - either show or edit the configuration + One of [ show | edit ] + +Flags: + -a, --paths show the configuration paths for the system + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen +``` + +Data models produced by this tool: + +- [chain](/data-model/admin/#chain) + +Links: + +- [api docs](/api/#operation/admin-config) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/config) + diff --git a/docs/content/readmes/admin-daemon.md b/docs/content/readmes/admin-daemon.md new file mode 100644 index 0000000000..1cf7439aec --- /dev/null +++ b/docs/content/readmes/admin-daemon.md @@ -0,0 +1,50 @@ +## chifra daemon + + +`chifra daemon` manages chifra's long-running processes include its JSON API server. Each of the +`chifra` commands along with all of its options, are provided not only by the command line, but +also the API server. We call this process the `flame` server, which is written in Go. + +In the future, this daemon may also manage other long-running processes. + +Another way to get help to run `chifra --help` or `chifra --help` on your command line. +See below for an example of converting command line options to a call to the API. There's a +one-to-one correspondence between the command line tools and options and the API routes and +their options. + +```[plaintext] +Purpose: + Initalize and control long-running processes such as the API and the scrapers. + +Usage: + chifra daemon [flags] + +Aliases: + daemon, serve + +Flags: + -p, --port string specify the server's port (default ":8080") + -a, --api string instruct the node to start the API server + One of [ off | on ] (default "on") + -s, --scrape string start the scraper, initialize it with either just blooms or entire index, generate for new blocks + One of [ off | blooms | index ] + -m, --monitor instruct the node to start the monitors tool + -g, --grpc run gRPC server to serve names + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - To start API open terminal window and run chifra daemon. + - See the API documentation (https://trueblocks.io/api) for more information. +``` + +Data models produced by this tool: + +- none + +Links: + +- no api for this command +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/daemon) + diff --git a/docs/content/readmes/admin-init.md b/docs/content/readmes/admin-init.md new file mode 100644 index 0000000000..242991aa2a --- /dev/null +++ b/docs/content/readmes/admin-init.md @@ -0,0 +1,55 @@ +## chifra init + + +When invoked, `chifra init` reads a value from a smart contract called **The Unchained Index** +(0x0c316b7042b419d07d343f2f4f5bd54ff731183d). + +This value (`manifestHashMap`) is an IPFS hash pointing to a pinned file (called the Manifest) that +contains a large collection of other IPFS hashes. These other hashes point to each of the Bloom +filter and Index Chunk. TrueBlocks periodically publishes the Manifest's hash to the smart contract. +This makes the index available for our software to use and impossible for us to withhold. Both of +these aspects of the manifest are by design. + +If you stop `chifra init` before it finishes, it will pick up again where it left off the next +time you run it. + +Certain parts of the system (`chifra list` and `chifra export` for example) if you have not +previously run `chifra init` or `chifra scrape`. You will be warned by the system until it's +satified. + +If you run `chifra init` and allow it to complete, the next time you run `chifra scrape`, it will +start where `init` finished. This means that only the blooms will be stored on your hard drive. +Subsequent scraping will produce both chunks and blooms, although you can, if you wish delete +chunks that are not being used. You may periodically run `chifra init` if you prefer not to scrape. + +```[plaintext] +Purpose: + Initialize the TrueBlocks system by downloading the Unchained Index from IPFS. + +Usage: + chifra init [flags] + +Flags: + -a, --all in addition to Bloom filters, download full index chunks (recommended) + -d, --dry_run display the results of the download without actually downloading + -F, --first_block uint do not download any chunks earlier than this block + -s, --sleep float seconds to sleep between downloads + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - If run with no options, this tool will download or freshen only the Bloom filters. + - The --first_block option will fall back to the start of the containing chunk. + - You may re-run the tool as often as you wish. It will repair or freshen the index. +``` + +Data models produced by this tool: + +- [manifest](/data-model/admin/#manifest) +- [chunkrecord](/data-model/admin/#chunkrecord) + +Links: + +- [api docs](/api/#operation/admin-init) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/init) + diff --git a/docs/content/readmes/admin-scrape.md b/docs/content/readmes/admin-scrape.md new file mode 100644 index 0000000000..f6f21662f0 --- /dev/null +++ b/docs/content/readmes/admin-scrape.md @@ -0,0 +1,108 @@ +## chifra scrape + + +The `chifra scrape` application creates TrueBlocks' chunked index of address appearances -- the +fundamental data structure of the entire system. It also, optionally, pins each chunk of the index +to IPFS. + +`chifra scrape` is a long running process, therefore we advise you run it as a service or in terminal +multiplexer such as `tmux`. You may start and stop `chifra scrape` as needed, but doing so means the +scraper will not be keeping up with the front of the blockchain. The next time it starts, it will +have to catch up to the chain, a process that may take several hours depending on how long ago it +was last run. See the section below and the "Papers" section of our website for more information +on how the scraping process works and prerequisites for it proper operation. + +You may adjust the speed of the index creation with the `--sleep` and `--block_cnt` options. On +some machines, or when running against some EVM node software, the scraper may overburden the +hardware. Slowing things down will ensure proper operation. Finally, you may optionally `--pin` +each new chunk to IPFS which naturally shards the database among all users. By default, pinning +is against a locally running IPFS node, but the `--remote` option allows pinning to an IPFS +pinning service such as Pinata or Estuary. + +```[plaintext] +Purpose: + Scan the chain and update the TrueBlocks index of appearances. + +Usage: + chifra scrape [flags] + +Flags: + -n, --block_cnt uint maximum number of blocks to process per pass (default 2000) + -i, --pin pin new chunks (requires locally-running IPFS daemon or --remote) + -r, --remote pin new chunks to the gateway (requires pinning service keys) + -s, --sleep float seconds to sleep between scraper passes (default 14) + -l, --start_block uint first block to visit when scraping (snapped back to most recent snap_to_grid mark) + -v, --verbose enable verbose output + -h, --help display this help screen +``` + +Data models produced by this tool: + +- [manifest](/data-model/admin/#manifest) +- [chunkrecord](/data-model/admin/#chunkrecord) + +Links: + +- [api docs](/api/#operation/admin-scrape) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/scrape) + +### configuration + +Each of the following additional configurable command line options are available. + +**Configuration file:** `$CONFIG/$CHAIN/blockScrape.toml` +**Configuration group:** `[settings]` + +| Item | Type | Default | Description / Default | +| ------------------ | ------------ | ------------ | --------- | +| apps_per_chunk | uint64 | 200000 | the number of appearances to build into a chunk before consolidating it | +| snap_to_grid | uint64 | 100000 | an override to apps_per_chunk to snap-to-grid at every modulo of this value, this allows easier corrections to the index | +| first_snap | uint64 | 0 | the first block at which snap_to_grid is enabled | +| unripe_dist | uint64 | 28 | the distance (in blocks) from the front of the chain under which (inclusive) a block is considered unripe | +| channel_count | uint64 | 20 | number of concurrent processing channels | +| allow_missing | bool | true | do not report errors for blockchains that contain blocks with zero addresses | + + +These items may be set in three ways, each overridding the preceeding method: + +-- in the above configuration file under the `[settings]` group, +-- in the environment by exporting the configuration item as UPPER_CASE, without underbars, and prepended with TB_SETTINGS_, or +-- on the command line using the configuration item with leading dashes (i.e., `--name`). + + +### further information + +Each time `chifra scrape` runs, it begins at the last block it completed processing (plus one). With +each pass, the scraper descends as deeply as is possible into each block's data. (This is why +TrueBlocks requires a `--tracing` node.) As the scraper encounters appearances of address in the +block's data, it adds those appearance to a growing index. Periodically (after processing the the +block that contains the 2,000,000th appearance), the system consolidates an **index chunk**. + +An **index chunk** is a portion of the index containing approximately 2,000,000 records (although, +this number is adjustable for different chains). As part of the consolidation, the scraper creates +a Bloom filter representing the set membership in the associated index portion. The Bloom filters +are an order of magnitude smaller than the index chunks. The system then pushes both the index +chunk and the Bloom filter to IPFS. In this way, TrueBlocks creates an immutable, uncapturable +index of appearances that can be used not only by TrueBlocks, but any member of the community who +needs it. (Hint: We all need it.) + +Users of the [TrueBlocks Explorer](https://github.com/TrueBlocks/trueblocks-explorer) (or any other +software) may subsequently download the Bloom filters, query them to determine which **index +chunks** need to be downloaded, and thereby build a historical list of transactions for a given +address. This is accomplished while imposing a minimum amount of resource requirement on the end +user's machine. + +Recently, we enabled the ability for the end user to pin these downloaded index chunks and blooms +on their own machines. The user needs the data for the software to operate--sharing requires +minimal effort and makes the data available to other people. Everyone is better off. A +naturally-occuring network effect. + +### prerequisites + +`chifra scrape` works with any EVM-based blockchain, but does not currently work without a "tracing, +archive" RPC endpoint. The Erigon blockchain node, given its minimal disc footprint for an archive +node and its support of the required `trace_` endpoint routines, is highly recommended. + +Please [see this article](https://trueblocks.io/blog/a-long-winded-explanation-of-trueblocks/) for +more information about running the scraper and building and sharing the index of appearances. + diff --git a/docs/content/readmes/admin-status.md b/docs/content/readmes/admin-status.md new file mode 100644 index 0000000000..5cda5687b9 --- /dev/null +++ b/docs/content/readmes/admin-status.md @@ -0,0 +1,38 @@ +## chifra status + + +```[plaintext] +Purpose: + Report on the state of the internal binary caches. + +Usage: + chifra status [mode...] [flags] + +Arguments: + modes - the (optional) name of the binary cache to report on, terse otherwise + One or more of [ index | blooms | blocks | transactions | traces | logs | statements | results | state | tokens | monitors | names | abis | slurps | staging | unripe | maps | some | all ] + +Flags: + -c, --first_record uint the first record to process + -e, --max_records uint the maximum number of records to process (default 10000) + -a, --chains include a list of chain configurations in the output + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - The some mode includes index, monitors, names, slurps, and abis. + - If no mode is supplied, a terse report is generated. +``` + +Data models produced by this tool: + +- [status](/data-model/admin/#status) +- [cacheitem](/data-model/admin/#cacheitem) +- [chain](/data-model/admin/#chain) + +Links: + +- [api docs](/api/#operation/admin-status) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/status) + diff --git a/docs/content/readmes/chaindata-blocks.md b/docs/content/readmes/chaindata-blocks.md new file mode 100644 index 0000000000..97ac26c208 --- /dev/null +++ b/docs/content/readmes/chaindata-blocks.md @@ -0,0 +1,71 @@ +## chifra blocks + + +The `chifra blocks` tool retrieves block data from your Ethereum node or, if previously cached, from the +TrueBlocks cache. You may specify multiple blocks per invocation. + +By default, `chifra blocks` queries the full transactional details of the block (including receipts). +You may optionally retrieve only the transaction hashes in the block (which is significantly faster). +Additionally, you may also use this tool to retrieve uncle blocks at a give height. + +Another useful feature of `chifra blocks` is the ability to extract address appearances from a block. +TrueBlocks uses a similar feature internally to build its index of appearances. This type of data +is very insightful when studying end user behavior and chain-wide adoption analysis. + +```[plaintext] +Purpose: + Retrieve one or more blocks from the chain or local cache. + +Usage: + chifra blocks [flags] [block...] + +Arguments: + blocks - a space-separated list of one or more block identifiers (required) + +Flags: + -e, --hashes display only transaction hashes, default is to display full transaction detail + -c, --uncles display uncle blocks (if any) instead of the requested block + -t, --traces export the traces from the block as opposed to the block data + -u, --uniq display a list of uniq address appearances per transaction + -f, --flow string for the --uniq option only, export only from or to (including trace from or to) + One of [ from | to | reward ] + -l, --logs display only the logs found in the block(s) + -m, --emitter strings for the --logs option only, filter logs to show only those logs emitted by the given address(es) + -B, --topic strings for the --logs option only, filter logs to show only those with this topic(s) + -a, --articulate for the --logs option only, articulate the retrieved data if ABIs can be found + -r, --big_range uint for the --logs option only, allow for block ranges larger than 500 (default 500) + -U, --count display the number of the lists of appearances for --addrs or --uniq + -w, --raw report JSON data from the source with minimal processing + -o, --cache force the results of the query into the cache + -D, --decache removes related items from the cache + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - Blocks is a space-separated list of values, a start-end range, a special, or any combination. + - Blocks may be specified as either numbers or hashes. + - Special blocks are detailed under chifra when --list. + - With the --logs option, optionally specify one or more --emitter, one or more --topics, either or both. + - The --logs option is significantly faster if you provide an --emitter and/or a --topic. + - Multiple topics match on topic0, topic1, and so on, not on different topic0's. + - For the --logs option, large block ranges may crash the node, use --big_range to specify a larger range. + - The --decache option removes the block(s), all transactions in those block(s), and all traces in those transactions from the cache. +``` + +Data models produced by this tool: + +- [appearance](/data-model/accounts/#appearance) +- [block](/data-model/chaindata/#block) +- [log](/data-model/chaindata/#log) +- [logfilter](/data-model/chaindata/#logfilter) +- [trace](/data-model/chaindata/#trace) +- [traceaction](/data-model/chaindata/#traceaction) +- [traceresult](/data-model/chaindata/#traceresult) +- [blockcount](/data-model/chaindata/#blockcount) + +Links: + +- [api docs](/api/#operation/chaindata-blocks) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/blocks) + diff --git a/docs/content/readmes/chaindata-logs.md b/docs/content/readmes/chaindata-logs.md new file mode 100644 index 0000000000..29a87087b9 --- /dev/null +++ b/docs/content/readmes/chaindata-logs.md @@ -0,0 +1,45 @@ +## chifra logs + + +`chifra logs` returns the given transaction's logs. You may specify multiple transaction identifiers +per invocation. + +The `--articulate` option fetches the ABI from each encountered smart contract to better describe +the reported data. The `--topic` and `--source` options allow you to filter your results. + +```[plaintext] +Purpose: + Retrieve logs for the given transaction(s). + +Usage: + chifra logs [flags] [tx_id...] + +Arguments: + transactions - a space-separated list of one or more transaction identifiers (required) + +Flags: + -a, --articulate articulate the retrieved data if ABIs can be found + -w, --raw report JSON data from the source with minimal processing + -o, --cache force the results of the query into the cache + -D, --decache removes related items from the cache + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - The transactions list may be one or more transaction hashes, blockNumber.transactionID pairs, or a blockHash.transactionID pairs. + - This tool checks for valid input syntax, but does not check that the transaction requested actually exists. + - If the queried node does not store historical state, the results for most older transactions are undefined. + - If you specify a 32-byte hash, it will be assumed to be a transaction hash, if it is not, the hash will be used as a topic. +``` + +Data models produced by this tool: + +- [log](/data-model/chaindata/#log) +- [logfilter](/data-model/chaindata/#logfilter) + +Links: + +- [api docs](/api/#operation/chaindata-logs) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/logs) + diff --git a/docs/content/readmes/chaindata-receipts.md b/docs/content/readmes/chaindata-receipts.md new file mode 100644 index 0000000000..3e53c315e2 --- /dev/null +++ b/docs/content/readmes/chaindata-receipts.md @@ -0,0 +1,47 @@ +## chifra receipts + + +`chifra receipts` returns the given transaction's receipt. You may specify multiple transaction identifiers +per invocation. + +The `--articulate` option fetches the ABI from each encountered smart contract (including those +encountered in a trace--if the `--trace` option is enabled) to better describe the reported data. + +Generally speaking, this tool is less useful than others as you may report the same data using +`chifra transactions` and more focused data using `chifra logs`. It is included here for +completeness, as the `receipt` is a fundamental data structure in Ethereum. + +```[plaintext] +Purpose: + Retrieve receipts for the given transaction(s). + +Usage: + chifra receipts [flags] [tx_id...] + +Arguments: + transactions - a space-separated list of one or more transaction identifiers (required) + +Flags: + -a, --articulate articulate the retrieved data if ABIs can be found + -w, --raw report JSON data from the source with minimal processing + -o, --cache force the results of the query into the cache + -D, --decache removes related items from the cache + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - The transactions list may be one or more transaction hashes, blockNumber.transactionID pairs, or a blockHash.transactionID pairs. + - This tool checks for valid input syntax, but does not check that the transaction requested actually exists. + - If the queried node does not store historical state, the results for most older transactions are undefined. +``` + +Data models produced by this tool: + +- [receipt](/data-model/chaindata/#receipt) + +Links: + +- [api docs](/api/#operation/chaindata-receipts) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/receipts) + diff --git a/docs/content/readmes/chaindata-traces.md b/docs/content/readmes/chaindata-traces.md new file mode 100644 index 0000000000..11da772825 --- /dev/null +++ b/docs/content/readmes/chaindata-traces.md @@ -0,0 +1,53 @@ +## chifra traces + + +The `chifra traces` tool retrieves a transaction's traces. You may specify multiple transaction +identifiers per invocation. + +The `--articulate` option fetches the ABI from each encountered smart contract to better describe +the reported data. + +The `--filter` option calls your node's `trace_filter` routine (if available) using a bang-separated +string of the same values used by `trace_fitler`. + +```[plaintext] +Purpose: + Retrieve traces for the given transaction(s). + +Usage: + chifra traces [flags] [tx_id...] + +Arguments: + transactions - a space-separated list of one or more transaction identifiers (required) + +Flags: + -a, --articulate articulate the retrieved data if ABIs can be found + -f, --filter string call the node's trace_filter routine with bang-separated filter + -U, --count show the number of traces for the transaction only (fast) + -w, --raw report JSON data from the source with minimal processing + -o, --cache force the results of the query into the cache + -D, --decache removes related items from the cache + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - The transactions list may be one or more transaction hashes, blockNumber.transactionID pairs, or a blockHash.transactionID pairs. + - This tool checks for valid input syntax, but does not check that the transaction requested actually exists. + - If the queried node does not store historical state, the results for most older transactions are undefined. + - A bang separated filter has the following fields (at least one of which is required) and is separated with a bang (!): fromBlk, toBlk, fromAddr, toAddr, after, count. +``` + +Data models produced by this tool: + +- [trace](/data-model/chaindata/#trace) +- [traceaction](/data-model/chaindata/#traceaction) +- [traceresult](/data-model/chaindata/#traceresult) +- [tracecount](/data-model/chaindata/#tracecount) +- [tracefilter](/data-model/chaindata/#tracefilter) + +Links: + +- [api docs](/api/#operation/chaindata-traces) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/traces) + diff --git a/docs/content/readmes/chaindata-transactions.md b/docs/content/readmes/chaindata-transactions.md new file mode 100644 index 0000000000..edfa5c5d14 --- /dev/null +++ b/docs/content/readmes/chaindata-transactions.md @@ -0,0 +1,62 @@ +## chifra transactions + + +The `chifra transactions` tool retrieves transactions directly from the Ethereum node (using the `--raw` +option) or from the TrueBlocks cache (if present). You may specify multiple transaction identifiers +per invocation. Unlike the Ethereum RPC, the reported transactions include the transaction's receipt +and generated logs. + +The `--articulate` option fetches the ABI from each encountered smart contract (including those +encountered in a trace--if the `--trace` option is enabled) to better describe the reported data. + +The `--trace` option attaches an array transaction traces to the output (if the node you're querying +has --tracing enabled), while the `--uniq` option displays a list of uniq address appearances +instead of the underlying data (including uniq addresses in traces if enabled). + +```[plaintext] +Purpose: + Retrieve one or more transactions from the chain or local cache. + +Usage: + chifra transactions [flags] [tx_id...] + +Arguments: + transactions - a space-separated list of one or more transaction identifiers (required) + +Flags: + -a, --articulate articulate the retrieved data if ABIs can be found + -t, --traces include the transaction's traces in the results + -u, --uniq display a list of uniq addresses found in the transaction + -f, --flow string for the uniq option only, export only from or to (including trace from or to) + One of [ from | to ] + -l, --logs display only the logs found in the transaction(s) + -m, --emitter strings for the --logs option only, filter logs to show only those logs emitted by the given address(es) + -B, --topic strings for the --logs option only, filter logs to show only those with this topic(s) + -A, --account_for string reconcile the transaction as per the provided address + -H, --ether specify value in ether + -w, --raw report JSON data from the source with minimal processing + -o, --cache force the results of the query into the cache + -D, --decache removes related items from the cache + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - The transactions list may be one or more transaction hashes, blockNumber.transactionID pairs, or a blockHash.transactionID pairs. + - This tool checks for valid input syntax, but does not check that the transaction requested actually exists. + - If the queried node does not store historical state, the results for most older transactions are undefined. + - The --traces option, when used with --account_for, will descend into traces to complete reconciliations. + - The --decache option removes the all transaction(s) and all traces in those transactions from the cache. +``` + +Data models produced by this tool: + +- [statement](/data-model/accounts/#statement) +- [transaction](/data-model/chaindata/#transaction) +- [transfer](/data-model/chaindata/#transfer) + +Links: + +- [api docs](/api/#operation/chaindata-transactions) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/transactions) + diff --git a/docs/content/readmes/chaindata-when.md b/docs/content/readmes/chaindata-when.md new file mode 100644 index 0000000000..1cf11c1781 --- /dev/null +++ b/docs/content/readmes/chaindata-when.md @@ -0,0 +1,57 @@ +## chifra when + + +The `chifra when` tool answers one of two questions: (1) "At what date and time did a given block +occur?" or (2) "What block occurred at or before a given date and time?" + +In the first case, supply a block number or hash and the date and time of that block are displayed. +In the later case, supply a date (and optionally a time) and the block number that occurred at or +just prior to that date is displayed. + +The values for `date` and `time` are specified in JSON format. `hour`/`minute`/`second` are +optional, and if omitted, default to zero in each case. Block numbers may be specified as either +integers or hexadecimal number or block hashes. You may specify any number of dates and/or blocks +per invocation. + +```[plaintext] +Purpose: + Find block(s) based on date, blockNum, timestamp, or 'special'. + +Usage: + chifra when [flags] < block | date > [ block... | date... ] + +Arguments: + blocks - one or more dates, block numbers, hashes, or special named blocks (see notes) + +Flags: + -l, --list export a list of the 'special' blocks + -t, --timestamps display or process timestamps + -U, --count with --timestamps only, returns the number of timestamps in the cache + -r, --repair with --timestamps only, repairs block(s) in the block range by re-querying from the chain + -c, --check with --timestamps only, checks the validity of the timestamp data + -u, --update with --timestamps only, bring the timestamp database forward to the latest block + -d, --deep with --timestamps --check only, verifies timestamps from on chain (slow) + -o, --cache force the results of the query into the cache + -D, --decache removes related items from the cache + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - The block list may contain any combination of number, hash, date, special named blocks. + - Block numbers, timestamps, or dates in the future are estimated with 13 second blocks. + - Dates must be formatted in JSON format: YYYY-MM-DD[THH[:MM[:SS]]]. +``` + +Data models produced by this tool: + +- [block](/data-model/chaindata/#block) +- [namedblock](/data-model/chaindata/#namedblock) +- [timestamp](/data-model/chaindata/#timestamp) +- [timestampcount](/data-model/chaindata/#timestampcount) + +Links: + +- [api docs](/api/#operation/chaindata-when) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/when) + diff --git a/docs/content/readmes/chainstate-state.md b/docs/content/readmes/chainstate-state.md new file mode 100644 index 0000000000..df629529ad --- /dev/null +++ b/docs/content/readmes/chainstate-state.md @@ -0,0 +1,56 @@ +## chifra state + + +The `chifra state` tool retrieves the balance of an address (or list of addresses) at the given block +(or blocks). Specify multiple addresses and/or multiple blocks if you wish, but you must specify +at least one address. If no block is specified, the latest block is reported. + +You may also query to see if an address is a smart contract as well as retrieve a contract's +byte code. + +```[plaintext] +Purpose: + Retrieve account balance(s) for one or more addresses at given block(s). + +Usage: + chifra state [flags]
[address...] [block...] + +Arguments: + addrs - one or more addresses (0x...) from which to retrieve balances (required) + blocks - an optional list of one or more blocks at which to report balances, defaults to 'latest' + +Flags: + -p, --parts strings control which state to export + One or more of [ none | some | all | balance | nonce | code | proxy | deployed | accttype ] + -c, --changes only report a balance when it changes from one block to the next + -z, --no_zero suppress the display of zero balance accounts + -l, --call string call a smart contract with a solidity syntax, a four-byte and parameters, or encoded call data + -a, --articulate for the --call option only, articulate the retrieved data if ABIs can be found + -r, --proxy_for string for the --call option only, redirects calls to this implementation + -H, --ether specify value in ether + -o, --cache force the results of the query into the cache + -D, --decache removes related items from the cache + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - An address must be either an ENS name or start with '0x' and be forty-two characters long. + - Blocks is a space-separated list of values, a start-end range, a special, or any combination. + - If the queried node does not store historical state, the results are undefined. + - Special blocks are detailed under chifra when --list. + - Balance is the default mode. To select a single mode use none first, followed by that mode. + - Valid parameters for --call include Solidity-like syntax: balanceOf(0x316b...183d), a four-byte followed by parameters: 0x70a08231(0x316b...183d), or encoded input data. + - You may specify multiple modes on a single line. +``` + +Data models produced by this tool: + +- [result](/data-model/chainstate/#result) +- [state](/data-model/chainstate/#state) + +Links: + +- [api docs](/api/#operation/chainstate-state) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/state) + diff --git a/docs/content/readmes/chainstate-tokens.md b/docs/content/readmes/chainstate-tokens.md new file mode 100644 index 0000000000..13089d9cd2 --- /dev/null +++ b/docs/content/readmes/chainstate-tokens.md @@ -0,0 +1,58 @@ +## chifra tokens + + +Given the address of an ERC20 token contract, the `chifra tokens` tool reports token balances for one or +more additional addresses. Alternatively, the tool can report the token balances for multiple ERC20 +tokens for a single addresses. + +In normal operation the **first item** in the `address_list` is assumed to be an ERC20 token +contract whose balances are being queried, whereas the remainder of the list is assumed to be +addresses on which to report. + +In `--byAcct` mode, **all addresses** in the `address_list` are assumed to be ERC20 token contracts, +except the final one which is the account whose token balances are reported. + +You may optionally specify one or more blocks at which to report. If no block is specified, the +latest block is assumed. You may also optionally specify which parts of the token data to extract. + +```[plaintext] +Purpose: + Retrieve token balance(s) for one or more addresses at given block(s). + +Usage: + chifra tokens [flags]
[address...] [block...] + +Arguments: + addrs - two or more addresses (0x...), the first is an ERC20 token, balances for the rest are reported (required) + blocks - an optional list of one or more blocks at which to report balances, defaults to 'latest' + +Flags: + -p, --parts strings which parts of the token information to retrieve + One or more of [ name | symbol | decimals | totalSupply | version | all ] + -b, --by_acct consider each address an ERC20 token except the last, whose balance is reported for each token + -c, --changes only report a balance when it changes from one block to the next + -z, --no_zero suppress the display of zero balance accounts + -o, --cache force the results of the query into the cache + -D, --decache removes related items from the cache + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - An address must be either an ENS name or start with '0x' and be forty-two characters long. + - Blocks is a space-separated list of values, a start-end range, a special, or any combination. + - If the token contract(s) from which you request balances are not ERC20 compliant, the results are undefined. + - If the queried node does not store historical state, the results are undefined. + - Special blocks are detailed under chifra when --list. + - If the --parts option is not empty, all addresses are considered tokens and each token's attributes are presented. +``` + +Data models produced by this tool: + +- [token](/data-model/chainstate/#token) + +Links: + +- [api docs](/api/#operation/chainstate-tokens) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/tokens) + diff --git a/docs/content/readmes/other-explore.md b/docs/content/readmes/other-explore.md new file mode 100644 index 0000000000..79e738394b --- /dev/null +++ b/docs/content/readmes/other-explore.md @@ -0,0 +1,32 @@ +## chifra explore + + +`chifra explore` opens Etherscan (and other explorers -- including our own) to the block identifier, +transaction identifier, or address you specify. It's a handy (configurable) way to open an explorer +from the command line, nothing more. + +```[plaintext] +Purpose: + Open a local or remote explorer for one or more addresses, blocks, or transactions. + +Usage: + chifra explore [flags] [term...] + +Arguments: + terms - one or more address, name, block, or transaction identifier + +Flags: + -l, --local open the local TrueBlocks explorer + -g, --google search google excluding popular blockchain explorers + -h, --help display this help screen +``` + +Data models produced by this tool: + +- none + +Links: + +- no api for this command +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/explore) + diff --git a/docs/content/readmes/other-slurp.md b/docs/content/readmes/other-slurp.md new file mode 100644 index 0000000000..93b9e0a27b --- /dev/null +++ b/docs/content/readmes/other-slurp.md @@ -0,0 +1,52 @@ +## chifra slurp + + +`chifra slurp` is the first tool we built in the Ethereum space. It even has its [own website](http://ethslurp.com). + +While it's useful, it has two shortcomings. First, it is fully centralized, pulling its data from +[http://etherscan.io](http://etherscan.io). Second, is that it does not report every transaction +for a given account. This is actually a shortcoming with Etherscan. It's too complicated to explain +here, but see our blog. + +While `chifra slurp` has its shortcomings, it does provides some nice features. You may use it to pull +any transaction initiated by an EOA for example or to explore mining rewards. Visit the above +referenced website for more information. + +```[plaintext] +Purpose: + Fetch data from Etherscan for any address. + +Usage: + chifra slurp [flags]
[address...] [block...] + +Arguments: + addrs - one or more addresses to slurp from Etherscan (required) + blocks - an optional range of blocks to slurp + +Flags: + -t, --types strings which types of transactions to request + One or more of [ ext | int | token | nfts | 1155 | miner | uncles | all ] + -p, --appearances show only the blocknumber.tx_id appearances of the exported transactions + -P, --per_page uint the number of records to request on each page (default 5000) + -s, --sleep float seconds to sleep between requests (default 0.25) + -w, --raw report JSON data from the source with minimal processing + -o, --cache force the results of the query into the cache + -D, --decache removes related items from the cache + -x, --fmt string export format, one of [none|json*|txt|csv] + -v, --verbose enable verbose output + -h, --help display this help screen + +Notes: + - An address must be either an ENS name or start with '0x' and be forty-two characters long. + - Portions of this software are Powered by Etherscan.io APIs. +``` + +Data models produced by this tool: + +- [slurp](/data-model/other/#slurp) + +Links: + +- [api docs](/api/#operation/other-slurp) +- [source code](https://github.com/TrueBlocks/trueblocks-core/tree/master/src/apps/chifra/internal/slurp) + diff --git a/docs/content/tutorials/index.md b/docs/content/tutorials/index.md new file mode 100644 index 0000000000..cfb831d49c --- /dev/null +++ b/docs/content/tutorials/index.md @@ -0,0 +1,10 @@ +--- +title: "Tutorials" +description: "Learn how to use TrueBlocks, chifra, and the Unchained Index." +date: 2023-09-05T19:25:12+02:00 +lastmod: 2023-09-05T19:25:12+02:00 +draft: false +images: [] +--- + +WHAT WHAT? diff --git a/src/apps/chifra/internal/scrape/save_timestamps.go b/src/apps/chifra/internal/scrape/save_timestamps.go index 952e85ee70..f517e2a3fe 100644 --- a/src/apps/chifra/internal/scrape/save_timestamps.go +++ b/src/apps/chifra/internal/scrape/save_timestamps.go @@ -59,7 +59,7 @@ func (bm *BlazeManager) WriteTimestamps(blocks []base.Blknum) error { } } - msg := fmt.Sprintf("Updating timestamps %-04d of %-04d (%-04d remaining)", + msg := fmt.Sprintf("Updating timestamps %-04d of %-04d (%-04d remaining)"+spaces, block, blocks[len(blocks)-1], blocks[len(blocks)-1]-block, From e402df2fc0ade3d07639dc565a5330031f710acb Mon Sep 17 00:00:00 2001 From: tjayrush Date: Tue, 5 Sep 2023 23:26:40 -0400 Subject: [PATCH 13/33] Cleans up reporting and validation. Verifies against sepolia. --- .../chifra/internal/scrape/handle_scrape.go | 6 ++-- .../internal/scrape/scrape_consolidate.go | 13 +++++-- .../internal/scrape/scrape_manager_utils.go | 35 ++++++++++++------- src/apps/chifra/internal/scrape/validate.go | 4 +++ src/apps/chifra/pkg/manifest/manifest.go | 3 +- src/apps/chifra/pkg/pinning/pin_file.go | 6 +--- 6 files changed, 42 insertions(+), 25 deletions(-) diff --git a/src/apps/chifra/internal/scrape/handle_scrape.go b/src/apps/chifra/internal/scrape/handle_scrape.go index c69fa1198c..197b271ac4 100644 --- a/src/apps/chifra/internal/scrape/handle_scrape.go +++ b/src/apps/chifra/internal/scrape/handle_scrape.go @@ -104,10 +104,10 @@ func (opts *ScrapeOptions) HandleScrape() error { if opts.Globals.Verbose { logger.Info("chain head: ", bm.meta.ChainHeight()) logger.Info("opts.BlockCnt: ", opts.BlockCnt) - logger.Info("opts.ChunkSize: ", opts.Settings.Apps_per_chunk) - logger.Info("block count: ", bm.blockCount) logger.Info("ripe block: ", bm.ripeBlock) - logger.Info("start block: ", bm.startBlock) + logger.Info("perChunk: ", bm.PerChunk()) + logger.Info("start block: ", bm.StartBlock()) + logger.Info("block count: ", bm.BlockCount()) logger.Info("len(blocks): ", len(blocks)) if len(blocks) > 0 { logger.Info("blocks[0]: ", blocks[0]) diff --git a/src/apps/chifra/internal/scrape/scrape_consolidate.go b/src/apps/chifra/internal/scrape/scrape_consolidate.go index f94eda6752..07e3d5d051 100644 --- a/src/apps/chifra/internal/scrape/scrape_consolidate.go +++ b/src/apps/chifra/internal/scrape/scrape_consolidate.go @@ -42,7 +42,6 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { } }() } - countThen := file.FileSize(stageFn) / asciiAppearanceSize // After this point if we fail the backup file will replace the original file, so // we can safely remove these the stage file (and ripe files) and it will get replaced @@ -62,6 +61,9 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { // Brand new stage. chunkRange = base.FileRange{First: bm.meta.Finalized + 1, Last: blocks[0]} } + nAppsThen := int(file.FileSize(stageFn) / asciiAppearanceSize) + nAddrsThen := len(appMap) + nAddrsNow := nAddrsThen // For each block... for _, block := range blocks { @@ -103,6 +105,8 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { report.Snapped = isSnap report.Report() } + nAddrsNow += len(appMap) + // reset for next chunk bm.meta, _ = bm.opts.Conn.GetMetaData(bm.IsTestMode()) appMap = make(index.AddressAppearanceMap, 0) @@ -113,6 +117,8 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { } if len(appMap) > 0 { // are there any appearances in this block range? + nAddrsNow += len(appMap) + newRange := base.FileRange{First: bm.meta.Finalized + 1, Last: 0} // We need an array because we're going to write it back to disc @@ -124,6 +130,7 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { newRange.Last = utils.Max(newRange.Last, uint64(app.BlockNumber)) } } + // The stage needs to be sorted because the end user queries it and we want the search to be fast sort.Strings(appearances) @@ -136,8 +143,8 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { // Let the user know what happened... stageFn, _ = file.LatestFileInFolder(bm.StageFolder()) // it may not exist... - countNow := file.FileSize(stageFn) / asciiAppearanceSize - bm.report(int64(bm.PerChunk()), countThen, countNow) + nAppsNow := int(file.FileSize(stageFn) / asciiAppearanceSize) + bm.report(len(blocks), int(bm.PerChunk()), nAppsNow, nAppsNow-nAppsThen, nAddrsNow-nAddrsThen) // Commit the change by deleting the backup file. os.Remove(backupFn) diff --git a/src/apps/chifra/internal/scrape/scrape_manager_utils.go b/src/apps/chifra/internal/scrape/scrape_manager_utils.go index b548fafb9c..36ecaf3c9d 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager_utils.go +++ b/src/apps/chifra/internal/scrape/scrape_manager_utils.go @@ -10,21 +10,30 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" ) +func colored(s string) string { + s = strings.Replace(s, "{", colors.Green, -1) + s = strings.Replace(s, "}", colors.Off, -1) + return s +} + // Report prints out a report of the progress of the scraper. -func (bm *BlazeManager) report(perChunk, nAppsThen, nAppsNow int64) { - need := perChunk - utils.Min(perChunk, nAppsNow) - seen := nAppsNow - if nAppsThen < nAppsNow { - seen = nAppsNow - nAppsThen - } - pct := float64(nAppsNow) / float64(perChunk) - pBlk := float64(seen) / float64(bm.BlockCount()) - height := bm.StartBlock() + bm.BlockCount() - 1 +func (bm *BlazeManager) report(nBlocks, perChunk, nAppsNow, nAppsFound, nAddrsFound int) { + nNeeded := perChunk - utils.Min(perChunk, nAppsNow) + pctFull := float64(nAppsNow) / float64(perChunk) + appsPerBlock := float64(nAppsFound) / float64(bm.BlockCount()) - const templ = `Block={%d} have {%d} appearances of {%d} ({%0.1f%%}). Need {%d} more. Added {%d} records ({%0.2f} apps/blk).` - msg := strings.Replace(templ, "{", colors.Green, -1) - msg = strings.Replace(msg, "}", colors.Off, -1) - logger.Info(fmt.Sprintf(msg, height, nAppsNow, perChunk, pct*100, need, seen, pBlk)) + msg := fmt.Sprintf(`At block {%d}, found {%d} apps for {%d} addrs in {%d} blocks ({%0.2f} apps/blk). Have {%d} of {%d} {%0.1f%%}. Need {%d} more.`, + bm.EndBlock(), + nAppsFound, + nAddrsFound, + nBlocks, + appsPerBlock, + nAppsNow, + perChunk, + pctFull*100, + nNeeded, + ) + logger.Info(colored(msg)) } // Pause goes to sleep for a period of time based on the settings. diff --git a/src/apps/chifra/internal/scrape/validate.go b/src/apps/chifra/internal/scrape/validate.go index 6db468018b..d51c9f4a5e 100644 --- a/src/apps/chifra/internal/scrape/validate.go +++ b/src/apps/chifra/internal/scrape/validate.go @@ -43,6 +43,10 @@ func (opts *ScrapeOptions) validateScrape() error { return validate.Usage("The {0} option ({1}) must {2}.", "--sleep", fmt.Sprintf("%f", opts.Sleep), "be at least .25") } + if opts.BlockCnt < 10 { + return validate.Usage("Specify at least {0} with {0}.", "10 blocks per round", "chifra scrape") + } + // We can't really test this code, so we just report and quit if opts.Globals.TestMode { return validate.Usage("Cannot test block scraper") diff --git a/src/apps/chifra/pkg/manifest/manifest.go b/src/apps/chifra/pkg/manifest/manifest.go index 13cc5aedc7..5aa8753493 100644 --- a/src/apps/chifra/pkg/manifest/manifest.go +++ b/src/apps/chifra/pkg/manifest/manifest.go @@ -11,6 +11,7 @@ import ( "strings" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/colors" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config/scrapeCfg" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/file" @@ -130,7 +131,7 @@ func UpdateManifest(chain string, chunk ChunkRecord) error { }) } - logger.Info("Updating manifest with", len(man.Chunks), "chunks", spaces) + logger.Info(colors.Magenta+"Updating manifest with", len(man.Chunks), "chunks", spaces, colors.Off) return man.SaveManifest(chain) } diff --git a/src/apps/chifra/pkg/pinning/pin_file.go b/src/apps/chifra/pkg/pinning/pin_file.go index 6b1a17da0c..1383963166 100644 --- a/src/apps/chifra/pkg/pinning/pin_file.go +++ b/src/apps/chifra/pkg/pinning/pin_file.go @@ -36,7 +36,6 @@ func PinItem(chain string, dbName, path string, isRemote bool) (hash base.IpfsHa } if isLocal { - logger.Info("Pinning", dbName, "to local service") localService, _ := NewPinningService(chain, Local) if hash, err = localService.PinFile(path, true); err != nil { err = fmt.Errorf("error pinning locally: %s", err) @@ -45,7 +44,6 @@ func PinItem(chain string, dbName, path string, isRemote bool) (hash base.IpfsHa } if isRemote { - logger.Info("Pinning", dbName, "to remote service") remoteService, _ := NewPinningService(chain, Pinata) var remoteHash base.IpfsHash if remoteHash, err = remoteService.PinFile(path, true); err != nil { @@ -60,7 +58,7 @@ func PinItem(chain string, dbName, path string, isRemote bool) (hash base.IpfsHa } } - logger.Info("Pinned", dbName, "file", path, "to", hash) + logger.Info(colors.Magenta+"Pinned", dbName, "file", path, "to", hash, colors.Off) return } @@ -77,7 +75,6 @@ func PinChunk(chain, bloomFile, indexFile string, isRemote bool) (PinResult, err isLocal := LocalDaemonRunning() if isLocal { - // logger.Info(colors.Magenta+"Pinning locally...", colors.Off) if result.Local.BloomHash, result.err = localService.PinFile(bloomFile, true); result.err != nil { return PinResult{}, result.err } @@ -90,7 +87,6 @@ func PinChunk(chain, bloomFile, indexFile string, isRemote bool) (PinResult, err } if isRemote { - logger.Info(colors.Magenta+"Pinning remotely...", colors.Off) if result.Remote.BloomHash, result.err = remoteService.PinFile(bloomFile, false); result.err != nil { return PinResult{}, result.err } From 6175e10bde63ee87d10a08798eb8b83444649428 Mon Sep 17 00:00:00 2001 From: tjayrush Date: Wed, 6 Sep 2023 00:58:50 -0400 Subject: [PATCH 14/33] Fixes tests --- .../internal/scrape/scrape_consolidate.go | 23 ++++++++++--------- .../internal/scrape/scrape_manager_utils.go | 9 +++++--- .../api_tests/blockScrape_nBlocks_invalid.txt | 2 +- 3 files changed, 19 insertions(+), 15 deletions(-) diff --git a/src/apps/chifra/internal/scrape/scrape_consolidate.go b/src/apps/chifra/internal/scrape/scrape_consolidate.go index 07e3d5d051..c2067418e0 100644 --- a/src/apps/chifra/internal/scrape/scrape_consolidate.go +++ b/src/apps/chifra/internal/scrape/scrape_consolidate.go @@ -54,6 +54,10 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { trapChannel := sigintTrap.Enable(ctx, cancel, cleanOnQuit) defer sigintTrap.Disable(trapChannel) + // Some counters... + nAppsFound := 0 + nAddrsFound := 0 + // Load the stage into the map... exists := file.FileExists(stageFn) // order matters appMap, chunkRange, nAppearances := bm.AsciiFileToAppearanceMap(stageFn) @@ -61,11 +65,9 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { // Brand new stage. chunkRange = base.FileRange{First: bm.meta.Finalized + 1, Last: blocks[0]} } - nAppsThen := int(file.FileSize(stageFn) / asciiAppearanceSize) - nAddrsThen := len(appMap) - nAddrsNow := nAddrsThen // For each block... + nChunks := 0 for _, block := range blocks { fn := fmt.Sprintf("%09d.txt", block) if file.FileExists(filepath.Join(bm.UnripeFolder(), fn)) { @@ -86,6 +88,8 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { // Read in the ripe file, add it to the appMap and... thisMap, _, thisCount := bm.AsciiFileToAppearanceMap(ripeFn) nAppearances += thisCount + nAppsFound += thisCount + nAddrsFound += len(thisMap) for addr, apps := range thisMap { appMap[addr] = append(appMap[addr], apps...) } @@ -105,7 +109,6 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { report.Snapped = isSnap report.Report() } - nAddrsNow += len(appMap) // reset for next chunk bm.meta, _ = bm.opts.Conn.GetMetaData(bm.IsTestMode()) @@ -113,12 +116,11 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { chunkRange.First = chunkRange.Last + 1 chunkRange.Last = chunkRange.Last + 1 nAppearances = 0 + nChunks++ } } if len(appMap) > 0 { // are there any appearances in this block range? - nAddrsNow += len(appMap) - newRange := base.FileRange{First: bm.meta.Finalized + 1, Last: 0} // We need an array because we're going to write it back to disc @@ -134,17 +136,16 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { // The stage needs to be sorted because the end user queries it and we want the search to be fast sort.Strings(appearances) - newStageFn := filepath.Join(bm.StageFolder(), fmt.Sprintf("%s.txt", newRange)) - if err := file.LinesToAsciiFile(newStageFn, appearances); err != nil { - os.Remove(newStageFn) + stageFn = filepath.Join(bm.StageFolder(), fmt.Sprintf("%s.txt", newRange)) + if err := file.LinesToAsciiFile(stageFn, appearances); err != nil { + os.Remove(stageFn) return err, true } } // Let the user know what happened... - stageFn, _ = file.LatestFileInFolder(bm.StageFolder()) // it may not exist... nAppsNow := int(file.FileSize(stageFn) / asciiAppearanceSize) - bm.report(len(blocks), int(bm.PerChunk()), nAppsNow, nAppsNow-nAppsThen, nAddrsNow-nAddrsThen) + bm.report(len(blocks), int(bm.PerChunk()), nChunks, nAppsNow, nAppsFound, nAddrsFound) // Commit the change by deleting the backup file. os.Remove(backupFn) diff --git a/src/apps/chifra/internal/scrape/scrape_manager_utils.go b/src/apps/chifra/internal/scrape/scrape_manager_utils.go index 36ecaf3c9d..7da61b1b73 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager_utils.go +++ b/src/apps/chifra/internal/scrape/scrape_manager_utils.go @@ -17,17 +17,20 @@ func colored(s string) string { } // Report prints out a report of the progress of the scraper. -func (bm *BlazeManager) report(nBlocks, perChunk, nAppsNow, nAppsFound, nAddrsFound int) { +func (bm *BlazeManager) report(nBlocks, perChunk, nChunks, nAppsNow, nAppsFound, nAddrsFound int) { nNeeded := perChunk - utils.Min(perChunk, nAppsNow) + appsPerBlock := float64(nAppsFound) / float64(nBlocks) + appsPerAddr := float64(nAppsFound) / float64(nAddrsFound) pctFull := float64(nAppsNow) / float64(perChunk) - appsPerBlock := float64(nAppsFound) / float64(bm.BlockCount()) - msg := fmt.Sprintf(`At block {%d}, found {%d} apps for {%d} addrs in {%d} blocks ({%0.2f} apps/blk). Have {%d} of {%d} {%0.1f%%}. Need {%d} more.`, + msg := fmt.Sprintf(`#{%d}, found {%6d} apps, {%5d} addrs ({%0.1f/addr}), in {%4d} blks ({%0.1f}/blk). Created {%d} chunks, staged {%5d} of {%d} ({%0.1f%%}). Need {%5d} more.`, bm.EndBlock(), nAppsFound, nAddrsFound, + appsPerAddr, nBlocks, appsPerBlock, + nChunks, nAppsNow, perChunk, pctFull*100, diff --git a/test/gold/apps/blockScrape/api_tests/blockScrape_nBlocks_invalid.txt b/test/gold/apps/blockScrape/api_tests/blockScrape_nBlocks_invalid.txt index 6703cd8542..6a56ab4d73 100644 --- a/test/gold/apps/blockScrape/api_tests/blockScrape_nBlocks_invalid.txt +++ b/test/gold/apps/blockScrape/api_tests/blockScrape_nBlocks_invalid.txt @@ -1,6 +1,6 @@ scrape?blockCnt=-12 { "errors": [ - "Cannot test block scraper" + "Specify at least 10 blocks per round with 10 blocks per round." ] } \ No newline at end of file From 6b9a84fe42ab6bba9e9b8f9c42795561225492e3 Mon Sep 17 00:00:00 2001 From: tjayrush Date: Wed, 6 Sep 2023 06:59:43 -0400 Subject: [PATCH 15/33] Sleeps for 1/4 second if remote to avoid rate limits --- src/apps/chifra/internal/scrape/scrape_consolidate.go | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/apps/chifra/internal/scrape/scrape_consolidate.go b/src/apps/chifra/internal/scrape/scrape_consolidate.go index c2067418e0..40443d5a92 100644 --- a/src/apps/chifra/internal/scrape/scrape_consolidate.go +++ b/src/apps/chifra/internal/scrape/scrape_consolidate.go @@ -8,6 +8,7 @@ import ( "path/filepath" "sort" "strings" + "time" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" @@ -109,6 +110,9 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { report.Snapped = isSnap report.Report() } + if bm.opts.Remote { + time.Sleep(250 * time.Millisecond) + } // reset for next chunk bm.meta, _ = bm.opts.Conn.GetMetaData(bm.IsTestMode()) From 0225cc52354ad4f803b22c670193f6849f77960f Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Wed, 6 Sep 2023 21:39:14 -0400 Subject: [PATCH 16/33] Fixes a few issues with chifra blocks uncles and chifra when. --- src/apps/chifra/pkg/pinning/fetch.go | 2 +- src/apps/chifra/pkg/rpc/get_uncle.go | 2 + src/apps/chifra/pkg/tslib/list.go | 99 ++++++++++++------- .../testRunner/testCases/tools/whenBlock.csv | 1 + 4 files changed, 70 insertions(+), 34 deletions(-) diff --git a/src/apps/chifra/pkg/pinning/fetch.go b/src/apps/chifra/pkg/pinning/fetch.go index 225fc34c6b..3c715fac33 100644 --- a/src/apps/chifra/pkg/pinning/fetch.go +++ b/src/apps/chifra/pkg/pinning/fetch.go @@ -33,7 +33,7 @@ func FetchFromGateway(ctx context.Context, gateway, hash string) (*FetchResult, } if response.StatusCode != 200 { // logger.Fatalln("DefaultClient.Do returned StatusCode not equal to 200 in FetFromGateway with", url) - return nil, fmt.Errorf("wrong status code: %d", response.StatusCode) + return nil, fmt.Errorf("url %s returned status code: %d", url, response.StatusCode) } body := response.Body diff --git a/src/apps/chifra/pkg/rpc/get_uncle.go b/src/apps/chifra/pkg/rpc/get_uncle.go index ae9114f6c3..1e1c4c105b 100644 --- a/src/apps/chifra/pkg/rpc/get_uncle.go +++ b/src/apps/chifra/pkg/rpc/get_uncle.go @@ -84,6 +84,8 @@ func (conn *Connection) GetUnclesCountInBlock(bn uint64) (uint64, error) { if count, err := query.Query[string](conn.Chain, method, params); err != nil { return 0, err + } else if count == nil || *count == "" { + return 0, nil } else { return strconv.ParseUint(fmt.Sprint(*count), 0, 64) } diff --git a/src/apps/chifra/pkg/tslib/list.go b/src/apps/chifra/pkg/tslib/list.go index ba206f1c86..aa1101873d 100644 --- a/src/apps/chifra/pkg/tslib/list.go +++ b/src/apps/chifra/pkg/tslib/list.go @@ -7,6 +7,7 @@ package tslib import ( "encoding/csv" "errors" + "fmt" "io" "os" @@ -18,8 +19,8 @@ import ( // GetSpecials returns a chain-specific list of special block names and numbers func GetSpecials(chain string) (specials []types.SimpleNamedBlock, err error) { - specialsPath := config.GetPathToChainConfig(chain) + "specials.csv" - _, err = os.Stat(specialsPath) + path := config.GetPathToChainConfig(chain) + "specials.csv" + _, err = os.Stat(path) if err != nil { // It's okay if there are no specials for a certain chain if chain == "mainnet" { @@ -29,42 +30,16 @@ func GetSpecials(chain string) (specials []types.SimpleNamedBlock, err error) { return specials, nil } - file, err := os.OpenFile(specialsPath, os.O_RDONLY, 0) - if err != nil { - return - } - reader := csv.NewReader(file) - // component,block/epoch,name,timestamp,date,description - reader.FieldsPerRecord = 6 - - for { - record, err1 := reader.Read() - if err1 == io.EOF { - break - } - if err1 != nil { - return specials, err1 - } - if len(record) == 6 { - bn := utils.MustParseUint(record[1]) - name := record[2] - ts := utils.MustParseInt(record[3]) - if bn == 0 && name != "frontier" { - continue + if specials, err = readSpecials(path, 6); err != nil { + if errors.Is(err, csv.ErrFieldCount) { + if specials, err = readSpecials(path, 4); err != nil { + return specials, err } - s := types.SimpleNamedBlock{ - BlockNumber: bn, - Name: name, - Timestamp: ts, - Component: record[0], - Description: record[5], - } - specials = append(specials, s) } } if len(specials) == 0 { - err = errors.New("found no special blocks") + err = fmt.Errorf("found no special blocks in file %s", path) } return @@ -75,3 +50,61 @@ func IsSpecialBlock(chain, needle string) bool { _, err := FromNameToBn(chain, needle) return err == nil } + +func readSpecials(path string, nFields int) (specials []types.SimpleNamedBlock, err error) { + file, err := os.OpenFile(path, os.O_RDONLY, 0) + if err != nil { + return + } + + reader := csv.NewReader(file) + // block,name,timestamp,date + reader.FieldsPerRecord = nFields + for { + if record, err := reader.Read(); err == io.EOF { + break + + } else if err != nil { + return specials, err + + } else { + if nFields == len(record) { + locs := map[string]int{ + "bn": 1, + "name": 2, + "ts": 3, + "component": 0, + // "date": 4, // skipped since we have timestamp + "description": 5, + } + if nFields == 4 { + locs = map[string]int{ + "bn": 0, + "name": 1, + "ts": 2, + // "date": 3, // skipped since we have timestamp + } + } + s := types.SimpleNamedBlock{ + BlockNumber: utils.MustParseUint(record[locs["bn"]]), + Name: record[locs["name"]], + Timestamp: utils.MustParseInt(record[locs["ts"]]), + } + // is this the header? + if s.BlockNumber == 0 && s.Name == "name" { + continue + } + + if nFields == 6 { + s.Component = record[locs["component"]] + s.Description = record[locs["description"]] + } else { + s.Component = "execution" + } + + specials = append(specials, s) + } + } + } + return +} diff --git a/src/dev_tools/testRunner/testCases/tools/whenBlock.csv b/src/dev_tools/testRunner/testCases/tools/whenBlock.csv index a331f168b6..4649b485e3 100644 --- a/src/dev_tools/testRunner/testCases/tools/whenBlock.csv +++ b/src/dev_tools/testRunner/testCases/tools/whenBlock.csv @@ -29,6 +29,7 @@ on ,both ,fast ,when ,tools/whenBlock ,invalid_option_2 ,y on ,both ,fast ,when ,tools/whenBlock ,invalid_option_3 ,y ,blocks = 0-0 on ,both ,fast ,when ,tools/whenBlock ,invalid_verbose_syntax ,y ,verbose on ,both ,slow ,when ,tools/whenBlock ,list_dates ,n ,list & fmt = txt +sep ,both ,slow ,when ,tools/whenBlock ,list_dates_sepolia ,n ,list & fmt = txt & chain = sepolia on ,both ,slow ,when ,tools/whenBlock ,list_dates_long ,y ,list on ,both ,fast ,when ,tools/whenBlock ,long_verbose_valid_block ,y ,verbose & blocks = 1000 on ,both ,slow ,when ,tools/whenBlock ,mixed_block_and_date ,y ,blocks = 2017-03-02 & blocks = 123123 From c88050959c209d2b0f6f81bf495ad0837e74ab9d Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Thu, 7 Sep 2023 05:59:25 +0100 Subject: [PATCH 17/33] Updates sepolia manifest --- .../install/per-chain/sepolia/manifest.json | 5537 +++++++++++++++++ 1 file changed, 5537 insertions(+) diff --git a/src/other/install/per-chain/sepolia/manifest.json b/src/other/install/per-chain/sepolia/manifest.json index d002495dcb..531d20a8ee 100644 --- a/src/other/install/per-chain/sepolia/manifest.json +++ b/src/other/install/per-chain/sepolia/manifest.json @@ -9074,6 +9074,5543 @@ "bloomSize": 131114, "indexHash": "QmXE7j4Fx8eNtSPiKDhRmYHPNwEBCpvQDbNFFcTAvtXbEY", "indexSize": 2593596 + }, + { + "range": "003937206-003937424", + "bloomHash": "QmeHAi6BLmSsu9Zaf3fCvRwyS2GZo5q1kTX8dhdQuZriZN", + "bloomSize": 131114, + "indexHash": "QmRrwb9c5a34xs9B9Wa1FhU8iS8Q1kTK4FoKEFve9yBSHM", + "indexSize": 2719176 + }, + { + "range": "003937425-003937629", + "bloomHash": "QmSUSGxjwmH2jpB1XEqrLQc2UnzcvaEBmMCWrDJDdVSzUK", + "bloomSize": 262190, + "indexHash": "Qmc4qD1v7H9bqp5Fuh2Dd35r1vx9G9Nr5PcuyXYDrWbv8Q", + "indexSize": 3348356 + }, + { + "range": "003937630-003937955", + "bloomHash": "QmcE1M1FHGCLZ9Q6N1XRCYqPHAPDGJiUxZn3xz1TsCBy8L", + "bloomSize": 262190, + "indexHash": "QmNtTbzSvBKPRDMWupCWHPGq8fwhdwmWLqwKng6R94Xywq", + "indexSize": 4025080 + }, + { + "range": "003937956-003938137", + "bloomHash": "QmbgdeBtEmSjxoMceWQRBkjCAFithyKTWzHf7scH222Yfd", + "bloomSize": 524342, + "indexHash": "Qmak4TAj49LQWBJbiJU6nGu16YMpSidbssip8thkZpXoKS", + "indexSize": 5829508 + }, + { + "range": "003938138-003938326", + "bloomHash": "QmNaWAWtgiRUzirBdd211YTvoDDTQvgN3psBhALyW7gnWH", + "bloomSize": 393266, + "indexHash": "QmSJ1kc6MVc7nSKVXJS46jaehoXKzmVCAP6mTGM1mS4LyU", + "indexSize": 5786904 + }, + { + "range": "003938327-003938587", + "bloomHash": "QmPCxWwXobPa6VZo7gbqmDLPov8VGWgUSFAxB2FyuVrd1e", + "bloomSize": 393266, + "indexHash": "QmZNRHYJKP7EQgXGLzCh6iG8mFAVsndksfm34NReNCXp2L", + "indexSize": 4651736 + }, + { + "range": "003938588-003938763", + "bloomHash": "QmYCC6gykh65npefEPYymmwNeG4x79bGZZd2D2moJg5EsK", + "bloomSize": 393266, + "indexHash": "QmX5aXQiXVykiLuPDEkAcGjvZ8hzr3B9m9SD3PnWq87LFA", + "indexSize": 5743536 + }, + { + "range": "003938764-003938924", + "bloomHash": "QmYM1VejRk868XqhDsrqJbECPVky3uv3Pvuvp3mT8aBTsF", + "bloomSize": 524342, + "indexHash": "QmbwUqGdm3S3vtGTRd2sH8J176ypqRoW9cGTqM82qFXt8e", + "indexSize": 6243272 + }, + { + "range": "003938925-003939131", + "bloomHash": "QmSFESFDgWYTaU8myu4dUC2cxHaxiEQy5MofN7gjuX8bbA", + "bloomSize": 524342, + "indexHash": "QmS8XMyouCqRhGBG2yzxpQZf5CvJvhStqH2WZW6cNveR7N", + "indexSize": 5902088 + }, + { + "range": "003939132-003939318", + "bloomHash": "QmbdL9HC964LF5yRaENYi8e2hf3gSHhPQEepC1JDmpBuHP", + "bloomSize": 524342, + "indexHash": "QmcCztzskMTGBUnRsenwyj3GRyWJwpqcjahUwEbL7rot4f", + "indexSize": 6003992 + }, + { + "range": "003939319-003939528", + "bloomHash": "QmVGjB1Lyn6UNEGvtjKSiFH2iKnaMpuzzw1NmgYvni1Nt4", + "bloomSize": 393266, + "indexHash": "Qmdug8VQ3wCQyZiuj1SGaE56dwpcAnjD9yK3HH5d3KgmtU", + "indexSize": 5809956 + }, + { + "range": "003939529-003939962", + "bloomHash": "QmXz9JW9MSjxUo1uzdTZ4R8tHUBLTFTbVPYZYgoV9xT3kB", + "bloomSize": 262190, + "indexHash": "QmYDXvzFRGmEyTRvpBHaHgGrFQfrCy85XqG3Qw36GNPqrC", + "indexSize": 4332632 + }, + { + "range": "003939963-003940135", + "bloomHash": "QmYWhA1yWQm6kXKRUA48RSKosPNS6JmVSbk6rjsERop7oH", + "bloomSize": 524342, + "indexHash": "QmdxfnqTXUKsVvrk6wTzgE4hAj8msKSTDkvTbYrvq9wABc", + "indexSize": 6029264 + }, + { + "range": "003940136-003940290", + "bloomHash": "QmUvHzWNoEbdxeKBCLxL59VZLmxvudth7cXty6EkSWcPSr", + "bloomSize": 524342, + "indexHash": "QmR9Jto4CuZvgz6VtrotPcEPjg1ZjxQCnj7PUDawS3tvQo", + "indexSize": 6098512 + }, + { + "range": "003940291-003940638", + "bloomHash": "QmXLk9HzrmDSnbtyjKU7SD56wVv8E1TXm9Z34HoWJYKDLv", + "bloomSize": 393266, + "indexHash": "QmWsTTE7kpdn6vSwuMxRT3HyPpWwyrdno2FFaPyMJtXfzc", + "indexSize": 4878616 + }, + { + "range": "003940639-003941229", + "bloomHash": "QmZ1JpTJa8CMxdPr7SqTYBmbyqoYKxvQiT17NzKxpQDK7U", + "bloomSize": 131114, + "indexHash": "QmdD3C7m4ookvLMMHAitjydncAkYTvL9vHivAzay5s3NHQ", + "indexSize": 2984384 + }, + { + "range": "003941230-003941775", + "bloomHash": "QmQ6UoaTGsjDawtJAgTcAv9cUb6fhAMwvpCgtxnfQnZkhi", + "bloomSize": 131114, + "indexHash": "QmZrNpDaPmGUSP1uxYaoRjJpnPjY66uBMxK1Eurzixxwxo", + "indexSize": 2952168 + }, + { + "range": "003941776-003942333", + "bloomHash": "QmYJu1FBW8ghfbRQQzJH3cBrF5wtqTAT5wUv2rACSCRgCg", + "bloomSize": 131114, + "indexHash": "QmeRdAc7sFprKXhPEBCnRd4eXZd4C3HS7i7bJSFHMYLHQS", + "indexSize": 2886948 + }, + { + "range": "003942334-003942846", + "bloomHash": "QmZx9DitC4i7TMU1RhHkC4QtKdauq8kt9ibDK97y2w5DVo", + "bloomSize": 131114, + "indexHash": "QmeHZneW2ZXYebymaSPBD3Aktp4yXQ8w63p7v9rfVFj6Rc", + "indexSize": 2266512 + }, + { + "range": "003942847-003943459", + "bloomHash": "QmYmu7jnzsBzJtw4GLgfsQLLqvH14CpHuB3cpYau12DR5n", + "bloomSize": 131114, + "indexHash": "Qmem6edj3rkZMeZUQq81gfqaZfNAWfsuK9Rt4oPiQygA4D", + "indexSize": 2352308 + }, + { + "range": "003943460-003944170", + "bloomHash": "QmVmf2ovY4teWXyq93rgBi6g3SN3d2FQbf7Dkv1cvcuvum", + "bloomSize": 131114, + "indexHash": "QmeK3oM3nu2C1Nr46sTZSZ4rFSN38bUo9oJuNVt29QWRN5", + "indexSize": 2405056 + }, + { + "range": "003944171-003944932", + "bloomHash": "QmWSxkp2NzwJQSa6HqjoPLeAE4ncBbaQDNry3rRqXJcNPz", + "bloomSize": 131114, + "indexHash": "QmQ1YWppL5vpDUMDZyt9hQFgqaTBdnPRSKPbGTtvD4US3T", + "indexSize": 2843916 + }, + { + "range": "003944933-003945767", + "bloomHash": "QmcBuD7pdHL8sKLdVreKyTTEmromq6L7GVPfRazJzWAQt5", + "bloomSize": 131114, + "indexHash": "QmdFsrmQ5W1LDDgdhvQiyvyXqVci2HyiJcuXMJXShTFDqj", + "indexSize": 2813884 + }, + { + "range": "003945768-003946535", + "bloomHash": "QmYJRULqnZkMcEtzCz6DA4juvdjj6bcGTnjmbYRxUxPDvB", + "bloomSize": 131114, + "indexHash": "QmaZgHeRKg7p5cJJpAr6DAztxDxd731Eoupt5vDwegcRqD", + "indexSize": 2624184 + }, + { + "range": "003946536-003946993", + "bloomHash": "QmX5BdC5TMTgox7wTewMLth5wS1xhNu1kMwkhwTMAkcmNd", + "bloomSize": 262190, + "indexHash": "QmXuxBPvsgkizEPPnbhA5wvji8bpJpRXtNTc97H2MtYsKo", + "indexSize": 4215592 + }, + { + "range": "003946994-003947211", + "bloomHash": "QmQ4S6wo9dJ3EBbK1h2VdpsCDZHiGmMGKBcrpAPuHeSpRe", + "bloomSize": 393266, + "indexHash": "QmW7Ht8AGvwrQEPCS96uM3wFDeHSrm8oN3SLucQ1P6kq9L", + "indexSize": 5427924 + }, + { + "range": "003947212-003947383", + "bloomHash": "QmeZKp5rifawZKvdzZPSszdXN8rLhTatPTRq796Non2vmn", + "bloomSize": 524342, + "indexHash": "QmcTzSAsYcHD38pAFhVinFBFV1gfJXzbDpDz9hVMmXHW92", + "indexSize": 5963984 + }, + { + "range": "003947384-003947550", + "bloomHash": "QmWjbVqiv2YmF9557TWiKwNZLyKaXPHmJKY7BM2ZCXUhpy", + "bloomSize": 524342, + "indexHash": "Qmf7isiXHC79DWgTmafAnGTnBaXjKtrJK66tjjCA1wfDrS", + "indexSize": 5898236 + }, + { + "range": "003947551-003947702", + "bloomHash": "QmaE5GDSKMJrFsF5iCzbSsJgoeGi5UXNghws5HjKVw3BnH", + "bloomSize": 524342, + "indexHash": "Qme1ZaktpVkdzZ9r8uwB9GcHCnR6qvBSYUHYu29SnDkmsD", + "indexSize": 6002076 + }, + { + "range": "003947703-003947863", + "bloomHash": "QmYBCEG6XtXn3bWsAjuMumyw3v43GhJUxLNS6VhGMDfG3n", + "bloomSize": 393266, + "indexHash": "QmT61hkRXXwGGHUiB8Ng9vTYq6ExocgiqE4CK63iNG9m77", + "indexSize": 5757836 + }, + { + "range": "003947864-003948271", + "bloomHash": "QmVR5ZH2PkkKHCH1PmvZPv3kKPqfyDNfUxg7s4Hi5iS1oe", + "bloomSize": 262190, + "indexHash": "QmbtsrxkSKnUVJNWRseNfUk8KjeCbiGYjrCUDyNKa4ve8u", + "indexSize": 3676900 + }, + { + "range": "003948272-003948828", + "bloomHash": "Qmb3LTUpH3BabADFvbxbkXyCS7GReL7gLCYR9xGRaQmhmS", + "bloomSize": 131114, + "indexHash": "QmRofBLY1xHqEyvoPCgzWb8fmeoXndXBye15MCSB6rmDiM", + "indexSize": 2462872 + }, + { + "range": "003948829-003949369", + "bloomHash": "QmYokUDpQRYDpfUG5rPVxMcr5nibtXhkSrbpf6M3QFmePu", + "bloomSize": 131114, + "indexHash": "QmYP83VMS31Wfpph2iFrttp8uT1zjkmuSZWkWBjfNd27xe", + "indexSize": 2372304 + }, + { + "range": "003949370-003949955", + "bloomHash": "QmUkZS2dnhap9NCXFmvGgVuU4TMAGUTKhSvURBaYm9NYDv", + "bloomSize": 131114, + "indexHash": "QmQ5hWSfBg3hH2GG3QcTGzZD3wkRdXSoiZmKiD9Te2Jwxf", + "indexSize": 2403356 + }, + { + "range": "003949956-003950257", + "bloomHash": "QmedJ9kLHyszeYkPFufh5B32wMSCEqwD6ojXrCY1TvhmU9", + "bloomSize": 393266, + "indexHash": "QmRFvMoxJJCUFHmPYmdawQyk9YZUEFgRfkajjA7gNXJX3z", + "indexSize": 5047324 + }, + { + "range": "003950258-003950406", + "bloomHash": "QmSCAgmjhP6vuW2juZyhADVzU3v7bHo9y7PZG6YP21ohDu", + "bloomSize": 524342, + "indexHash": "QmS89WCvcsmra3kZkjhYQEUpce3W1T3iyrL3rnyRSypoNE", + "indexSize": 6096920 + }, + { + "range": "003950407-003951190", + "bloomHash": "QmSnExgFLUEA9RMSkTTTpEdFXD4TcL6bUxoZxqt89JUFPi", + "bloomSize": 131114, + "indexHash": "QmetND4uxU4A8cHTTHKDeD7bCYe61jsoc7gJ7gEEhbPLBy", + "indexSize": 2403532 + }, + { + "range": "003951191-003952222", + "bloomHash": "QmR6BjAhDXZoeyYKHo4dFMkqDMT3V3uLFSN6aEAvW1wYB8", + "bloomSize": 131114, + "indexHash": "QmQzPr6krSPJyyHSkuJAnkfQf7sVL9jYfPK4M1WVgm3iRo", + "indexSize": 2141928 + }, + { + "range": "003952223-003952976", + "bloomHash": "QmQPak9scRgcKuAkAN4hsWjHhVCwpXRxSbnH1eVFzrC1fV", + "bloomSize": 131114, + "indexHash": "QmepS9QeLWfVA3bKeJqjVgczjCxNsggCGGxthYUnZ5TmHM", + "indexSize": 2530656 + }, + { + "range": "003952977-003953625", + "bloomHash": "QmTdqaHFXCYo5rH85rgBunVZWheTGghtYHzWUaLXsZ3ea6", + "bloomSize": 131114, + "indexHash": "QmYWGDpWJQu7V1WPPY3caJ4c6ixPRo31YnoynvzV3oGXg7", + "indexSize": 2586880 + }, + { + "range": "003953626-003954222", + "bloomHash": "QmZJsS6LuGgEPuw61nJbX1PGqXHixE16T1FG5qGw4QiDVK", + "bloomSize": 131114, + "indexHash": "QmR4vGmmoCKymeU8Xm418VwxGTWovSP9uzx3wTAnnbWLxQ", + "indexSize": 2737720 + }, + { + "range": "003954223-003954751", + "bloomHash": "QmcBnpfWuN283vXFC17dcRkqzQzZpC1fQH6bVAfEqW1Lht", + "bloomSize": 131114, + "indexHash": "Qmb1LxjXAhewc13idERj43YXjReAPipngrKGsNC94cvqaP", + "indexSize": 2660992 + }, + { + "range": "003954752-003955313", + "bloomHash": "QmcBvQhR8jEFWc4rcAh2TKxTthRmys3yBgSaCBk79J1NLE", + "bloomSize": 131114, + "indexHash": "QmddUr3cML9kjuqhNqSWcG8urSKWqUzxbEdzb1EM5qqLSZ", + "indexSize": 2697740 + }, + { + "range": "003955314-003955857", + "bloomHash": "QmRG8Yi6httK8ZjUCqAAArrLv1EbNMtTW8VkwtYNQdvyVX", + "bloomSize": 131114, + "indexHash": "Qmexv9irjVjLj7E3RPaQh2SdCKbZf84DyZvFWgsnDauXjy", + "indexSize": 2922100 + }, + { + "range": "003955858-003956443", + "bloomHash": "QmSzGFYunH4S9sQqHAJ1jnMfpET5BJNLSVkH6pvQsagDZC", + "bloomSize": 262190, + "indexHash": "QmUhLPwUjcHQE99X2JR9bjw3vBeyaqh8tJxq9FvzPvneRE", + "indexSize": 3033172 + }, + { + "range": "003956444-003957054", + "bloomHash": "QmYKE6fcuDyyyrxkYn4WwP1Yp3L3u5maymfEgUBc4NQgv9", + "bloomSize": 131114, + "indexHash": "Qme39o528VeMC5eZ2KZjPQQrXYiHKEpXEyfiKBDX4JJyP2", + "indexSize": 2951688 + }, + { + "range": "003957055-003957669", + "bloomHash": "QmPRDg2CGzV118JYsfeq1Pn9TX3X8J7fUM47CTusJvGFxS", + "bloomSize": 262190, + "indexHash": "QmYVSSPCKTt9jGRbfg5hzkxFVhCdtcGrpDeeSfj2ToAYWh", + "indexSize": 3277900 + }, + { + "range": "003957670-003958388", + "bloomHash": "Qmdo9uE8iVn69eqpVWFNp71hXjfa7eAFqXnqhy2FyZ76am", + "bloomSize": 262190, + "indexHash": "QmecpiHQXnLsZiXZ41zjDxbE5bN4t1c75gnUWgKf77TuaQ", + "indexSize": 3601076 + }, + { + "range": "003958389-003959098", + "bloomHash": "QmdEqEgkKF5tgpFUuL7mzx6Z7NM2EPZVSFN1JiexmyzYyX", + "bloomSize": 262190, + "indexHash": "QmR8jkqfrNd9F99fyKFwgDDWATCseu2V1UhNKKKhbePR1N", + "indexSize": 3451280 + }, + { + "range": "003959099-003959783", + "bloomHash": "QmNrEqi34m8Z1xRSiNEFNJ9RiDxiV1i6dRUEZ7fQkSg2og", + "bloomSize": 262190, + "indexHash": "QmdXYLN1pb5uD4hsnC4kp6j8q3hsBS2YF3CjYCz7dbqnAQ", + "indexSize": 3361332 + }, + { + "range": "003959784-003960400", + "bloomHash": "QmPaa7SSpJyWE4qznoTPE1S4G9C7iNY8G9QWGLH8KMLgJX", + "bloomSize": 262190, + "indexHash": "QmeJvjLj3mTS1gpphGsP7Eiw7nGES3qNnix4D4U6dsY3ne", + "indexSize": 3480916 + }, + { + "range": "003960401-003960988", + "bloomHash": "Qma3ZmkWpd9G6fEWMrHXdVVmHrV7Qjmh6ERfFJEyfS1XN8", + "bloomSize": 262190, + "indexHash": "QmWBySZVFEfPBBDnCk2V5oYkhtqNWY9Nk3akQsHWwCJy3t", + "indexSize": 3090960 + }, + { + "range": "003960989-003961585", + "bloomHash": "QmbHoSMvgdXMQMgCqKmHC5jZeZFom3ByEFwUXvky9p8YUJ", + "bloomSize": 131114, + "indexHash": "QmSihsf5obTdfHAv56cGEQhmmjYR1mKvjH6hmeX5uLq9tk", + "indexSize": 2935404 + }, + { + "range": "003961586-003962122", + "bloomHash": "QmQ4H2RB1H81dupdPK8GR23fB8FUF616munRzzmQj3vp6N", + "bloomSize": 131114, + "indexHash": "QmTG4SfkRs5pQMv4cFYSaDJEevprCiw3RfvAjqojaNcvxJ", + "indexSize": 2992532 + }, + { + "range": "003962123-003962591", + "bloomHash": "QmVaqPmcWas6HthbJ62mrbKqXjyk4KYbpGprjBpLETt3hP", + "bloomSize": 262190, + "indexHash": "QmZLi62cW2SunTtUmCW8Y2W9FokVwpkVTuXZ9aoCgyJS4z", + "indexSize": 3051552 + }, + { + "range": "003962592-003963042", + "bloomHash": "QmboGMdykXVZR3vz3gHeEXjnU3eFjYkGX34cb2xtH8kN1v", + "bloomSize": 131114, + "indexHash": "QmSxWKhTPELja8tpan4dAuRU2E7jYJ83VqNEcwmc4hf5GC", + "indexSize": 2943968 + }, + { + "range": "003963043-003963581", + "bloomHash": "QmZExPHKdmxU4yXV2qbhDFvtzmyVWX7n7ojv4GyVNuZ5Gg", + "bloomSize": 131114, + "indexHash": "QmUn6NFF5nuEHcijVWiTwwQCHvG5hFzYKCX5KtspvjZi7J", + "indexSize": 2835400 + }, + { + "range": "003963582-003964230", + "bloomHash": "QmbpvLs27DjvXGcbvpHG957T1CewTXYmQUrRji58sux1en", + "bloomSize": 131114, + "indexHash": "QmT5yw6ziDUb91BuFi4nVBMKovCQghuqLbXaLcUNvfaT85", + "indexSize": 2799036 + }, + { + "range": "003964231-003964902", + "bloomHash": "QmQ2DpdFXopcfsWWitBbKBq6fz4TAp26doykFgaMNqcwnz", + "bloomSize": 131114, + "indexHash": "QmcR8tfmANbg4Z2iMCqfY3thTwzXpQVw7vHV8QqMRZDY2g", + "indexSize": 2677156 + }, + { + "range": "003964903-003965491", + "bloomHash": "QmUPenwbt7aqBM9xs4sC1tQLpsBtEuG24HZog94K5WeBUT", + "bloomSize": 131114, + "indexHash": "QmaTDN4gruSHSDFmrNt928tYD56SQddvazr7R64UCZsrn1", + "indexSize": 2697708 + }, + { + "range": "003965492-003966125", + "bloomHash": "QmbQkGJvs4LfuXNB9MjxkZktEJV4kQXft2332WueeqfbHD", + "bloomSize": 131114, + "indexHash": "QmeASqY3Qv2EhAsbN9YcKuXEzS98nxvbX2xeHoAUF8z4g5", + "indexSize": 2495496 + }, + { + "range": "003966126-003966736", + "bloomHash": "QmeVWU5krWfdzY2RdRBcwccRPxR8KHXESu8BwMDr2w6yeh", + "bloomSize": 131114, + "indexHash": "QmNkSNKudEY6hzgL4wQpDdhBmnTcKDUuARhqb3K2FSJGNW", + "indexSize": 2459764 + }, + { + "range": "003966737-003967232", + "bloomHash": "QmWT8ViUA12QNtPJTJkFrV9jkqTVoZF4gnTam1xohvcnsP", + "bloomSize": 131114, + "indexHash": "QmYhZCNGfhoVD4ikBwkPfX3u2HgiFp7toDGm8KmzURbBiy", + "indexSize": 2389088 + }, + { + "range": "003967233-003967913", + "bloomHash": "QmdY9MPYoYL6gVZGE1g1HMqDtc8xL6TMwmXzvGRVwaARMJ", + "bloomSize": 131114, + "indexHash": "QmQUB6yQZC2CNTTuK5NwE46L66m6qUjysnoehupejbuwAz", + "indexSize": 2403724 + }, + { + "range": "003967914-003968568", + "bloomHash": "QmWQ55zobyGtMbbJBBBa6JAP3XvR9abK7tZ7HM3DCvkhkR", + "bloomSize": 131114, + "indexHash": "QmViouD4bNBntEcH1yECTntp8J1HhTFVuCZKjZDLe3fXHk", + "indexSize": 2368272 + }, + { + "range": "003968569-003969335", + "bloomHash": "QmTp2SacoxqCHDZTyUyKm5p8hLFvJPkCqeHYCsbaVt4uKQ", + "bloomSize": 131114, + "indexHash": "QmfVoAswGSaFVbxVMuxj3mkRHyju8XLJ3RgXce9Zp7xzdZ", + "indexSize": 2238108 + }, + { + "range": "003969336-003970215", + "bloomHash": "QmZ7po4neBEyTPrCBTqLnhrLy2J7ZGzkhXzC7KXo6Qxgoo", + "bloomSize": 131114, + "indexHash": "QmduDZGUgw3PgWn6P5NrHTLEjWne9HLh5dyuCY3nn58TBG", + "indexSize": 2187080 + }, + { + "range": "003970216-003971223", + "bloomHash": "QmQD53yAdeSzNvdGtKzGSxWhzmGRyzpxFFfPmLrd7cm8H8", + "bloomSize": 131114, + "indexHash": "QmP5wG8GDh1mKvi9baM6JBHiYTwQapYrmvCxJp3tDqHrEa", + "indexSize": 2078560 + }, + { + "range": "003971224-003971940", + "bloomHash": "QmYKWRSGVKzNPYXAGDmofAzim3QUR5imvdrUkPztiENwYP", + "bloomSize": 131114, + "indexHash": "QmTVBKk2L1eVcwxcDVtAczteoBFDT3pCVFBZQzNGgxs7jr", + "indexSize": 2289772 + }, + { + "range": "003971941-003972709", + "bloomHash": "QmYWBxe5kFKcTLa2aNZZCuxyB2DQK1wsYYyBZxm1CKcrWV", + "bloomSize": 131114, + "indexHash": "QmVns92DhWeLUActeCuGvQ1s3sLsGBV5NYCzJJ4fcFmoCG", + "indexSize": 2254664 + }, + { + "range": "003972710-003973358", + "bloomHash": "QmVKr7KTVY6gqVNAMpywvcirYehhj1vQZzwMW98c77YRig", + "bloomSize": 131114, + "indexHash": "QmWzGy3TgufCeCK9idZRGGzEskUCfvkLZgzcL1fXqzk7t5", + "indexSize": 2378660 + }, + { + "range": "003973359-003973985", + "bloomHash": "QmaUeUZ9v7t3zMPAjXE3Q1VK4bk1GkW68DfzfEcVKtMsCn", + "bloomSize": 131114, + "indexHash": "QmemNFkM38TtvjfUgveD9qKe9bDUXAtZtpMEPuiwKJf8Ps", + "indexSize": 2648316 + }, + { + "range": "003973986-003974575", + "bloomHash": "QmWenm7w34MZKGJJyibM15Zd5Zei574krbNG3yHUFFo38c", + "bloomSize": 262190, + "indexHash": "QmbDzwegecDwVXTaj9VbumXiNE61wH3KijX6scXC8We1uc", + "indexSize": 3134004 + }, + { + "range": "003974576-003974961", + "bloomHash": "Qmai2mQPyKD55eVtUsS6LKaMhYQ3HRmyDMoBqfNuvxo68B", + "bloomSize": 393266, + "indexHash": "QmbVhZW3gtL6YUAe5pv72vR12oj4QVniWWVnb65KDDmb8K", + "indexSize": 4515904 + }, + { + "range": "003974962-003975199", + "bloomHash": "QmSEANBCpJh2UPRuDDyMFFE8XfLV2cDeG9CWLTG3XeVTVQ", + "bloomSize": 393266, + "indexHash": "QmZVFJ66LL9NcRXFNo43H8MFUPYu6QJtZyXBstZgm3gR6y", + "indexSize": 5569164 + }, + { + "range": "003975200-003975325", + "bloomHash": "QmeZuTDWWYrsFGvDfU7rjfFuvqc7sTvFdk5ivSYwinp8YR", + "bloomSize": 524342, + "indexHash": "QmUvg8Sk1Xa8Hcm5KXxLA25MYCWrAEw6HTAFQtAzF2QU37", + "indexSize": 6303252 + }, + { + "range": "003975326-003975472", + "bloomHash": "QmP1h1jaLpBsUfdCc9yELx4kDFbb6qBA5jPKpxMZthL1ki", + "bloomSize": 524342, + "indexHash": "QmPjhfNXh4gSxiNPrbynd899AcmFBZSLbLwEwy6D7Ya4HV", + "indexSize": 5961720 + }, + { + "range": "003975473-003975647", + "bloomHash": "QmNvMQDdzjV8JhUt2Rjc6GZDJo4y2zesaP16xMnWgwSx36", + "bloomSize": 524342, + "indexHash": "QmYfYBzw2JzJ3iVYmWXEv5MHAawfFTpyyXjzZsfPBFYyin", + "indexSize": 5822864 + }, + { + "range": "003975648-003975799", + "bloomHash": "QmU1rwXjr4trANWhaVJkie1AiJtLS3xruL66ATuvjjVqub", + "bloomSize": 524342, + "indexHash": "QmY8g4tztr4K9Bt4h6heisaqLo1adgKvTFqT53Ks1CmXRy", + "indexSize": 6167040 + }, + { + "range": "003975800-003975943", + "bloomHash": "QmfXxLenMW6t2CRUeKgQXhbn6YtMdf7ZZ9wRFdixABoJ2R", + "bloomSize": 524342, + "indexHash": "QmUsG7Py6HLxn3dTCoMHGvPRA2fV3m2rycBDuXiLTe6Rkq", + "indexSize": 6259792 + }, + { + "range": "003975944-003976076", + "bloomHash": "QmWWvmYzgu4f7AWSG89R6VmNSqVm7R9Rf1xhbqfRyQNcnd", + "bloomSize": 524342, + "indexHash": "QmXat9L1CWDfhwDPFhNnEw7ryScv7UEqj781bZp8cH3TM8", + "indexSize": 6500316 + }, + { + "range": "003976077-003976217", + "bloomHash": "QmU9CHNsDf51knAxPcVLnbYWzz64NLUmbY1mNVL28R1xu6", + "bloomSize": 524342, + "indexHash": "QmXcanTti5djZ2YnVgZK61UWJQySCCeGHZ6zV2Cqrm1kcF", + "indexSize": 6427728 + }, + { + "range": "003976218-003976363", + "bloomHash": "Qmd94mftF8GpzejweYyMRtUfKwjpcD8hTLT8R8PX1795aD", + "bloomSize": 524342, + "indexHash": "QmYwx21x9DUWKLzaZGDriQBmXvpZEw786kwinc2pAJitGe", + "indexSize": 6491228 + }, + { + "range": "003976364-003976505", + "bloomHash": "QmbMvXNoL6NcujkffDV9B4Pg2hziRg4EFRAZ4nkAUdWvHg", + "bloomSize": 524342, + "indexHash": "QmR9vcSNavu99EWLQQ8pWR8Mr7dNXfmT1fYbjJ1SroZhSj", + "indexSize": 6496304 + }, + { + "range": "003976506-003976647", + "bloomHash": "QmdxtxTt6dDx2vKxABeMTezNwdnZP1m5BV8uzerXyEA8Sw", + "bloomSize": 524342, + "indexHash": "QmP85ehuLRakBCPZgV5jV7Bc75E2T1v2J5EmDRs1gHym8C", + "indexSize": 6478464 + }, + { + "range": "003976648-003976800", + "bloomHash": "QmUctHUUcUH4fijEzKnBhxfoXzmPcK2KMHSf4zQeBYRHgV", + "bloomSize": 524342, + "indexHash": "QmYN5KwfHpa6hdsxdftiL6WpzktUJhhoQcsnBKr6FzCFdJ", + "indexSize": 6448908 + }, + { + "range": "003976801-003976935", + "bloomHash": "QmfHzA2dsrBSpJX2Cy2GBHU5sAJku8k1MCKFeHPGwZmgtJ", + "bloomSize": 524342, + "indexHash": "QmWTMBgP11xfy5FmxiTjbznYGfmhKcPZPPLe5Mbz9irUhM", + "indexSize": 6534480 + }, + { + "range": "003976936-003977090", + "bloomHash": "QmVSRQau7JGgeDCfSP7UANHoTBXkyyUYWcGKY81mEGkXuf", + "bloomSize": 524342, + "indexHash": "QmTLkogR53Lf52afoMqRqHDjxAoHZLTjxK2L2K6dCKJeRD", + "indexSize": 6462044 + }, + { + "range": "003977091-003977505", + "bloomHash": "QmZVFFW3EAQ5r4yXh4Ewa1qHZGr9NNqphrDMFPofSf5sEh", + "bloomSize": 393266, + "indexHash": "QmQ2E21TkW8f2QmRYJj5xwb8pUMhGQU6PS2AYfZDieXu4A", + "indexSize": 5168148 + }, + { + "range": "003977506-003977647", + "bloomHash": "QmRZiTimiAv7iBCm2roNwM47dPtG7pDcySzt93N5A15HVp", + "bloomSize": 524342, + "indexHash": "Qmefaq7mXXRUe8UqdCZDnG1RH8wDY3zCkWQN7NAeMG6v2J", + "indexSize": 6457100 + }, + { + "range": "003977648-003977744", + "bloomHash": "QmWa5WFNLfNxUxii9o7zFVS2vM3t6XGqr92HvTzkvaaJVb", + "bloomSize": 524342, + "indexHash": "QmWiyqHnjSMU1T2mKp8CuFdxWD74mBbRGPtLrKTHEvgUs9", + "indexSize": 6245028 + }, + { + "range": "003977745-003977850", + "bloomHash": "QmQY52kbHUAqGYuHzZSmcGL6BKJQiKMkm5HqdtfLNoBPHM", + "bloomSize": 524342, + "indexHash": "QmcHJbjEepS76p364k5Zofy8vTRmYv5snKBjJuHiryNjpg", + "indexSize": 6382284 + }, + { + "range": "003977851-003977994", + "bloomHash": "QmbyChNGCqMGecG3Ha8D29pxVeFWHHx2PYTPUp6dnmZboX", + "bloomSize": 524342, + "indexHash": "QmZnZNPyz3cyGVZa3BY7rsKXXL7xmtnEuXb29SiQgpbcxz", + "indexSize": 6457264 + }, + { + "range": "003977995-003978177", + "bloomHash": "QmPqAfi6g2pqf7FZCFiUZ632iFw6HvgT1vYW6PEcXCCdvh", + "bloomSize": 524342, + "indexHash": "QmZyDNdwtq8AjMM5T3QHLhjwm2AokBPSaRqmQBW4P3xps7", + "indexSize": 6192768 + }, + { + "range": "003978178-003978368", + "bloomHash": "QmXueAxQ5WCjdGqNg36tvgnFDBmwCLFpsT8AAPAuon5JPe", + "bloomSize": 524342, + "indexHash": "QmR3RXe4zDBYaE9BYs3WBQq3QuAMpGpkvaPSfEvt7sr2QX", + "indexSize": 6108808 + }, + { + "range": "003978369-003978527", + "bloomHash": "QmSYrR7KEptioQy7zhqJ6eXoqHcHpue1PW74rWbM8W7NEe", + "bloomSize": 524342, + "indexHash": "QmTuRLGYf3AoXa7haAaZ2RYqU5gwwK7SK3iShFCPrdGgTT", + "indexSize": 6256616 + }, + { + "range": "003978528-003978680", + "bloomHash": "QmSB3vja9GAr3TG1fpFjX8M1PC9Ybn3bUkV371Bs1WeQat", + "bloomSize": 524342, + "indexHash": "QmZLwrpV7StDhc7m8oudsQSfoEuuHAh1fAjy4si9h2g7zH", + "indexSize": 6144260 + }, + { + "range": "003978681-003978831", + "bloomHash": "QmYULHhjXrfwiPLHWeGvxRbnFjQFD3Y4qB8vPPZ356k679", + "bloomSize": 524342, + "indexHash": "QmbyDWNxwfZPGF8wFBkNXv9GDGUDYbk9KQUwUQR8nacTiL", + "indexSize": 6316796 + }, + { + "range": "003978832-003979011", + "bloomHash": "QmSbquS8bQ7RyHFarneq2f6NJXL5ChRsFoumpY3781CPdi", + "bloomSize": 524342, + "indexHash": "Qmde2kgBRzPM6yDG6hdq9WHyc2Pe9yQYUGE1eSY4DHALRL", + "indexSize": 6232784 + }, + { + "range": "003979012-003979156", + "bloomHash": "QmNVhkiw9uiC7HyMZnAdqgH9yQHzZE4vfBkHg4naipmKW1", + "bloomSize": 524342, + "indexHash": "QmRUxoXoRWbEmf5TrK2zNPWQCt91HAMHeNyxeCjoKAZdfH", + "indexSize": 6433568 + }, + { + "range": "003979157-003979298", + "bloomHash": "QmVXTEnrURTs6jWv36ro758aVFaLnu52jfBUL5DDt4CNz8", + "bloomSize": 524342, + "indexHash": "QmZNPYCHSinDe4TVoFEUZF95Mj1jsva7JGRVmGaQdAVyAz", + "indexSize": 6249052 + }, + { + "range": "003979299-003979417", + "bloomHash": "Qmb5Ujknm5Vtvj32MTEmizAFJS8fSTe8Fcrr3sDKVDsgCT", + "bloomSize": 524342, + "indexHash": "QmYugWMF3yg3tdbFS3fH6UbQGfKjssUdnM3bNU6Njth8zu", + "indexSize": 6515532 + }, + { + "range": "003979418-003979556", + "bloomHash": "Qmbk5qgTEoYgsY4AHLmR58imz7RgJrttUTW9EYoqtdc214", + "bloomSize": 524342, + "indexHash": "QmRiT4Pc2VKYTo6yVQEyoQ8rSHP5CQFS4osvPRfZixmv12", + "indexSize": 6067032 + }, + { + "range": "003979557-003979721", + "bloomHash": "QmQpfX1EFt3yFtmASUFDX12EACFCZGTjy9FWnAWTtk14sj", + "bloomSize": 524342, + "indexHash": "QmdHomnMAUPFjYoudAe41mdbS9LouMrCBX32QPDgaZeZHU", + "indexSize": 6096408 + }, + { + "range": "003979722-003979848", + "bloomHash": "QmZPcFVadzUGKxjL4y5XibRRCMFzQouj5UgxmkstAY1DTJ", + "bloomSize": 524342, + "indexHash": "QmRHbvqvoWPW72UDsKUXikdqaKnfyREVWLRHLmbg39jjea", + "indexSize": 6575260 + }, + { + "range": "003979849-003979978", + "bloomHash": "QmSMP4NZZwmQzCLLr9jfxD47ELk88aJ2sjkUHgdJr1WE8W", + "bloomSize": 524342, + "indexHash": "QmRwqNdiDSLQbxw7Zj6s3bxfKoCUXUssU7oFBo2ufYTkVC", + "indexSize": 6392744 + }, + { + "range": "003979979-003980103", + "bloomHash": "QmePjwfsmwLNcbQLuBDsvooJd9kVz8CsCUb1mJ8cQs5CXH", + "bloomSize": 524342, + "indexHash": "QmWc1Y5ZpLXuoDqQBrBWW2nCQwkLwKEGRezGbCQqJotsm9", + "indexSize": 6442572 + }, + { + "range": "003980104-003980238", + "bloomHash": "QmUWZk2AmWCwCNUf1kvA9wqwP5B7WzkAcMnwKN985SwPhA", + "bloomSize": 524342, + "indexHash": "QmdZRx94AiwbH7r6Jb8H3jC8tCXZV9ozG2chphVVxrjTaG", + "indexSize": 6352080 + }, + { + "range": "003980239-003980373", + "bloomHash": "QmZuASxxAezpiKUqSiepUBVhbG3SKNd12RBiy1GofCnx12", + "bloomSize": 524342, + "indexHash": "QmQygqpJ6DUH7thKcuBU6S4qJhtUo83Ek5eeo6fNRaEftR", + "indexSize": 6407160 + }, + { + "range": "003980374-003980559", + "bloomHash": "QmeHJtqbnZViEFLL1L5HhAvMEwwraEHfJTZ5umnLeC2pq5", + "bloomSize": 524342, + "indexHash": "QmatfD1E9i4QXmx4RwrPoAe8Q7jzjg4kiu7kWW4QxefHgy", + "indexSize": 6174960 + }, + { + "range": "003980560-003980711", + "bloomHash": "QmPAV7e351sDLcoY13M34z48xFnEkTg2rbdog5pUxSoaYG", + "bloomSize": 524342, + "indexHash": "QmQEcLKHzTAtrPxMDtStYgLofbzoW471PJ9nkR8CmaRsng", + "indexSize": 6248624 + }, + { + "range": "003980712-003980857", + "bloomHash": "Qma3cTKnuuFmhpfvz1h7RUJ2GcGbYzEkR7mjkPQwPzpL2p", + "bloomSize": 524342, + "indexHash": "QmY7YCht5GSYzdztELbpUFDu86z5tcacTjruayHqC3bcyM", + "indexSize": 6097956 + }, + { + "range": "003980858-003981018", + "bloomHash": "QmRFuvkZicDGeA1Xz5S6A36LM6E9dkVo244LXftb3tF8pt", + "bloomSize": 524342, + "indexHash": "QmX8k4Muum68JGJvfDbkXbkdrb3Vged8m5SJN3aJhV3urf", + "indexSize": 6313256 + }, + { + "range": "003981019-003981200", + "bloomHash": "QmPLM6vtbwh9MFAkbYkE7PJ77JtQhjVGN4gEWCYkk6tbm6", + "bloomSize": 524342, + "indexHash": "QmbbcgaWwwbdxsvr1XU2xLdTnGDfB3T4ZEW6k2mKmcTjbQ", + "indexSize": 5965312 + }, + { + "range": "003981201-003981386", + "bloomHash": "QmYaSuvD6wFnjU9vpNL1SZF7rgp45ctFC3Kff3yT52P1Jm", + "bloomSize": 524342, + "indexHash": "QmVMoGYP3y5JEkwrvb47WuN5kppH92TbP1uXo7LXLBJTLg", + "indexSize": 5960316 + }, + { + "range": "003981387-003981552", + "bloomHash": "QmdTPdhyGXH8YFeudUUnvg9LvQ8RFfnCzt6AZ9JifFznB1", + "bloomSize": 524342, + "indexHash": "QmdsUYAGh6uegu68pjSGkZVJLdCV25Kx5DQFhzksV2a24k", + "indexSize": 6178944 + }, + { + "range": "003981553-003981719", + "bloomHash": "QmQGiJc7m2UdZiL3isk2fNKLrQYfR4193a4tiwykSDjPfB", + "bloomSize": 524342, + "indexHash": "QmVzQMxV8WRkU9DhKLCgM1XjwdBgCRURPfh2MVRKLTExrV", + "indexSize": 6179908 + }, + { + "range": "003981720-003981868", + "bloomHash": "QmXfNFHEfmKjeuNW3V6WReLM2drnjqNziBN2cXiYs9eB3J", + "bloomSize": 524342, + "indexHash": "QmZVH2ysCGRzjcSW3EsXeqkgJCZVCzC5QAAaS6qJcAnPcU", + "indexSize": 6347376 + }, + { + "range": "003981869-003982019", + "bloomHash": "QmXAnrfbrkf1vD7cjx2wyzxCbyn5YnMzSmJS8vyL5cp4w9", + "bloomSize": 524342, + "indexHash": "QmUBi1hgxucPhw2Uq5oCB1mWBYy1VWd1FcChz3UEdMTLaA", + "indexSize": 6103084 + }, + { + "range": "003982020-003982220", + "bloomHash": "QmT6f9H81PADzy7H9CZiqMTvq9GPufjA1Mk93n45gkJ7DT", + "bloomSize": 393266, + "indexHash": "QmZCYcxWpQvoSb3i2k5CuWbvk4nPGHeL3hx6oAve5ExAY3", + "indexSize": 5437284 + }, + { + "range": "003982221-003982406", + "bloomHash": "QmWHSdAVohcNosXFsfATm8RKrvrfZWrHLoZa1SgS7TJcrF", + "bloomSize": 524342, + "indexHash": "QmRA6Zj83AkCtWgzfUF7E812EG7f9TrepAZC67Du3NkpCe", + "indexSize": 6054736 + }, + { + "range": "003982407-003982560", + "bloomHash": "QmPLNywrwziRBkJP4xoCW4AJK2LKtUfvaM2D9yix2RRFUb", + "bloomSize": 524342, + "indexHash": "QmcAWEptdL1bk8J9M6xMnGZy9YA4mKSPuutxRyjCHjSwsB", + "indexSize": 6396268 + }, + { + "range": "003982561-003982917", + "bloomHash": "QmeHUUfS2gZbAxMa6M4Bp78Xhiar9jkxatr34jEcv8tiP3", + "bloomSize": 393266, + "indexHash": "QmP5RsoV5hYK8rff68DuBotJRJgpFHCk1Khaf5cievAzGP", + "indexSize": 5306548 + }, + { + "range": "003982918-003983867", + "bloomHash": "QmUDk4Gn6tSxxWWXsNnLs2pW9YCL4U11D4mFwL7yDe4FdD", + "bloomSize": 131114, + "indexHash": "QmeCPpE28dDqymgMjh8aV18kGUWZv1jdeUiMWpVDQ3ryh3", + "indexSize": 2633768 + }, + { + "range": "003983868-003984221", + "bloomHash": "QmRi11je2x5GeoL1Rf5PR4KzwmoCAKsoHMHmEifLUSuvRA", + "bloomSize": 393266, + "indexHash": "QmReo8xtCZFkVnNVcmPnNptVMLB9Yzte35txBpqb5ytRd8", + "indexSize": 5578540 + }, + { + "range": "003984222-003984374", + "bloomHash": "QmY61mmLK4Vn1ZoqAaDHSLvEGXFmJ2D8joF7cZ1tzyXRE8", + "bloomSize": 524342, + "indexHash": "QmVpvJKd2FaAmEpFxzgh52JMa9Qv9H7WR4PPdRtAPdKeV1", + "indexSize": 6513220 + }, + { + "range": "003984375-003984536", + "bloomHash": "QmdhXt6CWazLnPrfkS6LwD3zwkhzXkD5wQrsz6yLa1SzLs", + "bloomSize": 524342, + "indexHash": "QmdJJfuwLGWHPhcJSNWkcDW9UvvU7rouozePvMgbyM8DUw", + "indexSize": 6410816 + }, + { + "range": "003984537-003984703", + "bloomHash": "QmTbq89Zt9L6Dd9mQ4myUPXpoRCU2LhL593XGfSTzn5iBF", + "bloomSize": 524342, + "indexHash": "QmQEar11g5634PEccm4mzM6uuCjktmN5hqGFKRetvY1zqY", + "indexSize": 6298364 + }, + { + "range": "003984704-003984830", + "bloomHash": "QmX99daWAu4F2XxFEerBhWhoiiSAa65hfRjRtKS5ieHJ3A", + "bloomSize": 524342, + "indexHash": "QmbJCdLGJYWoDEJFwjpKsgjaJ3DcQ2jX2SPJGZoyUk7NQD", + "indexSize": 6415152 + }, + { + "range": "003984831-003984948", + "bloomHash": "Qmbh9KD2hdwSbDxsb6xjZW4FKGGPnGekUZNgxzUf4EABz2", + "bloomSize": 524342, + "indexHash": "QmRtWRDQL3JkNk3yGkEXFHckVsB7f1SDUFSmFBDupTpoeR", + "indexSize": 6507832 + }, + { + "range": "003984949-003985082", + "bloomHash": "QmTyickFwxyHgxWFPt3KKfBEEk3X9iocriAEW3mvjE3fgz", + "bloomSize": 524342, + "indexHash": "QmVUrJaCEBF5nhGqKvkSHSXQTbNUkKfvpfASsFjRg9SBJ5", + "indexSize": 6530724 + }, + { + "range": "003985083-003985217", + "bloomHash": "QmRqBtAPCzuPQQbvpp39ghGMDVNkryFnXEg1iVVeuQJ4NT", + "bloomSize": 524342, + "indexHash": "QmdYXMM3fkDdJY8qRHArBjXgftHA2hEtcxy3i7KizcxXpZ", + "indexSize": 6453380 + }, + { + "range": "003985218-003985374", + "bloomHash": "QmbY68KWZVNVJ93uHyZbnhv6U7bUjEnd45AUDaRnDARX5a", + "bloomSize": 524342, + "indexHash": "QmTmVLcGLsywZ5ZSpvFdPiBGLoC3RZJaoXtb2yN8B4T9hM", + "indexSize": 6564140 + }, + { + "range": "003985375-003985506", + "bloomHash": "QmUe3y9UqHQJodGarBnHnJkQdHqMaCWz4fN2k2czqMsQZR", + "bloomSize": 524342, + "indexHash": "QmXfz6NjQHRm8XHbBbeSU5Pccug9xJg7XLAaqX8cb6Kzqa", + "indexSize": 6415496 + }, + { + "range": "003985507-003985640", + "bloomHash": "QmU4sUkSvAx9A8Des8cjpJ5XJrE2b8vC7Vmub8kFpueefT", + "bloomSize": 524342, + "indexHash": "QmbdcQjH5sJd29pAJQf9cgKCL6JK5qGx3pU4yLwBZSALEu", + "indexSize": 6367984 + }, + { + "range": "003985641-003985770", + "bloomHash": "QmUkwZQzg5abVfKQ7eGqHZCzwdNySBfRUMtQDdmWCVQq5S", + "bloomSize": 524342, + "indexHash": "QmSYR1oGoJ73DR1Y9bRECYJBFSDyv4v9SNWDJ4mcim6Pkq", + "indexSize": 6418172 + }, + { + "range": "003985771-003985904", + "bloomHash": "QmeDNgkXEcXe2yWj9ui3kJgghgQDcuoF1k9GR8Xh5VCKu5", + "bloomSize": 524342, + "indexHash": "QmPzQcFVzLoHtLaWopkzXxexRTunABobqbY88XpveMGeRe", + "indexSize": 6380276 + }, + { + "range": "003985905-003986059", + "bloomHash": "QmeUuau3ZwrZCS6k4kBAgC1ZSSWirtY3GmXK1P731566LV", + "bloomSize": 524342, + "indexHash": "Qmdc2EvDR7JBdSehnJwbkjvsuDFcu8QTK2b8ZhTLJMSGV8", + "indexSize": 6272368 + }, + { + "range": "003986060-003986562", + "bloomHash": "QmeE3TGpBqTzp7Pg7aRgBcMHEwE7q1EVnHinYGzSKWh4Qq", + "bloomSize": 262190, + "indexHash": "QmXwMogBCLNfJ43oBEJ6eJqG69uN3gd19E8Z2UaQ51SquP", + "indexSize": 4067216 + }, + { + "range": "003986563-003986746", + "bloomHash": "QmTCwFPRdHqZFVQfGPazcTXto3YVzZGeyqmaPQMPqu2FoT", + "bloomSize": 524342, + "indexHash": "Qmbf4VW9t21izbQFxbNKCHnbJgJtMfSTmhD5SJUNEzcUmw", + "indexSize": 5964580 + }, + { + "range": "003986747-003986992", + "bloomHash": "Qmdh2PTtqgTafGZ4CcU9LXHrUjHScFxPE4krwZeSnPpsLZ", + "bloomSize": 393266, + "indexHash": "QmadSu34PCDAnU15HHQq66UvsCg3NaZJeQuRkTDTibHTL2", + "indexSize": 5438236 + }, + { + "range": "003986993-003987107", + "bloomHash": "QmdhafWT6BrmriYUfwspEe88wHpRr9TVWn2cBLeeQWPX3P", + "bloomSize": 524342, + "indexHash": "QmbKLTB7vJycTbdhFgeMBw2MvdVCvb8xCza7xSa6t93x35", + "indexSize": 6319084 + }, + { + "range": "003987108-003987234", + "bloomHash": "QmXS3ebPbsB6ZKi4RkTN6VoBtfjaX8birE4B6wKKpKNLmA", + "bloomSize": 524342, + "indexHash": "QmWjwAHFvfwQsSjuRf41WiR3RBoc47pzbSBX5nmX5DJs3j", + "indexSize": 6200796 + }, + { + "range": "003987235-003987369", + "bloomHash": "QmfDep97a6g7qtgYXow8qUWBgeAARvpReuKSQaT2atbJjE", + "bloomSize": 524342, + "indexHash": "QmaJKPTvpavQbqxUvbr1qXNfSnYYrhvYMywhTdrPhQ1DP9", + "indexSize": 6250816 + }, + { + "range": "003987370-003987487", + "bloomHash": "QmQrqsEksHiZ3Fh59zzYqJDMF9DeeDLsrzQi543SsJzrNH", + "bloomSize": 524342, + "indexHash": "QmSmAjxBPPtToGFpTp1hDFtAW5wycPJ72Tu4PZ6dp7kYJ7", + "indexSize": 6481360 + }, + { + "range": "003987488-003987634", + "bloomHash": "QmYXB9pSScx5mNjreu17uBQy93FB7xkbp3nS2k1DYauX5J", + "bloomSize": 524342, + "indexHash": "QmfPXLh48KC39xCQ5sMJhQAzug2tbXrGHpdV7tEUZSGPsN", + "indexSize": 6304460 + }, + { + "range": "003987635-003987805", + "bloomHash": "QmSjYyKrCUvcm1BQ4jtGpLYmGHDy8YwWhj4JzGPSMFFsuZ", + "bloomSize": 393266, + "indexHash": "QmeaxqTnB2xWK5n3AWU5rTEXpziYdsE6S23D63KxA9wHjZ", + "indexSize": 5206344 + }, + { + "range": "003987806-003987927", + "bloomHash": "QmUtfgdHqDH2pymj3EPeVa6hHk3VXbEMYxVGyrsAXRxiBF", + "bloomSize": 393266, + "indexHash": "QmStnbwXJhmn4B6CCNuQbLbE71vPLndNxHNSdhBSh1dPgt", + "indexSize": 5788876 + }, + { + "range": "003987928-003988027", + "bloomHash": "QmZhuURwwBUWET8oh4DQkpY7FpgVbC8AEZWzJAVJaBbK1A", + "bloomSize": 524342, + "indexHash": "QmVwJUp8wbD9SHYM6hWnG9A6w9Nh3F75BuDeUX4rLnHASU", + "indexSize": 6539760 + }, + { + "range": "003988028-003988125", + "bloomHash": "QmPBDx6V5pG9JhWk34qy2hQqg7PadHAomMv2X6P4UjqAq5", + "bloomSize": 524342, + "indexHash": "QmcWre1qkNDuvLMpRmUXhxVWB2sg4oEifM3sWoC6t7jne6", + "indexSize": 6531840 + }, + { + "range": "003988126-003988224", + "bloomHash": "QmbXqcs4fryFYCFLTRwo2ZdVdTvv1Z4mof8aqFKUewjcyn", + "bloomSize": 524342, + "indexHash": "QmZgYuidc643VEwR5RiQa3Mut6NMAM5K9CEmuP1KKQzF42", + "indexSize": 6642576 + }, + { + "range": "003988225-003988328", + "bloomHash": "QmRNA1EZbYEA6zUSDipR7cZ5QZWwSrNZKU1mew8Kt4saEt", + "bloomSize": 524342, + "indexHash": "QmPTw1dtmDpcPLZnJyiBRxSj4HncXQ6j2jjYKAt5R2HLZ6", + "indexSize": 6637012 + }, + { + "range": "003988329-003988455", + "bloomHash": "QmNLxC3hrsNtdJKSXcsP6mE4xU8jJg44KFJYmxMyTH5qh3", + "bloomSize": 524342, + "indexHash": "QmbdqgqpSkc5vRhCwLwT9oS1PbciAmqsN5DLM3VBqxDad6", + "indexSize": 6481836 + }, + { + "range": "003988456-003988564", + "bloomHash": "QmW4YjCVcHsbpsyVDocvCHsLtvhtZJev5yqwF132apjpAq", + "bloomSize": 524342, + "indexHash": "QmVXZx1xnPHhMeyaBQY8zZgbmDHdJtY7EthPsN8ELqtyG9", + "indexSize": 6696964 + }, + { + "range": "003988565-003988668", + "bloomHash": "Qmam7HNjxXW5R5JMqpbkE4mQ23CcQ2A3Jr5iq9HN4nSUJ2", + "bloomSize": 524342, + "indexHash": "Qmb4nTMAfPJDtVHWwVrZL33iHboaHmHQ1mP7sBRLTLRn7q", + "indexSize": 6653616 + }, + { + "range": "003988669-003988766", + "bloomHash": "QmctQqauyf2CscHovZEa1ZhvxAP2eSN1G4VwXmx1GJSeJS", + "bloomSize": 524342, + "indexHash": "Qmdq76haqNBQw1J1nB2Mc8JCKmfmh9H5oCBoY1fvdiioJ9", + "indexSize": 6730672 + }, + { + "range": "003988767-003988877", + "bloomHash": "Qme77GqgTzJBB7G8o7pvS1diAPucRDuQDNycRFr5BjSALj", + "bloomSize": 524342, + "indexHash": "QmZL8d5LpkKq8aJFdi82J3ewLQgbfjUrrh6A4hQ9HDMqf3", + "indexSize": 6575500 + }, + { + "range": "003988878-003989014", + "bloomHash": "QmfMzNjSbPjbRf4q7mWnh58qJ7EXcv4yimPtZku292DJ2U", + "bloomSize": 524342, + "indexHash": "QmatAEh3kYF8RPR7SLXP495DAA4xPE5YmGhY4pZGN2Em2a", + "indexSize": 6525288 + }, + { + "range": "003989015-003989157", + "bloomHash": "QmWzujeQSgbHBz8ugGHGDFGdPDgyfdrQcbHfnB8kg66Mq5", + "bloomSize": 524342, + "indexHash": "QmY1wztv1Nmg7SxPvZ95E1giZ9jx9nQbh46DyHYtaSBCWE", + "indexSize": 6466132 + }, + { + "range": "003989158-003989314", + "bloomHash": "QmZS67kAnLNUVGq6FzseDdS5gUcvPLRWT3HssQHwFUAdTu", + "bloomSize": 524342, + "indexHash": "QmbCUUm5bpY5s5nXz5Buhc9TdAEPrRMxgXA3HnbyLkb7CJ", + "indexSize": 6475740 + }, + { + "range": "003989315-003989433", + "bloomHash": "QmRdmRejzKqqTHztswz684au5v7ALCBFhD8VnePbNhCh5G", + "bloomSize": 524342, + "indexHash": "QmS1vNocbcqCr6SXx3nLcv4sGFDzNhPzjzDjR4s78gjh3Q", + "indexSize": 6674872 + }, + { + "range": "003989434-003989563", + "bloomHash": "QmZCyvtQjeMw3prTeLzTQuxQXCA3yL2SbGrL9YnMMebb9C", + "bloomSize": 524342, + "indexHash": "QmYxMY7AUtGK3r8kFYYgRs2iJyx4sT7t1F4HsnnCRiyF3c", + "indexSize": 6606052 + }, + { + "range": "003989564-003989701", + "bloomHash": "QmUhiJSAMNMgTLZToeDiBAFLV8ws15ehANxmqzK7aDCVD6", + "bloomSize": 524342, + "indexHash": "QmRqPFzS7SyZ1A65ndUGTu735mBYj2ezxCpjZGRkfZ7dwF", + "indexSize": 6782244 + }, + { + "range": "003989702-003989835", + "bloomHash": "Qmcfi2rwd84gVmkL7ctVPWFzDkAoPQYzLuBCh6rbok2pmo", + "bloomSize": 524342, + "indexHash": "QmcJUQkyPM7dmpFfnFK6iwYR3Y9LBR1e29RgSQhz79ZfSG", + "indexSize": 6774428 + }, + { + "range": "003989836-003989975", + "bloomHash": "QmYB5vtVpS4zFiL5Q1i4jo1XaWUn6JixtFxsV7oFfiypif", + "bloomSize": 524342, + "indexHash": "QmeCwvANsBpLVAnnGFPooSdWmzNRnGqBc1UTuwDrxZmeDR", + "indexSize": 6620836 + }, + { + "range": "003989976-003990112", + "bloomHash": "QmNPJiV5uAXm1teYTvknRyzZs1J9qYg6fdiPiVuiAo2k6w", + "bloomSize": 524342, + "indexHash": "QmWhpQdJ6up2AdAz9txnWsVaKnvDbMa1P4nLyCpwH1PLEa", + "indexSize": 6615560 + }, + { + "range": "003990113-003990236", + "bloomHash": "QmXwdJWEec3yDv62JMcRj5b5gpinP2FQt3Vr34kQHSjnyY", + "bloomSize": 524342, + "indexHash": "QmSexD8suAMSfCPfaZUjwaav66ou2HQy8ZV3qWqANRYxpt", + "indexSize": 6657232 + }, + { + "range": "003990237-003990359", + "bloomHash": "QmQ5dHTA1TKaWRf9NGRV3qRjSxC1dMJ4hkvheUG4Uds1GC", + "bloomSize": 524342, + "indexHash": "QmWF1Tn6bBT9srzWLuwnZGnvfUaCDPw2MsVi9cSikjBiGL", + "indexSize": 6725048 + }, + { + "range": "003990360-003990452", + "bloomHash": "QmbZDetYnGQvH5HPfStAPtmApda3AwC22FmDSLeQ3HkB4G", + "bloomSize": 524342, + "indexHash": "QmaoneKErMixCJWXeVZu6Ug6oU8bJS1PzeKy2Pq9bZk7wn", + "indexSize": 6478412 + }, + { + "range": "003990453-003990559", + "bloomHash": "QmdZQaG25ragr9rrJ5abVAoY3ZZ3fhwgi7nnSJPHMseob9", + "bloomSize": 524342, + "indexHash": "Qmd5c7YUxE2UEyGcJ6c7D3muBTbajAcrrFt6RR2Qz1oYJZ", + "indexSize": 6697676 + }, + { + "range": "003990560-003990690", + "bloomHash": "QmcS6EzNwZSVNuiFSwvoYGh5FemujUQGRrcfauijZDrmMS", + "bloomSize": 524342, + "indexHash": "QmTpXPE4riBEb29yxr3Houuz4VctebuWYThNyLzJowRLGE", + "indexSize": 6640160 + }, + { + "range": "003990691-003990852", + "bloomHash": "QmZ4sLz18TaPJd8kGFxY1Q9L6TqDV1z28FUsBAP7nDZXcr", + "bloomSize": 524342, + "indexHash": "Qmaday7KQTck5H193HnLGLawB9tdwKNSK18PW3ftBGbc9C", + "indexSize": 6385516 + }, + { + "range": "003990853-003990971", + "bloomHash": "QmNxqtuByRY3c2djeWfmcL6uzeBiHUbx4b6TD6ga4y6Xq9", + "bloomSize": 524342, + "indexHash": "QmbKj4swKDGipBqV8fM7BLEccZRMQqG1X2SaeiBRWX8Cfa", + "indexSize": 6708268 + }, + { + "range": "003990972-003991082", + "bloomHash": "QmXd93iPXrUkCvdY753wyKtbmxkQ1vVhUQe8GJENiNNa65", + "bloomSize": 524342, + "indexHash": "QmYArVwEuFwF6Mx4qZVhrFUJaU6wnJkBwKsrKKfeDsotrc", + "indexSize": 6701820 + }, + { + "range": "003991083-003991180", + "bloomHash": "QmcbyXGSXPhcA1Q1a6pWXSzyih6KEYrhxxrPKNo4z4ptu3", + "bloomSize": 524342, + "indexHash": "QmcjGhPuZFpsNGdSGKQ41u1Pe8RT6omJu8hMYFzFTxALNp", + "indexSize": 6817660 + }, + { + "range": "003991181-003991286", + "bloomHash": "QmbA6kn1fr2hJqtk5oUzLix75RR3fMCVfiphbgsAxnVtfb", + "bloomSize": 524342, + "indexHash": "QmQCUbJjeCBABNcf42Y1wo4FHkWyY9T53ELk8sa8Y4Tjvf", + "indexSize": 6596420 + }, + { + "range": "003991287-003991434", + "bloomHash": "QmRrADqAvNN7tiSe2JnFj5V8qZsxWxyRGD8vWtQQYAbqzu", + "bloomSize": 524342, + "indexHash": "QmcyUUb83ATx6KZrADkbaxn76PeijdP8fzt6uHj1qM3duH", + "indexSize": 6345272 + }, + { + "range": "003991435-003991963", + "bloomHash": "QmXtNHC9Z7bxTtdYdmRFz86R4TbswwKqt5cXcoqrqVv4uM", + "bloomSize": 262190, + "indexHash": "QmUdYBhuEySfb3nwaFvVudDhCqkNhkz2GXsBsxiSX2aHL8", + "indexSize": 3034768 + }, + { + "range": "003991964-003992534", + "bloomHash": "QmTbDvr1udvuyCQmCuhwp1EFB5AdiJQpSHbRGVNzWSkA5i", + "bloomSize": 262190, + "indexHash": "QmQxoMGVc5NpzGTcmW9Lh4y3j3KXJDgzsz8g7Pf6hSJjed", + "indexSize": 3586248 + }, + { + "range": "003992535-003992787", + "bloomHash": "QmTDfdTjBuDcmAAoTZy5q9irCQBgqv5F5mz48AkC7D8ZJb", + "bloomSize": 393266, + "indexHash": "QmVsjFwqLTVbSeBuZvvfhAC6ZwnEDR8rpVoV9vEJmrgiSa", + "indexSize": 4575280 + }, + { + "range": "003992788-003993040", + "bloomHash": "QmRKCSj6pzbFBhGs2nMXkREhX4Vwo8xAxS94Po5raUhAS7", + "bloomSize": 393266, + "indexHash": "QmSzvGhuQ9WVSBjVABwQ8Jgu183ztvjrUfpgm5Zm3XRAcd", + "indexSize": 5640428 + }, + { + "range": "003993041-003993753", + "bloomHash": "QmS1w5Q2WLxnPahcYyqHuwaA17o87EsVb4ddZBp65ySCXY", + "bloomSize": 131114, + "indexHash": "QmRppyvX43wPukAt2wnsWmAsKkKXzM9MSvqEAyEkefvtTc", + "indexSize": 2754884 + }, + { + "range": "003993754-003994490", + "bloomHash": "QmRiFUxZaqcTEoxUPvhdB6HG4AS1vmpq9GzF2QJAJQLA5e", + "bloomSize": 131114, + "indexHash": "QmUFoJ7mGs3Ny5j8ZwtYb316kBS37aLXNmUoGuHiYkXcjz", + "indexSize": 2731656 + }, + { + "range": "003994491-003995023", + "bloomHash": "QmRiKGpNKab3yc3UkPx7CcaUHyHj6f5XGrzgTkESYCeSjn", + "bloomSize": 131114, + "indexHash": "QmUs3qt39DxTHVE8pgS9ePjQJS3ifqLSSQwkVMFfjPn2D2", + "indexSize": 2468312 + }, + { + "range": "003995024-003995649", + "bloomHash": "Qmejkehv6woXcnnq6ZcGYSoDkG8h5YAgSmA4baAd1xyR2g", + "bloomSize": 262190, + "indexHash": "QmZKeg3js1AiBhAXcytdynKk9ZL9i46cg36iV6ZXxn7NMq", + "indexSize": 3451520 + }, + { + "range": "003995650-003996010", + "bloomHash": "QmY5NQRBrK63TYjgbaAZyiAXotpBSr82Wv9zWYg1ExZzSa", + "bloomSize": 131114, + "indexHash": "QmY2gv9EUhbnuUu4MtvMRPJbhnP3VEAze7WodMt4D7kVML", + "indexSize": 2847736 + }, + { + "range": "003996011-003996492", + "bloomHash": "QmYzAzcfjvUUo4qoWFxjpadeEekP8iqkLH6HNpWTGwdXVp", + "bloomSize": 262190, + "indexHash": "QmfSH8mjLp4Ty3p2oDrVjMJE8SNsGRxzM3ZcWtwmSDNfAu", + "indexSize": 3235632 + }, + { + "range": "003996493-003996640", + "bloomHash": "QmQ7rrzEHg25o28MYGwhWqMTZwXr4AUiFn3AmogU7KNfw5", + "bloomSize": 524342, + "indexHash": "QmWAKb2LCGvQWW4FErh2qVnDW5c8vY2o5gJshGxMkcSmS1", + "indexSize": 6457048 + }, + { + "range": "003996641-003996791", + "bloomHash": "QmUqjKPtCNkdqRGgKsr1RaXhgKhrhsPuCFAYsgA34jrSGF", + "bloomSize": 524342, + "indexHash": "QmboVSukj6jNm5ZRQqvxEXWByuWovcdAh432GXe38NJoAK", + "indexSize": 6639692 + }, + { + "range": "003996792-003996952", + "bloomHash": "QmZtzv2DdoTgEf9fVJYGVAnqy1BgbUNPbzuNrrsHdLq7rf", + "bloomSize": 524342, + "indexHash": "QmS6FsparrRhgMNnGM8dP6EdUBxsMXDMFogZ9CWCBFLt38", + "indexSize": 6532056 + }, + { + "range": "003996953-003997108", + "bloomHash": "QmS6Qt1FDJj92egBDd2VbxyDLxx9QhhV147wPAWmHnFrx5", + "bloomSize": 524342, + "indexHash": "QmZse9KfvuTmcGH4pvugs6ucRMSdUgKasJpmhai7MmDpyP", + "indexSize": 6384640 + }, + { + "range": "003997109-003997270", + "bloomHash": "QmZecFh3bbNK55nCxm8TQ4WpTs9VLfPEef6n3RwhKnut6f", + "bloomSize": 524342, + "indexHash": "QmPF4th9oUd698aHAB31PSCq1M2FB882xwEmDyRnsRaYgn", + "indexSize": 6116128 + }, + { + "range": "003997271-003997391", + "bloomHash": "QmPBWfWme6ebYqHgiMfRFAqZqcs1zEnKNHobd5NzTjNB2K", + "bloomSize": 524342, + "indexHash": "QmfHE5LQN1B3YTepHhXsWEWFcBVxF4eKSuFsEeXQN5LH4W", + "indexSize": 6596480 + }, + { + "range": "003997392-003997540", + "bloomHash": "QmUwwXMrLBqQgT43LkyqB2T5SjNZgs89ezfYD7KtjcALVT", + "bloomSize": 524342, + "indexHash": "QmYWSPJffunbHGoLtZa9KMrHqPXnmjtaUhiNqwYboyk64X", + "indexSize": 6468968 + }, + { + "range": "003997541-003997694", + "bloomHash": "QmWZV8nF2qTubXeP8jW5ih3SEjpW3FD8gMg4DKpWvMrvUB", + "bloomSize": 524342, + "indexHash": "QmYigxP7hNeNwyBjT93v2Yvbb8BGN9tNvaFTTwrTS8wGCr", + "indexSize": 6437968 + }, + { + "range": "003997695-003997890", + "bloomHash": "QmfVRwKXFsC5eK5RPK4nUhmWYskgx7LYDGaEL9SevLeyM9", + "bloomSize": 524342, + "indexHash": "QmPi4pYsDz2WiFVQvTN278yKoBqpD3DHRBL6z9BAikc589", + "indexSize": 6164944 + }, + { + "range": "003997891-003998082", + "bloomHash": "QmdMWhNu1raSkt8fS255hbFVwrqSAZHKKjpA7Nwck6TtH1", + "bloomSize": 524342, + "indexHash": "QmQbar1NT2DLK7r9MF1oCazFXDpgUgw2VyM7MJpmJgZvLn", + "indexSize": 5957464 + }, + { + "range": "003998083-003998226", + "bloomHash": "QmU7PFCUsKRwteK2NaamUtewP3TiYw1xQYj43CKzSJh1Ce", + "bloomSize": 524342, + "indexHash": "QmSW33UBePQjE9sVjhC777bUTF4TXF3uYtCRcTk7T421DA", + "indexSize": 6233808 + }, + { + "range": "003998227-003998365", + "bloomHash": "QmdnumeGMU4KkzaZugJTE7UNPnUWNjU29mA8kfFvv1gC4M", + "bloomSize": 524342, + "indexHash": "QmStVHUSXGSaKVvYF58aNsYg25shUjVe6C2bMtEtHK8rwe", + "indexSize": 6129676 + }, + { + "range": "003998366-003998567", + "bloomHash": "QmPnrUBUgmVGtGQSAufZ5Nh8An4JF8dXVR5T55x3riDK9j", + "bloomSize": 393266, + "indexHash": "QmUNQVvL9VsDS2Aba7322pD7igwRPRr5UxuSR4VXPHy38K", + "indexSize": 5589680 + }, + { + "range": "003998568-003998700", + "bloomHash": "QmeMNzwgnhimMmQ7ovZEQt7Z3o4LpfLahjgWuXn5kBPpcK", + "bloomSize": 524342, + "indexHash": "Qmf6d98Kv4JPi52yQvkv7f73svnrqaMJNaY65hbK8g5t4G", + "indexSize": 6303132 + }, + { + "range": "003998701-003998828", + "bloomHash": "QmWgPb6xYfvLLXTjGukxC3Bm4FCWAdPwow2dh16E263Bzh", + "bloomSize": 524342, + "indexHash": "QmbUGDGHsU6L1jPqgi2YDNqnEydnrGiuz1FJhbrdtiaNpk", + "indexSize": 6491312 + }, + { + "range": "003998829-003998977", + "bloomHash": "QmQgWrwSkths9BjYybx9Mpu6sVN8yswCA35rjSQPz2r4xM", + "bloomSize": 524342, + "indexHash": "QmQvowxCvebppgotYW246tdenwKqgnhysCLSyry9vE7LLR", + "indexSize": 6419968 + }, + { + "range": "003998978-003999101", + "bloomHash": "QmRz1bTgZjbzFftBGorw8bT5NnRkvckpobbQFQswPkbwby", + "bloomSize": 524342, + "indexHash": "QmXXaWnPn7kNjaxFQ4ezn2CrmDrgQv2raveR11ZUoFBRuk", + "indexSize": 6544044 + }, + { + "range": "003999102-003999254", + "bloomHash": "QmNrtgvJCtGoFf5Nro4mx1EGnMsGFCcgCSQg4Ee8tJWH18", + "bloomSize": 524342, + "indexHash": "QmQrb4XNx3V3Yy44m6KjfLb5sy5YyByE6SKcoSRUjEmQTS", + "indexSize": 6213328 + }, + { + "range": "003999255-003999413", + "bloomHash": "Qme71ereHDUXMFrXbWQ3bd8yGWJqdKiPrYVqFSJSMHFK68", + "bloomSize": 524342, + "indexHash": "QmbygvuEpLQ9b9vJ3z8dUkDRFupSs6i62ttMV3Us3Qtm55", + "indexSize": 6268620 + }, + { + "range": "003999414-003999837", + "bloomHash": "QmQViXj5aawxZWsLPTNKw6gggqEtXGdksGbcaK9nYmJQa9", + "bloomSize": 393266, + "indexHash": "Qma8LS6TLU2oQT9U9sVxt3sQkgnGtmoo4Sv48asQXDgq93", + "indexSize": 4540560 + }, + { + "range": "003999838-003999982", + "bloomHash": "Qmcx13sPgv6Em1PZ99kSYSzRLreow23LZoXEisSLZYmQky", + "bloomSize": 524342, + "indexHash": "QmSnXJZN4yJMPX4BR7B1a61ugmJfShqeWDJ2WJ1LCNMBtK", + "indexSize": 6279292 + }, + { + "range": "003999983-004000000", + "bloomHash": "QmajN7kRW2r6vLvFJNfH99tHBGwirC4sGNaayeKpyKbB2B", + "bloomSize": 131114, + "indexHash": "QmWe1kxn5hEE6DbjQh2F25gbBCPN4rUFq38QtqT9Q18p5n", + "indexSize": 700608 + }, + { + "range": "004000001-004000153", + "bloomHash": "QmcAJnrh5av1ZWARFMVxLEhq2uhpiF9q6ZwXp1mL2BbqRk", + "bloomSize": 524342, + "indexHash": "QmQJHdzKwgEh4qeMvA62imyiW9MArLSQVHEnW1A2aEAbBa", + "indexSize": 6137448 + }, + { + "range": "004000154-004000295", + "bloomHash": "Qmay3SSigS3oRBRyvJSnQkm3j72siwodfuh7jshEw4vNai", + "bloomSize": 524342, + "indexHash": "Qmf6jGMM5xn23tf5zEwVK31P9vDoyXBtgn29dqV6G6i2Fv", + "indexSize": 6323504 + }, + { + "range": "004000296-004000439", + "bloomHash": "QmRrU8CQ9KgpN7nQ3qQZBkZBcrsBNKUhdWZm4Kv5KAToth", + "bloomSize": 524342, + "indexHash": "QmdzX76XJqY1AwpCw3ud41he98rSC9uiC3p439GUAUnmoy", + "indexSize": 5943672 + }, + { + "range": "004000440-004000578", + "bloomHash": "QmZD1hgyd2mf66Ewp2kNtEgrgJMawyKPkba67ebxijAzhf", + "bloomSize": 524342, + "indexHash": "QmNVkC8y4NZtkiFps514igKnwkM42UYpXTg1wSFTnr25XH", + "indexSize": 5913696 + }, + { + "range": "004000579-004000723", + "bloomHash": "QmU6aabTarTTLDYCTXsQQbRFWGfxpTenrZv3XMJWucaPuD", + "bloomSize": 524342, + "indexHash": "QmXUACyNhVYzjoQTsbMxEmxSmxraN9mMLhyziiUyR6buma", + "indexSize": 6211580 + }, + { + "range": "004000724-004000905", + "bloomHash": "QmSAw2XLY1W822Su6a6FFQ2pgJQPwQqofZZAt3VgT2rzLz", + "bloomSize": 524342, + "indexHash": "QmPbwXetstrQtCMPYZm6eoBdnqLYK9U6bivQzvXQGTan3J", + "indexSize": 6183588 + }, + { + "range": "004000906-004001044", + "bloomHash": "QmRPg6G13dShTyuYg45QhYHxCKcVaDCnhiz9iTu1fLic4e", + "bloomSize": 524342, + "indexHash": "QmRkJBZ1LfFQgSeoV1aouGBHYt4xo1s2njv1tPbzJA182o", + "indexSize": 6372372 + }, + { + "range": "004001045-004001177", + "bloomHash": "QmTmeM772bLweB2cTBhVgNfNG1uv5jSoP1iUWSEtXxJRnR", + "bloomSize": 524342, + "indexHash": "QmYPfZei91G2AhS5YFL87fTQmUzym98Rio4h18NDNB76g5", + "indexSize": 6350612 + }, + { + "range": "004001178-004001304", + "bloomHash": "QmQfZbvpmHaquqMeXJ8PawAn4as3Bi6MzHiVLpUH48hK4k", + "bloomSize": 524342, + "indexHash": "QmcbhMRdYFdDtkeg2UigxnNqkHs7ruc1NEMxSya2ajkjA3", + "indexSize": 6573180 + }, + { + "range": "004001305-004001446", + "bloomHash": "QmRgbMU3q5VrJQNQkN1W9vaf3wBx91vcoTex3WgaQt3myU", + "bloomSize": 524342, + "indexHash": "QmP7AkJaWZFBvyV5Ni6x8epgiz7jqjes87j1VTAiaUk7rf", + "indexSize": 6564048 + }, + { + "range": "004001447-004001590", + "bloomHash": "QmVXSwCb8Su1VEpirs1yb2xZb1WowFyYtg8fW9nBpt7L6K", + "bloomSize": 524342, + "indexHash": "QmcGz9Yr56ropDUXHzBBXYcsxW3TZXeiYyFQbntTnEiwLF", + "indexSize": 6622924 + }, + { + "range": "004001591-004001735", + "bloomHash": "QmRdWjLmYQypnMySpJp4JxgzPgRUgSLsB2EdHYPGAKmctT", + "bloomSize": 524342, + "indexHash": "QmUSWBjPtXfLq6Suv9Vy3vgsQydiLijsNH9LuQ5sQ3F1tW", + "indexSize": 6559096 + }, + { + "range": "004001736-004001871", + "bloomHash": "QmZJ4iaKPjr1mCX75e1XRjnwfiZT2BtefsVfr1Z4GJKM1S", + "bloomSize": 524342, + "indexHash": "QmeirHjmeu5Q6fimD4kr3teDXpMHkMHTTFj35mJc6YVJGv", + "indexSize": 6677852 + }, + { + "range": "004001872-004002003", + "bloomHash": "QmVwzss7wQkGi3mFWArYyhSTxfvZTySSWWd9xuBD4fTNRW", + "bloomSize": 524342, + "indexHash": "QmUqW6Ad7xLsyLiW3bAEbrFzwuCsruX4Z3TWXK2NcR5TAE", + "indexSize": 6656188 + }, + { + "range": "004002004-004002150", + "bloomHash": "QmTJUrNyumM7nrn7ihpCJ6oL9Sn8xU6kd9B2vJS6LqQQCy", + "bloomSize": 524342, + "indexHash": "QmUtoyiG9ogxCYnndBkSpfH3FWnBMqZVxxr7iQPrNsZixj", + "indexSize": 6692668 + }, + { + "range": "004002151-004002303", + "bloomHash": "QmWF2Deascs9UvPRW61gVpf1TwdXmdtJooBgTYrh6TgXxX", + "bloomSize": 524342, + "indexHash": "QmWRgHGGvgrLHwrRiQ6yKN5UrvHuN6JZtkgRKtmxSsT5Wo", + "indexSize": 6758084 + }, + { + "range": "004002304-004002462", + "bloomHash": "QmaJNjrrdQsKfcda69kQXtykYN4Wj7hDh3pWP1GisH5geR", + "bloomSize": 524342, + "indexHash": "QmRRWZjNjEtNJeRi3w8RVW1cmC4gRx9kHpYqvE4Nc951aa", + "indexSize": 6697152 + }, + { + "range": "004002463-004002600", + "bloomHash": "QmULJaweMYVmcUPCaBp3Asi2kBnHSnRrxgo4qaaK2MRXYC", + "bloomSize": 524342, + "indexHash": "Qmcohwp63tuzCoTE5geZwB3Emd5iQXpMn2hh1yZqBmejCB", + "indexSize": 6811000 + }, + { + "range": "004002601-004002736", + "bloomHash": "Qmf9VbEd3CFcf8J12hj28HPjE66tYQYrLLWDqjLz2fknVM", + "bloomSize": 524342, + "indexHash": "QmfH5raPuQaWwu7ggCNy9aygnBxfj8htDyfSt1y9jWmFn9", + "indexSize": 6817532 + }, + { + "range": "004002737-004002873", + "bloomHash": "QmWAGrQjksALi9VWQdXAiZaAv9QiFQFm9TYmY2w91LF4e5", + "bloomSize": 524342, + "indexHash": "QmZefZHvecuRtr8u42G8ny4rRfPthvejGyvmHbGZrrbuN7", + "indexSize": 6708988 + }, + { + "range": "004002874-004003265", + "bloomHash": "QmefJYvEUw9am75y1FjsPX1fgHuUaPA3xAJEkNUCwdcn6G", + "bloomSize": 393266, + "indexHash": "QmPqhoGNEMmmgJJAqnMm3TSSWxKuVESELb8qNBTYpBmNPP", + "indexSize": 5265180 + }, + { + "range": "004003266-004003968", + "bloomHash": "QmQcMqbahW7StVFa9dCkTMu8qGo2v5SpZW7LyVdbhokNPH", + "bloomSize": 262190, + "indexHash": "QmQp4uvQMmbgzr5JEPPYyDS6GLePdNSDe3GD7PXCDqofVN", + "indexSize": 3499556 + }, + { + "range": "004003969-004004807", + "bloomHash": "QmNvcdb8c9YqKqZ5Tt7rwtJPgssWP72PBCRmZF2ohF3JPv", + "bloomSize": 131114, + "indexHash": "Qmea5FdPSWGKpUAFPh1PfMX25Vi9JUDY9Mi65XifevJQfN", + "indexSize": 2983336 + }, + { + "range": "004004808-004005543", + "bloomHash": "QmdFkJtT5T9rau5tyfPa7qum4sSUUaZMWkC7mJt6bmGseJ", + "bloomSize": 131114, + "indexHash": "QmT4FBqDt1SNeMZ5yY1W1wWbyqyYnto8SvsLXFfjS4gVWJ", + "indexSize": 2905560 + }, + { + "range": "004005544-004006338", + "bloomHash": "QmZZ63hMq5sVSQMY4LEjvQ8c1e2erH1KVSV4GByn3JUUyv", + "bloomSize": 262190, + "indexHash": "QmRPoeNBxGUzJXT9mtFMXkZ7BXBWL3jQmdMATMmkcifZXw", + "indexSize": 3091552 + }, + { + "range": "004006339-004007212", + "bloomHash": "QmTdQe4PDX6xS5PDzv3JGNxbs849nemZMZYoXA85fjsVQ2", + "bloomSize": 262190, + "indexHash": "Qme1x9BoXsQEY2VfVTK9ix8AKVhRB5xvp6wZuCjdSBL8Rc", + "indexSize": 3557044 + }, + { + "range": "004007213-004008075", + "bloomHash": "QmYTUiGEnFzbEZyyYEgCNxir84vdgXq5zqYC8AERz7EwXj", + "bloomSize": 262190, + "indexHash": "QmPcsmpRJ8AhQj8jMZvAGc7d2A83KWe5HWLfpGMqVKkSsh", + "indexSize": 3519748 + }, + { + "range": "004008076-004008346", + "bloomHash": "Qmb8vAAig1Rj3pyXYnPnoN69s4wEnkmS2WVCdEq26T1diY", + "bloomSize": 131114, + "indexHash": "QmZRaqknG7grCb9DGwN4shYxZHtWVETv5jKN3AowNue3XQ", + "indexSize": 2375008 + }, + { + "range": "004008347-004009099", + "bloomHash": "QmXCBaouQzxGGQ5N77h1YSt5nv7nyGTFFs5PJk94UrZgA9", + "bloomSize": 131114, + "indexHash": "QmSUp23m9RMYCy7rdadv5qd4S74i1KjBwCN6XcCWNkBa17", + "indexSize": 2876252 + }, + { + "range": "004009100-004009429", + "bloomHash": "Qme5QkReZhxJHeyncXNUfGNZ2tmwcYFUA7PzoddVHaLuHm", + "bloomSize": 131114, + "indexHash": "QmXdKGp2kkzsCbKBc2wCZ4o6m88deE7onNDRuZEGrVtf72", + "indexSize": 2172372 + }, + { + "range": "004009430-004009503", + "bloomHash": "QmPtWfLEmwETPzWsYMywe3Eq4pRexdEK8EBLeCdnBKwwLp", + "bloomSize": 131114, + "indexHash": "QmYv6294dscDoAipmpEggXeM4hjqZ1yFhwVhdrQAj62shS", + "indexSize": 2036412 + }, + { + "range": "004009504-004010375", + "bloomHash": "QmVjqxeJeBsWctR8gg3b3yD3VUb6rJbx4HJeS5aWu5KXgG", + "bloomSize": 262190, + "indexHash": "QmWoSaUXhtfYstMvB7LXzmAkyCKVHHhayDGWB2uskUM4Wj", + "indexSize": 3254136 + }, + { + "range": "004010376-004010993", + "bloomHash": "Qmf7PH3AyVCm4b1Eij2iGuFmpRd4fmR3QwTp8bu4Qwnhn3", + "bloomSize": 262190, + "indexHash": "QmZF7sRtcCLE2tWW9uSGrgXs7QtM4DNdK1v4pVRkG8s6WX", + "indexSize": 3659756 + }, + { + "range": "004010994-004011597", + "bloomHash": "QmfCpQu9LWaDjFzF2jrBH5ffQ8KCeTw885kvsYTsvkaqtX", + "bloomSize": 262190, + "indexHash": "QmcGQWtyQ6JTDq5mnNsxh3BQZVJNnmGRyiRGL8HzceKrPx", + "indexSize": 3796772 + }, + { + "range": "004011598-004012253", + "bloomHash": "QmVKoaK8WG8zSBLturcQ6Hue9WewuQv3xy5ard8NMm3uoP", + "bloomSize": 262190, + "indexHash": "QmfHGcytxqNkezep3eUq5rb2GWmCSFP5W4icwH4fod4vJc", + "indexSize": 4234716 + }, + { + "range": "004012254-004012849", + "bloomHash": "QmcJ7JNpFg6TrH4pVW9JNxerN57auoVn3zf1mvqhWqCqjv", + "bloomSize": 262190, + "indexHash": "QmZTNLC7x5ChaAjHuFEajeKRZKYhrr7tcahvfwKBSL2VW8", + "indexSize": 3904476 + }, + { + "range": "004012850-004013560", + "bloomHash": "QmbgDFNhvgxNzGxddBEoXRivtNU1xSNkmLjRFhbvBuGxcw", + "bloomSize": 262190, + "indexHash": "QmZg9HhDfQ5aHz4CJPAsiEZQ4ruVok7djeDVAQHHwxT6B8", + "indexSize": 3642164 + }, + { + "range": "004013561-004014245", + "bloomHash": "QmXbR4BfwjZb3tUPkrg54WRkSrgZNWRwVdQRYwdD8kygCy", + "bloomSize": 262190, + "indexHash": "QmcHtxp9Ykr95gzsjBMreaB2d1mqsW26dxdrQH57SVLqM3", + "indexSize": 3738772 + }, + { + "range": "004014246-004015026", + "bloomHash": "QmZMoL9tDAzgruVTka7GnuCyYLs2xWm8XrUK7pK8wAbPJv", + "bloomSize": 262190, + "indexHash": "QmU4LBkac7LZbA8tMv3tjMTqsQDett8ZRD7jW63w7nCeiM", + "indexSize": 3243648 + }, + { + "range": "004015027-004015797", + "bloomHash": "QmP4UqB2Vnx9SevgxDseq3JGdpcLnjswxTtSqtBhoSD5XQ", + "bloomSize": 131114, + "indexHash": "QmduzsqezgyDS9BPpsFqPNEMB2PwfYYQBNNmUCiAhFsKpw", + "indexSize": 2857520 + }, + { + "range": "004015798-004016705", + "bloomHash": "QmcTFrxhZrduckvrmrSaQuQC9zp2dyBcjv3ryf7qBSr8MM", + "bloomSize": 131114, + "indexHash": "QmYUgE6X71gzVWZM6feE3eNiJoShnfKJfvfUFnNzQRTbUc", + "indexSize": 2890216 + }, + { + "range": "004016706-004017439", + "bloomHash": "QmU4mMmNu4sJssBY65xQnevwPJoX8De4QXybJAapEsZ5Kq", + "bloomSize": 131114, + "indexHash": "QmPUy9UjdMGutdnG3qiQvcMNbLGnLdhDpJNtrjwfaRibJB", + "indexSize": 2823600 + }, + { + "range": "004017440-004018189", + "bloomHash": "QmSwPxymnHGNGzH3doq9zM4Df4tGGPtKRxSZ3rmi5oULC7", + "bloomSize": 131114, + "indexHash": "QmaXjYcbFFLpgtHvQFuDTLSGP3grgow1gm9H6oXfNgvLbo", + "indexSize": 2820496 + }, + { + "range": "004018190-004018851", + "bloomHash": "QmaNfA9wFtdQjZqVrYZWmuJqxGDNf55Cji8YobPbkZKE4k", + "bloomSize": 262190, + "indexHash": "QmRtBgQtTTcNfaAXnEUGXS3AZxL8LQjHduG3tuwsmNj8vg", + "indexSize": 3318164 + }, + { + "range": "004018852-004019530", + "bloomHash": "QmZWzdQDXT5S8M4EKLkAdiWSyh4HNi98MmYQ2RDcVn6MMb", + "bloomSize": 262190, + "indexHash": "QmPS9KpnPkfFziQzsGEEHwjmyLENXbLPEWQDexHPifLamS", + "indexSize": 3782432 + }, + { + "range": "004019531-004020171", + "bloomHash": "QmPcpdb3Zu2wy6iZtfVpU19fTqNtovDhw3upUxt138VTh2", + "bloomSize": 262190, + "indexHash": "QmfASBnxbYe4rt2fm5FxeZcpRG362usCAMHAAVW6JnkKGG", + "indexSize": 4084988 + }, + { + "range": "004020172-004021018", + "bloomHash": "QmUTwdCngLfUfJjAn2DsUxAS7ZwEA5TusbBK9u1BFJ8sgJ", + "bloomSize": 262190, + "indexHash": "QmUAVzR28SiUd39HMw3P4cfJs5LogbJfeq13PtPgEEr71m", + "indexSize": 3523812 + }, + { + "range": "004021019-004021761", + "bloomHash": "QmX8xWYGtWV3dCLUtLoh5YZLe7VgFJMPpkE9Mx55JjjBee", + "bloomSize": 262190, + "indexHash": "QmPUX794QvJdrjFsqbBh9Zoj33JtGS46SfCy4JKWJAhJv1", + "indexSize": 3950664 + }, + { + "range": "004021762-004022674", + "bloomHash": "QmRkuGS6VabMvUVe1JgsmKJfKzshtAAtd9bEDrJDxLGa2S", + "bloomSize": 131114, + "indexHash": "QmRxpHH5QZgUE24wjaRXMx8KA1NpuEGfju9Ey6t9oDEMai", + "indexSize": 2977904 + }, + { + "range": "004022675-004023404", + "bloomHash": "QmVBjgwP6LCVmeg2uVrPnLaJhEyJZhqw63iKegefSBQkEp", + "bloomSize": 262190, + "indexHash": "QmWhumzHJAeKP2vNHa5bEoXKo8XrLN6BRQ1cy6irtAyt2F", + "indexSize": 3402736 + }, + { + "range": "004023405-004024184", + "bloomHash": "QmP2qrWCCoVJ6aKjn5kbp71e6EukAs3aKpUvy4a8UcC6va", + "bloomSize": 131114, + "indexHash": "QmVhrm6JLaU5zXVjFvMnTp2KJzKXDfU8kEwMittCxroJm7", + "indexSize": 2890312 + }, + { + "range": "004024185-004024833", + "bloomHash": "QmbbpFKH8fxqTtUQ65QzW9n39EfGmPLZEu4Qbme73axazQ", + "bloomSize": 131114, + "indexHash": "QmXMyZJJTdw9r9gpLUqT8cPyazT9xwoWGqCmCktR2HVY9m", + "indexSize": 2572872 + }, + { + "range": "004024834-004025242", + "bloomHash": "QmVidByYzTiY3WY1j9cr9tdWWfUP8TnowSXQDh46ziKV8d", + "bloomSize": 131114, + "indexHash": "QmQ49LtHXGktCHuY25Pd78nv2NstPxJFQzn4v3oZ7UV7sG", + "indexSize": 2459000 + }, + { + "range": "004025243-004025931", + "bloomHash": "QmWShG5x8a8uw6qbgk5fnxjDHNXgpeAEsA8GCytYKx32ns", + "bloomSize": 262190, + "indexHash": "QmX6yXjbiUhEe92cYeUGQwuBXvdTpzNqwmfy6bQpY28S5e", + "indexSize": 3521780 + }, + { + "range": "004025932-004026670", + "bloomHash": "QmZf23pZXdsRp7mrWKg6GaZtnxPbJCUz47PQnftraox2fj", + "bloomSize": 262190, + "indexHash": "QmfU2w2ff1SYENN6FYpMR3tK9fsCoUreoxcsC7EB7Ps9r3", + "indexSize": 3668232 + }, + { + "range": "004026671-004027257", + "bloomHash": "QmWS662BEN4gkKo8SLv5m7nedjqfRReJxvz2P9sEqjgzzB", + "bloomSize": 393266, + "indexHash": "QmdiLdWyqmHdkYSqXzKVwQrpypb3TUztwajV1VqdHz923B", + "indexSize": 4527560 + }, + { + "range": "004027258-004027833", + "bloomHash": "QmT3tpCPYJobFrEqCKbaVq9ovyWJqUEZm9XHkzfW2sutyw", + "bloomSize": 262190, + "indexHash": "QmNva9iyoFL2aBjuSymoQz3nAsK8WA96MabVeL7v8bRfLq", + "indexSize": 4221928 + }, + { + "range": "004027834-004028505", + "bloomHash": "QmRntkvSKK1P34pvgAFMLSow9k3chc31GVvugr6AjFeEUn", + "bloomSize": 262190, + "indexHash": "QmRE7b1T6hQCKAHCD1v6xX773EdA9DcGbUyeFYSRywBvFG", + "indexSize": 3768188 + }, + { + "range": "004028506-004029107", + "bloomHash": "QmPx5rWsYWaLJkRePJNeJNk28TXwUpHDYr4RZGJvT64Rfd", + "bloomSize": 262190, + "indexHash": "QmPG59BkHnuSNW1Friib39rKNc39CDLwx1GiSK3kekN6sd", + "indexSize": 3811460 + }, + { + "range": "004029108-004029712", + "bloomHash": "QmRwJhZDmAf75J4nANq96R7VcXdTdEdcu91mvjVcveJCni", + "bloomSize": 262190, + "indexHash": "QmU5KKt69vc93ARRzL5BD6T5QJcxjhEs9k27oX6i1pitQx", + "indexSize": 3610872 + }, + { + "range": "004029713-004030442", + "bloomHash": "QmWbHbELmYe4Z5S4PUwP4YFVUDb1WRMcB6BGCDSn8ZgTqZ", + "bloomSize": 131114, + "indexHash": "QmYi4CmrETUkhPsz5HW8XSkpVLa4eXNBguniBZR2tuq5B8", + "indexSize": 3001644 + }, + { + "range": "004030443-004031013", + "bloomHash": "QmUh2TuC3WET5CZ5GqkSrjdgqKUMm7E3rmR7xmNdLPGjRf", + "bloomSize": 262190, + "indexHash": "QmS7TK9MTepUSsHeaw6kXgQHKLWayNi8ubocwmFxN6BMfF", + "indexSize": 3198048 + }, + { + "range": "004031014-004031574", + "bloomHash": "QmUBhZr7rv6pae6K9mRFeHy6oJgTSGChHyV7mDtDjVWHrE", + "bloomSize": 262190, + "indexHash": "QmNcPcikhgN883mTyZ3PaDZAXTfhcSazs93zuxXmRqBD8h", + "indexSize": 3587116 + }, + { + "range": "004031575-004032189", + "bloomHash": "QmdK9JVYYzhxcLBVE7NgHGCu6H7Qvyp9XZf2BLE5WY1NrP", + "bloomSize": 262190, + "indexHash": "QmS2fGibmdRLFwBfujoiASR7TargT3MUAmJKGckJNAk9Uy", + "indexSize": 3686112 + }, + { + "range": "004032190-004032882", + "bloomHash": "QmVNmBqbTBECF5FJwKZsED8HABcLqcyeWJBUKDk2xAgevE", + "bloomSize": 262190, + "indexHash": "QmQbRstBpkQMMXuc7CZwbUxmiJ1D5LUXm4qya6vWBwZMtw", + "indexSize": 3184076 + }, + { + "range": "004032883-004033535", + "bloomHash": "Qma1L66wQAhemecfYJjBWJqV2onvnb8beprL4Da6MLtUnK", + "bloomSize": 262190, + "indexHash": "Qmcg9aTUHpVGzpifi97f41ncCLD5X2BhtDMJDc5j6jSnWV", + "indexSize": 3531528 + }, + { + "range": "004033536-004034349", + "bloomHash": "QmVcQTktuoQCApMmzqCJ7M3arbRotZ67MCa1ibMUAsRq6j", + "bloomSize": 262190, + "indexHash": "QmWvsZKyeWtsRzoB1hVgYtGDMqD5zv5tfXQfRaJFJsgFq6", + "indexSize": 3390008 + }, + { + "range": "004034350-004035169", + "bloomHash": "Qmc8duNKuR4FDc9BRg7D3cgB4PXbS3GfHwqmg3H9tFWfZX", + "bloomSize": 262190, + "indexHash": "QmZFQER6b84ocXbCH4HYyfY6JYwCsD8Cg7SnEspbaeipBN", + "indexSize": 3224904 + }, + { + "range": "004035170-004035694", + "bloomHash": "QmWF35PaftjRUeJRpAJRczGUcqvGkMAodE9rnaxPZSdTDs", + "bloomSize": 262190, + "indexHash": "QmWUZ9Uao8o9pRtfVrHftVnfKwsADXsbvYRW5kL17Kqdr3", + "indexSize": 4182624 + }, + { + "range": "004035695-004036259", + "bloomHash": "QmdTcZr2t1ChwfaDH5yaJJF69pSzNJEKhVbYH6GwSg3EBL", + "bloomSize": 262190, + "indexHash": "QmdijQ1hTfCjXpcu9R5YtDVs2o9RaV6TYNBiW8EkDhsSnu", + "indexSize": 3664420 + }, + { + "range": "004036260-004036850", + "bloomHash": "QmenmqCj3oNUNk8CDhL8ZU3APWuJ8Qc9TBRsXx3chyU2sM", + "bloomSize": 262190, + "indexHash": "Qme2SC34RE2Km7fLiVCoU6EDzzU9QdqWSniAsyLwke6wB6", + "indexSize": 3560976 + }, + { + "range": "004036851-004037477", + "bloomHash": "QmfJCFAQ7rB2VkgV1d1SRwbBqnQ6r8bE96UzecSfn8VA2r", + "bloomSize": 262190, + "indexHash": "QmTf8CdocLVMXzvXF48xcTnwqbtjRnd7z7QBapH6yomywu", + "indexSize": 3142184 + }, + { + "range": "004037478-004038102", + "bloomHash": "QmXveWWzZBiV7YgZX7NMvaQvcue665UCbfKhgywV5Esb4R", + "bloomSize": 262190, + "indexHash": "QmbZQENP8XDvmPpRvFA3JWPXyRmyWsbTtxq1pWB43AsfzB", + "indexSize": 3038460 + }, + { + "range": "004038103-004038857", + "bloomHash": "QmdfT4thz7TSUmYSsUTFYEwfWzfgaYC6zAHmhpkWTHbZ11", + "bloomSize": 262190, + "indexHash": "QmeVBnqrs8MMVANF65gpHWzccZQQpPCa4LkroKnj1Qwihm", + "indexSize": 3103956 + }, + { + "range": "004038858-004039681", + "bloomHash": "Qmat57yjnBMeakYAMiqc19jDccPpjsrGk33V5cHb2iCFhe", + "bloomSize": 262190, + "indexHash": "QmcUhcUesGHeEDixNBjW4kNTyP8vLEdSNHCqrvyxFrmXrE", + "indexSize": 3075248 + }, + { + "range": "004039682-004040465", + "bloomHash": "QmYHP93eFxvXgXUMB944YbEKB2CBEhH7y3dwFetZhzT72m", + "bloomSize": 262190, + "indexHash": "QmQJTQBfZGtb4uyYZCnVEzBBr1AUci2ctvD4LqqZmGgrHP", + "indexSize": 3055432 + }, + { + "range": "004040466-004041219", + "bloomHash": "QmbNorDwwpvwf7CL3xaa4ew4iYqM2qEwagCGNiZGZfQMuS", + "bloomSize": 131114, + "indexHash": "QmTQbWk4ktfTbz8tevK4kXwENaj15szeaZ8s44NpGvfLh6", + "indexSize": 2825232 + }, + { + "range": "004041220-004041702", + "bloomHash": "QmZ2md5jGnm1Vf453Tr55D9QmtYupDxVbJNnkp4bo8Rvf3", + "bloomSize": 262190, + "indexHash": "QmYA3ALuLWJ4LwmZpWRP38kK1VNtZN5Fdmjhaf7Wk8fNhS", + "indexSize": 3850096 + }, + { + "range": "004041703-004042377", + "bloomHash": "QmdmWpGmVFrfwpFU2dDJM5XmgZocTiEytcpyfU5NXnYBah", + "bloomSize": 131114, + "indexHash": "Qmdxg6zLPbu44N5CLNTR9X45sAiPsz3s7jXkCK8eDvQuEN", + "indexSize": 2741592 + }, + { + "range": "004042378-004042558", + "bloomHash": "QmdHJrQMMArt62yf5QqMiECA2SywLxX3MeTh5qFRUhrnMh", + "bloomSize": 131114, + "indexHash": "QmUjrdgLpLEWab3XWnyJUDwEkdKjYRxkqVSEbXh42oXYNe", + "indexSize": 1886264 + }, + { + "range": "004042559-004042710", + "bloomHash": "QmU4DGf9UWZKxiKDvL4uXCM6QPFMmP7iMKWJ9cLtruSR7Z", + "bloomSize": 131114, + "indexHash": "QmavetfCxbAXPj5548DMANw7vfUftwrSVQEcGeyC7sKHkR", + "indexSize": 2046464 + }, + { + "range": "004042711-004043361", + "bloomHash": "QmRjyVNNjSDsKwkxERDRfkiXrTWtnjjqFRyVjdbdWi9tsf", + "bloomSize": 131114, + "indexHash": "QmPSp6ViVjU1opnGoSF4rVo7FuZLpcdfhgawNs52hTajbE", + "indexSize": 2696748 + }, + { + "range": "004043362-004044144", + "bloomHash": "QmXTv7gAcxSgDzEmpLqLXkKaiy9DGv7nYfNNCbp5rv8Edq", + "bloomSize": 131114, + "indexHash": "QmbbeVq6f7UNzhSBLRtxYsRrEpVvXJyYS3U1SQksJg96Si", + "indexSize": 2829292 + }, + { + "range": "004044145-004044832", + "bloomHash": "QmU4UeNN1G53crsB59KkeNgvtsAsqDMqXKigy4wm9D7uFH", + "bloomSize": 262190, + "indexHash": "QmTjdXAytBN17LXZVz83nWShKspSP3Eps7Ct63joD99xJc", + "indexSize": 3200364 + }, + { + "range": "004044833-004045489", + "bloomHash": "Qmbm9ZJ3d9B677UbzbwNnvj5jruAQjUEwrRN5mf8E9gwXW", + "bloomSize": 262190, + "indexHash": "QmbYJnZrLzcc35dj3L7NMWqYPj53uGnaMKcoYuFdU3gqSB", + "indexSize": 3549964 + }, + { + "range": "004045490-004046016", + "bloomHash": "QmegdkwgRcW2GWt5Gcko6UTRtdFjdkyw91Zv2nbihMd1sJ", + "bloomSize": 131114, + "indexHash": "QmVoKq6PcT7ZP8Wcbh7qpxmo5np4p5ivWTHeFjqqRFViJ1", + "indexSize": 2390504 + }, + { + "range": "004046017-004046114", + "bloomHash": "QmSa6szHnSvswpUm8tJXp8WgnoqouvT7z4LyfvzE4sgnwR", + "bloomSize": 131114, + "indexHash": "QmacAoJb57twyBVVxpTi418um4dz4XugpoL9atNESGrChn", + "indexSize": 1814428 + }, + { + "range": "004046115-004046195", + "bloomHash": "Qmd2Qot2kw5oji6hgUTHZnheKv7hFNyLdvniWNGMHjdkyy", + "bloomSize": 131114, + "indexHash": "QmUsd7tFLpqXrHRbe4969v7gn2gArJ9e4WMDimHyECukw9", + "indexSize": 1823608 + }, + { + "range": "004046196-004046569", + "bloomHash": "QmecWLoATVaygZd87dyRJSvbKZjhmvNToFcY2QjGATvrXr", + "bloomSize": 131114, + "indexHash": "QmUiHRT5zQMqFeQ3b6mtaK29eWg7kgNKnBxDZvgyfL1Q2A", + "indexSize": 2206976 + }, + { + "range": "004046570-004047338", + "bloomHash": "QmUHcUxjiitycVAKuJFj3XPQNQywWGZ4YnRwD78gx9jTjb", + "bloomSize": 131114, + "indexHash": "QmdaHXxX5x4CwfkR3piFSgbYXFzdqLmJhTDrPJVujxvk4V", + "indexSize": 2333116 + }, + { + "range": "004047339-004048014", + "bloomHash": "QmRXF7yR678CpuVGGrB9AoVDe6dzMoF6WLpwL9PwnWYNPi", + "bloomSize": 131114, + "indexHash": "QmWrW6Ca8bzNei3oVoByhenx43KTukz4nUqS963TwHHyqq", + "indexSize": 2772084 + }, + { + "range": "004048015-004048733", + "bloomHash": "Qmf8WKdqyqGPDs8c2W9a9AAxHtSkKAqgJFzQFTyhzmP5Xw", + "bloomSize": 131114, + "indexHash": "QmWJ6Nh4vD2tFtuiree3CN2xrCrqZuP5zoiqNtfzoFvpbQ", + "indexSize": 2777040 + }, + { + "range": "004048734-004049290", + "bloomHash": "QmY2nxYC8zieQyvGQWdSydefwgbg9xPkULivM5ymkhxzWf", + "bloomSize": 262190, + "indexHash": "QmaSzXXtfR3pDg82HEm4youUEzGtnhSmcsKj5QY6wqcHDt", + "indexSize": 3639500 + }, + { + "range": "004049291-004049823", + "bloomHash": "QmeD3E5QiaHxdtwf8zHXdKerX86un2zLsPeKD7gFRAy8A6", + "bloomSize": 262190, + "indexHash": "QmekWPmZZtvdruzVaUVXX5D3pqFvD5tBuefwSNpUk9Cux8", + "indexSize": 4005520 + }, + { + "range": "004049824-004050312", + "bloomHash": "QmWzXP9ttzs9XxbSwW5gUCsBjfVbccpsevw7KNPncTAXWQ", + "bloomSize": 393266, + "indexHash": "QmRGo1cUHHPVdBC71hJK4pgv5ajbv8xZEgStreroxLtav1", + "indexSize": 4438592 + }, + { + "range": "004050313-004050732", + "bloomHash": "QmbBnQqUDXUh6k3nnVjYdVswBRQtQdZ1XnCnT2un4U5eRx", + "bloomSize": 393266, + "indexHash": "QmS6dS5zR618fRfKZ3r85oE7NyMaUMcCEE2vUUMd7JHkoo", + "indexSize": 4752504 + }, + { + "range": "004050733-004051157", + "bloomHash": "QmanozPfG53uD7kVErtR5CaCpXr5qkFYHnvvoLWdgxUHxf", + "bloomSize": 393266, + "indexHash": "QmYfSu9tTovdJSqi9pfxNvv3is4NpUj2fZqsGV3WGA1JoD", + "indexSize": 4681192 + }, + { + "range": "004051158-004051621", + "bloomHash": "QmYnrssAXU62nPhSZJk6kWe7EFL6woCMrG52zBLgL1iDRQ", + "bloomSize": 393266, + "indexHash": "QmRLqn2NNRs3gDUQkevLWX4vn6eMCCkX9JBF4pj8DB7krU", + "indexSize": 4953924 + }, + { + "range": "004051622-004052426", + "bloomHash": "QmWKgfCQCKhgxb5rLhTkcwvHWbmW1masaqSFzmz5wNaiiA", + "bloomSize": 262190, + "indexHash": "QmXhHXit964xnfMDVTJHNsQSbZrAA65KbYNfgztceAuFh4", + "indexSize": 3228348 + }, + { + "range": "004052427-004052812", + "bloomHash": "QmXZPcgrUKwBFuNBKZzKLd4e9HrGsxnoT3CfPuzSaCDoHe", + "bloomSize": 393266, + "indexHash": "QmdTu3RXviY8gppDi3pZP2kamFJkkcBYfAKLPtX45CVPgG", + "indexSize": 5131228 + }, + { + "range": "004052813-004052979", + "bloomHash": "QmTpehrvWWLN6x9c1EVJ3zWhQdndeKnvjJqz4jyp7msrfF", + "bloomSize": 524342, + "indexHash": "QmZAjtAbHJ8w1acMaFAVpdmcybRW8ArLSHCu3m46KXYVbd", + "indexSize": 6385908 + }, + { + "range": "004052980-004053153", + "bloomHash": "QmcqWn6RVdL4FdT4A1LzgRUF9sLF2VMN84vZ5obkC48CHV", + "bloomSize": 524342, + "indexHash": "Qme4N5BNTtRPGAqEAjQN6nybc1fgrEUdPzzqR7iEJ13hhT", + "indexSize": 6428452 + }, + { + "range": "004053154-004053774", + "bloomHash": "QmcgFcsLifnh9WZzTZRPiLSxctPwNvY3jfc7tHzpZu3yKF", + "bloomSize": 262190, + "indexHash": "QmaMQW9zkL7RErMHPzM5bdEHUvrBUnJmNACLcWtgdRXahD", + "indexSize": 3769620 + }, + { + "range": "004053775-004054484", + "bloomHash": "QmZ7EjXYGx6U4idHPiHwauKaMKM3TNE3bpLt6qNqmrR2Fx", + "bloomSize": 131114, + "indexHash": "QmfAoozZPj4eUjkFofkkNmgNtxo4EzpaF6ibiCcfGrMhbz", + "indexSize": 2586456 + }, + { + "range": "004054485-004055327", + "bloomHash": "QmYdDcbWFKf8fwf8SfjbChNoJPP4qnKCX6dq3Lp23BDNUm", + "bloomSize": 131114, + "indexHash": "QmVxJ9aVnmim6MoLR5ot41di6qByuTGahyQ8CtNxYMkF5k", + "indexSize": 2536516 + }, + { + "range": "004055328-004055837", + "bloomHash": "QmU3uwWCM662HDorfSRDa34Jx8mLdfje4bcnTYFumtkL8g", + "bloomSize": 262190, + "indexHash": "QmdXB7pv25b2gM3nLYipj8VGgXpSwKS5vj4ivAXQBtDXju", + "indexSize": 3896600 + }, + { + "range": "004055838-004056435", + "bloomHash": "QmPaAyq9sevV1WVavRsmiiTXDBK5QgNWrBp5psRAU9bKb2", + "bloomSize": 131114, + "indexHash": "QmV97CBW9b53v8GYbDTi37RLr1H6ny4ynsh8fmDUMK1YpS", + "indexSize": 2781636 + }, + { + "range": "004056436-004057019", + "bloomHash": "QmdcycyswWwo2o34uoDh29UG2pmZ1HkqU97xgy5bCsMovB", + "bloomSize": 131114, + "indexHash": "QmXrfeYqsuGApatvFkwxRYj8nQ9VZ8qXaEL8w5W9Q3yTJu", + "indexSize": 2809992 + }, + { + "range": "004057020-004057554", + "bloomHash": "QmVMjvt8sVLGpZgGDHAyB9UbHBs6hYHekaUwXrafJbdTqD", + "bloomSize": 262190, + "indexHash": "QmaVrHR34AMc7jQAhbxZ1e6qHxaeHsoQNQjdsZxqeWeMYB", + "indexSize": 3227280 + }, + { + "range": "004057555-004058141", + "bloomHash": "QmVhfaK27bAigyr95GUMQAMyV5AjutTQQie7yP8EBWhybL", + "bloomSize": 131114, + "indexHash": "QmdR8Q8Ngy5ed2ecs7vCoYjkLwit2ofdRV1gFL444ZWG1Y", + "indexSize": 2697044 + }, + { + "range": "004058142-004058745", + "bloomHash": "QmV9CAkfAiBMSyDsuCezcJewFNNQEb8GAkeBKYUrAkZZG6", + "bloomSize": 131114, + "indexHash": "Qmdvm5TEPBEdLYMxg9HQoAQjnRtJdbCQjxDmQbKMPiff8Y", + "indexSize": 2824540 + }, + { + "range": "004058746-004059255", + "bloomHash": "QmajiHfnfFTEDPXNuYwEn6PykKMNJ3kTe2ix5RdA5Tjozj", + "bloomSize": 131114, + "indexHash": "Qme8uWH2B7o7SPqto7r7E8vq5VZbH8TJYSETmAekV2Yhb1", + "indexSize": 2974008 + }, + { + "range": "004059256-004059801", + "bloomHash": "QmREnD2gTgMxG7LAcUjzJyzC1GGDKoZCLP6vgTZzohE9eK", + "bloomSize": 131114, + "indexHash": "Qmco9qXdgSSDkWpQaCBcysYLXRacnkiDB2FB6ykYtJx6GB", + "indexSize": 2905132 + }, + { + "range": "004059802-004060295", + "bloomHash": "QmYmjQRbTxLeU3nudTixqxzeFmqXjRskxbVjwPUR6HwaBP", + "bloomSize": 131114, + "indexHash": "QmYZosojubxueoK2UyNVhbBuf8PVfDnmCRTMYskXr2tckC", + "indexSize": 2808832 + }, + { + "range": "004060296-004060735", + "bloomHash": "QmVxeWQeAN6NVwDahTa6MM45wkGriAAoGbzWGw4D5uyt8e", + "bloomSize": 131114, + "indexHash": "QmWABtvVkQq1K7BSDW6LtaCRZE4QwmPTv5GXgMvfw64sUx", + "indexSize": 2528624 + }, + { + "range": "004060736-004061174", + "bloomHash": "QmdGxSd5XE32SwVuKzv1vMEPEgwaQjjNN4eL8BPCGhCBVr", + "bloomSize": 131114, + "indexHash": "QmQH8KVwArSuavjiTiyXhxMAVixiuFR6QANvjgLiaBovT2", + "indexSize": 2460176 + }, + { + "range": "004061175-004061645", + "bloomHash": "QmNMKpVP5LaV48ZLTP3cDoA5qs7P34n8icRXrV8HW8ec6P", + "bloomSize": 131114, + "indexHash": "QmU9aTh6krkg8ideduwAPn4bVyV9dtCG6BY9DjSEQco9cJ", + "indexSize": 2461860 + }, + { + "range": "004061646-004062107", + "bloomHash": "QmSqWMW6C4ThiNFJ42tQaHm5KgMkWApCuBVmo5cYRLe4yS", + "bloomSize": 131114, + "indexHash": "QmTEZ8p8RnQa5fiufxq8ScyfuAABZZzvZf6RA8fg2oTF6i", + "indexSize": 2488816 + }, + { + "range": "004062108-004062605", + "bloomHash": "QmTdmVYTP5qyN34Mp5i2xCVZjJ5WuM4DMwR8TjWhfifChR", + "bloomSize": 131114, + "indexHash": "QmS5nNd7nDFQu5cxBfPsAgDJW1voPQxfFi9nMNoJvDoe7m", + "indexSize": 2451408 + }, + { + "range": "004062606-004063105", + "bloomHash": "QmYPTuxx58NsCYq9PYW3seeaiiNMzV5k6wy7LcLPHmpNFD", + "bloomSize": 131114, + "indexHash": "QmNtuYN1Soe2p7fZjyyBpuMiFPvDawmF216MoLWx9WY9iV", + "indexSize": 2449328 + }, + { + "range": "004063106-004063649", + "bloomHash": "QmWJQAJpUhEiVZHUm6ZPw3TkHRyuwgZJLPZ78UoZcNdFwz", + "bloomSize": 131114, + "indexHash": "QmXf3q9V72yaeP1Jf5fF2qX15WNnhHwaydHfhgUnZ5QexS", + "indexSize": 2657980 + }, + { + "range": "004063650-004064184", + "bloomHash": "QmSPams1jj2ZXAHbpcZq4LsLYWXWG6hDShsSg3SFKPJRJ4", + "bloomSize": 131114, + "indexHash": "QmPaquTspP2sZcbKKJMfkBQ12J4DAYrP64UDC8BeVg2ART", + "indexSize": 2538092 + }, + { + "range": "004064185-004064823", + "bloomHash": "Qme38Dj5gRxnjwD5EwBtpT3FKVMifNd9tkHWwmKm4Ap7MX", + "bloomSize": 131114, + "indexHash": "QmbANRZLXtVPgV525VYbpxZDTU2nvxeaM1dyFx5gFjoxdP", + "indexSize": 2668856 + }, + { + "range": "004064824-004065440", + "bloomHash": "QmVNKJvu1wk4hoHWPbYzHckRaCTsQBQRsZhzydC4crxT3M", + "bloomSize": 131114, + "indexHash": "QmU5iVE5PNRLudwBZbt2KDHQwaYq3T2xc7TMWypoY7MTq6", + "indexSize": 2648476 + }, + { + "range": "004065441-004065946", + "bloomHash": "QmTeSwrqeUH2t9xwAJXfenjEQSsgFi6ipjnyPQM1dQC8Cs", + "bloomSize": 131114, + "indexHash": "QmPhPDzK4shpLxeYbpSfk4SJnNFmwVtWbe6h47Q6yQ3u53", + "indexSize": 2750668 + }, + { + "range": "004065947-004066446", + "bloomHash": "QmZeQZTJspbTi2n4e64KmbokamknPEnnwyQ1qwEMhVxfSV", + "bloomSize": 131114, + "indexHash": "QmfJfZV5suYfbcLpDMxHwf9iM9FPpL51jqQRwCYRJZiUJ5", + "indexSize": 2643900 + }, + { + "range": "004066447-004066908", + "bloomHash": "QmZqiGqERDJoF9jQ5ffFG2ootLK5gGSo59vEcNnjiy3fv2", + "bloomSize": 131114, + "indexHash": "Qme3116WQtDDwi8Tk1XzVdF1ce9oDL5gG5nHbUugzpHKG5", + "indexSize": 2619688 + }, + { + "range": "004066909-004067365", + "bloomHash": "QmSfbKpvVUB9tEMh7c4MwwEKLhmasRpQuP6p2eG3VUQFvY", + "bloomSize": 131114, + "indexHash": "QmZLSikeN6CvsiwCohtSaPi3pahW4pf8S67PcLKYj2Btau", + "indexSize": 2527676 + }, + { + "range": "004067366-004067831", + "bloomHash": "QmNj6DBaHjGxiqAtGSfve7VcmAH88yf1AqZAtvNc4SnunD", + "bloomSize": 131114, + "indexHash": "QmWmcdoou9ZtqVNtFa8z9CC474SqfQcw72DmW8J1v22bQM", + "indexSize": 2722204 + }, + { + "range": "004067832-004068286", + "bloomHash": "QmRuvPGgb9D2qRvDo86NDE1FTFWi3MTP5DWAJ2VwPdLJKe", + "bloomSize": 131114, + "indexHash": "QmTvDLhk347KrRYMhrg99Kap44nM1aJ4dFvAUMg6sc8QsY", + "indexSize": 2716188 + }, + { + "range": "004068287-004068756", + "bloomHash": "QmfFFtE4JGPN6RCYzQr3jHMrNtDDNxUGHWgcMsZ576bbQn", + "bloomSize": 131114, + "indexHash": "Qmbxk62xp1ooULzDkAkDMf77j8K1vDuECfzahtQPhjGH7e", + "indexSize": 2653052 + }, + { + "range": "004068757-004069231", + "bloomHash": "QmY8rKLFQLy2LZ17ZTTqMDJhhJkcY7RMrimDANr784v1Qm", + "bloomSize": 131114, + "indexHash": "QmNN8Xz4c4W3NUT7oSYpxEYZAMACK5dPFvk5TxQnzxN7b3", + "indexSize": 2705832 + }, + { + "range": "004069232-004069751", + "bloomHash": "QmY5aMPnKVXWD7rzbR34ZUj4MjWVVPsj72c9q5JFsBjV3M", + "bloomSize": 131114, + "indexHash": "QmVvo2pCsbf2ocXtbUg7eRwigkrSQZ6BoQvTB19T8tdvpD", + "indexSize": 2635628 + }, + { + "range": "004069752-004070283", + "bloomHash": "QmPpwwugDE8BowGTdX6rcPinj2dHQdu5WjF2uoHnWDAD5Y", + "bloomSize": 131114, + "indexHash": "QmTmeX263qN7ySMahBfburu6r2TaYfn7sHF2RSLkS8GVcH", + "indexSize": 2555220 + }, + { + "range": "004070284-004070826", + "bloomHash": "QmUW3js4gZDQqxESKKcicR3gjh61c3raZUe9CuaiVubWt9", + "bloomSize": 131114, + "indexHash": "QmYRUa7xmVmb82qK53XUvxURMj7y5yNuUcb5YKhBDReHGk", + "indexSize": 2562320 + }, + { + "range": "004070827-004071404", + "bloomHash": "QmY7TeWts2MnVbB4ytEBCZ8fUtgKyzFB7vRs442cGfkfwg", + "bloomSize": 131114, + "indexHash": "QmWyfufvi6nS9uhNTRz9qHERezZL5b6BFYEBAAkwMB2u1h", + "indexSize": 2611508 + }, + { + "range": "004071405-004071949", + "bloomHash": "QmdiGazMWBVtJmnfj5MnTrfhopuTqW9i9zPCzmCb3iEs2C", + "bloomSize": 131114, + "indexHash": "QmZff6dxiACEt4bxniSiCiAPkMHiFtUTJ3gmtaPMLb7p7z", + "indexSize": 2653504 + }, + { + "range": "004071950-004072505", + "bloomHash": "QmRRkLJa6dVaKwScKnSJVZFEERrKvrK97d32Cq2Z6sc7pW", + "bloomSize": 131114, + "indexHash": "QmUMV7ZbWWphhKfAtgKjWm4kAYjRFL8zTjWm1tVm318E5J", + "indexSize": 2629184 + }, + { + "range": "004072506-004073039", + "bloomHash": "Qmc5zyzgic6aW3hPf7wrGxpcLLHmtEitgmxihPUjyvwhWX", + "bloomSize": 131114, + "indexHash": "QmSquZ3N8Y1XVJKLYDWfDPiRGzN2vqPqbretHs4r3C9iFW", + "indexSize": 2586908 + }, + { + "range": "004073040-004073571", + "bloomHash": "QmVPDKdQZ5bLZT4gyBQFpPooXHNTtoKF4g3UGfct5KFYVk", + "bloomSize": 131114, + "indexHash": "QmekWjhtZMGoVaCm85bTzGByz8dFaPwnJRehBYYQppLbUW", + "indexSize": 2717560 + }, + { + "range": "004073572-004074103", + "bloomHash": "QmUeKPu1Xq13fzBLKL5Zhq4irYhHJRUdebyFDCTA9kj9yd", + "bloomSize": 131114, + "indexHash": "QmctC81Bqbpp7oFwdWcp9EYYQb3kcrBywrpfy9Euv2ux1J", + "indexSize": 2781484 + }, + { + "range": "004074104-004074644", + "bloomHash": "Qmarf1sqN79uGkbwott3JjN7oYWAdu3FUsBWup7A4GvPNn", + "bloomSize": 131114, + "indexHash": "QmPo3F3RHS9SNTUpJQLcJSzEiE6fST822r22VP1aKiuTad", + "indexSize": 2790216 + }, + { + "range": "004074645-004075150", + "bloomHash": "Qmcs2JTcBgCcbgsSYJFh2u27tA2EaWmXMRzRYpWrWy2sJz", + "bloomSize": 131114, + "indexHash": "Qmey5m9bcdzjpSkSUug4r3aXDSwUpNT2jV7uMQytJeWYeY", + "indexSize": 2631212 + }, + { + "range": "004075151-004075679", + "bloomHash": "QmXNWHstwGvsVxP9VMh9Z65inrDp6fM2FdoHFwVbtNQqdo", + "bloomSize": 131114, + "indexHash": "Qmd2Sxzw91AtxMMd9xUvRYPficoDVo4kspjV5UhycfZXYd", + "indexSize": 2726848 + }, + { + "range": "004075680-004076235", + "bloomHash": "QmPnmYayuiyczYFdFAchKua67ZizzF9v83XCWGMVbnpaEu", + "bloomSize": 131114, + "indexHash": "QmUC5Vf5GhHvt93jS58RkLX9JRQ9uT4b9tvnW5Bu9efrVD", + "indexSize": 2822032 + }, + { + "range": "004076236-004076832", + "bloomHash": "QmaRVvnWvGygGfEqkLxr5xAb4QcVoVgTgfPyPeTLUmU5hQ", + "bloomSize": 131114, + "indexHash": "QmfKW4iJgZ95JbFvHDXnJW1AxT14voFWLLggGM4cgrvrNo", + "indexSize": 2932660 + }, + { + "range": "004076833-004077495", + "bloomHash": "Qmb43ijRucBSP5KZwvNbSJo4mKNRxZG7Ymq6Fr6hMFmTvT", + "bloomSize": 131114, + "indexHash": "QmWLYHvcTG5jUe34Lsnvkfv7uunS8JUbFEztD69iWWyLP6", + "indexSize": 2959436 + }, + { + "range": "004077496-004078304", + "bloomHash": "QmZ2ukAEAz4Ab7Z9gmsuN2i9YSWAFSaZF7K5k87SswVc6m", + "bloomSize": 131114, + "indexHash": "QmeqNNUDmhHCtQDsRLuXFRuvp8ipynqCeKiJ8EeV9KgZuA", + "indexSize": 2906288 + }, + { + "range": "004078305-004078994", + "bloomHash": "QmWvJYvGb25TFMGMmVpeFWiLbCVjB1mfRqxbb18sNLyh88", + "bloomSize": 131114, + "indexHash": "QmP6uLp3sLCZAEMSQhDKpeAYZe9kWHH8DuavQDc3BVhqaq", + "indexSize": 2740196 + }, + { + "range": "004078995-004079608", + "bloomHash": "QmdhGyEmrHPFjLyMANma8Yf8KCksx7ZaZH3tAUz5q1pz8t", + "bloomSize": 131114, + "indexHash": "QmTeFLBxaigC8YAjCdGBAYdy8ronpvqu7qVL1GZEJiVFKP", + "indexSize": 2454980 + }, + { + "range": "004079609-004080214", + "bloomHash": "QmXvjjwXJUyzQeRLY6rx9fW6wRjmAixv9oQWAEL95L4wgG", + "bloomSize": 131114, + "indexHash": "QmYb5cfRKzkgH7tAo29jxg7kjYJNQGfrqwEMfLVN8zb1hs", + "indexSize": 2462136 + }, + { + "range": "004080215-004080846", + "bloomHash": "QmUNRs7t6fEBsjhjKpJJH8ZBrFLwSdReiqhfZRg8zVQeZ3", + "bloomSize": 131114, + "indexHash": "QmcWY6mkj18hu9d5N7Z7krmN9Amvh9zwdNZ7Q5F1NLqzuE", + "indexSize": 2540624 + }, + { + "range": "004080847-004081399", + "bloomHash": "QmR1j29Hgks354fXjf3wuivj9smssG4GwdQMyMFZ6uaoCp", + "bloomSize": 131114, + "indexHash": "Qmbe4bhu6JsRgsY1KGYFpUbSo5QSxw6V6DU8v9UiijWRSo", + "indexSize": 2821064 + }, + { + "range": "004081400-004081956", + "bloomHash": "QmWDpUxrSBnZMjFbkraP4zfJbe6HgPvdBFSj2VDkEnp2La", + "bloomSize": 131114, + "indexHash": "QmUgeCyuNEC1ij2awrZepx9gm81kn5tNRy7SEc5sKg12Sd", + "indexSize": 2550460 + }, + { + "range": "004081957-004082550", + "bloomHash": "Qmbs64tBDbUcyKCYQ1YKvBFwv24fU2g8WikbsvQtpY3Sdg", + "bloomSize": 131114, + "indexHash": "QmVCpsCLadT35qd6MYTTmj9sbtYhapFydt6Vzi5cLWHoqz", + "indexSize": 2828476 + }, + { + "range": "004082551-004083057", + "bloomHash": "QmYeRg2vbdRRuLJ4Key2Sth5mEGU4Rj6jF6Dq7915F4Qnc", + "bloomSize": 131114, + "indexHash": "QmcwK94kR7uSPJXLpFM9AASnvTZC4N9H6az5g1dZ6Bjjin", + "indexSize": 2689928 + }, + { + "range": "004083058-004083594", + "bloomHash": "QmeU84HxQxGaAY3wtwY3WiZWsJ3aLdEeViAhFc9DgPgQBe", + "bloomSize": 131114, + "indexHash": "QmTCUFG2oTygFqHHWgpGT5TXSFXh5oV4CcScy5A9zET5tj", + "indexSize": 2526264 + }, + { + "range": "004083595-004084216", + "bloomHash": "QmPJZZ4n22LXwJmHc9CmEsx7iXW192DxVpk4DsbFoqNysc", + "bloomSize": 131114, + "indexHash": "QmZxuLvj7mXwZEUsoYiSVczytRK5phfdfqnUck9fm15EW3", + "indexSize": 2342352 + }, + { + "range": "004084217-004084741", + "bloomHash": "QmWemzkNYo2DCwTg9WXxgUqP2ebNPz9jNGf7CaB1abTwfD", + "bloomSize": 131114, + "indexHash": "QmPFp34Kqrkafh4LLkptUz52GbqDE1B9eJCJS2eQv3reAb", + "indexSize": 2437492 + }, + { + "range": "004084742-004085183", + "bloomHash": "QmVd1Q3GgrwUH76nd4ukQKL8Ust3JssxgdeBHXwCVY69Pp", + "bloomSize": 131114, + "indexHash": "QmXLf5obfsKe3S49xDw2J2B9jkdzPkHPkteTZ1X8kbk5UV", + "indexSize": 2480228 + }, + { + "range": "004085184-004085565", + "bloomHash": "QmVkrXiWfwLbyNnmwTFAwNU9hLwbxGsbgzxmNQTvq3hLZ2", + "bloomSize": 131114, + "indexHash": "Qmf2eKqrujRwGHAfcq1ZieXoDtNbHDs9G1N2yYGbDaL9VX", + "indexSize": 2723764 + }, + { + "range": "004085566-004085974", + "bloomHash": "QmeYYojKQpHV4oT5U69KA1jecLH9xGSGt18GNTcbnCmsHN", + "bloomSize": 131114, + "indexHash": "QmbnzN4zLtr4U56NrTvVCfzAxApK7Bds6RsaYdr88bx6mf", + "indexSize": 2646272 + }, + { + "range": "004085975-004086388", + "bloomHash": "QmSpvNkgJxw4YvyD67Lv7t9TqkuW7FTaQbCsmtWKDVrWvY", + "bloomSize": 131114, + "indexHash": "QmNmuCZEaVM9xEzQEXtutvg38Yi1SSpwnFEdmKxcKtY5Hp", + "indexSize": 2537600 + }, + { + "range": "004086389-004086840", + "bloomHash": "QmR6jQPbbhnbJDpF1xRgZ3KH6kt4Q6PDcpinUwT4uCZWV9", + "bloomSize": 131114, + "indexHash": "QmYxJX5KJ4devXeS11f5kQqRUVKsoSqrUDvtsBRcgZz2Fp", + "indexSize": 2385356 + }, + { + "range": "004086841-004087344", + "bloomHash": "Qmc5MBQGpB8mcxVXPtfc8Tzt4QC5L6aTmNpLBkM52bBjhh", + "bloomSize": 131114, + "indexHash": "QmXMbRKwdGvjkFEKZkRCKHGDbs41TAwKSMVfRWEQFfeUTW", + "indexSize": 2461024 + }, + { + "range": "004087345-004087818", + "bloomHash": "QmVzbcMieRNfupyrow4qwHjEgZ7pesXNbigtC2oXwVbcq6", + "bloomSize": 131114, + "indexHash": "QmQLuHt7xkfkRUsg4dhbeziEvbtEAweVPiHrTmCoPpY45d", + "indexSize": 2411104 + }, + { + "range": "004087819-004088281", + "bloomHash": "QmfHVH3q6QBWZEnyFgyyMBzmNmWRN9SYGQ6bhjQHpRbDTF", + "bloomSize": 131114, + "indexHash": "QmVz7eEvp8LUXifFwQAqaTM8n3jnTLcbLhBiUszaemAijR", + "indexSize": 2564280 + }, + { + "range": "004088282-004088538", + "bloomHash": "Qmby7Z7tYgf2Csh71JG3njkLBQkqL4VhHLTVSgSeL6MKiL", + "bloomSize": 393266, + "indexHash": "QmXUx3DsoLfE4SMm6cBwdpDX4Ay8zqJ6F9x6aBRCJkz9Ar", + "indexSize": 4477224 + }, + { + "range": "004088539-004088947", + "bloomHash": "QmTxNJTP9tiTXtL64nfhtnzj6CENea1c37Ae14ynaujWfD", + "bloomSize": 262190, + "indexHash": "Qmaz465WnZMHZpc3tuPZoEdDJbpmUL3GtezxKnFPvHVtAR", + "indexSize": 3438920 + }, + { + "range": "004088948-004089477", + "bloomHash": "QmVV7vNh4kBhjHyBFtfGZ1BDq4JwWXYGE3ibEjMCXWGiZX", + "bloomSize": 131114, + "indexHash": "QmU4qzApdCdNPxVZEaaWbq8Z8X9LAvUVnrivUJBYmkyD3Q", + "indexSize": 2500360 + }, + { + "range": "004089478-004089999", + "bloomHash": "QmQdgiNM1bLZUAtSqVxjnRQBFmhHKd7pMj5BPG1ktBmg7L", + "bloomSize": 131114, + "indexHash": "QmP8jrPUn3no8TzfU3UCeAeTGWE3vYppbt2MXoj1LW1ZB2", + "indexSize": 2363588 + }, + { + "range": "004090000-004090579", + "bloomHash": "QmYmKCxAsprCjju27C6SPPYdoMxhmMiDPLTHxVL2Dcps3J", + "bloomSize": 131114, + "indexHash": "QmXFveqt65fejtDDsf1htvLVehSdmU3ExTpyJxcUvsz2i9", + "indexSize": 2452104 + }, + { + "range": "004090580-004091103", + "bloomHash": "QmeY9L4cU5vzXynVj1bwAzDmYFsWkmpe6JY13QsjxfaYCU", + "bloomSize": 131114, + "indexHash": "QmciVhzUFW2HCettN3zgjmZzGLQFkSuazwppvkFuQp8CBo", + "indexSize": 2215444 + }, + { + "range": "004091104-004091590", + "bloomHash": "QmQDALRD2rRCmxtwfy4wWPAsN1g9M9AwJa4nmT4n3Wmb5j", + "bloomSize": 131114, + "indexHash": "QmUox6L3njnR7Mmw7M9hLuMuRK7HEPuGg1Y8ZzEWEpGFjA", + "indexSize": 2403512 + }, + { + "range": "004091591-004092040", + "bloomHash": "QmfCFGzYf1BgxLiD634Q2SmeRWAE6Hviro6eLCHYr4FFvy", + "bloomSize": 131114, + "indexHash": "QmQjs3xksCnZzVZERjCewWQvEbkZ5eF5AmdwFQ7ftzDcbS", + "indexSize": 2382600 + }, + { + "range": "004092041-004092493", + "bloomHash": "QmSEpZ3fxQRveKeHUoFoxSZxR9F4bmKMQx1rnNFLFcuteA", + "bloomSize": 131114, + "indexHash": "QmWuoCLSqMMXJCiEYtDS7KxdKvP4LxLMvq9aSyBoZyY6t7", + "indexSize": 2356868 + }, + { + "range": "004092494-004092886", + "bloomHash": "QmPcjRfFGkNiozTNR6Sos78dZwsRdjyr682mLvQojb14vZ", + "bloomSize": 131114, + "indexHash": "QmUMxtZQjHqKMAyHNFo3V5nAEdkTH8UhKv1Sfuo7JoL7gU", + "indexSize": 2332988 + }, + { + "range": "004092887-004093310", + "bloomHash": "QmZSUNQV5LjPhLjDye6ngKaRGT7q9cwp78Fv3ubVEwYQEX", + "bloomSize": 131114, + "indexHash": "Qmd8d6KiED9pRmZp1DNNSQkPrMBnJZp11tJVj1moaENPcy", + "indexSize": 2369196 + }, + { + "range": "004093311-004093724", + "bloomHash": "Qmf89JjtwWFJS1aTR8bVoyuDQ2BMPtuy5K5JSsGPxtpRaR", + "bloomSize": 131114, + "indexHash": "QmeoZY4iXcxReHLgjSicxYvtuTRnBkd9swfDUgXz5Q8NX8", + "indexSize": 2490564 + }, + { + "range": "004093725-004094172", + "bloomHash": "QmQsNZ24CVrVwBW4bY97eV6jm5XTvQqprcLUbpf3VPXH62", + "bloomSize": 131114, + "indexHash": "QmPuYAH8TR9q8M89dggwSRZrPAdJps2DUXxrtktVcrAW9o", + "indexSize": 2342884 + }, + { + "range": "004094173-004094582", + "bloomHash": "QmPwwqeYe6TkUC5JR3G3VugCrmUst6oL3mnBi1rKbLzKt6", + "bloomSize": 131114, + "indexHash": "QmdCnvd9Yjx99Gqt3CZkWPoLcMWfzF3ujfZzdDnH2eAKMx", + "indexSize": 2418436 + }, + { + "range": "004094583-004094978", + "bloomHash": "QmXo12xFJKKhi8zMJSe5g5vn1xgm2DZ9jqQU4TrVm46x3q", + "bloomSize": 131114, + "indexHash": "QmWoc2bFBK14oMBWLSGKkGRQrYvEfwJaqstYN9QX8YgNym", + "indexSize": 2393876 + }, + { + "range": "004094979-004095370", + "bloomHash": "QmbAKi44gGC8UjgWWtTSNePyVf6ZH2F4uBVNy3C5inDKAk", + "bloomSize": 131114, + "indexHash": "QmPNEc4moSR3BFApxDaLPGnLYoZS89WMR2m537p7K8GEjf", + "indexSize": 2253664 + }, + { + "range": "004095371-004095756", + "bloomHash": "QmPCsMU5Q5apiweozLDK4QYLno9cfaFYNYDUiYqZfR2Zc8", + "bloomSize": 131114, + "indexHash": "QmZb65U8LCDM85gmYF8kvfmzSqaPfWcgtTJx3fZyemK7i5", + "indexSize": 2397900 + }, + { + "range": "004095757-004096166", + "bloomHash": "QmQoBHvBvhLJbdenAyB4vS3ab4VSSjTZV6NF2VwgnFMrvu", + "bloomSize": 131114, + "indexHash": "QmVuoFvSiyHKiPXmCB5iYv63qbTEnzYUzQC4ABWtyicFm1", + "indexSize": 2373180 + }, + { + "range": "004096167-004096536", + "bloomHash": "QmQURLfFLhY2oRgNuwQD44efiaB3Sr21juLFhWr7Gbp4if", + "bloomSize": 131114, + "indexHash": "QmPP9PHn3RUGep8L1tn9rgExoFAv2enUu5AazWJkNhtgSZ", + "indexSize": 2402216 + }, + { + "range": "004096537-004096911", + "bloomHash": "Qmep9ijpsWsW5ZES35JWbQ6SbmJf7inZi72DvmwcYF1tGN", + "bloomSize": 131114, + "indexHash": "QmQDQDrqk2FZuTsDeFxeAfq4ZZFvyCJevJRT7s5W9kWh72", + "indexSize": 2524252 + }, + { + "range": "004096912-004097266", + "bloomHash": "QmRu66PUqzB3bXQYpqzr7wjxcesFBYfAXTBMjZEyKWLaKE", + "bloomSize": 131114, + "indexHash": "QmdgCuGtbNzB8T3rq535VW2uz2RAjeHhfXccgNnQ5MtuY2", + "indexSize": 2474864 + }, + { + "range": "004097267-004097609", + "bloomHash": "QmRap6RYjNbt7SjE4CGZ23KhnFnbyYwQHG3wj34WjN28Ui", + "bloomSize": 131114, + "indexHash": "QmcNNC3Ltd1CLi9QHuvC3uFdx44c5byTEAjMrXQaneD6uh", + "indexSize": 2439176 + }, + { + "range": "004097610-004097935", + "bloomHash": "QmV7L5H4PzsZ5XYUULvAnbCZnqiQrrfDuKZxDvSGgmsXAT", + "bloomSize": 131114, + "indexHash": "QmRTWtJm8Lao7dTewkE1qeRVLrBgVj5b4MkPeb31CMG4gJ", + "indexSize": 2459076 + }, + { + "range": "004097936-004098261", + "bloomHash": "QmVuid2kRqtbRhhA7cZQW49z5Jt47bzVCVvCozHN1JFuwV", + "bloomSize": 131114, + "indexHash": "QmXuWmADVNd52za933Dn8BrNApYR8CaUGSKBSzW1gUDMHC", + "indexSize": 2404416 + }, + { + "range": "004098262-004098585", + "bloomHash": "QmP4pQa7nSZaexUTG8LV1PYcjYEyeSXszH9JcUvgoCGutp", + "bloomSize": 131114, + "indexHash": "QmTTWYSDKbZ3iRQjkpxqBYtMmDhZ5L9d7CojSsQhAF4DFF", + "indexSize": 2492096 + }, + { + "range": "004098586-004098917", + "bloomHash": "QmdpcsKvMAbnMT8iTiDe6SQZ3A6iaH8NHPP9hB6Qn7QvM7", + "bloomSize": 131114, + "indexHash": "QmTSvnDiXD1McKLDmeJUnduU2gJfVDmBS4NK3jGCjgprZv", + "indexSize": 2451576 + }, + { + "range": "004098918-004099231", + "bloomHash": "QmQsPxJFRZUoCwhP5Cm26vg7WP91XCg3ELr59nnGE9pFez", + "bloomSize": 131114, + "indexHash": "QmZCHTQb9AjtFkFgSi6q6w669kkcADkdfwGg8EyzCT9F7L", + "indexSize": 2317780 + }, + { + "range": "004099232-004099542", + "bloomHash": "QmeBNMc44uBsAm91kEDVoDBKKTybmMoLURH2PCJgnYnqD7", + "bloomSize": 131114, + "indexHash": "QmP1Jq4J1C37Zy5P6VpfBQi86ioiM3oxn7bwrFBR58xTyi", + "indexSize": 2451672 + }, + { + "range": "004099543-004099937", + "bloomHash": "QmePVwg4uw4b3f4CQTkepeMwnmJef6fjVNtFCYDDtFxJPE", + "bloomSize": 131114, + "indexHash": "QmeW5kmjKkszsZtZPJy1eS5cxFz95vie137T1A8hHDwDLN", + "indexSize": 2491396 + }, + { + "range": "004099938-004100000", + "bloomHash": "Qmahh6tGu9Y673zMrgZdtsYmid3U68kfKtiWzkV4VcVJu3", + "bloomSize": 131114, + "indexHash": "QmZ5kwDvuZnmh3USWyUVx228Q1NCzVvvLXKDAoS4DuCv8Q", + "indexSize": 502760 + }, + { + "range": "004100001-004100406", + "bloomHash": "QmW9dF334ikmyhmKT3LQudPqPURrYoc9qfsisM2ZLw9AoR", + "bloomSize": 131114, + "indexHash": "QmboFXzregmDrdxaqf15muEwwNPqeNxHBi1CAetbZNok4X", + "indexSize": 2673224 + }, + { + "range": "004100407-004100828", + "bloomHash": "Qmc917AmAD2SVnAK9RZhKfZRsvugGfjjYKiCjPZRay6w73", + "bloomSize": 131114, + "indexHash": "QmNTqgfqozGgWr97krvfsuYuDasXcXwj6FEgtRU5jPtKzD", + "indexSize": 2557968 + }, + { + "range": "004100829-004101199", + "bloomHash": "QmPDdoHmAxh54HP7sjgdm4nDYvCqqUDXJdqUzFkNW8yvWS", + "bloomSize": 131114, + "indexHash": "QmdjCoMB93MtFg9rGyq7DNB48CdUeTW1aUZDzh2fu59vzv", + "indexSize": 2348816 + }, + { + "range": "004101200-004101618", + "bloomHash": "QmerG4zgc45o6v55TpY6ttuJuNnF1f2vx5vEhyDkUZBpTb", + "bloomSize": 131114, + "indexHash": "QmT2TjyQ61S2gj59BmC6ET897nF5Mj1eAzpULQVQDRfC9R", + "indexSize": 2411836 + }, + { + "range": "004101619-004101990", + "bloomHash": "QmXN7MDuaevoYkdy8oiVkH1zP3v4DeiMpKREsKtWQiaGMJ", + "bloomSize": 131114, + "indexHash": "QmSLcXXGwaWn4aL1LG4W2Jmgzw72QjmHfw9rfGXBEXxqfH", + "indexSize": 2405596 + }, + { + "range": "004101991-004102412", + "bloomHash": "QmeCq8j4Hpn34z2BwFo7Sn5oa9HHpBiAMemMy5hWpe1Sju", + "bloomSize": 131114, + "indexHash": "QmPLYUu747RgPq8HYWo1n3n74sGhnMMQTBmGunNhAjLVUG", + "indexSize": 2491180 + }, + { + "range": "004102413-004102810", + "bloomHash": "QmUcb93WJMnduGeSmeo5JDjtg9WJjiSLdvVEXBnc2hKxVU", + "bloomSize": 131114, + "indexHash": "QmQYWtsr9dfo5QWAaf74Ymc2oJZ51RYBBj7Fb2GDbuVUit", + "indexSize": 2529772 + }, + { + "range": "004102811-004103205", + "bloomHash": "QmXALDiNTUM4p1fWaqarUa8EpaY8tYyjJFY9toFNFBFpUk", + "bloomSize": 131114, + "indexHash": "QmUdXX2Y6vXEcyKuLnEFRwRjmkTsEAAceWavLWaAhtRufk", + "indexSize": 2500008 + }, + { + "range": "004103206-004103660", + "bloomHash": "QmQrVYcSfED3CHNtD3Y2o9iDsXF9rDqvRVHwwNAFhbHseB", + "bloomSize": 131114, + "indexHash": "QmUjVjD3fzvNSLm52AA7AAsiUvaJCzzmHHhpZgaRjPqbg8", + "indexSize": 2709372 + }, + { + "range": "004103661-004104089", + "bloomHash": "Qmd3AmQzyYxpCWpvNU9ddSYYcdP87tbQe2jnncrQ994HwM", + "bloomSize": 131114, + "indexHash": "QmPHB9yQdzaMfefTNQXiQCCE9Ev2uzsxnkFQcAQifZdFEk", + "indexSize": 2674852 + }, + { + "range": "004104090-004104511", + "bloomHash": "QmaL1X3FYTLYJJ6dRjcncXKUnbbbM5vSTBgRC3wV2gZSNh", + "bloomSize": 131114, + "indexHash": "QmUCKUUGuN9fE9cB99drVpNr1AvBNJS9L2CotkjYQXfVV1", + "indexSize": 2521348 + }, + { + "range": "004104512-004104947", + "bloomHash": "QmeDpzRgPmvYDGTT6VbpgsqAQ41ChLcSq1sv2RCiuVQbcQ", + "bloomSize": 131114, + "indexHash": "QmScqtEZ781C3iztfu9a4rqigAKiHNiq68FbGfdJPbt56s", + "indexSize": 2449560 + }, + { + "range": "004104948-004105421", + "bloomHash": "QmPS7RY9B1A1BMLxSrtMZTuP4tYFSTEfSr8RqK6KddsMBa", + "bloomSize": 131114, + "indexHash": "QmfZRe4HVzmhnndrTnEyb1Xu6sQrypRfp4Nis4vV7pEPaZ", + "indexSize": 2326832 + }, + { + "range": "004105422-004105852", + "bloomHash": "QmcqYVhbujT3DvXqUsrZAB42qwkP4Xe57a4DTQUiGAmSxZ", + "bloomSize": 131114, + "indexHash": "QmR4sc6saCPd4dqBJPW16upaqLk4v5xdgdV2vXcPd2GnJp", + "indexSize": 2434432 + }, + { + "range": "004105853-004106319", + "bloomHash": "QmX3GhpPRvfhXSpazVMZrHs5mgoDGonkpdsH1TRr3yHPPn", + "bloomSize": 131114, + "indexHash": "QmdEeuZSedarG9QZY5AZXWZLsvpvrZHhW1DeBsFPZ8N6EA", + "indexSize": 2489364 + }, + { + "range": "004106320-004106803", + "bloomHash": "QmfLUPdQAqpwxbt9da2q1qVDTF5NK5uKANJMmbwTgxU1Ke", + "bloomSize": 131114, + "indexHash": "QmcssmdKaayVSCEuRPjNKrUeHfBGxeu3nMjEv6i18kHyY3", + "indexSize": 2555296 + }, + { + "range": "004106804-004107263", + "bloomHash": "QmbvJpkCHPDoALt7QCp3QF6RVgPYeEuvtEjwshSTvmCBf6", + "bloomSize": 131114, + "indexHash": "QmdVtFYibLy1zTeYLyiEpzhwXQNXScPaLtuLT9cABNCj2Y", + "indexSize": 2389248 + }, + { + "range": "004107264-004107730", + "bloomHash": "QmaXGpobd9hYMRBKnot8wd6rnb2eA68b8mNgd1JN6xsPiR", + "bloomSize": 131114, + "indexHash": "QmQ7TRBBNcTVBF4YUhqEG5QshybCK5HXfAUGQay3eHadBm", + "indexSize": 2028504 + }, + { + "range": "004107731-004108209", + "bloomHash": "QmPQWAXZ9VqKTgJ4Jt7P7g2ygXji8vML7nts1nGC4iEmFQ", + "bloomSize": 131114, + "indexHash": "QmSS5kt12phoxzdfbA6t51tBSsnFhXaLNEA5xydsjo8SqU", + "indexSize": 2017776 + }, + { + "range": "004108210-004108709", + "bloomHash": "QmY3bYBVXZ63K7oewfZ4NoaHsJ1TchVS9MxTWSzUH8j9RD", + "bloomSize": 131114, + "indexHash": "QmZFosmJV8VoErCDzm14ZtsRrkVAxHWhXEmh45N8AWJtgG", + "indexSize": 1949936 + }, + { + "range": "004108710-004109239", + "bloomHash": "QmdjfaP8ybfmcVaBBvL8ZZxX2ZTZMA4AnSybJbs6zincRW", + "bloomSize": 131114, + "indexHash": "QmZBusUsKJGiANN9GJHiyrNSghKkXHJZLEjnNDCrUsf2cA", + "indexSize": 1931376 + }, + { + "range": "004109240-004109587", + "bloomHash": "QmdFdo6tkbnFjFxGY7w3ffe5U6baVxU3hJYgNgrV58yMMq", + "bloomSize": 131114, + "indexHash": "QmcMow854TKTcxECMkPe3vk4s7c5aNE1aZdi1cXRQCoovB", + "indexSize": 2126492 + }, + { + "range": "004109588-004109983", + "bloomHash": "QmYmQadE9EnuRH4pweA9BQjU44paXxQxib85pfrCSFhHYm", + "bloomSize": 131114, + "indexHash": "QmTq9W11a1eKNr1rEpUPkMvz85TUrbQ1hRNSYcxQu8AMfw", + "indexSize": 2272672 + }, + { + "range": "004109984-004110508", + "bloomHash": "QmbesRPAXr8v3Herd83oNRb2fXNfWoHK7gWumuDVcW4vdE", + "bloomSize": 131114, + "indexHash": "QmXSZh4rKDpUshLMJfHExh6LW75CDzdWazRxWwNsVSf52Q", + "indexSize": 2126056 + }, + { + "range": "004110509-004111017", + "bloomHash": "QmZLXeq4dQ4h1HQpUwACdyLxnvY4sNVz7nbvYo2DqBigZX", + "bloomSize": 131114, + "indexHash": "QmPdTMBW3p3XnjhwjQaQ42DmvACLcnXBo62Bf4Tk2uU85T", + "indexSize": 2292908 + }, + { + "range": "004111018-004111438", + "bloomHash": "QmegZFoFQncWtud5AUWZ4N8zGh9BpJqafM1JwnhjLdM7Uh", + "bloomSize": 131114, + "indexHash": "QmZu35TVTUMJ7SRW7yvvNHTaGsQ3fTh9TVH9VEDfJr6Pwm", + "indexSize": 2189804 + }, + { + "range": "004111439-004111810", + "bloomHash": "QmeMav4mse7cntoCPnMsDpkZum3CLqDDd6yg2zhkofzvYX", + "bloomSize": 131114, + "indexHash": "QmRQFNRzr6nKFC7p5uvRAsaRS2KCbf1N3S1Teq29iLUu93", + "indexSize": 2365952 + }, + { + "range": "004111811-004112176", + "bloomHash": "QmNbxucqJ9381cRALdRnLqq2gwGU7sMvAmYNWW3m3WANKT", + "bloomSize": 131114, + "indexHash": "QmRPRNm7FxZSL92c6kBmmYpP2zYb37sB8RLeZXF2zoeakP", + "indexSize": 2201136 + }, + { + "range": "004112177-004112622", + "bloomHash": "QmdBT9F8m9bcqom1d6eHFnMbVMTCTVNFjMbKtCifNNb4AZ", + "bloomSize": 131114, + "indexHash": "QmRTGF7kfyAGLCfoZBCrjivS5W4CCS7hdGSPtEnXjqnqvs", + "indexSize": 2185472 + }, + { + "range": "004112623-004112992", + "bloomHash": "QmRQAX2mBfEDUvBeBnDoyRvvLfjFw3xcbx8fTw2Snzhu2d", + "bloomSize": 131114, + "indexHash": "QmTrSYwVgMkxwQEDfyfH6sLh211Bw1mEmkRgWjSNAvYeKE", + "indexSize": 2290376 + }, + { + "range": "004112993-004113333", + "bloomHash": "QmRAZWpJeuHtXhkeXMACrkJZQ37MydVs2HKr616b1WEm6M", + "bloomSize": 131114, + "indexHash": "QmXJrMn1pY4y44EyjR8NA2cBzXwxJ3TZaa5LRdxJNM21Ck", + "indexSize": 2327052 + }, + { + "range": "004113334-004113705", + "bloomHash": "QmbQfVmaSGfUbxeJm1HTDm4YNyhS7KxwsXmJVpLyjwvEfe", + "bloomSize": 131114, + "indexHash": "QmNviBR7iFVgHM8htLb4WqqfotwMwgoUX5VN4FPTmkDMCp", + "indexSize": 2332192 + }, + { + "range": "004113706-004114087", + "bloomHash": "QmWxaZxng6JbUKZexTMS5buZkk26iy9ymt9LHXDkmZyoFL", + "bloomSize": 131114, + "indexHash": "QmYArDGGgatVfSVfhhtjBzK3NGWR4uYVbjVQwVBhJ1hZ54", + "indexSize": 2388140 + }, + { + "range": "004114088-004114533", + "bloomHash": "QmV24BEUwiTMvVsuGCsANW6AknYpToeBEzDBNEzk1jWEHy", + "bloomSize": 131114, + "indexHash": "QmcBxoxR6tq9CzzQmpPrF2LCUC9PuVU94yeN9AiVY2BtFH", + "indexSize": 2258080 + }, + { + "range": "004114534-004114968", + "bloomHash": "QmXgyngRgPbeeAKWeDMMVAykKbJcxuiAGogJFahqsuZ9wB", + "bloomSize": 131114, + "indexHash": "QmSvJtpHLkPAdMea41H32fMn2htc3vXUjUesjANYtHQ9mx", + "indexSize": 2291552 + }, + { + "range": "004114969-004115458", + "bloomHash": "QmcbrzNyqECcDfsUfaKpFrP1C553ZESJhvpzfdEPXgoSaL", + "bloomSize": 131114, + "indexHash": "QmcYJdDJAiaA2ZutyJNh5tPnVZxKQGFzii8XfJRXgWxHKs", + "indexSize": 2402448 + }, + { + "range": "004115459-004115941", + "bloomHash": "QmTXxTXPme2QvGLWEC6f6xJcPyFVa2xacwFMfVWJgBX4UT", + "bloomSize": 131114, + "indexHash": "QmdqWP74kF9yyHMGG8duBT9f2Xo3mttAPvZ2kTwzJZnsSk", + "indexSize": 2576980 + }, + { + "range": "004115942-004116445", + "bloomHash": "QmdJ714eTAjcGycryR2iDj3Hexkz7gzZH9UhmBgpX77AJX", + "bloomSize": 131114, + "indexHash": "QmXuyvawrB5So87Wp8ApNtqciSSmV53sNj7cQw929eiNb2", + "indexSize": 2731900 + }, + { + "range": "004116446-004116931", + "bloomHash": "QmewFENKSPceoukqp6SpM4eczQkqqfqXmwFHGt2jmybhae", + "bloomSize": 131114, + "indexHash": "QmbrWALmnssiRZyhp6Fvu8zSVzo7ynwqFX76cgNKy3Q3A2", + "indexSize": 2433916 + }, + { + "range": "004116932-004117389", + "bloomHash": "QmPXjWoMFBmcQup4GWVwY7Gnk3jBJeZUe3pxuQA1xAjUgG", + "bloomSize": 131114, + "indexHash": "QmdRCB9ZPJrvSxAMmKyxkihGQzj7MW3bMcdWf3oiwuMWXx", + "indexSize": 2479428 + }, + { + "range": "004117390-004117822", + "bloomHash": "QmcNfv8ujCAvMHehd6pCxT7Eu8BXmvbAGNXXVv88fddbzR", + "bloomSize": 131114, + "indexHash": "QmadcU1dhsoeP4nnyS24WTasfUgwj4JtKVjuMur3Ad5JvU", + "indexSize": 2731908 + }, + { + "range": "004117823-004118210", + "bloomHash": "QmecDTAQVPnrYf6r2E1xWtotgw7xx15QGDj3BmBSTZX1oB", + "bloomSize": 131114, + "indexHash": "Qmdjzf2iP8nJqqbdAHkXCQsVcgUmzCCcyPwrgJQYNuBEDY", + "indexSize": 2335968 + }, + { + "range": "004118211-004118574", + "bloomHash": "QmYh8rfbrLZ4boepQuCB17eSest7VCnNFd67fuZ1NBQ5AT", + "bloomSize": 131114, + "indexHash": "QmR1ZLDLi6rCCdmTFxnSR2GT4d2wP4aSSZAfcdsgyuBtFU", + "indexSize": 2188148 + }, + { + "range": "004118575-004118932", + "bloomHash": "QmULLto7y4BSrgASji73UjUXFPYjc8kgibSNXJXGiJFiov", + "bloomSize": 131114, + "indexHash": "QmQkdAqhF9oEbAJPEsSaQcvagLUKkkUcfDnHimNyHAtkKT", + "indexSize": 2204196 + }, + { + "range": "004118933-004119257", + "bloomHash": "QmZhX3KdT9aitwboenAvws8G8XVgGjAArozsxzCDXc9xPD", + "bloomSize": 131114, + "indexHash": "QmV3wVQBhMMryzNj2EAMGEXpqpLD8qe5hFThdmWrM65786", + "indexSize": 2261660 + }, + { + "range": "004119258-004119585", + "bloomHash": "QmVrjDuyPjY3qob7S5TTTorYVNpECt3oXSaYBeG2VUyPkd", + "bloomSize": 131114, + "indexHash": "QmNzxb2BqdPt1YTaLpFZM8UrQMUKA4yziGAeC9XHCtAu5L", + "indexSize": 2446860 + }, + { + "range": "004119586-004119945", + "bloomHash": "QmWj2kkZFVn2DCX3xAgCCMSHSUd1f49cwSXTwLuddc9GTp", + "bloomSize": 131114, + "indexHash": "QmXKukxU2xVK95ShEHf8EysKw5zU4iH5ktBy2XfkMkVyhG", + "indexSize": 2480660 + }, + { + "range": "004119946-004120313", + "bloomHash": "QmaHj4VzB69EEsa9FK1H87zXtrMQ5tazdBJSHbjSfrxS8j", + "bloomSize": 131114, + "indexHash": "QmRsrxAqtu2XB9jqdKE6PFLXsh6mZmv1iQce7WUas1CFZZ", + "indexSize": 2601344 + }, + { + "range": "004120314-004120707", + "bloomHash": "QmNi7pQzQ27hnBik9aPBLj7K7cxwxKZENwEficJoxzRhef", + "bloomSize": 131114, + "indexHash": "QmWgEYrpCoMK3j7MuM3AzEptFKrjRJzpTLLwTCvAfibsJY", + "indexSize": 2632192 + }, + { + "range": "004120708-004121148", + "bloomHash": "QmVy5wzzLTsWNdEGkijhncJF4xTFL1U4xSt9KaNbGa5h5j", + "bloomSize": 131114, + "indexHash": "QmXa8481GPdVyUXi1pcMQUMAe1PemGSZZhpVAQVcAUd7cT", + "indexSize": 2539160 + }, + { + "range": "004121149-004121566", + "bloomHash": "QmNt6UFnq1zGs254ectkx8TnvyCtDDX5HvEVQED6fXLUnY", + "bloomSize": 131114, + "indexHash": "Qmc8U2TKSWXcgX8ENXj9fxjaKQWqUxEgdUM42sNeFHybkj", + "indexSize": 2682976 + }, + { + "range": "004121567-004122026", + "bloomHash": "QmQnfR9wPAQsVdjZPyWxpHAzzuqJjjs4r7TJiaZatsLz2S", + "bloomSize": 131114, + "indexHash": "QmXza2T9pFEy6zjXBkTu32GqYGx1bD6uNRe1MeQZUMWs3t", + "indexSize": 2695356 + }, + { + "range": "004122027-004122454", + "bloomHash": "QmdaQkpgwPAccXiwLnYZJ5ZKuwnSgZ2sz42CqRS1tEUfsN", + "bloomSize": 131114, + "indexHash": "QmaHrJrdcehTU7g5p3RTmtceymcD3NoD6twgKYZhB4xHD1", + "indexSize": 2550060 + }, + { + "range": "004122455-004122843", + "bloomHash": "QmXLequD2TmJaTQTsKyGK9MdrUKaBcWtD5QhZ8z7RbeAbg", + "bloomSize": 131114, + "indexHash": "QmNU4zr9P2Yh1KD3hvfKLKB59Y6CgQqnEezU2xVYAeMjBx", + "indexSize": 2445960 + }, + { + "range": "004122844-004123191", + "bloomHash": "QmTx1pxk3cmJUKFyoHggURzQaQbGRGXjdewAtgU4dkRBua", + "bloomSize": 131114, + "indexHash": "Qmbv9NtKHXxs7jABiPoibiQxmJvKGnXDqFZwz5aYdqMgGg", + "indexSize": 2385136 + }, + { + "range": "004123192-004123559", + "bloomHash": "QmXpU2ChNYQXuwZt9HVpgwbcBxmND23oHaMwa5dqH5Cxfo", + "bloomSize": 131114, + "indexHash": "QmPrpANNYsLiqdLmXWeZxuCtTSyVtxyyf4Kf2R2CSHSijk", + "indexSize": 2433364 + }, + { + "range": "004123560-004123932", + "bloomHash": "QmaaBhksUSPaMeRPXiNRqUGyAr28UqsuCEwWsKa2Jhtjgq", + "bloomSize": 131114, + "indexHash": "QmNg5pdzJoBXUgK2nxLfLyofuQ2LwoZ1AQwxp9x1NepN3x", + "indexSize": 2445952 + }, + { + "range": "004123933-004124277", + "bloomHash": "QmTe1PkyGXQiJPUPSxMjYkuX3LBBvU4D3vQ4p3uRrFMrqQ", + "bloomSize": 131114, + "indexHash": "QmUVCaxZz97qduvU1G2Y7ykJHMMuY9WYZQdNaQgzeCDxYZ", + "indexSize": 2468872 + }, + { + "range": "004124278-004124623", + "bloomHash": "QmW18cY3tJcLTwWfcmjcrbHRkCdVypnEJcADm9i9rBz3Uo", + "bloomSize": 131114, + "indexHash": "QmZ6oPmAWSHDAj8GwJVZE5nmRvZGMBxBF7fwmU5gyzBZB2", + "indexSize": 2604348 + }, + { + "range": "004124624-004124988", + "bloomHash": "QmfXJmpeuxC76C1zbcZ5qM8qE6GhrJXnvEABfV5T4FYmfT", + "bloomSize": 131114, + "indexHash": "QmWKg7Fg7EJUTFZGgCu5q8Quf55mzMxZV1mkRKFNh3cmhZ", + "indexSize": 2515664 + }, + { + "range": "004124989-004125322", + "bloomHash": "QmZjroDBmzxkYun858EZpjsQWP5dcGRq5kkzmruq7XnV8k", + "bloomSize": 131114, + "indexHash": "QmfJTB45c4Rmmiw4u64QMxJwQMh46cbfQcAoamG7sKLHow", + "indexSize": 2474616 + }, + { + "range": "004125323-004125650", + "bloomHash": "QmdeZrxVotrx5r5gjAbcU1zrdVoPRsfCNiKfD1Dmd7gAQS", + "bloomSize": 131114, + "indexHash": "QmfCWNiqYdiRGsLYWPrRsdpLRB3ANaX9duE7piqSFTXgCH", + "indexSize": 2500192 + }, + { + "range": "004125651-004125972", + "bloomHash": "QmZ1vBBmm8dC1gMHB2CrLYnAVm7cjaRcFU5cgRMHPVCgbH", + "bloomSize": 131114, + "indexHash": "QmSbB8CR3m8Cg1sEDxtVYuXDzj4atwsGrzxAturdGHDNE5", + "indexSize": 2551532 + }, + { + "range": "004125973-004126313", + "bloomHash": "QmZvszA6bh3t7NHurDwUf3RckRrCHkzeQMRVCWpoYgGvwq", + "bloomSize": 131114, + "indexHash": "QmSMdp73PTArV3KPnSGdjat98jwv4VeS2mkG2kPLSQcgCD", + "indexSize": 2498344 + }, + { + "range": "004126314-004126753", + "bloomHash": "QmdNNoghxusus8YXmsJv1WPwevmYfLVthytfbX2Cy3JSPN", + "bloomSize": 131114, + "indexHash": "QmUGDrjABGMdhPGYJN5LWwKcxQWDYH9oSsJDdYXAZYyqJX", + "indexSize": 2498960 + }, + { + "range": "004126754-004127139", + "bloomHash": "QmSYFhGM5hKRbFwuAGS5ED9x7aEoNwqX2RD61o3AgC5XBH", + "bloomSize": 131114, + "indexHash": "QmNbEnpJN3WHnZbfo2BquRPQWPoB516HxVW287SCqBEP6r", + "indexSize": 2634568 + }, + { + "range": "004127140-004127516", + "bloomHash": "Qmdx1CMNXsoBi4etvR5gA5CPnNZJdTy9uKXNB4kT2uiYCj", + "bloomSize": 131114, + "indexHash": "Qmf4Re1AjeA46wN3xf4U318MbSyCq7dvZhPUYiJwLT1KRY", + "indexSize": 2523144 + }, + { + "range": "004127517-004128041", + "bloomHash": "QmUnfAHdZpWe9FMa7iteU3bqVDMMe71cV4cMMYbPMryPD3", + "bloomSize": 131114, + "indexHash": "QmZeFxah87eqEp4ui7DGBSXwCHbLYs5CF6gRerDvtXcZXF", + "indexSize": 2751712 + }, + { + "range": "004128042-004128533", + "bloomHash": "QmR5CuaD6hB6iBRxTw1273u62SxogeiE82qFTXHiW9MFuW", + "bloomSize": 131114, + "indexHash": "QmSLPtJWijVEg6HfKQtNGnfhBZrNQPH52RcZSTjDE5uK7d", + "indexSize": 2601936 + }, + { + "range": "004128534-004128929", + "bloomHash": "QmRq1LLZ8ANTgeTZSysNCKkAU9Dbkpm2V3qgJ9LuiHbqUi", + "bloomSize": 131114, + "indexHash": "QmYrizbsyPbiJCbpEXBAdDEPv7QFxqGPGJX4NXg9M8G2Wb", + "indexSize": 2491452 + }, + { + "range": "004128930-004129307", + "bloomHash": "QmXJVdSJB1zmBfTg995yDrAbnYRnd9T5fxvSemTFjanhVB", + "bloomSize": 131114, + "indexHash": "QmdNtJfB7Uc7H1w9HsP1NNg3eLzZw1vtFaYuSc6iRCGdXD", + "indexSize": 2480052 + }, + { + "range": "004129308-004129753", + "bloomHash": "QmYqQTN2SPTWVnMyb3SzhLTPBJU5f2srLGKG9p5d1v8ckL", + "bloomSize": 131114, + "indexHash": "QmVmMdvmaGyYbpAufGcXDwDFxjKu2g6usbSL6nrxvxB4By", + "indexSize": 2338532 + }, + { + "range": "004129754-004130144", + "bloomHash": "QmbAEy3dux1tAfR9dRFHTQFtiJSMU4SWfSBhXy622XDkxa", + "bloomSize": 131114, + "indexHash": "QmVcLUVCGg59UR4kMtrbJLV9XzRwMxwSUcfb5BNBWqGL1x", + "indexSize": 2348900 + }, + { + "range": "004130145-004130504", + "bloomHash": "QmUnjMGwE1KqGbRj4EoWA9kq6x1jmj1vrp7PtRbhQCcwaH", + "bloomSize": 131114, + "indexHash": "QmRrL3MmGJ3vDMnMG3Jdrvqt5wezJ1TSqUFsWkqiajCwtG", + "indexSize": 2341436 + }, + { + "range": "004130505-004130861", + "bloomHash": "QmQkHa98FYoEEE68WA3uwjrs7J315RAGQ7rCU8gQvzckjB", + "bloomSize": 131114, + "indexHash": "QmT6zEx2J73nqYfxEg5zfNXmHmpuaPa5v1EWDX2VherxDA", + "indexSize": 2456972 + }, + { + "range": "004130862-004131258", + "bloomHash": "QmcGmLTYov7rY9GWS2iS7EqCYzryVSqL4mWqJnvNNM532p", + "bloomSize": 131114, + "indexHash": "QmRvVYk5QwihD2j546jPiqaMbqbQPw7yM8eMCTyJ2Msrb5", + "indexSize": 2542096 + }, + { + "range": "004131259-004131599", + "bloomHash": "QmdbiyBJxty7fZdqDjdJPWv4ZXfewx4RBmbAWrUMBi8KL5", + "bloomSize": 131114, + "indexHash": "QmV1JqktDCWR8vc5QrYtrSMdv14RaQZBeqXufhctxgkJEf", + "indexSize": 2539648 + }, + { + "range": "004131600-004131927", + "bloomHash": "QmQiPzo6J3QE3BGZ1PxTgZjcdmZGYokv5Q4qfpzjZ9TXnQ", + "bloomSize": 131114, + "indexHash": "QmeZJvyZPFwxDaxeWiFSYA3vXkZFzZyePZzGph9R7e3WDJ", + "indexSize": 2406792 + }, + { + "range": "004131928-004132233", + "bloomHash": "QmVyw8k2S5vy82JdHB8nFj6WWuuZRrWxyo7FNwNwuzaPbu", + "bloomSize": 131114, + "indexHash": "QmcmntkVm5NR3kJe9soEtqV1NL4ycskaPr1eKQLQr14nNk", + "indexSize": 2439936 + }, + { + "range": "004132234-004132516", + "bloomHash": "QmdKZPpMqzVPsbi7NBu4agmoQdxe6hyRg1QsbuUZBLe7c1", + "bloomSize": 131114, + "indexHash": "QmcqdiftHiPsLeLGZ5DvsjknMFpKPM271U5RK97AuChSzv", + "indexSize": 2339300 + }, + { + "range": "004132517-004132903", + "bloomHash": "QmZihF6Lqr6VZzcPwpGH8JYppPxsqH6wZ5ZpqP4LdCDHEg", + "bloomSize": 131114, + "indexHash": "QmRR73gHRqy2fqTGXRjUBWMwpL49hh61F2gZECfS9sST5K", + "indexSize": 2393884 + }, + { + "range": "004132904-004133218", + "bloomHash": "QmacfL1mRFggenKE9PeWnLkrrt8Gy9z6BGfYpzxRD4roJF", + "bloomSize": 131114, + "indexHash": "QmcSHTHVg5XnZG126B2sKePyKwkS3qD8ZkAWANf8Vp46gi", + "indexSize": 2340612 + }, + { + "range": "004133219-004133485", + "bloomHash": "QmaPLniqSBGs4YhpRWp1ddLZPx5HQNHFZGZn3ETTzPQCoG", + "bloomSize": 131114, + "indexHash": "QmTnhij9BryXACfzgvACkue5vwQgrXmYacN7RJVAAXMGYc", + "indexSize": 2173808 + }, + { + "range": "004133486-004133789", + "bloomHash": "QmYcCSWKzSsXPyJgkCkua65nenwnRxF4jCfnyX6Nw7iRif", + "bloomSize": 131114, + "indexHash": "QmT9epK1jekVpwLbbv2xL9boCDmrZtA4onKwbzePxiQYKu", + "indexSize": 2177784 + }, + { + "range": "004133790-004134007", + "bloomHash": "QmNwdJPtAxozEr76mFs8tLPv1117ZZpiyKNcRtwvRw6teX", + "bloomSize": 131114, + "indexHash": "Qme8NJSuz2hW2n6BgdrMFv7xqEcwwDtNaQDegyAQqj9eGC", + "indexSize": 1984732 + }, + { + "range": "004134008-004134211", + "bloomHash": "QmXqskQbJRzM4thWRkJ6Bu6tyEeJAthB4X5RfiaZv2zHvG", + "bloomSize": 131114, + "indexHash": "QmQqb1ifZBdcLH1EJNMeoR5Do2qHZUkjU1v5aLNvaaDR3X", + "indexSize": 1942396 + }, + { + "range": "004134212-004134497", + "bloomHash": "QmU7tZHGgi4mUY8Xj7ACY54bjPbuxs4DFMdmjEZavAoo1f", + "bloomSize": 131114, + "indexHash": "QmRD1CfxGh52SjZ8bNMiUVZhgQuYuHD56ZHQsQCXMobwQv", + "indexSize": 2012348 + }, + { + "range": "004134498-004134673", + "bloomHash": "QmbcmbFYsd5VNWKdvti8tjY93BvJZyhKdXA7QxDJXppahY", + "bloomSize": 131114, + "indexHash": "Qmby5QPLYpNawUefatFSDnngCHVQqBguRZjr4n3SSDpyri", + "indexSize": 1916916 + }, + { + "range": "004134674-004134965", + "bloomHash": "QmQVizUzTyPNurvbD8gULE8fqCSE8uvSfgeyzHvr3Qsj8C", + "bloomSize": 131114, + "indexHash": "QmVc5Tw3hRtQenRn66fdzgK4azNBYnnydjJQ3EnCQMiaL9", + "indexSize": 2159584 + }, + { + "range": "004134966-004135316", + "bloomHash": "QmYEDveP8kgEUpYyTsdeHGDMngDnium86DaRswKRh7hcF4", + "bloomSize": 131114, + "indexHash": "QmPZYEpPWFiLwQZ9qCUn2xkKtUzaAmyB89wm9mM9GnoR23", + "indexSize": 2317488 + }, + { + "range": "004135317-004135586", + "bloomHash": "QmNMWkFDu8JvzMpL94PkvFr3KhnathL6yaxksDKWs4YwYb", + "bloomSize": 131114, + "indexHash": "QmahMDpMdke1JrLtHJScY1BEYDi96zGczFheabwJykZqDG", + "indexSize": 2625220 + }, + { + "range": "004135587-004135758", + "bloomHash": "QmPkVVF129rnzPDwvzUcdg1Byn7BZQMq8MWzLGZejZy1W4", + "bloomSize": 131114, + "indexHash": "QmUsqxjorVSSRbmvUWmTAzghLYWdwDY3Z4JrQChN5bJQsX", + "indexSize": 2196624 + }, + { + "range": "004135759-004136036", + "bloomHash": "QmV1wM2iUPZdpUikdQYQU8VNApykXpFMtTeZaRJT9Az7eg", + "bloomSize": 262190, + "indexHash": "QmR8WiFhoHiiXHZNLKkKqbbqedGR7ycmUJgUCmorE9W1pv", + "indexSize": 3197976 + }, + { + "range": "004136037-004136294", + "bloomHash": "QmVcoVGqMLQJSe3sr3XxuGrhPWRF92xE6q9RC5fKCtFQfh", + "bloomSize": 262190, + "indexHash": "QmTPkNaxqDx2iPj2ko6JtFWSmpTKgHe9z6JS4tpXCChhwt", + "indexSize": 3528204 + }, + { + "range": "004136295-004136618", + "bloomHash": "QmNoPXhbEDodCJNDvNADcW7gQVez2d8yng1cUmgnTGz22X", + "bloomSize": 131114, + "indexHash": "QmaRqJkBDXwCoDJw7q5MifXm427VreDvY2xKtvRhMuoChd", + "indexSize": 2566156 + }, + { + "range": "004136619-004136919", + "bloomHash": "QmPYVtNyQXZnbynWXyJHYePvpDJnjLKhxbkjJ1YyByXw6q", + "bloomSize": 262190, + "indexHash": "QmStxq2ATwX987rchphMz4ZWxAvJqrcEi4CDfgrxZiBN6o", + "indexSize": 3027088 + }, + { + "range": "004136920-004137250", + "bloomHash": "QmdS79bb4CVuyqAEqa356rYVip4tSjXRkzfPLKswbNUg8R", + "bloomSize": 131114, + "indexHash": "QmeuWGDzKj47LkrH1aWBZaCbm8bpDLq7MkPCNQaCyK6VyR", + "indexSize": 2494900 + }, + { + "range": "004137251-004137561", + "bloomHash": "QmRgARK7QNKoYKhqrUFE6eMsTt9ChPXMx6CJPAzLD4aQD6", + "bloomSize": 131114, + "indexHash": "QmRv2CGYqTpNZvJVFW8EJDxNNTqSJGEPq2swXeyLTLSyjt", + "indexSize": 2277792 + }, + { + "range": "004137562-004137911", + "bloomHash": "QmUoR6RupYqHaHz2aGi2uxeo8nerfdKsgjtzc3YrSWPrJy", + "bloomSize": 131114, + "indexHash": "QmT1qptp8NaEwXBdT8Rnpkb9iuxDmBhtzbf88Evqjbqbjj", + "indexSize": 2268924 + }, + { + "range": "004137912-004138284", + "bloomHash": "QmUT4mV43ndC2UJd7DxBJUnxcHfmFGSpwAuR6bWwUSB4Q8", + "bloomSize": 131114, + "indexHash": "QmQxbrLSMs2y2hWUqEoTCaqtubapRzJ6Etjs4jtuZHYQ4b", + "indexSize": 2360108 + }, + { + "range": "004138285-004138663", + "bloomHash": "QmdvLqnFnK2wWZCQSX7LWELJBPsbyJ7Qqg3s24WaHTVgAU", + "bloomSize": 131114, + "indexHash": "QmTyREnr2SdRL4rCj1yiNm9eNPmPA2h3oWXmDJMUTEBsRN", + "indexSize": 2346832 + }, + { + "range": "004138664-004138986", + "bloomHash": "QmUrUKMixE5hpdVyvyGMMJ7GXFiujnZyFRnaHd3rs4sPD3", + "bloomSize": 262190, + "indexHash": "QmUxgQFBC8HKavyWW5sWVzwfHeQqxskyHy2cpDB6WKvicD", + "indexSize": 3221244 + }, + { + "range": "004138987-004139252", + "bloomHash": "QmRAp2iRnYXEq194ybvED2P51RAhBXncRkjTX3mPRG4zEN", + "bloomSize": 262190, + "indexHash": "Qmc5iYg7JcDdd7Jkf3qA7jb3hmiqx32doC3imWPv2rA3oD", + "indexSize": 3439712 + }, + { + "range": "004139253-004139606", + "bloomHash": "Qmcq5NensrtcNzeLgopZbi9VJJuJpUDZYoB5LowqFWKwnT", + "bloomSize": 131114, + "indexHash": "Qmbh43arCQvnneMX6SENLXeCkkD2QPgNerR9s2rb1Af3A5", + "indexSize": 2186780 + }, + { + "range": "004139607-004139889", + "bloomHash": "QmZChao1AhvwNsFNSPijxfwJJ1UsssxrK8RP9TXAXXoDbu", + "bloomSize": 131114, + "indexHash": "QmZasU4Vzcz66HZfWHP1P4WLCbTrXsuyPwugMj6gDGwvba", + "indexSize": 2033640 + }, + { + "range": "004139890-004140156", + "bloomHash": "QmY2uZaGn9Q2keJ1n92ZgQYhQq3ryvQgiPHZ9VjLUzE4Kf", + "bloomSize": 131114, + "indexHash": "QmZQG5TSat6xwQmXj4jiw41PGTby2KTu3qdsCdh9LK8uHy", + "indexSize": 1939872 + }, + { + "range": "004140157-004140416", + "bloomHash": "QmNcDofxAdEQRbnP1bSJ8DRapu8cjxKMCWBEiTb4aoCG7v", + "bloomSize": 131114, + "indexHash": "QmcUhdUXAUvEnJv6UKsYiXtLUN8gXfCZDW3kncg4XnXWkV", + "indexSize": 2033588 + }, + { + "range": "004140417-004140800", + "bloomHash": "QmPKSLv7EEAb8dUxBaTXXitiHhp9XG2n6xpK5hDBCJ4MAQ", + "bloomSize": 131114, + "indexHash": "QmVcBT8foAyv1yZBsHM3m6519MhGXS7aasVqi8CaUMQ5kG", + "indexSize": 2058660 + }, + { + "range": "004140801-004141240", + "bloomHash": "Qme7TsJtQhVWG55k2D26NR4JaFgYuB8mQ7wg8UX2jzGjMn", + "bloomSize": 131114, + "indexHash": "QmcbpD4RmZGUZHr5xpM7qK7Qg8APFocTKXuUahgJ7YBkWi", + "indexSize": 2069072 + }, + { + "range": "004141241-004141606", + "bloomHash": "QmbAjVmn1NvaHedLioYYCytaJdmUv6DE4MPqCRvaEmc4ZH", + "bloomSize": 131114, + "indexHash": "QmPyU8Tyd2pkugFmch29CN4jzvEhDNhYJJdk9tpkT7JvLf", + "indexSize": 2237060 + }, + { + "range": "004141607-004141907", + "bloomHash": "QmdsSBSEAq6d5GUXgnAtcTDjyguQoCPRT9tCZJnnTqhwrq", + "bloomSize": 131114, + "indexHash": "QmXxtFMiyxAy5eYghcc5Sqsz3HjtFoaLMNRBYkyfmDtAC2", + "indexSize": 2303860 + }, + { + "range": "004141908-004142218", + "bloomHash": "QmYCkuy6kKuZfpy4yK5BPb1NTnvuAQcm9G7pgbT9v2mjkz", + "bloomSize": 131114, + "indexHash": "QmNzoTZoa4HvbyMhzp9Fdxk8xnVavQVSkMfcqdePvcQ579", + "indexSize": 2244104 + }, + { + "range": "004142219-004142492", + "bloomHash": "QmZLQNeZo9kiZ1uAvqji936djuK4gnXMib2dYKwkGK7ZDm", + "bloomSize": 131114, + "indexHash": "QmSV9gxSD4L8Qk2A5c9ZLiPaXLXdk3BzBUQZCtkXXYnweB", + "indexSize": 2250176 + }, + { + "range": "004142493-004142800", + "bloomHash": "QmbJLMAX6WLY52QQj3sdKdFcNydcugEL7gfY3GGkeQ7dLf", + "bloomSize": 131114, + "indexHash": "QmdiHeFi7yR5fh3D94V9aTkaQwqak4RJy8Vxt3qvxq3TX1", + "indexSize": 2272008 + }, + { + "range": "004142801-004143060", + "bloomHash": "QmW7iem8qb8FDHzAWdRiDo4x236Ja8e6jSq7GRLSJ1WHCd", + "bloomSize": 131114, + "indexHash": "QmUCKBtrrwFTB8tgyAzGyLKWPBvVwJwKnNSV5TC1vkvbP9", + "indexSize": 2419780 + }, + { + "range": "004143061-004143330", + "bloomHash": "QmR4JHrGa5TpJDQzp5yt1S6T3gX4aqqK8nXzrR9CUpQ83a", + "bloomSize": 131114, + "indexHash": "QmUKrE4yCFisMND7WTNT6mdit6qndBjt3iUbZN3SjBM2Fs", + "indexSize": 2522540 + }, + { + "range": "004143331-004143579", + "bloomHash": "QmPZM1qFpqku5EZzxqjSNir8K4QNNfuL6jWzM1yAaHnyrr", + "bloomSize": 131114, + "indexHash": "QmRTKRCeFAfv592ebjzRHMpLoqb8Kz2qYAcFXk6UapDBfv", + "indexSize": 2514152 + }, + { + "range": "004143580-004143862", + "bloomHash": "QmQtvRjh2u56xuhpbCoxbeV1zdoCMUrv3GV6ASeDzukQoY", + "bloomSize": 131114, + "indexHash": "QmZ3Skqt1H8HJkMNufkHVprgXE2T2rBub98tpgbE5yFoDv", + "indexSize": 2434884 + }, + { + "range": "004143863-004144181", + "bloomHash": "QmaJ15qJhhcVpdNqbDRPoTrEW4exyQoZ2M8HHm5cpATn7x", + "bloomSize": 131114, + "indexHash": "Qmdiki5xMTE7dKdJ74JD78pMWGFEDxVyRmjYR1R8cr7a8E", + "indexSize": 2423312 + }, + { + "range": "004144182-004144474", + "bloomHash": "QmRqo4NaXyLwMaUKpkpNVRpjXeaXhnSyYv9AmKXKHH7mSk", + "bloomSize": 131114, + "indexHash": "QmcS66wuJeoKvtbv6UmPo9FZwGDNDyREfiZh3gdbi8j3cq", + "indexSize": 2377084 + }, + { + "range": "004144475-004144801", + "bloomHash": "QmS1kRkhzicJNHGD8hXRhjF8LUdzkFFLECEQHhzAF8FUK7", + "bloomSize": 131114, + "indexHash": "QmZ9KCtfRdo56tpCesyK2o5xXc47Enc7KZYfhtqHvg4aJm", + "indexSize": 2470056 + }, + { + "range": "004144802-004145123", + "bloomHash": "QmQgKz23A3MrxxhY5oYXuq3oBU7rS6vb3Zx3vysdWJBXVB", + "bloomSize": 131114, + "indexHash": "QmWNf6eQFGaes3ejarEsJa2kGGNgCaCZ8gKbLEyaEqKPx9", + "indexSize": 2392372 + }, + { + "range": "004145124-004145426", + "bloomHash": "Qmdw6c96jXwx8f52B3eAqR6CxEoJmSy4SbCucAgRsj15Rt", + "bloomSize": 131114, + "indexHash": "QmQFKawYN8PdTzFEGBTncz1auRQdranhDZbP9mmbjSQLdh", + "indexSize": 2309392 + }, + { + "range": "004145427-004145801", + "bloomHash": "QmXK7DnMageAU2CyAtBk7pGBi1AXWjk91KukwPGZAW2cyM", + "bloomSize": 131114, + "indexHash": "QmP5QQd4HHuS96JBeCXZDGDhpMZZ5pBCZhbS1XC3hvn97X", + "indexSize": 2412164 + }, + { + "range": "004145802-004146068", + "bloomHash": "QmQh4JWveA4zpyhc16rrHCoPvTSohGrmfhZBvf6JAZ9YPa", + "bloomSize": 131114, + "indexHash": "QmZP8ah7gwc1cNjSKvRWPFmiHw9JvSGCC7sizkQP44yFfz", + "indexSize": 2486860 + }, + { + "range": "004146069-004146456", + "bloomHash": "QmZBAKDeYZzL9VmyphBbk54Vycctrz6jRaemBAqZdtWza7", + "bloomSize": 131114, + "indexHash": "QmaFWKv5aGYk4ngrZRPBtSzG5AtUioT68Hp31sRUpRjjH1", + "indexSize": 2303352 + }, + { + "range": "004146457-004146884", + "bloomHash": "QmUB1AQEgg6AhhY3tkjcGx71qvxZa1bkA4UdNK46Ymhbvt", + "bloomSize": 131114, + "indexHash": "QmRsv1bexEn7wZpWXKBquXEbNCfTGuP69mdNmyryHxw5o3", + "indexSize": 2221136 + }, + { + "range": "004146885-004147336", + "bloomHash": "Qmcb2DnohnY7Jc6x9oEYy3f3FgCeeZGbHEQXRvQgAwxMQC", + "bloomSize": 131114, + "indexHash": "QmeVkGDVKkaQC4AeLScPqUyo995yYMSvz3EacTiwVWUGhj", + "indexSize": 2211604 + }, + { + "range": "004147337-004147799", + "bloomHash": "QmeqMtmns6Tg9DxEeJKKdv3QbUcjjmn7dA5ARPwT1Zj2Rw", + "bloomSize": 131114, + "indexHash": "QmTL7kabMBnq5MwfrBZVd4Y7BGwZiA5JA7z9cVBLdNCpA9", + "indexSize": 2303312 + }, + { + "range": "004147800-004148206", + "bloomHash": "QmRxM2F584974T8WEvkwenjtvyKUu1Az3qwQRMmk6FjqRh", + "bloomSize": 131114, + "indexHash": "QmTtRqtv15yHRMV4jvWaZs1p31JVpx3N388rse3av2ZoN2", + "indexSize": 2238484 + }, + { + "range": "004148207-004148592", + "bloomHash": "QmRurEDLx3XZFMJPSmeCTF3nh5uvjihUPcc5YpAizjVkqg", + "bloomSize": 131114, + "indexHash": "QmdwygPKM27VasfjVGsKuMcixoYauTb5xQN8p44ct8o2Rs", + "indexSize": 2247484 + }, + { + "range": "004148593-004148940", + "bloomHash": "QmNXRCJ2NeGccTb2PFf6nsyZ5W8LpFxpPFp4c9YtkDAr9m", + "bloomSize": 131114, + "indexHash": "QmWeKmCtt1kMU88n1xcWRmVkdNSB2ja2YYGxfsoNuX8mWX", + "indexSize": 2243072 + }, + { + "range": "004148941-004149270", + "bloomHash": "QmWcoWSguvHwDLTg41TxgVqZ53GDcrj3zeVUQUBs83GGwV", + "bloomSize": 131114, + "indexHash": "QmaF3HnEvkmozHo5N6Zgz7FpSA4skmmU1CwU65eGNwH5fG", + "indexSize": 2167612 + }, + { + "range": "004149271-004149591", + "bloomHash": "QmTfUHZqxEeXjihzHC4GDEMhKBCkPJxyB5fkj7qoHX4ugr", + "bloomSize": 131114, + "indexHash": "QmRQF77zDvCS5gVBLiTrfReFhg1UX2dR8kQDaFQh4wZA42", + "indexSize": 2155004 + }, + { + "range": "004149592-004149878", + "bloomHash": "Qmbe2qN9uB2HsWwygokBGEwM6aPeUAFy2b34PHJvNFnNY1", + "bloomSize": 131114, + "indexHash": "QmR31KZGNpFEDiPb5Yc99AeWdVRopgphtR9iXD1v31qmYV", + "indexSize": 2144476 + }, + { + "range": "004149879-004150176", + "bloomHash": "Qmb1syvV9MuRtE3tdbvmdjogAUrjACHg8juoBiGtEoj4zp", + "bloomSize": 131114, + "indexHash": "QmQ3Dpjc2D6P96bYHEL44e3PQv8QjUgAxBdDzQertG1Hp9", + "indexSize": 2242744 + }, + { + "range": "004150177-004150466", + "bloomHash": "QmXNZTE6FETqpP9MuJya8BdnpYi12Xi85tjs4tRjcvYEYh", + "bloomSize": 131114, + "indexHash": "QmfEWHw3YTfkVwmLgLkB85mGH73MFAd8qpB1GgnpBbSkMN", + "indexSize": 2208104 + }, + { + "range": "004150467-004150804", + "bloomHash": "QmT5FR1hZB1vgixJFEaFgerbejpJW9GPo7YKvRuPWDeVYv", + "bloomSize": 131114, + "indexHash": "QmXt9Bbrzmtdvs14vAXygodVF7wrWjU66m2S9oReKASzuS", + "indexSize": 2209068 + }, + { + "range": "004150805-004151246", + "bloomHash": "QmR9xjkbJpnzZTjeUpDcma9d8rqcyA5hzdgh59BE8iPd7R", + "bloomSize": 131114, + "indexHash": "QmRNFz12mrqbC5U76UVX8YfefUPMU4PbTFeTrguAuQDrv5", + "indexSize": 2344460 + }, + { + "range": "004151247-004151758", + "bloomHash": "Qme8ExdQeyGTTCFUeNj7GWACJ9SXsKQ8QxCkKJZXa18Ng2", + "bloomSize": 131114, + "indexHash": "QmbV6USiaAGVEqacoU8MbCsMDanmyvzZzdMNN5A5FZo1XT", + "indexSize": 2495180 + }, + { + "range": "004151759-004152232", + "bloomHash": "QmRhT9e9bA3ntakTnhfbNyFmAX2N9JhYouHXFWZWmJu6yx", + "bloomSize": 131114, + "indexHash": "QmTZBH1XwrbYip6ZXViyGirqMfTrwCr7ZzyjKRKPnzLfmG", + "indexSize": 2834312 + }, + { + "range": "004152233-004152542", + "bloomHash": "QmazwJhU7UUc1vHSwKwNHJGdQunw2DMKgbmqeY2gsFWoxX", + "bloomSize": 131114, + "indexHash": "QmUZi6gkqksB62vfKnho5ksKbZKAQeS6rugwnwJLY4Bqvf", + "indexSize": 2388884 + }, + { + "range": "004152543-004152848", + "bloomHash": "QmUZEgnAnDHFSN7gxhAdghLcLx9QPJKuqcFcHSdW4ig6F3", + "bloomSize": 131114, + "indexHash": "QmPH1RbYeYKAQYdyUQi1Hdg47PSz3UaLzy4qu1MSqWNsSv", + "indexSize": 2286356 + }, + { + "range": "004152849-004153203", + "bloomHash": "QmX56wziJ6uuWPvn8C9zKsfcYqfEAEV1SCtFE887JwPVqj", + "bloomSize": 131114, + "indexHash": "QmcovpVN4aMG2LH4qxwmnKYmEExvmdwYNVD7XaWSSQ86U2", + "indexSize": 2101536 + }, + { + "range": "004153204-004153599", + "bloomHash": "QmSntU2eyXYBLHVaW6mFHXb5hcQZoMmQ6i9EsPyK7pz2CX", + "bloomSize": 131114, + "indexHash": "QmVRHqkmrybxpostpsN7sXCgX22SWyRZJovPfL8wRhL5AP", + "indexSize": 2055892 + }, + { + "range": "004153600-004153884", + "bloomHash": "QmYAMc1WL94LMYGoxwemy2f7HVMK5u9dALCsv3ws2aUKWt", + "bloomSize": 131114, + "indexHash": "QmdUnssChujV3iYUe7yu4k3wGKbF1sGHDgx6r7TUmtNhkF", + "indexSize": 1970184 + }, + { + "range": "004153885-004154187", + "bloomHash": "QmQKabM73amZ1Rx5RUWbfKDfi1yovEtyHZfXv4iJtFjD1a", + "bloomSize": 131114, + "indexHash": "Qme3fJuUivRtZb1FFBQwako6WEwhzXjM4od3BUJekiZxfD", + "indexSize": 1999812 + }, + { + "range": "004154188-004154676", + "bloomHash": "QmWhuCdDoACZrNUsV62FEECTu9AMqkHrRXjj6t4U3q3Uvw", + "bloomSize": 131114, + "indexHash": "QmRQuRtXF9ac9gRf9W9w3XnNFD9BStvLJDE94gF5nb27Rg", + "indexSize": 2074948 + }, + { + "range": "004154677-004154921", + "bloomHash": "QmaqZyBg3okEJrEW5akTeEeezbnFBjdnRX8zLtYiEVgvDS", + "bloomSize": 131114, + "indexHash": "QmcagoGA8SKwVWFNJStRtko3WGxg9YUQdVo2PqKkSidKkm", + "indexSize": 2249048 + }, + { + "range": "004154922-004155275", + "bloomHash": "Qmd5K77pb9JKjFTPwD3UUgum4NoiFijFphGaDQQfD1BeFv", + "bloomSize": 131114, + "indexHash": "QmSfS2Kh2e9hqg3iAJjx3fXFBz5dywrhbAtFE2EvubebUA", + "indexSize": 2478148 + }, + { + "range": "004155276-004155637", + "bloomHash": "Qma4QvhEjJ5k6daLeAVau9crVAJ4yGBH3qiajRFophRxjk", + "bloomSize": 131114, + "indexHash": "QmfTfpHsuf2hN5FU31Xy4NPxpQX7RWhvQvsoRXUaPSZP5K", + "indexSize": 2250496 + }, + { + "range": "004155638-004156045", + "bloomHash": "QmY1e3sPAW3itfZE5LZB4JQUuUjRez46DoDL6auSdaUeYa", + "bloomSize": 131114, + "indexHash": "QmQnUu1QbsvqdMfN5aobGKH5iz1ALu7eqmuxme9b8DTQAJ", + "indexSize": 2565360 + }, + { + "range": "004156046-004156437", + "bloomHash": "QmZZDb9xDP86RmMFVhNiE3bEUi7UxUybbmoCFRKgD1ixZ7", + "bloomSize": 131114, + "indexHash": "QmdLLe7SG8M2R1bUSutnYJGMo4T7ed5vdPgdYGhbFcMLau", + "indexSize": 2349188 + }, + { + "range": "004156438-004156793", + "bloomHash": "QmNPYMY6f8kQtp5Bq5dCkKrZQf1BGUHwzoEwJyn7zctUp5", + "bloomSize": 131114, + "indexHash": "QmRPUCCn8iW1AxNqjAsMsqcVfoS8yzU68sJcEVsUcTFwiR", + "indexSize": 2509008 + }, + { + "range": "004156794-004157101", + "bloomHash": "QmZgoEMJ4hNBXMB9TgZrgW4BGEGQrtBG4VjrBp7PrYYbcz", + "bloomSize": 131114, + "indexHash": "QmPmbHviQMFzkGYZU11gDCGgRjXxuxpdAJWx16WaZ69yUz", + "indexSize": 2880020 + }, + { + "range": "004157102-004157474", + "bloomHash": "QmYGM8BSvhTKSc8wqNQuD8TnhEN2ortsfPCLDfizaPu48a", + "bloomSize": 131114, + "indexHash": "QmdkqQartNihGY9ZU4kyk47fYRL8xtdA66pYkAm37UWFda", + "indexSize": 2548908 + }, + { + "range": "004157475-004157878", + "bloomHash": "QmUJs1TqCg6RANQDSCGs5DKmfrBCRVqvakuPsRgt7uwTKS", + "bloomSize": 131114, + "indexHash": "QmeUJRQWkn4LegXKxsdwxHC5PUUjwN2wCTibF9Jyp6ZGLd", + "indexSize": 2436276 + }, + { + "range": "004157879-004158166", + "bloomHash": "QmSKwVZUba5rro4FVXeoQkHEqVLUAVViy9puFhG2LzDGZC", + "bloomSize": 131114, + "indexHash": "QmcEM8WqvK9aeTXTeZxVFaHyGnnJLfcdXCKNaWVa5zy7no", + "indexSize": 2781360 + }, + { + "range": "004158167-004158461", + "bloomHash": "QmYNWkMbsr3c3Nj3a7gDPcc7udFkY2z8jtqEpv7V3WxPK2", + "bloomSize": 262190, + "indexHash": "QmcSj1gMkwRykuspRgFy8HJo1rqQjo5GUuFrZA1SZiLkjh", + "indexSize": 3212164 + }, + { + "range": "004158462-004158772", + "bloomHash": "QmWZnmBTZx9uuVwJEaRGw8tWTHcvnGfoWuoeB2XFpuAtJp", + "bloomSize": 131114, + "indexHash": "QmSvU6nkpxikV7czLurM8hYsLddrdWmvvsttedL8kHwJb3", + "indexSize": 2946284 + }, + { + "range": "004158773-004159020", + "bloomHash": "Qmau9WjfwXL6fFbcznhX6smv8kW1Xby2hqhLWHnxhTpXdG", + "bloomSize": 131114, + "indexHash": "QmQPT2kDcsaqjuWrvNJWRkKED8JubEpshwcw6cKwSNTWiQ", + "indexSize": 2615052 + }, + { + "range": "004159021-004159345", + "bloomHash": "QmX83NK8XjmMjy3hxymRFypdZYLmNmQ9rC4cYfEiy6qngt", + "bloomSize": 131114, + "indexHash": "QmQXZeyf817ML53SVyxugtPLJ5tzh4UkB94i2T4yHnmzHc", + "indexSize": 2540208 + }, + { + "range": "004159346-004159696", + "bloomHash": "QmcRDdsXNRoVapHZPn8GDwPyLLRCMNnYcKg4vjpC4cx7zU", + "bloomSize": 131114, + "indexHash": "QmeFPDuVQLdTXE6BwZRLEntMgpWv1Xi5arW2oZcNcW9sXn", + "indexSize": 2106300 + }, + { + "range": "004159697-004160080", + "bloomHash": "QmRCSUj7vcyBcDaYgiuQqFrZVgFBPZALNQTJQgRw7raqf2", + "bloomSize": 131114, + "indexHash": "QmXfrKCzC1b3EESS5kWySsfeuREgsVR7pQoB1ENo8c3xqW", + "indexSize": 2115424 + }, + { + "range": "004160081-004160473", + "bloomHash": "QmfNezLogVQQRXJsByHWYe8bfRx7zGbCpJkwtb5h3UrvyM", + "bloomSize": 131114, + "indexHash": "QmeXn3uK5oVgDUS9vZeSWHRp6PgST4NGUnHs2JtAJW8EQm", + "indexSize": 2090652 + }, + { + "range": "004160474-004160889", + "bloomHash": "Qmaski3RJGHEKmFSiboTeH9hRToPiDxojsB3UZwhowNfUh", + "bloomSize": 131114, + "indexHash": "Qmcbk5n6EavSQFaF7VphLtbzTJT57p1k2siKARGhRA34oF", + "indexSize": 2145556 + }, + { + "range": "004160890-004161259", + "bloomHash": "Qmdrvd5hAeErVzKSLffFks5aVqvwu3p4BygWCb6gBhtzoM", + "bloomSize": 131114, + "indexHash": "QmYRpsqZcg5cgh2dJb5vha8encHHLprKv7mLv8UsaK6AJf", + "indexSize": 2272328 + }, + { + "range": "004161260-004161594", + "bloomHash": "QmcgYoJSiuzcFDaAWUjkjYL8XnskGU77u53qAemS56prkt", + "bloomSize": 131114, + "indexHash": "Qme3pUuuzwKkSq12e5JKqivHJpbgTcNWXu14qGXenAij7R", + "indexSize": 2246820 + }, + { + "range": "004161595-004161967", + "bloomHash": "QmPk3eaJFqwTBtuV298Lo5BgXSv2kgJWaPoybk1fMxvxmQ", + "bloomSize": 131114, + "indexHash": "QmU2Av58nHj36h9UZDquTMsdGk8WfyVXvhF9vMhPBmMFjt", + "indexSize": 2217812 + }, + { + "range": "004161968-004162280", + "bloomHash": "Qmdu2PWwAWRWCS1E3HvvtXm7moWSgEUqHVugy6kuAFSgxK", + "bloomSize": 131114, + "indexHash": "QmYq5aVQp2dcYA9Z2Ju5sXwetMXfLif6iaiX85Kp7qvfNa", + "indexSize": 2051236 + }, + { + "range": "004162281-004162612", + "bloomHash": "QmVk2YeKy7RoPB3yAZRg1XFPQQsrYcfQFdLGjBDXvLqghH", + "bloomSize": 131114, + "indexHash": "QmaJw6yJpVL7EY8chQMoRJ6xPxMwrwtYAD2PdFQWo4amsS", + "indexSize": 2129636 + }, + { + "range": "004162613-004162916", + "bloomHash": "QmSLqS9wosxXqcXbeMSkjgqmZXmsWWBY3Xt7V6UG8DFki8", + "bloomSize": 131114, + "indexHash": "QmYyBm1MwRPqGWA1sEsGG79jyp6GCFqH3NgTWfwNDfF4Xo", + "indexSize": 2242592 + }, + { + "range": "004162917-004163184", + "bloomHash": "Qmb51b571eaAB6xdEye9t7Gf9UKZoqn1o9fyx9rkxURv7C", + "bloomSize": 131114, + "indexHash": "QmSfCb24H32FSSZ1s4E7ik97KbDEP6erb8XhLmc8X7NHU7", + "indexSize": 2462408 + }, + { + "range": "004163185-004163536", + "bloomHash": "Qmbd9utmDVCvxtCrjnr1dk8kBob1aEnL9aV8dEboJz9ufR", + "bloomSize": 131114, + "indexHash": "QmNUWT8u3ebqxMXGYcgMHpJsJdZPyDGwC4yqh87h8kckRH", + "indexSize": 2352376 + }, + { + "range": "004163537-004163914", + "bloomHash": "Qmctty6G1pX6mXAtYBQDek9VZxXJRqqVF7sqXqoAUEU6Xj", + "bloomSize": 131114, + "indexHash": "QmQmncctEiPkmbE9NaXnubFAxEWzNVTrh6YwLFhUkFivHL", + "indexSize": 2248840 + }, + { + "range": "004163915-004164349", + "bloomHash": "QmcXpym8AfdwagJ6gNWeT9r1LQ9dda5V4fxz7bZ6aC53LS", + "bloomSize": 131114, + "indexHash": "QmQGxaLfMVKnaR81yPvawttR5WGk92b9dmFRq1EoU8gEhw", + "indexSize": 2198912 + }, + { + "range": "004164350-004164715", + "bloomHash": "QmWjbCpxYHCSBBeLqbXPkpRYPEcHPQxystby9As9ChDQ1L", + "bloomSize": 131114, + "indexHash": "QmUjYKQrjSK5uJ8cpcV8DSrBb6NTHFCxaTqZtChwnDUZDy", + "indexSize": 2114384 + }, + { + "range": "004164716-004165074", + "bloomHash": "QmcXWsnhh4TFCArCwWkGDDSwR5RH3YgPZNdr9FV9SLyUYq", + "bloomSize": 131114, + "indexHash": "QmRTCA8bVXwdW8rcyy62GfXeoWWfpmq3xG9ZjMjqzeboXk", + "indexSize": 2924704 + }, + { + "range": "004165075-004165428", + "bloomHash": "QmSf22vxr545DbuXBu1w8FH7jFgyVxfvgtj3FKdotiHxtp", + "bloomSize": 131114, + "indexHash": "QmbASsN1o9PURNm9kvkJRtmhMDVWia7jfxRZqQqfmhMVYw", + "indexSize": 2955052 + }, + { + "range": "004165429-004165781", + "bloomHash": "QmQaufwfQZ4wSKfMq3a28gvu5VaHFnakyEk2BBzN9ybJ1a", + "bloomSize": 131114, + "indexHash": "QmZ6bYv97r8AY1NYukcTkDpSnYxM46B9k9yG7UrUUPt6db", + "indexSize": 2475216 + }, + { + "range": "004165782-004166197", + "bloomHash": "QmbDMsVhdqC3e2ocKCTnfdqH4zxP9pc1ADbCaPvF2dN46t", + "bloomSize": 131114, + "indexHash": "QmWJzYTQTYPVFikVjgvQQQ6jtnopDNNex45hHPTKkHaApD", + "indexSize": 2996840 + }, + { + "range": "004166198-004166629", + "bloomHash": "QmeVdeYmBQEDJCwCsBxtXVWQ9BdKdndosYFDWj8ZkCyuee", + "bloomSize": 131114, + "indexHash": "QmbH6muDL41Dju6FmgX9HYpCBonJsdfp6HZXfHCsUAMix7", + "indexSize": 2541452 + }, + { + "range": "004166630-004167120", + "bloomHash": "QmULFTsepT1URU4EavjYtqj3dA2TV3ZujgvXDV1Wg6D6VM", + "bloomSize": 131114, + "indexHash": "QmPfoXRmzaRAm5RMLt79pbrEtfbxL9Xbb77nZmipRPZzEV", + "indexSize": 2364716 + }, + { + "range": "004167121-004167592", + "bloomHash": "QmPVdgPoQhMf6eaoX2VHYGiVM8hz98atN2xSM5CfYNUHkg", + "bloomSize": 131114, + "indexHash": "QmZC95EvNyUcw1G4qobNLqeWy1tG5WCKCV1no1sWhmzcsJ", + "indexSize": 2293200 + }, + { + "range": "004167593-004168122", + "bloomHash": "QmdSFtRzdzpc34xEhJjYTPGvByp8Eo2kzqnwCdSL2a5LAe", + "bloomSize": 131114, + "indexHash": "QmcRq13bEkKVKmRPei37CNioumvpsE9APsfzoxcdMKeJP9", + "indexSize": 2113636 + }, + { + "range": "004168123-004168600", + "bloomHash": "QmbVFG4QhsfvvbHcrVPVNhz2xC3a9HKWx68P7GZNDVHdWw", + "bloomSize": 131114, + "indexHash": "QmahsQjtEYQQAH1azNgpY2ArTWahAXcQNS2HV6jyVKKAE8", + "indexSize": 2098824 + }, + { + "range": "004168601-004168993", + "bloomHash": "QmNwknmdvzcbu1NumCxK8Pp2THFawUig24TDhRyqpBxaP1", + "bloomSize": 131114, + "indexHash": "QmPdEJZL5PVuK58dgnkx6RdzidWEYK1jfmEvvDsKT8SbQL", + "indexSize": 2606412 + }, + { + "range": "004168994-004169362", + "bloomHash": "QmSD7Y6cRFVNvidYmbEo5eNk1np45XgoU4TRxwBfigPvej", + "bloomSize": 131114, + "indexHash": "QmSvDTeCtRjEodBrhDHpVZfvHKdo1q28Xvvwjv4XtupRyb", + "indexSize": 2710124 + }, + { + "range": "004169363-004169704", + "bloomHash": "QmQRcq3pPM2sRUot9xiNxRmnVcqxTLGmBDTTF9v3wKcJFE", + "bloomSize": 262190, + "indexHash": "QmQgKarktuj4DvGP7QafpPDecmb7WoM28XzYBR542CgxyX", + "indexSize": 3129940 + }, + { + "range": "004169705-004170075", + "bloomHash": "QmVzJXPP9qarZqcjrXzW9k3J85GS7mhH1rBhLYjKqqHS9W", + "bloomSize": 131114, + "indexHash": "QmYJd8dsjFMyKKQVEq6SiWgnJGRnxTAo78xuuGrVxd1SoJ", + "indexSize": 2689172 + }, + { + "range": "004170076-004170492", + "bloomHash": "QmdUHXz2EqDuz7e8AuPnLRy7Derx9q6ndahG2V8X1N1usL", + "bloomSize": 131114, + "indexHash": "QmRZNBDoBWc6YiLB5wUWe4DtBrFoiWjbfMGyt2pbnqvNkT", + "indexSize": 2321800 + }, + { + "range": "004170493-004170824", + "bloomHash": "QmX1cRUsZUjQ9tAaEcwUDYFv7381ZtX3gaQXndhGhXX6Xv", + "bloomSize": 131114, + "indexHash": "QmVos6egzaujBoKM5VVUc1gF7kTc8bAiGmh8Jd8t6dk8JP", + "indexSize": 2803912 + }, + { + "range": "004170825-004171275", + "bloomHash": "QmVbTZLWLA5Qn8HL36YsDVJG9F13C3irYGM7ZpQ94Dc9Fo", + "bloomSize": 131114, + "indexHash": "Qma3RV6RHVhYvSvvcn5kaU3PkGXV5Eo6xXfFzkfAKJP3Ys", + "indexSize": 2428472 + }, + { + "range": "004171276-004171711", + "bloomHash": "QmfJJtDKstowaXhkZGyYx7akepHAZkfUQHxDA2dRdDDq2V", + "bloomSize": 131114, + "indexHash": "QmWMyneh5yy48eyLRH2uD9e4EXA8PmUCe2YjV5Uhscy8w4", + "indexSize": 2654592 + }, + { + "range": "004171712-004172163", + "bloomHash": "QmUFGq1rGbSCyktjs32zdZvP5tpMHEM9XeqKbFJaGGn8K3", + "bloomSize": 131114, + "indexHash": "QmS6RRkfCH2WWk7if7H4JeW27o56Wrx6tdke3wMqVKzfX9", + "indexSize": 2688648 + }, + { + "range": "004172164-004172675", + "bloomHash": "QmSeVtGSCcuKcjuXGXLSBmQ4E6oCm68Lf7KaXc55YrWqd4", + "bloomSize": 131114, + "indexHash": "QmVdLsawwHLrBWQ6fGzb49w9CQB6WL2yNyJ2PA3ufmSvoD", + "indexSize": 2887264 + }, + { + "range": "004172676-004173150", + "bloomHash": "QmSK2EfuFk4a7Cm4EVF7QF7JaGKggJWitTTYVS2eSyP3mH", + "bloomSize": 262190, + "indexHash": "QmUySJByVnnjzqRLyLg8fYZbDjN4LJNpeuXmsteoSGxcTC", + "indexSize": 3254708 + }, + { + "range": "004173151-004173633", + "bloomHash": "QmPzD52R4AhB69et2DUhLxRosiVr6kE6wPjQrjQ7rVu7ss", + "bloomSize": 262190, + "indexHash": "Qma8ditQ9XBcC2KxR4Fxs1nPFmsDJEJbT9xH1anxF5bF4w", + "indexSize": 3167732 + }, + { + "range": "004173634-004174071", + "bloomHash": "QmPPuun3EGrAjWuBSoZs8vaVQ9owM1yhbp1rZBiURdTv37", + "bloomSize": 262190, + "indexHash": "QmcKk6mJ4vkmQNei1C2iD5VkAnYyK5FwmavHNXMhtroy58", + "indexSize": 3045768 + }, + { + "range": "004174072-004174514", + "bloomHash": "QmPxZsyXMsa9ohEWQjEqTp8yxVU5dD6iEYQguVEskwmQt7", + "bloomSize": 262190, + "indexHash": "QmeYjRKEkymY1jqF8WoWKUzJSqok32XhByN5BuDZ1vsKVY", + "indexSize": 3090552 + }, + { + "range": "004174515-004174963", + "bloomHash": "QmNkpwp3ZPCusAm9rEpUsDNaDWbpd94SZFsDqaG9YTzJe1", + "bloomSize": 131114, + "indexHash": "QmbBw3ByRr2Y77kgvzghXYj6ny4tUJkU2PgsXM1hVbZVr7", + "indexSize": 2662296 + }, + { + "range": "004174964-004175369", + "bloomHash": "QmYyWsW78WDSZDWtRMAUKwtq6Ci9GL9fP216igP349EKwL", + "bloomSize": 131114, + "indexHash": "QmZno6pVvT5xToo7VQQ67q9HT2fAUvreicKR1V73wNKDtF", + "indexSize": 2248272 + }, + { + "range": "004175370-004175816", + "bloomHash": "QmRyMvxUDLdZjNGn9Z4CbjMtf8qVCTvot4fz9aQK2SmffK", + "bloomSize": 131114, + "indexHash": "QmTtDYeQC6vAstocrZyqTEJmmnqjvC9nG9P2qoy6pUDLAE", + "indexSize": 2279988 + }, + { + "range": "004175817-004176207", + "bloomHash": "QmS4vwRiM4MpdSRNoVrErhvxV9DfD6ghgqASkshcswu5xN", + "bloomSize": 131114, + "indexHash": "QmcC1Ko3K88m6EfFQEppnRzD6Fwx3v5RcXu9CimkbGa8tT", + "indexSize": 2149012 + }, + { + "range": "004176208-004176580", + "bloomHash": "QmY5us4gc2yYCVuSsYvoZ5Tz663PDwwmLE2Wk7Fd5aiGu2", + "bloomSize": 131114, + "indexHash": "QmdXM8THitk9YuRR2rJUCN3Myda86NrM1p1ZmvxpDQQGLJ", + "indexSize": 2194888 + }, + { + "range": "004176581-004176946", + "bloomHash": "QmRC64ZUYNPDSCpsaniV4n31hvnDHbzQRrVJfbufUjH89F", + "bloomSize": 131114, + "indexHash": "QmdNksGSQcf6n4BtDGRSpWHqLosQrDCtKmvPXgYh2w17eh", + "indexSize": 2236416 + }, + { + "range": "004176947-004177262", + "bloomHash": "QmPYgbZhKv7sLZ6b2H1CpRUBqsQ9N4pEwG5sWMpi2eWngq", + "bloomSize": 131114, + "indexHash": "QmUxCH5yveYCJqEGM3DwmTeYvMzqRmxJmSbjbzQbvAUuHj", + "indexSize": 2388288 + }, + { + "range": "004177263-004177632", + "bloomHash": "QmTNUrjoK3w8Y6bhfdpSco3DNmH4iPVPB96RF95pfcC5jz", + "bloomSize": 131114, + "indexHash": "QmZvN5G8JB5mkAnMEJAJ83hRhw7egBQmz9iLjnGhY3YSTK", + "indexSize": 2322892 + }, + { + "range": "004177633-004178027", + "bloomHash": "QmdZSAcLtA5oDbhPSaD9Bo7dafyXtFAEPrg2dHPxzNkbAC", + "bloomSize": 131114, + "indexHash": "QmPHLXfkZLzrXUR1ZXb82LPPYC7pETbvwgBAdVRhWzM4Wx", + "indexSize": 2343092 + }, + { + "range": "004178028-004178378", + "bloomHash": "QmPQLfrg2ZNqEK527r9ey1A2RhUNVnLjbtXP46bASgcbSi", + "bloomSize": 131114, + "indexHash": "QmRPke8QuGob1LAhVDzmqoCbf4nLVxAiXjk3mtrThUZkHy", + "indexSize": 2350432 + }, + { + "range": "004178379-004178756", + "bloomHash": "QmdQ3n2m7EsjsFAv1cfGZE3sDjqdxC4XcDjKcUWGXPPMGM", + "bloomSize": 131114, + "indexHash": "QmaEPsZQL7pdS7wmAYporPKhDenX7w56wNRh2AVTYVAAmk", + "indexSize": 2567640 + }, + { + "range": "004178757-004179173", + "bloomHash": "Qmakk2khHQaigKkEU6tAXVQmXXZhpbuexkspFzgzBQW5Yd", + "bloomSize": 131114, + "indexHash": "QmRsvGcrWrfdUxSZJ4MVxe9Q5mTMVGnutAuGYZYK3sUZxj", + "indexSize": 2645644 + }, + { + "range": "004179174-004179566", + "bloomHash": "QmW4L5PCTiGCvzwDDYFNFTAVamSiHh5AGWgcmC1XQKf1XM", + "bloomSize": 131114, + "indexHash": "Qmdzc44NUWdNDhkGVLPFPS3evpN1yn7Ymg25ZUSWXArnpB", + "indexSize": 2529952 + }, + { + "range": "004179567-004180028", + "bloomHash": "Qme3Fbz9XbJtGzhdvc18FbGDp7ScCzzX1BPRDH3qpKBGmE", + "bloomSize": 131114, + "indexHash": "QmeHDQa1WWsGcvA3dsvdNmCHxERH7zzuAhsYiQLoENuksj", + "indexSize": 2687404 + }, + { + "range": "004180029-004180504", + "bloomHash": "QmQQPM6mJDHxawGLJDyaB922JBzc1R2aS3Wk5H1c43kfwo", + "bloomSize": 131114, + "indexHash": "QmR93hwLyuj8Vzby94Pd4NTRUYdeuN9NbvyBisJKRVzz31", + "indexSize": 2606832 + }, + { + "range": "004180505-004180920", + "bloomHash": "QmPHX1FND2d3azVfg2UdtFZcdLTT81mUQqC3CgCEyUyTnc", + "bloomSize": 131114, + "indexHash": "QmbRXo8NjuFzRCvS4vXVVpbrHUmqZfhMWwxRyLjbsVVGqg", + "indexSize": 2577480 + }, + { + "range": "004180921-004181358", + "bloomHash": "QmdQYa4r7YTTkPosPzhGepss48DQSkXsAtZVsJVPyCJpR2", + "bloomSize": 131114, + "indexHash": "QmaW1siZ142C1d3yBYKLfuGxAfQ25EzjbavexEtgonaVN2", + "indexSize": 2483292 + }, + { + "range": "004181359-004181804", + "bloomHash": "QmanGcx2kccz6oniVFQLhy37ToZq4hwj2qtghtGQGnE6sH", + "bloomSize": 131114, + "indexHash": "QmYYTx2oXkLLZa1Z517E4mAV4XC4E6NPfGK9fN3n8JqWoo", + "indexSize": 2680348 + }, + { + "range": "004181805-004182366", + "bloomHash": "QmU2TPAGAU9njSwmoYX8cCv4hmiJPrxh4KAoEkiqacyDE9", + "bloomSize": 131114, + "indexHash": "QmNrQazxhDwB8jYBbn8X76UAD7PGjWoqEgbV6XT98aGBaL", + "indexSize": 2823204 + }, + { + "range": "004182367-004182891", + "bloomHash": "QmdCTwo2nimP3bwaK9QriyxwGJDkurtP2JeUjeMevT4jK9", + "bloomSize": 131114, + "indexHash": "QmcU3EcGDBzhfNf5VRdGAgFM9jz4DdNWe3uCisnEma9CZe", + "indexSize": 2733844 + }, + { + "range": "004182892-004183379", + "bloomHash": "QmTYA7LVpomj3Fd2rqc8QaCFhHEmj8h7WJ8sLQwxD2T8bo", + "bloomSize": 131114, + "indexHash": "Qmbrgk5duHyLax3kpX6sdMeRmZ7jFroVBrbW6VEf6nMrdY", + "indexSize": 2586940 + }, + { + "range": "004183380-004183680", + "bloomHash": "QmWtpEi5GKrBH2zPMoLAYYsik4kHXrQkkg2zJH6ARE2fXN", + "bloomSize": 131114, + "indexHash": "QmWnuERkfMnHJxwVGV99jgg9QiCKTREWhpiykPV4Zpp3dp", + "indexSize": 2136704 + }, + { + "range": "004183681-004184011", + "bloomHash": "QmPmreWe58ejiMmnByrovGik73FwQPoZ9ZBxR2Q2toqmTu", + "bloomSize": 131114, + "indexHash": "QmRCkNFFQw8ydzKza3shWYLkHX9PdW1ysjjqMET4sQXNLf", + "indexSize": 2136184 + }, + { + "range": "004184012-004184524", + "bloomHash": "QmdkipDkzCkDhd8BocABP1kJToSdE8xV2seZkYrhbu3qFc", + "bloomSize": 131114, + "indexHash": "QmQZZJj6J2ty7sN4ozzXfte7WRhnUxZhKtF1tVEzGR2hzF", + "indexSize": 2759256 + }, + { + "range": "004184525-004184839", + "bloomHash": "QmUf3XhXmfxVZGGeE6B6BHV4nVpUtKeACfk9b2CgA1dcgx", + "bloomSize": 131114, + "indexHash": "QmZb43uBEemLuiVs6FbtwH8ExjoXEpZDaw9eHDbdos4jdR", + "indexSize": 2271928 + }, + { + "range": "004184840-004185226", + "bloomHash": "QmfU55y6wjhUYuRX3r7k5vfoFowPtyqxPE8ZAYSf4HjwNs", + "bloomSize": 131114, + "indexHash": "QmYLVETYvomTQatezjZidhsZHRCgHNRG1i6NXAEwGqLYwF", + "indexSize": 2440316 + }, + { + "range": "004185227-004185646", + "bloomHash": "QmTPB1Vr5P3PsRm3V22dHjLWp7P2f8gFgwabjkkVNJQvs5", + "bloomSize": 131114, + "indexHash": "QmdULm6duuVPVqofZjznrjAnAztz6QpTQAXRDs7qmm12ud", + "indexSize": 2419692 + }, + { + "range": "004185647-004186036", + "bloomHash": "QmTWK6b4HhPiAugdEfKiay54Nc1xz2wXabSNbdZCLDGs6e", + "bloomSize": 131114, + "indexHash": "QmSMaGgc2zsEGubj9SMK3QoBHYG3SLchsWSiNJE3hVmwij", + "indexSize": 2473260 + }, + { + "range": "004186037-004186350", + "bloomHash": "Qmf7r4y7ZcVET65Bg7W9SiFE112daaqqRRsHjK61mEj1wd", + "bloomSize": 131114, + "indexHash": "QmUV3YiMEJcJHG1KuZB1USw54fTxRyTthTbia1gGTGKuha", + "indexSize": 2209172 + }, + { + "range": "004186351-004186836", + "bloomHash": "Qme1RpgUyNq2wJoui64RWvGUaHbRF4fFE84Uvjb8EkbBvh", + "bloomSize": 131114, + "indexHash": "QmRNokBXDLMnArbc225DdsGMKbxC9A1PTSkcTZcAbYv6Q2", + "indexSize": 2407460 + }, + { + "range": "004186837-004187246", + "bloomHash": "QmV4J84c6cYeseCtbKG1R8B4fBvnnmEQ1iyR1QeBmH2pBt", + "bloomSize": 131114, + "indexHash": "QmZXstzsiM7UaE72V1mqAN157QV1mWD2ivf62SnnVtgaYE", + "indexSize": 2294928 + }, + { + "range": "004187247-004187625", + "bloomHash": "QmVhU1bR4gJ2FkhinboVZmYRawF8v3HJHa9kUdjTsEx1iU", + "bloomSize": 131114, + "indexHash": "QmYADURi75ntEaDS1y7Yq1vgx44XV9ertwgg8E3CNfXSx5", + "indexSize": 2025948 + }, + { + "range": "004187626-004188006", + "bloomHash": "QmQ1W4J7wS4TWPPzUJ6k6o6uqLfgqE8sQHXBqDZ8Pi7vac", + "bloomSize": 131114, + "indexHash": "QmREesyE74PNHvxmt6knuz7X3CGSjB37w9JKctBzF3wPdQ", + "indexSize": 1930500 + }, + { + "range": "004188007-004188408", + "bloomHash": "QmQcUP7pDXd3CF5XPkqU8JUbrqHQeHkWeQzn3qLnEFE1v9", + "bloomSize": 131114, + "indexHash": "QmTpVNxrY3uQnxE2wq6pYt7k7k8F4JBUPg5v8wL1VH2qm9", + "indexSize": 1942308 + }, + { + "range": "004188409-004188767", + "bloomHash": "QmS9XEn43SMDkkYWHo3Hs9JRXv2Ah7wX774CQ3TwtSy9Xn", + "bloomSize": 131114, + "indexHash": "QmQVc98Zw5Yac17chfqeTA1adiMoTg6qGzq9gG9fDRNcbY", + "indexSize": 2146452 + }, + { + "range": "004188768-004189183", + "bloomHash": "QmPK5KsWWvWuPtGNJv1a7z249Hwu2EPHtWdNtHnhn87ckL", + "bloomSize": 131114, + "indexHash": "QmQBun5HdwctsCM8kRiWdSKpCHFmCqmWaAz8i25TzNLfAt", + "indexSize": 1981528 + }, + { + "range": "004189184-004189548", + "bloomHash": "QmRsxDkms1BhApMzghn51roHEAXs3FdVum8wwUP9MZy6fg", + "bloomSize": 131114, + "indexHash": "QmVSRgZLGP5iTWnWkgWfTmbiumFeVBzHrHg7HPis8NCBaM", + "indexSize": 2259716 + }, + { + "range": "004189549-004189919", + "bloomHash": "QmaLZUZ1A38qeYfnLvmVvmVzmYbidsSJMtFi7ZG9u48K4h", + "bloomSize": 131114, + "indexHash": "QmUst4T79kyzMvRWrmSdnbBdKPkrrZVRbanZ351rW3uevm", + "indexSize": 2430828 + }, + { + "range": "004189920-004190402", + "bloomHash": "QmZp2x9yYJZyi8xnB8hpXCaPzh5nukX7nctHohJ1wjmngY", + "bloomSize": 131114, + "indexHash": "QmWLzrNQwQV6BD9L3nA4cpFJHzAFjRg8iGHH3w9fqtneQ7", + "indexSize": 2611320 + }, + { + "range": "004190403-004190902", + "bloomHash": "QmczZSq638A6na5dfxfAVzJT1ERrPCc26gQyrksBi43AvX", + "bloomSize": 131114, + "indexHash": "QmQrrWuNF4F4vfcJp9TBYAKY1QTcWkxUxZp7kJDUa5ZSyL", + "indexSize": 2592828 + }, + { + "range": "004190903-004191357", + "bloomHash": "QmWV3wWQwwXNwCHVgNUpRVJhCDJDv18aKRg9uDH59dxM9p", + "bloomSize": 131114, + "indexHash": "QmXUEHNf7tnWVy1R4iyRZ16rgVXUaARWPSA19DkhQpfiTD", + "indexSize": 2607308 + }, + { + "range": "004191358-004191788", + "bloomHash": "QmXDboh8V8m3ruxTks4q5p5mojHvX4wgJWh8FyApAg5XRU", + "bloomSize": 131114, + "indexHash": "QmdsaM9GNwokJ79oXsswBuV1uHjyr5fmdwxvmk6Urg3Les", + "indexSize": 2625848 + }, + { + "range": "004191789-004192254", + "bloomHash": "QmYT7YrzkP94iPfrfpUbfNpcdLgRpNbvErbQqZitL9yZLT", + "bloomSize": 131114, + "indexHash": "QmPd92gyrGYH6fV9rah6PV5d7g5pxTRLyVXgsj1dBhtfW7", + "indexSize": 2611608 + }, + { + "range": "004192255-004192649", + "bloomHash": "QmWfLi9jjY4o6zGRg726nEkGJQw9F5H2VAPDtsMfz8YvL8", + "bloomSize": 131114, + "indexHash": "QmYyENA4Uuuu1VvsN2Csf8KeAxSnkNxKUK6dsQkjkJM97v", + "indexSize": 2491580 + }, + { + "range": "004192650-004192974", + "bloomHash": "QmaPJiAobsxWWkuZPYz2QR2K33aAXqKvbV4QAqiFXLuTVK", + "bloomSize": 131114, + "indexHash": "QmWzCbPTAkS2jY5We7gYK7eZsVzSoyqB8fJHQAbKEqQ9kk", + "indexSize": 2288432 + }, + { + "range": "004192975-004193256", + "bloomHash": "QmTJpeLmiW73SphiXXvPnWmbo4suqofFF2hvaMTNQ5Kh7i", + "bloomSize": 131114, + "indexHash": "Qme32gHiBq5AjCcTbUKH12ds9B4ZG8GeofvwLmZrxJQ4Ey", + "indexSize": 2137640 + }, + { + "range": "004193257-004193565", + "bloomHash": "QmSCkrzGsRKpNSs5kt7AKaf6t9rzuqFre2CtAUqioMEftZ", + "bloomSize": 131114, + "indexHash": "QmSb8UJL6YNGfenYTANkkskjom22sLpEkkUFh4NqzovzUY", + "indexSize": 2262280 + }, + { + "range": "004193566-004193958", + "bloomHash": "QmSFirbiTWzLx637UuQuXtjuAavPm8LJQNNMWhHTzkPD9D", + "bloomSize": 131114, + "indexHash": "Qmekyi4VML8y96zZCCR6pXK9RLH2Lrx5Q8BbETnaB6iRa5", + "indexSize": 2064224 + }, + { + "range": "004193959-004194411", + "bloomHash": "QmZRdaa4LR9HEDoWDkqV2BSH4RUY4bny9f1TqRjcH6amcw", + "bloomSize": 131114, + "indexHash": "QmQoJeEgC3YiMjbrZKV5Gdd1oBQVqSNjcSLLHbtYDEpoBV", + "indexSize": 2067768 + }, + { + "range": "004194412-004194817", + "bloomHash": "QmZNNLv5dxEiuRKiU7rhjKt37ER3BHMseRDxngx3FFcX8R", + "bloomSize": 131114, + "indexHash": "QmVFaxzYH8NWfXCUfmMJLkQqTwcy2wuzvhhyEawkZ6yh4V", + "indexSize": 2344464 + }, + { + "range": "004194818-004195228", + "bloomHash": "QmXkA79XqNzMUyYQN5Bjj1z3yZ45FATo4M6nm67Kw7GsSJ", + "bloomSize": 131114, + "indexHash": "QmZi9em8pYW3ARFPvoX5WSSzzbKraWHbNJ4c2ZdgkJqVUW", + "indexSize": 2386652 + }, + { + "range": "004195229-004195660", + "bloomHash": "QmUaJtyrmx2P97zeGF19H8v5eG898MZi2kT99LjBWcpP65", + "bloomSize": 131114, + "indexHash": "QmWt5x6JnMP6cDMgFTQCY5WcgZDN1JE46zVp6ne2hYhbrV", + "indexSize": 2476548 + }, + { + "range": "004195661-004196047", + "bloomHash": "QmcswuKARRgHckC5afmq9cxQu9rh52XPn6epZnbZgfe7bR", + "bloomSize": 131114, + "indexHash": "QmQkGFef91fzL9SsZHEV47FcfwBMfhULgkq6dvb9ifENi2", + "indexSize": 2427112 + }, + { + "range": "004196048-004196398", + "bloomHash": "Qmb6py2nT7sSXST8Mu9LQ2JmbjMq1mv4KVgFfRXyucp9yA", + "bloomSize": 131114, + "indexHash": "QmS6bqJgt4W7yb4aLj3SFH4w5cXjMCrfyK8kdgkpkJxDXi", + "indexSize": 2496724 + }, + { + "range": "004196399-004196774", + "bloomHash": "QmeSqyFgrA5GB7P6bouiN47xtwAYNCyYjMor1AvsVGYVPT", + "bloomSize": 131114, + "indexHash": "QmaqFKeRzUrRgtL8qYkMJYH7reFmfDQgh34MNPEu1BRpY5", + "indexSize": 2660376 + }, + { + "range": "004196775-004197190", + "bloomHash": "Qmc3N3qXkpnAtPwxxbQ9qmzrhzXWU7HGqgs1siPodiWzpx", + "bloomSize": 131114, + "indexHash": "Qmb1k27UeMFEZ3RtvAzfqRjAD4bjDCsbpdNAgT1HPvUUKu", + "indexSize": 2536372 + }, + { + "range": "004197191-004197560", + "bloomHash": "QmX8sawBGutFdJDMvbfPisJXagu1aaaNhpgAKWNPbj8jo9", + "bloomSize": 131114, + "indexHash": "QmTHC95UzNc5AtykhEwgrg9UQqPkXwsipLBrd3CiBy3DhX", + "indexSize": 2493196 + }, + { + "range": "004197561-004197939", + "bloomHash": "QmWrNRkkCXXGnAR1fK9dmffSKbmRq1ijtdqEMDcw3NnAY1", + "bloomSize": 131114, + "indexHash": "QmQXPA1iAuPoeKTaTmC7zpedZF7dSwSY1WcdEjLTmYJfPW", + "indexSize": 2687808 + }, + { + "range": "004197940-004198289", + "bloomHash": "QmTyxYbBPTaep7jNdRAKLzDTc9VHpF8uGBuzkJEJpxv5P7", + "bloomSize": 131114, + "indexHash": "QmYaQ3fTSufgG2PNpTsSyUgsETygFXYYZqNwN5h29Zc337", + "indexSize": 2776512 + }, + { + "range": "004198290-004198626", + "bloomHash": "QmYYp9QierbKDDdoicqebCe4p5hfLe7x3GkuvELEo3KJC5", + "bloomSize": 131114, + "indexHash": "QmbUxZB23gDyy3ML1oPo2BPK9GSEM21VZ3VirPRCChp6et", + "indexSize": 2653408 + }, + { + "range": "004198627-004199001", + "bloomHash": "QmZXLxN8fCqYyt9izyGBnW89wMqRcBrZ8CF7rdntwzZVJ2", + "bloomSize": 131114, + "indexHash": "QmPhfDg58Brvw6awz7mBkLeqpHnFJ1QDyU3eJZ8dXoG7vb", + "indexSize": 2549048 + }, + { + "range": "004199002-004199372", + "bloomHash": "QmUu4UUbVcjpbTQm2mch27uTLNRH698TYbodd6mpzPREs4", + "bloomSize": 131114, + "indexHash": "Qmb4kAWohiWveLfoidmPqsNL3dVqnHJQ337qVAWGhnarhQ", + "indexSize": 2624600 + }, + { + "range": "004199373-004199757", + "bloomHash": "QmZ9ds2cMLCxZpzHZd6XZ1kMH4i7Sxjmj5ND2o7o3xdux1", + "bloomSize": 131114, + "indexHash": "QmWYk76vE18JkrzEV9cWEL2RjSW6JNW9Tm3QrPUgRhCToY", + "indexSize": 2650012 + }, + { + "range": "004199758-004200000", + "bloomHash": "QmPdLbTte2fBsVdVF68uoJZhUSdM4bZoFUpC5e4yAZkzP6", + "bloomSize": 131114, + "indexHash": "QmQeEjKyQtXNSTMYQHfuhQvy1XF31xjfjbH7UUPtHk3aDq", + "indexSize": 1670404 + }, + { + "range": "004200001-004200385", + "bloomHash": "QmZqEeYUu1xsmWDbZmzBMP9a2RynTe3WkEpvJS74rSzypf", + "bloomSize": 131114, + "indexHash": "QmXeorbFisQ17rD29ccVqCGWQ4gS7jBUP6MtRdMow8KhsT", + "indexSize": 2511692 + }, + { + "range": "004200386-004200770", + "bloomHash": "QmSJHug1LBdvSjCNVy3ACe5MvjEQXDd3KvG2r9JFYbw3Vt", + "bloomSize": 131114, + "indexHash": "QmTgTp7qXBE36oWTKxQVHzH2tv8tmRE6zqJ9uF4uKvyWWw", + "indexSize": 2639780 + }, + { + "range": "004200771-004201110", + "bloomHash": "QmVgzoh1UYQwXqyJnBxzK4APXtmsVokL5kBzp3eTdkZ35q", + "bloomSize": 131114, + "indexHash": "Qmf1KrTQgArPgC2EKYPEQaqAeNFGUkZqZmPvkL4JtQ8GcB", + "indexSize": 2070636 + }, + { + "range": "004201111-004201406", + "bloomHash": "QmXdN6YGewp1WLa4gMf4nwT5JqV4KgM6JAneoAY8BPyWc2", + "bloomSize": 131114, + "indexHash": "QmbhkdmqiWZ1Ryh9gjaKSTQg1w7BwnKPRLMxPLKQ2smyUi", + "indexSize": 1961740 + }, + { + "range": "004201407-004201727", + "bloomHash": "QmZmuZZg48qvo6PKtorKyHX87V6GbZQCFWTWxLMNn8w3Yr", + "bloomSize": 131114, + "indexHash": "QmUUsLxs4nU2FCtzYGmXFxD4zLQRhNWpPJHhYQJbUD6Ycn", + "indexSize": 1967948 + }, + { + "range": "004201728-004201959", + "bloomHash": "QmVVCJqaZtpLj632wCL88exobjaQsqw4q766hsfeuLS5xS", + "bloomSize": 131114, + "indexHash": "QmWKkmwvrGGvJkaHPMFDjkCKjJWTBnbtPY6aToK3saqASg", + "indexSize": 1849556 + }, + { + "range": "004201960-004202213", + "bloomHash": "QmPqxwpFY1WZZ2H3kFbhx2k8DQTG9ndo5iM3tQ9feNimTv", + "bloomSize": 131114, + "indexHash": "QmQrZ52fse6srqbGNPKyH6UG9XypzCXCu4VRcd2mnzV75H", + "indexSize": 1925216 + }, + { + "range": "004202214-004202469", + "bloomHash": "QmRiBESEwazfGYdBXbEKkSXzHdb3mAgHcMxrhaHC7ZRMFb", + "bloomSize": 131114, + "indexHash": "QmUEqJf1LPpYQJN1nkzVLRGKHDRt2JRU7TVJVkhbs8K4V3", + "indexSize": 1935136 + }, + { + "range": "004202470-004202767", + "bloomHash": "QmUQX4t3CixRfWiZWEAteE9S5JQM3tNuex1TePxRsLqY8Z", + "bloomSize": 131114, + "indexHash": "QmQZyDXLUXGWNVq4Necxepei4oJPpdaYX5BLCySQr1iw8g", + "indexSize": 2020364 + }, + { + "range": "004202768-004203029", + "bloomHash": "Qmc2bMZT3ubwpEa8DF4rhd3QXLvubcKq8UEU4xQqUwi1NB", + "bloomSize": 131114, + "indexHash": "QmbbhxEX37eVsEDBBv34gdTSDDkyBqLLFgF211BASx1zLz", + "indexSize": 2181824 + }, + { + "range": "004203030-004203312", + "bloomHash": "QmWogH85RztrzCZEHjDReCCLtfaLT9EDDr7BsMZeLDA5S6", + "bloomSize": 131114, + "indexHash": "QmQZ3d6Jg1PraX1QJf4CCbetAeP9gKEhSY1FKGb1vYMCet", + "indexSize": 2118584 + }, + { + "range": "004203313-004203577", + "bloomHash": "QmTmtFCT82JYj6Y8hHtZ5sdtTGZre8V6j71fKDQdTfS9UC", + "bloomSize": 131114, + "indexHash": "QmSTKyzL5aCgwRCPYy2b7D6X6bx19khytjpcgf2dqnKP6S", + "indexSize": 2191920 + }, + { + "range": "004203578-004203855", + "bloomHash": "Qmc4w3wBQmJQugPbpKLZ1pdyqFx1tumCeyAnBMyxe4WqNR", + "bloomSize": 131114, + "indexHash": "QmPurtpUgy4dBC2d3c2hNHuo1Zjkud72Jq96NzmQaYifn5", + "indexSize": 2098100 + }, + { + "range": "004203856-004204145", + "bloomHash": "QmSZumnQiw7xouCkn7DXUK6tYNHLndUZi155TaUd19uHFH", + "bloomSize": 131114, + "indexHash": "QmWFnKaQEEGxfcNKnTw7DnjByYMgMX42ZzYtGRoiPod5E1", + "indexSize": 2079644 + }, + { + "range": "004204146-004204433", + "bloomHash": "QmPzepfK5eF552mBdRvBfJQRiiXHh32MZM6Wv6wxm3BLXs", + "bloomSize": 131114, + "indexHash": "Qmc55mPAccjcqVSrcw52sYKQbDBK9rCdEx5dvp3cdeU5f5", + "indexSize": 2248248 + }, + { + "range": "004204434-004204741", + "bloomHash": "QmTTDVGQtPhZ4mcBBDktpgcAruARLwiqEmXx1eCzdnaVQG", + "bloomSize": 131114, + "indexHash": "QmUKRtNSt8ZFoyijjtDn1TXePN3vCBF9FJ9YEUA5k6FsrE", + "indexSize": 2231592 + }, + { + "range": "004204742-004205034", + "bloomHash": "QmcxGU9rfgTFNZYQxkvadLzw7aKCEgSqZ3XUQt7BCwhnUL", + "bloomSize": 131114, + "indexHash": "QmabVJxBG4Cy9EhPThBjc1D3TVriXmFuyNQXepsCVPTqxN", + "indexSize": 2168068 + }, + { + "range": "004205035-004205336", + "bloomHash": "QmYTrFMvgVx1v26Mur6eu3QbExUyMRSFaFVjPEusk52iUC", + "bloomSize": 131114, + "indexHash": "QmXrsbLtCwoLjrMbhGX6rrNK8NKvbgTYEVVsZGos7KLtU2", + "indexSize": 2160796 + }, + { + "range": "004205337-004205650", + "bloomHash": "QmVscgmkN1gzg6ogg54peRVH1CfxNPq2P289RAxxHeVSM9", + "bloomSize": 131114, + "indexHash": "QmeJaqQgJDUWWfSuUoTNSLtuayqfFpyxGksL812weEUC8M", + "indexSize": 2144832 + }, + { + "range": "004205651-004205909", + "bloomHash": "QmX5fHZGwqSyQfmKeoT4cgJnHgRJEjYexpkoHZw86vHkyy", + "bloomSize": 131114, + "indexHash": "QmPYHHcPBnLbeJ5ogaNqfV7UUJ41fq9PXh5ga3tP8s9W4Y", + "indexSize": 1908680 + }, + { + "range": "004205910-004206167", + "bloomHash": "QmVzpHQi3C67MFNXvkCSCtbMFav494sxnbciY4ar79QCVf", + "bloomSize": 131114, + "indexHash": "Qmf4Szqub6x4pGDYq6yy7UuvjztQmihntHQCZoNyiqXvy1", + "indexSize": 1919328 + }, + { + "range": "004206168-004206416", + "bloomHash": "QmQRF2x4s6hSwzwWXSrY3qDvbN9YdGyxeTTp3gPzbyDAkP", + "bloomSize": 131114, + "indexHash": "QmWcgcdjUQ5T9pU1eKgTkE5MfaFNVQENWriCKZ8rXN6zjZ", + "indexSize": 1848868 + }, + { + "range": "004206417-004206672", + "bloomHash": "QmQBC4WxgFxLCqkD1SyatKyxYKk1yn9fYgMeE3iX9kmXis", + "bloomSize": 131114, + "indexHash": "QmaFQ7Wb5KP8dUtnmdUwkY3hYwk2hfy3nL8GzYxQvFWQRx", + "indexSize": 1827480 + }, + { + "range": "004206673-004206932", + "bloomHash": "QmZPCsseJkNjpXyHYMnwyDowYZEbqnL43h7LdtH1fouFvm", + "bloomSize": 131114, + "indexHash": "QmVNo1xoeg4y2bNmmyrMZ1KLch9N4qNnkNNVyBMsB37E3S", + "indexSize": 1846320 + }, + { + "range": "004206933-004207212", + "bloomHash": "QmQr2dqyZADYCmQnthaJBZnVznvEidkn9JVwWFeuxDiXG1", + "bloomSize": 131114, + "indexHash": "QmYewuhyhoti2ekryRBp35eokK1tz9zJH7NL2twVTqXsUA", + "indexSize": 1924408 + }, + { + "range": "004207213-004207506", + "bloomHash": "QmaAgWuWbTihZveG1BasYhHRL1yLC2KKfrT6EgM44vqffY", + "bloomSize": 131114, + "indexHash": "QmSZqtiGJP3ZiLdKHXgHm1aSQVEcb5ymxCQGjd8hWw4TAs", + "indexSize": 2108456 + }, + { + "range": "004207507-004207762", + "bloomHash": "QmZEEdmF7Rh2mVxTCyvV5pWV15QbjsxKAAXCoXj2zczDmt", + "bloomSize": 131114, + "indexHash": "QmdBBSccS1TELhnV8agE2y3aZMS5ruVJdbF5jV4vAwKgPT", + "indexSize": 1985072 + }, + { + "range": "004207763-004208084", + "bloomHash": "QmeT9wuGa2Z5MyUa75LcXKDLvJ3nzvmWfJu6qaMSaeEtNo", + "bloomSize": 131114, + "indexHash": "QmNkB5xsUV6o6rBVDw38Ex9YnE41EQ6gLXFGLY5GvZWFx8", + "indexSize": 1912964 + }, + { + "range": "004208085-004208428", + "bloomHash": "QmUUNdH2JMGMdk3MRTCA6AANj94aZAvcs9sVzuvQdWVMAP", + "bloomSize": 131114, + "indexHash": "QmZQrfMpWYdRt8t5bSPq7kxBSKGEFx8ojQoczFrFivsxwQ", + "indexSize": 1920580 + }, + { + "range": "004208429-004208769", + "bloomHash": "QmVyA1xd8mkJM6XpjaNaJFZ2EJ34WtBKwNAk1dDpF6MNY6", + "bloomSize": 131114, + "indexHash": "QmVyMRc8iisyp3n7CJ4zU5tWg9uSXoENYFpJmF6dnFuQmg", + "indexSize": 1897332 + }, + { + "range": "004208770-004209079", + "bloomHash": "QmSs2Cp7uKvTwU18Uu66mbbg2MwNDq6jcVyJNZXXhwzUdm", + "bloomSize": 131114, + "indexHash": "QmZh4YHiuqTDKGSsaTiacJQ1xLtmHn8ydS8CJyV1GMfe8o", + "indexSize": 1888816 + }, + { + "range": "004209080-004209436", + "bloomHash": "QmTtUQxMpSUg1fcFpxaBX81pWyL35SwrbGg7gtGusvBdDT", + "bloomSize": 131114, + "indexHash": "QmXtNxfVQFhBXxPbTBmBVUPZRMcgVAkHxXnddVy83GBSrJ", + "indexSize": 2020588 + }, + { + "range": "004209437-004209738", + "bloomHash": "QmczVHcYjwoDpqpfTvZmZXyoeqBRhUCpMSJ2Bx4ckMJxbm", + "bloomSize": 131114, + "indexHash": "QmXBBWM4p8sCbkwSYnbziCWEgRSm3Bu3GTShXUHuDvhDga", + "indexSize": 1971568 + }, + { + "range": "004209739-004209993", + "bloomHash": "QmeGmWUcp71fAgRcWsu2T4fgSDWmypLQ5nkvjLqZAKJKWB", + "bloomSize": 131114, + "indexHash": "QmbU4xZ5J8HHQpJuzyLEHMEe9CAt9MWmq8VoAWxVKaEdUa", + "indexSize": 1916716 + }, + { + "range": "004209994-004210304", + "bloomHash": "QmeGTpUq8RmBKUaas13cvstvSt4CV1Moq8Bpeo4bDR8eAN", + "bloomSize": 131114, + "indexHash": "Qmd4W5NteEMtJ9azVphC9CXpMc8qQbY9AWX4K79rCrj3pL", + "indexSize": 1939948 + }, + { + "range": "004210305-004210591", + "bloomHash": "QmeLF4nR7f3eD1yNBr3SNYBJXFrftSARjX9k7j76iqFLaY", + "bloomSize": 131114, + "indexHash": "QmV1QQrJoLWd9pxkJS3TKqsG9L9G2eHFVSefL5yfm2bm2Q", + "indexSize": 1954164 + }, + { + "range": "004210592-004210867", + "bloomHash": "Qme9VkQRgeARhVpzm9MKPrpVuPcNXzKiLhnDwE89XWKnM7", + "bloomSize": 131114, + "indexHash": "QmWtdbJPJaLCorXiD7gn4Tt89XVQLqfN4KTjwTb7udw4gd", + "indexSize": 1972876 + }, + { + "range": "004210868-004211116", + "bloomHash": "QmTrn1LbAkqXFtbaiV4A7sqqEbPgJgDfMNw1hYxPAxnJc7", + "bloomSize": 131114, + "indexHash": "QmdLii4eoFXwXVcsEwFXLVzBZ2bS5KHVTrcrYj16xdFav1", + "indexSize": 1940112 + }, + { + "range": "004211117-004211387", + "bloomHash": "QmWFwcGW4ugP5VrUrAnwUYgUadXKcc438wqF6VP4gn7ZFV", + "bloomSize": 131114, + "indexHash": "Qmb62u8vef8doK2bfFsTBZSXUEMYf9iWvLmV4d2SEWzDxy", + "indexSize": 1954192 + }, + { + "range": "004211388-004211683", + "bloomHash": "QmTovW79yQQWwucDNbwjn7fxuX9jGJcLvoZC2FbpHZVLR8", + "bloomSize": 131114, + "indexHash": "QmQ9UgVorCYi5RcfNajy3we7Sds9S883EHUpCe4M9SSjx4", + "indexSize": 1991836 + }, + { + "range": "004211684-004211954", + "bloomHash": "QmUdjPJu1J9RaF8o1TcxY3W8axDKGimGnFuUHzLgVqubMY", + "bloomSize": 131114, + "indexHash": "QmQ5Z8EnoneeX8jEJwsv4iw4XBPqvmZJxaAy6rfLv6atii", + "indexSize": 2045668 + }, + { + "range": "004211955-004212229", + "bloomHash": "QmeJaGeTfcopcXVE8VDMgyRHnZCbvbfWLaGVpWMaB2FsTy", + "bloomSize": 131114, + "indexHash": "QmeW2mAooztMZKaq4nA2vngHCWZdRoy9NaShzosyxSqtnu", + "indexSize": 1948836 + }, + { + "range": "004212230-004212544", + "bloomHash": "QmZe6Rp2QgnHQtNoFTdskUP4tm6nEeHUjcsYzhuyqD6nwt", + "bloomSize": 131114, + "indexHash": "QmdQCJLTWrp2tdimDDFN7FZmKJRZrNG6V5dhtapJ8f3Hxd", + "indexSize": 1948176 + }, + { + "range": "004212545-004212813", + "bloomHash": "QmbxU5MpY1eL9LLj9Y2mk7DXZZLNMYsj7NF3rN5Wi656eW", + "bloomSize": 131114, + "indexHash": "QmXT67je9YQeucgf77GqKhSV7wnxx3b3UdpQyCWS8SdCp4", + "indexSize": 1840388 + }, + { + "range": "004212814-004213121", + "bloomHash": "QmVvMM6nmGodGoxfTCuZaNAirm8mcPFEMUFf66oy8rRmya", + "bloomSize": 131114, + "indexHash": "Qmeu5QPFN1V5MWdJi6yWag7ts4DD7TfMkoHL4cwMKLAPiR", + "indexSize": 1886084 + }, + { + "range": "004213122-004213437", + "bloomHash": "QmPQZyB7cyKy8USXovbNWTMGMXyAbvsU4L1E5ZW8NLEakm", + "bloomSize": 131114, + "indexHash": "Qmf4V3Xkqntg5N5gW3tykmnDSd6njcSMwjtr5CdStEMcEc", + "indexSize": 1839144 + }, + { + "range": "004213438-004213724", + "bloomHash": "QmWZKSnnAMZyZ6sf2o8ncM8J5G8GjKVCFgX7f35aWuwJ7n", + "bloomSize": 131114, + "indexHash": "QmS4fTUrFr1HaFW5nRMBxw3vjwHX6p5vGYZpoTuPCFepQa", + "indexSize": 1808804 + }, + { + "range": "004213725-004214015", + "bloomHash": "QmWHGC1suf3ZhqHd8ZNrKG3NSmQTepF4moasdbA6PNZswt", + "bloomSize": 131114, + "indexHash": "QmZWPu6ybSSJAf4iNWaXF7hrbpXUmTRX7KE4v239ftouwG", + "indexSize": 1931736 + }, + { + "range": "004214016-004214276", + "bloomHash": "QmWmAHQaccqFWKxTajB4osm99tnN4ogJbMJ9fppycMc2Yt", + "bloomSize": 131114, + "indexHash": "QmPGq4aWWrYgKR1YTgaYuwtzRPXZTLaeUv7VUDikpBBEm8", + "indexSize": 1883864 + }, + { + "range": "004214277-004214565", + "bloomHash": "QmXJDV4KfaUanuTSUQN7z8mhM5ubci8SCdWwE44boaMCin", + "bloomSize": 131114, + "indexHash": "QmPdCAon9Gknj1NuwMuEGauQuNWqhdMDvd9Ss7VA6ek8pp", + "indexSize": 1833696 + }, + { + "range": "004214566-004214867", + "bloomHash": "QmYY1u3qZds3jAxcKbwYduKsNNAYQazwLaroJADqmweTt9", + "bloomSize": 131114, + "indexHash": "QmQ4ZMhGvwHLQxCVW81kDB9bbc2hsg2DZFvqEc8StQr9b7", + "indexSize": 1829924 + }, + { + "range": "004214868-004215166", + "bloomHash": "QmWRENqeDG9iFkpzDoqVCvt88vi88Wm1VQYBSN1QEKU4ey", + "bloomSize": 131114, + "indexHash": "QmY6witNELL21XKVty5t7dG25HQ3arN1vZKt5GmaEjB8u2", + "indexSize": 1917228 + }, + { + "range": "004215167-004215409", + "bloomHash": "QmZwio5LoY44sjJHx2u38wWGC2FekLtweXaZRyLUHLsdHX", + "bloomSize": 131114, + "indexHash": "QmW17fCZuG6vFbZe9S5uzi7qYLBwh1yF5fuKEBKgvwUNTf", + "indexSize": 2192168 + }, + { + "range": "004215410-004215670", + "bloomHash": "QmXvQ6LE8WTVwpbouBLykmJ8JxvutDsTMR4FKrppN5Gbcv", + "bloomSize": 131114, + "indexHash": "QmcuhjguwZPNWSFSuS3m5Dnw8N7RxjHQ2L1Wo8XUWwfBLt", + "indexSize": 2230344 + }, + { + "range": "004215671-004215971", + "bloomHash": "QmXdTXwtc4hUApKPAkaZuhY8ZEGoLnGu4ZzmvEMuUMco6P", + "bloomSize": 131114, + "indexHash": "QmZsAWaG2j7wP7Lh5CutV2g6eYHiUFTZ2bnqu1vrpaSaUp", + "indexSize": 2301832 + }, + { + "range": "004215972-004216227", + "bloomHash": "QmVbCzVk4GWnYvmdvmD7iGeTznbviJUaNaNnR2Jm7KdgvE", + "bloomSize": 131114, + "indexHash": "QmW3HVxcwU6TbiY96iK9CZFuiMTeNzfawA5DVnvjLcf7t9", + "indexSize": 2268140 + }, + { + "range": "004216228-004216477", + "bloomHash": "QmYdTUWoUogF8M16B4VRbMSEYBLtWVSxD71Cqo54rj1x41", + "bloomSize": 131114, + "indexHash": "QmNac69i1BnCTzHnR48h7zfze6kQBLBYRTfwMGMCzB8RF5", + "indexSize": 2196040 + }, + { + "range": "004216478-004216805", + "bloomHash": "QmWMZAD1ieRrfZPzRkN2mnEfgg4WnQDh764VWfXAnnoxco", + "bloomSize": 131114, + "indexHash": "QmZfZmwmCkg9LmQF9PJzyukXe9Y9JSiXsT7vwYapozD5RB", + "indexSize": 2229120 + }, + { + "range": "004216806-004217109", + "bloomHash": "QmU5oTYYUQ2k2R2mdY7u7vrywt2SuaiBUZNDakAbreipLS", + "bloomSize": 131114, + "indexHash": "QmW6nsg3aubwzonGkAYD72cMMPWg6sxisi4PtWL6MrUFDg", + "indexSize": 2191900 + }, + { + "range": "004217110-004217404", + "bloomHash": "QmTWizEMFAF6TDVnqiDjNUUVVKXJQfghihz8yro3SzBVnz", + "bloomSize": 131114, + "indexHash": "Qmf7mZfGNG4yjVkrVW9Ygkb9YC6dx9ZuM1sJze7dpRg91F", + "indexSize": 2231788 + }, + { + "range": "004217405-004217694", + "bloomHash": "QmSmnSHz4r2yWrgZ4jZmwmXsXoLNmYwC4sG3Fn9Tf3QdVW", + "bloomSize": 131114, + "indexHash": "QmcfbDD9bgnETgQZBsxGCaL6R575bmrCws2Hd3urv8Snpr", + "indexSize": 2304932 + }, + { + "range": "004217695-004217995", + "bloomHash": "QmT3ANrR9Vqr4r6pom5RLJFXLHkk1JGEX3ocZnHHvL1WYh", + "bloomSize": 131114, + "indexHash": "QmaXg7MDCd4uPnpPr864JhmG8ssuLeqrpdcse35bSJrpbW", + "indexSize": 2345256 + }, + { + "range": "004217996-004218328", + "bloomHash": "QmNTSNKchnrkK8j3559sp2NmurYoevZtgnTpPF6HBCSWpK", + "bloomSize": 131114, + "indexHash": "QmSNDQQm79qA54VGyQy3eoFvv1NUS4eucdufuXh7EJeHgb", + "indexSize": 2444536 + }, + { + "range": "004218329-004218667", + "bloomHash": "QmUaJbK8MA37bQpC9yMvzjRFFRqGHqLqTE9QHM7cdV2x1A", + "bloomSize": 131114, + "indexHash": "QmSaSCUdJD5gsudNa2GU1KjpREXTKNXEhXkRQ2GYJTGjnM", + "indexSize": 2390996 + }, + { + "range": "004218668-004219032", + "bloomHash": "QmVorr116oBLYeaujpR5VMyGkKmJVT8cf3eUmJ9vk9prn2", + "bloomSize": 131114, + "indexHash": "QmU571CsjgcygN1Uyfk9dEKXRx85QshRpx3oDDBxoiEVAt", + "indexSize": 2483920 + }, + { + "range": "004219033-004219372", + "bloomHash": "QmStd29Q2HTbZSXc1Fu5Zo6fvcwHAGNBYgNmga3oxhjNmu", + "bloomSize": 131114, + "indexHash": "QmdkPzMrPGGcwqnJjADScMAAfx6gxPaUQbzhobzxiCmKKh", + "indexSize": 2338352 + }, + { + "range": "004219373-004219699", + "bloomHash": "QmVyb254qpPCFgPEpKLNrYdCKRZvWdCdEZ2obEstyZHF1E", + "bloomSize": 131114, + "indexHash": "QmP4U7L1EvxPgzP1ZqfH2YMqp1NmAmMhHLJjfwdcQzwZYT", + "indexSize": 2278072 + }, + { + "range": "004219700-004220037", + "bloomHash": "Qmcn2iFnoSrRXntPDLgYZDbdRkrm75sG81rQaPGfj7EPsK", + "bloomSize": 131114, + "indexHash": "QmU5kd99fvrshqbED3LYWYgoLRMxtKL33ovH2GwgzPapsk", + "indexSize": 2251128 + }, + { + "range": "004220038-004220358", + "bloomHash": "QmZ3jKZELimy321JBqTuzkU9J3ygtUyH5acQpVA9kpyzHc", + "bloomSize": 131114, + "indexHash": "QmeiweMqP7ennGkhv6wYh4JVhydpMfxtEbeWykHcAQEz6T", + "indexSize": 2255980 + }, + { + "range": "004220359-004220672", + "bloomHash": "QmUhNkjKgNrU5uC9gGxfGKvDWBNzoQ9FoQLzVYcw6Yffkt", + "bloomSize": 131114, + "indexHash": "QmTp16ECPfMoTeUpNhQjZFN8uVhHiwNqpCPUthqNRKxRdG", + "indexSize": 2261596 + }, + { + "range": "004220673-004221005", + "bloomHash": "QmebD9CSKqeqthL5DUUjrfCHK3BTRyX4JiNvH3YZfeXdNF", + "bloomSize": 131114, + "indexHash": "QmTEMhq5FKA3M2ZmkqMMrjbYaqnumqu23vzX4Y81XzRdgQ", + "indexSize": 2283088 + }, + { + "range": "004221006-004221332", + "bloomHash": "QmbH9LNUdm9PKCe7LhXAxrvYwgvuXftZ5Pc4NmREh35JZw", + "bloomSize": 131114, + "indexHash": "Qmf16Ubj1vp6goj1ViaZY2LbDVHip3R5FqhVFvibN9QBLb", + "indexSize": 2271328 + }, + { + "range": "004221333-004221646", + "bloomHash": "QmdYsR5sCAPFnkXMU5BJepGKQq3Vd8qACXBM6wK5NBSNtc", + "bloomSize": 131114, + "indexHash": "QmeDGJYog2Y91c7J9AipeF8iPBgndCkDLfZF7R791x35a3", + "indexSize": 2160608 + }, + { + "range": "004221647-004221922", + "bloomHash": "QmSz4P3pj19zB5zcLBPiCYKLS6pX5Wk1XhAdsBmBRwx457", + "bloomSize": 131114, + "indexHash": "QmagqSrosrbLFUZ93YgLERAVzHDH7bNF9bQV1KpPDz5zPS", + "indexSize": 2194508 + }, + { + "range": "004221923-004222214", + "bloomHash": "QmUvZjKZMbceCzLjN9n8nyNqxDBt7NhpR6Bym5eK5aEA8Q", + "bloomSize": 131114, + "indexHash": "QmSDqc3f9WWyRDgpv6qhxZYDiDn5mrF2QarnzfQvbmzp7U", + "indexSize": 2240888 + }, + { + "range": "004222215-004222485", + "bloomHash": "QmT8adYHevA28K3ZM9qR4FmsMbCSDHriDWsWBJ84SKcy3B", + "bloomSize": 131114, + "indexHash": "QmQRMehW91RpFi6q4Ff8S6Aw1fuYKLh3oAty9zC7tpHWEe", + "indexSize": 2325264 + }, + { + "range": "004222486-004222751", + "bloomHash": "QmZBV9FyQ6obVJnqhy7zVMijgqmvA6zmdwG26zRYM8tRjg", + "bloomSize": 131114, + "indexHash": "QmT65vDkVG8yhXGCLfQXuCZdFJvhBuH6GykhHebWUxDpSp", + "indexSize": 2362948 + }, + { + "range": "004222752-004223029", + "bloomHash": "QmcFzBTRFZBe7nLfvFHuQBZaBNev82CjD8VT53QYACRsMR", + "bloomSize": 131114, + "indexHash": "Qmcb3mZtRrV8d8HvvxL7vGX6YenBMMgGLSCqAhxETeatsp", + "indexSize": 2342564 + }, + { + "range": "004223030-004223322", + "bloomHash": "QmXNg9myFACTxUqvqxBL6JeWRqC5mAxExD5qtCMoLYZjAP", + "bloomSize": 131114, + "indexHash": "QmctLCicbY4RnVJ1EHFvqL2o6azLa7NrLHRaiHcRpqWy7K", + "indexSize": 2294752 + }, + { + "range": "004223323-004223584", + "bloomHash": "QmdhosDi1zFtuscgDPZKCt9LWQTnT1bANmuSm4jd2do6NV", + "bloomSize": 131114, + "indexHash": "QmcFZaVK3HqgaxmLTnae13fAf1X3amKUZsNFCw5WSAZavv", + "indexSize": 2214512 + }, + { + "range": "004223585-004223828", + "bloomHash": "QmUH2x7eBXxRbeM4KCHupFem2TAQC1Us9sV9vn1iDKNadc", + "bloomSize": 131114, + "indexHash": "QmPFmjQnCUQCwfLUGSeYWZiNkbhP8tZSXV67EXSJ6L7mKZ", + "indexSize": 2280532 + }, + { + "range": "004223829-004224111", + "bloomHash": "QmS1uN1UC5JRnD7UeApwE8zPCy4Us7gmSVY3FwmERMsHJs", + "bloomSize": 131114, + "indexHash": "Qmcs8QwEwxNdLLsnnFA2FSUBfLfvV5CGhkYzPXb8QpX6Ce", + "indexSize": 2109888 + }, + { + "range": "004224112-004224442", + "bloomHash": "QmZqDUKfTU3kpTKD1YcrJfnqi8JstAsumQvRgNHUDSD4P4", + "bloomSize": 131114, + "indexHash": "QmR4BMqNqxvFDYnpmtV9FLqVqoajTZJvw3XrFZkip3qPYj", + "indexSize": 2353256 + }, + { + "range": "004224443-004224799", + "bloomHash": "QmS8fZ7iZQsftL128ztyHBLCMvpDmvzBmRqLyoRzQXs4Cx", + "bloomSize": 131114, + "indexHash": "QmfWiZdb6N359qe7s5mVvn1SdZoRuKC1qqmjmHwGCALJPx", + "indexSize": 2328720 + }, + { + "range": "004224800-004225118", + "bloomHash": "QmQMeDnA9F2oqZLmy6RVRgKCZ5Ft5GMnSbjvofwyy9AgBi", + "bloomSize": 131114, + "indexHash": "QmVC9biswFiGkST5wFLDiXLLX285YQ9GBSz3Dj1UkZF7ZR", + "indexSize": 2411944 + }, + { + "range": "004225119-004225446", + "bloomHash": "QmNiaEUK7oSrDt4qSRnHhziG2NC4ZYtcQ4bKduxibtS86B", + "bloomSize": 131114, + "indexHash": "QmcFnVJWVBZfjjBdjqU7jMUHGvwo7oRmiQw8D3iqTp4AFN", + "indexSize": 2397872 + }, + { + "range": "004225447-004225785", + "bloomHash": "QmUNWje28UtFWSc5EMqAntGPyHmU4R1SqLPSGgbeSoAQBU", + "bloomSize": 131114, + "indexHash": "QmWjzk5F7189QVaD38Raxetdkn7PTJYJobLiZZHwwhfawc", + "indexSize": 2439656 + }, + { + "range": "004225786-004226125", + "bloomHash": "QmbSVrJ2xERyUmVSXYMx3meNdDoH8GzZqdPMcWjWxcLKgh", + "bloomSize": 131114, + "indexHash": "Qmcur1XLzsYBjTCPNfQ5pRDZLum9wqH24z7BDzFF5r5Mxy", + "indexSize": 2244516 + }, + { + "range": "004226126-004226440", + "bloomHash": "QmVTapkPQwS5gUqr227ioKhrAQPBnCm5GAAkeMt1Qrc4FU", + "bloomSize": 131114, + "indexHash": "QmVFpJQte7z4u6gVyFdFxApebjqmsN29emftL3AdmvpFRL", + "indexSize": 2232692 + }, + { + "range": "004226441-004226782", + "bloomHash": "QmadmWR2Kwo67CQzUbB8Tsstp2Vpdh7k7Z2mqAYUZX4z82", + "bloomSize": 131114, + "indexHash": "QmPtohVVbRnx8zczoQNRvm6sF7foY2HdEiVdvUM8h2jNBo", + "indexSize": 2473760 + }, + { + "range": "004226783-004227139", + "bloomHash": "QmRyMckk6SX4Vzjoa9Kq4zUTitjPqPqZ4afvJoov2B6Qtz", + "bloomSize": 131114, + "indexHash": "QmWZoMroADGfnqtJVMyP6Ke7DVhaKN7hxa4WzqcP7gndAg", + "indexSize": 2427844 + }, + { + "range": "004227140-004227428", + "bloomHash": "Qme367vYyWPfoP7PJ7QPx5T4qBjzcRFnAMpxNi1QQm4yb9", + "bloomSize": 131114, + "indexHash": "QmUDPgd1rohEo1Fhh4svy4FNyuHDQgY5DZssVeVFxW27mv", + "indexSize": 2179728 + }, + { + "range": "004227429-004227764", + "bloomHash": "Qmad6npWSjwJ2P9PKXjHa6jA95DJsH67C1NNJc7QLAJnCp", + "bloomSize": 131114, + "indexHash": "QmaRNgyQuYJTorRTZ96to4W7AX5CdsF87iisBwRP1dcDuW", + "indexSize": 1885508 + }, + { + "range": "004227765-004228082", + "bloomHash": "QmPCNk6ntCSxx27utkhEFyGhGGuL7Y2XHG16gNT92YQjh6", + "bloomSize": 131114, + "indexHash": "QmYrKzNvCZJNRwZDsB7xrwuEaJf8pm8UZrXPS32pdx27EJ", + "indexSize": 1933584 + }, + { + "range": "004228083-004228399", + "bloomHash": "QmVLwQPfhYkvhGbEgqqprCPxxkzUmzYk1XjNnqMJzccc72", + "bloomSize": 131114, + "indexHash": "Qmf3jUwBst6WefEoQBGqWUHwbsY5iArJi7NmiD7DeVSiqX", + "indexSize": 1986672 + }, + { + "range": "004228400-004228712", + "bloomHash": "Qmc8wTW9631QgbHF3TZ6DwrByMVVxqG5WkRFPKkMjWWj89", + "bloomSize": 131114, + "indexHash": "QmdGoNuJaNzub8vcWjiedWTqQw6YD3ECbCu2gky7v2gvij", + "indexSize": 2128100 + }, + { + "range": "004228713-004228998", + "bloomHash": "QmQrDPxyGQmvyy55i5MVnbmy23nNWcHJkWxgwun4ZoWprn", + "bloomSize": 131114, + "indexHash": "QmdAYcQi18eTqwfMQaE3mxWy63Ay97jGME1Hdedx3QkR9H", + "indexSize": 2112740 + }, + { + "range": "004228999-004229325", + "bloomHash": "QmfKjyUgy2wqtmpxXW9vY3KsJLPSxNdHxPY935kxZA1TZZ", + "bloomSize": 131114, + "indexHash": "QmVZoHVWo5QLTuifuuP9uvA9ov5iUHxiCikqPz15cm6PTc", + "indexSize": 2120272 + }, + { + "range": "004229326-004229626", + "bloomHash": "QmYm6FzJ2NxQy9PWAPucYK3zQJBVYpvy1Qs7GUuasiYXEV", + "bloomSize": 131114, + "indexHash": "QmbaDsff5BRbviJtn8EPgR56RgMiX3PitqRfTZdRpkCKqZ", + "indexSize": 2088080 + }, + { + "range": "004229627-004229954", + "bloomHash": "QmWcusjZVcFGZkDARCKboxw3zw8nLGjtgzow8qc24BM1F6", + "bloomSize": 131114, + "indexHash": "QmbgctFcJJn64QvGC71N8V4fA7bazUrFnVeKmVNNkJot8V", + "indexSize": 2090540 + }, + { + "range": "004229955-004230260", + "bloomHash": "QmQuzYgSqGLk9DPcjkkF5VSR3S3YVABqhFfA2gde8fa45z", + "bloomSize": 131114, + "indexHash": "QmR7PhCDypi8k736b7TpTRm7K91TQU6TbQqRcKzGwQ4aaF", + "indexSize": 2048988 + }, + { + "range": "004230261-004230574", + "bloomHash": "QmfDi9cnvHgL8J2GAxQcr1ARmszF547qRFt6eyXXc98nEP", + "bloomSize": 131114, + "indexHash": "QmRP5pSk9g9rzYSB2C5RrD67TefAFJX7oBjdpc7pmnUfwi", + "indexSize": 2092568 + }, + { + "range": "004230575-004230901", + "bloomHash": "QmcufFQQMHxxhFVKPGfZQzX5nfCk3s8voGd3vYC3RAkpDd", + "bloomSize": 131114, + "indexHash": "QmQpZKwPBKT5v3myARTYBWdW24amwtWEKBNHWq7tWbbGQK", + "indexSize": 2531292 + }, + { + "range": "004230902-004231060", + "bloomHash": "QmUmutQVFErk4VeXPK91PeRjjZhtd4po5nRyhLJdqQRGqT", + "bloomSize": 131114, + "indexHash": "Qmau6X7EJJSyEHGXUU5SXyHYUtKBFf6EZ1UMa1CD4gkFbQ", + "indexSize": 2893468 + }, + { + "range": "004231061-004231458", + "bloomHash": "QmRXyiCBy5qMA3wPwMDqmyRKvXomk9r5vfVsryFTbFGqdK", + "bloomSize": 131114, + "indexHash": "QmZMeEA9piJEKg9DZkkDHhmGN5hnNrqZrGm6X4Ei6YRWkb", + "indexSize": 2222904 + }, + { + "range": "004231459-004231870", + "bloomHash": "QmaZLWFKZarHZdo3D6fFcYVqbd1Ud3hNfkgA3fJsZXdo3V", + "bloomSize": 131114, + "indexHash": "QmboxTL7c1DUCXvVVR2AWGY6qx25vetTRqsFNurzEFUTWo", + "indexSize": 2014288 + }, + { + "range": "004231871-004232367", + "bloomHash": "QmNeRnuH8tPxBDz9HC5oKZMUEgorKfofTRb31e3TzPTV6m", + "bloomSize": 131114, + "indexHash": "QmTX6bgZKrxVVGxW4SxmhCrx3w82wJnvGBrRzZFVy3Dx7f", + "indexSize": 2500628 + }, + { + "range": "004232368-004232795", + "bloomHash": "QmSC7HnHRCVwD8PYjBVEvg1AApg9SfMQW6gSdAbe86E55H", + "bloomSize": 131114, + "indexHash": "QmQCSLMFPRHGHzfUYQy5JZADJuLMjh54QbNfJCsdVvdHYi", + "indexSize": 2444776 + }, + { + "range": "004232796-004233160", + "bloomHash": "QmXNmZ3m78DstTiwJ5wabTdBpJXiYaMV5UafgNA2DMovun", + "bloomSize": 131114, + "indexHash": "Qma8U59zBJ8AEt9GwYdtsxGsKpkjEjEoL6KMTZpsQoJPu8", + "indexSize": 2480976 + }, + { + "range": "004233161-004233599", + "bloomHash": "QmNSnLyY4rfu49KTKzG5MDGBk1DxNjdKvdbdCjLhTZiBzs", + "bloomSize": 131114, + "indexHash": "QmcorAqwo3igCbzbprRZTdZEGM7RFbhESuPR6WBGS83Dx4", + "indexSize": 2657656 + }, + { + "range": "004233600-004234000", + "bloomHash": "QmT5ZnwgH49ZV9JtGjp8sLi1MQg8JtEA9ZYqjDd2VWpAk2", + "bloomSize": 131114, + "indexHash": "QmZN86Wids2rcquPeX6VdkRMc5BZTz8JpbCLZRvGdFmGcu", + "indexSize": 2669396 + }, + { + "range": "004234001-004234340", + "bloomHash": "QmUtR8qTYeh56MNwk3UiWDBhE4zS4htns4vYQofGLeSX7n", + "bloomSize": 131114, + "indexHash": "QmPkS7ikLxcqRFngkpQeB7DuotZXodco7EQE72o5imAszK", + "indexSize": 2534296 + }, + { + "range": "004234341-004234721", + "bloomHash": "QmRonME8rGb2A9TRAdU7mvwX6WXj3QMsSHEp4V8b9xS4VK", + "bloomSize": 131114, + "indexHash": "QmUWuvzDwqhHFMuEoJ7EC7SAyct9SwzwujcprGs3d3ksbc", + "indexSize": 2323372 + }, + { + "range": "004234722-004235149", + "bloomHash": "QmQ7bsuWC2Bu3cmAvjJ8ySWgjsnwMJiPxpve1xrzp6JR6p", + "bloomSize": 131114, + "indexHash": "QmQeczbFGmTudADmsLdZ8Utet4wZMm85RgjtRVxaJKvLzQ", + "indexSize": 2360060 + }, + { + "range": "004235150-004235592", + "bloomHash": "QmZYvY6tjvFZrEDza8MCNXAd54WNZYVU3yv6iM3NL6sADV", + "bloomSize": 131114, + "indexHash": "QmUJ8Kawmx52AyLtooDfjT23pf99xwyyHKCs58mS2uTaJE", + "indexSize": 2837112 + }, + { + "range": "004235593-004236058", + "bloomHash": "QmR6d8FA6TzMggGmUaDk1KA1P5AJCYZRSo7DdgdTpUGRdh", + "bloomSize": 131114, + "indexHash": "QmR2mZjU4bXDpTRnnCHt5t23RAWbt9VQvb1cAWop6poSeQ", + "indexSize": 2805392 + }, + { + "range": "004236059-004236511", + "bloomHash": "QmczNbe6vhpek8tyvrKQREgCgQYX9aUM7p8PvwDPupadiE", + "bloomSize": 131114, + "indexHash": "QmWE5asUr5MpueaRXRnX32FwDzAhpXXYUtBiPhEkLAtVKN", + "indexSize": 2577016 + }, + { + "range": "004236512-004237003", + "bloomHash": "QmPJs7FGHzz8zN5hVE2m1rpNMDvKn7tQuawXjbR7WM2zHF", + "bloomSize": 262190, + "indexHash": "QmchqQxifdr64uvk9DhUbnyjYaawAob84fDfx74pZZvouh", + "indexSize": 3077156 + }, + { + "range": "004237004-004237510", + "bloomHash": "QmZNKacp9bEJyrqMKrDJ1MfiwcHz8B1JSHsuAkPC9wo6uU", + "bloomSize": 131114, + "indexHash": "QmNzNkYgadzYTTdn953cDr7yfVwh2deGx23SDUdew5x12E", + "indexSize": 2564516 + }, + { + "range": "004237511-004238205", + "bloomHash": "QmRrcVSFTLGWRpX15VSGxHaqhPwSQQsVd1aDLTTJ2egWLk", + "bloomSize": 131114, + "indexHash": "QmeRd4ag7h5sbXHdvEMs3RKix7X56FuwVnQqvSABXqrKKm", + "indexSize": 2477748 + }, + { + "range": "004238206-004238810", + "bloomHash": "QmUn8QgN2aBKWXwAwXgieraJMp2Ns8rhtQ9R5PCQ7LRbQW", + "bloomSize": 131114, + "indexHash": "QmbUCB8rmrcyysEPyzJhxRaRQ7q9eRbXTwMF2yDdJWF7jK", + "indexSize": 2102624 } ] } \ No newline at end of file From e66f7e63ff4cb73b066ded9ee929ebdb95299fe7 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Thu, 7 Sep 2023 05:59:37 +0100 Subject: [PATCH 18/33] Fixes tests --- src/apps/chifra/pkg/tslib/list_integration_test.go | 6 ++---- src/apps/chifra/pkg/tslib/list_test.go | 2 +- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/src/apps/chifra/pkg/tslib/list_integration_test.go b/src/apps/chifra/pkg/tslib/list_integration_test.go index a5ac629e9d..abce2a05f9 100644 --- a/src/apps/chifra/pkg/tslib/list_integration_test.go +++ b/src/apps/chifra/pkg/tslib/list_integration_test.go @@ -1,6 +1,3 @@ -//go:build integration -// +build integration - // Copyright 2021 The TrueBlocks Authors. All rights reserved. // Use of this source code is governed by a license that can // be found in the LICENSE file. @@ -38,7 +35,8 @@ func TestGetSpecials2(t *testing.T) { t.Error(err) } - if len(specials) != 39 { + if len(specials) != 42 { + fmt.Println(specials) t.Error("Wrong number of special blocks ", len(specials), ". Should have 39.") } diff --git a/src/apps/chifra/pkg/tslib/list_test.go b/src/apps/chifra/pkg/tslib/list_test.go index 4efe7c0a66..1bc93235ca 100644 --- a/src/apps/chifra/pkg/tslib/list_test.go +++ b/src/apps/chifra/pkg/tslib/list_test.go @@ -73,7 +73,7 @@ func TestGetSpecials(t *testing.T) { t.Error(err) } - if len(specials) != 39 { + if len(specials) != 42 { t.Error("Wrong number of special blocks ", len(specials), ". Should have 39.") } From 709b181524f6e51824e6e818f2a0b68b39dc3844 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Thu, 7 Sep 2023 06:07:21 +0100 Subject: [PATCH 19/33] Fixes tests --- src/apps/chifra/pkg/tslib/list_integration_test.go | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/apps/chifra/pkg/tslib/list_integration_test.go b/src/apps/chifra/pkg/tslib/list_integration_test.go index abce2a05f9..dd5521610d 100644 --- a/src/apps/chifra/pkg/tslib/list_integration_test.go +++ b/src/apps/chifra/pkg/tslib/list_integration_test.go @@ -1,3 +1,6 @@ +//go:build integration +// +build integration + // Copyright 2021 The TrueBlocks Authors. All rights reserved. // Use of this source code is governed by a license that can // be found in the LICENSE file. @@ -36,7 +39,6 @@ func TestGetSpecials2(t *testing.T) { } if len(specials) != 42 { - fmt.Println(specials) t.Error("Wrong number of special blocks ", len(specials), ". Should have 39.") } From 889e435abc54da5a49d87ae5eee8e3c744936331 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Fri, 8 Sep 2023 06:21:44 +0200 Subject: [PATCH 20/33] Fixing chifra chunks manifest --pin --- src/apps/chifra/internal/chunks/handle_pin.go | 29 ++++++++++++------- src/apps/chifra/internal/chunks/validate.go | 4 +-- src/apps/chifra/pkg/pinning/pin_file.go | 2 +- .../chifra/pkg/types/types_chunkrecord.go | 7 +++++ 4 files changed, 29 insertions(+), 13 deletions(-) diff --git a/src/apps/chifra/internal/chunks/handle_pin.go b/src/apps/chifra/internal/chunks/handle_pin.go index edb4a92447..2dbde253b9 100644 --- a/src/apps/chifra/internal/chunks/handle_pin.go +++ b/src/apps/chifra/internal/chunks/handle_pin.go @@ -20,12 +20,12 @@ import ( ) func (opts *ChunksOptions) HandlePin(blockNums []uint64) error { - firstBlock := mustParseUint(os.Getenv("TB_CHUNK_PIN_FIRST_BLOCK")) + firstBlock := mustParseUint(os.Getenv("TB_CHUNKS_PINFIRSTBLOCK")) chain := opts.Globals.Chain ctx, cancel := context.WithCancel(context.Background()) fetchData := func(modelChan chan types.Modeler[types.RawModeler], errorChan chan error) { - man := simpleChunkPinReport{ + report := simpleChunkPinReport{ Version: version.ManifestVersion, Chain: chain, Schemas: unchained.Schemas, @@ -34,14 +34,14 @@ func (opts *ChunksOptions) HandlePin(blockNums []uint64) error { if len(blockNums) == 0 { var err error tsPath := config.GetPathToIndex(chain) + "ts.bin" - if man.TsHash, err = pinning.PinItem(chain, "timestamps", tsPath, opts.Remote); err != nil { + if report.TsHash, err = pinning.PinItem(chain, "timestamps", tsPath, opts.Remote); err != nil { errorChan <- err cancel() return } manPath := config.GetPathToChainConfig(chain) + "manifest.json" - if man.ManifestHash, err = pinning.PinItem(chain, "manifest", manPath, opts.Remote); err != nil { + if report.ManifestHash, err = pinning.PinItem(chain, "manifest", manPath, opts.Remote); err != nil { errorChan <- err cancel() return @@ -65,17 +65,19 @@ func (opts *ChunksOptions) HandlePin(blockNums []uint64) error { result, err := pinning.PinChunk(chain, index.ToBloomPath(path), index.ToIndexPath(path), opts.Remote) if err != nil { - return false, err + errorChan <- err + cancel() // keep going... + return true, nil } if pinning.LocalDaemonRunning() { - man.Pinned = append(man.Pinned, result.Local.BloomHash) - man.Pinned = append(man.Pinned, result.Local.IndexHash) + report.Pinned = append(report.Pinned, result.Local.BloomHash) + report.Pinned = append(report.Pinned, result.Local.IndexHash) } if opts.Remote { - man.Pinned = append(man.Pinned, result.Remote.BloomHash) - man.Pinned = append(man.Pinned, result.Remote.IndexHash) + report.Pinned = append(report.Pinned, result.Remote.BloomHash) + report.Pinned = append(report.Pinned, result.Remote.IndexHash) } if !result.Matches { @@ -86,6 +88,13 @@ func (opts *ChunksOptions) HandlePin(blockNums []uint64) error { } else if opts.Remote && pinning.LocalDaemonRunning() { logger.Info(colors.BrightGreen+"Matches: "+result.Remote.BloomHash.String(), "-", result.Remote.IndexHash, colors.Off) } + if opts.Globals.Verbose { + if opts.Remote { + fmt.Println("result.Remote:", result.Remote.String()) + } else { + fmt.Println("result.Local:", result.Local.String()) + } + } sleep := opts.Sleep if sleep > 0 { @@ -113,7 +122,7 @@ func (opts *ChunksOptions) HandlePin(blockNums []uint64) error { } } - modelChan <- &man + modelChan <- &report } return output.StreamMany(ctx, fetchData, opts.Globals.OutputOpts()) diff --git a/src/apps/chifra/internal/chunks/validate.go b/src/apps/chifra/internal/chunks/validate.go index 91fe9bcc0e..50590c68fd 100644 --- a/src/apps/chifra/internal/chunks/validate.go +++ b/src/apps/chifra/internal/chunks/validate.go @@ -72,8 +72,8 @@ func (opts *ChunksOptions) validateChunks() error { if opts.Mode == "index" { // do nothing } else if opts.Mode == "manifest" { - if !pinning.LocalDaemonRunning() { - return validate.Usage("The {0} option requires {1}.", "manifest --deep", "a locally running IPFS daemon") + if !opts.Remote && !pinning.LocalDaemonRunning() { + return validate.Usage("The {0} option requires {1}.", "manifest --deep", "a locally running IPFS daemon or --remote") } } else { return validate.Usage("The {0} option requires mode {1}.", "--deep", "index or manifest") diff --git a/src/apps/chifra/pkg/pinning/pin_file.go b/src/apps/chifra/pkg/pinning/pin_file.go index 1383963166..701ca70a5c 100644 --- a/src/apps/chifra/pkg/pinning/pin_file.go +++ b/src/apps/chifra/pkg/pinning/pin_file.go @@ -46,7 +46,7 @@ func PinItem(chain string, dbName, path string, isRemote bool) (hash base.IpfsHa if isRemote { remoteService, _ := NewPinningService(chain, Pinata) var remoteHash base.IpfsHash - if remoteHash, err = remoteService.PinFile(path, true); err != nil { + if remoteHash, err = remoteService.PinFile(path, false); err != nil { err = fmt.Errorf("error pinning remotely: %s", err) return } diff --git a/src/apps/chifra/pkg/types/types_chunkrecord.go b/src/apps/chifra/pkg/types/types_chunkrecord.go index dc224f9d00..2401ac2881 100644 --- a/src/apps/chifra/pkg/types/types_chunkrecord.go +++ b/src/apps/chifra/pkg/types/types_chunkrecord.go @@ -10,6 +10,8 @@ package types // EXISTING_CODE import ( + "encoding/json" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" ) @@ -72,4 +74,9 @@ func (s *SimpleChunkRecord) Model(verbose bool, format string, extraOptions map[ } // EXISTING_CODE +func (s *SimpleChunkRecord) String() string { + bytes, _ := json.MarshalIndent(s, "", " ") + return string(bytes) +} + // EXISTING_CODE From 35a61397bc4acd00e9eeda82878e63206165f8ee Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Fri, 8 Sep 2023 06:43:25 +0200 Subject: [PATCH 21/33] Cleaning --- src/apps/chifra/pkg/pinning/fetch.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/apps/chifra/pkg/pinning/fetch.go b/src/apps/chifra/pkg/pinning/fetch.go index 3c715fac33..29d3e74d08 100644 --- a/src/apps/chifra/pkg/pinning/fetch.go +++ b/src/apps/chifra/pkg/pinning/fetch.go @@ -33,7 +33,7 @@ func FetchFromGateway(ctx context.Context, gateway, hash string) (*FetchResult, } if response.StatusCode != 200 { // logger.Fatalln("DefaultClient.Do returned StatusCode not equal to 200 in FetFromGateway with", url) - return nil, fmt.Errorf("url %s returned status code: %d", url, response.StatusCode) + return nil, fmt.Errorf("fetch to %s returned status code: %d", url, response.StatusCode) } body := response.Body From b0a8e58502ff74e85b3e3a413c242d6408ca6a57 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Fri, 8 Sep 2023 08:39:22 +0200 Subject: [PATCH 22/33] Adds the --publisher option to chifra init and chifra chunks but does not implement it. --- CHANGES.md | 5 +++-- docs/content/api/openapi.yaml | 16 ++++++++++++++++ docs/content/chifra/admin.md | 3 +++ docs/readmes/admin-chunks.md | 2 ++ docs/readmes/admin-init.md | 1 + sdk/python/src/_chunks.py | 1 + sdk/python/src/_init.py | 1 + sdk/typescript/src/paths/chunks.ts | 1 + sdk/typescript/src/paths/init.ts | 1 + src/apps/chifra/cmd/chunks.go | 4 +++- src/apps/chifra/cmd/init.go | 1 + src/apps/chifra/internal/chunks/README.md | 2 ++ src/apps/chifra/internal/chunks/options.go | 5 +++++ src/apps/chifra/internal/init/README.md | 1 + src/apps/chifra/internal/init/options.go | 8 +++++++- src/cmd-line-options.csv | 5 ++++- src/dev_tools/makeClass/results/report.csv | 6 ++++++ 17 files changed, 58 insertions(+), 5 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 77f55c1698..baac10e4cb 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -13,12 +13,13 @@ Removes a number of previously deprecated options. `chifra abis --sol`, `chifra Better error reporting when running against non-tracing nodes. Fixed an issue with Content-Type in the server. Fixed an issue where user could hit cntl+c during caching and corrupt the database. -Fixed an issue where scraper missed some smart contract addresses created during out of gas transactions. +Fixed an issue where scraper was missing some smart contract addresses created during out of gas transactions. Better error handling from the RPC. Added a verbose mode to `chifra when` to include more specials and a description for each special block. Now disallows running `chifra scrape` if the node is not a tracing archive node. Near complete re-write of block scraper to fix few bugs and prepare for writing v1.0.0 of the spec. -Add run_count to both the scraper and the monitor --watch code, hides both because they are intended for debugging only. +Add `--run_count` to both the `chifra scrape` and the `chifra monitors --watch`, hides both because they are intended for debugging only. +Added `--publisher` option to `chifra init` and `chifra chunks`. ## v1.0.0 (2023/08/20) diff --git a/docs/content/api/openapi.yaml b/docs/content/api/openapi.yaml index 957d4a27d8..385f804c73 100644 --- a/docs/content/api/openapi.yaml +++ b/docs/content/api/openapi.yaml @@ -2068,6 +2068,14 @@ paths: explode: true schema: type: boolean + - name: publisher + description: for some query options, the publisher of the index + required: false + style: form + in: query + explode: true + schema: + type: string - name: remote description: > prior to processing, retreive the manifest from the Unchained Index smart contract @@ -2198,6 +2206,14 @@ paths: explode: true schema: type: boolean + - name: publisher + description: the publisher of the index to download + required: false + style: form + in: query + explode: true + schema: + type: string - name: firstBlock description: do not download any chunks earlier than this block required: false diff --git a/docs/content/chifra/admin.md b/docs/content/chifra/admin.md index e87b873dae..a9da631fd2 100644 --- a/docs/content/chifra/admin.md +++ b/docs/content/chifra/admin.md @@ -282,6 +282,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk -F, --first_block uint first block to process (inclusive) @@ -302,6 +303,7 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. ``` Data models produced by this tool: @@ -355,6 +357,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/docs/readmes/admin-chunks.md b/docs/readmes/admin-chunks.md index c778bff2e7..c92be6f2a5 100644 --- a/docs/readmes/admin-chunks.md +++ b/docs/readmes/admin-chunks.md @@ -23,6 +23,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk -F, --first_block uint first block to process (inclusive) @@ -43,6 +44,7 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. ``` Data models produced by this tool: diff --git a/docs/readmes/admin-init.md b/docs/readmes/admin-init.md index 242991aa2a..690da38195 100644 --- a/docs/readmes/admin-init.md +++ b/docs/readmes/admin-init.md @@ -32,6 +32,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/sdk/python/src/_chunks.py b/sdk/python/src/_chunks.py index 8734de590f..9cc8d7c9b3 100644 --- a/sdk/python/src/_chunks.py +++ b/sdk/python/src/_chunks.py @@ -10,6 +10,7 @@ "check": {"hotkey": "-c", "type": "switch"}, "pin": {"hotkey": "-i", "type": "switch"}, "publish": {"hotkey": "-p", "type": "switch"}, + "publisher": {"hotkey": "-P", "type": "flag"}, "remote": {"hotkey": "-r", "type": "switch"}, "belongs": {"hotkey": "-b", "type": "flag"}, "firstBlock": {"hotkey": "-F", "type": "flag"}, diff --git a/sdk/python/src/_init.py b/sdk/python/src/_init.py index 229c528d2a..4d726b5b16 100644 --- a/sdk/python/src/_init.py +++ b/sdk/python/src/_init.py @@ -9,6 +9,7 @@ initOpts = { "all": {"hotkey": "-a", "type": "switch"}, "dryRun": {"hotkey": "-d", "type": "switch"}, + "publisher": {"hotkey": "-P", "type": "flag"}, "firstBlock": {"hotkey": "-F", "type": "flag"}, "sleep": {"hotkey": "-s", "type": "flag"}, "fmt": {"hotkey": "-x", "type": "flag"}, diff --git a/sdk/typescript/src/paths/chunks.ts b/sdk/typescript/src/paths/chunks.ts index 3c91721a85..d99262a023 100644 --- a/sdk/typescript/src/paths/chunks.ts +++ b/sdk/typescript/src/paths/chunks.ts @@ -13,6 +13,7 @@ export function getChunks( check?: boolean, pin?: boolean, publish?: boolean, + publisher?: string, remote?: boolean, belongs?: address[], firstBlock?: blknum, diff --git a/sdk/typescript/src/paths/init.ts b/sdk/typescript/src/paths/init.ts index 963bbe4ea0..efe8f477dc 100644 --- a/sdk/typescript/src/paths/init.ts +++ b/sdk/typescript/src/paths/init.ts @@ -10,6 +10,7 @@ export function getInit( parameters?: { all?: boolean, dryRun?: boolean, + publisher?: string, firstBlock?: blknum, sleep?: double, chain: string, diff --git a/src/apps/chifra/cmd/chunks.go b/src/apps/chifra/cmd/chunks.go index e893ee6013..ae0161ee93 100644 --- a/src/apps/chifra/cmd/chunks.go +++ b/src/apps/chifra/cmd/chunks.go @@ -57,7 +57,8 @@ Notes: - The --belongs option is only available in the index mode. - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - - The --publish option requires a private key.` + - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher.` func init() { var capabilities = caps.Default // Additional global caps for chifra chunks @@ -69,6 +70,7 @@ func init() { chunksCmd.Flags().BoolVarP(&chunksPkg.GetOptions().Check, "check", "c", false, "check the manifest, index, or blooms for internal consistency") chunksCmd.Flags().BoolVarP(&chunksPkg.GetOptions().Pin, "pin", "i", false, "pin the manifest or each index chunk and bloom") chunksCmd.Flags().BoolVarP(&chunksPkg.GetOptions().Publish, "publish", "p", false, "publish the manifest to the Unchained Index smart contract") + chunksCmd.Flags().StringVarP(&chunksPkg.GetOptions().Publisher, "publisher", "P", "trueblocks.eth", "for some query options, the publisher of the index") chunksCmd.Flags().Uint64VarP(&chunksPkg.GetOptions().Truncate, "truncate", "n", 0, "truncate the entire index at this block (requires a block identifier) (hidden)") chunksCmd.Flags().BoolVarP(&chunksPkg.GetOptions().Remote, "remote", "r", false, "prior to processing, retreive the manifest from the Unchained Index smart contract") chunksCmd.Flags().StringSliceVarP(&chunksPkg.GetOptions().Belongs, "belongs", "b", nil, "in index mode only, checks the address(es) for inclusion in the given index chunk") diff --git a/src/apps/chifra/cmd/init.go b/src/apps/chifra/cmd/init.go index b814c534bf..e46af2c81c 100644 --- a/src/apps/chifra/cmd/init.go +++ b/src/apps/chifra/cmd/init.go @@ -63,6 +63,7 @@ func init() { initCmd.Flags().BoolVarP(&initPkg.GetOptions().All, "all", "a", false, "in addition to Bloom filters, download full index chunks (recommended)") initCmd.Flags().BoolVarP(&initPkg.GetOptions().DryRun, "dry_run", "d", false, "display the results of the download without actually downloading") + initCmd.Flags().StringVarP(&initPkg.GetOptions().Publisher, "publisher", "P", "trueblocks.eth", "the publisher of the index to download") initCmd.Flags().Uint64VarP(&initPkg.GetOptions().FirstBlock, "first_block", "F", 0, "do not download any chunks earlier than this block") initCmd.Flags().Float64VarP(&initPkg.GetOptions().Sleep, "sleep", "s", 0.0, "seconds to sleep between downloads") globals.InitGlobals(initCmd, &initPkg.GetOptions().Globals, capabilities) diff --git a/src/apps/chifra/internal/chunks/README.md b/src/apps/chifra/internal/chunks/README.md index ca8daf4f7b..ec4769da52 100644 --- a/src/apps/chifra/internal/chunks/README.md +++ b/src/apps/chifra/internal/chunks/README.md @@ -23,6 +23,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk -F, --first_block uint first block to process (inclusive) @@ -43,6 +44,7 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. ``` Data models produced by this tool: diff --git a/src/apps/chifra/internal/chunks/options.go b/src/apps/chifra/internal/chunks/options.go index 001ebc0a3d..c1da128909 100644 --- a/src/apps/chifra/internal/chunks/options.go +++ b/src/apps/chifra/internal/chunks/options.go @@ -30,6 +30,7 @@ type ChunksOptions struct { Check bool `json:"check,omitempty"` // Check the manifest, index, or blooms for internal consistency Pin bool `json:"pin,omitempty"` // Pin the manifest or each index chunk and bloom Publish bool `json:"publish,omitempty"` // Publish the manifest to the Unchained Index smart contract + Publisher string `json:"publisher,omitempty"` // For some query options, the publisher of the index Truncate uint64 `json:"truncate,omitempty"` // Truncate the entire index at this block (requires a block identifier) Remote bool `json:"remote,omitempty"` // Prior to processing, retreive the manifest from the Unchained Index smart contract Belongs []string `json:"belongs,omitempty"` // In index mode only, checks the address(es) for inclusion in the given index chunk @@ -46,6 +47,7 @@ type ChunksOptions struct { } var defaultChunksOptions = ChunksOptions{ + Publisher: "trueblocks.eth", Truncate: utils.NOPOS, LastBlock: utils.NOPOS, MaxAddrs: utils.NOPOS, @@ -58,6 +60,7 @@ func (opts *ChunksOptions) testLog() { logger.TestLog(opts.Check, "Check: ", opts.Check) logger.TestLog(opts.Pin, "Pin: ", opts.Pin) logger.TestLog(opts.Publish, "Publish: ", opts.Publish) + logger.TestLog(len(opts.Publisher) > 0, "Publisher: ", opts.Publisher) logger.TestLog(opts.Truncate != utils.NOPOS, "Truncate: ", opts.Truncate) logger.TestLog(opts.Remote, "Remote: ", opts.Remote) logger.TestLog(len(opts.Belongs) > 0, "Belongs: ", opts.Belongs) @@ -100,6 +103,8 @@ func chunksFinishParseApi(w http.ResponseWriter, r *http.Request) *ChunksOptions opts.Pin = true case "publish": opts.Publish = true + case "publisher": + opts.Publisher = value[0] case "truncate": opts.Truncate = globals.ToUint64(value[0]) case "remote": diff --git a/src/apps/chifra/internal/init/README.md b/src/apps/chifra/internal/init/README.md index 31c4427132..0ac26629b8 100644 --- a/src/apps/chifra/internal/init/README.md +++ b/src/apps/chifra/internal/init/README.md @@ -32,6 +32,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/src/apps/chifra/internal/init/options.go b/src/apps/chifra/internal/init/options.go index bcb563f0d1..9513b4f8a0 100644 --- a/src/apps/chifra/internal/init/options.go +++ b/src/apps/chifra/internal/init/options.go @@ -22,6 +22,7 @@ import ( type InitOptions struct { All bool `json:"all,omitempty"` // In addition to Bloom filters, download full index chunks (recommended) DryRun bool `json:"dryRun,omitempty"` // Display the results of the download without actually downloading + Publisher string `json:"publisher,omitempty"` // The publisher of the index to download FirstBlock uint64 `json:"firstBlock,omitempty"` // Do not download any chunks earlier than this block Sleep float64 `json:"sleep,omitempty"` // Seconds to sleep between downloads Globals globals.GlobalOptions `json:"globals,omitempty"` // The global options @@ -31,12 +32,15 @@ type InitOptions struct { // EXISTING_CODE } -var defaultInitOptions = InitOptions{} +var defaultInitOptions = InitOptions{ + Publisher: "trueblocks.eth", +} // testLog is used only during testing to export the options for this test case. func (opts *InitOptions) testLog() { logger.TestLog(opts.All, "All: ", opts.All) logger.TestLog(opts.DryRun, "DryRun: ", opts.DryRun) + logger.TestLog(len(opts.Publisher) > 0, "Publisher: ", opts.Publisher) logger.TestLog(opts.FirstBlock != 0, "FirstBlock: ", opts.FirstBlock) logger.TestLog(opts.Sleep != float64(0.0), "Sleep: ", opts.Sleep) opts.Conn.TestLog(opts.getCaches()) @@ -61,6 +65,8 @@ func initFinishParseApi(w http.ResponseWriter, r *http.Request) *InitOptions { opts.All = true case "dryRun": opts.DryRun = true + case "publisher": + opts.Publisher = value[0] case "firstBlock": opts.FirstBlock = globals.ToUint64(value[0]) case "sleep": diff --git a/src/cmd-line-options.csv b/src/cmd-line-options.csv index 0efccb2657..2d41df58b4 100644 --- a/src/cmd-line-options.csv +++ b/src/cmd-line-options.csv @@ -259,6 +259,7 @@ num,group,tags,api_route,tool,longName,hotKey,def_val,is_required,is_customizabl 31920,apps,Admin,chunks,chunkMan,check,c,,false,false,true,true,gocmd,switch,,check the manifest, index, or blooms for internal consistency 31935,apps,Admin,chunks,chunkMan,pin,i,,false,false,true,true,gocmd,switch,,pin the manifest or each index chunk and bloom 31940,apps,Admin,chunks,chunkMan,publish,p,,false,false,true,true,gocmd,switch,,publish the manifest to the Unchained Index smart contract +31942,apps,Admin,chunks,chunkMan,publisher,P,trueblocks.eth,false,false,true,true,gocmd,flag,,for some query options, the publisher of the index 31925,apps,Admin,chunks,chunkMan,truncate,n,,false,false,false,false,gocmd,flag,,truncate the entire index at this block (requires a block identifier) 31945,apps,Admin,chunks,chunkMan,remote,r,,false,false,true,true,gocmd,switch,,prior to processing, retreive the manifest from the Unchained Index smart contract 31950,apps,Admin,chunks,chunkMan,belongs,b,,false,false,true,true,gocmd,flag,list,in index mode only, checks the address(es) for inclusion in the given index chunk @@ -276,9 +277,11 @@ num,group,tags,api_route,tool,longName,hotKey,def_val,is_required,is_customizabl 31980,apps,Admin,chunks,chunkMan,n5,,,false,false,false,false,--,note,,The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. 31990,apps,Admin,chunks,chunkMan,n7,,,false,false,false,false,--,note,,The --pin option requires a locally running IPFS node or a pinning service API key. 31995,apps,Admin,chunks,chunkMan,n8,,,false,false,false,false,--,note,,The --publish option requires a private key. +31997,apps,Admin,chunks,chunkMan,n9,,,false,false,false,false,--,note,,The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. 11905,apps,Admin,init,init,all,a,,false,false,true,true,gocmd,switch,,in addition to Bloom filters, download full index chunks (recommended) -11910,apps,Admin,init,init,dry_run,d,,false,false,true,true,gocmd,switch,,display the results of the download without actually downloading +11907,apps,Admin,init,init,dry_run,d,,false,false,true,true,gocmd,switch,,display the results of the download without actually downloading +11910,apps,Admin,init,init,publisher,P,trueblocks.eth,false,false,true,true,gocmd,flag,,the publisher of the index to download 11915,apps,Admin,init,init,first_block,F,0,false,false,true,true,gocmd,flag,,do not download any chunks earlier than this block 11920,apps,Admin,init,init,sleep,s,0.0,false,false,true,true,gocmd,flag,,seconds to sleep between downloads 11925,apps,Admin,init,init,,,,false,false,true,true,--,description,,Initialize the TrueBlocks system by downloading the Unchained Index from IPFS. diff --git a/src/dev_tools/makeClass/results/report.csv b/src/dev_tools/makeClass/results/report.csv index c4041b7b74..53613900cb 100644 --- a/src/dev_tools/makeClass/results/report.csv +++ b/src/dev_tools/makeClass/results/report.csv @@ -145,6 +145,9 @@ chunks,publish,cmds chunks,publish,python chunks,publish,readme chunks,publish,typescript +chunks,publisher,api +chunks,publisher,python +chunks,publisher,typescript chunks,raw,typescript chunks,remote,api chunks,remote,cmds @@ -338,6 +341,9 @@ init,firstBlock,typescript init,first_block,cmds init,first_block,readme init,noHeader,typescript +init,publisher,api +init,publisher,python +init,publisher,typescript init,raw,typescript init,sleep,api init,sleep,cmds From b131ac1bca45d6cf62340945ca71718c15df220a Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Fri, 8 Sep 2023 09:35:22 +0200 Subject: [PATCH 23/33] Fixed auto-code-gen --- sdk/typescript/src/paths/chunks.ts | 2 +- sdk/typescript/src/paths/init.ts | 4 ++-- src/apps/chifra/internal/chunks/options.go | 4 +++- src/apps/chifra/internal/daemon/options.go | 4 ++-- src/apps/chifra/internal/init/options.go | 4 +++- src/cmd-line-options.csv | 4 ++-- src/dev_tools/makeClass/handle_gocmds.cpp | 21 +++++++++++++++++---- 7 files changed, 30 insertions(+), 13 deletions(-) diff --git a/sdk/typescript/src/paths/chunks.ts b/sdk/typescript/src/paths/chunks.ts index d99262a023..cfc14c9521 100644 --- a/sdk/typescript/src/paths/chunks.ts +++ b/sdk/typescript/src/paths/chunks.ts @@ -13,7 +13,7 @@ export function getChunks( check?: boolean, pin?: boolean, publish?: boolean, - publisher?: string, + publisher?: address, remote?: boolean, belongs?: address[], firstBlock?: blknum, diff --git a/sdk/typescript/src/paths/init.ts b/sdk/typescript/src/paths/init.ts index efe8f477dc..51d3d5c5c9 100644 --- a/sdk/typescript/src/paths/init.ts +++ b/sdk/typescript/src/paths/init.ts @@ -4,13 +4,13 @@ * This file was generated with makeClass --sdk. Do not edit it. */ import * as ApiCallers from '../lib/api_callers'; -import { blknum, ChunkRecord, double, Manifest } from '../types'; +import { address, blknum, ChunkRecord, double, Manifest } from '../types'; export function getInit( parameters?: { all?: boolean, dryRun?: boolean, - publisher?: string, + publisher?: address, firstBlock?: blknum, sleep?: double, chain: string, diff --git a/src/apps/chifra/internal/chunks/options.go b/src/apps/chifra/internal/chunks/options.go index c1da128909..715c3592ed 100644 --- a/src/apps/chifra/internal/chunks/options.go +++ b/src/apps/chifra/internal/chunks/options.go @@ -60,7 +60,7 @@ func (opts *ChunksOptions) testLog() { logger.TestLog(opts.Check, "Check: ", opts.Check) logger.TestLog(opts.Pin, "Pin: ", opts.Pin) logger.TestLog(opts.Publish, "Publish: ", opts.Publish) - logger.TestLog(len(opts.Publisher) > 0, "Publisher: ", opts.Publisher) + logger.TestLog(len(opts.Publisher) > 0 && opts.Publisher != "trueblocks.eth", "Publisher: ", opts.Publisher) logger.TestLog(opts.Truncate != utils.NOPOS, "Truncate: ", opts.Truncate) logger.TestLog(opts.Remote, "Remote: ", opts.Remote) logger.TestLog(len(opts.Belongs) > 0, "Belongs: ", opts.Belongs) @@ -131,6 +131,7 @@ func chunksFinishParseApi(w http.ResponseWriter, r *http.Request) *ChunksOptions } } opts.Conn = opts.Globals.FinishParseApi(w, r, opts.getCaches()) + opts.Publisher, _ = opts.Conn.GetEnsAddress(opts.Publisher) // EXISTING_CODE // EXISTING_CODE @@ -157,6 +158,7 @@ func chunksFinishParse(args []string) *ChunksOptions { defFmt := "txt" opts := GetOptions() opts.Conn = opts.Globals.FinishParse(args, opts.getCaches()) + opts.Publisher, _ = opts.Conn.GetEnsAddress(opts.Publisher) // EXISTING_CODE if len(args) > 0 { diff --git a/src/apps/chifra/internal/daemon/options.go b/src/apps/chifra/internal/daemon/options.go index d0bfbdd1a1..d13a988cbc 100644 --- a/src/apps/chifra/internal/daemon/options.go +++ b/src/apps/chifra/internal/daemon/options.go @@ -39,8 +39,8 @@ var defaultDaemonOptions = DaemonOptions{ // testLog is used only during testing to export the options for this test case. func (opts *DaemonOptions) testLog() { - logger.TestLog(len(opts.Port) > 0, "Port: ", opts.Port) - logger.TestLog(len(opts.Api) > 0, "Api: ", opts.Api) + logger.TestLog(len(opts.Port) > 0 && opts.Port != ":8080", "Port: ", opts.Port) + logger.TestLog(len(opts.Api) > 0 && opts.Api != "on", "Api: ", opts.Api) logger.TestLog(len(opts.Scrape) > 0, "Scrape: ", opts.Scrape) logger.TestLog(opts.Monitor, "Monitor: ", opts.Monitor) logger.TestLog(opts.Grpc, "Grpc: ", opts.Grpc) diff --git a/src/apps/chifra/internal/init/options.go b/src/apps/chifra/internal/init/options.go index 9513b4f8a0..7119cc157e 100644 --- a/src/apps/chifra/internal/init/options.go +++ b/src/apps/chifra/internal/init/options.go @@ -40,7 +40,7 @@ var defaultInitOptions = InitOptions{ func (opts *InitOptions) testLog() { logger.TestLog(opts.All, "All: ", opts.All) logger.TestLog(opts.DryRun, "DryRun: ", opts.DryRun) - logger.TestLog(len(opts.Publisher) > 0, "Publisher: ", opts.Publisher) + logger.TestLog(len(opts.Publisher) > 0 && opts.Publisher != "trueblocks.eth", "Publisher: ", opts.Publisher) logger.TestLog(opts.FirstBlock != 0, "FirstBlock: ", opts.FirstBlock) logger.TestLog(opts.Sleep != float64(0.0), "Sleep: ", opts.Sleep) opts.Conn.TestLog(opts.getCaches()) @@ -78,6 +78,7 @@ func initFinishParseApi(w http.ResponseWriter, r *http.Request) *InitOptions { } } opts.Conn = opts.Globals.FinishParseApi(w, r, opts.getCaches()) + opts.Publisher, _ = opts.Conn.GetEnsAddress(opts.Publisher) // EXISTING_CODE // EXISTING_CODE @@ -103,6 +104,7 @@ func initFinishParse(args []string) *InitOptions { defFmt := "txt" opts := GetOptions() opts.Conn = opts.Globals.FinishParse(args, opts.getCaches()) + opts.Publisher, _ = opts.Conn.GetEnsAddress(opts.Publisher) // EXISTING_CODE if len(args) > 0 { diff --git a/src/cmd-line-options.csv b/src/cmd-line-options.csv index 2d41df58b4..371ab34267 100644 --- a/src/cmd-line-options.csv +++ b/src/cmd-line-options.csv @@ -259,7 +259,7 @@ num,group,tags,api_route,tool,longName,hotKey,def_val,is_required,is_customizabl 31920,apps,Admin,chunks,chunkMan,check,c,,false,false,true,true,gocmd,switch,,check the manifest, index, or blooms for internal consistency 31935,apps,Admin,chunks,chunkMan,pin,i,,false,false,true,true,gocmd,switch,,pin the manifest or each index chunk and bloom 31940,apps,Admin,chunks,chunkMan,publish,p,,false,false,true,true,gocmd,switch,,publish the manifest to the Unchained Index smart contract -31942,apps,Admin,chunks,chunkMan,publisher,P,trueblocks.eth,false,false,true,true,gocmd,flag,,for some query options, the publisher of the index +31942,apps,Admin,chunks,chunkMan,publisher,P,trueblocks.eth,false,false,true,true,gocmd,flag,
,for some query options, the publisher of the index 31925,apps,Admin,chunks,chunkMan,truncate,n,,false,false,false,false,gocmd,flag,,truncate the entire index at this block (requires a block identifier) 31945,apps,Admin,chunks,chunkMan,remote,r,,false,false,true,true,gocmd,switch,,prior to processing, retreive the manifest from the Unchained Index smart contract 31950,apps,Admin,chunks,chunkMan,belongs,b,,false,false,true,true,gocmd,flag,list,in index mode only, checks the address(es) for inclusion in the given index chunk @@ -281,7 +281,7 @@ num,group,tags,api_route,tool,longName,hotKey,def_val,is_required,is_customizabl 11905,apps,Admin,init,init,all,a,,false,false,true,true,gocmd,switch,,in addition to Bloom filters, download full index chunks (recommended) 11907,apps,Admin,init,init,dry_run,d,,false,false,true,true,gocmd,switch,,display the results of the download without actually downloading -11910,apps,Admin,init,init,publisher,P,trueblocks.eth,false,false,true,true,gocmd,flag,,the publisher of the index to download +11910,apps,Admin,init,init,publisher,P,trueblocks.eth,false,false,true,true,gocmd,flag,
,the publisher of the index to download 11915,apps,Admin,init,init,first_block,F,0,false,false,true,true,gocmd,flag,,do not download any chunks earlier than this block 11920,apps,Admin,init,init,sleep,s,0.0,false,false,true,true,gocmd,flag,,seconds to sleep between downloads 11925,apps,Admin,init,init,,,,false,false,true,true,--,description,,Initialize the TrueBlocks system by downloading the Unchained Index from IPFS. diff --git a/src/dev_tools/makeClass/handle_gocmds.cpp b/src/dev_tools/makeClass/handle_gocmds.cpp index befa8f1d87..7e926ccfd7 100644 --- a/src/dev_tools/makeClass/handle_gocmds.cpp +++ b/src/dev_tools/makeClass/handle_gocmds.cpp @@ -384,6 +384,7 @@ string_q get_godefaults(const CCommandOption& cmd) { for (auto p : *((CCommandOptionArray*)cmd.members)) { if (!isDef(p)) { string_q val = substitute(p.def_val, "NOPOS", "utils.NOPOS"); + val = substitute(val, "trueblocks.eth", "\"trueblocks.eth\""); os << "\t" << padRight(p.Format("[{VARIABLE}]") + ": ", wid + 2, ' ') << val << "," << endl; } } @@ -408,9 +409,17 @@ string_q get_testlogs(const CCommandOption& cmd) { } else if (startsWith(p.data_type, "list<") || p.data_type == "" || p.data_type == "
" || contains(p.data_type, "enum")) { - const char* STR_TESTLOG_STRING = - "\tlogger.TestLog(len(opts.[{VARIABLE}]) > 0, \"[{VARIABLE}]: \", opts.[{VARIABLE}])"; - os << p.Format(STR_TESTLOG_STRING) << endl; + if (!p.def_val.empty() && p.def_val != "\"\"" && p.def_val != "utils.NOPOS") { + const char* STR_TESTLOG_STRING = + "\tlogger.TestLog(len(opts.[{VARIABLE}]) > 0 && opts.[{VARIABLE}] != \"[{DEF_VAL}]\", " + "\"[{VARIABLE}]: \", opts.[{VARIABLE}])"; + p.def_val = substitute(p.def_val, "\"", ""); + os << p.Format(STR_TESTLOG_STRING) << endl; + } else { + const char* STR_TESTLOG_STRING = + "\tlogger.TestLog(len(opts.[{VARIABLE}]) > 0, \"[{VARIABLE}]: \", opts.[{VARIABLE}])"; + os << p.Format(STR_TESTLOG_STRING) << endl; + } } else if (p.data_type == "" || p.data_type == "") { const char* STR_TESTLOG_UINT = @@ -640,7 +649,11 @@ string_q get_goDefault(const CCommandOption& p) { return p.def_val; return "0.0"; } else if (p.go_intype == "string") { - return p.def_val; + if (contains(p.def_val, ".eth")) { // an address + return "\"" + p.def_val + "\""; + } else { + return p.def_val; + } } else if (p.go_intype == "uint64") { if (contains(p.def_val, "NOPOS")) { return "0"; From deab89eedfac00afbdced75bf45e9c7866b73ab7 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Fri, 8 Sep 2023 10:48:13 +0200 Subject: [PATCH 24/33] Fixes chifra when --- src/apps/chifra/pkg/tslib/list.go | 5 +++++ src/apps/chifra/pkg/tslib/list_integration_test.go | 2 +- src/apps/chifra/pkg/tslib/list_test.go | 2 +- test/gold/apps/chifra/chifra_help_chunks.txt | 2 ++ test/gold/apps/chifra/chifra_help_init.txt | 1 + test/gold/apps/chunkMan/chunkMan_addresses_default.txt | 1 + test/gold/apps/chunkMan/chunkMan_addresses_json.txt | 1 + test/gold/apps/chunkMan/chunkMan_appearances_default.txt | 1 + test/gold/apps/chunkMan/chunkMan_appearances_txt.txt | 1 + test/gold/apps/chunkMan/chunkMan_belongs.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_belongs2.txt | 1 + test/gold/apps/chunkMan/chunkMan_block_not_found.txt | 1 + test/gold/apps/chunkMan/chunkMan_blooms_default.txt | 1 + test/gold/apps/chunkMan/chunkMan_blooms_txt.txt | 1 + test/gold/apps/chunkMan/chunkMan_caps_allowed.txt | 2 ++ test/gold/apps/chunkMan/chunkMan_caps_disallowed_1.txt | 2 ++ test/gold/apps/chunkMan/chunkMan_caps_disallowed_2.txt | 2 ++ test/gold/apps/chunkMan/chunkMan_caps_disallowed_3.txt | 2 ++ test/gold/apps/chunkMan/chunkMan_caps_disallowed_4.txt | 2 ++ test/gold/apps/chunkMan/chunkMan_caps_disallowed_5.txt | 2 ++ test/gold/apps/chunkMan/chunkMan_check_bad.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_clean_bad.txt | 2 ++ test/gold/apps/chunkMan/chunkMan_fail_belongs_addr_wrong.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_fail_belongs_no_addr.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_fail_belongs_no_blocks.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_fail_belongs_wrong_mode.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_fail_both.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_fail_detail_belong.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_fail_neither.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_header_csv.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_header_default.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_help.txt | 2 ++ test/gold/apps/chunkMan/chunkMan_help_long.txt | 2 ++ test/gold/apps/chunkMan/chunkMan_index_default.txt | 1 + test/gold/apps/chunkMan/chunkMan_index_json.txt | 1 + test/gold/apps/chunkMan/chunkMan_invalid_param_1.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_list_check.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_manifest_csv.txt | 1 + test/gold/apps/chunkMan/chunkMan_manifest_default.txt | 1 + test/gold/apps/chunkMan/chunkMan_no_params.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_pin_chunks_bad.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_pin_chunks_bad2.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_pin_data_bad.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_pin_data_bad2.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_redir_output.txt | 1 + test/gold/apps/chunkMan/chunkMan_redir_output_append.txt | 1 + test/gold/apps/chunkMan/chunkMan_remote_fail.txt | 3 +++ test/gold/apps/chunkMan/chunkMan_stats_default.txt | 1 + test/gold/apps/chunkMan/chunkMan_stats_txt.txt | 1 + test/gold/apps/init/init_by_file_bad.txt | 1 + test/gold/apps/init/init_caps_allowed.txt | 1 + test/gold/apps/init/init_caps_disallowed_1.txt | 1 + test/gold/apps/init/init_caps_disallowed_2.txt | 1 + test/gold/apps/init/init_caps_disallowed_3.txt | 1 + test/gold/apps/init/init_caps_disallowed_4.txt | 1 + test/gold/apps/init/init_caps_disallowed_5.txt | 1 + test/gold/apps/init/init_caps_disallowed_6.txt | 1 + test/gold/apps/init/init_caps_disallowed_7.txt | 1 + test/gold/apps/init/init_first_block.txt | 2 ++ test/gold/apps/init/init_fmt_api.txt | 1 + test/gold/apps/init/init_fmt_csv.txt | 1 + test/gold/apps/init/init_fmt_default.txt | 2 ++ test/gold/apps/init/init_fmt_json.txt | 1 + test/gold/apps/init/init_fmt_junk.txt | 1 + test/gold/apps/init/init_fmt_txt.txt | 1 + test/gold/apps/init/init_help.txt | 1 + test/gold/apps/init/init_help_long.txt | 1 + test/gold/apps/init/init_invalid_param_1.txt | 1 + test/gold/apps/init/init_list_local.txt | 2 ++ test/gold/apps/init/init_no_params.txt | 2 ++ 70 files changed, 126 insertions(+), 2 deletions(-) diff --git a/src/apps/chifra/pkg/tslib/list.go b/src/apps/chifra/pkg/tslib/list.go index aa1101873d..e6bc5a5c52 100644 --- a/src/apps/chifra/pkg/tslib/list.go +++ b/src/apps/chifra/pkg/tslib/list.go @@ -10,6 +10,7 @@ import ( "fmt" "io" "os" + "strings" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" @@ -85,6 +86,10 @@ func readSpecials(path string, nFields int) (specials []types.SimpleNamedBlock, // "date": 3, // skipped since we have timestamp } } + if strings.Contains(record[locs["bn"]], "-") { + // before block zero + continue + } s := types.SimpleNamedBlock{ BlockNumber: utils.MustParseUint(record[locs["bn"]]), Name: record[locs["name"]], diff --git a/src/apps/chifra/pkg/tslib/list_integration_test.go b/src/apps/chifra/pkg/tslib/list_integration_test.go index dd5521610d..a5ac629e9d 100644 --- a/src/apps/chifra/pkg/tslib/list_integration_test.go +++ b/src/apps/chifra/pkg/tslib/list_integration_test.go @@ -38,7 +38,7 @@ func TestGetSpecials2(t *testing.T) { t.Error(err) } - if len(specials) != 42 { + if len(specials) != 39 { t.Error("Wrong number of special blocks ", len(specials), ". Should have 39.") } diff --git a/src/apps/chifra/pkg/tslib/list_test.go b/src/apps/chifra/pkg/tslib/list_test.go index 1bc93235ca..4efe7c0a66 100644 --- a/src/apps/chifra/pkg/tslib/list_test.go +++ b/src/apps/chifra/pkg/tslib/list_test.go @@ -73,7 +73,7 @@ func TestGetSpecials(t *testing.T) { t.Error(err) } - if len(specials) != 42 { + if len(specials) != 39 { t.Error("Wrong number of special blocks ", len(specials), ". Should have 39.") } diff --git a/test/gold/apps/chifra/chifra_help_chunks.txt b/test/gold/apps/chifra/chifra_help_chunks.txt index d234f14524..592939a8c6 100644 --- a/test/gold/apps/chifra/chifra_help_chunks.txt +++ b/test/gold/apps/chifra/chifra_help_chunks.txt @@ -14,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,3 +36,4 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chifra/chifra_help_init.txt b/test/gold/apps/chifra/chifra_help_init.txt index a79014affa..48420c8dc1 100644 --- a/test/gold/apps/chifra/chifra_help_init.txt +++ b/test/gold/apps/chifra/chifra_help_init.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/chunkMan/chunkMan_addresses_default.txt b/test/gold/apps/chunkMan/chunkMan_addresses_default.txt index f963e167cb..297834abd1 100644 --- a/test/gold/apps/chunkMan/chunkMan_addresses_default.txt +++ b/test/gold/apps/chunkMan/chunkMan_addresses_default.txt @@ -1,5 +1,6 @@ chifra chunks addresses TEST[DATE|TIME] Mode: addresses +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt address range offset count 0x000d836201318ec6899a67540690382780743280 000000000-000000000 0 1 diff --git a/test/gold/apps/chunkMan/chunkMan_addresses_json.txt b/test/gold/apps/chunkMan/chunkMan_addresses_json.txt index 2b266a8161..f9226660a7 100644 --- a/test/gold/apps/chunkMan/chunkMan_addresses_json.txt +++ b/test/gold/apps/chunkMan/chunkMan_addresses_json.txt @@ -1,5 +1,6 @@ chifra chunks addresses --fmt json TEST[DATE|TIME] Mode: addresses +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: json { "data": [ diff --git a/test/gold/apps/chunkMan/chunkMan_appearances_default.txt b/test/gold/apps/chunkMan/chunkMan_appearances_default.txt index af878a5849..b9d3c42924 100644 --- a/test/gold/apps/chunkMan/chunkMan_appearances_default.txt +++ b/test/gold/apps/chunkMan/chunkMan_appearances_default.txt @@ -1,5 +1,6 @@ chifra chunks appearances TEST[DATE|TIME] Mode: appearances +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt blockNumber transactionIndex 0 0 diff --git a/test/gold/apps/chunkMan/chunkMan_appearances_txt.txt b/test/gold/apps/chunkMan/chunkMan_appearances_txt.txt index 9b68162a68..0656022295 100644 --- a/test/gold/apps/chunkMan/chunkMan_appearances_txt.txt +++ b/test/gold/apps/chunkMan/chunkMan_appearances_txt.txt @@ -1,5 +1,6 @@ chifra chunks appearances --fmt txt TEST[DATE|TIME] Mode: appearances +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt blockNumber transactionIndex 0 0 diff --git a/test/gold/apps/chunkMan/chunkMan_belongs.txt b/test/gold/apps/chunkMan/chunkMan_belongs.txt index f389363895..1d6ea77379 100644 --- a/test/gold/apps/chunkMan/chunkMan_belongs.txt +++ b/test/gold/apps/chunkMan/chunkMan_belongs.txt @@ -1,6 +1,7 @@ chifra chunks index --belongs 0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9 2287625 --fmt csv TEST[DATE|TIME] Mode: index TEST[DATE|TIME] Blocks: [2287625] +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Belongs: [0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9] TEST[DATE|TIME] Format: csv Error: The --belongs option only works with --fmt json @@ -16,6 +17,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -37,4 +39,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_belongs2.txt b/test/gold/apps/chunkMan/chunkMan_belongs2.txt index 20169a53ad..5ebc7ff761 100644 --- a/test/gold/apps/chunkMan/chunkMan_belongs2.txt +++ b/test/gold/apps/chunkMan/chunkMan_belongs2.txt @@ -1,6 +1,7 @@ chifra chunks index --belongs 0x0000000000000214156104345760006001905490 0x12 TEST[DATE|TIME] Mode: index TEST[DATE|TIME] Blocks: [0x12] +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Belongs: [0x0000000000000214156104345760006001905490] TEST[DATE|TIME] Format: json { diff --git a/test/gold/apps/chunkMan/chunkMan_block_not_found.txt b/test/gold/apps/chunkMan/chunkMan_block_not_found.txt index 7833bf54f5..05a1ec3379 100644 --- a/test/gold/apps/chunkMan/chunkMan_block_not_found.txt +++ b/test/gold/apps/chunkMan/chunkMan_block_not_found.txt @@ -1,6 +1,7 @@ chifra chunks index --belongs 0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9 0x0f1217b92276cd17608d4212879739e6a5ec388bd7a03bef9798655234afd2b2 TEST[DATE|TIME] Mode: index TEST[DATE|TIME] Blocks: [0x0f1217b92276cd17608d4212879739e6a5ec388bd7a03bef9798655234afd2b2] +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Belongs: [0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9] TEST[DATE|TIME] Format: json { diff --git a/test/gold/apps/chunkMan/chunkMan_blooms_default.txt b/test/gold/apps/chunkMan/chunkMan_blooms_default.txt index dac6ecbbf8..7eeb8b5f78 100644 --- a/test/gold/apps/chunkMan/chunkMan_blooms_default.txt +++ b/test/gold/apps/chunkMan/chunkMan_blooms_default.txt @@ -1,5 +1,6 @@ chifra chunks blooms TEST[DATE|TIME] Mode: blooms +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt range magic hash nBlooms nInserted size byteWidth 000000000-000000000 0xdead 0x81ae14ba68e372bc9bd4a295b844abd8e72b1de10fcd706e624647701d911da1 1 8893 131114 131072 diff --git a/test/gold/apps/chunkMan/chunkMan_blooms_txt.txt b/test/gold/apps/chunkMan/chunkMan_blooms_txt.txt index 519d6eb826..4f92fa2167 100644 --- a/test/gold/apps/chunkMan/chunkMan_blooms_txt.txt +++ b/test/gold/apps/chunkMan/chunkMan_blooms_txt.txt @@ -1,5 +1,6 @@ chifra chunks blooms --fmt txt TEST[DATE|TIME] Mode: blooms +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt range magic hash nBlooms nInserted size byteWidth 000000000-000000000 0xdead 0x81ae14ba68e372bc9bd4a295b844abd8e72b1de10fcd706e624647701d911da1 1 8893 131114 131072 diff --git a/test/gold/apps/chunkMan/chunkMan_caps_allowed.txt b/test/gold/apps/chunkMan/chunkMan_caps_allowed.txt index 8f64486fb2..ff85c2ccd4 100644 --- a/test/gold/apps/chunkMan/chunkMan_caps_allowed.txt +++ b/test/gold/apps/chunkMan/chunkMan_caps_allowed.txt @@ -14,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +36,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_caps_disallowed_1.txt b/test/gold/apps/chunkMan/chunkMan_caps_disallowed_1.txt index 0f790f2ae6..2886c2d217 100644 --- a/test/gold/apps/chunkMan/chunkMan_caps_disallowed_1.txt +++ b/test/gold/apps/chunkMan/chunkMan_caps_disallowed_1.txt @@ -14,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +36,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_caps_disallowed_2.txt b/test/gold/apps/chunkMan/chunkMan_caps_disallowed_2.txt index 037a018acf..b45dee9a4a 100644 --- a/test/gold/apps/chunkMan/chunkMan_caps_disallowed_2.txt +++ b/test/gold/apps/chunkMan/chunkMan_caps_disallowed_2.txt @@ -14,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +36,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_caps_disallowed_3.txt b/test/gold/apps/chunkMan/chunkMan_caps_disallowed_3.txt index b1a0d19161..95a8e1c488 100644 --- a/test/gold/apps/chunkMan/chunkMan_caps_disallowed_3.txt +++ b/test/gold/apps/chunkMan/chunkMan_caps_disallowed_3.txt @@ -14,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +36,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_caps_disallowed_4.txt b/test/gold/apps/chunkMan/chunkMan_caps_disallowed_4.txt index 9c8fe0aac3..6585d1adc5 100644 --- a/test/gold/apps/chunkMan/chunkMan_caps_disallowed_4.txt +++ b/test/gold/apps/chunkMan/chunkMan_caps_disallowed_4.txt @@ -14,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +36,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_caps_disallowed_5.txt b/test/gold/apps/chunkMan/chunkMan_caps_disallowed_5.txt index 74dd6e27cc..003d13ee77 100644 --- a/test/gold/apps/chunkMan/chunkMan_caps_disallowed_5.txt +++ b/test/gold/apps/chunkMan/chunkMan_caps_disallowed_5.txt @@ -14,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +36,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_check_bad.txt b/test/gold/apps/chunkMan/chunkMan_check_bad.txt index 60b15ca7d3..e18c222b55 100644 --- a/test/gold/apps/chunkMan/chunkMan_check_bad.txt +++ b/test/gold/apps/chunkMan/chunkMan_check_bad.txt @@ -1,6 +1,7 @@ chifra chunks addresses --check TEST[DATE|TIME] Mode: addresses TEST[DATE|TIME] Check: true +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: The --check option is not available in addresses mode. Usage: @@ -15,6 +16,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -36,4 +38,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_clean_bad.txt b/test/gold/apps/chunkMan/chunkMan_clean_bad.txt index efd5b583f9..397122641a 100644 --- a/test/gold/apps/chunkMan/chunkMan_clean_bad.txt +++ b/test/gold/apps/chunkMan/chunkMan_clean_bad.txt @@ -14,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +36,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_fail_belongs_addr_wrong.txt b/test/gold/apps/chunkMan/chunkMan_fail_belongs_addr_wrong.txt index 8581d1e3f4..a1498b2406 100644 --- a/test/gold/apps/chunkMan/chunkMan_fail_belongs_addr_wrong.txt +++ b/test/gold/apps/chunkMan/chunkMan_fail_belongs_addr_wrong.txt @@ -1,5 +1,6 @@ chifra chunks index 0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9 TEST[DATE|TIME] Mode: index +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Belongs: [0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9] TEST[DATE|TIME] Format: json Error: You must specify at least one block identifier with the --belongs option @@ -15,6 +16,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -36,4 +38,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_fail_belongs_no_addr.txt b/test/gold/apps/chunkMan/chunkMan_fail_belongs_no_addr.txt index eca1a00b63..252f7e3673 100644 --- a/test/gold/apps/chunkMan/chunkMan_fail_belongs_no_addr.txt +++ b/test/gold/apps/chunkMan/chunkMan_fail_belongs_no_addr.txt @@ -1,5 +1,6 @@ chifra chunks index --belongs 2287625 TEST[DATE|TIME] Mode: index +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Belongs: [2287625] TEST[DATE|TIME] Format: json Error: Please specify at least one valid Ethereum address. @@ -15,6 +16,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -36,4 +38,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_fail_belongs_no_blocks.txt b/test/gold/apps/chunkMan/chunkMan_fail_belongs_no_blocks.txt index 43f8a94832..5f5880c58c 100644 --- a/test/gold/apps/chunkMan/chunkMan_fail_belongs_no_blocks.txt +++ b/test/gold/apps/chunkMan/chunkMan_fail_belongs_no_blocks.txt @@ -1,5 +1,6 @@ chifra chunks index --belongs 0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9 TEST[DATE|TIME] Mode: index +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Belongs: [0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9] TEST[DATE|TIME] Format: json Error: You must specify at least one block identifier with the --belongs option @@ -15,6 +16,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -36,4 +38,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_fail_belongs_wrong_mode.txt b/test/gold/apps/chunkMan/chunkMan_fail_belongs_wrong_mode.txt index 07b0cca752..81be686e3c 100644 --- a/test/gold/apps/chunkMan/chunkMan_fail_belongs_wrong_mode.txt +++ b/test/gold/apps/chunkMan/chunkMan_fail_belongs_wrong_mode.txt @@ -1,5 +1,6 @@ chifra chunks addresses --belongs 0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9 TEST[DATE|TIME] Mode: addresses +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Belongs: [0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9] TEST[DATE|TIME] Format: json Error: The --belongs option requires the index mode. @@ -15,6 +16,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -36,4 +38,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_fail_both.txt b/test/gold/apps/chunkMan/chunkMan_fail_both.txt index f57396fd51..dd517d1cd2 100644 --- a/test/gold/apps/chunkMan/chunkMan_fail_both.txt +++ b/test/gold/apps/chunkMan/chunkMan_fail_both.txt @@ -1,5 +1,6 @@ chifra chunks --check TEST[DATE|TIME] Check: true +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: Please choose at least one of [manifest|index|blooms|addresses|appearances|stats]. Usage: @@ -14,6 +15,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +37,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_fail_detail_belong.txt b/test/gold/apps/chunkMan/chunkMan_fail_detail_belong.txt index 62e12e6443..9827557935 100644 --- a/test/gold/apps/chunkMan/chunkMan_fail_detail_belong.txt +++ b/test/gold/apps/chunkMan/chunkMan_fail_detail_belong.txt @@ -1,6 +1,7 @@ chifra chunks index --belongs 0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9 2287625 --verbose TEST[DATE|TIME] Mode: index TEST[DATE|TIME] Blocks: [2287625] +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Belongs: [0x0101f3be8ebb4bbd39a2e3b9a3639d4259832fd9] TEST[DATE|TIME] Verbose: true TEST[DATE|TIME] Format: json @@ -17,6 +18,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -38,4 +40,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_fail_neither.txt b/test/gold/apps/chunkMan/chunkMan_fail_neither.txt index 2abb699da2..d3dceb9985 100644 --- a/test/gold/apps/chunkMan/chunkMan_fail_neither.txt +++ b/test/gold/apps/chunkMan/chunkMan_fail_neither.txt @@ -1,4 +1,5 @@ chifra chunks +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: Please choose at least one of [manifest|index|blooms|addresses|appearances|stats]. Usage: @@ -13,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -34,4 +36,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_header_csv.txt b/test/gold/apps/chunkMan/chunkMan_header_csv.txt index eaa12a3267..0a8cf14226 100644 --- a/test/gold/apps/chunkMan/chunkMan_header_csv.txt +++ b/test/gold/apps/chunkMan/chunkMan_header_csv.txt @@ -1,5 +1,6 @@ chifra chunks header --fmt csv TEST[DATE|TIME] Mode: header +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: csv Error: The mode option (header) must be one of [ manifest | index | blooms | addresses | appearances | stats ] Usage: @@ -14,6 +15,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +37,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_header_default.txt b/test/gold/apps/chunkMan/chunkMan_header_default.txt index 05c543b640..a56c87fc33 100644 --- a/test/gold/apps/chunkMan/chunkMan_header_default.txt +++ b/test/gold/apps/chunkMan/chunkMan_header_default.txt @@ -1,5 +1,6 @@ chifra chunks header TEST[DATE|TIME] Mode: header +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: The mode option (header) must be one of [ manifest | index | blooms | addresses | appearances | stats ] Usage: @@ -14,6 +15,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +37,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_help.txt b/test/gold/apps/chunkMan/chunkMan_help.txt index 881330d563..92f77c7a26 100644 --- a/test/gold/apps/chunkMan/chunkMan_help.txt +++ b/test/gold/apps/chunkMan/chunkMan_help.txt @@ -14,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,3 +36,4 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_help_long.txt b/test/gold/apps/chunkMan/chunkMan_help_long.txt index dada649588..68eeb8a856 100644 --- a/test/gold/apps/chunkMan/chunkMan_help_long.txt +++ b/test/gold/apps/chunkMan/chunkMan_help_long.txt @@ -14,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,3 +36,4 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_index_default.txt b/test/gold/apps/chunkMan/chunkMan_index_default.txt index 921cf19065..a3f0cd381b 100644 --- a/test/gold/apps/chunkMan/chunkMan_index_default.txt +++ b/test/gold/apps/chunkMan/chunkMan_index_default.txt @@ -1,5 +1,6 @@ chifra chunks index TEST[DATE|TIME] Mode: index +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt range magic hash nAddresses nAppearances fileSize 000000000-000000000 0xdeadbeef 0x81ae...1da1 8893 8893 320192 diff --git a/test/gold/apps/chunkMan/chunkMan_index_json.txt b/test/gold/apps/chunkMan/chunkMan_index_json.txt index d3af12e40f..ff737dca60 100644 --- a/test/gold/apps/chunkMan/chunkMan_index_json.txt +++ b/test/gold/apps/chunkMan/chunkMan_index_json.txt @@ -1,5 +1,6 @@ chifra chunks index --fmt json TEST[DATE|TIME] Mode: index +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: json { "data": [ diff --git a/test/gold/apps/chunkMan/chunkMan_invalid_param_1.txt b/test/gold/apps/chunkMan/chunkMan_invalid_param_1.txt index 9f675d573d..a9379f9de3 100644 --- a/test/gold/apps/chunkMan/chunkMan_invalid_param_1.txt +++ b/test/gold/apps/chunkMan/chunkMan_invalid_param_1.txt @@ -1,5 +1,6 @@ chifra chunks junk TEST[DATE|TIME] Mode: junk +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: The mode option (junk) must be one of [ manifest | index | blooms | addresses | appearances | stats ] Usage: @@ -14,6 +15,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +37,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_list_check.txt b/test/gold/apps/chunkMan/chunkMan_list_check.txt index 705fb735ea..22104cff0b 100644 --- a/test/gold/apps/chunkMan/chunkMan_list_check.txt +++ b/test/gold/apps/chunkMan/chunkMan_list_check.txt @@ -1,6 +1,7 @@ chifra chunks monitors --check TEST[DATE|TIME] Mode: monitors TEST[DATE|TIME] Check: true +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: The mode option (monitors) must be one of [ manifest | index | blooms | addresses | appearances | stats ] Usage: @@ -15,6 +16,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -36,4 +38,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_manifest_csv.txt b/test/gold/apps/chunkMan/chunkMan_manifest_csv.txt index b143ecdde9..66b01e709e 100644 --- a/test/gold/apps/chunkMan/chunkMan_manifest_csv.txt +++ b/test/gold/apps/chunkMan/chunkMan_manifest_csv.txt @@ -1,5 +1,6 @@ chifra chunks manifest --fmt csv TEST[DATE|TIME] Mode: manifest +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: csv range,bloomHash,bloomSize,indexHash,indexSize 000000000-000000000,QmYhuaJu9bHAGpSsuaQug7bjnLcoE6B5PCZsg4XZGVFbKy,131114,QmaKUsfH5AXqPJgjGAQFZWRheF1jtc5PqGi8cYrwmMCXdu,320192 diff --git a/test/gold/apps/chunkMan/chunkMan_manifest_default.txt b/test/gold/apps/chunkMan/chunkMan_manifest_default.txt index 1b0b095a52..51ec4c568f 100644 --- a/test/gold/apps/chunkMan/chunkMan_manifest_default.txt +++ b/test/gold/apps/chunkMan/chunkMan_manifest_default.txt @@ -1,5 +1,6 @@ chifra chunks manifest TEST[DATE|TIME] Mode: manifest +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt range bloomHash bloomSize indexHash indexSize 000000000-000000000 QmYhuaJu9bHAGpSsuaQug7bjnLcoE6B5PCZsg4XZGVFbKy 131114 QmaKUsfH5AXqPJgjGAQFZWRheF1jtc5PqGi8cYrwmMCXdu 320192 diff --git a/test/gold/apps/chunkMan/chunkMan_no_params.txt b/test/gold/apps/chunkMan/chunkMan_no_params.txt index 2abb699da2..d3dceb9985 100644 --- a/test/gold/apps/chunkMan/chunkMan_no_params.txt +++ b/test/gold/apps/chunkMan/chunkMan_no_params.txt @@ -1,4 +1,5 @@ chifra chunks +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: Please choose at least one of [manifest|index|blooms|addresses|appearances|stats]. Usage: @@ -13,6 +14,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -34,4 +36,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_pin_chunks_bad.txt b/test/gold/apps/chunkMan/chunkMan_pin_chunks_bad.txt index 6020cd67f9..dfe10975ea 100644 --- a/test/gold/apps/chunkMan/chunkMan_pin_chunks_bad.txt +++ b/test/gold/apps/chunkMan/chunkMan_pin_chunks_bad.txt @@ -1,6 +1,7 @@ chifra chunks addresses --pin TEST[DATE|TIME] Mode: addresses TEST[DATE|TIME] Pin: true +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: The --pin option is not available in addresses mode. Usage: @@ -15,6 +16,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -36,4 +38,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_pin_chunks_bad2.txt b/test/gold/apps/chunkMan/chunkMan_pin_chunks_bad2.txt index 6020cd67f9..dfe10975ea 100644 --- a/test/gold/apps/chunkMan/chunkMan_pin_chunks_bad2.txt +++ b/test/gold/apps/chunkMan/chunkMan_pin_chunks_bad2.txt @@ -1,6 +1,7 @@ chifra chunks addresses --pin TEST[DATE|TIME] Mode: addresses TEST[DATE|TIME] Pin: true +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: The --pin option is not available in addresses mode. Usage: @@ -15,6 +16,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -36,4 +38,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_pin_data_bad.txt b/test/gold/apps/chunkMan/chunkMan_pin_data_bad.txt index e72bdaadcf..cb18e0e6bc 100644 --- a/test/gold/apps/chunkMan/chunkMan_pin_data_bad.txt +++ b/test/gold/apps/chunkMan/chunkMan_pin_data_bad.txt @@ -1,6 +1,7 @@ chifra chunks manifest --publish TEST[DATE|TIME] Mode: manifest TEST[DATE|TIME] Publish: true +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: The --publish option is not available in test mode. Usage: @@ -15,6 +16,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -36,4 +38,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_pin_data_bad2.txt b/test/gold/apps/chunkMan/chunkMan_pin_data_bad2.txt index 2625843f07..fe665df348 100644 --- a/test/gold/apps/chunkMan/chunkMan_pin_data_bad2.txt +++ b/test/gold/apps/chunkMan/chunkMan_pin_data_bad2.txt @@ -1,6 +1,7 @@ chifra chunks addresses --publish TEST[DATE|TIME] Mode: addresses TEST[DATE|TIME] Publish: true +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: The --publish option is not available in addresses mode. Usage: @@ -15,6 +16,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -36,4 +38,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_redir_output.txt b/test/gold/apps/chunkMan/chunkMan_redir_output.txt index 45eab117b6..c9452dc524 100644 --- a/test/gold/apps/chunkMan/chunkMan_redir_output.txt +++ b/test/gold/apps/chunkMan/chunkMan_redir_output.txt @@ -1,6 +1,7 @@ chifra chunks index 2000000 --output output_test_file TEST[DATE|TIME] Mode: index TEST[DATE|TIME] Blocks: [2000000] +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] OutputFn: output_test_file TEST[DATE|TIME] Format: txt ---- diff --git a/test/gold/apps/chunkMan/chunkMan_redir_output_append.txt b/test/gold/apps/chunkMan/chunkMan_redir_output_append.txt index 7f3a341d72..0b6bd90a44 100644 --- a/test/gold/apps/chunkMan/chunkMan_redir_output_append.txt +++ b/test/gold/apps/chunkMan/chunkMan_redir_output_append.txt @@ -1,6 +1,7 @@ chifra chunks index 3000000 --output output_test_file --no_header --append TEST[DATE|TIME] Mode: index TEST[DATE|TIME] Blocks: [3000000] +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] NoHeader: true TEST[DATE|TIME] OutputFn: output_test_file TEST[DATE|TIME] Append: true diff --git a/test/gold/apps/chunkMan/chunkMan_remote_fail.txt b/test/gold/apps/chunkMan/chunkMan_remote_fail.txt index 8fe79ca8ed..537c53ad6e 100644 --- a/test/gold/apps/chunkMan/chunkMan_remote_fail.txt +++ b/test/gold/apps/chunkMan/chunkMan_remote_fail.txt @@ -1,5 +1,6 @@ chifra chunks remote TEST[DATE|TIME] Mode: remote +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt Error: The mode option (remote) must be one of [ manifest | index | blooms | addresses | appearances | stats ] Usage: @@ -14,6 +15,7 @@ Flags: -c, --check check the manifest, index, or blooms for internal consistency -i, --pin pin the manifest or each index chunk and bloom -p, --publish publish the manifest to the Unchained Index smart contract + -P, --publisher string for some query options, the publisher of the index (default "trueblocks.eth") -n, --truncate uint truncate the entire index at this block (requires a block identifier) (hidden) -r, --remote prior to processing, retreive the manifest from the Unchained Index smart contract -b, --belongs strings in index mode only, checks the address(es) for inclusion in the given index chunk @@ -35,4 +37,5 @@ Notes: - The --first_block and --last_block options apply only to addresses, appearances, and index --belongs mode. - The --pin option requires a locally running IPFS node or a pinning service API key. - The --publish option requires a private key. + - The --publisher option is ignored with the --publish option since the sender of the transaction is recorded as the publisher. diff --git a/test/gold/apps/chunkMan/chunkMan_stats_default.txt b/test/gold/apps/chunkMan/chunkMan_stats_default.txt index 91be3a0348..a899949bf6 100644 --- a/test/gold/apps/chunkMan/chunkMan_stats_default.txt +++ b/test/gold/apps/chunkMan/chunkMan_stats_default.txt @@ -1,5 +1,6 @@ chifra chunks stats TEST[DATE|TIME] Mode: stats +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt range rangeEnd nAddrs nApps nBlocks nBlooms recWid bloomSz chunkSz addrsPerBlock appsPerBlock appsPerAddr ratio 000000000-000000000 2015-07-30 15:26:15 UTC 8893 8893 1 1 131076 131114 320192 8893 8893 1 2.4420885641502816 diff --git a/test/gold/apps/chunkMan/chunkMan_stats_txt.txt b/test/gold/apps/chunkMan/chunkMan_stats_txt.txt index 19955c3614..4a5ad84d56 100644 --- a/test/gold/apps/chunkMan/chunkMan_stats_txt.txt +++ b/test/gold/apps/chunkMan/chunkMan_stats_txt.txt @@ -1,5 +1,6 @@ chifra chunks stats --fmt txt TEST[DATE|TIME] Mode: stats +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Format: txt range rangeEnd nAddrs nApps nBlocks nBlooms recWid bloomSz chunkSz addrsPerBlock appsPerBlock appsPerAddr ratio 000000000-000000000 2015-07-30 15:26:15 UTC 8893 8893 1 1 131076 131114 320192 8893 8893 1 2.4420885641502816 diff --git a/test/gold/apps/init/init_by_file_bad.txt b/test/gold/apps/init/init_by_file_bad.txt index f62e45e6e6..da0c15f091 100644 --- a/test/gold/apps/init/init_by_file_bad.txt +++ b/test/gold/apps/init/init_by_file_bad.txt @@ -6,6 +6,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_caps_allowed.txt b/test/gold/apps/init/init_caps_allowed.txt index 9943a6054a..2fe51c0fcc 100644 --- a/test/gold/apps/init/init_caps_allowed.txt +++ b/test/gold/apps/init/init_caps_allowed.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_caps_disallowed_1.txt b/test/gold/apps/init/init_caps_disallowed_1.txt index a58cfe94cf..3d62cc8d45 100644 --- a/test/gold/apps/init/init_caps_disallowed_1.txt +++ b/test/gold/apps/init/init_caps_disallowed_1.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_caps_disallowed_2.txt b/test/gold/apps/init/init_caps_disallowed_2.txt index d9113c9dd1..aad40d6d9f 100644 --- a/test/gold/apps/init/init_caps_disallowed_2.txt +++ b/test/gold/apps/init/init_caps_disallowed_2.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_caps_disallowed_3.txt b/test/gold/apps/init/init_caps_disallowed_3.txt index 3057e11442..f5f2c54713 100644 --- a/test/gold/apps/init/init_caps_disallowed_3.txt +++ b/test/gold/apps/init/init_caps_disallowed_3.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_caps_disallowed_4.txt b/test/gold/apps/init/init_caps_disallowed_4.txt index 426e4be9c2..ccee6b12d4 100644 --- a/test/gold/apps/init/init_caps_disallowed_4.txt +++ b/test/gold/apps/init/init_caps_disallowed_4.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_caps_disallowed_5.txt b/test/gold/apps/init/init_caps_disallowed_5.txt index ac40f20e5c..b38600976c 100644 --- a/test/gold/apps/init/init_caps_disallowed_5.txt +++ b/test/gold/apps/init/init_caps_disallowed_5.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_caps_disallowed_6.txt b/test/gold/apps/init/init_caps_disallowed_6.txt index cf1d10ae53..b3773f630b 100644 --- a/test/gold/apps/init/init_caps_disallowed_6.txt +++ b/test/gold/apps/init/init_caps_disallowed_6.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_caps_disallowed_7.txt b/test/gold/apps/init/init_caps_disallowed_7.txt index 1f02c7d4cf..743fababde 100644 --- a/test/gold/apps/init/init_caps_disallowed_7.txt +++ b/test/gold/apps/init/init_caps_disallowed_7.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_first_block.txt b/test/gold/apps/init/init_first_block.txt index 3c649b237a..c82ad8e10a 100644 --- a/test/gold/apps/init/init_first_block.txt +++ b/test/gold/apps/init/init_first_block.txt @@ -1,4 +1,5 @@ chifra init --first_block 12 +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] FirstBlock: 12 TEST[DATE|TIME] Caps: -fmt,-noHeader,-file,-output,-append TEST[DATE|TIME] Format: txt @@ -9,6 +10,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_fmt_api.txt b/test/gold/apps/init/init_fmt_api.txt index 6aa1546a68..9beb80c404 100644 --- a/test/gold/apps/init/init_fmt_api.txt +++ b/test/gold/apps/init/init_fmt_api.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_fmt_csv.txt b/test/gold/apps/init/init_fmt_csv.txt index 38db782e02..5fe7d16276 100644 --- a/test/gold/apps/init/init_fmt_csv.txt +++ b/test/gold/apps/init/init_fmt_csv.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_fmt_default.txt b/test/gold/apps/init/init_fmt_default.txt index f976d41d67..85bd438edd 100644 --- a/test/gold/apps/init/init_fmt_default.txt +++ b/test/gold/apps/init/init_fmt_default.txt @@ -1,4 +1,5 @@ chifra init +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Caps: -fmt,-noHeader,-file,-output,-append TEST[DATE|TIME] Format: txt Error: integration testing was skipped for chifra init @@ -8,6 +9,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_fmt_json.txt b/test/gold/apps/init/init_fmt_json.txt index 9cec668489..28ba7d3bd8 100644 --- a/test/gold/apps/init/init_fmt_json.txt +++ b/test/gold/apps/init/init_fmt_json.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_fmt_junk.txt b/test/gold/apps/init/init_fmt_junk.txt index f143e3b63a..1d29ebdbf1 100644 --- a/test/gold/apps/init/init_fmt_junk.txt +++ b/test/gold/apps/init/init_fmt_junk.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_fmt_txt.txt b/test/gold/apps/init/init_fmt_txt.txt index d7cc7769b1..f1b653e975 100644 --- a/test/gold/apps/init/init_fmt_txt.txt +++ b/test/gold/apps/init/init_fmt_txt.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_help.txt b/test/gold/apps/init/init_help.txt index ca0de64a7d..84edada084 100644 --- a/test/gold/apps/init/init_help.txt +++ b/test/gold/apps/init/init_help.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_help_long.txt b/test/gold/apps/init/init_help_long.txt index 2636510ebf..b5aaa86ef2 100644 --- a/test/gold/apps/init/init_help_long.txt +++ b/test/gold/apps/init/init_help_long.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_invalid_param_1.txt b/test/gold/apps/init/init_invalid_param_1.txt index c2175021ad..651a794a9c 100644 --- a/test/gold/apps/init/init_invalid_param_1.txt +++ b/test/gold/apps/init/init_invalid_param_1.txt @@ -8,6 +8,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_list_local.txt b/test/gold/apps/init/init_list_local.txt index f976d41d67..85bd438edd 100644 --- a/test/gold/apps/init/init_list_local.txt +++ b/test/gold/apps/init/init_list_local.txt @@ -1,4 +1,5 @@ chifra init +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Caps: -fmt,-noHeader,-file,-output,-append TEST[DATE|TIME] Format: txt Error: integration testing was skipped for chifra init @@ -8,6 +9,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output diff --git a/test/gold/apps/init/init_no_params.txt b/test/gold/apps/init/init_no_params.txt index f976d41d67..85bd438edd 100644 --- a/test/gold/apps/init/init_no_params.txt +++ b/test/gold/apps/init/init_no_params.txt @@ -1,4 +1,5 @@ chifra init +TEST[DATE|TIME] Publisher: 0xf503017d7baf7fbc0fff7492b751025c6a78179b TEST[DATE|TIME] Caps: -fmt,-noHeader,-file,-output,-append TEST[DATE|TIME] Format: txt Error: integration testing was skipped for chifra init @@ -8,6 +9,7 @@ Usage: Flags: -a, --all in addition to Bloom filters, download full index chunks (recommended) -d, --dry_run display the results of the download without actually downloading + -P, --publisher string the publisher of the index to download (default "trueblocks.eth") -F, --first_block uint do not download any chunks earlier than this block -s, --sleep float seconds to sleep between downloads -v, --verbose enable verbose output From 1b7e22579bcf5072dfef10d672fa3928b1925236 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Sat, 9 Sep 2023 04:47:06 +0200 Subject: [PATCH 25/33] Establishes the zero address --- src/apps/chifra/internal/export/handle_neighbors.go | 2 +- src/apps/chifra/internal/monitors/handle_watch.go | 2 +- src/apps/chifra/internal/names/output_test.go | 2 +- src/apps/chifra/internal/traces/handle_filter.go | 4 ++-- src/apps/chifra/internal/transactions/handle_source.go | 2 +- src/apps/chifra/pkg/base/address.go | 6 ++++++ src/apps/chifra/pkg/index/establish.go | 2 +- src/apps/chifra/pkg/manifest/remove_chunk.go | 2 +- src/apps/chifra/pkg/monitor/monitor.go | 7 ++----- src/apps/chifra/pkg/prefunds/prefunds.go | 2 +- src/apps/chifra/pkg/pricing/maker.go | 6 +++--- src/apps/chifra/pkg/pricing/reporting.go | 6 +++--- src/apps/chifra/pkg/rpc/contract_integration_test.go | 2 +- src/apps/chifra/pkg/types/types_name.go | 2 +- src/apps/chifra/pkg/types/types_transaction_test.go | 4 ++-- src/dev_tools/utillib/conversions.cpp | 7 +------ src/dev_tools/utillib/conversions.h | 1 - 17 files changed, 28 insertions(+), 31 deletions(-) diff --git a/src/apps/chifra/internal/export/handle_neighbors.go b/src/apps/chifra/internal/export/handle_neighbors.go index 660da7b519..93b10f4c60 100644 --- a/src/apps/chifra/internal/export/handle_neighbors.go +++ b/src/apps/chifra/internal/export/handle_neighbors.go @@ -422,7 +422,7 @@ bool visitBloom(const string_q& path, void* data) { //----------------------------------------------------------------------- void COptions::addNeighbor(CAddressUintMap& map, const address_t& addr) { - if ((addr == ledgerManager.accountedFor || isZeroAddr(addr))) + if ((addr == ledgerManager.accountedFor || isZ eroAddr(addr))) return; map[addr]++; } diff --git a/src/apps/chifra/internal/monitors/handle_watch.go b/src/apps/chifra/internal/monitors/handle_watch.go index 3c7a573969..549abe1d22 100644 --- a/src/apps/chifra/internal/monitors/handle_watch.go +++ b/src/apps/chifra/internal/monitors/handle_watch.go @@ -319,7 +319,7 @@ func (opts *MonitorsOptions) getMonitorList() []monitor.Monitor { for result := range monitorChan { switch result.Address { - case monitor.SentinalAddr: + case base.SentinalAddr: close(monitorChan) default: if result.Count() > 500000 { diff --git a/src/apps/chifra/internal/names/output_test.go b/src/apps/chifra/internal/names/output_test.go index 194534d560..111f52329e 100644 --- a/src/apps/chifra/internal/names/output_test.go +++ b/src/apps/chifra/internal/names/output_test.go @@ -157,7 +157,7 @@ func TestNamesOptions_getCrudDataEnv(t *testing.T) { // invalid envs expected = &CrudData{ Address: crudDataField[base.Address]{ - Value: base.HexToAddress("0x0"), + Value: base.ZeroAddr, Updated: true, }, Name: crudDataField[string]{ diff --git a/src/apps/chifra/internal/traces/handle_filter.go b/src/apps/chifra/internal/traces/handle_filter.go index 181dba47b1..1c54f50862 100644 --- a/src/apps/chifra/internal/traces/handle_filter.go +++ b/src/apps/chifra/internal/traces/handle_filter.go @@ -179,7 +179,7 @@ void getTracesByFilter(CTraceArray& traces, const CTraceFilter& filter) { string_q toAddrs; for (auto addr : filter.toAddress) { - if (!isZeroAddr(addr)) + if (!isZ eroAddr(addr)) toAddrs += ("\"" + addr + "\","); } if (!toAddrs.empty()) { @@ -188,7 +188,7 @@ void getTracesByFilter(CTraceArray& traces, const CTraceFilter& filter) { string_q fromAddrs; for (auto addr : filter.fromAddress) { - if (!isZeroAddr(addr)) + if (!isZ eroAddr(addr)) fromAddrs += ("\"" + addr + "\","); } if (!fromAddrs.empty()) { diff --git a/src/apps/chifra/internal/transactions/handle_source.go b/src/apps/chifra/internal/transactions/handle_source.go index acb589af73..4a4ef14afb 100644 --- a/src/apps/chifra/internal/transactions/handle_source.go +++ b/src/apps/chifra/internal/transactions/handle_source.go @@ -44,7 +44,7 @@ bool CSourceSearch::traceTransfer(CTransfer& transfer, const string_q& id) { CTransfer* tt = &man.transfers[i]; string_q ttId = transfer_2_Id(tt); ostringstream msg; - if (!isZeroAddr(tt->sender)) { + if (!isZ eroAddr(tt->sender)) { if (tt->amount > 0) { if (transfer.sender == tt->recipient) { if (isWhiteListed(tt)) { diff --git a/src/apps/chifra/pkg/base/address.go b/src/apps/chifra/pkg/base/address.go index cbc7372e5e..720aef93ae 100644 --- a/src/apps/chifra/pkg/base/address.go +++ b/src/apps/chifra/pkg/base/address.go @@ -19,6 +19,12 @@ type Address struct { common.Address } +// A few well known address. ZeroAddr, of course, is 0x0. SentinalAddr is a marker to signify the end of the monitor list produced by ListMonitors +var ( + SentinalAddr = HexToAddress("0xdeaddeaddeaddeaddeaddeaddeaddeaddeaddead") + ZeroAddr = HexToAddress("0x0") +) + // Hex returns string representation of an address func (a *Address) Hex() string { if a.IsZero() { diff --git a/src/apps/chifra/pkg/index/establish.go b/src/apps/chifra/pkg/index/establish.go index 2c35eee357..68a1e8e87e 100644 --- a/src/apps/chifra/pkg/index/establish.go +++ b/src/apps/chifra/pkg/index/establish.go @@ -28,7 +28,7 @@ import ( func EstablishIndexChunk(chain string, fileRange base.FileRange) (bool, error) { exists, fileName := fileRange.RangeToFilename(chain) - chunkManifest, err := manifest.ReadManifest(chain, manifest.FromCache) + chunkManifest, err := manifest.ReadManifest(chain, base.ZeroAddr, manifest.FromCache) if err != nil { return exists, err } diff --git a/src/apps/chifra/pkg/manifest/remove_chunk.go b/src/apps/chifra/pkg/manifest/remove_chunk.go index be377a1386..f0dc1f4b24 100644 --- a/src/apps/chifra/pkg/manifest/remove_chunk.go +++ b/src/apps/chifra/pkg/manifest/remove_chunk.go @@ -59,7 +59,7 @@ func RemoveChunk(chain, bloomFn, indexFn string) (err error) { } var man *Manifest - man, err = ReadManifest(chain, FromCache) + man, err = ReadManifest(chain, base.ZeroAddr, FromCache) removedRange, err1 := base.RangeFromFilenameE(bloomFn) if err1 != nil { diff --git a/src/apps/chifra/pkg/monitor/monitor.go b/src/apps/chifra/pkg/monitor/monitor.go index 55b43f5589..ed6aa7c5ca 100644 --- a/src/apps/chifra/pkg/monitor/monitor.go +++ b/src/apps/chifra/pkg/monitor/monitor.go @@ -170,14 +170,11 @@ func (mon *Monitor) Remove() (bool, error) { return !file.FileExists(mon.Path()), nil } -// SentinalAddr is a marker to signify the end of the monitor list produced by ListMonitors -var SentinalAddr = base.HexToAddress("0xdeaddeaddeaddeaddeaddeaddeaddeaddeaddead") - // ListMonitors puts a list of Monitors into the monitorChannel. The list of monitors is built from // a file called addresses.tsv in the current folder or, if not present, from existing monitors func ListMonitors(chain, watchList string, monitorChan chan<- Monitor) { defer func() { - monitorChan <- Monitor{Address: SentinalAddr} + monitorChan <- Monitor{Address: base.SentinalAddr} }() if watchList != "existing" { @@ -250,7 +247,7 @@ func GetMonitorMap(chain string) (map[base.Address]*Monitor, []*Monitor) { for mon := range monitorChan { mon := mon switch mon.Address { - case SentinalAddr: + case base.SentinalAddr: close(monitorChan) default: monMap[mon.Address] = &mon diff --git a/src/apps/chifra/pkg/prefunds/prefunds.go b/src/apps/chifra/pkg/prefunds/prefunds.go index f4b4ff592a..394c4c128c 100644 --- a/src/apps/chifra/pkg/prefunds/prefunds.go +++ b/src/apps/chifra/pkg/prefunds/prefunds.go @@ -21,7 +21,7 @@ type Allocation struct { } // emptyAllocs is a list of empty allocations. We use this to return at least one allocation -var emptyAllocs = []Allocation{{Address: base.HexToAddress("0x0"), Balance: *big.NewInt(0)}} +var emptyAllocs = []Allocation{{Address: base.ZeroAddr, Balance: *big.NewInt(0)}} type allocCallback func(*Allocation, *any) (bool, error) diff --git a/src/apps/chifra/pkg/pricing/maker.go b/src/apps/chifra/pkg/pricing/maker.go index 385b6a19c6..0c849ca598 100644 --- a/src/apps/chifra/pkg/pricing/maker.go +++ b/src/apps/chifra/pkg/pricing/maker.go @@ -66,10 +66,10 @@ func PriceUsdMaker(conn *rpc.Connection, testMode bool, statement *types.SimpleS blockNumber: statement.BlockNumber, source1: makerMedianizer, theCall1: theCall, - source2: base.HexToAddress("0x0"), + source2: base.ZeroAddr, theCall2: "0x" + rawHex, - first: base.HexToAddress("0x0"), - second: base.HexToAddress("0x0"), + first: base.ZeroAddr, + second: base.ZeroAddr, reversed: false, int0: int0, int1: int1, diff --git a/src/apps/chifra/pkg/pricing/reporting.go b/src/apps/chifra/pkg/pricing/reporting.go index ae04bfab0f..aa8b6ec0bd 100644 --- a/src/apps/chifra/pkg/pricing/reporting.go +++ b/src/apps/chifra/pkg/pricing/reporting.go @@ -48,14 +48,14 @@ func (r *priceDebugger) report(msg string, testMode bool) { logger.TestLog(true, "blockNumber: ", r.blockNumber) logger.TestLog(true, "source1 (uni/mkr): ", r.source1) logger.TestLog(true, "str0 (call): ", r.theCall1) - if r.source2 != base.HexToAddress("0x0") { + if r.source2 != base.ZeroAddr { logger.TestLog(true, "source2 (pair): ", r.source2) } logger.TestLog(true, "str1 (call/data): ", r.theCall2) - if r.first != base.HexToAddress("0x0") { + if r.first != base.ZeroAddr { logger.TestLog(true, "first: ", r.first) } - if r.second != base.HexToAddress("0x0") { + if r.second != base.ZeroAddr { logger.TestLog(true, "second: ", r.second) logger.TestLog(true, "reversed: ", r.reversed) } diff --git a/src/apps/chifra/pkg/rpc/contract_integration_test.go b/src/apps/chifra/pkg/rpc/contract_integration_test.go index 11225c0839..68a752697d 100644 --- a/src/apps/chifra/pkg/rpc/contract_integration_test.go +++ b/src/apps/chifra/pkg/rpc/contract_integration_test.go @@ -43,7 +43,7 @@ func TestGetContractDeployBlock(t *testing.T) { } // Error when the address is not a contract - _, err = conn.GetContractDeployBlock(base.HexToAddress("0x0")) + _, err = conn.GetContractDeployBlock(base.ZeroAddr) if err != ErrNotAContract { t.Fatal("expected ErrNotAContract, but got", err) } diff --git a/src/apps/chifra/pkg/types/types_name.go b/src/apps/chifra/pkg/types/types_name.go index a9ea3505cf..5319ad5fc3 100644 --- a/src/apps/chifra/pkg/types/types_name.go +++ b/src/apps/chifra/pkg/types/types_name.go @@ -104,7 +104,7 @@ func (s *SimpleName) Model(verbose bool, format string, extraOptions map[string] "petname", } - if len(s.Address.Bytes()) > 0 && s.Address != base.HexToAddress("0x0") { + if len(s.Address.Bytes()) > 0 && s.Address != base.ZeroAddr { model["address"] = strings.ToLower(s.Address.String()) } diff --git a/src/apps/chifra/pkg/types/types_transaction_test.go b/src/apps/chifra/pkg/types/types_transaction_test.go index 60bb9d7b93..0695676465 100644 --- a/src/apps/chifra/pkg/types/types_transaction_test.go +++ b/src/apps/chifra/pkg/types/types_transaction_test.go @@ -22,7 +22,7 @@ func TestTransactionCache(t *testing.T) { Input: "0x", Nonce: 2377519, Receipt: &SimpleReceipt{ - ContractAddress: base.HexToAddress("0x0"), + ContractAddress: base.ZeroAddr, EffectiveGasPrice: 21000000000, GasUsed: 22966, Logs: []SimpleLog{ @@ -93,7 +93,7 @@ func TestTransactionCacheArticulated(t *testing.T) { Input: "0x1fee5cd200000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000a7365706f6c69612d747300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002e516d5441623866625233674d6f777876624e466f584339337a6762386f66395567756d3363554b564366414e3172000000000000000000000000000000000000", Nonce: 551, Receipt: &SimpleReceipt{ - ContractAddress: base.HexToAddress("0x0"), + ContractAddress: base.ZeroAddr, EffectiveGasPrice: 0, GasUsed: 41738, Logs: []SimpleLog{ diff --git a/src/dev_tools/utillib/conversions.cpp b/src/dev_tools/utillib/conversions.cpp index 6aca1f8ccb..e800e03abc 100644 --- a/src/dev_tools/utillib/conversions.cpp +++ b/src/dev_tools/utillib/conversions.cpp @@ -240,7 +240,7 @@ address_t topic_2_Addr(const topic_t& topic) { //-------------------------------------------------------------------------------- address_t str_2_Addr(const string_q& str) { - if (isZeroAddr(str)) + if (isZeroHash(str)) return "0x0"; string_q ret = substitute(str, "0x", ""); @@ -357,11 +357,6 @@ bool isEtherAddr(const address_t& addr) { return toLower(addr) == FAKE_ETH_ADDRESS; } -//-------------------------------------------------------------------------------- -bool isZeroAddr(const address_t& addr) { - return isZeroHash(addr); -} - //-------------------------------------------------------------------------------- bool isNumeral(const string_q& test) { for (size_t i = 0; i < test.length(); i++) diff --git a/src/dev_tools/utillib/conversions.h b/src/dev_tools/utillib/conversions.h index 03429822d9..ebc3fe9d33 100644 --- a/src/dev_tools/utillib/conversions.h +++ b/src/dev_tools/utillib/conversions.h @@ -122,7 +122,6 @@ extern timestamp_t date_2_Ts(const string_q& str); //-------------------------------------------------------------------- extern bool isZeroHash(const hash_t& hash); -extern bool isZeroAddr(const address_t& addr); extern bool isEtherAddr(const address_t& addr); extern bool isNumeral(const string_q& test); extern bool isDouble(const string_q& test); From f5d618ecd70ad2e91a0c32110222584f6774be09 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Sat, 9 Sep 2023 04:47:43 +0200 Subject: [PATCH 26/33] Forces use of publisher for chifra init and chifra chunks --- src/apps/chifra/internal/chunks/handle_check.go | 4 ++-- src/apps/chifra/internal/chunks/handle_manifest.go | 3 ++- src/apps/chifra/internal/init/handle_dryrun.go | 3 ++- src/apps/chifra/internal/init/handle_init.go | 3 ++- .../chifra/internal/scrape/scrape_consolidate.go | 3 ++- src/apps/chifra/internal/scrape/scrape_prepare.go | 4 +++- src/apps/chifra/pkg/index/chunk_write.go | 4 ++-- src/apps/chifra/pkg/manifest/download.go | 14 ++++++-------- src/apps/chifra/pkg/manifest/manifest.go | 11 +++++++---- src/apps/chifra/pkg/tslib/establish.go | 4 +++- 10 files changed, 31 insertions(+), 22 deletions(-) diff --git a/src/apps/chifra/internal/chunks/handle_check.go b/src/apps/chifra/internal/chunks/handle_check.go index 9f4c9f233e..ce86df0074 100644 --- a/src/apps/chifra/internal/chunks/handle_check.go +++ b/src/apps/chifra/internal/chunks/handle_check.go @@ -71,12 +71,12 @@ func (opts *ChunksOptions) HandleCheck(blockNums []uint64) error { return fileNames[i] < fileNames[j] }) - cacheManifest, err := manifest.ReadManifest(chain, manifest.FromCache) + cacheManifest, err := manifest.ReadManifest(chain, base.HexToAddress(opts.Publisher), manifest.FromCache) if err != nil { return err } - remoteManifest, err := manifest.ReadManifest(chain, manifest.FromContract) + remoteManifest, err := manifest.ReadManifest(chain, base.HexToAddress(opts.Publisher), manifest.FromContract) if err != nil { return err } diff --git a/src/apps/chifra/internal/chunks/handle_manifest.go b/src/apps/chifra/internal/chunks/handle_manifest.go index 279ff2ed87..58450463b2 100644 --- a/src/apps/chifra/internal/chunks/handle_manifest.go +++ b/src/apps/chifra/internal/chunks/handle_manifest.go @@ -7,6 +7,7 @@ package chunksPkg import ( "context" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/manifest" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/output" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/types" @@ -20,7 +21,7 @@ var sourceMap = map[bool]manifest.Source{ func (opts *ChunksOptions) HandleManifest(blockNums []uint64) error { chain := opts.Globals.Chain testMode := opts.Globals.TestMode - man, err := manifest.ReadManifest(chain, sourceMap[opts.Remote]) + man, err := manifest.ReadManifest(chain, base.HexToAddress(opts.Publisher), sourceMap[opts.Remote]) if err != nil { return err } diff --git a/src/apps/chifra/internal/init/handle_dryrun.go b/src/apps/chifra/internal/init/handle_dryrun.go index 8cdd4d9a6c..2e6a02f13b 100644 --- a/src/apps/chifra/internal/init/handle_dryrun.go +++ b/src/apps/chifra/internal/init/handle_dryrun.go @@ -5,6 +5,7 @@ import ( "errors" "fmt" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/manifest" @@ -16,7 +17,7 @@ import ( func (opts *InitOptions) HandleDryRun() error { chain := opts.Globals.Chain - remoteManifest, err := manifest.ReadManifest(chain, manifest.FromContract) + remoteManifest, err := manifest.ReadManifest(chain, base.HexToAddress(opts.Publisher), manifest.FromContract) if err != nil { return err } diff --git a/src/apps/chifra/internal/init/handle_init.go b/src/apps/chifra/internal/init/handle_init.go index ac47dd35b0..9b966e5d18 100644 --- a/src/apps/chifra/internal/init/handle_init.go +++ b/src/apps/chifra/internal/init/handle_init.go @@ -12,6 +12,7 @@ import ( "sync" "time" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/colors" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/index" @@ -32,7 +33,7 @@ func (opts *InitOptions) HandleInit() error { // scraper starts, it starts on the correct block. _ = index.CleanTempIndexFolders(chain, []string{"ripe", "unripe", "staging"}) - remoteManifest, err := manifest.ReadManifest(chain, manifest.FromContract) + remoteManifest, err := manifest.ReadManifest(chain, base.HexToAddress(opts.Publisher), manifest.FromContract) if err != nil { return err } diff --git a/src/apps/chifra/internal/scrape/scrape_consolidate.go b/src/apps/chifra/internal/scrape/scrape_consolidate.go index 40443d5a92..1273c73e87 100644 --- a/src/apps/chifra/internal/scrape/scrape_consolidate.go +++ b/src/apps/chifra/internal/scrape/scrape_consolidate.go @@ -102,7 +102,8 @@ func (bm *BlazeManager) Consolidate(blocks []base.Blknum) (error, bool) { if isSnap || isOvertop { // Make a chunk - i.e., consolidate chunkPath := indexPath + "finalized/" + chunkRange.String() + ".bin" - if report, err := index.WriteChunk(chain, chunkPath, appMap, nAppearances, bm.opts.Pin, bm.opts.Remote); err != nil { + publisher := base.ZeroAddr + if report, err := index.WriteChunk(chain, chunkPath, publisher, appMap, nAppearances, bm.opts.Pin, bm.opts.Remote); err != nil { return err, false } else if report == nil { logger.Fatal("Should not happen, write chunk returned empty report") diff --git a/src/apps/chifra/internal/scrape/scrape_prepare.go b/src/apps/chifra/internal/scrape/scrape_prepare.go index e9230301fe..8ab8786cb6 100644 --- a/src/apps/chifra/internal/scrape/scrape_prepare.go +++ b/src/apps/chifra/internal/scrape/scrape_prepare.go @@ -7,6 +7,7 @@ package scrapePkg import ( "path/filepath" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/file" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/index" @@ -56,7 +57,8 @@ func (opts *ScrapeOptions) Prepare() (ok bool, err error) { logger.Info("Writing block zero allocations for", len(prefunds), "prefunds, nAddresses:", len(appMap)) indexPath := index.ToIndexPath(bloomPath) - if report, err := index.WriteChunk(chain, indexPath, appMap, len(prefunds), opts.Pin, opts.Remote); err != nil { + publisher := base.ZeroAddr + if report, err := index.WriteChunk(chain, indexPath, publisher, appMap, len(prefunds), opts.Pin, opts.Remote); err != nil { return false, err } else if report == nil { logger.Fatal("Should not happen, write chunk returned empty report") diff --git a/src/apps/chifra/pkg/index/chunk_write.go b/src/apps/chifra/pkg/index/chunk_write.go index 3652bd485e..91fe2a451a 100644 --- a/src/apps/chifra/pkg/index/chunk_write.go +++ b/src/apps/chifra/pkg/index/chunk_write.go @@ -46,7 +46,7 @@ func (c *WriteChunkReport) Report() { } } -func WriteChunk(chain, fileName string, addrAppearanceMap AddressAppearanceMap, nApps int, pin, remote bool) (*WriteChunkReport, error) { +func WriteChunk(chain, fileName string, publisher base.Address, addrAppearanceMap AddressAppearanceMap, nApps int, pin, remote bool) (*WriteChunkReport, error) { // We're going to build two tables. An addressTable and an appearanceTable. We do this as we spin // through the map @@ -167,7 +167,7 @@ func WriteChunk(chain, fileName string, addrAppearanceMap AddressAppearanceMap, report.PinRecord.BloomHash = rec.BloomHash report.PinRecord.IndexSize = rec.IndexSize report.PinRecord.BloomSize = rec.BloomSize - return &report, manifest.UpdateManifest(chain, rec) + return &report, manifest.UpdateManifest(chain, publisher, rec) } else { return nil, err diff --git a/src/apps/chifra/pkg/manifest/download.go b/src/apps/chifra/pkg/manifest/download.go index 677d33ad35..0abf077f4d 100644 --- a/src/apps/chifra/pkg/manifest/download.go +++ b/src/apps/chifra/pkg/manifest/download.go @@ -25,8 +25,9 @@ import ( // fromRemote gets the CID from the smart contract, calls // the gateway and returns the parsed manifest -func fromRemote(chain string) (*Manifest, error) { - cid, err := ReadUnchainedIndex(chain, "", unchained.GetPreferredPublisher()) +func fromRemote(chain string, publisher base.Address) (*Manifest, error) { + database := chain + cid, err := ReadUnchainedIndex(chain, publisher, database) if err != nil { return nil, err } @@ -35,6 +36,8 @@ func fromRemote(chain string) (*Manifest, error) { logger.InfoTable("Chain:", chain) logger.InfoTable("Gateway:", gatewayUrl) + logger.InfoTable("Publisher:", publisher) + logger.InfoTable("Database:", database) logger.InfoTable("CID:", cid) return downloadManifest(chain, gatewayUrl, cid) @@ -42,17 +45,12 @@ func fromRemote(chain string) (*Manifest, error) { // ReadUnchainedIndex calls UnchainedIndex smart contract to get the current manifest IPFS CID as // published by the given publisher -func ReadUnchainedIndex(chain, reason string, publisher base.Address) (string, error) { +func ReadUnchainedIndex(chain string, publisher base.Address, database string) (string, error) { cid := os.Getenv("TB_OVERRIDE_CID") if cid != "" { return cid, nil } - database := chain - if reason != "" { - database += ("-" + reason) - } - unchainedChain := "mainnet" // the unchained index is on mainnet theCall := fmt.Sprintf("manifestHashMap(%s, \"%s\")", publisher, database) conn := rpc.TempConnection(unchainedChain) diff --git a/src/apps/chifra/pkg/manifest/manifest.go b/src/apps/chifra/pkg/manifest/manifest.go index 5aa8753493..766171b663 100644 --- a/src/apps/chifra/pkg/manifest/manifest.go +++ b/src/apps/chifra/pkg/manifest/manifest.go @@ -64,9 +64,12 @@ const ( var ErrManifestNotFound = errors.New("could not find manifest.json or it was empty") // ReadManifest reads the manifest from either the local cache or the Unchained Index smart contract -func ReadManifest(chain string, source Source) (*Manifest, error) { +func ReadManifest(chain string, publisher base.Address, source Source) (*Manifest, error) { if source == FromContract { - man, err := fromRemote(chain) + if publisher.IsZero() { + publisher = unchained.GetPreferredPublisher() + } + man, err := fromRemote(chain, publisher) if man != nil { man.LoadChunkMap() } @@ -98,8 +101,8 @@ func (m *Manifest) LoadChunkMap() { // TODO: Protect against overwriting files on disc -func UpdateManifest(chain string, chunk ChunkRecord) error { - man, err := ReadManifest(chain, FromCache) +func UpdateManifest(chain string, publisher base.Address, chunk ChunkRecord) error { + man, err := ReadManifest(chain, publisher, FromCache) if err != nil { if err != ErrManifestNotFound { return err diff --git a/src/apps/chifra/pkg/tslib/establish.go b/src/apps/chifra/pkg/tslib/establish.go index cd9561c89d..860053d710 100644 --- a/src/apps/chifra/pkg/tslib/establish.go +++ b/src/apps/chifra/pkg/tslib/establish.go @@ -30,7 +30,9 @@ func EstablishTsFile(chain string) error { return nil } - cid, err := manifest.ReadUnchainedIndex(chain, "ts", unchained.GetPreferredPublisher()) + database := chain + "-ts" + publisher := unchained.GetPreferredPublisher() + cid, err := manifest.ReadUnchainedIndex(chain, publisher, database) if err != nil { return err } From 02a3d876bcdc7c5753dc72678a9f8c2a6de459d4 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Sun, 10 Sep 2023 04:01:16 +0200 Subject: [PATCH 27/33] Adds chifra config edit --- src/apps/chifra/internal/config/output.go | 2 ++ src/apps/chifra/internal/config/validate.go | 7 +++++++ 2 files changed, 9 insertions(+) diff --git a/src/apps/chifra/internal/config/output.go b/src/apps/chifra/internal/config/output.go index 739fd564ea..ef932a8e62 100644 --- a/src/apps/chifra/internal/config/output.go +++ b/src/apps/chifra/internal/config/output.go @@ -60,6 +60,8 @@ func (opts *ConfigOptions) ConfigInternal() (err error, handled bool) { handled = true if opts.Paths { err = opts.HandlePaths() + } else if opts.Mode == "edit" { + err = opts.HandleEdit() } else { logger.Warn("The config tool is current unavailable. Please use the 'chifra status' tool instead.") } diff --git a/src/apps/chifra/internal/config/validate.go b/src/apps/chifra/internal/config/validate.go index 26f4586917..8947bc7c75 100644 --- a/src/apps/chifra/internal/config/validate.go +++ b/src/apps/chifra/internal/config/validate.go @@ -5,6 +5,9 @@ package configPkg import ( + "fmt" + "os" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/validate" ) @@ -20,5 +23,9 @@ func (opts *ConfigOptions) validateConfig() error { return err } + if opts.Mode == "edit" && os.Getenv("EDITOR") == "" { + return fmt.Errorf("You must set the EDITOR environment variable to use the 'edit' mode.") + } + return opts.Globals.Validate() } From 60d5926a36dc2633c920d01fd5e677bb6660a1fd Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Sun, 10 Sep 2023 04:01:38 +0200 Subject: [PATCH 28/33] Adds chifra config edit --- src/apps/chifra/internal/config/handle_edit.go | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 src/apps/chifra/internal/config/handle_edit.go diff --git a/src/apps/chifra/internal/config/handle_edit.go b/src/apps/chifra/internal/config/handle_edit.go new file mode 100644 index 0000000000..5e20c65583 --- /dev/null +++ b/src/apps/chifra/internal/config/handle_edit.go @@ -0,0 +1,17 @@ +package configPkg + +import ( + "os" + "path/filepath" + + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" +) + +func (opts *ConfigOptions) HandleEdit() error { + editor := os.Getenv("EDITOR") + configFolder := config.GetPathToRootConfig() + configFile := filepath.Join(configFolder + "trueBlocks.toml") + utils.System(editor + " " + configFile) + return nil +} From 00dbebe0625907109501df9de907f39130b87a5f Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Mon, 11 Sep 2023 22:04:52 +0200 Subject: [PATCH 29/33] Removes apiProvider as unused --- docs/content/api/openapi.yaml | 3 -- docs/content/data-model/admin.md | 19 ++++--- docs/templates/api/components.txt | 3 -- sdk/typescript/src/types/chain.ts | 1 - src/apps/chifra/cmd/root_initialize.go | 2 +- .../chifra/internal/config/handle_edit.go | 2 +- src/apps/chifra/internal/scrape/validate.go | 5 +- .../chifra/internal/status/types_status.go | 1 - src/apps/chifra/pkg/abi/load_abi.go | 2 +- src/apps/chifra/pkg/config/per_chain.go | 6 --- src/apps/chifra/pkg/config/root_config.go | 3 +- src/apps/chifra/pkg/types/types_chain.go | 4 -- src/apps/chifra/pkg/version/version.go | 9 ++-- src/dev_tools/testRunner/options.cpp | 1 + .../classDefinitions/fields/chain.csv | 7 ++- src/other/install/trueBlocks.toml | 54 ++++++++----------- .../cacheStatus_items_blocks_chains.txt | 1 - .../cacheStatus_items_blocks_chains.txt | 1 - .../gold/apps/cacheStatus/tests/ethslurp.toml | 1 - 19 files changed, 46 insertions(+), 79 deletions(-) diff --git a/docs/content/api/openapi.yaml b/docs/content/api/openapi.yaml index 385f804c73..eae26cea5d 100644 --- a/docs/content/api/openapi.yaml +++ b/docs/content/api/openapi.yaml @@ -3872,9 +3872,6 @@ components: rpcProvider: type: string description: "A valid RPC provider for the chain" - apiProvider: - type: string - description: "A valid API provider for the explorer" remoteExplorer: type: string description: "A remote explorer for the chain such as Etherscan" diff --git a/docs/content/data-model/admin.md b/docs/content/data-model/admin.md index 33ae0ef3f7..e8d3bc33a2 100644 --- a/docs/content/data-model/admin.md +++ b/docs/content/data-model/admin.md @@ -234,15 +234,15 @@ The following commands produce and manage CacheItems: CacheItems consist of the following fields: -| Field | Description | Type | -| ----------- | --------------------------------------------------- | --------- | -| type | the type of the cache | string | -| items | the individual items in the cache (if --verbose) | Any | -| lastCached | the date of the most recent item added to the cache | string | -| nFiles | the number of items in the cache | uint64 | -| nFolders | the number of folders holding that many items | uint64 | -| path | the path to the top of the given cache | string | -| sizeInBytes | the size of the cache in bytes | int64 | +| Field | Description | Type | +| ----------- | --------------------------------------------------- | ------ | +| type | the type of the cache | string | +| items | the individual items in the cache (if --verbose) | Any | +| lastCached | the date of the most recent item added to the cache | string | +| nFiles | the number of items in the cache | uint64 | +| nFolders | the number of folders holding that many items | uint64 | +| path | the path to the top of the given cache | string | +| sizeInBytes | the size of the cache in bytes | int64 | ## ReportCheck @@ -304,7 +304,6 @@ Chains consist of the following fields: | chainId | The chain id as reported by the RPC | uint64 | | symbol | The symbol of the base currency on the chain | string | | rpcProvider | A valid RPC provider for the chain | string | -| apiProvider | A valid API provider for the explorer | string | | remoteExplorer | A remote explorer for the chain such as Etherscan | string | | localExplorer | The local explorer for the chain (typically TrueBlocks Explorer) | string | | ipfsGateway | An IPFS gateway for pinning the index if enabled | string | diff --git a/docs/templates/api/components.txt b/docs/templates/api/components.txt index 5a7f465703..1e857c1f62 100644 --- a/docs/templates/api/components.txt +++ b/docs/templates/api/components.txt @@ -1494,9 +1494,6 @@ components: rpcProvider: type: string description: "A valid RPC provider for the chain" - apiProvider: - type: string - description: "A valid API provider for the explorer" remoteExplorer: type: string description: "A remote explorer for the chain such as Etherscan" diff --git a/sdk/typescript/src/types/chain.ts b/sdk/typescript/src/types/chain.ts index 0795ff8605..bf0bde2b8c 100644 --- a/sdk/typescript/src/types/chain.ts +++ b/sdk/typescript/src/types/chain.ts @@ -10,7 +10,6 @@ export type Chain = { chainId: uint64 symbol: string rpcProvider: string - apiProvider: string remoteExplorer: string localExplorer: string ipfsGateway: string diff --git a/src/apps/chifra/cmd/root_initialize.go b/src/apps/chifra/cmd/root_initialize.go index 4dd222e07c..5164d8aa27 100644 --- a/src/apps/chifra/cmd/root_initialize.go +++ b/src/apps/chifra/cmd/root_initialize.go @@ -183,7 +183,7 @@ func VerifyMigrations() { } // We need at least this version... - requiredVersion := "v0.40.0-beta" + requiredVersion := "v1.0.0-release" if !config.IsAtLeastVersion(requiredVersion) { msg := strings.Replace(backVersion, "{0}", "{"+requiredVersion+"}", -1) msg = strings.Replace(msg, "[{VERSION}]", versionText, -1) diff --git a/src/apps/chifra/internal/config/handle_edit.go b/src/apps/chifra/internal/config/handle_edit.go index 5e20c65583..b19bedb864 100644 --- a/src/apps/chifra/internal/config/handle_edit.go +++ b/src/apps/chifra/internal/config/handle_edit.go @@ -12,6 +12,6 @@ func (opts *ConfigOptions) HandleEdit() error { editor := os.Getenv("EDITOR") configFolder := config.GetPathToRootConfig() configFile := filepath.Join(configFolder + "trueBlocks.toml") - utils.System(editor + " " + configFile) + utils.System(editor + " \"" + configFile + "\"") return nil } diff --git a/src/apps/chifra/internal/scrape/validate.go b/src/apps/chifra/internal/scrape/validate.go index d51c9f4a5e..3a57971a03 100644 --- a/src/apps/chifra/internal/scrape/validate.go +++ b/src/apps/chifra/internal/scrape/validate.go @@ -11,7 +11,6 @@ import ( "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config/scrapeCfg" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/index" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/pinning" - "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/rpc" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/validate" ) @@ -32,11 +31,11 @@ func (opts *ScrapeOptions) validateScrape() error { } if !opts.Conn.IsNodeTracing() { - return validate.Usage("{0} requires tracing, err: {1}", "chifra scrape", rpc.ErrTraceBlockMissing) + return validate.Usage("{0} requires {1}, try {2} instead.", "chifra scrape", "tracing", "chifra init") } if !opts.Conn.IsNodeArchive() { - return validate.Usage("{0} requires {1}.", "chifra scrape", "an archive node") + return validate.Usage("{0} requires {1}, try {2} instead.", "chifra scrape", "an archive node", "chifra init") } if opts.Sleep < .25 { diff --git a/src/apps/chifra/internal/status/types_status.go b/src/apps/chifra/internal/status/types_status.go index 6b7defbbbe..884665461c 100644 --- a/src/apps/chifra/internal/status/types_status.go +++ b/src/apps/chifra/internal/status/types_status.go @@ -132,7 +132,6 @@ func (s *simpleStatus) Model(verbose bool, format string, extraOptions map[strin LocalExplorer: chain.LocalExplorer, RemoteExplorer: chain.RemoteExplorer, RpcProvider: chain.RpcProvider, - ApiProvider: chain.ApiProvider, IpfsGateway: chain.IpfsGateway, Symbol: chain.Symbol, } diff --git a/src/apps/chifra/pkg/abi/load_abi.go b/src/apps/chifra/pkg/abi/load_abi.go index 36e8c0657f..fdc7bf32fe 100644 --- a/src/apps/chifra/pkg/abi/load_abi.go +++ b/src/apps/chifra/pkg/abi/load_abi.go @@ -750,7 +750,7 @@ func LoadAbiFromAddress(chain string, address base.Address, destination *Functio if err = fromJson(localFile, destination); err != nil { return } - // File is correct + // File is correct, cache it if err = insertAbi(chain, address, localFile); err != nil { return } diff --git a/src/apps/chifra/pkg/config/per_chain.go b/src/apps/chifra/pkg/config/per_chain.go index d70085f759..bf340c790f 100644 --- a/src/apps/chifra/pkg/config/per_chain.go +++ b/src/apps/chifra/pkg/config/per_chain.go @@ -49,12 +49,6 @@ func GetRpcProvider(chain string) (string, error) { return cleaned, nil } -// GetApiProvider returns the RPC provider for a chain -func GetApiProvider(chain string) string { - ch := GetRootConfig().Chains[chain] - return cleanUrl(ch.ApiProvider) -} - // GetSymbol returns the expected chain id for a given chain func GetSymbol(chain string) string { ch := GetRootConfig().Chains[chain] diff --git a/src/apps/chifra/pkg/config/root_config.go b/src/apps/chifra/pkg/config/root_config.go index afb58641b6..9380a74079 100644 --- a/src/apps/chifra/pkg/config/root_config.go +++ b/src/apps/chifra/pkg/config/root_config.go @@ -32,7 +32,6 @@ type chainGroup struct { LocalExplorer string `toml:"localExplorer"` RemoteExplorer string `toml:"remoteExplorer"` RpcProvider string `toml:"rpcProvider"` - ApiProvider string `toml:"apiProvider"` IpfsGateway string `toml:"ipfsGateway"` Symbol string `toml:"symbol"` } @@ -138,7 +137,7 @@ func IsAtLeastVersion(needle string) bool { return true } - return !current.IsEarlierThan(desired) + return desired.IsEarlierThan(current) || desired == current } // GetPathToRootConfig returns the path where to find configuration files diff --git a/src/apps/chifra/pkg/types/types_chain.go b/src/apps/chifra/pkg/types/types_chain.go index 9e4600d4ec..b8565b6e83 100644 --- a/src/apps/chifra/pkg/types/types_chain.go +++ b/src/apps/chifra/pkg/types/types_chain.go @@ -13,7 +13,6 @@ package types // EXISTING_CODE type RawChain struct { - ApiProvider string `json:"apiProvider"` Chain string `json:"chain"` ChainId string `json:"chainId"` IpfsGateway string `json:"ipfsGateway"` @@ -26,7 +25,6 @@ type RawChain struct { } type SimpleChain struct { - ApiProvider string `json:"apiProvider"` Chain string `json:"chain"` ChainId uint64 `json:"chainId"` IpfsGateway string `json:"ipfsGateway"` @@ -53,7 +51,6 @@ func (s *SimpleChain) Model(verbose bool, format string, extraOptions map[string // EXISTING_CODE model = map[string]interface{}{ - "apiProvider": s.ApiProvider, "chain": s.Chain, "chainId": s.ChainId, "ipfsGateway": s.IpfsGateway, @@ -63,7 +60,6 @@ func (s *SimpleChain) Model(verbose bool, format string, extraOptions map[string "symbol": s.Symbol, } order = []string{ - "apiProvider", "chain", "chainId", "ipfsGateway", diff --git a/src/apps/chifra/pkg/version/version.go b/src/apps/chifra/pkg/version/version.go index b09a63bbb6..ce9c8c6983 100644 --- a/src/apps/chifra/pkg/version/version.go +++ b/src/apps/chifra/pkg/version/version.go @@ -17,27 +17,28 @@ type Version struct { } func NewVersion(str string) (vers Version, err error) { - if str[:3] == "GHC" { - // First, remove the part that cannot be parsed with ParseInt - str = str[len("GHC-TrueBlocks//"):] - } + str = strings.Replace(str, "GHC-TrueBlocks//", "", -1) str = strings.Replace(strings.Replace(str, "-", ".", -1), "v", "", -1) parts := strings.Split(str, ".") + if len(parts) > 0 { if vers.Major, err = strconv.ParseInt(parts[0], 10, 32); err != nil { return vers, err } } + if len(parts) > 1 { if vers.Minor, err = strconv.ParseInt(parts[1], 10, 32); err != nil { return vers, err } } + if len(parts) > 2 { if vers.Build, err = strconv.ParseInt(parts[2], 10, 32); err != nil { return vers, err } } + if len(parts) > 3 { vers.Aspect = parts[3] } diff --git a/src/dev_tools/testRunner/options.cpp b/src/dev_tools/testRunner/options.cpp index 0ae45749b3..471ff15596 100644 --- a/src/dev_tools/testRunner/options.cpp +++ b/src/dev_tools/testRunner/options.cpp @@ -167,6 +167,7 @@ bool COptions::parseArguments(string_q& command) { SHOW_FIELD(CTestCase, "test_id"); + // Note that if this value is not in the config file, add it apiProvider = getGlobalConfig("testRunner")->getConfigStr("settings", "api_provider", "http://localhost:8080"); if (!endsWith(apiProvider, '/')) apiProvider += "/"; diff --git a/src/other/data-models/classDefinitions/fields/chain.csv b/src/other/data-models/classDefinitions/fields/chain.csv index cea81967de..82c40d6403 100644 --- a/src/other/data-models/classDefinitions/fields/chain.csv +++ b/src/other/data-models/classDefinitions/fields/chain.csv @@ -3,7 +3,6 @@ chain ,string , , , , , chainId ,uint64 , , , , , , , ,2 , , ,The chain id as reported by the RPC symbol ,string , , , , , , , ,3 , , ,The symbol of the base currency on the chain rpcProvider ,string ,http://localhost:8545 , , , , , , ,4 , , ,A valid RPC provider for the chain -apiProvider ,string ,http://localhost:8080 , , , , , , ,5 , , ,A valid API provider for the explorer -remoteExplorer ,string ,http://etherscan.io , , , , , , ,6 , , ,A remote explorer for the chain such as Etherscan -localExplorer ,string ,http://localhost:1234 , , , , , , ,7 , , ,The local explorer for the chain (typically TrueBlocks Explorer) -ipfsGateway ,string ,http://gateway.ipfs.io/ipfs , , , , , , ,8 , , ,An IPFS gateway for pinning the index if enabled +remoteExplorer ,string ,http://etherscan.io , , , , , , ,5 , , ,A remote explorer for the chain such as Etherscan +localExplorer ,string ,http://localhost:1234 , , , , , , ,6 , , ,The local explorer for the chain (typically TrueBlocks Explorer) +ipfsGateway ,string ,http://gateway.ipfs.io/ipfs , , , , , , ,7 , , ,An IPFS gateway for pinning the index if enabled diff --git a/src/other/install/trueBlocks.toml b/src/other/install/trueBlocks.toml index eaf3afe85d..f678ac6ea9 100644 --- a/src/other/install/trueBlocks.toml +++ b/src/other/install/trueBlocks.toml @@ -1,10 +1,10 @@ [version] -current = "v0.40.0-beta" +current = "v1.0.0-release" [settings] -cachePath = "" defaultChain = "mainnet" defaultGateway = "https://ipfs.unchainedindex.io/ipfs/" +cachePath = "" indexPath = "" [keys] @@ -17,55 +17,45 @@ apiKey = "" jwt = "" secret = "" +[keys.trueblocks] +license = "" + #[dev] #debug_curl=true [chains] [chains.mainnet] -apiProvider = "http://localhost:8080" -chainId = "1" -localExplorer = "http://localhost:1234" -remoteExplorer = "https://etherscan.io" rpcProvider = "http://localhost:8545" -symbol = "ETH" - -[chains.gnosis] -apiProvider = "http://localhost:8080" -chainId = "100" +remoteExplorer = "https://etherscan.io" localExplorer = "http://localhost:1234" -remoteExplorer = "https://gnosisscan.io/" -rpcProvider = "https://gnosischain-rpc.gateway.pokt.network" -symbol = "XDAI" +symbol = "ETH" +chainId = "1" [chains.sepolia] -apiProvider = "http://localhost:8080" -chainId = "11155111" -localExplorer = "http://localhost:1234" -remoteExplorer = "https://sepolia.otterscan.io/" rpcProvider = "http://localhost:8548" +remoteExplorer = "https://sepolia.otterscan.io/" +localExplorer = "http://localhost:1234" symbol = "ETH" +chainId = "11155111" -[chains.goerli] -apiProvider = "http://localhost:8080" -chainId = "5" +[chains.gnosis] +rpcProvider = "https://gnosischain-rpc.gateway.pokt.network" +remoteExplorer = "https://gnosisscan.io/" localExplorer = "http://localhost:1234" -remoteExplorer = "https://goerli.etherscan.io" -rpcProvider = "https://goerli-light.eth.linkpool.io" -symbol = "ETH" +symbol = "XDAI" +chainId = "100" [chains.optimism] -apiProvider = "http://localhost:8080" -chainId = "10" -localExplorer = "http://localhost:1234" -remoteExplorer = "https://optimistic.etherscan.io" rpcProvider = "https://mainnet.optimism.io" +remoteExplorer = "https://optimistic.etherscan.io" +localExplorer = "http://localhost:1234" symbol = "OPT" +chainId = "10" [chains.polygon] -apiProvider = "http://localhost:8080" -chainId = "80001" -localExplorer = "http://localhost:1234" -remoteExplorer = "https://mumbai.polygonscan.com" rpcProvider = "https://rpc-mumbai.maticvigil.com" +remoteExplorer = "https://mumbai.polygonscan.com" +localExplorer = "http://localhost:1234" symbol = "MATIC" +chainId = "80001" diff --git a/test/gold/apps/cacheStatus/api_tests/cacheStatus_items_blocks_chains.txt b/test/gold/apps/cacheStatus/api_tests/cacheStatus_items_blocks_chains.txt index 211b140786..746a3637ff 100644 --- a/test/gold/apps/cacheStatus/api_tests/cacheStatus_items_blocks_chains.txt +++ b/test/gold/apps/cacheStatus/api_tests/cacheStatus_items_blocks_chains.txt @@ -17,7 +17,6 @@ status?modes=blocks&maxRecords=100&chains "chainConfig": "--paths--", "chains": [ { - "apiProvider": "", "chain": "testChain", "chainId": 12345, "ipfsGateway": "", diff --git a/test/gold/apps/cacheStatus/cacheStatus_items_blocks_chains.txt b/test/gold/apps/cacheStatus/cacheStatus_items_blocks_chains.txt index 68305f3715..bd7a55716a 100644 --- a/test/gold/apps/cacheStatus/cacheStatus_items_blocks_chains.txt +++ b/test/gold/apps/cacheStatus/cacheStatus_items_blocks_chains.txt @@ -21,7 +21,6 @@ TEST[DATE|TIME] Format: json "chainConfig": "--paths--", "chains": [ { - "apiProvider": "", "chain": "testChain", "chainId": 12345, "ipfsGateway": "", diff --git a/test/gold/apps/cacheStatus/tests/ethslurp.toml b/test/gold/apps/cacheStatus/tests/ethslurp.toml index 1c2cf14159..360e03d952 100644 --- a/test/gold/apps/cacheStatus/tests/ethslurp.toml +++ b/test/gold/apps/cacheStatus/tests/ethslurp.toml @@ -1,5 +1,4 @@ [settings] -apiProvider = "EtherScan" api_key = "" api_url = "http://etherscan.io/apis" From a5566a37ba212171b3471714222014e2874562ca Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Tue, 12 Sep 2023 07:52:12 +0200 Subject: [PATCH 30/33] Working --- src/CMakeLists.txt | 2 +- src/apps/chifra/internal/config/handle_edit.go | 15 +++++++++++---- .../internal/scrape/scrape_manager_utils.go | 3 ++- src/apps/chifra/pkg/base/known_blocks.go | 7 +++++++ src/apps/chifra/pkg/rpc/get_block.go | 4 ++-- src/apps/chifra/pkg/rpc/get_transaction.go | 5 ----- src/apps/chifra/pkg/rpc/get_uncle.go | 16 ++++++++++------ src/apps/chifra/pkg/types/types_transaction.go | 3 +-- .../api_tests/blockScrape_no_tracing.txt | 2 +- test/gold/apps/config/config_config_edit.txt | 1 - test/gold/tools/getLogs/getLogs_three_ways.txt | 2 +- 11 files changed, 36 insertions(+), 24 deletions(-) create mode 100644 src/apps/chifra/pkg/base/known_blocks.go diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 7e6655cddb..9d124ffbbd 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -96,7 +96,7 @@ add_custom_target(test-all COMMAND "${BIN_TEST_DIR}/test-all.sh" WORKING_DIRECTO # ---------------------------------------------------------------------------------------- # Compilation order for the subfolders -add_subdirectory(dev_tools) +# add_subdirectory(dev_tools) add_subdirectory(apps) add_subdirectory(examples) add_subdirectory(other) diff --git a/src/apps/chifra/internal/config/handle_edit.go b/src/apps/chifra/internal/config/handle_edit.go index b19bedb864..7b3e808217 100644 --- a/src/apps/chifra/internal/config/handle_edit.go +++ b/src/apps/chifra/internal/config/handle_edit.go @@ -5,13 +5,20 @@ import ( "path/filepath" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/logger" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/utils" ) func (opts *ConfigOptions) HandleEdit() error { - editor := os.Getenv("EDITOR") - configFolder := config.GetPathToRootConfig() - configFile := filepath.Join(configFolder + "trueBlocks.toml") - utils.System(editor + " \"" + configFile + "\"") + testMode := opts.Globals.TestMode + if testMode { + logger.Info("would have opened config file for edit") + } else { + editor := os.Getenv("EDITOR") + configFolder := config.GetPathToRootConfig() + configFile := filepath.Join(configFolder + "trueBlocks.toml") + utils.System(editor + " \"" + configFile + "\"") + } + return nil } diff --git a/src/apps/chifra/internal/scrape/scrape_manager_utils.go b/src/apps/chifra/internal/scrape/scrape_manager_utils.go index 7da61b1b73..0440c487d6 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager_utils.go +++ b/src/apps/chifra/internal/scrape/scrape_manager_utils.go @@ -23,7 +23,8 @@ func (bm *BlazeManager) report(nBlocks, perChunk, nChunks, nAppsNow, nAppsFound, appsPerAddr := float64(nAppsFound) / float64(nAddrsFound) pctFull := float64(nAppsNow) / float64(perChunk) - msg := fmt.Sprintf(`#{%d}, found {%6d} apps, {%5d} addrs ({%0.1f/addr}), in {%4d} blks ({%0.1f}/blk). Created {%d} chunks, staged {%5d} of {%d} ({%0.1f%%}). Need {%5d} more.`, + msg := fmt.Sprintf(`%s #{%d}, found {%6d} apps, {%5d} addrs ({%0.1f/addr}), in {%4d} blks ({%0.1f}/blk). Created {%d} chunks, staged {%5d} of {%d} ({%0.1f%%}). Need {%5d} more.`, + bm.chain, bm.EndBlock(), nAppsFound, nAddrsFound, diff --git a/src/apps/chifra/pkg/base/known_blocks.go b/src/apps/chifra/pkg/base/known_blocks.go new file mode 100644 index 0000000000..2dd4f89b28 --- /dev/null +++ b/src/apps/chifra/pkg/base/known_blocks.go @@ -0,0 +1,7 @@ +package base + +const ( + ByzantiumBlock = Blknum(4370000) + ConstantinopleBlock = Blknum(7280000) + LondonBlock = Blknum(12965000) +) diff --git a/src/apps/chifra/pkg/rpc/get_block.go b/src/apps/chifra/pkg/rpc/get_block.go index 0cccd29d48..9a6d15cc46 100644 --- a/src/apps/chifra/pkg/rpc/get_block.go +++ b/src/apps/chifra/pkg/rpc/get_block.go @@ -273,9 +273,9 @@ func (conn *Connection) getBlockRaw(bn uint64, withTxs bool) (*types.RawBlock, e func (conn *Connection) getBlockReward(bn uint64) *big.Int { if bn == 0 { return big.NewInt(0) - } else if bn < byzantiumBlock { + } else if bn < base.ByzantiumBlock { return big.NewInt(5000000000000000000) - } else if bn < constantinopleBlock { + } else if bn < base.ConstantinopleBlock { return big.NewInt(3000000000000000000) } else { return big.NewInt(2000000000000000000) diff --git a/src/apps/chifra/pkg/rpc/get_transaction.go b/src/apps/chifra/pkg/rpc/get_transaction.go index c827d2c399..c2df579d7c 100644 --- a/src/apps/chifra/pkg/rpc/get_transaction.go +++ b/src/apps/chifra/pkg/rpc/get_transaction.go @@ -249,11 +249,6 @@ const ( // return 0; // } -const ( - byzantiumBlock = uint64(4370000) - constantinopleBlock = uint64(7280000) -) - // TODO: This is not cross-chain correct func (conn *Connection) GetTransactionRewardByTypeAndApp(rt RewardType, appearance *types.RawAppearance) (*types.SimpleTransaction, error) { diff --git a/src/apps/chifra/pkg/rpc/get_uncle.go b/src/apps/chifra/pkg/rpc/get_uncle.go index 1e1c4c105b..14df4ee682 100644 --- a/src/apps/chifra/pkg/rpc/get_uncle.go +++ b/src/apps/chifra/pkg/rpc/get_uncle.go @@ -18,7 +18,9 @@ import ( func (conn *Connection) GetUncleBodiesByNumber(bn uint64) ([]types.SimpleBlock[types.SimpleTransaction], error) { if count, err := conn.GetUnclesCountInBlock(bn); err != nil { return nil, err - } else if count > 0 { + } else if count == 0 { + return []types.SimpleBlock[types.SimpleTransaction]{}, nil + } else { ret := make([]types.SimpleBlock[types.SimpleTransaction], count) for i := uint64(0); i < count; i++ { method := "eth_getUncleByBlockNumberAndIndex" @@ -46,8 +48,6 @@ func (conn *Connection) GetUncleBodiesByNumber(bn uint64) ([]types.SimpleBlock[t } } return ret, nil - } else { - return []types.SimpleBlock[types.SimpleTransaction]{}, nil } } @@ -55,7 +55,9 @@ func (conn *Connection) GetUncleBodiesByNumber(bn uint64) ([]types.SimpleBlock[t func (conn *Connection) GetUnclesHashesByNumber(bn uint64) ([]base.Hash, error) { if count, err := conn.GetUnclesCountInBlock(bn); err != nil { return nil, err - } else if count > 0 { + } else if count == 0 { + return []base.Hash{}, nil + } else { ret := make([]base.Hash, count) for i := uint64(0); i < count; i++ { method := "eth_getUncleByBlockNumberAndIndex" @@ -72,13 +74,15 @@ func (conn *Connection) GetUnclesHashesByNumber(bn uint64) ([]base.Hash, error) } } return ret, nil - } else { - return []base.Hash{}, nil } } // GetUnclesCountInBlock returns the number of uncles in a block. func (conn *Connection) GetUnclesCountInBlock(bn uint64) (uint64, error) { + // if bn > base.LondonBlock { + // return 0, nil + // } + method := "eth_getUncleCountByBlockNumber" params := query.Params{fmt.Sprintf("0x%x", bn)} diff --git a/src/apps/chifra/pkg/types/types_transaction.go b/src/apps/chifra/pkg/types/types_transaction.go index 6814a942da..9ade0e457f 100644 --- a/src/apps/chifra/pkg/types/types_transaction.go +++ b/src/apps/chifra/pkg/types/types_transaction.go @@ -209,9 +209,8 @@ func (s *SimpleTransaction) Model(verbose bool, format string, extraOptions map[ // TODO: this should not be hardcoded here. We have tslib.GetSpecials(), but there // TODO: are 2 issues with it: 1. circular dependency with types package, 2. every // TODO: call to GetSpecials parses CSV file, so we need to call it once and cache - byzantiumBlock := uint64(4370000) status := &s.Receipt.Status - if s.BlockNumber < byzantiumBlock || *status == 4294967295-1 { + if s.BlockNumber < base.ByzantiumBlock || *status == 4294967295-1 { status = nil } diff --git a/test/gold/apps/blockScrape/api_tests/blockScrape_no_tracing.txt b/test/gold/apps/blockScrape/api_tests/blockScrape_no_tracing.txt index b6cf6729e9..1dec90d61e 100644 --- a/test/gold/apps/blockScrape/api_tests/blockScrape_no_tracing.txt +++ b/test/gold/apps/blockScrape/api_tests/blockScrape_no_tracing.txt @@ -1,6 +1,6 @@ scrape?chain=non-tracing { "errors": [ - "chifra scrape requires tracing, err: trace_block is missing" + "chifra scrape requires tracing, try chifra init instead." ] } diff --git a/test/gold/apps/config/config_config_edit.txt b/test/gold/apps/config/config_config_edit.txt index bb16a963fa..c865a3155d 100644 --- a/test/gold/apps/config/config_config_edit.txt +++ b/test/gold/apps/config/config_config_edit.txt @@ -1,3 +1,2 @@ chifra config edit TEST[DATE|TIME] Mode: edit -WARN[DATE|TIME] The config tool is current unavailable. Please use the 'chifra status' tool instead. diff --git a/test/gold/tools/getLogs/getLogs_three_ways.txt b/test/gold/tools/getLogs/getLogs_three_ways.txt index e3b807bad2..990e995c63 100644 --- a/test/gold/tools/getLogs/getLogs_three_ways.txt +++ b/test/gold/tools/getLogs/getLogs_three_ways.txt @@ -6,4 +6,4 @@ TEST[DATE|TIME] Format: txt 3657480 36 21 0x0f1217b92276cd17608d4212879739e6a5ec388bd7a03bef9798655234afd2b9 0xbed082f2f6fb5f91924df74eb79ac37ea9490e49dda0b29a01db27d171efb422 1494031419 2017-05-06 00:43:39 UTC 0x6090a6e47849629b7245dfa1ca21d94cd15878ef 0xb556ff269c1b6714f432c36431e2041d28436a73b6c3f19c021827bbdc6bfc29 0xb2c16e803407972d90eeec38de7937a77320f329f450a3e61c4b5bc1dbf6b55a 0x0000000000000000000000004811e6996291cd78b9f9272ead30da18774db174 0x000000000000000000000000000000000000000000000000002aa1efb94e0000 3657480 36 21 0x0f1217b92276cd17608d4212879739e6a5ec388bd7a03bef9798655234afd2b9 0xbed082f2f6fb5f91924df74eb79ac37ea9490e49dda0b29a01db27d171efb422 1494031419 2017-05-06 00:43:39 UTC 0x6090a6e47849629b7245dfa1ca21d94cd15878ef 0xb556ff269c1b6714f432c36431e2041d28436a73b6c3f19c021827bbdc6bfc29 0xb2c16e803407972d90eeec38de7937a77320f329f450a3e61c4b5bc1dbf6b55a 0x0000000000000000000000004811e6996291cd78b9f9272ead30da18774db174 0x000000000000000000000000000000000000000000000000002aa1efb94e0000 3657480 36 21 0x0f1217b92276cd17608d4212879739e6a5ec388bd7a03bef9798655234afd2b9 0xbed082f2f6fb5f91924df74eb79ac37ea9490e49dda0b29a01db27d171efb422 1494031419 2017-05-06 00:43:39 UTC 0x6090a6e47849629b7245dfa1ca21d94cd15878ef 0xb556ff269c1b6714f432c36431e2041d28436a73b6c3f19c021827bbdc6bfc29 0xb2c16e803407972d90eeec38de7937a77320f329f450a3e61c4b5bc1dbf6b55a 0x0000000000000000000000004811e6996291cd78b9f9272ead30da18774db174 0x000000000000000000000000000000000000000000000000002aa1efb94e0000 -EROR[DATE|TIME] transaction at 1001010.12 returned an error: not found +EROR[DATE|TIME] transaction at 0xbed082f2f6fb5f91924df74eb79ac37ea9490e49dda0b29a01db27d171efb424 returned an error: not found From b06bf949f8414b8199a9a4c1431959c19998c34b Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Wed, 13 Sep 2023 08:55:09 +0200 Subject: [PATCH 31/33] Fresh --- docs/content/data-model/admin.md | 18 +++++++++--------- .../makeClass/handle_sdk_py_paths.cpp | 3 +-- .../makeClass/handle_sdk_ts_paths.cpp | 3 +-- 3 files changed, 11 insertions(+), 13 deletions(-) diff --git a/docs/content/data-model/admin.md b/docs/content/data-model/admin.md index e8d3bc33a2..aa39ca7e80 100644 --- a/docs/content/data-model/admin.md +++ b/docs/content/data-model/admin.md @@ -234,15 +234,15 @@ The following commands produce and manage CacheItems: CacheItems consist of the following fields: -| Field | Description | Type | -| ----------- | --------------------------------------------------- | ------ | -| type | the type of the cache | string | -| items | the individual items in the cache (if --verbose) | Any | -| lastCached | the date of the most recent item added to the cache | string | -| nFiles | the number of items in the cache | uint64 | -| nFolders | the number of folders holding that many items | uint64 | -| path | the path to the top of the given cache | string | -| sizeInBytes | the size of the cache in bytes | int64 | +| Field | Description | Type | +| ----------- | --------------------------------------------------- | --------- | +| type | the type of the cache | string | +| items | the individual items in the cache (if --verbose) | Any | +| lastCached | the date of the most recent item added to the cache | string | +| nFiles | the number of items in the cache | uint64 | +| nFolders | the number of folders holding that many items | uint64 | +| path | the path to the top of the given cache | string | +| sizeInBytes | the size of the cache in bytes | int64 | ## ReportCheck diff --git a/src/dev_tools/makeClass/handle_sdk_py_paths.cpp b/src/dev_tools/makeClass/handle_sdk_py_paths.cpp index 70ae8c3542..5f14977391 100644 --- a/src/dev_tools/makeClass/handle_sdk_py_paths.cpp +++ b/src/dev_tools/makeClass/handle_sdk_py_paths.cpp @@ -75,8 +75,7 @@ bool COptions::handle_sdk_py_paths(CStringArray& pathsOut) { ostringstream params; for (auto p : members) { string_q line = " \"[{LONGNAME}]\": {\"hotkey\": \"[{HOTKEY}]\", \"type\": \"[{TYPE}]\"},\n"; - string_q optionName = substitute(toCamelCase(p.longName), "deleteme", "delete"); - + string_q optionName = toCamelCase(p.longName); if (!p.is_visible_docs && !contains(optionName, "cache")) { continue; } diff --git a/src/dev_tools/makeClass/handle_sdk_ts_paths.cpp b/src/dev_tools/makeClass/handle_sdk_ts_paths.cpp index a0f53d72f9..40f5631fa8 100644 --- a/src/dev_tools/makeClass/handle_sdk_ts_paths.cpp +++ b/src/dev_tools/makeClass/handle_sdk_ts_paths.cpp @@ -118,8 +118,7 @@ bool COptions::handle_sdk_ts_paths(CStringArray& pathsOut) { if (!p.is_visible_docs) { continue; } - string_q optionName = substitute(toCamelCase(p.longName), "deleteme", "delete"); - + string_q optionName = toCamelCase(p.longName); params << " " << optionName; params << (p.is_required ? "" : "?") << ": "; params << toTsType(p.data_type, imports) << ","; From ad9cd40123dbc8daf9513d8966f7bd64c1e37605 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Wed, 13 Sep 2023 09:00:42 +0200 Subject: [PATCH 32/33] Fresh --- src/CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 9d124ffbbd..7e6655cddb 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -96,7 +96,7 @@ add_custom_target(test-all COMMAND "${BIN_TEST_DIR}/test-all.sh" WORKING_DIRECTO # ---------------------------------------------------------------------------------------- # Compilation order for the subfolders -# add_subdirectory(dev_tools) +add_subdirectory(dev_tools) add_subdirectory(apps) add_subdirectory(examples) add_subdirectory(other) From 8f354e996889313b6c46d8d02d003570e71c8f57 Mon Sep 17 00:00:00 2001 From: Thomas Jay Rush Date: Wed, 13 Sep 2023 23:08:16 +0200 Subject: [PATCH 33/33] Removing unused code. --- docs/content/tutorials/index.md | 10 ---------- 1 file changed, 10 deletions(-) delete mode 100644 docs/content/tutorials/index.md diff --git a/docs/content/tutorials/index.md b/docs/content/tutorials/index.md deleted file mode 100644 index cfb831d49c..0000000000 --- a/docs/content/tutorials/index.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: "Tutorials" -description: "Learn how to use TrueBlocks, chifra, and the Unchained Index." -date: 2023-09-05T19:25:12+02:00 -lastmod: 2023-09-05T19:25:12+02:00 -draft: false -images: [] ---- - -WHAT WHAT?