Function name changes to improve scraper interfaces

CHANGES.md

@@ -4,6 +4,21 @@
 
 This file details changes made to TrueBlocks over time. See the [migration notes](./MIGRATIONS.md) for any required actions you must take to stay up to date.
 
+## v1.1.0 (2023/09/05)
+
+Added `--watch_list`, `--commands`, `--batch_size`, and `--run_count` options to `chifra monitors`.
+Added `--run_count` to `chifra scrape` (for debugging purposes).
+Changed names of some rarely used special blocks in `chifra when`. Breaking, but minimal impact.
+Removes a number of previously deprecated options. `chifra abis --sol`, `chifra names --named`, `chifra names --to_custom`, `chifra transactions --trace`, and `chifra --log_level` for all commands.
+Better error reporting when running against non-tracing nodes.
+Fixed an issue with Content-Type in the server.
+Fixed an issue where user could hit cntl+c during caching and corrupt the database.
+Fixed an issue where scraper missed some smart contract addresses created during out of gas transactions.
+Better error handling from the RPC.
+Added a verbose mode to `chifra when` to include more specials and a description for each special block.
+Now disallows running `chifra scrape` if the node is not a tracing archive node.
+Near complete re-write of block scraper to fix few bugs and prepare for writing v1.0.0 of the spec.
+
 ## v1.0.0 (2023/08/20)
 
 **OUR FIRST OFFICIAL RELEASE!**

src/dev_tools/testRunner/testCases/apps/acctExport.csv

@@ -79,7 +79,7 @@ on      ,both ,fast  ,monitors ,apps/acctExport ,monitors_err_sleep       ,y
 on      ,both ,fast  ,monitors ,apps/acctExport ,monitors_err_batch_size2 ,y    ,addrs = 0xf503017d7baf7fbc0fff7492b751025c6a78179b & fmt = json & batch_size = 1
 on      ,both ,fast  ,monitors ,apps/acctExport ,monitors_err_run_once    ,y    ,addrs = 0xf503017d7baf7fbc0fff7492b751025c6a78179b & fmt = json & run_count = 1
 on      ,both ,fast  ,monitors ,apps/acctExport ,monitors_err_watch       ,y    ,watch & batch_size = 0 & fmt = json
-local   ,both ,fast  ,monitors ,apps/acctExport ,monitors_watch           ,y    ,watch & commands = ./command.fil & watchlist = ./watches.txt & run_once & fmt = json
+local   ,both ,fast  ,monitors ,apps/acctExport ,monitors_watch           ,y    ,watch & commands = ./command.fil & watchlist = ./watches.txt & run_count = 1 & fmt = json
 
 on      ,both ,fast  ,list     ,apps/acctExport ,list_prepare_1           ,y    ,addrs = 0x001d14804b399c6ef80e64576f657660804fec0b & fmt = json & last_block = 1501460
 on      ,both ,fast  ,monitors ,apps/acctExport ,monitors_clean           ,y    ,addrs = 0x001d14804b399c6ef80e64576f657660804fec0b & clean

test/gold/apps/acctExport/acctExport_caps_allowed_m.txt

@@ -18,15 +18,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).

test/gold/apps/acctExport/acctExport_caps_disallowed_1_m.txt

@@ -18,15 +18,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).

test/gold/apps/acctExport/acctExport_caps_disallowed_2_m.txt

@@ -18,15 +18,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).

test/gold/apps/acctExport/acctExport_caps_disallowed_3_m.txt

@@ -18,15 +18,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).

test/gold/apps/acctExport/acctExport_caps_disallowed_4_m.txt

@@ -18,15 +18,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).

test/gold/apps/acctExport/acctExport_monitors_decache.txt

@@ -18,15 +18,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).

test/gold/apps/acctExport/acctExport_monitors_decache_again.txt

@@ -18,15 +18,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).

test/gold/apps/acctExport/acctExport_monitors_delete_1_fail.txt

@@ -19,15 +19,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).

test/gold/apps/acctExport/acctExport_monitors_err_batch_size2.txt

@@ -19,15 +19,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).

test/gold/apps/acctExport/acctExport_monitors_err_not_both1.txt

@@ -20,15 +20,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).

test/gold/apps/acctExport/acctExport_monitors_err_not_both2.txt

@@ -20,15 +20,15 @@ Flags:
   -a, --watchlist string   available with --watch option only, a file containing the addresses to watch
   -c, --commands string    available with --watch option only, the file containing the list of commands to apply to each watched address
   -b, --batch_size uint    available with --watch option only, the number of monitors to process in each batch (default 8)
-  -r, --run_once           available with --watch option only, only run the monitor --watch commands once then quit
+  -u, --run_count uint     available with --watch option only, run the monitor this many times, then quit (hidden)
   -s, --sleep float        available with --watch option only, the number of seconds to sleep between runs (default 14)
   -x, --fmt string         export format, one of [none|json*|txt|csv]
   -v, --verbose            enable verbose output
   -h, --help               display this help screen
 
 Notes:
   - An address must be either an ENS name or start with '0x' and be forty-two characters long.
-  - If no address is presented to the --clean command, all existing monitors are be cleaned.
+  - If no address is presented to the --clean command, all existing monitors will be cleaned.
   - The --watch option requires two additional parameters to be specified: --watchlist and --commands.
   - Addresses provided on the command line are ignored in --watch mode.
   - Providing the value existing to the --watchlist monitors all existing monitor files (see --list).