You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I succeeded in streaming block data into a postgresql database from a local openethereum node. When I try streaming transactions, traces, or any other of ther possible entities I get the below listed output. If I do a select statement in the database, I don't see any entries (except for the blocks).
Does it maybe have to do with openethereum (since as I saw form another issue, the devs use parity 2.5) or is it a bug in the script?
Is it also correct that contracts are not yet supported for the streaming into postgresql?
Output
The below listed output is just a snippet, it goes on like this for quite some time.
2020-12-17 10:08:18,245 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,246 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,247 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.001733.
2020-12-17 10:08:18,247 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,247 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000099.
2020-12-17 10:08:18,247 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,247 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000091.
2020-12-17 10:08:18,247 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,251 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,251 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.003582.
2020-12-17 10:08:18,251 - root [INFO] - Exporting with PostgresItemExporter
2020-12-17 10:08:18,251 - root [INFO] - Writing last synced block 1838
2020-12-17 10:08:18,253 - root [INFO] - Current block 1740260, target block 1839, last synced block 1838, blocks to sync 1
2020-12-17 10:08:18,254 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,255 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,256 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.001944.
2020-12-17 10:08:18,256 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,256 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000101.
2020-12-17 10:08:18,256 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,256 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000084.
2020-12-17 10:08:18,256 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,260 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,261 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.004245.
2020-12-17 10:08:18,261 - root [INFO] - Exporting with PostgresItemExporter
2020-12-17 10:08:18,261 - root [INFO] - Writing last synced block 1839
2020-12-17 10:08:18,263 - root [INFO] - Current block 1740260, target block 1840, last synced block 1839, blocks to sync 1
2020-12-17 10:08:18,263 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,265 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,266 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.002211.
2020-12-17 10:08:18,266 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,266 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000096.
2020-12-17 10:08:18,266 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,266 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000079.
2020-12-17 10:08:18,266 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,270 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,270 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.003434.
2020-12-17 10:08:18,270 - root [INFO] - Exporting with PostgresItemExporter
2020-12-17 10:08:18,270 - root [INFO] - Writing last synced block 1840
2020-12-17 10:08:18,273 - root [INFO] - Current block 1740260, target block 1841, last synced block 1840, blocks to sync 1
2020-12-17 10:08:18,273 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,275 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,275 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.001923.
2020-12-17 10:08:18,275 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,275 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000077.
2020-12-17 10:08:18,276 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,276 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000118.
2020-12-17 10:08:18,276 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,279 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,279 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.003494.
2020-12-17 10:08:18,280 - root [INFO] - Exporting with PostgresItemExporter
2020-12-17 10:08:18,280 - root [INFO] - Writing last synced block 1841
2020-12-17 10:08:18,283 - root [INFO] - Current block 1740260, target block 1842, last synced block 1841, blocks to sync 1
2020-12-17 10:08:18,283 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,285 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,285 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.001863.
2020-12-17 10:08:18,285 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,285 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000081.
2020-12-17 10:08:18,285 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,285 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000064.
2020-12-17 10:08:18,285 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,288 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,289 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.003132.
2020-12-17 10:08:18,289 - root [INFO] - Exporting with PostgresItemExporter
2020-12-17 10:08:18,289 - root [INFO] - Writing last synced block 1842
2020-12-17 10:08:18,292 - root [INFO] - Current block 1740260, target block 1843, last synced block 1842, blocks to sync 1
2020-12-17 10:08:18,292 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,294 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,294 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.001705.
2020-12-17 10:08:18,294 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,294 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000087.
2020-12-17 10:08:18,294 - ProgressLogger [INFO] - Started work.
2020-12-17 10:08:18,294 - ProgressLogger [INFO] - Finished work. Total items processed: 0. Took 0:00:00.000072.
2020-12-17 10:08:18,295 - ProgressLogger [INFO] - Started work. Items to process: 1.
2020-12-17 10:08:18,298 - ProgressLogger [INFO] - 1 items processed. Progress is 100%.
2020-12-17 10:08:18,298 - ProgressLogger [INFO] - Finished work. Total items processed: 1. Took 0:00:00.003387.
2020-12-17 10:08:18,298 - root [INFO] - Exporting with PostgresItemExporter
2020-12-17 10:08:18,298 - root [INFO] - Writing last synced block 1843
2020-12-17 10:08:18,302 - root [INFO] - Current block 1740260, target block 1844, last synced block 1843, blocks to sync 1
The text was updated successfully, but these errors were encountered:
SirFrankieCrisp
changed the title
Streaming blocks to Postgresql Database works, others don't
Streaming transactions, logs, token_transfer, trace to PostgreSQL doesn't work
Dec 17, 2020
@medvedev1088 Thanks for your reponse. I now tried streaming blocks, transactions, traces, token_transfers and logs all at the same time, but besides blocks there is nothing in the database. There are now at least 15'000 blocks in the db but still no transactions or anything. Also no traces :/ No error messages are displayed, it looks like it is really just importing blocks or so
You where right. The first transaction happens on block 46167. Now transactions show up. Still no logs, traces, receipts or token_transfers though:/ Let's see if anything changes a few tens of thousand of blocks down the line.
I succeeded in streaming block data into a postgresql database from a local openethereum node. When I try streaming transactions, traces, or any other of ther possible entities I get the below listed output. If I do a select statement in the database, I don't see any entries (except for the blocks).
Does it maybe have to do with openethereum (since as I saw form another issue, the devs use parity 2.5) or is it a bug in the script?
Is it also correct that contracts are not yet supported for the streaming into postgresql?
Command
ethereumetl stream --provider-uri file://$HOME/.local/share/openethereum/jsonrpc.ipc --start-block 0 --output postgresql+pg8000://postgres:[email protected]:5432/openeth_etl -e transaction,log,token_transfer,trace
Output
The below listed output is just a snippet, it goes on like this for quite some time.
The text was updated successfully, but these errors were encountered: