Fix interface_zmq functional test case #116

gitcoindev · 2022-12-20T13:09:37Z

Description

The goal of this pull request is to fix interface_zmq functional test case.

Notes

The test case needed two separate fixes: first was the bugfix from Bitcoin core bitcoin/bitcoin#17185 , then my bugfix for Bitgesell, to verify against keccak256 and single sha256.

$ test/functional/interface_zmq.py
2022-12-20T13:03:41.840000Z TestFramework (INFO): Initializing test directory /tmp/BGL_func_test_06l12t_m
2022-12-20T13:03:44.361000Z TestFramework (INFO): Generate 5 blocks (and 5 coinbase txes)
2022-12-20T13:03:44.391000Z TestFramework (INFO): Wait for tx from second node
2022-12-20T13:03:45.532000Z TestFramework (INFO): Test the getzmqnotifications RPC
2022-12-20T13:03:45.534000Z TestFramework (INFO): Testing 'sequence' publisher
2022-12-20T13:03:46.444000Z TestFramework (INFO): Wait for tx from second node
2022-12-20T13:03:47.564000Z TestFramework (INFO): Testing sequence notifications with mempool sequence values
2022-12-20T13:03:47.564000Z TestFramework (INFO): Testing RBF notification
2022-12-20T13:03:49.821000Z TestFramework (INFO): Testing reorg notifications
2022-12-20T13:03:51.847000Z TestFramework (INFO): Evict mempool transaction by block conflict
2022-12-20T13:03:52.924000Z TestFramework (INFO): Testing 'mempool sync' usage of sequence notifier
2022-12-20T13:03:59.154000Z TestFramework (INFO): Testing IPv6
2022-12-20T13:04:00.001000Z TestFramework (INFO): Stopping nodes
2022-12-20T13:04:00.257000Z TestFramework (INFO): Cleaning up /tmp/BGL_func_test_06l12t_m on exit
2022-12-20T13:04:00.257000Z TestFramework (INFO): Tests successful

Hi @madnadyka @wu-emma @janus I'd be grateful if you found time for the review

I opened a batch of new pull requests fixing functional tests including this one

#107
#108
#109
#110
#111
#112
#113
#114
#115
#116

After those pull request will be reviewed and merged there are last remaining failing tests (all of them not trivial or requiring a lot of integration, I estimate 5-10 days to fix) that I would like to tackle in January next year:

Test case name	Analysis
feature_signet.py	Bitgesell signet blocks needed
p2p_dos_header_tree.py	Forked Bitgesell testnet headers needed
rpc_psbt.py --descriptors	Lots of integration and changes in Bitcoin core
mempool_package_limits.py	Should be synced with Bitcoin core, and after initial investigation package limits are not failing in all cases, needs more analysis
feature_nulldummy.py --descriptors	Fails with unexpected witness data when simulating Segwit fork

After those are fixed in the new batch of pull requests we should have all functional tests passing and could integrate all new tests from Bitcoin core if functionality is already available in Bitgesell and you would be interested to integrate them.

PR reward address

0x7e92476D69Ff1377a8b45176b1829C4A5566653a (ETH mainnet or Polygon)

madnadyka · 2022-12-21T21:25:57Z

Hi! Some PRs didn't pass tests:

#109 - passed before and failed after
#116 - failed

interface_BGL_cli.py                               | ✖ Failed  | 6 s
interface_zmq.py                                   | ✖ Failed  | 4 s

Please check additional.

gitcoindev · 2022-12-22T08:10:19Z

@madnadyka you were right, I did a fresh sync after merges, clean and rebuild and interface_BGL_cli.py 10/10 PASS
I will skip it then as it is passing always now and check interface_zmq.py asap now.

$ for i in {1..10}; do test/functional/test_runner.py test/functional/interface_BGL_cli.py | grep interface ; done
Temporary test directory at /tmp/test_runner_20221222_090250
..........
----------------------------------------------------------------------
Ran 10 tests in 0.514s

OK
1/1 - interface_BGL_cli.py passed, Duration: 12 s
Remaining jobs: [interface_BGL_cli.py]
interface_BGL_cli.py | ✓ Passed  | 12 s
Temporary test directory at /tmp/test_runner_20221222_090302
..........
----------------------------------------------------------------------
Ran 10 tests in 0.523s

OK
1/1 - interface_BGL_cli.py passed, Duration: 12 s
Remaining jobs: [interface_BGL_cli.py]
interface_BGL_cli.py | ✓ Passed  | 12 s
Temporary test directory at /tmp/test_runner_20221222_090315
..........
----------------------------------------------------------------------
Ran 10 tests in 0.524s

OK
1/1 - interface_BGL_cli.py passed, Duration: 11 s
Remaining jobs: [interface_BGL_cli.py]
interface_BGL_cli.py | ✓ Passed  | 11 s
Temporary test directory at /tmp/test_runner_20221222_090327
..........
----------------------------------------------------------------------
Ran 10 tests in 0.503s

OK
1/1 - interface_BGL_cli.py passed, Duration: 11 s
Remaining jobs: [interface_BGL_cli.py]
interface_BGL_cli.py | ✓ Passed  | 11 s
Temporary test directory at /tmp/test_runner_20221222_090339
..........
----------------------------------------------------------------------
Ran 10 tests in 0.518s

OK
1/1 - interface_BGL_cli.py passed, Duration: 11 s
Remaining jobs: [interface_BGL_cli.py]
interface_BGL_cli.py | ✓ Passed  | 11 s
Temporary test directory at /tmp/test_runner_20221222_090352
..........
----------------------------------------------------------------------
Ran 10 tests in 0.520s

OK
1/1 - interface_BGL_cli.py passed, Duration: 12 s
Remaining jobs: [interface_BGL_cli.py]
interface_BGL_cli.py | ✓ Passed  | 12 s
Temporary test directory at /tmp/test_runner_20221222_090404
..........
----------------------------------------------------------------------
Ran 10 tests in 0.504s

OK
1/1 - interface_BGL_cli.py passed, Duration: 12 s
Remaining jobs: [interface_BGL_cli.py]
interface_BGL_cli.py | ✓ Passed  | 12 s
Temporary test directory at /tmp/test_runner_20221222_090417
..........
----------------------------------------------------------------------
Ran 10 tests in 0.525s

OK
1/1 - interface_BGL_cli.py passed, Duration: 11 s
Remaining jobs: [interface_BGL_cli.py]
interface_BGL_cli.py | ✓ Passed  | 11 s
Temporary test directory at /tmp/test_runner_20221222_090429
..........
----------------------------------------------------------------------
Ran 10 tests in 0.500s

OK
1/1 - interface_BGL_cli.py passed, Duration: 11 s
Remaining jobs: [interface_BGL_cli.py]
interface_BGL_cli.py | ✓ Passed  | 11 s
Temporary test directory at /tmp/test_runner_20221222_090441
..........
----------------------------------------------------------------------
Ran 10 tests in 0.522s

OK
1/1 - interface_BGL_cli.py passed, Duration: 11 s
Remaining jobs: [interface_BGL_cli.py]
interface_BGL_cli.py | ✓ Passed  | 11 s

Use correct keccak256 hash and single sha256 to verify block and transactions.

gitcoindev · 2022-12-22T08:34:44Z

Hi @madnadyka I did a fresh rebase against latest master the interface_zmq.py is 10/10 PASSing now.

I would be grateful if you checked this pull request and the test case again. Please make sure that you have pyzmq installed:

pip install pyzmq
Requirement already satisfied: pyzmq in /usr/lib/python3/dist-packages (18.1.1)

and when you call autogen.sh and configure you see in the last configure logs that zmq is enabled i.e.:

$ export BDB_PREFIX=`pwd`/db4
$ ./configure BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" BDB_CFLAGS="-I${BDB_PREFIX}/include"

...
...
...

Options used to compile and link:
  external signer = yes
  multiprocess    = no
  with experimental syscall sandbox support = yes
  with libs       = yes
  with wallet     = yes
    with sqlite   = yes
    with bdb      = yes
  with gui / qt   = yes
    with qr       = no
  with zmq        = yes   <============ HERE
  with test       = yes
  with bench      = yes
  with upnp       = yes
  with natpmp     = yes

after that with a rebuild you should hopefully see:

$ test/functional/interface_zmq.py
2022-12-22T08:32:44.350000Z TestFramework (INFO): Initializing test directory /tmp/BGL_func_test_arogrrnt
2022-12-22T08:32:46.883000Z TestFramework (INFO): Generate 5 blocks (and 5 coinbase txes)
2022-12-22T08:32:46.915000Z TestFramework (INFO): Wait for tx from second node
2022-12-22T08:32:48.054000Z TestFramework (INFO): Test the getzmqnotifications RPC
2022-12-22T08:32:48.056000Z TestFramework (INFO): Testing 'sequence' publisher
2022-12-22T08:32:48.967000Z TestFramework (INFO): Wait for tx from second node
2022-12-22T08:32:50.086000Z TestFramework (INFO): Testing sequence notifications with mempool sequence values
2022-12-22T08:32:50.087000Z TestFramework (INFO): Testing RBF notification
2022-12-22T08:32:52.332000Z TestFramework (INFO): Testing reorg notifications
2022-12-22T08:32:54.359000Z TestFramework (INFO): Evict mempool transaction by block conflict
2022-12-22T08:32:55.384000Z TestFramework (INFO): Testing 'mempool sync' usage of sequence notifier
2022-12-22T08:33:01.612000Z TestFramework (INFO): Testing IPv6
2022-12-22T08:33:02.455000Z TestFramework (INFO): Stopping nodes
2022-12-22T08:33:02.710000Z TestFramework (INFO): Cleaning up /tmp/BGL_func_test_arogrrnt on exit
2022-12-22T08:33:02.710000Z TestFramework (INFO): Tests successful

madnadyka · 2022-12-22T11:41:12Z

Now it is passed

gitcoindev · 2022-12-22T19:26:23Z

Thank you for merging everything!

wu-emma · 2022-12-31T13:35:53Z

HI @gitcoindev !
thanks for your contributions, good job!
Send you 2k USDT (for now) https://etherscan.io/tx/0x55c8092a842aa5f8f29f8311a54b24b71be1201ce127d5275883fdb8d3ffccf6
Guys would check the PRs volume just in case.
All the best in New Year!

gitcoindev · 2023-01-02T07:15:35Z

Dear @wu-emma, thank you a lot, I really appreciate it. This is a great start into the New Year. I will try submit new PRs by mid/end of January. sob., 31 gru 2022 o 14:36 wu-emma ***@***.***> napisał(a):

…

HI @gitcoindev <https://github.com/gitcoindev> ! thanks for your contributions, good job! Send you 2k USDT (for now) https://etherscan.io/tx/0x55c8092a842aa5f8f29f8311a54b24b71be1201ce127d5275883fdb8d3ffccf6 Guys would check the PRs volume just in case. All the best in New Year! — Reply to this email directly, view it on GitHub <#116 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVFGLNPZV3EJIOAHCYMWV5LWQAZEJANCNFSM6AAAAAATEPQRVI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

wu-emma · 2023-01-14T08:29:28Z

Hi @gitcoindev
hope you had good holidays!
Guys checked the volume of PRs (of course we know it's a rough estimation) and decided to payout $1000 extra to you for the contributions. Thank you!
0xcbbb831ba1d3e80a139a5a7e8fe15dfd860e5708a304801a7b399275e5a9bc11

ooops... I occasionally sent it via BSC network. Hope you would be able to retrieve/bridge it, sorry.

gitcoindev · 2023-01-14T11:59:35Z

Hi @wu-emma thank you a lot! I really appreciate this. I was able to bridge this one, no worries!

promag and others added 2 commits December 22, 2022 09:12

zmq: Fix due to invalid argument and multiple notifiers

c5ee760

test: functional: interface_zmq: Fix the test for Bitgesell.

307c9ac

Use correct keccak256 hash and single sha256 to verify block and transactions.

gitcoindev force-pushed the bugfix/fix-interface_zmq-test branch from db873a0 to 307c9ac Compare December 22, 2022 08:12

madnadyka merged commit 2f3fafb into BitgesellOfficial:master Dec 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix interface_zmq functional test case #116

Fix interface_zmq functional test case #116

gitcoindev commented Dec 20, 2022 •

edited

Loading

madnadyka commented Dec 21, 2022

gitcoindev commented Dec 22, 2022

gitcoindev commented Dec 22, 2022 •

edited

Loading

madnadyka commented Dec 22, 2022

gitcoindev commented Dec 22, 2022

wu-emma commented Dec 31, 2022

gitcoindev commented Jan 2, 2023 via email

wu-emma commented Jan 14, 2023 •

edited

Loading

gitcoindev commented Jan 14, 2023

Fix interface_zmq functional test case #116

Fix interface_zmq functional test case #116

Conversation

gitcoindev commented Dec 20, 2022 • edited Loading

Description

Notes

PR reward address

madnadyka commented Dec 21, 2022

gitcoindev commented Dec 22, 2022

gitcoindev commented Dec 22, 2022 • edited Loading

madnadyka commented Dec 22, 2022

gitcoindev commented Dec 22, 2022

wu-emma commented Dec 31, 2022

gitcoindev commented Jan 2, 2023 via email

wu-emma commented Jan 14, 2023 • edited Loading

gitcoindev commented Jan 14, 2023

gitcoindev commented Dec 20, 2022 •

edited

Loading

gitcoindev commented Dec 22, 2022 •

edited

Loading

wu-emma commented Jan 14, 2023 •

edited

Loading