Add LogIndex and TxIndex into logs/event response body #862

paologalligit · 2024-10-11T08:16:12Z

Description

This PR adds logIndex and txIndex fields into logs/event response body.

GH Issue

Type of change

Please delete options that are not relevant.

Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

Manual test
Unit test
E2E test

Checklist:

codecov-commenter · 2024-10-11T08:18:57Z

Codecov Report

Attention: Patch coverage is 82.60870% with 16 lines in your changes missing coverage. Please review.

Project coverage is 60.70%. Comparing base (49d9704) to head (a2a74f8).

Files with missing lines	Patch %	Lines
cmd/thor/sync_logdb.go	0.00%	10 Missing ⚠️
logdb/sequence.go	73.33%	2 Missing and 2 partials ⚠️
logdb/logdb.go	93.10%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #862      +/-   ##
==========================================
+ Coverage   60.62%   60.70%   +0.08%     
==========================================
  Files         215      215              
  Lines       23099    23125      +26     
==========================================
+ Hits        14003    14038      +35     
+ Misses       7947     7935      -12     
- Partials     1149     1152       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

* feat: add thorclient * refactor: remove roundTripper * refactor: change null check * clean: remove commented code * feat: add account revision and pending tx * fix: add licence headers and fix linter issue * refactor: rename package * refactor: change revision type to string * refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers * refactor: change FilterEvents and FilterTransactions request type to EventFilter * Adding common.EventWrapper to handle channel errors * tweak * update rawclient + update account tests * tidy up names * update tests * pr comments * adding raw tx * Tidy up method names and calls * options client * tweaks * pr comments * Update thorclient/common/common.go Co-authored-by: libotony <[email protected]> * pr comments * Adding Subscriptions * Pr comments * adjust func orders * pr comments * changing subscribe to use the channel close vs multiple channels * adding go-doc * no error after unsubscribe * pr comments * checking status code is 2xx * fix: change FilterTransfers argument --------- Co-authored-by: otherview <[email protected]> Co-authored-by: libotony <[email protected]>

* Show all issues on lint * fix lint

* fix(docker): using AWS docker repo for trivy * fix(docker): using AWS docker repo for trivy

thor/params.go

api/events/types.go

api/doc/thor.yaml

logdb/logdb.go

* ehancement: create a cache for block based subscriptions * minor: change function names for subscriptions * test: add unit test for message cache * chore: add license headers * refactor: fix up error handling * fix: remove bad test * fix: PR comments * fix: PR comments - remove block cache * refactor(subscriptions): store structs in cache, not bytes * fix(license): add license header * chore(subscriptions): revert unit test changes * enhancement: resolve pr comments to use simplelru * enhancement: resolve pr comments - use id as key

libotony · 2024-11-06T09:14:52Z

api/doc/thor.yaml

+          nullable: true
+          example: 1
+        logIndex:
+          descrption: The index of the log in the receipt's outputs.


Need to highlight it's an overall index among all clauses.

libotony · 2024-11-06T09:31:21Z

logdb/logdb.go

@@ -504,7 +507,7 @@ func (w *Writer) Write(b *block.Block, receipts tx.Receipts) error {

 				if err := w.exec(
 					query,
-					newSequence(blockNum, eventCount),
+					newSequence(blockNum, uint32(txIndex), eventCount),


eventCount and transferCount needs to be reset for every receipt/tx.

And the logic of insert blockID into ref needs to be updated.

eventCount and transferCount needs to be reset for every receipt/tx.

Are we changing the indexes logic? Before the count was not reset to 0 every receipt

I think by adding txIndex we are trying to align with ethereum's filter API, so we need to change the concept of logIndex.

blockID insertion logic updated

darrenvechain · 2024-11-06T12:28:53Z

@paologalligit possible to trigger CI? You may have to rebase with target branch. I think @libotony had issues on his mainDB branch, not sure how he solved

libotony · 2024-11-07T02:47:30Z

@paologalligit possible to trigger CI? You may have to rebase with target branch. I think @libotony had issues on his mainDB branch, not sure how he solved

Here, #868 (comment)

Wired that tests won't run with this PR only, might be caused by the on-pull-request.yaml wasn't updated in the master branch, but after PR #871 was opened, CI was trigger for targeting master then with the same commit hash of this PR, the workflow was also linked here.

libotony · 2024-11-07T10:02:07Z

@paologalligit I think the logIndex and txIndex also need to be applied to logs/transfers.

* Refactor thor node * thorchain allows insertion of blocks * remove thorNode, added testchain * clean up + comments * adding license headers * adding templating tests for thorclient * Remove test event hacks * remove types * removed chain_builder + added logdb to testchain * pr comments * Update test/testchain/chain.go Co-authored-by: libotony <[email protected]> --------- Co-authored-by: libotony <[email protected]>

* chore(docs): update spec for validator nodes * chore(docs): update cores * chore(docs): remove public node stuff

…d only once

otherview

lgtm !

api/events/events.go

libotony · 2024-11-08T10:02:22Z

@paologalligit Appended a commit a2a74f8

* Thor client (#818) * feat: add thorclient * refactor: remove roundTripper * refactor: change null check * clean: remove commented code * feat: add account revision and pending tx * fix: add licence headers and fix linter issue * refactor: rename package * refactor: change revision type to string * refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers * refactor: change FilterEvents and FilterTransactions request type to EventFilter * Adding common.EventWrapper to handle channel errors * tweak * update rawclient + update account tests * tidy up names * update tests * pr comments * adding raw tx * Tidy up method names and calls * options client * tweaks * pr comments * Update thorclient/common/common.go Co-authored-by: libotony <[email protected]> * pr comments * Adding Subscriptions * Pr comments * adjust func orders * pr comments * changing subscribe to use the channel close vs multiple channels * adding go-doc * no error after unsubscribe * pr comments * checking status code is 2xx * fix: change FilterTransfers argument --------- Co-authored-by: otherview <[email protected]> Co-authored-by: libotony <[email protected]> * Show all issues on lint (#869) * Show all issues on lint * fix lint * fix(docker): using AWS docker repo for trivy (#872) * fix(docker): using AWS docker repo for trivy * fix(docker): using AWS docker repo for trivy * Darren/feat/add subscription cache (#866) * ehancement: create a cache for block based subscriptions * minor: change function names for subscriptions * test: add unit test for message cache * chore: add license headers * refactor: fix up error handling * fix: remove bad test * fix: PR comments * fix: PR comments - remove block cache * refactor(subscriptions): store structs in cache, not bytes * fix(license): add license header * chore(subscriptions): revert unit test changes * enhancement: resolve pr comments to use simplelru * enhancement: resolve pr comments - use id as key * Add additional block tests (#863) * enhancement(logging): leverage trace level (#873) * Add testchain package (#844) * Refactor thor node * thorchain allows insertion of blocks * remove thorNode, added testchain * clean up + comments * adding license headers * adding templating tests for thorclient * Remove test event hacks * remove types * removed chain_builder + added logdb to testchain * pr comments * Update test/testchain/chain.go Co-authored-by: libotony <[email protected]> --------- Co-authored-by: libotony <[email protected]> * chore(docs): update spec for validator nodes (#875) * chore(docs): update spec for validator nodes * chore(docs): update cores * chore(docs): remove public node stuff * Darren/logdb remove leading zeros (#865) * feat: add new txIndex column to event meta response * test: add convert event test * feat: make txLog and txIndex as optional return params * chore: update swagger with new event optional data * feat: save logIndex in sequence * feat: tweaked bits in sequence * refactor: rename optional log meta field * refactor: comments, yaml and txIndex counts * rebase to master * fix: remove stale struct * add txIndex to returned logdb query * reset to 0 eventCount and transferCount each receipt and write blockId only once * fix lint * rephrase logIndex description in yaml file * refactor: use filter.Option instead of eventFilter.Option * move includeIndexes to api --------- Co-authored-by: otherview <[email protected]> Co-authored-by: libotony <[email protected]> Co-authored-by: Darren Kelly <[email protected]> Co-authored-by: Makis Christou <[email protected]>

paologalligit requested a review from a team as a code owner October 11, 2024 08:16

paologalligit changed the base branch from master to feat/db October 29, 2024 10:37

otherview and others added 2 commits October 30, 2024 11:36

Show all issues on lint (#869)

af88757

* Show all issues on lint * fix lint

fix(docker): using AWS docker repo for trivy (#872)

ad5bfbd

* fix(docker): using AWS docker repo for trivy * fix(docker): using AWS docker repo for trivy

libotony reviewed Nov 5, 2024

View reviewed changes

thor/params.go Outdated Show resolved Hide resolved

libotony reviewed Nov 5, 2024

View reviewed changes

api/events/types.go Outdated Show resolved Hide resolved

libotony reviewed Nov 5, 2024

View reviewed changes

api/doc/thor.yaml Show resolved Hide resolved

libotony reviewed Nov 5, 2024

View reviewed changes

logdb/logdb.go Outdated Show resolved Hide resolved

darrenvechain added the resync-required label Nov 5, 2024

libotony reviewed Nov 6, 2024

View reviewed changes

Add additional block tests (#863)

7a89780

libotony reviewed Nov 6, 2024

View reviewed changes

paologalligit force-pushed the feat/tx-log-seq branch from 6edc877 to b74f714 Compare November 6, 2024 15:27

enhancement(logging): leverage trace level (#873)

8be8574

paologalligit changed the base branch from feat/db to master November 7, 2024 09:33

otherview and others added 2 commits November 7, 2024 10:57

chore(docs): update spec for validator nodes (#875)

49d9704

* chore(docs): update spec for validator nodes * chore(docs): update cores * chore(docs): remove public node stuff

darrenvechain changed the base branch from master to feat/db November 7, 2024 12:06

darrenvechain changed the base branch from feat/db to master November 7, 2024 12:07

darrenvechain and others added 5 commits November 7, 2024 13:12

Darren/logdb remove leading zeros (#865)

ca8b38b

feat: add new txIndex column to event meta response

f9173e4

test: add convert event test

f3bd272

feat: make txLog and txIndex as optional return params

c5b3c63

chore: update swagger with new event optional data

76a38d9

paologalligit added 8 commits November 7, 2024 13:12

feat: save logIndex in sequence

ad6204e

feat: tweaked bits in sequence

1415b40

refactor: rename optional log meta field

94f4070

refactor: comments, yaml and txIndex counts

464f530

rebase to master

9114c0b

fix: remove stale struct

a80a62a

add txIndex to returned logdb query

9217741

reset to 0 eventCount and transferCount each receipt and write blockI…

297ffd5

…d only once

paologalligit force-pushed the feat/tx-log-seq branch from fbca57e to 297ffd5 Compare November 7, 2024 13:37

paologalligit added 2 commits November 7, 2024 14:40

fix lint

9ff47cb

rephrase logIndex description in yaml file

6f4c9ef

paologalligit requested review from libotony and darrenvechain November 7, 2024 13:55

otherview previously approved these changes Nov 7, 2024

View reviewed changes

api/events/events.go Show resolved Hide resolved

refactor: use filter.Option instead of eventFilter.Option

f81bc85

paologalligit dismissed otherview’s stale review via f81bc85 November 7, 2024 16:35

paologalligit requested a review from otherview November 7, 2024 16:43

otherview previously approved these changes Nov 7, 2024

View reviewed changes

move includeIndexes to api

a2a74f8

libotony dismissed otherview’s stale review via a2a74f8 November 8, 2024 10:01

libotony approved these changes Nov 8, 2024

View reviewed changes

libotony changed the base branch from master to feat/db November 8, 2024 10:04

darrenvechain approved these changes Nov 8, 2024

View reviewed changes

paologalligit merged commit 368e5d2 into feat/db Nov 8, 2024
15 checks passed

paologalligit deleted the feat/tx-log-seq branch November 8, 2024 16:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add LogIndex and TxIndex into logs/event response body #862

Add LogIndex and TxIndex into logs/event response body #862

paologalligit commented Oct 11, 2024

codecov-commenter commented Oct 11, 2024 •

edited

Loading

libotony Nov 6, 2024

libotony Nov 6, 2024

libotony Nov 6, 2024

paologalligit Nov 6, 2024

libotony Nov 7, 2024

paologalligit Nov 7, 2024

darrenvechain commented Nov 6, 2024 •

edited

Loading

libotony commented Nov 7, 2024

libotony commented Nov 7, 2024

otherview left a comment

libotony commented Nov 8, 2024

Add LogIndex and TxIndex into logs/event response body #862

Add LogIndex and TxIndex into logs/event response body #862

Conversation

paologalligit commented Oct 11, 2024

Description

Type of change

How Has This Been Tested?

Checklist:

codecov-commenter commented Oct 11, 2024 • edited Loading

Codecov Report

libotony Nov 6, 2024

Choose a reason for hiding this comment

libotony Nov 6, 2024

Choose a reason for hiding this comment

libotony Nov 6, 2024

Choose a reason for hiding this comment

paologalligit Nov 6, 2024

Choose a reason for hiding this comment

libotony Nov 7, 2024

Choose a reason for hiding this comment

paologalligit Nov 7, 2024

Choose a reason for hiding this comment

darrenvechain commented Nov 6, 2024 • edited Loading

libotony commented Nov 7, 2024

libotony commented Nov 7, 2024

otherview left a comment

Choose a reason for hiding this comment

libotony commented Nov 8, 2024

codecov-commenter commented Oct 11, 2024 •

edited

Loading

darrenvechain commented Nov 6, 2024 •

edited

Loading