feat(manager): max skew based on time instead of batches #1140

srene · 2024-10-16T09:58:06Z

PR Standards

This PR substitutes the max batch skew calculation, and instead of pausing block production after N unsubmitted batches (Config parameter), it pauses block production when the gap between the last submitted block and the last produced bock is higher than a config value.

Opening a pull request should be able to meet the following requirements

--

PR naming convention: https://hackmd.io/@nZpxHZ0CT7O5ngTp0TP9mg/HJP_jrm7A

Close #1139

<-- Briefly describe the content of this pull request -->

For Author:

Targeted PR against correct branch
included the correct type prefix in the PR title
Linked to Github issue with discussion and accepted design
Targets only one github issue
Wrote unit and integration tests
All CI checks have passed
Added relevant godoc comments

For Reviewer:

confirmed the correct type prefix in the PR title
Reviewers assigned
confirmed all author checklist items have been addressed

After reviewer approval:

In case targets main branch, PR should be squashed and merged.
In case PR targets a release branch, PR should be rebased.

block/state.go

block/submit.go

block/state.go

block/executor.go

types/state.go

block/state.go

block/manager.go

block/state.go

block/submit.go

block/state.go

codecov · 2024-10-18T09:36:37Z

Codecov Report

Attention: Patch coverage is 71.65354% with 36 lines in your changes missing coverage. Please review.

Project coverage is 17.03%. Comparing base (54e2fa5) to head (f8938b3).
Report is 22 commits behind head on main.

Files with missing lines	Patch %	Lines
types/pb/dymint/state.pb.go	43.18%	24 Missing and 1 partial ⚠️
settlement/grpc/grpc.go	0.00%	10 Missing ⚠️
settlement/dymension/utils.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1140      +/-   ##
==========================================
- Coverage   19.54%   17.03%   -2.51%     
==========================================
  Files         165      174       +9     
  Lines       41268    51520   +10252     
==========================================
+ Hits         8065     8776     +711     
- Misses      32034    41366    +9332     
- Partials     1169     1378     +209

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

block/executor.go

block/initchain.go

@@ -22,11 +23,16 @@
 	if err != nil {
 		return err
 	}
+	err = m.SetLastSettlementBlockTime(time.Now())


block/state.go

@@ -110,6 +111,7 @@
 	}
 	// We update the last results hash with the empty hash, to conform with RFC-6962.
 	copy(s.LastResultsHash[:], merkle.HashFromByteSlices(nil))
+	s.SetLastBlockTime(time.Now())


mtsitrin · 2024-11-20T19:47:16Z

block/produce.go

@@ -76,16 +76,16 @@ func (m *Manager) ProduceBlockLoop(ctx context.Context, bytesProducedC chan int)
 			}

 			bytesProducedN := block.SizeBytes() + commit.SizeBytes()
-			m.logger.Info("New block.", "size", uint64(block.ToProto().Size()))
+			m.logger.Info("New block.", "size", bytesProducedN)


I would remove the size value
change it to more useful values (height, num of tx)

i removed this log and added size to the existing block created log in produceblock func

mtsitrin · 2024-11-20T19:50:00Z

config/config.go

-	// BatchSkew is the number of batches waiting to be submitted. Block production will be paused if this limit is reached.
-	BatchSkew uint64 `mapstructure:"max_batch_skew"`
+	// MaxSkewTime is the number of batches waiting to be submitted. Block production will be paused if this limit is reached.
+	MaxBatchSkewTime time.Duration `mapstructure:"max_skew_time"`


Suggested change

MaxBatchSkewTime time.Duration `mapstructure:"max_skew_time"`

MaxSkewTime time.Duration `mapstructure:"max_skew_time"`

mtsitrin · 2024-11-21T07:51:33Z

block/manager.go

@@ -303,20 +310,28 @@ func (m *Manager) updateFromLastSettlementState() error {
 		// The SL hasn't got any batches for this chain yet.
 		m.logger.Info("No batches for chain found in SL.")
 		m.LastSettlementHeight.Store(uint64(m.Genesis.InitialHeight - 1))
+		m.SetLastBlockTimeInSettlementFromHeight(uint64(m.Genesis.InitialHeight))


Suggested change

m.SetLastBlockTimeInSettlementFromHeight(uint64(m.Genesis.InitialHeight))

m.LastBlockTimeInSettlement.Store(m.Genesis.GenesisTime.UTC().UnixNano())

GenesisTime

srene requested a review from a team as a code owner October 16, 2024 09:58

srene marked this pull request as draft October 16, 2024 09:58

github-actions bot added the dym-internal label Oct 16, 2024

srene linked an issue Oct 16, 2024 that may be closed by this pull request

block production not stopping after max_batch_skew #1077

Closed

github-advanced-security bot found potential problems Oct 16, 2024

View reviewed changes

block/state.go Fixed Show fixed Hide fixed

block/state.go Fixed Show fixed Hide fixed

block/submit.go Fixed Show fixed Hide fixed

github-advanced-security bot found potential problems Oct 16, 2024

View reviewed changes

block/state.go Fixed Show fixed Hide fixed

srene self-assigned this Oct 17, 2024

srene marked this pull request as ready for review October 17, 2024 11:51

danwt reviewed Oct 17, 2024

View reviewed changes

block/executor.go Outdated Show resolved Hide resolved

types/state.go Outdated Show resolved Hide resolved

block/state.go Outdated Show resolved Hide resolved

block/manager.go Outdated Show resolved Hide resolved

srene marked this pull request as draft October 17, 2024 13:07

github-advanced-security bot found potential problems Oct 17, 2024

View reviewed changes

block/state.go Fixed Show fixed Hide fixed

block/submit.go Fixed Show fixed Hide fixed

github-advanced-security bot found potential problems Oct 17, 2024

View reviewed changes

block/state.go Fixed Show fixed Hide fixed

srene marked this pull request as ready for review October 18, 2024 09:36

srene marked this pull request as draft October 24, 2024 14:03

srene force-pushed the srene/1139-max-skew-time branch from f8938b3 to 63affef Compare November 4, 2024 08:49

github-advanced-security bot found potential problems Nov 4, 2024

View reviewed changes

block/executor.go Fixed Show fixed Hide fixed

srene force-pushed the srene/1139-max-skew-time branch from 120347e to 4f0976b Compare November 4, 2024 15:13

github-advanced-security bot found potential problems Nov 4, 2024

View reviewed changes

block/initchain.go Fixed Show fixed Hide fixed

srene force-pushed the srene/1139-max-skew-time branch from bab82bb to c2bef84 Compare November 5, 2024 09:02

github-advanced-security bot found potential problems Nov 5, 2024

View reviewed changes

srene marked this pull request as ready for review November 5, 2024 12:30

srene added 7 commits November 8, 2024 09:06

skew based on time

2f66f5e

test fix

3e92814

new skey claculatuin

8f7c678

fixing test + bds in mock

72f014a

lint fix

02f5ab7

test adapted

306f6e4

fix + test

a48066b

srene marked this pull request as draft November 19, 2024 14:20

srene added 16 commits November 19, 2024 15:25

fix test

c8d0078

last block time fix

c190fe6

lint fix

113fec4

log removed

8b284c3

addressing comments

f227659

move update block time

caf8f5c

param rename

fac407c

lint fix

3f87777

wip

fe1af66

time to atomic int

e703849

fix race cond

373c2cb

fix test

764fc02

merge

4405eb5

merge fix

3dd3e9a

merge fix

cf34896

test fix

86bbc31

srene marked this pull request as ready for review November 20, 2024 08:40

srene added 2 commits November 20, 2024 09:59

improving logs

4e9dc27

improving comments

e28cec4

srene requested review from mtsitrin and omritoptix November 20, 2024 11:06

mtsitrin approved these changes Nov 20, 2024

View reviewed changes

mtsitrin previously approved these changes Nov 21, 2024

View reviewed changes

comments

87d785d

srene dismissed mtsitrin’s stale review via 87d785d November 21, 2024 09:15

srene added 2 commits November 21, 2024 10:21

fix

e4de0e6

lint fix

e398d25

mtsitrin merged commit 34018b9 into main Nov 21, 2024
4 checks passed

mtsitrin deleted the srene/1139-max-skew-time branch November 21, 2024 09:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(manager): max skew based on time instead of batches #1140

feat(manager): max skew based on time instead of batches #1140

srene commented Oct 16, 2024 •

edited

Loading

codecov bot commented Oct 18, 2024

mtsitrin Nov 20, 2024

srene Nov 21, 2024

mtsitrin Nov 20, 2024

srene Nov 21, 2024

mtsitrin Nov 21, 2024

srene Nov 21, 2024

	MaxBatchSkewTime time.Duration `mapstructure:"max_skew_time"`
	MaxSkewTime time.Duration `mapstructure:"max_skew_time"`

	m.SetLastBlockTimeInSettlementFromHeight(uint64(m.Genesis.InitialHeight))
	m.LastBlockTimeInSettlement.Store(m.Genesis.GenesisTime.UTC().UnixNano())

feat(manager): max skew based on time instead of batches #1140

feat(manager): max skew based on time instead of batches #1140

Conversation

srene commented Oct 16, 2024 • edited Loading

PR Standards

Opening a pull request should be able to meet the following requirements

codecov bot commented Oct 18, 2024

Codecov Report

mtsitrin Nov 20, 2024

Choose a reason for hiding this comment

srene Nov 21, 2024

Choose a reason for hiding this comment

mtsitrin Nov 20, 2024

Choose a reason for hiding this comment

srene Nov 21, 2024

Choose a reason for hiding this comment

mtsitrin Nov 21, 2024

Choose a reason for hiding this comment

srene Nov 21, 2024

Choose a reason for hiding this comment

srene commented Oct 16, 2024 •

edited

Loading