Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: peerdas - partially downloaded batch #7354

Open
wants to merge 58 commits into
base: peerDAS
Choose a base branch
from

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jan 13, 2025

Motivation

  • when we partially download a batch, right now we trigger batch processor

Description

  • trigger batch downloader again instead, because we have pending data columns to download
  • once batch is fully downloaded, we trigger batch processor
  • improve log

cc @matthewkeil @g11tech

g11tech and others added 30 commits August 9, 2024 13:56
add types stub and epoch config

fix types
add some presets

add further params and types

add data column to types repo and network

move to max request data columns to preset

add the datacolumns data in blockinput and fix breaking errors in seen gossip blockinput

handle data columns in gossip and the seengossip

further propagate forkaware blockdata and resolve build/type issues

further handle datacolumns sync by range by root and forkaware data handling

fix issues

chore: update c-kzg to peerDas version

feat: add peerDas ckzg functions to interface

fix the lookups

handle the publishing flow

various sync try fixes

fixes

compute blob side car

various misl debuggings and fixes

debug and apply fixes and get range and by root sync to work will full custody

enable syncing with lower custody requirement

use node peerid rather than a dummy string

get and use the nodeid from enr and correctly compute subnets and column indexes

filterout and connect to peers only matching out custody requiremnt

try adding custody requirement

add protection for subnet calc

get the sync working with devnet 0

correctly set the enr with custody subnet info

rebase fixes

small refactor
wip: REPLACE THIS COMMIT

commit yarn lock

rebase fixes

fix: update c-zkg install workflow

feat: add trustedSetupPrecompute cli flag

fix: update trusted-setup for testing

fix: update c-zkg install workflow to remove sudo

fix: add rsync to apk deps
some network options to control peering behavior

allow setting node custody capability via --params

use eip754 names for the peerdas config
rebase fixes
fixes for metadata, working locally

change the condition to update metadata csc change
g11tech and others added 27 commits September 12, 2024 13:57
* refactor: getDataColumnSidecars

* test: unit test getDataColumnSidecars with mocks from c-kzg library

* refactor: use fromHex util

* chore: update numbering on mocks

* chore: update c-kzg to latest version

* chore: fix type export syntax

* test: add verification for cells from sidecars

* test: add verification to DataColumnSidecars tests

* refactor: getDataColumnSidecars for PR comments

* feat: narrow type and remove unnecessary conditional

* fix: getDataColumnSidecars param type

* refactor: rename to computeDataColumnSidecars
* feat: update c-kzg to final DAS version

* refactor: use trusted-setup from c-kzg package
* fix: remove unused ckzg build script

* fix: remove unused rsync dep from Dockerfile
* feat: update c-kzg to final DAS version

* refactor: use trusted-setup from c-kzg package

* feat: implement validateDataColumnsSidecars

* feat: check block and column commitments match

* test: add unit test for validateDataColumnsSidecars

* fix: invalid build and update validity condition of validateDataColumnsSidecars

* fix: make error messages better

* fix: electra vs peerdas type conflict
…on (#7184)

* feat: check for no commitments on block or column in sidecar validation

* test: add sanity check for empty blob commitments in column validation

* fix: test bug

* fix: but in test passing commitments
* feat: log disconnect reason

* feat: log peerScore update

* fix: pretty print peerId

* fix: use prettyPrintPeerId
* refactor: organize peerDAS types

* refactor: DataColumnsData

* refactor: rename BlockInputBlobs BlockInputColumnData

* refactor: split up and rename BlockInputData

* refactor: clean up BlobsData

* refactor: clean up CachedData types

* refactor: change from interface to type and update enum values for grafana

* chore: lint

* fix: remove extraneous lint fix
@twoeths twoeths requested a review from a team as a code owner January 13, 2025 03:57
@philknows philknows mentioned this pull request Feb 5, 2025
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants