Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(p2p): avoid receiving duplicated blocks #818

Merged
merged 11 commits into from
May 15, 2024
Merged

fix(p2p): avoid receiving duplicated blocks #818

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
22e476e
adding ttl to gossipsub router seen messages
srene May 7, 2024
dd39da9
update new client
srene May 7, 2024
0603f78
skipping onNewGossipedBlock for duplicates
srene May 7, 2024
2f3fa90
merge
srene May 8, 2024
22ddcf6
modifying gossipsub seen ttl
srene May 9, 2024
b082aa0
mutex fix
srene May 9, 2024
62f0476
mutex fix
srene May 9, 2024
7805c69
removed extra log
srene May 9, 2024
4d8a382
update gossipsub version and remove seen ttl messages option
srene May 13, 2024
126fbc5
comment added
srene May 13, 2024
a4c4ae8
Update block/manager.go
srene May 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions block/manager.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,21 +237,28 @@ func (m *Manager) onNodeHealthStatus(event pubsub.Message) {
// TODO: move to gossip.go
// onNewGossippedBlock will take a block and apply it
func (m *Manager) onNewGossipedBlock(event pubsub.Message) {
m.retrieverMutex.Lock() // needed to protect blockCache access
eventData := event.Data().(p2p.GossipedBlock)
block := eventData.Block
commit := eventData.Commit
m.logger.Debug("Received new block via gossip", "height", block.Header.Height, "n cachedBlocks", len(m.blockCache))
m.retrieverMutex.Lock() // needed to protect blockCache access
_, found := m.blockCache[block.Header.Height]
mtsitrin marked this conversation as resolved.
Show resolved Hide resolved
// It is not strictly necessary to return early, for correctness, but doing so helps us avoid mutex pressure and unnecessary repeated attempts to apply cached blocks
if found {
m.retrieverMutex.Unlock()
return
}
danwt marked this conversation as resolved.
Show resolved Hide resolved

m.logger.Debug("Received new block via gossip", "block height", block.Header.Height, "store height", m.Store.Height(), "n cachedBlocks", len(m.blockCache))
danwt marked this conversation as resolved.
Show resolved Hide resolved

nextHeight := m.Store.NextHeight()
if block.Header.Height >= nextHeight {
m.blockCache[block.Header.Height] = CachedBlock{
Block: &block,
Commit: &commit,
}
m.logger.Debug("caching block", "block height", block.Header.Height, "store height", m.Store.Height())
}
m.retrieverMutex.Unlock() // have to give this up as it's locked again in attempt apply, and we're not re-entrant

err := m.attemptApplyCachedBlocks()
if err != nil {
m.logger.Error("applying cached blocks", "err", err)
Expand Down
79 changes: 37 additions & 42 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ go 1.22.2
require (
code.cloudfoundry.org/go-diodes v0.0.0-20220725190411-383eb6634c40
cosmossdk.io/errors v1.0.1
cosmossdk.io/math v1.3.0
github.com/avast/retry-go/v4 v4.5.0
github.com/celestiaorg/celestia-openrpc v0.4.0-rc.1
github.com/celestiaorg/go-cnc v0.4.2
Expand All @@ -21,15 +20,15 @@ require (
github.com/google/uuid v1.4.0
github.com/gorilla/mux v1.8.1
github.com/gorilla/rpc v1.2.0
github.com/gorilla/websocket v1.5.0
github.com/gorilla/websocket v1.5.1
github.com/hashicorp/go-multierror v1.1.1
github.com/ignite/cli v0.26.1
github.com/informalsystems/tm-load-test v1.3.0
github.com/libp2p/go-libp2p v0.30.0
github.com/libp2p/go-libp2p-kad-dht v0.21.1
github.com/libp2p/go-libp2p v0.33.1
github.com/libp2p/go-libp2p-kad-dht v0.25.2
github.com/libp2p/go-libp2p-pubsub v0.9.3
github.com/multiformats/go-multiaddr v0.11.0
github.com/prometheus/client_golang v1.16.0
github.com/multiformats/go-multiaddr v0.12.2
github.com/prometheus/client_golang v1.18.0
github.com/rs/cors v1.9.0
github.com/spf13/cobra v1.8.0
github.com/spf13/viper v1.15.0
Expand Down Expand Up @@ -72,25 +71,23 @@ require (
github.com/ethereum/go-ethereum v1.12.0 // indirect
github.com/filecoin-project/go-jsonrpc v0.3.1 // indirect
github.com/ghodss/yaml v1.0.0 // indirect
github.com/go-logr/logr v1.2.4 // indirect
github.com/go-logr/logr v1.3.0 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-stack/stack v1.8.1 // indirect
github.com/golang/mock v1.6.0 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/pprof v0.0.0-20230817174616-7a8ec2ada47b // indirect
github.com/google/pprof v0.0.0-20240207164012-fb44976bdcd5 // indirect
github.com/hashicorp/go-uuid v1.0.1 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.5 // indirect
github.com/ipfs/go-log v1.0.5 // indirect
github.com/klauspost/reedsolomon v1.11.8 // indirect
github.com/lib/pq v1.10.7 // indirect
github.com/libp2p/go-yamux/v4 v4.0.1 // indirect
github.com/minio/highwayhash v1.0.2 // indirect
github.com/onsi/ginkgo/v2 v2.11.0 // indirect
github.com/onsi/ginkgo/v2 v2.15.0 // indirect
github.com/pierrec/xxHash v0.1.5 // indirect
github.com/quic-go/qpack v0.4.0 // indirect
github.com/quic-go/qtls-go1-20 v0.3.2 // indirect
github.com/quic-go/quic-go v0.37.6 // indirect
github.com/quic-go/webtransport-go v0.5.3 // indirect
github.com/quic-go/quic-go v0.42.0 // indirect
github.com/quic-go/webtransport-go v0.6.0 // indirect
github.com/satori/go.uuid v1.2.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/tidwall/btree v1.5.0 // indirect
Expand All @@ -104,9 +101,9 @@ require (
go.opentelemetry.io/otel v1.19.0 // indirect
go.opentelemetry.io/otel/metric v1.19.0 // indirect
go.opentelemetry.io/otel/trace v1.19.0 // indirect
go.uber.org/dig v1.17.0 // indirect
go.uber.org/fx v1.20.0 // indirect
golang.org/x/exp v0.0.0-20230817173708-d852ddb80c63 // indirect
go.uber.org/dig v1.17.1 // indirect
go.uber.org/fx v1.20.1 // indirect
golang.org/x/exp v0.0.0-20240213143201-ec583247a57a // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240125205218-1f4bbc51befe // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240125205218-1f4bbc51befe // indirect
Expand Down Expand Up @@ -135,7 +132,7 @@ require (
github.com/dustin/go-humanize v1.0.1-0.20200219035652-afde56e7acac // indirect
github.com/dvsekhvalnov/jose2go v1.5.0 // indirect
github.com/elastic/gosigar v0.14.2 // indirect
github.com/flynn/noise v1.0.0 // indirect
github.com/flynn/noise v1.1.0 // indirect
github.com/francoispqt/gojay v1.2.13 // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/go-kit/log v0.2.1 // indirect
Expand All @@ -161,36 +158,33 @@ require (
github.com/hashicorp/golang-lru v0.5.5-0.20210104140557-80c98217689d // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hdevalence/ed25519consensus v0.0.0-20220222234857-c00d1f31bab3 // indirect
github.com/huin/goupnp v1.2.0 // indirect
github.com/huin/goupnp v1.3.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/ipfs/go-cid v0.4.1 // indirect
github.com/ipfs/go-datastore v0.6.0 // indirect
github.com/ipfs/go-ipfs-util v0.0.2 // indirect
github.com/ipfs/go-ipns v0.2.0 // indirect
github.com/ipfs/go-log/v2 v2.5.1 // indirect
github.com/ipld/go-ipld-prime v0.9.0 // indirect
github.com/ipld/go-ipld-prime v0.20.0 // indirect
github.com/jackpal/go-nat-pmp v1.0.2 // indirect
github.com/jbenet/go-temp-err-catcher v0.1.0 // indirect
github.com/jbenet/goprocess v0.1.4 // indirect
github.com/jmhodges/levigo v1.0.0 // indirect
github.com/klauspost/compress v1.16.7 // indirect
github.com/klauspost/cpuid/v2 v2.2.5 // indirect
github.com/klauspost/compress v1.17.6 // indirect
github.com/klauspost/cpuid/v2 v2.2.7 // indirect
github.com/koron/go-ssdp v0.0.4 // indirect
github.com/libp2p/go-buffer-pool v0.1.0 // indirect
github.com/libp2p/go-cidranger v1.1.0 // indirect
github.com/libp2p/go-flow-metrics v0.1.0 // indirect
github.com/libp2p/go-libp2p-asn-util v0.3.0 // indirect
github.com/libp2p/go-libp2p-kbucket v0.5.0 // indirect
github.com/libp2p/go-libp2p-asn-util v0.4.1 // indirect
github.com/libp2p/go-libp2p-kbucket v0.6.3 // indirect
github.com/libp2p/go-libp2p-record v0.2.0 // indirect
github.com/libp2p/go-msgio v0.3.0 // indirect
github.com/libp2p/go-nat v0.2.0 // indirect
github.com/libp2p/go-netroute v0.2.1 // indirect
github.com/libp2p/go-reuseport v0.4.0 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/miekg/dns v1.1.55 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/miekg/dns v1.1.58 // indirect
github.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b // indirect
github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc // indirect
github.com/mimoo/StrobeGo v0.0.0-20220103164710-9a04d6ca976b // indirect
Expand All @@ -205,17 +199,17 @@ require (
github.com/multiformats/go-multibase v0.2.0 // indirect
github.com/multiformats/go-multicodec v0.9.0 // indirect
github.com/multiformats/go-multihash v0.2.3 // indirect
github.com/multiformats/go-multistream v0.4.1 // indirect
github.com/multiformats/go-multistream v0.5.0 // indirect
github.com/multiformats/go-varint v0.0.7 // indirect
github.com/opencontainers/runtime-spec v1.1.0 // indirect
github.com/opencontainers/runtime-spec v1.2.0 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect
github.com/pelletier/go-toml/v2 v2.0.7 // indirect
github.com/petermattis/goid v0.0.0-20230317030725-371a4b8eda08 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/polydawn/refmt v0.0.0-20201211092308-30ac6d18308e // indirect
github.com/prometheus/client_model v0.4.0 // indirect
github.com/prometheus/procfs v0.10.1 // indirect
github.com/polydawn/refmt v0.89.0 // indirect
github.com/prometheus/client_model v0.6.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
github.com/raulk/go-watchdog v1.3.0 // indirect
github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
github.com/sasha-s/go-deadlock v0.3.1 // indirect
Expand All @@ -233,10 +227,10 @@ require (
github.com/zondax/hid v0.9.2 // indirect
go.etcd.io/bbolt v1.3.8 // indirect
go.opencensus.io v0.24.0 // indirect
go.uber.org/zap v1.25.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/crypto v0.22.0 // indirect
golang.org/x/mod v0.12.0 // indirect
golang.org/x/sync v0.5.0 // indirect
golang.org/x/mod v0.15.0 // indirect
golang.org/x/sync v0.6.0 // indirect
golang.org/x/sys v0.19.0 // indirect
golang.org/x/term v0.19.0 // indirect
golang.org/x/text v0.14.0 // indirect
Expand All @@ -248,6 +242,7 @@ require (
)

require (
cosmossdk.io/math v1.3.0 // indirect
github.com/DataDog/zstd v1.5.2 // indirect
github.com/agl/ed25519 v0.0.0-20170116200512-5312a6153412 // indirect
github.com/blang/semver/v4 v4.0.0 // indirect
Expand All @@ -264,25 +259,25 @@ require (
github.com/getsentry/sentry-go v0.18.0 // indirect
github.com/gopherjs/gopherjs v0.0.0-20190812055157-5d271430af9f // indirect
github.com/holiman/uint256 v1.2.2 // indirect
github.com/ipfs/boxo v0.10.0 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/libp2p/go-libp2p-routing-helpers v0.7.2 // indirect
github.com/linxGnu/grocksdb v1.8.12 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/prometheus/common v0.42.0 // indirect
github.com/prometheus/common v0.47.0 // indirect
github.com/regen-network/cosmos-proto v0.3.1 // indirect
github.com/rogpeppe/go-internal v1.11.0 // indirect
github.com/smartystreets/assertions v1.0.1 // indirect
golang.org/x/tools v0.12.1-0.20230815132531-74c255bcf846 // indirect
go.uber.org/mock v0.4.0 // indirect
golang.org/x/tools v0.18.0 // indirect
)

replace (
github.com/centrifuge/go-substrate-rpc-client/v4 => github.com/availproject/go-substrate-rpc-client/v4 v4.0.12-avail-1.4.0-rc1-5e286e3
github.com/evmos/evmos/v12 => github.com/dymensionxyz/evmos/v12 v12.1.6-dymension-v0.3

github.com/gogo/protobuf => github.com/regen-network/protobuf v1.3.2-alpha.regen.4
github.com/gorilla/rpc => github.com/dymensionxyz/rpc v1.3.1
github.com/libp2p/go-libp2p-pubsub => github.com/dymensionxyz/go-libp2p-pubsub v0.0.0-20231220125656-9e904c648367

github.com/libp2p/go-libp2p-pubsub => github.com/dymensionxyz/go-libp2p-pubsub v0.0.0-20240513081713-3ecd83c19ea2
github.com/tendermint/tendermint => github.com/cometbft/cometbft v0.34.28

)
Expand Down
Loading
Loading