Operator node reachability scanner #506

pschork · 2024-04-20T01:45:32Z

The EigenDA Node Reachability Scanner allows Operators to initiate port scans from the EigenDA backend to validate e2e reachability.

┌─────────────────────────────────────────────┐               ┌─────────────────────────────────────────────┐
│                                             │               │                                             │
│            EigenDA Operator Node            │               │            EigenDA Operator Node            │
│                                             │               │                                             │
│                                             │               │                                             │
└───────┬────────▲────────────────▲───────────┘               └───────┬─────────────────────────────────────┘
┌───────┴────────┴────────────────┴───────────┐               ┌───────┴─────────────────────────────────────┐
│       Operator Firewall Allowing 32005      │               │       Operator Firewall Blocking 32005      │
└───────┬────────┬────────────────┬───────────┘               └───────┬─────────────────────────────────────┘
        │        │                │                                   │        X                X
    PortCheck    │                │                               PortCheck    │                │
     Request     │                │                                Request     │                │
                 │                │                                            │                │
        │        │                │                                   │        │                │
        │        │                │                                   │        │                │
        │        │                │                                   │        │                │
        │        │                │                                   │        │                │
        │  Reachability       Dispersal                               │  Reachability       Dispersal
        │      Check           Request                                │      Check           Request
        │      32005            32005                                 │      32005            32005
┌───────┴────────┴────────────────┴───────────┐               ┌───────┴────────┴────────────────┴───────────┐
│                  Nat Gateway                │               │                  Nat Gateway                │
└───────┬────────┬────────────────┬───────────┘               └───────┬────────┬────────────────┬───────────┘
┌───────┼────────┼────────────────┼───────────┐               ┌───────┼────────┼────────────────┼───────────┐
│       │        │                │           │               │       │        │                │           │
│       ▼        │                │           │               │       ▼        │                │           │
│ ┌──────────────┴────┐┌──────────┴────────┐  │               │ ┌──────────────┴────┐┌──────────┴────────┐  │
│ │                   ││                   │  │               │ │                   ││                   │  │
│ │      dataapi      ││     disperser     │  │               │ │      dataapi      ││     disperser     │  │
│ │                   ││                   │  │               │ │                   ││                   │  │
│ └───────────────────┘└───────────────────┘  │               │ └───────────────────┘└───────────────────┘  │
│                EigenDA VPC                  │               │                EigenDA VPC                  │
└─────────────────────────────────────────────┘               └─────────────────────────────────────────────┘

jianoaix

Can we just add an API to dataapi for this check? It shouldn't need another server and new interface under api/

scanner/metrics.go

scanner/scanner.go

ian-shim

Few comments:

Can we move this under tools/ directory?
I think it should have some automated testing

scanner/grpc/server.go

api/proto/scanner/scanner.proto

scanner/grpc/server.go

jianoaix · 2024-04-25T18:16:42Z

Can we just add an API to dataapi for this check? It shouldn't need another server and new interface under api/

@pschork

jianoaix

LG

jianoaix · 2024-04-26T19:50:10Z

disperser/dataapi/server.go

@@ -586,6 +599,36 @@ func (s *server) FetchDeregisteredOperators(c *gin.Context) {
 	})
 }



Do we need another API which takes the socket as parameter? That could be used by CLI before registration.

This OperatorPortCheck will still be useful for creating a metric at Node to check reachability periodically.

Adding generic socket port check opens the door to it being abused as a general purpose penetration testing service. Forcing operator_id avoids a swath of potential attacks on the endpoint/dataapi in general.

pschork · 2024-04-26T21:29:07Z

200 OK

curl -X 'GET' \
  'https://dataapi-preprod-holesky.eigenda.xyz/api/v1/operators-info/port-check?operator_id=0xa96bfb4a7ca981ad365220f336dc5a3de0816ebd5130b79bbc85aca94bc9b6ab' \
  -H 'accept: application/json'

{
  "operator_id": "0xa96bfb4a7ca981ad365220f336dc5a3de0816ebd5130b79bbc85aca94bc9b6ab",
  "disperser_socket": "23.93.76.39:32005",
  "retriever_socket": "23.93.76.39:32004",
  "disperser_online": true,
  "retriever_online": true
}

404 Not Found

curl -X 'GET' \
  'https://dataapi-preprod-holesky.eigenda.xyz/api/v1/operators-info/port-check?operator_id=asdf' \
  -H 'accept: application/json'

{
"error": "not found"
}

jianoaix

LGTM! We can add a metric to node based on this API

disperser/dataapi/docs/swagger.yaml

…rt scans from the EigenDA backend to validate e2e reachability. ``` ┌─────────────────────────────────────────────┐ ┌─────────────────────────────────────────────┐ │ │ │ │ │ EigenDA Operator Node │ │ EigenDA Operator Node │ │ │ │ │ │ │ │ │ └───────┬────────▲────────────────▲───────────┘ └───────┬─────────────────────────────────────┘ ┌───────┴────────┴────────────────┴───────────┐ ┌───────┴─────────────────────────────────────┐ │ Operator Firewall Allowing 32005 │ │ Operator Firewall Blocking 32005 │ └───────┬────────┬────────────────┬───────────┘ └───────┬─────────────────────────────────────┘ │ │ │ │ X X PortCheck │ │ PortCheck │ │ Request │ │ Request │ │ 32005 │ │ 32005 │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ Reachability Dispersal │ Reachability Dispersal │ Check Request │ Check Request │ 32005 32005 │ 32005 32005 ┌───────┴────────┴────────────────┴───────────┐ ┌───────┴────────┴────────────────┴───────────┐ │ Nat Gateway │ │ Nat Gateway │ └───────┬────────┬────────────────┬───────────┘ └───────┬────────┬────────────────┬───────────┘ ┌───────┼────────┼────────────────┼───────────┐ ┌───────┼────────┼────────────────┼───────────┐ │ │ │ │ │ │ │ │ │ │ │ ▼ │ │ │ │ ▼ │ │ │ │ ┌──────────────┴────┐┌──────────┴────────┐ │ │ ┌──────────────┴────┐┌──────────┴────────┐ │ │ │ ││ │ │ │ │ ││ │ │ │ │ dataapi ││ disperser │ │ │ │ dataapi ││ disperser │ │ │ │ ││ │ │ │ │ ││ │ │ │ └───────────────────┘└───────────────────┘ │ │ └───────────────────┘└───────────────────┘ │ │ EigenDA VPC │ │ EigenDA VPC │ └─────────────────────────────────────────────┘ └─────────────────────────────────────────────┘ ```

Lint

Fix tests

pschork requested review from siddimore, dmanc and jianoaix April 20, 2024 01:45

jianoaix reviewed Apr 20, 2024

View reviewed changes

dmanc reviewed Apr 20, 2024

View reviewed changes

scanner/metrics.go Outdated Show resolved Hide resolved

scanner/scanner.go Outdated Show resolved Hide resolved

scanner/scanner.go Outdated Show resolved Hide resolved

ian-shim reviewed Apr 23, 2024

View reviewed changes

scanner/grpc/server.go Outdated Show resolved Hide resolved

scanner/grpc/server.go Outdated Show resolved Hide resolved

scanner/grpc/server.go Outdated Show resolved Hide resolved

api/proto/scanner/scanner.proto Outdated Show resolved Hide resolved

siddimore reviewed Apr 23, 2024

View reviewed changes

scanner/grpc/server.go Outdated Show resolved Hide resolved

pschork force-pushed the pschork/scanner branch 2 times, most recently from 8ce4434 to e2236b4 Compare April 25, 2024 17:02

pschork force-pushed the pschork/scanner branch 3 times, most recently from e9eb1bd to fdc52a8 Compare April 26, 2024 01:13

jianoaix reviewed Apr 26, 2024

View reviewed changes

pschork force-pushed the pschork/scanner branch 2 times, most recently from aba080f to 2734155 Compare April 26, 2024 20:46

pschork marked this pull request as ready for review April 26, 2024 21:29

pschork force-pushed the pschork/scanner branch 3 times, most recently from 272d5f8 to 1063b0a Compare April 26, 2024 21:41

jianoaix approved these changes Apr 26, 2024

View reviewed changes

jianoaix reviewed Apr 26, 2024

View reviewed changes

disperser/dataapi/docs/swagger.yaml Outdated Show resolved Hide resolved

disperser/dataapi/docs/swagger.yaml Outdated Show resolved Hide resolved

jianoaix reviewed Apr 26, 2024

View reviewed changes

disperser/dataapi/docs/swagger.yaml Outdated Show resolved Hide resolved

pschork force-pushed the pschork/scanner branch 2 times, most recently from 2e697bd to 07e5a6f Compare April 27, 2024 00:19

pschork added 3 commits April 29, 2024 16:54

Not found error handling + metrics

30b2805

Update swagger docs

ae8bee9

Lint

pschork force-pushed the pschork/scanner branch from 07e5a6f to cb67a1e Compare April 30, 2024 00:03

Unit tests

6c789dc

Fix tests

pschork force-pushed the pschork/scanner branch from c8bdcf4 to 6c789dc Compare April 30, 2024 07:44

pschork added 3 commits April 30, 2024 01:29

Debug

43478ad

Debug

ae643c1

Fix IPv6 validation

d7f5a4b

pschork merged commit 979707d into master Apr 30, 2024
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Operator node reachability scanner #506

Operator node reachability scanner #506

pschork commented Apr 20, 2024 •

edited

Loading

jianoaix left a comment

ian-shim left a comment

jianoaix commented Apr 25, 2024

jianoaix left a comment

jianoaix Apr 26, 2024

pschork Apr 26, 2024

pschork commented Apr 26, 2024

jianoaix left a comment •

edited

Loading

		@@ -586,6 +599,36 @@ func (s server) FetchDeregisteredOperators(c gin.Context) {
		})
		}

Operator node reachability scanner #506

Operator node reachability scanner #506

Conversation

pschork commented Apr 20, 2024 • edited Loading

jianoaix left a comment

Choose a reason for hiding this comment

ian-shim left a comment

Choose a reason for hiding this comment

jianoaix commented Apr 25, 2024

jianoaix left a comment

Choose a reason for hiding this comment

jianoaix Apr 26, 2024

Choose a reason for hiding this comment

pschork Apr 26, 2024

Choose a reason for hiding this comment

pschork commented Apr 26, 2024

200 OK

404 Not Found

jianoaix left a comment • edited Loading

Choose a reason for hiding this comment

pschork commented Apr 20, 2024 •

edited

Loading

jianoaix left a comment •

edited

Loading