Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stakerShares state model #18

Merged
merged 12 commits into from
Sep 9, 2024
Merged

Add stakerShares state model #18

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
0304da0
Add stakerShares state model
seanmcgary Sep 9, 2024
6758331
Stub out tests, remove arm building for now
seanmcgary Sep 9, 2024
d3bbbae
Fix percent complete log value
seanmcgary Sep 9, 2024
1aba8d8
wip
seanmcgary Sep 9, 2024
25b2a1c
Better handling of numeric shares parsing
seanmcgary Sep 9, 2024
817befb
Better numeric parsing
seanmcgary Sep 9, 2024
b232ad4
Normalize addresses
seanmcgary Sep 9, 2024
a9430bb
Fix go mod naming
seanmcgary Sep 9, 2024
e34480c
Fix funky gitignore issue
seanmcgary Sep 9, 2024
c7b07c0
ignore sqlite* directories
seanmcgary Sep 9, 2024
c1e5229
use deps-linux in ci env
seanmcgary Sep 9, 2024
4b84ec1
make test
seanmcgary Sep 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
with:
go-version: 1.22
- name: Run tests
run: make test
run: make ci-test
build:
# needs: test
runs-on: ubuntu-latest
Expand All @@ -40,7 +40,7 @@ jobs:
REGISTRY: "public.ecr.aws/z6g0f8n7"
REPOSITORY: ${{ github.event.repository.name }}
IMAGE_TAG: ${{ github.sha }}
PLATFORMS: "linux/amd64,linux/arm64"
PLATFORMS: "linux/amd64"
run: |
if [[ $GITHUB_REF == refs/heads/master ]]; then
docker buildx build --platform $PLATFORMS -t $REGISTRY/$REPOSITORY:$IMAGE_TAG -t $REGISTRY/$REPOSITORY:latest --push .
Expand Down
7 changes: 2 additions & 5 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,9 @@
*.sw*
*.terraform
*.terraform*
scripts/runLocal/runBackfillBlockIndexerWorker.sh
scripts/runBackfillTransactionLogIndexerWorker.sh
block-lake/preprod-values.yaml
/bin
./block-lake/preprod-values.yaml
/scripts/runLocal
sidecar.db*
*.db*
sqlite
/sqlite
/sqlite*
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,4 @@ test:
TESTING=true go test -v -p 1 ./...

.PHONY: ci-test
ci-test: deps test
ci-test: test
42 changes: 23 additions & 19 deletions cmd/sidecar/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,26 @@ package main
import (
"context"
"fmt"
"github.com/Layr-Labs/sidecar/internal/clients/ethereum"
"github.com/Layr-Labs/sidecar/internal/clients/etherscan"
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/sidecar/internal/contractManager"
"github.com/Layr-Labs/sidecar/internal/contractStore/sqliteContractStore"
"github.com/Layr-Labs/sidecar/internal/eigenState/avsOperators"
"github.com/Layr-Labs/sidecar/internal/eigenState/operatorShares"
"github.com/Layr-Labs/sidecar/internal/eigenState/stakerDelegations"
"github.com/Layr-Labs/sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/sidecar/internal/fetcher"
"github.com/Layr-Labs/sidecar/internal/indexer"
"github.com/Layr-Labs/sidecar/internal/logger"
"github.com/Layr-Labs/sidecar/internal/metrics"
"github.com/Layr-Labs/sidecar/internal/pipeline"
"github.com/Layr-Labs/sidecar/internal/shutdown"
"github.com/Layr-Labs/sidecar/internal/sidecar"
"github.com/Layr-Labs/sidecar/internal/sqlite"
"github.com/Layr-Labs/sidecar/internal/sqlite/migrations"
sqliteBlockStore "github.com/Layr-Labs/sidecar/internal/storage/sqlite"
"github.com/Layr-Labs/go-sidecar/internal/clients/ethereum"
"github.com/Layr-Labs/go-sidecar/internal/clients/etherscan"
"github.com/Layr-Labs/go-sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/contractManager"
"github.com/Layr-Labs/go-sidecar/internal/contractStore/sqliteContractStore"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/avsOperators"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/operatorShares"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/stakerDelegations"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/stakerShares"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/go-sidecar/internal/fetcher"
"github.com/Layr-Labs/go-sidecar/internal/indexer"
"github.com/Layr-Labs/go-sidecar/internal/logger"
"github.com/Layr-Labs/go-sidecar/internal/metrics"
"github.com/Layr-Labs/go-sidecar/internal/pipeline"
"github.com/Layr-Labs/go-sidecar/internal/shutdown"
"github.com/Layr-Labs/go-sidecar/internal/sidecar"
"github.com/Layr-Labs/go-sidecar/internal/sqlite"
"github.com/Layr-Labs/go-sidecar/internal/sqlite/migrations"
sqliteBlockStore "github.com/Layr-Labs/go-sidecar/internal/storage/sqlite"
"go.uber.org/zap"
"log"
"time"
Expand Down Expand Up @@ -79,6 +80,9 @@ func main() {
if _, err := stakerDelegations.NewStakerDelegationsModel(sm, grm, cfg.Network, cfg.Environment, l, cfg); err != nil {
l.Sugar().Fatalw("Failed to create StakerDelegationsModel", zap.Error(err))
}
if _, err := stakerShares.NewStakerSharesModel(sm, grm, cfg.Network, cfg.Environment, l, cfg); err != nil {
l.Sugar().Fatalw("Failed to create StakerSharesModel", zap.Error(err))
}

fetchr := fetcher.NewFetcher(client, cfg, l)

Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
module github.com/Layr-Labs/sidecar
module github.com/Layr-Labs/go-sidecar

go 1.22.2

Expand Down
2 changes: 1 addition & 1 deletion internal/clients/etherscan/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package etherscan
import (
"encoding/json"
"fmt"
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/config"
"go.uber.org/zap"
"io"
"math/rand/v2"
Expand Down
2 changes: 1 addition & 1 deletion internal/contractCaller/contractCaller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package contractCaller
import (
"context"
"fmt"
"github.com/Layr-Labs/sidecar/internal/clients/ethereum"
"github.com/Layr-Labs/go-sidecar/internal/clients/ethereum"
"github.com/ethereum/go-ethereum/accounts/abi"
"github.com/ethereum/go-ethereum/accounts/abi/bind"
"github.com/ethereum/go-ethereum/common"
Expand Down
10 changes: 5 additions & 5 deletions internal/contractManager/contractManager.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ import (
"context"
"fmt"
"github.com/DataDog/datadog-go/v5/statsd"
"github.com/Layr-Labs/sidecar/internal/clients/ethereum"
"github.com/Layr-Labs/sidecar/internal/clients/etherscan"
"github.com/Layr-Labs/sidecar/internal/contractStore"
"github.com/Layr-Labs/sidecar/internal/metrics"
"github.com/Layr-Labs/sidecar/internal/utils"
"github.com/Layr-Labs/go-sidecar/internal/clients/ethereum"
"github.com/Layr-Labs/go-sidecar/internal/clients/etherscan"
"github.com/Layr-Labs/go-sidecar/internal/contractStore"
"github.com/Layr-Labs/go-sidecar/internal/metrics"
"github.com/Layr-Labs/go-sidecar/internal/utils"
"go.uber.org/zap"
)

Expand Down
6 changes: 3 additions & 3 deletions internal/contractStore/sqliteContractStore/sqliteContractStore.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ import (
"encoding/json"
"errors"
"fmt"
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/sidecar/internal/contractStore"
"github.com/Layr-Labs/sidecar/internal/sqlite"
"github.com/Layr-Labs/go-sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/contractStore"
"github.com/Layr-Labs/go-sidecar/internal/sqlite"
"go.uber.org/zap"
"golang.org/x/xerrors"
"gorm.io/gorm"
Expand Down
10 changes: 5 additions & 5 deletions internal/contractStore/sqliteContractStore/sqliteContractStore_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
package sqliteContractStore

import (
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/sidecar/internal/contractStore"
"github.com/Layr-Labs/sidecar/internal/logger"
"github.com/Layr-Labs/sidecar/internal/sqlite/migrations"
"github.com/Layr-Labs/sidecar/internal/tests"
"github.com/Layr-Labs/go-sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/contractStore"
"github.com/Layr-Labs/go-sidecar/internal/logger"
"github.com/Layr-Labs/go-sidecar/internal/sqlite/migrations"
"github.com/Layr-Labs/go-sidecar/internal/tests"
"github.com/stretchr/testify/assert"
"go.uber.org/zap"
"gorm.io/gorm"
Expand Down
16 changes: 8 additions & 8 deletions internal/eigenState/avsOperators/avsOperators.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ package avsOperators
import (
"database/sql"
"fmt"
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/sidecar/internal/eigenState/base"
"github.com/Layr-Labs/sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/sidecar/internal/eigenState/types"
"github.com/Layr-Labs/sidecar/internal/storage"
"github.com/Layr-Labs/sidecar/internal/utils"
"github.com/Layr-Labs/go-sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/base"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/types"
"github.com/Layr-Labs/go-sidecar/internal/storage"
"github.com/Layr-Labs/go-sidecar/internal/utils"
"github.com/wealdtech/go-merkletree/v2"
"github.com/wealdtech/go-merkletree/v2/keccak256"
orderedmap "github.com/wk8/go-ordered-map/v2"
Expand Down Expand Up @@ -123,8 +123,8 @@ func (a *AvsOperatorsModel) GetStateTransitions() (types.StateTransitions[Accumu
return nil, xerrors.Errorf("No state accumulator found for block %d", log.BlockNumber)
}

avs := arguments[0].Value.(string)
operator := arguments[1].Value.(string)
avs := strings.ToLower(arguments[0].Value.(string))
operator := strings.ToLower(arguments[1].Value.(string))

registered := false
if val, ok := outputData["status"]; ok {
Expand Down
12 changes: 6 additions & 6 deletions internal/eigenState/avsOperators/avsOperators_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@ package avsOperators

import (
"database/sql"
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/sidecar/internal/logger"
"github.com/Layr-Labs/sidecar/internal/sqlite/migrations"
"github.com/Layr-Labs/sidecar/internal/storage"
"github.com/Layr-Labs/sidecar/internal/tests"
"github.com/Layr-Labs/go-sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/go-sidecar/internal/logger"
"github.com/Layr-Labs/go-sidecar/internal/sqlite/migrations"
"github.com/Layr-Labs/go-sidecar/internal/storage"
"github.com/Layr-Labs/go-sidecar/internal/tests"
"github.com/stretchr/testify/assert"
"go.uber.org/zap"
"gorm.io/gorm"
Expand Down
4 changes: 2 additions & 2 deletions internal/eigenState/base/baseEigenState.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ package base
import (
"encoding/json"
"fmt"
"github.com/Layr-Labs/sidecar/internal/parser"
"github.com/Layr-Labs/sidecar/internal/storage"
"github.com/Layr-Labs/go-sidecar/internal/parser"
"github.com/Layr-Labs/go-sidecar/internal/storage"
"go.uber.org/zap"
"slices"
"strings"
Expand Down
14 changes: 7 additions & 7 deletions internal/eigenState/eigenstate_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
package eigenState

import (
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/sidecar/internal/eigenState/avsOperators"
"github.com/Layr-Labs/sidecar/internal/eigenState/operatorShares"
"github.com/Layr-Labs/sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/sidecar/internal/logger"
"github.com/Layr-Labs/sidecar/internal/sqlite/migrations"
"github.com/Layr-Labs/sidecar/internal/tests"
"github.com/Layr-Labs/go-sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/avsOperators"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/operatorShares"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/go-sidecar/internal/logger"
"github.com/Layr-Labs/go-sidecar/internal/sqlite/migrations"
"github.com/Layr-Labs/go-sidecar/internal/tests"
"github.com/stretchr/testify/assert"
"go.uber.org/zap"
"gorm.io/gorm"
Expand Down
43 changes: 30 additions & 13 deletions internal/eigenState/operatorShares/operatorShares.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,14 @@ package operatorShares

import (
"database/sql"
"encoding/json"
"fmt"
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/sidecar/internal/eigenState/base"
"github.com/Layr-Labs/sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/sidecar/internal/eigenState/types"
"github.com/Layr-Labs/sidecar/internal/storage"
"github.com/Layr-Labs/sidecar/internal/utils"
"github.com/Layr-Labs/go-sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/base"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/types"
"github.com/Layr-Labs/go-sidecar/internal/storage"
"github.com/Layr-Labs/go-sidecar/internal/utils"
"github.com/holiman/uint256"
"github.com/wealdtech/go-merkletree/v2"
"github.com/wealdtech/go-merkletree/v2/keccak256"
Expand Down Expand Up @@ -97,6 +98,24 @@ func (osm *OperatorSharesModel) GetModelName() string {
return "OperatorSharesModel"
}

type operatorSharesOutput struct {
Strategy string `json:"strategy"`
Shares json.Number `json:"shares"`
}

func parseLogOutputForOperatorShares(outputDataStr string) (*operatorSharesOutput, error) {
outputData := &operatorSharesOutput{}
decoder := json.NewDecoder(strings.NewReader(outputDataStr))
decoder.UseNumber()

err := decoder.Decode(&outputData)
if err != nil {
return nil, err
}
outputData.Strategy = strings.ToLower(outputData.Strategy)
return outputData, err
}

func (osm *OperatorSharesModel) GetStateTransitions() (types.StateTransitions[AccumulatedStateChange], []uint64) {
stateChanges := make(types.StateTransitions[AccumulatedStateChange])

Expand All @@ -105,7 +124,7 @@ func (osm *OperatorSharesModel) GetStateTransitions() (types.StateTransitions[Ac
if err != nil {
return nil, err
}
outputData, err := osm.ParseLogOutput(log)
outputData, err := parseLogOutputForOperatorShares(log.OutputData)
if err != nil {
return nil, err
}
Expand All @@ -114,11 +133,9 @@ func (osm *OperatorSharesModel) GetStateTransitions() (types.StateTransitions[Ac
if _, ok := osm.stateAccumulator[log.BlockNumber]; !ok {
return nil, xerrors.Errorf("No state accumulator found for block %d", log.BlockNumber)
}
operator := strings.ToLower(arguments[0].Value.(string))

operator := arguments[0].Value.(string)
strategy := outputData["strategy"].(string)
sharesStr := outputData["shares"].(string)

sharesStr := outputData.Shares.String()
shares, err := uint256.FromDecimal(sharesStr)
if err != nil {
osm.logger.Sugar().Errorw("Failed to convert shares to uint256",
Expand All @@ -136,12 +153,12 @@ func (osm *OperatorSharesModel) GetStateTransitions() (types.StateTransitions[Ac
shares = shares.Neg(shares)
}

slotId := NewSlotId(operator, strategy)
slotId := NewSlotId(operator, outputData.Strategy)
record, ok := osm.stateAccumulator[log.BlockNumber][slotId]
if !ok {
record = &AccumulatedStateChange{
Operator: operator,
Strategy: strategy,
Strategy: outputData.Strategy,
Shares: shares,
BlockNumber: log.BlockNumber,
}
Expand Down
12 changes: 6 additions & 6 deletions internal/eigenState/operatorShares/operatorShares_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ package operatorShares
import (
"database/sql"
"fmt"
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/sidecar/internal/logger"
"github.com/Layr-Labs/sidecar/internal/sqlite/migrations"
"github.com/Layr-Labs/sidecar/internal/storage"
"github.com/Layr-Labs/sidecar/internal/tests"
"github.com/Layr-Labs/go-sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/go-sidecar/internal/logger"
"github.com/Layr-Labs/go-sidecar/internal/sqlite/migrations"
"github.com/Layr-Labs/go-sidecar/internal/storage"
"github.com/Layr-Labs/go-sidecar/internal/tests"
"github.com/stretchr/testify/assert"
"go.uber.org/zap"
"gorm.io/gorm"
Expand Down
16 changes: 8 additions & 8 deletions internal/eigenState/stakerDelegations/stakerDelegations.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ package stakerDelegations
import (
"database/sql"
"fmt"
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/sidecar/internal/eigenState/base"
"github.com/Layr-Labs/sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/sidecar/internal/eigenState/types"
"github.com/Layr-Labs/sidecar/internal/storage"
"github.com/Layr-Labs/sidecar/internal/utils"
"github.com/Layr-Labs/go-sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/base"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/types"
"github.com/Layr-Labs/go-sidecar/internal/storage"
"github.com/Layr-Labs/go-sidecar/internal/utils"
"github.com/wealdtech/go-merkletree/v2"
"github.com/wealdtech/go-merkletree/v2/keccak256"
orderedmap "github.com/wk8/go-ordered-map/v2"
Expand Down Expand Up @@ -107,8 +107,8 @@ func (s *StakerDelegationsModel) GetStateTransitions() (types.StateTransitions[A
return nil, xerrors.Errorf("No state accumulator found for block %d", log.BlockNumber)
}

staker := arguments[0].Value.(string)
operator := arguments[1].Value.(string)
staker := strings.ToLower(arguments[0].Value.(string))
operator := strings.ToLower(arguments[1].Value.(string))

slotId := NewSlotId(staker, operator)
record, ok := s.stateAccumulator[log.BlockNumber][slotId]
Expand Down
12 changes: 6 additions & 6 deletions internal/eigenState/stakerDelegations/stakerDelegations_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@ package stakerDelegations

import (
"database/sql"
"github.com/Layr-Labs/sidecar/internal/config"
"github.com/Layr-Labs/sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/sidecar/internal/logger"
"github.com/Layr-Labs/sidecar/internal/sqlite/migrations"
"github.com/Layr-Labs/sidecar/internal/storage"
"github.com/Layr-Labs/sidecar/internal/tests"
"github.com/Layr-Labs/go-sidecar/internal/config"
"github.com/Layr-Labs/go-sidecar/internal/eigenState/stateManager"
"github.com/Layr-Labs/go-sidecar/internal/logger"
"github.com/Layr-Labs/go-sidecar/internal/sqlite/migrations"
"github.com/Layr-Labs/go-sidecar/internal/storage"
"github.com/Layr-Labs/go-sidecar/internal/tests"
"github.com/stretchr/testify/assert"
"go.uber.org/zap"
"gorm.io/gorm"
Expand Down
Loading
Loading