Solana staking improvements #6496
Conversation
jeff-dude
added
the
ready-for-review
|
hmm this used to run in 37 minutes but idk why it is suddenly now timing out. maybe we can bump the cluster for this one, it really shouldn't be that compute expensive |
|
oh it ran @jeff-dude , data looks good to me |
jeff-dude
added
in review
...ubprojects/solana/models/_sector/staking/solana/staking_solana_stake_account_delegations.sql
Show resolved
Hide resolved
...jects/solana/models/_sector/staking/solana/staking_solana_validator_stake_account_epochs.sql
Show resolved
Hide resolved
...s/solana/models/_sector/staking/solana/staking_solana_validator_stake_account_epochs_raw.sql
Show resolved
Hide resolved
| , materialized = 'incremental' | ||
| , file_format = 'delta' | ||
| , incremental_strategy = 'merge' | ||
| , unique_key = ['block_time', 'block_slot', 'stake_account', 'vote_account'] |
There was a problem hiding this comment.
looks like this model is failing the unique test, but only after incremental run. that usually means incorrect keys. i noticed this model has 4 total columns, 4 keys, meaning nothing to update when match found. can we simplify the keys to less columns?
then the other thing to consider is null values in a key, which would fail the lookup. can any be null?
There was a problem hiding this comment.
none of the keys can be full. and yes this would only ever append, and not update. I put this unique key here so that we just don't end up with duplicates for overlapping incremental runs. we can't remove any of them because stake accounts could move delegations between vote accounts (and vice versa) any time.
There was a problem hiding this comment.
okay, only other option should be the target filter. i've removed to try again.
jeff-dude
added
ready-for-merging
and removed
WIP
Adding a few solana improvement tables.
"solana_utils.epochs": this is a simple table that I'm creating for easier query UX around epochs on Solana
"staking_solana.stake_account_delegations": vote delegations of accounts on solana are non-trivial to pull because of merge/split/withdraw accounts. This is an intermediary table to pull all historical validators delegated by a stake account
"staking_solana.validator_stake_account_epochs": this is a table that combines the prior two to get all stake accounts delegated to a validator BEFORE an epoch began. This then allows us to more accurately calculate active stake and stake rewards in in-app queries, which has been heavily requested by solflare, steakhouse, helius, and other teams.