Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: validator set persistence #200

Draft
wants to merge 24 commits into
base: master
Choose a base branch
from
Draft

WIP: validator set persistence #200

wants to merge 24 commits into from

Conversation

pk910
Copy link
Member

@pk910 pk910 commented Dec 18, 2024

This PR adds a validators table to the db and persists the finalized validator set there.

The validator cache is responsible for writing and updating validator entries.
It uses a batched approach for mass updates to keep the db load manageable.

This PR touches a lot of code as the validator set accessors changed.
Instead of fetching the full validator set and doing filtering/aggregation logic in the page handlers, the page handlers will now only fetch a filtered subset of the validator set.

This PR should significantly lower the memory consumption of dora when running in a network with large validator sets (holesky, mainnet)

@pk910 pk910 added the build-docker-image Automatically build docker image for PR branch label Jan 24, 2025
@pk910
Copy link
Member Author

pk910 commented Jan 25, 2025

This branch is now running on https://dora-holesky.pk910.de/ for the holesky testnet.
The first results look very promising - dora runs with a total memory consumption of ~3-4GB :)
Compared to a consumption of 10GB before the update, that's quite an improvement.

Issues found:

  • Active Validators, Staked Ether & Average Balance stats on index page show 0 during the first few slots of a new epoch (latest epoch stats are still loading)
  • Validators list status filter does not work

@pk910 pk910 linked an issue Feb 2, 2025 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build-docker-image Automatically build docker image for PR branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Duty calculation is wrong with unbalanced validator set
1 participant