Skip to content

Commit

Permalink
merge develop
Browse files Browse the repository at this point in the history
  • Loading branch information
@F4ever
F4ever committed Apr 13, 2023
2 parents 84e0cb3 + 3848dd9 commit d162e95
Show file tree
Hide file tree
Showing 19 changed files with 590 additions and 101 deletions.
34 changes: 34 additions & 0 deletions .github/workflows/build-image.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
name: Build Docker image

on:
workflow_dispatch:
pull_request:
branches:
- develop
- master
paths-ignore:
- ".github/**"
- "README.md"

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v3
with:
persist-credentials: false

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Build image
uses: docker/build-push-action@v3
with:
tags: app:ci
push: false
1 change: 1 addition & 0 deletions .github/workflows/ci-dev.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ on:
- develop
paths-ignore:
- ".github/**"
- "README.md"

permissions: {}

Expand Down
1 change: 1 addition & 0 deletions .github/workflows/ci-staging.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ on:
- master
paths-ignore:
- ".github/**"
- "README.md"

permissions: {}

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/create-tag-and-trigger-deploy.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
- name: Get tag value
id: tag
run: |
TAG="$(grep -oP '\d+\.\d+\.\d+' <<< "$MESSAGE")"
TAG="$(grep -oP '^chore\(release\).*\K(\d+\.\d+\.\d+)' <<< "$MESSAGE")"
echo "$TAG"
echo "tag=$TAG" >> $GITHUB_OUTPUT
env:
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Changes
- Oracle split in two separate modules: ejector and accounting

## [2.6.1](https://github.com/lidofinance/lido-oracle/releases/tag/2.6.1) - 2023-04-08
### Fixed
- Correctly handle missed slot.

## [2.4.0](https://github.com/lidofinance/lido-oracle/releases/tag/2.4.0) - 2022-10-04
### Changed
- Client for Prysm node has been changed to the same as for the lighthouse node.
Expand Down
45 changes: 24 additions & 21 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ Full variables list could be found [here](https://github.com/lidofinance/lido-or
Set required values. It will be enough to run the oracle in _check mode_.
2. Check that your environment is ready to run the oracle using the following command:
```bash
docker run --env-file .env --rm lidofinance/oracle:{tag} check
docker run -ti --env-file .env --rm lidofinance/oracle:{tag} check
```
If everything is ok, you will see that all required checks are passed
and your environment is ready to run the oracle.
Expand Down Expand Up @@ -142,25 +142,28 @@ Full variables list could be found [here](https://github.com/lidofinance/lido-or
## Env variables
| Name | Description | Required | Example value |
|----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------------------------|
| `EXECUTION_CLIENT_URI` | URI of the Execution Layer client | True | `http://localhost:8545` |
| `CONSENSUS_CLIENT_URI` | URI of the Consensus Layer client | True | `http://localhost:5052` |
| `KEYS_API_URI` | URI of the Keys API | True | `http://localhost:8080` |
| `LIDO_LOCATOR_ADDRESS` | Address of the Lido contract | True | `0x1...` |
| `MEMBER_PRIV_KEY` | Private key of the Oracle member account | False | `0x1...` |
| `MEMBER_PRIV_KEY_FILE` | A path to the file contained the private key of the Oracle member account. It takes precedence over `MEMBER_PRIV_KEY` | False | `/app/private_key` |
| `FINALIZATION_BATCH_MAX_REQUEST_COUNT` | The size of the batch to be finalized per request (The larger the batch size, the more memory of the contract is used but the fewer requests are needed) | False | `1000` |
| `ALLOW_REPORTING_IN_BUNKER_MODE` | Allow the Oracle to do report if bunker mode is active | False | `True` |
| `TX_GAS_ADDITION` | Used to modify gas parameter that used in transaction. (gas = estimated_gas + TX_GAS_ADDITION) | False | `1.75` |
| `CYCLE_SLEEP_IN_SECONDS` | The time between cycles of the oracle's activity | False | `12` |
| `SUBMIT_DATA_DELAY_IN_SLOTS` | The difference in slots between submit data transactions from Oracles. It is used to prevent simultaneous sending of transactions and, as a result, transactions revert. | False | `6` |
| `HTTP_REQUEST_RETRY_COUNT` | Total number of retries to fetch data from endpoint | False | `5` |
| `HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS` | The delay http provider sleeps if API is stuck | False | `12` |
| `HTTP_REQUEST_TIMEOUT` | Timeout for HTTP requests | False | `300` |
| `PRIORITY_FEE_PERCENTILE` | Priority fee percentile from prev block that would be used to send tx | False | `3` |
| `MIN_PRIORITY_FEE` | Min priority fee that would be used to send tx | False | `50000000` |
| `MAX_PRIORITY_FEE` | Max priority fee that would be used to send tx | False | `100000000000` |
| Name | Description | Required | Example value |
|--------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-------------------------|
| `EXECUTION_CLIENT_URI` | URI of the Execution Layer client | True | `http://localhost:8545` |
| `CONSENSUS_CLIENT_URI` | URI of the Consensus Layer client | True | `http://localhost:5052` |
| `KEYS_API_URI` | URI of the Keys API | True | `http://localhost:8080` |
| `LIDO_LOCATOR_ADDRESS` | Address of the Lido contract | True | `0x1...` |
| `MEMBER_PRIV_KEY` | Private key of the Oracle member account | False | `0x1...` |
| `MEMBER_PRIV_KEY_FILE` | A path to the file contained the private key of the Oracle member account. It takes precedence over `MEMBER_PRIV_KEY` | False | `/app/private_key` |
| `FINALIZATION_BATCH_MAX_REQUEST_COUNT` | The size of the batch to be finalized per request (The larger the batch size, the more memory of the contract is used but the fewer requests are needed) | False | `1000` |
| `ALLOW_REPORTING_IN_BUNKER_MODE` | Allow the Oracle to do report if bunker mode is active | False | `True` |
| `TX_GAS_ADDITION` | Used to modify gas parameter that used in transaction. (gas = estimated_gas + TX_GAS_ADDITION) | False | `100000` |
| `CYCLE_SLEEP_IN_SECONDS` | The time between cycles of the oracle's activity | False | `12` |
| `SUBMIT_DATA_DELAY_IN_SLOTS` | The difference in slots between submit data transactions from Oracles. It is used to prevent simultaneous sending of transactions and, as a result, transactions revert. | False | `6` |
| `HTTP_REQUEST_TIMEOUT_CONSENSUS` | Timeout for HTTP consensus layer requests | False | `300` |
| `HTTP_REQUEST_RETRY_COUNT_CONSENSUS` | Total number of retries to fetch data from endpoint for consensus layer requests | False | `5` |
| `HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS_CONSENSUS` | The delay http provider sleeps if API is stuck for consensus layer | False | `12` |
| `HTTP_REQUEST_TIMEOUT_KEYS_API` | Timeout for HTTP keys api requests | False | `10` |
| `HTTP_REQUEST_RETRY_COUNT_KEYS_API` | Total number of retries to fetch data from endpoint for keys api requests | False | `300` |
| `HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS_KEYS_API` | The delay http provider sleeps if API is stuck for keys api | False | `300` |
| `PRIORITY_FEE_PERCENTILE` | Priority fee percentile from prev block that would be used to send tx | False | `3` |
| `MIN_PRIORITY_FEE` | Min priority fee that would be used to send tx | False | `50000000` |
| `MAX_PRIORITY_FEE` | Max priority fee that would be used to send tx | False | `100000000000` |
### Alerts
Expand All @@ -178,7 +181,7 @@ groups:
severity: critical
annotations:
summary: "Dangerously low account balance"
description: "Account balance is less than 3 ETH. Address: {.labels.address}: {.value} ETH"
description: "Account balance is less than 1 ETH. Address: {.labels.address}: {.value} ETH"
- alert: OutdatedData
expr: (lido_oracle_genesis_time + ignoring (state) lido_oracle_slot_number{state="head"} * 12) < time() - 300
for: 1h
Expand Down
9 changes: 7 additions & 2 deletions src/modules/accounting/accounting.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,10 +71,15 @@ def execute_module(self, last_finalized_blockstamp: BlockStamp) -> ModuleExecute
return ModuleExecuteDelay.NEXT_FINALIZED_EPOCH

def process_extra_data(self, blockstamp: ReferenceBlockStamp):
latest_blockstamp = self._get_latest_blockstamp()
if not self.can_submit_extra_data(latest_blockstamp):
logger.info({'msg': 'Extra data can not be submitted.'})
return

chain_config = self.get_chain_config(blockstamp)
slots_to_sleep = self._get_slot_delay_before_data_submit(blockstamp)
seconds_to_sleep = slots_to_sleep * chain_config.seconds_per_slot
logger.info({'msg': f'Sleep for {seconds_to_sleep} before sending extra data.'})
logger.info({'msg': f'Sleep for {seconds_to_sleep} seconds before sending extra data.'})
sleep(seconds_to_sleep)

latest_blockstamp = self._get_latest_blockstamp()
Expand Down Expand Up @@ -237,7 +242,7 @@ def simulate_cl_rebase(self, blockstamp: ReferenceBlockStamp) -> LidoReportRebas
Simulate rebase excluding any execution rewards.
This used to check worst scenarios in bunker service.
"""
return self.simulate_rebase_after_report(blockstamp, Wei(0))
return self.simulate_rebase_after_report(blockstamp, el_rewards=Wei(0))

def simulate_full_rebase(self, blockstamp: ReferenceBlockStamp) -> LidoReportRebase:
el_rewards = self.w3.lido_contracts.get_el_vault_balance(blockstamp)
Expand Down
20 changes: 15 additions & 5 deletions src/providers/http_provider.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
from requests.exceptions import ConnectionError as RequestsConnectionError
from urllib3 import Retry

from src.variables import HTTP_REQUEST_RETRY_COUNT, HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS, HTTP_REQUEST_TIMEOUT
from src.web3py.extensions.consistency import ProviderConsistencyModule


logger = logging.getLogger(__name__)


Expand All @@ -35,17 +35,27 @@ class HTTPProvider(ProviderConsistencyModule, ABC):
Base HTTP Provider with metrics and retry strategy integrated inside.
"""
PROMETHEUS_HISTOGRAM: Histogram
request_timeout: int

def __init__(self, hosts: list[str]):
def __init__(
self,
hosts: list[str],
request_timeout: int,
retry_total: int,
retry_backoff_factor: int,
):
if not hosts:
raise NoHostsProvided(f"No hosts provided for {self.__class__.__name__}")

self.hosts = hosts
self.request_timeout = request_timeout
self.retry_count = retry_total
self.backoff_factor = retry_backoff_factor

retry_strategy = Retry(
total=HTTP_REQUEST_RETRY_COUNT,
total=self.retry_count,
status_forcelist=[418, 429, 500, 502, 503, 504],
backoff_factor=HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS,
backoff_factor=self.backoff_factor,
)

adapter = HTTPAdapter(max_retries=retry_strategy)
Expand Down Expand Up @@ -114,7 +124,7 @@ def _get_without_fallbacks(
response = self.session.get(
self._urljoin(host, complete_endpoint if path_params else endpoint),
params=query_params,
timeout=HTTP_REQUEST_TIMEOUT,
timeout=self.request_timeout,
)
except RequestsConnectionError as error:
logger.debug({'msg': str(error)})
Expand Down
9 changes: 4 additions & 5 deletions src/providers/keys/client.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
from src.typings import BlockStamp
from src.utils.dataclass import list_of_dataclasses
from src.utils.cache import global_lru_cache as lru_cache
from src import variables


class KeysOutdatedException(Exception):
Expand All @@ -33,17 +32,17 @@ def _get_with_blockstamp(self, url: str, blockstamp: BlockStamp, params: Optiona
"""
Returns response if blockstamp < blockNumber from response
"""
for i in range(variables.HTTP_REQUEST_RETRY_COUNT):
for i in range(self.retry_count):
data, meta = self._get(url, query_params=params)
blocknumber_meta = meta['meta']['elBlockSnapshot']['blockNumber']
KEYS_API_LATEST_BLOCKNUMBER.set(blocknumber_meta)
if blocknumber_meta >= blockstamp.block_number:
return data

if i != variables.HTTP_REQUEST_RETRY_COUNT - 1:
sleep(variables.HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS)
if i != self.retry_count - 1:
sleep(self.backoff_factor)

raise KeysOutdatedException(f'Keys API Service stuck, no updates for {variables.HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS * variables.HTTP_REQUEST_RETRY_COUNT} seconds.')
raise KeysOutdatedException(f'Keys API Service stuck, no updates for {self.backoff_factor * self.retry_count} seconds.')

@lru_cache(maxsize=1)
@list_of_dataclasses(LidoKey.from_response)
Expand Down
2 changes: 1 addition & 1 deletion src/services/validator_state.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from src.metrics.prometheus.accounting import (
ACCOUNTING_STUCK_VALIDATORS,
ACCOUNTING_EXITED_VALIDATORS,
ACCOUNTING_DELAYED_VALIDATORS
ACCOUNTING_DELAYED_VALIDATORS,
)
from src.modules.accounting.extra_data import ExtraDataService, ExtraData
from src.modules.accounting.typings import OracleReportLimits
Expand Down
15 changes: 12 additions & 3 deletions src/variables.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,18 @@
# If contract is reportable each member in order will submit data with difference with this amount of slots
SUBMIT_DATA_DELAY_IN_SLOTS = int(os.getenv('SUBMIT_DATA_DELAY_IN_SLOTS', 6))
CYCLE_SLEEP_IN_SECONDS = int(os.getenv('CYCLE_SLEEP_IN_SECONDS', 12))
HTTP_REQUEST_RETRY_COUNT = int(os.getenv('HTTP_REQUEST_RETRY_COUNT', 5))
HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS = int(os.getenv('HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS', 5))
HTTP_REQUEST_TIMEOUT = int(os.getenv('HTTP_REQUEST_TIMEOUT', 5 * 60))

HTTP_REQUEST_TIMEOUT_CONSENSUS = int(os.getenv('HTTP_REQUEST_TIMEOUT_CONSENSUS', 5 * 60))
HTTP_REQUEST_RETRY_COUNT_CONSENSUS = int(os.getenv('HTTP_REQUEST_RETRY_COUNT_CONSENSUS', 5))
HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS_CONSENSUS = int(
os.getenv('HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS_CONSENSUS', 5)
)

HTTP_REQUEST_TIMEOUT_KEYS_API = int(os.getenv('HTTP_REQUEST_TIMEOUT_KEYS_API', 10))
HTTP_REQUEST_RETRY_COUNT_KEYS_API = int(os.getenv('HTTP_REQUEST_RETRY_COUNT_KEYS_API', 5))
HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS_KEYS_API = int(
os.getenv('HTTP_REQUEST_SLEEP_BEFORE_RETRY_IN_SECONDS_KEYS_API', 5)
)

# - Metrics -
PROMETHEUS_PORT = int(os.getenv('PROMETHEUS_PORT', 9000))
Expand Down
Loading

0 comments on commit d162e95

Please sign in to comment.