Skip to content

Commit

Permalink
feat(fill): add a config file for the eels resolver to pin versions (#…
Browse files Browse the repository at this point in the history
…872)

* fix(fill): add a config file for the eels resolver to pin versions

This config is always used by fill and the framework tests. If the EELS_RESOLUTIONS_FILE is already set, this value is taken instead. This is done via environment variables (no command-line flag added) to be consistent with the resolver behavior.

* feat(pytest): copy eels resolutions file to the fixture .meta dir

* docs: update changelog

* tests(fw): fix filler tests due to introduction of eels_resolution.json

* Update src/pytest_plugins/eels_resolver.py

---------

Co-authored-by: Mario Vega <[email protected]>
  • Loading branch information
danceratopz and marioevz authored Oct 3, 2024
1 parent 96efd73 commit 7371ace
Show file tree
Hide file tree
Showing 7 changed files with 155 additions and 6 deletions.
7 changes: 4 additions & 3 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,10 @@ Test fixtures for use by clients are available for each release on the [Github r
- 🐞 Fix erroneous fork message in pytest session header with development forks ([#806](https://github.com/ethereum/execution-spec-tests/pull/806)).
- 🐞 Fix `Conditional` code generator in EOF mode ([#821](https://github.com/ethereum/execution-spec-tests/pull/821)).
- 🐞 Ensure that `Block` objects keep track of their `fork`, for example, when the block's `rlp_modifier` is not `None` ([#854](https://github.com/ethereum/execution-spec-tests/pull/854)).
- 🔀 `ethereum_test_rpc` library has been created with what was previously `ethereum_test_tools.rpc` ([#822](https://github.com/ethereum/execution-spec-tests/pull/822))
- ✨ Add `Wei` type to `ethereum_test_base_types` which allows parsing wei amounts from strings like "1 ether", "1000 wei", "10**2 gwei", etc ([#825](https://github.com/ethereum/execution-spec-tests/pull/825))
- 🔀 Replace `ethereum.base_types` with `ethereum-types` ([#850](https://github.com/ethereum/execution-spec-tests/pull/850))
- 🔀 `ethereum_test_rpc` library has been created with what was previously `ethereum_test_tools.rpc` ([#822](https://github.com/ethereum/execution-spec-tests/pull/822)).
- ✨ Add `Wei` type to `ethereum_test_base_types` which allows parsing wei amounts from strings like "1 ether", "1000 wei", "10**2 gwei", etc ([#825](https://github.com/ethereum/execution-spec-tests/pull/825)).
- ✨ Pin EELS versions in `eels_resolutions.json` and include this file in fixture releases ([#872](https://github.com/ethereum/execution-spec-tests/pull/872)).
- 🔀 Replace `ethereum.base_types` with `ethereum-types` ([#850](https://github.com/ethereum/execution-spec-tests/pull/850)).

### 🔧 EVM Tools

Expand Down
42 changes: 42 additions & 0 deletions eels_resolutions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"EELSMaster": {
"git_url": "https://github.com/ethereum/execution-specs.git",
"branch": "master",
"commit": "9b95554a88d2a8485f8180254d0f6a493a593fda"
},
"Frontier": {
"same_as": "EELSMaster"
},
"Homestead": {
"same_as": "EELSMaster"
},
"Byzantium": {
"same_as": "EELSMaster"
},
"Constantinople": {
"same_as": "EELSMaster"
},
"Istanbul": {
"same_as": "EELSMaster"
},
"Berlin": {
"same_as": "EELSMaster"
},
"London": {
"same_as": "EELSMaster"
},
"Paris": {
"same_as": "EELSMaster"
},
"Shanghai": {
"same_as": "EELSMaster"
},
"Cancun": {
"same_as": "EELSMaster"
},
"Prague": {
"git_url": "https://github.com/ethereum/execution-specs.git",
"branch": "prague",
"commit": "2875a733d6b1e9e751e437c7894d3ebe6ff58ecc"
}
}
1 change: 1 addition & 0 deletions pytest-framework.ini
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ markers =
run_in_serial
addopts =
-p pytester
-p pytest_plugins.eels_resolver
--ignore=src/pytest_plugins/consume/
--ignore=src/pytest_plugins/consume/direct/test_via_direct.py
--ignore=src/pytest_plugins/consume/hive_simulators/engine/test_via_engine.py
Expand Down
1 change: 1 addition & 0 deletions pytest.ini
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ addopts =
-p pytest_plugins.filler.filler
-p pytest_plugins.forks.forks
-p pytest_plugins.spec_version_checker.spec_version_checker
-p pytest_plugins.eels_resolver
-p pytest_plugins.help.help
-m "not eip_version_check"
--tb short
Expand Down
93 changes: 93 additions & 0 deletions src/pytest_plugins/eels_resolver.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
"""
Pytest plugin to help working with the `ethereum-spec-evm-resolver`.
This plugin sets the `EELS_RESOLUTIONS_FILE` environment variable to the path
of the `eels_resolutions.json` file in the pytest root directory. If the
environment variable is already set, the plugin will not override it.
"""

import os
import shutil
from pathlib import Path

import pytest
from pytest_metadata.plugin import metadata_key # type: ignore


def pytest_configure(config: pytest.Config) -> None:
"""
Set the EELS_RESOLUTIONS_FILE environment variable.
Args:
config (pytest.Config): The pytest configuration object.
"""
evm_bin = config.getoption("evm_bin", default=None)
if evm_bin and "resolver" not in str(evm_bin):
# evm_bin is not set for the framework tests: always set the env var.
return

env_var_name = "EELS_RESOLUTIONS_FILE"
eels_resolutions_file = os.getenv(env_var_name)

if os.getenv("EELS_RESOLUTIONS"):
# If the user sets this variable, assume they know what they're doing.
return

if eels_resolutions_file:
file_path = Path(eels_resolutions_file)
if not file_path.is_absolute():
raise ValueError(f"The path provided in {env_var_name} must be an absolute path.")
if not file_path.exists():
raise FileNotFoundError(
f"The file {file_path} does not exist. "
f"Ensure the {env_var_name} points to an existing file."
)
else:
root_dir = config.rootpath
default_file_path = root_dir / "eels_resolutions.json"
os.environ[env_var_name] = str(default_file_path)
eels_resolutions_file = str(default_file_path)

if "Tools" not in config.stash[metadata_key]:
config.stash[metadata_key]["Tools"] = {}
config.stash[metadata_key]["Tools"]["EELS Resolutions"] = str(eels_resolutions_file)

config._eels_resolutions_file = eels_resolutions_file # type: ignore


def pytest_report_header(config: pytest.Config, startdir: Path) -> str:
"""
Report the EELS_RESOLUTIONS_FILE path to the pytest report header.
Args:
config (pytest.Config): The pytest configuration object.
startdir (Path): The starting directory for the test run.
Returns:
str: A string to add to the pytest report header.
"""
eels_resolutions_file = getattr(config, "_eels_resolutions_file", None)
if eels_resolutions_file:
return f"EELS resolutions file: {eels_resolutions_file}"
return ""


@pytest.fixture(scope="session", autouse=True)
def output_metadata_dir_with_teardown(request):
"""
A session-scoped fixture that attempts to retrieve the filler's
"output_metadata_dir" fixture value and copies the EELS resolutions
file there, if `_eels_resolutions_file` is set on the config object.
"""
yield
try:
output_metadata_dir = request.getfixturevalue("output_metadata_dir")
except pytest.FixtureLookupError:
output_metadata_dir = None

eels_resolutions_file = getattr(request.config, "_eels_resolutions_file", None)
if output_metadata_dir and eels_resolutions_file:
shutil.copy(
Path(eels_resolutions_file),
Path(output_metadata_dir) / Path(eels_resolutions_file).name,
)
14 changes: 11 additions & 3 deletions src/pytest_plugins/filler/tests/test_filler.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

import pytest

from evm_transition_tool import ExecutionSpecsTransitionTool, TransitionTool
from pytest_plugins.filler.filler import default_output_directory


Expand Down Expand Up @@ -537,6 +538,9 @@ def test_fixture_output_based_on_command_line_args(

expected_ini_file = "fixtures.ini"
expected_index_file = "index.json"
expected_resolver_file = None
if TransitionTool.default_tool == ExecutionSpecsTransitionTool:
expected_resolver_file = "eels_resolutions.json"

ini_file = None
index_file = None
Expand All @@ -545,10 +549,14 @@ def test_fixture_output_based_on_command_line_args(
ini_file = file
elif file.name == expected_index_file:
index_file = file
elif expected_resolver_file and file.name == expected_resolver_file:
resolver_file = file
assert resolver_file.exists(), f"{resolver_file} does not exist"

all_fixtures = [
file for file in all_files if file.name not in {expected_ini_file, expected_index_file}
]
expected_additional_files = {expected_ini_file, expected_index_file}
if resolver_file:
expected_additional_files.add(expected_resolver_file)
all_fixtures = [file for file in all_files if file.name not in expected_additional_files]
for fixture_file, fixture_count in zip(expected_fixture_files, expected_fixture_counts):
assert fixture_file.exists(), f"{fixture_file} does not exist"
assert fixture_count == count_keys_in_fixture(
Expand Down
3 changes: 3 additions & 0 deletions whitelist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -511,6 +511,7 @@ parametrization
parametrizing
popen
prevrandao
pytestconfig
pytester
pytestmark
readline
Expand All @@ -521,8 +522,10 @@ ret
rglob
ripemd
rjust
rootpath
runpytest
runtest
startdir
subclasses
subcommand
subcontainer
Expand Down

0 comments on commit 7371ace

Please sign in to comment.