Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V13 backports #9862

Merged
merged 48 commits into from
Jan 28, 2025
Merged
Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
30266c6
Make checking parent alive more portable
eivindjahren Jan 20, 2025
b437827
Add check-all to justfile
eivindjahren Jan 20, 2025
5518481
Add unit tests to pre-commit
oyvindeide Jan 20, 2025
bf7d182
Remove redundant apt download in run-ert-test-data
larsevj Jan 20, 2025
a1a2d59
Add integration test label to potentially conflicting tests
xjules Jan 21, 2025
91f0484
Trust the queue system when a job is finished in tests
berland Jan 20, 2025
fe97d1b
Mark ES_MDA as recommended algorithm in GUI
berland Jan 16, 2025
b4ac65e
Fix rerun button not disabled for ES-MDA
jonathan-eq Jan 21, 2025
70c89e6
zmq: replace server_started Event with Future to handle exceptions
xjules Jan 15, 2025
411bfc5
Add everest to rapid tests
eivindjahren Jan 22, 2025
0dfb678
Adapt Slurm setup to Ubuntu 24.04
berland Jan 21, 2025
e70740d
Skip slow tests in mac ci
eivindjahren Jan 22, 2025
53f78e3
Fix mac select string
eivindjahren Jan 22, 2025
bef4261
Add tests for flow vs OMP_NUM_THREADS
berland Jan 21, 2025
f4eb706
Fix unable to have path object in argslist for localdriver
jonathan-eq Jan 21, 2025
8dfe8ab
Divide instead of multiply with observation errors in misfit preproce…
larsevj Jan 22, 2025
c065695
Add event serialization testing
DanSava Dec 17, 2024
9824ff6
Reduce test matrix for flow and slurm tests
eivindjahren Jan 23, 2025
ceade12
Increase timeout of killed_ert_does_not_leave_storage_server_process
eivindjahren Jan 23, 2025
f0019af
Skip testing lsf/openpbs in slurm specific test
berland Jan 23, 2025
e2b6898
Do not split integration_test in everest
eivindjahren Jan 23, 2025
aad9552
Have parameter config keep parameters types for design matrix
jonathan-eq Jan 22, 2025
0937c6e
Fix bug where only literal error had position info
frode-aarstad Jan 21, 2025
9fde13f
zmq: implement task for heeartbeats on clients to detect reconnection
xjules Jan 17, 2025
eb8e652
Make the run dialogue snappier by tuning batching interval
berland Jan 21, 2025
f7fcf5a
Remove tests for OMP_NUM_THREADS
berland Jan 23, 2025
516578b
Fix mac failing on main
eivindjahren Jan 23, 2025
4c136de
Install correct dependency and adjust slurm config
berland Jan 23, 2025
7e179f6
Combine running of unit and performance tests
eivindjahren Jan 23, 2025
92ce748
Fix deprecated use of get_event_loop
eivindjahren Jan 23, 2025
993b3d8
Remove unused async_utils.py
eivindjahren Jan 23, 2025
234fc9f
Ensure SETENV is also substituted when writing json
berland Jan 23, 2025
3abeea9
Fix get_priors test
eivindjahren Jan 24, 2025
4c337a0
Raise ConfigValidationError on short parameter file lines
eivindjahren Jan 24, 2025
1d199a0
Make design_matrix parameter_configuration a single GenKw element
xjules Jan 22, 2025
785eb69
zmq: Speed up tests using MockZMQServer
eivindjahren Jan 24, 2025
920efa8
zmq: Add integration_test marker to slow tests
eivindjahren Jan 24, 2025
54e6444
zmq: Speed up test_invalid_server
eivindjahren Jan 24, 2025
b7795d9
Speed up test_default_installed_jobs
eivindjahren Jan 24, 2025
7914ce3
Speed up lsf driver tests
eivindjahren Jan 24, 2025
421971c
Fix everest tests failing when using editable installs
larsevj Jan 24, 2025
9e44c8a
zmq: add test that monitor can't connect to a secured server
xjules Dec 30, 2024
625b256
Use tmp_path for test_everest_to_ert_controls
larsevj Jan 24, 2025
4a2e5e3
Ensure cpu_seconds is always reported increasingly
berland Jan 24, 2025
dce4e05
zmq: make sure client disconnects before moczmqserver closes
xjules Jan 24, 2025
b01c868
Add no_dealers Event to mockzmqserver
xjules Jan 24, 2025
ed7ce1a
Change port range of dark storage
JHolba Jan 28, 2025
ad04429
zmq: await server started first then create all evaluator tasks
xjules Jan 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Skip slow tests in mac ci
eivindjahren authored and xjules committed Jan 24, 2025
commit e70740d2b9546261d454bb952652b69b6e101c4d
1 change: 1 addition & 0 deletions .github/workflows/build_and_test.yml
Original file line number Diff line number Diff line change
@@ -109,6 +109,7 @@ jobs:
test-type: [ 'performance-tests', 'unit-tests', 'gui-tests', 'cli-tests' ]
python-version: [ '3.12' ]
os: [ 'macos-latest']
select-string: ['"not skip_mac_ci"']

uses: ./.github/workflows/test_ert.yml
with:
11 changes: 7 additions & 4 deletions .github/workflows/test_ert.yml
Original file line number Diff line number Diff line change
@@ -7,6 +7,9 @@ on:
type: string
test-type:
type: string
select-string:
type: string
default: "''"

env:
ERT_SHOW_BACKTRACE: 1
@@ -40,7 +43,7 @@ jobs:
- name: GUI Test
if: inputs.test-type == 'gui-tests'
run: |
pytest --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -v --mpl --benchmark-disable tests/ert/ui_tests/gui --durations=25
pytest -m ${{ inputs.select-string }} --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -v --mpl --benchmark-disable tests/ert/ui_tests/gui --durations=25

- name: Upload artifact images
uses: actions/upload-artifact@v4
@@ -53,18 +56,18 @@ jobs:
- name: CLI Test
if: inputs.test-type == 'cli-tests'
run: |
pytest --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -v --benchmark-disable --dist loadgroup tests/ert/ui_tests/cli --durations=25
pytest -m ${{ inputs.select-string }} --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -v --benchmark-disable --dist loadgroup tests/ert/ui_tests/cli --durations=25

- name: Unit Test
if: inputs.test-type == 'unit-tests'
run: |
pytest --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -n logical --show-capture=stderr -v --benchmark-disable --mpl --dist loadgroup tests/ert/unit_tests --durations=25
pytest -m ${{ inputs.select-string }} --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -n logical --show-capture=stderr -v --benchmark-disable --mpl --dist loadgroup tests/ert/unit_tests --durations=25
pytest --doctest-modules --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov2.xml src/ --ignore src/ert/dark_storage

- name: Performance Test
if: inputs.test-type == 'performance-tests'
run: |
pytest --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -n logical --show-capture=stderr -v --benchmark-disable --dist loadgroup tests/ert/performance_tests --durations=25
pytest -m ${{ inputs.select-string }} --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -n logical --show-capture=stderr -v --benchmark-disable --dist loadgroup tests/ert/performance_tests --durations=25

- name: Upload coverage to Codecov
id: codecov1
4 changes: 2 additions & 2 deletions .github/workflows/test_everest.yml
Original file line number Diff line number Diff line change
@@ -46,7 +46,7 @@ jobs:
- name: Run Tests macOS
if: ${{ inputs.test-type == 'test' && runner.os == 'macOS'}}
run: |
python -m pytest tests/everest -n 4 --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -m "not integration_test and not fails_on_macos_github_workflow" --dist loadgroup -sv
python -m pytest tests/everest -n 4 --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -m "not integration_test and not skip_mac_ci" --dist loadgroup -sv

- name: Run Integration Tests Linux
if: ${{inputs.test-type == 'integration-test' && runner.os != 'macOS'}}
@@ -56,7 +56,7 @@ jobs:
- name: Run Integration Tests macOS
if: ${{inputs.test-type == 'integration-test' && runner.os == 'macOS'}}
run: |
python -m pytest tests/everest -n 4 --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -m "integration_test and not fails_on_macos_github_workflow" --dist loadgroup
python -m pytest tests/everest -n 4 --cov=ert --cov=everest --cov=_ert --cov-report=xml:cov1.xml --junit-xml=junit.xml -o junit_family=legacy -m "integration_test and not skip_mac_ci" --dist loadgroup

- name: Build Documentation
if: inputs.test-type == 'doc'
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -171,7 +171,7 @@ markers = [
"slow",
"everest_models_test",
"integration_test",
"fails_on_macos_github_workflow", # Tests marked fail due to gui-related issues
"skip_mac_ci",
]
log_cli = "false"
asyncio_mode = "auto"
3 changes: 2 additions & 1 deletion tests/ert/ui_tests/cli/test_parameter_passing.py
Original file line number Diff line number Diff line change
@@ -19,7 +19,7 @@
import xtgeo
from hypothesis import given, note, settings
from hypothesis.extra.numpy import arrays
from pytest import MonkeyPatch, TempPathFactory
from pytest import MonkeyPatch, TempPathFactory, mark
from resdata import ResDataType
from resdata.grid import GridGenerator
from resdata.resfile import ResdataKW
@@ -409,6 +409,7 @@ def check(self, io_source: IoProvider, mask, num_realizations: int):
max_size=3,
),
)
@mark.skip_mac_ci # test is slow
def test_that_parameters_are_placed_in_the_runpath_as_expected(
io_source: IoProvider,
grid_format: Literal["grid", "egrid"],
1 change: 1 addition & 0 deletions tests/ert/ui_tests/gui/test_plotting_of_snake_oil.py
Original file line number Diff line number Diff line change
@@ -121,6 +121,7 @@ def plot_figure(qtbot, heat_equation_storage, snake_oil_case_storage, request):
# The tolerance is chosen by guess, in one bug we observed a
# mismatch of 58 which would fail the test by being above 10.0
@pytest.mark.mpl_image_compare(tolerance=10.0)
@pytest.mark.skip_mac_ci # test is slow
def test_that_all_snake_oil_visualisations_matches_snapshot(plot_figure):
return plot_figure

4 changes: 3 additions & 1 deletion tests/ert/unit_tests/dark_storage/test_dark_storage_state.py
Original file line number Diff line number Diff line change
@@ -111,4 +111,6 @@ def teardown(self):
del os.environ["ERT_STORAGE_ENS_PATH"]


TestDarkStorage = pytest.mark.integration_test(DarkStorageStateTest.TestCase)
TestDarkStorage = pytest.mark.skip_mac_ci(
pytest.mark.integration_test(DarkStorageStateTest.TestCase)
)
1 change: 1 addition & 0 deletions tests/ert/unit_tests/gui/model/test_snapshot.py
Original file line number Diff line number Diff line change
@@ -10,6 +10,7 @@


@pytest.mark.integration_test
@pytest.mark.skip_mac_ci # slow
def test_using_qt_model_tester(qtmodeltester, full_snapshot):
model = SnapshotModel()

10 changes: 5 additions & 5 deletions tests/everest/entry_points/test_everexport.py
Original file line number Diff line number Diff line change
@@ -101,7 +101,7 @@ def test_everexport_entry_empty(mocked_func, cached_example):
side_effect=validate_export_mock,
)
@patch("everest.bin.utils.export_data")
@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
def test_everexport_entry_batches(mocked_func, validate_export_mock, cached_example):
"""Test running everexport with the --batches flag"""
_, config_file, _ = cached_example("math_func/config_minimal.yml")
@@ -156,7 +156,7 @@ def test_everexport_entry_empty_export(mocked_func, cached_example):


@patch("everest.bin.utils.export_data")
@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
def test_everexport_entry_no_usr_def_ecl_keys(mocked_func, cached_example):
"""Test running everexport with config file containing only the
keywords label without any list of keys"""
@@ -190,7 +190,7 @@ def condition(config):


@patch("everest.bin.utils.export_data")
@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
def test_everexport_entry_internalized_usr_def_ecl_keys(mocked_func, cached_example):
"""Test running everexport with config file containing a key in the
list of user defined ecl keywords, that has been internalized on
@@ -226,7 +226,7 @@ def condition(config):


@patch("everest.bin.utils.export_data")
@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
def test_everexport_entry_non_int_usr_def_ecl_keys(mocked_func, caplog, cached_example):
"""Test running everexport when config file contains non internalized
ecl keys in the user defined keywords list"""
@@ -269,7 +269,7 @@ def condition(config):


@patch("everest.bin.utils.export_data")
@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
def test_everexport_entry_not_available_batches(mocked_func, caplog, cached_example):
"""Test running everexport when config file contains non existing
batch numbers in the list of user defined batches"""
2 changes: 1 addition & 1 deletion tests/everest/functional/test_main_everest_entry.py
Original file line number Diff line number Diff line change
@@ -41,7 +41,7 @@ def test_everest_main_entry_bad_command():


@pytest.mark.flaky(reruns=5)
@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
@pytest.mark.integration_test
def test_everest_entry_run(cached_example):
_, config_file, _ = cached_example("math_func/config_minimal.yml")
2 changes: 1 addition & 1 deletion tests/everest/test_detached.py
Original file line number Diff line number Diff line change
@@ -42,7 +42,7 @@

@pytest.mark.flaky(reruns=5)
@pytest.mark.integration_test
@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
@pytest.mark.xdist_group(name="starts_everest")
async def test_https_requests(copy_math_func_test_data_to_tmp):
everest_config = EverestConfig.load_file("config_minimal.yml")
2 changes: 1 addition & 1 deletion tests/everest/test_everlint.py
Original file line number Diff line number Diff line change
@@ -333,7 +333,7 @@ def test_date_type(date, valid, min_config):
EverestConfig(**min_config)


@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
def test_lint_everest_models_jobs():
pytest.importorskip("everest_models")
config_file = relpath("../../test-data/everest/egg/everest/model/config.yml")
2 changes: 1 addition & 1 deletion tests/everest/test_res_initialization.py
Original file line number Diff line number Diff line change
@@ -241,7 +241,7 @@ def test_summary_default(copy_egg_test_data_to_tmp):
@hide_opm
@skipif_no_everest_models
@pytest.mark.everest_models_test
@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
def test_summary_default_no_opm(copy_egg_test_data_to_tmp):
config_dir = "everest/model"
config_file = os.path.join(config_dir, "config.yml")
4 changes: 2 additions & 2 deletions tests/everest/test_yaml_parser.py
Original file line number Diff line number Diff line change
@@ -106,7 +106,7 @@ def test_valid_config_file(copy_test_data_to_tmp, monkeypatch):
assert "could not find expected ':'" in parser.get_error()


@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
@skipif_no_everest_models
@pytest.mark.everest_models_test
def test_valid_forward_model_config_files(copy_test_data_to_tmp, monkeypatch):
@@ -121,7 +121,7 @@ def test_valid_forward_model_config_files(copy_test_data_to_tmp, monkeypatch):

@skipif_no_everest_models
@pytest.mark.everest_models_test
@pytest.mark.fails_on_macos_github_workflow
@pytest.mark.skip_mac_ci
def test_invalid_forward_model_config_files(copy_test_data_to_tmp, monkeypatch):
monkeypatch.chdir("valid_config_file/forward_models")
parser = MockParser()