V13 backports #9862
Merged
V13 backports #9862
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Checking if parent process is 1 does not e.g. work on ubuntu where the adopting process is systemd --user which is unlikely to be 1.
This avoids always spending 120 billable cpu-seconds on every test. Instead we trust that the queue system is correct when it claims a job is finished, instead of waiting it out.
This commit fixes the issue where the rerun button was enabled for ES-MDA and ensemble smoother. The issue was due to us setting support_restart before calling the super class's constructor which overwrote it to False.
In a very special case zmq server might fail during initialization and all occurrences of server_started.wait() will wait indefinitely and therefore replacing it with asyncio.Future which provides additional exception trigger.
This serves as documentation and verification of the current interplay between ert, the plugin configuration of ert and the flowrun wrapper, and the flow binary itself.
Also add more debug info
Also use multiple workers since many tests do a lot of sleeping.
This commit makes enkf_main's _value_export_json keep types when dumping the json. Before this commit, all values (also numerical) were turned to strings if used alongside categorical values.
Heartbeat task sends HEARTBEAT to all the clients (ie. Monitor) at client.HEARTBEAT_TIMEOUT intervals. Clients do not reply, just process the message. If client detects longer delay between two heartbeats, the client will send CONNECT to evaluator in addition; ie. getting the connection re-established after a break. This is to simulate re-connection. Each CONNECT_MSG will then trigger sending FullSnapshot from the ensemble evaluator. Initially HEARTBEAT_TIMEOUT is set to 5 seconds while Monitor accepts 10 seconds at max as a delay. Additionally, initial connection will now undergo same amount of retries as standard messages.
The batching interval of 2 seconds is legacy from the time when Ert was a mixture of Python and C, and a lot of threading issues attached. The underlying message structure and message processing infrastructure now handles a lot more messages, and the GUI can thus appear more responsive to the incoming messages from compute nodes.
This test depends on a separate implementation of a flowrun executable, and an example of this executabe just changed implementation to always overwrite the environment variable OMP_NUM_THREADS in order to successfully control the behaviour of flow.
Due to integration-test not being a test type any more
Dramatic speedup in test time (~10-fold)
This is a regression from 1ae12f6
eivindjahren
approved these changes
Jan 24, 2025
CodSpeed Performance ReportMerging #9862 will not alter performanceComparing Summary
Benchmarks breakdown
|
eivindjahren
approved these changes
Jan 28, 2025
The dictionary introduced in parameter configuration in design matrix config seems to be not necessary and thus removing it. The type will those become GenKwConfig directly.
A simple test that provides either wrong or good server key.
The summation of cpu_seconds for a process and all its descendants can never work properly during teardown of a process tree, as the root process typically outlives its children. Thus, the maximum observed cpu_seconds for a process tree is always the best estimate of the correct sum.
This also makes sure that there are no lingering events when closing zmq server socket.
There is an issue where dark storage would try to use the same port as ensemble evaluator. Switching the port range for dark storage should fix this problem until we can fix the bug.
Additionally, _server_started can be removed from the tasks. The commits also adds a test for zmq start up failure when port already in use
eivindjahren
approved these changes
Jan 28, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue
Backports to version-13
Approach
Short description of the approach
(Screenshot of new behavior in GUI if applicable)
git rebase -i main --exec 'pytest tests/ert/unit_tests tests/everest -n auto --hypothesis-profile=fast -m "not integration_test"')When applicable