Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UCP/PROTO: Consider RNDV_PERF_DIFF #10401

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

rakhmets
Copy link
Contributor

What?

Applies UCX_RNDV_PERF_DIFF setting effect for protov2.

Cherry-picked from #10292.

Why?

This control effect was removed during introducing perf-factors logic. This PR brings it back.

That how it looks like with that patch:
image

@rakhmets
Copy link
Contributor Author

Test failures should be fixed with #10397.

@@ -717,6 +717,45 @@ UCS_TEST_F(test_proto_perf, intersect_first)
expect_empty_range(5000, SIZE_MAX);
}

UCS_TEST_F(test_proto_perf, apply_zero_bias) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering if it makes sense to add a mock test with RNDV_PERF_DIFF and make an assertion on selected protocols

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean remove this test, and add test case to test_ucp_proto_mock_rcx?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't insist on removing this test
But adding new test to test_ucp_proto_mock_rcx looks like superior alternative, because we test the entire chain

@@ -403,6 +403,7 @@ static void ucp_proto_rndv_ctrl_variant_probe(
cfg_thresh = remote_proto->cfg_thresh;
}

ucp_proto_perf_apply_bias(perf, params->perf_bias);
Copy link
Contributor

@iyastreb iyastreb Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems applying this bias leads to slightly different performance numbers and therefore breaks the mock test:
log1, log2, log3

Expected protocol:
{8247, 377094, "multi-frag copy-in"},
Actual protocol:
8247-363935 multi-frag copy-in

Could it be due to adding this bias? If so, we should update here the numbers in mock test as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is because of adding the bias. UCX selects rendezvous earlier with this changes.

@iyastreb
Copy link
Contributor

iyastreb commented Jan 6, 2025

Test failures should be fixed with #10397.

In fact none of the test failure can be fixed by that PR. They are triggered by changed perf calculation logic I guess

iyastreb
iyastreb previously approved these changes Jan 14, 2025
Copy link
Contributor

@iyastreb iyastreb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing my comments
Please note that there is approved PR for mock refactoring, that will be merged soon. You might need to slightly adapt mock test code

@rakhmets rakhmets force-pushed the topic/proto-consider-rndv-perf-diff branch from aaa08bd to db2470a Compare January 15, 2025 14:31
@rakhmets rakhmets requested review from iyastreb and brminich January 17, 2025 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants