Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use Index map in PGM #1639

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from
Open

Use Index map in PGM #1639

wants to merge 5 commits into from

Conversation

MarcelKoch
Copy link
Member

@MarcelKoch MarcelKoch commented Jul 2, 2024 •
edited
Loading

This PR enables using an index map to create the coarse level using PGM for distributed matrices. IMO this simplifies the PGM implementation, since there is no 'manual' implementation needed anymore.
This also has some consequences for the distributed::Matrix class. Namely, when creating a matrix with from local and non-local parts, an index map has to be supplied, instead of the list of vectors. Again, I think this should simplify the interface. Note that currently there is no accessor to the index map, since it's not necessary. But it might be useful for consistency.

I guess this is also an interface breaking change.

Todo:

@MarcelKoch MarcelKoch self-assigned this Jul 2, 2024
@MarcelKoch MarcelKoch requested a review from yhmtsai July 2, 2024 14:50
@ginkgo-bot ginkgo-bot added reg:testing This is related to testing. mod:core This is related to the core module. mod:cuda This is related to the CUDA module. mod:reference This is related to the reference module. mod:hip This is related to the HIP module. type:multigrid This is related to multigrid labels Jul 2, 2024
@MarcelKoch MarcelKoch requested a review from a team July 2, 2024 14:51
@MarcelKoch MarcelKoch marked this pull request as draft July 2, 2024 14:53
@MarcelKoch MarcelKoch force-pushed the index-map-pgm branch 2 times, most recently from 56d235d to b0e5f6d Compare July 5, 2024 11:07
@MarcelKoch MarcelKoch marked this pull request as ready for review July 5, 2024 11:07
@MarcelKoch MarcelKoch force-pushed the index-map-pgm branch 2 times, most recently from f594348 to 87d62d3 Compare July 10, 2024 09:23
yhmtsai
yhmtsai reviewed Jul 10, 2024
View reviewed changes
core/multigrid/pgm.cpp Outdated
Comment on lines 284 to 285
auto seng_global_agg = imap.map_to_global(
send_agg, experimental::distributed::index_space::local);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I guess it correctly, you map the local index back to the global index to avoid the collision.
this is what I want to avoid because it needs twice of memory transferring between nodes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would guess that the actual volume of the data transfer is negligible compared to the latency cost. Also, this is significantly simpler to what was there before.

@MarcelKoch MarcelKoch force-pushed the index-map-pgm branch 3 times, most recently from 9e52a2c to ded4dd3 Compare July 16, 2024 07:43
@MarcelKoch MarcelKoch force-pushed the index-map-pgm branch 2 times, most recently from 9e52a2c to ded4dd3 Compare July 18, 2024 16:40
@MarcelKoch MarcelKoch added the 1:ST:ready-for-review This PR is ready for review label Aug 15, 2024
@MarcelKoch MarcelKoch added this to the Ginkgo 1.9.0 milestone Aug 26, 2024
@MarcelKoch MarcelKoch requested review from yhmtsai and a team and removed request for a team August 27, 2024 12:02
@MarcelKoch MarcelKoch requested review from yhmtsai and removed request for yhmtsai September 16, 2024 12:48
@upsj upsj self-requested a review February 13, 2025 11:50
@fritzgoebel fritzgoebel self-requested a review February 13, 2025 11:50
fritzgoebel
fritzgoebel requested changes Feb 14, 2025
View reviewed changes
Copy link
Collaborator

@fritzgoebel fritzgoebel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM generally, minor changes

@MarcelKoch MarcelKoch mentioned this pull request Feb 14, 2025
Open
4 tasks
yhmtsai
yhmtsai requested changes Feb 14, 2025
View reviewed changes
Copy link
Member

@yhmtsai yhmtsai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does the #1589 require this?
I think some of matrix function are deleted in another pr.

@MarcelKoch
Copy link
Member Author

@yhmtsai Yes #1589 needs this, because it removes some data members from the dist matrix class which the PGM accessed. Getting this data is not possible anymore when the dist matrix uses the row gatherer, since the data is hidden by the collective communicator from #1780. The issue is solved in the dist row gatherer pr, by letting the collective communicator handle the communication in the PGM.
I guess it should be possible in principle to also use the collective communicator in the PGM, without these changes, but I don't understand the current implementation, so I'm not able to work on that.

@MarcelKoch MarcelKoch force-pushed the index-map-pgm branch 2 times, most recently from c432ffc to f8cb0e8 Compare February 18, 2025 15:46
@MarcelKoch MarcelKoch requested a review from yhmtsai February 20, 2025 09:31
@MarcelKoch MarcelKoch force-pushed the global-to-local branch 2 times, most recently from 285ff97 to 71ac6a4 Compare February 27, 2025 12:37
MarcelKoch added a commit that referenced this pull request Feb 28, 2025
@MarcelKoch
Merge Add local to global index mapping
7404a82 
This PR adds a local-to-global mapping to the `index_map` class. It is necessary for the follow-up PR #1639. This also introduces a device-side view of a partition, and adds another member to the partition which stores the range-ids segmented by their part-id.

Related PR: #1707
Base automatically changed from global-to-local to develop February 28, 2025 07:52
MarcelKoch and others added 5 commits March 3, 2025 11:48
@MarcelKoch
[dist-mat] store index map
9812671 
this also changes one constructor

Signed-off-by: Marcel Koch <[email protected]>
@MarcelKoch
[pgm] use index map to create coarse matrix
df866fc 
Signed-off-by: Marcel Koch <[email protected]>
@MarcelKoch
[dist] keep old interface, but deprecate it
a5c45ab
@MarcelKoch @fritzgoebel
[pgm] review updates:
2f1c552 
- documentation
- refactor

Co-authored-by: Fritz Goebel <[email protected]>
Co-authored-by: Yu-Hsiang M. Tsai <[email protected]>u
@MarcelKoch
[bench] update benchmark tests
b734fec
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fritzgoebel fritzgoebel fritzgoebel approved these changes

@upsj upsj Awaiting requested review from upsj

@yhmtsai yhmtsai Awaiting requested review from yhmtsai

Requested changes must be addressed to merge this pull request.

Assignees

@MarcelKoch MarcelKoch

Labels
1:ST:ready-for-review This PR is ready for review mod:core This is related to the core module. mod:cuda This is related to the CUDA module. mod:hip This is related to the HIP module. mod:reference This is related to the reference module. reg:testing This is related to testing. type:multigrid This is related to multigrid
Projects
None yet
Milestone
Ginkgo 1.10.0
Development

Successfully merging this pull request may close these issues.
4 participants
@MarcelKoch @yhmtsai @fritzgoebel @ginkgo-bot