Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prov/shm: Re-enable ze-ipc with new pidfd method #10256

Merged
merged 2 commits into from
Aug 6, 2024

Conversation

zachdworkin
Copy link
Contributor

Refactor close_handle to add new field for ze pidfd method
Implement ze pidfd method for ze-ipc copies
Remove old method's (socket by path) code

@zachdworkin zachdworkin force-pushed the ze-shm branch 3 times, most recently from faa05be to b8c58ca Compare August 1, 2024 16:30
prov/shm/src/smr_ep.c Outdated Show resolved Hide resolved
src/hmem_ze.c Outdated Show resolved Hide resolved
@zachdworkin zachdworkin force-pushed the ze-shm branch 2 times, most recently from cfcee28 to bfbc5a7 Compare August 2, 2024 16:11
prov/shm/src/smr_ep.c Outdated Show resolved Hide resolved
Copy link
Contributor

@j-xiong j-xiong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one more comment.

src/hmem_ze.c Show resolved Hide resolved
@j-xiong j-xiong requested a review from aingerson August 2, 2024 16:29
@zachdworkin
Copy link
Contributor Author

I will re-run ci after updates from #10272 are done.

prov/shm/src/smr_ep.c Show resolved Hide resolved
include/ofi_hmem.h Show resolved Hide resolved
prov/shm/src/smr_progress.c Outdated Show resolved Hide resolved
prov/shm/src/smr.h Outdated Show resolved Hide resolved
prov/shm/src/smr_util.h Outdated Show resolved Hide resolved
include/ofi_hmem.h Outdated Show resolved Hide resolved
@zachdworkin zachdworkin force-pushed the ze-shm branch 5 times, most recently from ceb170c to e4b8709 Compare August 2, 2024 19:08
prov/shm/src/smr.h Outdated Show resolved Hide resolved
Add a void **handle to close_handle for level-zero to use
in its new ipc protocol

Signed-off-by: Zach Dworkin <zachary.dworkin@intel.com>
Move away from using a socket to initialize level-zero ipc by path
and start using pidfd instead.

This new protocol will open the pidfd
(process id's fd containing all open fd's for that process) for a
peer and save it in their region. This is done to avoid multiple
open/close calls on the same fd. We then use this pidfd to lookup
each open fd from level-zero's open_handle and cache them for later
re-use. These are stored as a new data type "ze_pid_handle" which
contains the fd from handle_get, the fd from handle_open, and
the pid_fd. When a cache entry is evicted then the handle's open fds
will be closed.

These changes re-enable the level-zero ipc protocol.

Signed-off-by: Zach Dworkin <zachary.dworkin@intel.com>
@j-xiong j-xiong merged commit ff881d6 into ofiwg:main Aug 6, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants