Support Multiple Authorization Keys per Endpoint #9476
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
bot:aws:retest |
2 similar comments
|
bot:aws:retest |
|
bot:aws:retest |
|
@iziemba Please rebase and resolve the conflicts. |
iziemba
force-pushed
the
av_auth_key_support
branch
from
f0cf844 to
79954aa
Compare
j-xiong
reviewed
Oct 28, 2023
iziemba
commented
Oct 30, 2023
iziemba
force-pushed
the
av_auth_key_support
branch
from
79954aa to
c7c7b5f
Compare
|
Comments addressed. In addition, FI_AV_USER_ID with authorization key fi_addr_t support defined. |
|
@iziemba Please rebase to allow AWS CI to pass. |
j-xiong
approved these changes
Oct 30, 2023
fi_domain_attr::max_ep_auth_key is used to reported the number of
authorization keys supported by an endpoint. If this value is non-zero,
connectionless endpoints must implement FI_AV_AUTH_KEY.
FI_AV_AUTH_KEY is set by libfabric users via
fi_domain_attr:::auth_key_size to denoted if MR and EP authorization
keys from the AV instead of MR and EP attrs. When set, providers will
ignore fi_ep_attr::auth_key during endpoint enable. From MRs,
fi_mr_regattr() must be used with fi_mr_attr::auth_key pointing to
a struct fi_mr_auth_key and fi_mr_attr:auth_key_size equal to
sizeof(struct fi_mr_auth_key). fi_mr_auth_key::av should point to the AV
the MR authorization keys should come from. If the domain is configured
with FI_DIRECTED_RECV, fi_mr_auth_key::src_addr is used to restrict the
MR to a specific fi_addr_t including authorization key fi_addr_t's.
fi_av_insert_auth_key() output is an fi_addr_t handle specific to this
authorization key. All operations, including AV operations data transfer
operations, which may accept an auth_key fi_addr_t are required to pass
in the FI_AUTH_KEY flag. If the EP is configured with FI_DIRECTED_RECV,
this auth_key fi_addr_t can be used to match all EP addrs associated
with this authorization key. Calling fi_av_remove() with this fi_addr_t
will delete the authorization key. -FI_EBUSY will be returned from
fi_av_remove() should this key still be used by en EP. In other words,
all EPs using this authorization key need to be closed for
fi_av_remove() to succeed.
Once the AV is bound to an EP and the EP is successfully enabled, the
EP will be configured to support all auth keys in the AV at that point
in time.
Users must provide an authorization key fi_addr_t with
fi_av_insert_{addr, svc, sym}. This is done by using the fi_addr as
input and setting the FI_AUTH_KEY flag. For fi_av_insert_{addr, sym},
since fi_addr may be an array, authorization key fi_addr_t's need to be
specified for each index. The output of fi_av_insert_{addr, svc, sym} is
an fi_addr_t mapping to a specific <EP addr, auth_key> tuple.
For FI_EADDRNOTAVAIL CQ errors, fi_cq_err_entry::src_addr will return
the authorization key handle associated with the incoming data transfer.
This, combined with the existing behavior of fi_cq_err_entry::err_data
enables users to generate a fi_addr_t mapping to the specific
<EP addr, auth_key> tuple which triggered the FI_EADDRNOTAVAIL event.
Signed-off-by: Ian Ziemba <[email protected]>
FI_AV_AUTH_KEY is used to enable multiple auth keys per connectionless endpoint. Signed-off-by: Ian Ziemba <[email protected]>
fi_av_set_user_id() is used to set the user id when the AV is opened with FI_AV_USER_ID. Signed-off-by: Ian Ziemba <[email protected]>
Document FI_AV_USER_ID as a primary cap. In addition, define FI_AV_USER_ID as a new domain primary cap. This enables AVs to be opened with FI_AV_USER_ID. Define AV opened with FI_AV_USER_ID behavior. In addition, document how the existing FI_AV_USER_ID behavior can be used if FI_AV_USER_ID is not requested as a capability. Signed-off-by: Ian Ziemba <[email protected]>
Signed-off-by: Ian Ziemba <[email protected]>
When FI_AV_AUTH_KEY is used, domain_attr::auth_key and ep_attr::auth_key must be NULL. In addition, ep_attr::auth_key_size must be zero. Signed-off-by: Ian Ziemba <[email protected]>
iziemba
force-pushed
the
av_auth_key_support
branch
from
c7c7b5f to
202876b
Compare
|
Rebased on main. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Formal PR from #9319.
Use-case described in #9204.