Support for tag metadata in one-sided communication

[omor1]

After my meeting with @snirmarc earlier today, we came to the conclusion that it may be useful to add tags to the LCI one-sided communication calls. These are obviously not used for tag matching, but they are used to disambiguate between different puts or gets when using a handler or queue for completion.

This is helpful for implementing PaRSEC with LCI; their 'put' and 'get' calls use callbacks for completion on both the local and remote endpoints and need some mechanism to connect the callback to the particular communication that's being completed.

[omor1]

As a side note, what would be the difference in terms of functionality of having a handler function versus having a completion queue and calling some function for each request? Other than the handler is invoked from within LCI and the function would be called by the user.

[danghvu]

add tags to the LCI one-sided communication calls.

No problem, I already have this feature at some point.

As a side note, what would be the difference in terms of functionality of having a handler function versus having a completion queue and calling some function for each request? Other than the handler is invoked from within LCI and the function would be called by the user.

This is the only difference, and usually comes down to the drawback of active messages.

For example, since the handler is executed in the context of a LCI_progress invocation, the handler code cannot be so complex (possibly cannot call another LCI_progress, cannot be very long, cannot block, etc..).

A handler also means a function call, enqueue can be done in hardware, or a few instructions. Runtime with only AMs end up having people called functions just to enqueue data to the consumer tasks.

[omor1]

Runtime with only AMs end up having people called functions just to enqueue data to the consumer tasks.

That's effectively what's being done with the way they've set up the PaRSEC communication engine layer. We should talk to them to figure out if we can get around that.

[omor1]

I think we have this now?