Skip to content

Latest commit

 

History

History
1034 lines (768 loc) · 53.2 KB

Protocol.md

File metadata and controls

1034 lines (768 loc) · 53.2 KB

Status

This protocol is currently a draft for the final specifications. Current version of the protocol is 0.2 (Major Version: 0, Minor Version: 2).

Introduction

Specify an application protocol for Reactive Streams semantics across an asynchronous, binary boundary. For more information, please see Reactive Socket.

ReactiveSockets assumes an operating paradigm. These assumptions are:

  • one-to-one communication
  • non-proxied communication. Or if proxied, the ReactiveSocket semantics and assumptions are preserved across the proxy.
  • no state preserved across transport protocol sessions by the protocol

Key words used by this document conform to the meanings in RFC 2119.

Byte ordering is big endian for all fields.

Terminology

  • Frame: A single message containing a request, response, or protocol processing.
  • Fragment: A portion of an application message that has been partitioned for inclusion in a Frame. See Fragmentation and Reassembly.
  • Transport: Protocol used to carry ReactiveSockets protocol. One of WebSockets, TCP, or Aeron. The transport MUST provide capabilities mentioned in the transport protocol section.
  • Stream: Unit of operation (request/response, etc.). See Motivations.
  • Request: A stream request. May be one of four types. As well as request for more items or cancellation of previous request.
  • Response: A stream response. Contains data associated with previous request.
  • Client: The side initiating a connection.
  • Server: The side accepting connections from clients.
  • Connection: The instance of a transport session between client and server.
  • Requester: The side sending a request. A connection has at most 2 Requesters. One in each direction.
  • Responder: The side receiving a request. A connection has at most 2 Responders. One in each direction.

Versioning Scheme

ReactiveSocket follows a versioning scheme consisting of a numeric major version and a numeric minor version.

Cross version compatibility

ReactiveSocket assumes that all version changes (major and minor) are backward incompatible. A client can pass a version that it supports via the Setup Frame. It is up to a server to accept clients of lower versions than what it supports.

Data And Metadata

ReactiveSocket provides mechanisms for applications to distinguish payload into two types. Data and Metadata. The distinction between the types in an application is left to the application.

The following are features of Data and Metadata.

  • Metadata can be encoded differently than Data. Default metadata encoding is specified in this document
  • Metadata can be "attached" (i.e. correlated) with the following entities:
    • Connection via Metadata Push and Stream ID of 0
    • Individual Request or Response

Framing

Transport Protocol

The ReactiveSocket protocol uses a lower level transport protocol to carry ReactiveSocket frames. A transport protocol MUST provide the following:

  1. Unicast Reliable Delivery.
  2. Connection-Oriented and preservation of frame ordering. Frame A sent before Frame B must arrive in source order. i.e. if Frame A is sent by the same source as Frame B, then Frame A will always arrive before Frame B. No assumptions about ordering across sources is assumed.
  3. FCS is assumed to be in use either at the transport protocol or at each MAC layer hop. But no protection against malicious corruption is assumed.

An implementation MAY "close" a transport connection due to protocol processing. When this occurs, it is assumed that that connection will have no further frames sent and all frames will be ignored.

ReactiveSocket as specified here has been designed for and tested with TCP, WebSocket, and Aeron as transport protocols.

Framing Protocol Usage

Some of the supported transport protocols for ReactiveSocket may not support specific framing that preserves message boundaries. For these protocols, a framing protocol must be used with the ReactiveSocket frame that prepends the ReactiveSocket Frame Length.

The frame length field MUST be omitted if the transport protocol preserves message boundaries e.g. provides compatible framing. If, however, the transport protocol only provides a stream abstraction or can merge messages without preserving boundaries, or multiple transport protocols may be used, then the frame header MUST be used.

Transport Protocol Frame Length Field Required
TCP YES
WebSocket NO
Aeron NO
HTTP/2 YES

Framing Format

When using a transport protocol providing framing, the ReactiveSocket frame is simply encapsulated into the transport protocol messages directly.

    +-----------------------------------------------+
    |                ReactiveSocket Frame          ...
    |                                              
    +-----------------------------------------------+

When using a transport protocol that does not provide compatible framing, the Frame Length must be prepended to the ReactiveSocket Frame.

     0                   1                   2
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                    Frame Length               |
    +-----------------------------------------------+
    |                ReactiveSocket Frame          ...
    |                                              
    +-----------------------------------------------+
  • Frame Length: (24 = max 16,777,216 bytes) Length of Frame. Excluding Framing Length Field.

NOTE: Byte ordering is big endian.

Operation

Frame Header Format

ReactiveSocket frames begin with a ReactiveSocket Frame Header. The general layout is given below.

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-----------+-------------------------------+
    |   Frame Type  |I|M|   Flags   |     Depends on Frame Type    ...
    +-------------------------------+
  • Stream ID: (32) Positive signed integer representing the stream Identifier for this frame or 0 to indicate the entire connection.
    • Transport protocols that include demultiplexing, such as HTTP/2, MAY omit the Stream ID field if all parties agree. The means of negotiation and agreement is left to the transport protocol.
  • Frame Type: (8) Type of Frame.
  • Flags: Any Flag bit not specifically indicated in the frame type should be set to 0 when sent and not interpreted on reception. Flags generally depend on Frame Type, but all frame types must provide space for the following flags:
    • (I)gnore: Ignore frame if not understood
    • (M)etadata: Metadata present

NOTE: Byte ordering is big endian.

Handling Ignore Flag

The (I)gnore flag is used for extension of the protocol. A value of 0 in a frame for this flag indicates the protocol can't ignore this frame. An implementation MAY send an ERROR frame (with CONNECTION_ERROR error code) and close the underlying transport connection on reception of a frame that it does not understand with this bit not set.

Frame Validation

ReactiveSocket implementations may provide their own validation at the metadata level for specific frames. However, this is an application concern and not necessary for protocol processing.

Metadata Optional Header

Specific Frame Types MAY contain an optional metadata header that provides metadata about a frame. This metadata header is between the Frame Header and any payload.

Metadata Length MUST be less than or equal to the Frame Length minus the length of the Frame Header. If Metadata Length is greater than this value, the entire frame MUST be ignored.

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |              Metadata Length                  |
    +-+-------------------------------------------------------------+
    |                       Metadata Payload                       ...
    +---------------------------------------------------------------+
    |                       Payload of Frame                       ...
    +---------------------------------------------------------------+
  • Metadata Length: (24 = max 16,777,216 bytes) unsigned positive Integer representing the length of Metadata in bytes. Excluding Metadata Length field.

Stream Identifiers

Generation

Stream IDs are generated by a Requester. The lifetime of a Stream ID is determined by the request type and the semantics of the stream based on its type.

Stream ID value of 0 is reserved for any operation involving the connection.

A stream ID must be locally unique for a Requester in a connection.

Stream ID generation follows general guidelines for HTTP/2 with respect to odd/even values. In other words, a client MUST generate odd Stream IDs and a server MUST generate even Stream IDs.

Frame Types

Type Value Description
RESERVED 0x00 Reserved
SETUP 0x01 Setup: Sent by client to initiate protocol processing.
LEASE 0x02 Lease: Sent by Responder to grant the ability to send requests.
KEEPALIVE 0x03 Keepalive: Connection keepalive.
REQUEST_RESPONSE 0x04 Request Response: Request single response.
REQUEST_FNF 0x05 Fire And Forget: A single one-way message.
REQUEST_STREAM 0x06 Request Stream: Request a completable stream.
REQUEST_CHANNEL 0x07 Request Channel: Request a completable stream in both directions.
REQUEST_N 0x08 Request N: Request N more items with ReactiveStreams semantics.
CANCEL 0x09 Cancel Request: Cancel outstanding request.
RESPONSE 0x0A Response: Response to a request.
ERROR 0x0B Error: Error at connection or application level.
METADATA_PUSH 0x0C Metadata: Asynchronous Metadata frame
RESUME 0x0D Resume: Replaces SETUP for Resuming Operation (optional)
RESUME_OK 0x0E Resume OK : Sent in response to a RESUME if resuming operation possible (optional)
EXT 0xFF Extension Header: Used To Extend more frame types as well as extensions.

Setup Frame

Setup frames MUST always use Stream ID 0 as they pertain to the connection.

The SETUP frame is sent by the client to inform the server of the parameters under which it desires to operate. The usage and message sequence used is shown in Connection Establishment.

One of the important parameters for a connection is the format, layout, and any schema of the data and metadata for frames. This is, for lack of a better term, referred to here as "MIME Type". An implementation MAY use typical MIME type values or MAY decide to use specific non-MIME type values to indicate format, layout, and any schema for data and metadata. The protocol implementation MUST NOT interpret the MIME type itself. This is an application concern only.

The encoding format for Data and Metadata are included separately in the SETUP.

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-+-+-+-----+-------------------------------+
    |   Frame Type  |0|M|L|S|R|     |
    +---------------+-+-+-+-+-+-----+-------------------------------+
    |     Major Version             |         Minor Version         |
    +-------------------------------+-------------------------------+
    |                   Time Between KEEPALIVE Frames               |
    +---------------------------------------------------------------+
    |                         Max Lifetime                          |
    +---------------------------------------------------------------+
    |                    Resume Identification Token                |
    +                                                               +
    |                                                               |
    +                                                               +
    |                                                               |
    +                                                               +
    |                                                               |
    +---------------+-----------------------------------------------+
    |  MIME Length  |   Metadata Encoding MIME Type                ...
    +---------------+-----------------------------------------------+
    |  MIME Length  |     Data Encoding MIME Type                  ...
    +---------------+-----------------------------------------------+
                          Metadata & Setup Payload
  • Flags:
    • (M)etadata: Metadata present
    • (L)ease: Will honor LEASE (or not).
    • (S)trict: Adhere to strict interpretation of Data and Metadata.
    • (R)esume Enable: Client requests resume capability is possible (Optional).
  • Major Version: (16) Major version number of the protocol.
  • Minor Version: (16) Minor version number of the protocol.
  • Time Between KEEPALIVE Frames: Time (in milliseconds) between KEEPALIVE frames that the client will send.
  • Max Lifetime: Time (in milliseconds) that a client will allow a server to not respond to a KEEPALIVE before it is assumed to be dead.
  • Resume Identification Token: (128) Token used for client resume identification. (Optional - set to all 0s if not supported)
  • MIME Length: Encoding MIME Type Length in bytes.
  • Encoding MIME Type: MIME Type for encoding of Data and Metadata. This SHOULD be a US-ASCII string that includes the Internet media type specified in RFC 2045. Many are registered with IANA such as CBOR. Suffix rules MAY be used for handling layout. For example, application/x.netflix+cbor or application/x.reactivesocket+cbor or application/x.netflix+json. The string MUST NOT be null terminated.
  • Setup Data: includes payload describing connection capabilities of the endpoint sending the Setup header.

NOTE: A server that receives a SETUP frame that has (R)esume Enabled set, but does not support resuming operation, must reject the SETUP with an ERROR.

Error Frame

Error frames are used for errors on individual requests/streams as well as connection errors and in response to SETUP frames. The latter is referred to as a SETUP_ERROR.

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-----------+-------------------------------+
    |   Frame Type  |0|M|    Flags  |
    +---------------+-+-+-----------+-------------------------------+
    |                          Error Code                           |
    +---------------------------------------------------------------+
                        Metadata & Error Data
  • Flags:
    • (M)etadata: Metadata present
  • Error Code: Type of Error.
  • Error Data: includes payload describing error information. Error Data SHOULD be a UTF-8 encoded string. The string MUST NOT be null terminated.

A Stream ID of 0 means the error pertains to the connection. Including connection establishment. A positive non-0 Stream ID means the error pertains to a given stream.

The Error Data is typically an Exception message, but could include stringified stacktrace information if appropriate.

Error Codes

Type Value Description
RESERVED 0x00000000 Reserved
INVALID_SETUP 0x00000001 The Setup frame is invalid for the server (it could be that the client is too recent for the old server). Stream ID MUST be 0.
UNSUPPORTED_SETUP 0x00000002 Some (or all) of the parameters specified by the client are unsupported by the server. Stream ID MUST be 0.
REJECTED_SETUP 0x00000003 The server rejected the setup, it can specify the reason in the payload. Stream ID MUST be 0.
CONNECTION_ERROR 0x00000101 The connection is being terminated. Stream ID MUST be 0.
CONNECTION_ERROR_NO_RETRY 0x00000102 The connection is being terminated. May NOT be resumed. Stream ID MUST be 0.
APPLICATION_ERROR 0x00000201 Application layer logic generating a Reactive Streams onError event. Stream ID MUST be non-0.
REJECTED 0x00000202 Despite being a valid request, the Responder decided to reject it. The Responder guarantees that it didn't process the request. The reason for the rejection is explained in the metadata section. Stream ID MUST be non-0.
CANCELED 0x00000203 The responder canceled the request but potentially have started processing it (almost identical to REJECTED but doesn't garantee that no side-effect have been started). Stream ID MUST be non-0.
INVALID 0x00000204 The request is invalid. Stream ID MUST be non-0.
RESERVED 0xFFFFFFFF Reserved for Extension Use

NOTE: Values in the range of 0x0001 to 0x00FF are reserved for use as SETUP error codes. Values in the range of 0x00101 to 0x001FF are reserved for connection error codes. Values in the range of 0x00201 to 0xFFFFFFFE are reserved for application layer errors.

Lease Frame

Lease frames MUST always use Stream ID 0 as they pertain to the Connection.

Lease frames MAY be sent by the client-side or server-side Responders and inform the Requester that it may send Requests for a period of time and how many it may send during that duration. See Lease Semantics for more information.

The last received LEASE frame overrides all previous LEASE frame values.

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-----------+-------------------------------+
    |   Frame Type  |0|M|    Flags  |
    +---------------+-+-+-----------+-------------------------------+
    |                         Time-To-Live                          |
    +---------------------------------------------------------------+
    |                       Number of Requests                      |
    +---------------------------------------------------------------+
                                Metadata
  • Flags:
    • (M)etadata: Metadata present
  • Time-To-Live (TTL): Time (in milliseconds) for validity of LEASE from time of reception
  • Number of Requests: Number of Requests that may be sent until next LEASE

A Responder implementation MAY stop all further requests by sending a LEASE with a value of 0 for Number of Requests or Time-To-Live.

When a LEASE expires due to time, the value of the Number of Requests that a Requester may make is implicitly 0.

Keepalive Frame

KEEPALIVE frames MUST always use Stream ID 0 as they pertain to the Connection.

KEEPALIVE frames MUST be initiated by the client and sent periodically with the (R)espond flag set. A reasonable time interval between client KEEPALIVE frames SHOULD be 500ms.

KEEPALIVE frames MAY be initiated by the server and sent upon application request with the (R)espond flag set.

Reception of a KEEPALIVE frame with the (R)espond flag set MUST cause a client or server to send back a KEEPALIVE with the (R)espond flag NOT set. The data in the received KEEPALIVE MUST be echoed back in the generated KEEPALIVE.

Reception of a KEEPALIVE by a server indicates to the server that the client is alive.

Reception of a KEEPALIVE by a client indicates to the client that the server is alive.

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-+---------+-------------------------------+
    |   Frame Type  |0|0|R|  Flags  |
    +---------------+-+-+-+---------+-------------------------------+
    |                    Last Received Position                     |
    +                                                               +
    |                                                               |
    +---------------------------------------------------------------+
                                  Data
  • Flags:
    • (M)etadata: Metadata never present
    • (R)espond with KEEPALIVE or not
  • Last Received Position: (64) Resume Last Received Position (optional. Set to all 0s when not supported.)
  • Data: Data attached to a KEEPALIVE.

Request Response Frame

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-+---------+-------------------------------+
    |   Frame Type  |0|M|F|  Flags  |
    +-------------------------------+
                         Metadata & Request Data
  • Flags:
    • (M)etadata: Metadata present
    • (F)ollows: More Fragments Follow This Fragment.
  • Request Data: identification of the service being requested along with parameters for the request.

Request Fire-n-Forget Frame

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-+---------+-------------------------------+
    |   Frame Type  |0|M|F|  Flags  |
    +-------------------------------+
                          Metadata & Request Data
  • Flags:
    • (M)etadata: Metadata present
    • (F)ollows: More Fragments Follow This Fragment.
  • Request Data: identification of the service being requested along with parameters for the request.

Request Stream Frame

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-+---------+-------------------------------+
    |   Frame Type  |0|M|F|  Flags  |
    +-------------------------------+-------------------------------+
    |                      Initial Request N                        |
    +---------------------------------------------------------------+
                          Metadata & Request Data
  • Flags:
    • (M)etadata: Metadata present
    • (F)ollows: More Fragments Follow This Fragment.
  • Initial Request N: 32-bit signed integer representing the initial request N value for the stream. Only positive values are allowed.
  • Request Data: identification of the service being requested along with parameters for the request.

Please note that this explicitly does NOT follow rule number 17 in https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.0/README.md#3-subscription-code

While ReactiveStreams supports a demand of up to 2^63-1, and treats 2^63-1 as a magic number signaling to not track demand, this is not the case for ReactiveSocket. ReactiveSocket prioritizes byte size and only uses 4 bytes instead of 8 so the magic number is unavailable.

Request Channel Frame

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-+-+-+-----+-------------------------------+
    |   Frame Type  |0|M|F|C|N|     |
    +-------------------------------+-------------------------------+
    |              Initial Request N (only if N bit set)            |
    +---------------------------------------------------------------+
                           Metadata & Request Data
  • Flags:
    • (M)etadata: Metadata present
    • (F)ollows: More Fragments Follow This Fragment.
    • (C)omplete: bit to indicate COMPLETE.
    • (N): Is Initial Request N present or not
  • Initial Request N: 32-bit signed integer representing the initial request N value for channel. Only positive values are allowed.
  • Request Data: identification of the service being requested along with parameters for the request.

See Request Stream Frame for additional information.

Request N Frame

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-----------+-------------------------------+
    |   Frame Type  |0|0|   Flags   |
    +-------------------------------+-------------------------------+
    |                           Request N                           |
    +---------------------------------------------------------------+
  • Flags:
    • (M)etadata: Metadata NOT present
  • Request N: 32-bit signed integer value of items to request. Only positive values are allowed.

See Request Stream Frame for additional information.

Cancel Frame

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-----------+-------------------------------+
    |   Frame Type  |0|M|  Flags    |
    +-------------------------------+-------------------------------+
                                Metadata
  • Flags:
    • (M)etadata: Metadata present

Response Frame

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-+-+-------+-------------------------------+
    |   Frame Type  |0|M|F|N|C| ... |
    +-------------------------------+-------------------------------+
                         Metadata & Response Data
  • Flags:
    • (M)etadata: Metadata Present.
    • (F)ollows: More fragments follow this fragment.
    • (N)ext: bit to indicate Next (Response Data and/or Metadata present).
      • If set, onNext(Payload) or equivalent will be invoked on Subscriber/Observer.
    • (C)omplete: bit to indicate COMPLETE.
      • If set, onComplete() or equivalent will be invoked on Subscriber/Observer.
  • Response Data: payload for Reactive Streams onNext.

A Response is generally referred to as a NEXT.

A Response with the Complete Bit set is referred to as a COMPLETE.

Metadata Push Frame

A Metadata Push frame can be used to send asynchronous metadata notifications from a Requester or Responder to its peer. Metadata MUST be scoped to the connection by setting Stream ID to 0.

Metadata tied to a particular Request, Response, etc. uses the individual frames Metadata flag.

Frame Contents

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-----------+-------------------------------+
    |   Frame Type  |0|1|   Flags   |
    +-------------------------------+-------------------------------+
                                Metadata
  • Flags:
    • (M)etadata: Metadata always present
  • Stream ID: Must be 0 to pertain to the entire connection.

Extension Frame

The general format for an extension frame is given below.

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-----------+-------------------------------+
    |   Frame Type  |I|M|  Flags    |
    +-------------------------------+-------------------------------+
    |                        Extended Type                          |
    +---------------------------------------------------------------+
                          Depends on Extended Type...
  • Frame Type: (16) 0xFFFF for Extension Header.
  • Flags:
    • (I)gnore: Can be frame be ignored if not understood?
    • (M)etadata: Metadata Present.
  • Extended Type: Extended type information

Connection Establishment

NOTE: The semantics are similar to TLS False Start.

The term SETUP_ERROR is used below to indicate an ERROR frame that has a Stream ID of 0 and an Error Code that indicates a SETUP error.

Immediately upon successful connection, the client MUST send a SETUP frame with Stream ID of 0. Any other frame received that is NOT a SETUP frame or a SETUP frame with a non-0 Stream ID, MUST cause the server to send a SETUP_ERROR (with INVALID_SETUP) and close the connection.

The client-side Requester can inform the server-side Responder as to whether it will honor LEASEs or not based on the presence of the L flag in the SETUP frame.

The client-side Requester that has NOT set the L flag in the SETUP frame may send requests immediately if it so desires without waiting for a LEASE from the server.

The client-side Requester that has set the L flag in the SETUP frame MUST wait for the server-side Responder to send a LEASE frame before it can send Requests.

If the server accepts the contents of the SETUP frame, it MUST send a LEASE frame if the SETUP frame set the L flag. The server-side Requester may send requests immediately upon receiving a SETUP frame that it accepts if the L flag is not set in the SETUP frame.

If the server does NOT accept the contents of the SETUP frame, the server MUST send back a SETUP_ERROR and then close the connection.

The S flag of the SETUP indicates the client requires the server to adhere to strict interpretation of the Data and Metadata of the SETUP. Anything in the Data and/or Metadata that is not understood or can be provided by the server should require the SETUP to be rejected.

The server-side Requester mirrors the LEASE requests of the client-side Requester. If a client-side Requester sets the L flag in the SETUP frame, the server-side Requester MUST wait for a LEASE frame from the client-side Responder before it can send a request. The client-side Responder MUST send a LEASE frame after a SETUP frame with the L flag set.

A client assumes a SETUP is accepted if it receives a response to a request, a LEASE frame, or if it sees a REQUEST type.

A client assumes a SETUP is rejected if it receives a SETUP_ERROR.

Until connection establishment is complete, a Requester MUST NOT send any Request frames.

Until connection establishment is complete, a Responder MUST NOT emit any RESPONSE frames.

Negotiation

The assumption is that the client will be dictating to the server what it desires to do. The server will decide to support that SETUP (accept it) or not (reject it). The SETUP_ERROR error code indicates the reason for the rejection.

Sequences without LEASE

The possible sequences without LEASE are below.

  1. Client-side Request, Server-side accepts SETUP
    • Client connects & sends SETUP & sends REQUEST
    • Server accepts SETUP, handles REQUEST, sends back normal sequence based on REQUEST type
  2. Client-side Request, Server-side rejects SETUP
    • Client connects & sends SETUP & sends REQUEST
    • Server rejects SETUP, sends back SETUP_ERROR, closes connection
  3. Server-side Request, Server-side accepts SETUP
    • Client connects & sends SETUP
    • Server accepts SETUP, sends back REQUEST type
  4. Server-side Request, Server-side rejects SETUP
    • Client connects & sends SETUP
    • Server rejects SETUP, sends back SETUP_ERROR, closes connection

Sequences with LEASE

The possible sequences with LEASE are below.

  1. Client-side Request, Server-side accepts SETUP
    • Client connects & sends SETUP with L flag
    • Server accepts SETUP, sends back LEASE frame
    • Client-side sends REQUEST
  2. Client-side Request, Server-side rejects SETUP
    • Client connects & sends SETUP with L flag
    • Server rejects SETUP, sends back SETUP_ERROR, closes connection
  3. Server-side Request, Server-side accepts SETUP
    • Client connects & sends SETUP with L flag
    • Server accepts SETUP, sends back LEASE frame
    • Client sends LEASE frame
    • Server sends REQUEST
  4. Server-side Request, Server-side rejects SETUP
    • Client connects & sends SETUP with L flag
    • Server rejects SETUP, sends back SETUP_ERROR, closes connection

Fragmentation And Reassembly

RESPONSE frames and all REQUEST frames may represent a large object and MAY need to be fragmented to fit within the Frame Data size. When this occurs, the F flag indicates if more fragments follow the current frame (or not).

Stream Sequences and Lifetimes

Streams exists for a specific period of time. So an implementation may assume that Stream IDs are valid for a finite period of time. This period of time is bound by, at most, the lifetime of the underlying transport protocol connection lifetime. Beyond that, each interaction pattern imposes lifetime based on a sequence of interactions between Requester and Responder.

In the section below, "RQ -> RS" refers to Requester sending a frame to a Responder. And "RS -> RQ" refers to Responder sending a frame to a Requester.

In the section below, "*" refers to 0 or more and "+" refers to 1 or more.

Once a stream has "terminated", the Stream ID can be "forgotten" by the Requester and Responder. An implementation MAY re-use an ID at this time, but it is recommended that an implementation not aggressively re-use IDs.

Request Response

  1. RQ -> RS: REQUEST_RESPONSE
  2. RS -> RQ: RESPONSE with COMPLETE

or

  1. RQ -> RS: REQUEST_RESPONSE
  2. RS -> RQ: ERROR

or

  1. RQ -> RS: REQUEST_RESPONSE
  2. RQ -> RS: CANCEL

Upon sending a response, the stream is terminated on the Responder.

Upon receiving a CANCEL, the stream is terminated on the Responder and the response SHOULD not be sent.

Upon sending a CANCEL, the stream is terminated on the Requester.

Upon receiving a COMPLETE or ERROR, the stream is terminated on the Requester.

Request Fire-n-Forget

  1. RQ -> RS: REQUEST_FNF

Upon reception, the stream is terminated by the Responder.

Upon being sent, the stream is terminated by the Requester.

REQUEST_FNF are assumed to be best effort and MAY not be processed due to: (1) SETUP rejection, (2) mis-formatting, (3) etc.

Request Stream

  1. RQ -> RS: REQUEST_STREAM
  2. RS -> RQ: RESPONSE*
  3. RS -> RQ: ERROR

or

  1. RQ -> RS: REQUEST_STREAM
  2. RS -> RQ: RESPONSE*
  3. RS -> RQ: RESPONSE with COMPLETE

or

  1. RQ -> RS: REQUEST_STREAM
  2. RS -> RQ: RESPONSE*
  3. RQ -> RS: CANCEL

At any time, a client may send REQUEST_N frames.

Upon receiving a CANCEL, the stream is terminated on the Responder.

Upon sending a CANCEL, the stream is terminated on the Requester.

Upon receiving a COMPLETE or ERROR, the stream is terminated on the Requester.

Upon sending a COMPLETE or ERROR, the stream is terminated on the Responder.

Request Channel

  1. RQ -> RS: REQUEST_CHANNEL* intermixed with
  2. RS -> RQ: RESPONSE*
  3. RS -> RQ: COMPLETE | ERROR

or

  1. RQ -> RS: REQUEST_CHANNEL* intermixed with
  2. RS -> RQ: RESPONSE*
  3. RQ -> RS: CANCEL

At any time, a Requester may send REQUEST_CHANNEL frames with F bit set to indicate fragmentation.

At any time, a Requester, as well as a Responder, may send REQUEST_N frames.

An implementation MUST only send a single initial REQUEST_CHANNEL frame from the Requester to the Responder. And a Responder MUST respond to an initial REQUEST_CHANNEL frame with a REQUEST_N frame.

A Requester may indicate end of REQUEST_CHANNEL frames by setting the C bit. A Requester MUST NOT send any additional REQUEST_CHANNEL frames after sending a frame with the C bit set.

Upon receiving a CANCEL, the stream is terminated on the Responder.

Upon sending a CANCEL, the stream is terminated on the Requester.

Upon receiving a COMPLETE or ERROR, the stream is terminated on the Requester.

Upon sending a COMPLETE or ERROR, the stream is terminated on the Responder.

Per Stream State

Requester

  1. CLOSED: implicit starting/ending state of all stream IDs
  2. Requested (sent REQUEST_*)
  3. CLOSED (received COMPLETE or sent REQUEST_FNF)
  4. CLOSED (received ERROR)

Responder

  1. CLOSED: implicit starting/ending state of all stream IDs
  2. Responding: sending RESPONSEs and processing REQUEST_N
  3. CLOSED (received CANCEL)
  4. CLOSED (sent COMPLETE or received REQUEST_FNF)
  5. CLOSED (sent ERROR)

Flow Control

There are multiple flow control mechanics provided by the protocol.

Reactive Stream Semantics

Reactive Stream semantics for flow control of Streams, Subscriptions, and Channels.

The Requester and the Responder MUST respect the reactive-streams semantics.

e.g. here's an example of a successful stream call with flow-control.

  1. RQ -> RS: REQUEST_STREAM (REQUEST_N=3)
  2. RS -> RQ: RESPONSE
  3. RS -> RQ: RESPONSE
  4. RS -> RQ: RESPONSE
  5. RS needs to wait for a new REQUEST_N at that point
  6. RQ -> RS: REQUEST_N (n=3)
  7. RS -> RQ: RESPONSE
  8. RS -> RQ: RESPONSE with COMPLETE

Lease Semantics

The LEASE semantics are to control the number of indivdiual requests (all types) that a Requester may send in a given period. The only responsibility the protocol implementation has for the LEASE is to honor it on the Requester side. The Responder application is responsible for the logic of generation and informing the Responder it should send a LEASE to the peer Requester.

Requester MUST respect the LEASE contract. The Requester MUST NOT send more than Number of Requests specified in the LEASE frame within the Time-To-Live value in the LEASE.

A Responder that receives a REQUEST that it can not honor due to LEASE restrictions MUST respond with an ERROR frame with error code of LEASE_ERROR. This includes an initial LEASE sent as part of Connection Establishment.

QoS and Prioritization

Quality of Service and Prioritization of streams are considered application or network layer concerns and are better dealt with at those layers. The metadata capabilities, including METADATA_PUSH, are tools that applications can use for effective prioritization. DiffServ via IP QoS are best handled by the underlying network layer protocols.

Handling the Unexpected

This protocol attempts to be very lenient in processing of received frames and SHOULD ignore conditions that do not make sense given the current context. Clarifications are given below:

  1. TCP half-open connections (and WebSockets) or other dead transports are detectable by lack of KEEPALIVE frames as specified under Keepalive Frame. The decision to close a connection due to inactivity is the applications choice.
  2. Request keepalive and timeout semantics are the responsibility of the application.
  3. Lack of REQUEST_N frames that stops a stream is an application concern and SHALL NOT be handled by the protocol.
  4. Lack of LEASE frames that stops new Requests is an application concern and SHALL NOT be handled by the protocol.
  5. If a RESPONSE for a REQUEST_RESPONSE is received that does not have a COMPLETE flag set, the implementation MUST assume it is set and act accordingly.
  6. Reassembly of RESPONSEs and REQUEST_CHANNELs MUST assume the possibility of an infinite stream.
  7. Stream ID values MAY be re-used after completion or error of a stream.
  8. A RESPONSE with both F and C flags set, implicitly ignores the F flag.
  9. All other received frames that are not accounted for in previous sections MUST be ignored. Thus, for example:
    1. Receiving a Request frame on a Stream ID that is already in use MUST be ignored.
    2. Receiving a CANCEL on an unknown Stream ID (including 0) MUST be ignored.
    3. Receiving an ERROR on an unknown Stream ID MUST be ignored.
    4. Receiving a RESPONSE on an unknown Stream ID (including 0) MUST be ignored.
    5. Receiving a METADATA_PUSH with a non-0 Stream ID MUST be ignored.
    6. A server MUST ignore a SETUP frame after it has accepted a previous SETUP.
    7. A server MUST ignore a SETUP_ERROR frame.
    8. A client MUST ignore a SETUP_ERROR after it has completed connection establishment.
    9. A client MUST ignore a SETUP frame.

Resuming Operation

Due to the large number of active requests for ReactiveSocket, it is often necessary to provide the ability for resuming operation on transport failure. This behavior is totally optional for operation and may be supported or not based on an implementation choice.

Assumptions

ReactiveSocket resumption exists only for specific cases. It is not intended to be an “always works” solution. If resuming operation is not possible, the connection should be terminated with an ERROR as specified by the protocol definition.

  1. Resumption is optional behavior for implementations. But highly suggested. Clients and Servers should assume NO resumption capability by default.
  2. Resumption is an optimistic operation. It may not always succeed.
  3. Resumption is desired to be fast and require a minimum of state to be exchanged.
  4. Resumption is designed for loss of connectivity and assumes client and server state is maintained across connectivity loss. I.e. there is no assumption of loss of state by either end. This is very important as without it, all the requirements of "guaranteed messaging" come into play.
  5. Resumption assumes no changes to Lease, Data format (encoding), etc. for resuming operation. i.e. A client is not allowed to change the metadata MIME type or the data MIME type or version, etc. when resuming operation.
  6. Resumption is always initiated by the client and either allowed or denied by the server.
  7. Resumption makes no assumptions of application state for delivered frames with respect to atomicity, transactionality, etc. See above.

Implied Position

Resuming operation requires knowing the position of data reception of the previous connection. For this to be simplified, the underlying transport is assumed to support contiguous delivery of data on a per frame basis. In other words, partial frames are not delivered for processing nor are gaps allowed in the stream of frames sent by either the client or server. The current list of supported transports (TCP, WebSocket, and Aeron) all satisfy this requirement or can be made to do so in the case of TCP.

As a Requester or Responder sends REQUEST, CANCEL, or RESPONSE frames, it maintains a position of that frame within the connection in that direction. This is a 64-bit value that starts at 0. As a Requester or Responder receives REQUEST, CANCEL, or RESPONSE frames, it maintains an implied position of that frame within the connection in that direction. This is also a 64-bit value that starts at 0.

The reason this is “implied” is that the position is not included in each frame and is inferred simply by the message being sent/received on the connection in relation to previous frames.

This position will be used to identify the location for resuming operation to begin.

Frame types outside REQUEST, CANCEL, ERROR, and RESPONSE do not have assigned (nor implied) positions.

Client Lifetime Management

Client lifetime management for servers must be extended to incorporate the length of time a client may successfully attempt resumption passed a transport disconnect. The means of client lifetime management are totally up to the implementation.

Resume Operation

All ERROR frames sent MUST be CONNECTION_ERROR or CONNECTION_ERROR_NO_RETRY error code.

Client side resumption operation starts when the client desires to try to resume and starts a new transport connection. The operation then proceeds as the following:

  • Client sends RESUME frame. The client must NOT send any other frame types until resumption succeeds. The RESUME Identification Token MUST be the token used in the original SETUP frame. The RESUME Last Received Position field MUST be the last successfully received implied position from the server.
  • Client waits for either a RESUME_OK or ERROR frame from the server.
  • On receiving an ERROR frame, the client MUST NOT attempt resumption again if the error code was CONNECTION_ERROR_NO_RETRY.
  • On receiving a RESUME_OK, the client:
    • MUST assume that the next REQUEST, CANCEL, ERROR, and RESPONSE frames have an implied position commencing from the last implied positions
    • MAY retransmit all REQUEST, CANCEL, ERROR, and RESPONSE frames starting at the RESUME_OK Last Received Position field value from the server.
    • MAY send an ERROR frame indicating the end of the connection and MUST NOT attempt resumption again

Server side resumption operation starts when the client sends a RESUME frame. The operation then proceeds as the following:

  • On receiving a RESUME frame, the server:
    • MUST send an ERROR frame if the server does not support resuming operation. This is accomplished by handling the Ignore bit in the RESUME frame.
    • use the RESUME Identification Token field to determine which client the resume pertains to. If the client is identified successfully, resumption MAY be continued. If not identified, then the server MUST send an ERROR frame.
    • if successfully identified, then the server MAY send a RESUME_OK and then:
      • MUST assume that the next REQUEST, CANCEL, ERROR, and RESPONSE frames have an implied position commencing from the last implied positions
      • MAY retransmit all REQUEST, CANCEL, ERROR, and RESPONSE frames starting at the RESUME Last Received Position field value from the client.
    • if successfully identified, then the server MAY send an ERROR frame if the server can not resume operation given the value of RESUME Last Received Position if the position is not one it deems valid to resume operation from or other extenuating circumstances.

A Server that receives a RESUME frame after a SETUP frame, SHOULD send an ERROR.

A Server that receives a RESUME frame after a previous RESUME frame, SHOULD send an ERROR.

A Server implementation MAY use CONNECTION_ERROR or CONNECTION_ERROR_NO_RETRY as it sees fit for each error condition.

Leasing semantics are NOT assumed to carry over from previous connections when resuming. LEASE semantics MUST be restarted upon a new connection by sending a LEASE frame from the server.

Resume Frame

The general format for a Resume frame is given below.

RESUME frames MUST always use Stream ID 0 as they pertain to the connection.

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-----------+-------------------------------+
    |   Frame Type  |0|0|  Flags    |
    +-------------------------------+-------------------------------+
    |                    Resume Identification Token                |
    +                                                               +
    |                                                               |
    +                                                               +
    |                                                               |
    +                                                               +
    |                                                               |
    +---------------------------------------------------------------+
    |                  Last Received Position (Client)              |
    +                                                               +
    |                                                               |
    +---------------------------------------------------------------+
  • Frame Type: (16) 0x0D for Resume Header.
  • Flags:
    • (I)gnore: Frame can NOT be ignored if not understood.
    • (M)etadata: Metadata never Present.
  • Resume Identification Token: (128) Token used for client resume identification. Same Resume Identification used in the initial SETUP by the client.
  • Last Received Position: (64) The last implied position the client received from the server

Resume OK Frame

The general format for a Resume OK frame is given below.

RESUME OK frames MUST always use Stream ID 0 as they pertain to the connection.

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Stream ID                           |
    +---------------+-+-+-----------+-------------------------------+
    |   Frame Type  |0|0|  Flags    |
    +-------------------------------+-------------------------------+
    |                 Last Received Position (Server)               |
    +                                                               +
    |                                                               |
    +---------------------------------------------------------------+
  • Frame Type: (16) 0x0E for Resume OK Header.
  • Flags:
    • (I)gnore: Frame can NOT be ignored if not understood.
    • (M)etadata: Metadata never Present.
  • Last Received Position: (64) The last implied position the server received from the client

Keepalive Position Field

Keepalive frames include the implied position of the client (or server). When sent, they act as a means for the other end of the connection to know the position of the other side.

This information MAY be used to update state for possible retransmission, such as trimming a retransmit buffer, or possible associations with individual stream status.

Identification Token Handling

The requirements for the Resume Identification Token are implementation dependent. However, some guidelines and considerations are:

  • Tokens may be generated by the client.
  • Tokens may be generated outside the client and the server and managed externally to the protocol.
  • Tokens should uniquely identify a connection on the server. The server should not assume a generation method of the token and should consider the token opaque. This allows a client to be compatible with any ReactiveSocket implementation that supports resuming operation and allows the client full control of Identification Token generation.
  • Tokens must be valid for the lifetime of an individual ReactiveSocket including possible resumption.
  • A server should not accept a SETUP with a Token that is currently already being used
  • Tokens should be resilient to replay attacks and thus should only be valid for the lifetime of an individual connection
  • Tokens should not be predictable by an attacking 3rd party