Request: WebSocket send message in fragments (IDFGH-10198)

[projectgus]

(Hi David & colleagues, hope you're all doing well!)

Is your feature request related to a problem?

Currently sending messages via the esp_websocket_client API requires a caller to always provide a buffer holding the full message contents. Cases such as large messages, or when some piece of dynamic data is placed inside a header/footer such as wrapped in a JSON object, or when the transmitted data is generated on the fly from another format, then the caller has to allocate an additional larger buffer to hold all of the data.

Here's a contrived example:

void app_send(const void *buf1, size_t buf1_len, const void *buf2, size_t buf2_len, const void *buf3, size_t buf3_len)
{
    uint8_t *large_buf = malloc(buf1_len + buf2_len + buf3_len);
   memcpy(large_buf, buf1, buf1_len);
   memcpy(large_buf + buf1_len, buf2, buf2_len);
   memcpy(large_buf + buf1_len + buf2_len, buf3, buf3_len);

   esp_websocket_client_send_text(client, large_buf, buf1_len + buf2_len + buf3_len, portMAX_DELAY);
   free(large_buf);
}

(Not a real application ofc, the application that inspired me to make this request is serializing large blobs of JSON using jems and sending those to a WebSocket.)

Although the caller has to supply the entire message in one chunk, the transport implementation will still fragment any message buffer that's larger than its transmit buffer internally!

All of the front-end API send functions call through to esp_websocket_client_send_with_opcode which fragments the message but then sets the FIN bit on the last fragment, so there's no way to not set this.

Describe the solution you'd like.

One possible way to extend the current API could be:

esp_websocket_client_send_text_partial(client, buf1, buf1_len, portMAX_DELAY);
esp_websocket_client_send_text_partial(client, buf2, buf2_len, portMAX_DELAY);
esp_websocket_client_send_text(client, buf3, buf3_len, portMAX_DELAY);

(The result of this would be a single WebSocket message, the same as in the example using large_buf but without the additional memory allocation.)

Describe alternatives you've considered.

Other than allocating the large buffer, alternatives might be:

• Code against transport_ws.h layer directly (seems complex)
• Use a third party implementation like libwebsockets (which supports this, but requires the project to be converted to the libwebsockets build system.)

Additional context.

No response

[david-cermak]

Hi Angus!

Nice to hear from you 😄 ...and happy to hear that you're using Espressif's websocket client!

Thanks for requesting this useful feature, we will implement it; this looks like a simple change. (as you know we welcome contributions and PRs)
Your suggestion with esp_websocket_client_send_text_partial() looks good as an API extension (we might need to add `esp_websocket_client_send_bin_partial() in future).

Alternatively we can add esp_websocket_client_send_with_exact_opcode() which wouldn't add the FIN flag to the last fragment, so the example would go as follows:

esp_websocket_client_send_with_exact_opcode(client, WS_TRANSPORT_OPCODES_TEXT, buf1, buf1_len, portMAX_DELAY);
esp_websocket_client_send_with_exact_opcode(client, WS_TRANSPORT_OPCODES_TEXT, buf2, buf2_len, portMAX_DELAY);
...
esp_websocket_client_send_with_exact_opcode(client, WS_TRANSPORT_OPCODES_TEXT|WS_TRANSPORT_OPCODES_FIN, buf3, buf3_len, portMAX_DELAY);

(but the original proposal with esp_websocket_client_send_text_partial() is more straight-forward IMO)

[projectgus]

Thanks David! A set of new functions like you suggest, consisting of:

• esp_websocket_client_send_text_partial()
• esp_websocket_client_send_bin_partial()
• esp_websocket_client_send_with_exact_opcode

Would be very handy and solve this problem well! It seems like the first two functions could be be thin convenience wrappers around the third.

If I have time to work on a PR for this then I'll let you know, but I might not as we've got the "single large buffer" version working for our requirements at the moment.