permessage-deflate issues spotted when talking to web browser

[jsdw]

When the Deflate extention is added to a Soketto server, it seems as if there is a problem decoding messages.

To reproduce run this in one terminal:

# The example here has logging and a browser can connect, so easy place to test:
git checkout jsdw-hyper-soketto
RUST_LOG=trace cargo run --example hyper_server --features 'http deflate'

And then in a web browser (I used firefox), connect to localhost:3000 and run this in the console:

var socket = new WebSocket("ws://localhost:3000");
socket.onmessage = function(msg) { console.log(msg) };
socket.send("aaa");
socket.send("aaa");

After running the second socket.send, we don't get an echo response, and the logs indicate that the server failed and the loop ended:

[2021-09-17T15:52:07Z DEBUG soketto::connection] deda8acc: using extension: permessage-deflate
[2021-09-17T15:52:09Z TRACE soketto] read 2 bytes
[2021-09-17T15:52:09Z TRACE soketto] read 4 bytes
[2021-09-17T15:52:09Z TRACE soketto::connection] deda8acc: recv: (Text (fin 1) (rsv 100) (mask (1 e3ac149b)) (len 5))
[2021-09-17T15:52:09Z TRACE soketto::connection] deda8acc: decoding with extension: permessage-deflate
[2021-09-17T15:52:09Z TRACE soketto::extension::deflate] deflate: decoding (Text (fin 1) (rsv 100) (mask (1 e3ac149b)) (len 5))
[2021-09-17T15:52:09Z TRACE soketto::connection] deda8acc: encoding with extension: permessage-deflate
[2021-09-17T15:52:09Z TRACE soketto::extension::deflate] deflate: encoding (Text (fin 1) (rsv 000) (mask (0 0)) (len 0))
[2021-09-17T15:52:09Z TRACE soketto::connection] deda8acc: send: (Text (fin 1) (rsv 100) (mask (0 0)) (len 5))
[2021-09-17T15:52:09Z TRACE soketto::connection] deda8acc: Sender flushing connection
[2021-09-17T15:52:10Z TRACE soketto] read 2 bytes
[2021-09-17T15:52:10Z TRACE soketto] read 4 bytes
[2021-09-17T15:52:10Z TRACE soketto::connection] deda8acc: recv: (Text (fin 1) (rsv 100) (mask (1 c666e6b5)) (len 4))
[2021-09-17T15:52:10Z TRACE soketto::connection] deda8acc: decoding with extension: permessage-deflate
[2021-09-17T15:52:10Z TRACE soketto::extension::deflate] deflate: decoding (Text (fin 1) (rsv 100) (mask (1 c666e6b5)) (len 4))
Websocket connection error: extension error: corrupt deflate stream
[2021-09-17T15:52:10Z TRACE mio::poll] deregistering event source from poller

Interestingly, if you send a message like "a" or "aa" instead of "aaa" it seems OK.

My guess is that the way we decode messages should be stateful, but isn't (we create a decoder each time a message comes in, attempt to decode it, and then throw the decoder away). I get the impression (see links below) that the default decoding strategy may use a sliding window approach which cares about past state to help decode future state. We can see in the log messages that the length of the message payload decreases for the second "aaa" sent, implying that the encoding is different, and so probably based on past state (or that we are dropping bytes, which I think unlikely as no issues seen without the deflate extension).

A couple of issues that cropped up in my searching and may be useful:

• Potential issue in flate2, or at least a bit of chatter around how to resolve it "corrupt deflate stream" error rust-lang/flate2-rs#251
• does autobahn catch such issues; maybe not? Test 13 WebSocket Compression (different parameters) crossbario/autobahn-testsuite#112

And the RFC around how this should be implemented:

https://datatracker.ietf.org/doc/html/rfc7692#section-7.2.2