Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

snappy decompression doesn't work #349

Open
andysworkshop opened this issue Nov 5, 2018 · 1 comment
Open

snappy decompression doesn't work #349

andysworkshop opened this issue Nov 5, 2018 · 1 comment
Labels

Comments

@andysworkshop
Copy link
Contributor

Unfortunately the snappy decompression will not work as currently implemented. The reason is that snappy_uncompress is being called directly on data returned from a socket read regardless of the whether that read results in a complete snappy block or not. Sockets are streams; you may get back 1 byte, everything the peer wrote, or any number in between depending on prevailing network conditions.

The snappy framing format was invented to solve this problem. Basically you need to read frame headers from the stream to inform you of the size of the incoming compressed data which needs to be buffered up using as many reads as it takes to get it all in before the complete buffer can be passed to the decompressor.

This is exactly the same problem that the LZ4 decompressor suffered from.

@grobian
Copy link
Owner

grobian commented Nov 5, 2018

hmmm, sorry to bother you with this obvious fail ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants