Consider replacing safe-encode feature by unsafe APIs

[jorgecarleitao]

Used with lz4_flex = { version = "0.9.0", default-features = false }, we currently violate Rust's premise that code without unsafe must not result in UB/unsound code.

Specifically, it is surprising that I lose soundness guarantees by changing from

lz4_flex = { version = "0.9.0" }

to

lz4_flex = { version = "0.9.0", default-features = false }

since it is quite difficult to detect that such a change has soundness implications.

One mechanism to avoid this issue is to offer two APIs: an unsafe that does not perform checks, and a safe one, that performs the checks (instead of a feature-based safety checking).

[PSeitz]

Sorry for the late reply.

I agree that two different API has some advantages for soundness and documentation.
Initially I kept them together to minimize the code size and add checks without runtime overhead, but I think two APIs may be better. One serious downside is, that it is a breaking API change.

[PSeitz]

I think I will go forward with this by inverting the feature flags.

e.g. unsafe-decode and unsafe-encode will not be default features and can be enabled to get more performance. Those will not cause UB on any input.

Additionally there will be unchecked-decode which will be unsound. It can be used on trusted sources or when targeting wasm.

That means default-features = false will still result in code that uses no unsafe.
It will require deliberate action to enable the features that use unsafe.

[PSeitz]

With version 0.11 I inverted the feature flag from checked-decode to unchecked-decode. So the API will be sound with:

lz4_flex = { version = "0.11.0", default-features = false }

I didn't invert the other feature flags as I want to minimize the API breakage.

[arthurprs]

This is an old issue but if I may add a few thoughts.

Currently, if a crate transitively enables the safe compressor/decompressor then all lz4 usages in the build will also use that. And of course the reverse could apply, if someone wanted safe code they could get unsafe. -- I've been bit by this one.