Testing strategy: C, Rust, Miri

[jsha]

Right now we have a small C-based integration test (client.c and server.c), and a few Rust-based unittests. I want to expand both our unittest and integration test coverage, with unittests first.

One open question: To what extent should our tests be in Rust vs C?

My initial thinking was that the tests should be in C, since that is what will ultimately be using the library, and we can catch issues that only occur on the C->Rust boundary. More recently, thinking about Miri, I realized we can only get Miri coverage for tests written in Rust. That's a strong reason to write tests in Rust.

However, in #208 I realized that most non-trivial rustls operations will wind up invoking FFI code in *ring*, which causes Miri to fail. Unfortunately I think that means we can't use Miri for most of our test, unless we develop a way to mock out *ring*, which seems unlikely.

The tooling for unittests in Rust is so much more convenient that even with the finding above, I'm still leaning towards writing most of the tests in Rust (and expanding on / improving the integration test in C). Thoughts @djc @tgeoghegan @icing?

[kevinburke]

I'd prefer to write tests in Rust as well. One barrier for me was figuring out how to create C objects, in Rust, that I can pass to the public API's the same way that C code would. I ended up managing it here but it took me a lot of time to figure out.

More examples of how to do that or maybe helper functions would be useful there.

https://github.com/rustls/rustls-ffi/pull/147/files#diff-0f44c0313de2467f5f0ca40474beb8ab7a2f91c8cfb7f3a26a51f56788014bc3R116-R128

[djc]

I don't have strong opinions on all this, though I'd agree that Rust is generally the friendlier language to write tests in. Your plan basically sounds okay to me.

[divergentdave]

Now that rustls supports pluggable crypto providers, we can run end-to-end rustls-ffi tests in Miri, since we can avoid inline assembly. Miri doesn't support socket APIs, but it does support file I/O with MIRIFLAGS="-Zmiri-disable-isolation". Thus, the workload inside Miri could talk to a client or server over pipes set up by the following shell script.

mkfifo input_pipe
mkfifo output_pipe
netcat localhost 4443 <input_pipe >output_pipe