Experimenting with no_std for capnp-rpc

[nicholastmosher]

Hi there, I'm back! Thanks for adding the changes for no_std a while back. I only recently saw those changes land, so I figured I would try my hand again at putting together a sample program for my embedded project. It seemed that I was able to compile my project when I added capnp as a dependency and capnpc as a build dependency, but that capnp-rpc was still not compiling for no_std. I wanted to see if it'd be possible to get capnp-rpc compiling for no_std using just alloc, so I started playing around with things. I'll describe what I've discovered so far.

A quick TLDR

Right now the biggest thing that seems to be blocking this from working is the fact that capnp-futures/src/write_queue depends on futures::channel::mpsc::unbounded(), which seems truly unavailable on no_std, even with alloc present. Any solution to making capnp-rpc no_std compatible would probably hinge on finding a different channel solution which uses alloc or less.

Features all the way down

When I added capnp-rpc as a dependency to my blank no_std project, the first thing that I noticed is that something seemed to be depending on std, causing the build to fail. I looked at the dependencies of capnp-rpc itself and saw this:

[dependencies.futures]
version = "0.3.0"
default-features = false
features = ["std"]

So, capnp-rpc depends on the std feature of capnp-futures. So I decided to dig deeper and see if there was any way that we could disable the std feature in capnp-futures and still retain a minimal working base on which capnp-rpc could run using just alloc. In capnp-futures, I made a std feature and set it on by default. I also turned off default features for the futures crate to avoid the std dependency there, but made sure to add back futures/std and futures/executor as a dependency of the capnp-futures std feature. Basically, I was hoping to create a big chain of std features which would all turn off if the top-level dependency opted out of default-features.

AsyncRead and AsyncWrite

At about this point, I realized that AsyncRead and AsyncWrite have a hard dependency on std again because of their functions returning io::Result. So I figured I'd take a shot at reproducing the solution you came up with last time for no_std, creating approximations of those traits which are no_std compatible and defining blanket impls for the canonical versions of the traits when std is enabled. I dug down this rabbit hole for awhile, eventually coming up with capnp-futures/async_io, which has the trait approximations as well as some watered-down versions of the AsyncReadExt and AsyncWriteExt traits from futures-util. It turns out that all of the futures you used in capnp-futures (e.g. Read, ReadExact, etc.) are all just basic structs that don't depend on std at all, even though they're not published in a no_std compatible module. So I reproduced just those helper Futures from futures-util that you were using.

futures::channel::mpsc

At this point, my compiler errors basically watered down to just about one problem: that futures::channel::mpsc was not available without std, not even if alloc is present. I tried looking for alternative channel solutions which only required alloc, but I was unable to find any. This is pretty much the largest blocking point that I can see right now. I think it might require an alloc-only implementation of a mpsc channel, which would probably need to be made custom for this. I don't know if you're aware of any crates that would meet that need or whether there's an alternative solution where the dependency on mpsc could be gotten rid of, but that's where this is all stuck at right now.

---

So, I'm not sure where this leaves this issue. I haven't spent way too long thinking of other approaches yet, but I figured I'd share my findings so far in case others have any ideas for how to proceed. I've never worked with concurrency "primitives" before, so while I think the idea of creating an alloc-only mpsc channel is interesting, I'm not sure I would trust myself to come up with a correct or good enough implementation to be upstreamed to this project. I'd love to hear other thoughts on this though!

[dwrensha]

Cool! Thanks for sharing.

I wonder if we could sidestep the AsyncRead/AsyncWrite issues by building things on top of futures::Stream<Item=capnp::message::Reader> and futures::Sink<capnp::message::Builder> instead.