Split backend and frontend apart

[AyrA]

I create this issue in response to my comments in #73 and #186 to have this properly tracked.

The goal of this is to split the backend and frontend into individual components, removing anything remotely security related from the UI and have it confined to the backend. This provides means to fix #73 by appearing always online, but it also should ensure that people don't have to touch safety critical components when tinkering with ricochet.

If this change is implemented, the basic ricochet client will consist of two components. For a regular user there is no visible change to the application, but for expert users, the application provides maximized customizability.

To reiterate from my comment:

• Easy to write your own UI without potentially touching or misusing cryptographic routines
• Your contacts can always send you messages and you don't depend on both using Ricochet at the same time for message delivery because the backend is always online to send and receive (provided the backend is run as service executable)
• Cryptographic keys can be confined to the service and don't ever need to enter the UI
• Protocol elements can be exposed in a high level (JSON) format that doesn't requires protocol knowledge for encoding and decoding in the UI. In other words, even large protocol changes have no impact on the UI. For example, the switch from onion v2 to v3 would have been much easier.
• Enables integration with other applications. For example someone could write an SMTP/IMAP/POP3 adapter on top of it to make it E-mail like or an XMPP adapter to integrate it with other chat software
• Some new features can be implemented without needing a protocol change. An identity could for example be configured as a chat room simply by making it relay all inbound messages to all contacts.
• Service can be used for automated messaging. For example a bot that acts like a search engine and provides results for your messages.
• The service can update independently of the UI, for example when a new Tor client is released.

Suggested HTTP endpoints

This was very long, so I've put it into a dedicated file here: https://gist.github.com/AyrA/dda323440311622adacd14915ba1f360

[morganava]

So for non-localhost, the obvious solution for the 'reverse proxy' that comes to mind is to use a client-auth protected onion-service, maybe even single-hop though i would need to refresh myself on the security/privacy tradeoffs there. I would also very likely use Gosling's own Honk-RPC for the RPC protocol (basically JSON-RPC but BSON), though HTTP as a transport may make sense.

[AyrA]

So for non-localhost, the obvious solution for the 'reverse proxy' that comes to mind is to use a client-auth protected onion-service, maybe even single-hop though i would need to refresh myself on the security/privacy tradeoffs there.

Since power users are going to host this on a machine they fully trust, they're more likely going to use SSH tunnels, a VPN, or a simple nginx or apache reverse proxy with TLS and an authentication mechanism. Tor of course is possible, but hidden services are slow and error prone. It also means you now run two tor nodes, one that ricochet uses itself and one the user uses to host the API hidden service. It also means the frontend needs socks proxy support to reach the onion service. Although amusing, it increases the complexity a lot

I would also very likely use Gosling's own Honk-RPC for the RPC protocol (basically JSON-RPC but BSON), though HTTP as a transport may make sense.

The idea behind my proposal it to use ready made components rather than developing new ones. Doing your own usually just ends up costing effort for nothing. Especially with an API like this because you're not going to sustain dozens of requests per second, so the tiny performance overhead of JSON compared to binary formats is probably preferrable compared to having the developer learn a new format.

by XKCD

If you want people to develop for this API it should be as easy to use as possible to be accessible to as many developers as possible. Ideally this is done by using tools that modern languages already provide. For this, a simple JSON API is king. It makes implementation in most environments trivial, including web browsers.

This is also why my API description is super generic. You could present this to someone and they can implement it without having to know what the underlying protocol is. Could be ricochet, bitmessage, xmpp, etc.

One downside of JSON is that it's not very suitable for binary data, which is why I included a storage API to bypass this.
You can of course make API endpoints accept multiple formats by evaluating the Content-Type header of the data sent by the client and deliver multiple formats by evaluating the Accept header. This way a client can pick which format is best suitable for them.