Pubsub support

[RangerMauve]

• Add a /pubsub/ path to the gateway
• GET /pubsub/{topic}/ with Accept=event-stream => Get an event stream of events by subscriing to that topic
  • Base format on hypercore-fetch
  • Use peer ID for events as the id of the event
  • Use topic as the event name, or omit the event name entirely so it'll be the default "message"
  • Should we encode peer data in the message body?
• POST /pubsub/{topic}/ {body here} publish data to that topic

Limitations: We need to either use text mode or a base64 encoding to make things easier to work with browsers. e.g. with https://developer.mozilla.org/en-US/docs/Web/API/atob

[makew0rld]

In progress on pubsub branch.

[makew0rld]

Current pubsub message format, received as an event in an event-stream:

id: FuhB2orEIwE=
data: {"from":"QmfULfLaRtF7Kg3ShjFdiYL97P7eEcyL4PcvQLhkzCPCeG","data":"aGVsbG8=","topics":["test"]}

aGVsbG8= is base64-encoded hello

[makew0rld]

There is also the possibility of an error message:

[removed]

[RangerMauve]

There is also the possibility of an error message:

data: something failed blah blah
id: error

Just double checked this, and the format is more like

data: The error message
type: error
id: ""

Which I think is the the best approach. 😁

[makew0rld]

Ah I miswrote that, I think you did too though. It would actually be like:

data: something failed blah blah
event: error

The id is not included which is allowed.

[makew0rld]

If you're happy with this format, I will add it to the spec in #4. I know there were some concerns about the levels of encodings.

[RangerMauve]

Yeah, one thing that's feeling a bit off is the encoding bit.

What do you think about a querystring param for the encoding of the message body?

E.g. ?format=json will have the gateway parse the data as json and emit errors for invalid messages, or ?format=utf8 will parse the data as utf8 strings.

Then the default could be the same as ?format=base64 to base64 encode the binary data.

[RangerMauve]

Also, is there a way to publish a data to be sent to a particular peer identified by it's from value? Or can we only broadcast to everyone (not sure if its even relevant actually).

[makew0rld]

You can only send to specific topics using pubsub, not to specific peers. You could send a message to a secret topic only you and your peer know.

[RangerMauve]

Cool, sounds good then, forgot I mentioned it. 😁

[makew0rld]

Added support for format parameters in 1998b75. Supported formats are json, utf8, base64. Default is base64, and an unrecognized format will cause an error. It is case-insensitive.

This adds a new kind of error message, for when decoding fails.

event: error-decode
data: not valid UTF-8