Design user management process for results distribution

[sanjiva]

Basic process:

• EC will register organizations who are able to subscribe
• The orgs need to be given access credentials
• Some orgs may not have proper SSL certificates - should we consider support HTTP and signing the payload? The data itself is not confidential but its important to have a tamper proof delivery model. Something like https://en.wikipedia.org/wiki/JSON_Web_Signature? An alternative is to allow orgs to upload a self-signed cert but unnecessary pain

[MaryamZi]

• Some orgs may not have proper SSL certificates - should we consider support HTTP and signing the payload? The data itself is not confidential but its important to have a tamper proof delivery model. Something like https://en.wikipedia.org/wiki/JSON_Web_Signature? An alternative is to allow orgs to upload a self-signed cert but unnecessary pain

When a secret is specified for the WebSub subscriber (which is mandatory in the current implementation), signing is already done to some degree (https://www.w3.org/TR/websub/#signing-content).

Is this suggestion for something more advanced?

[sanjiva]

Oh I didn't remember that! OK so does this mean there's really no issue with delivering the result over HTTP?

[MaryamZi]

Yeah, IMO there shouldn't be any issue.

Since the hub service will be on HTTPS, the subscription request sent specifying the secret, at service start up, shouldn't cause any issues IINM.

With the following assumptions

• a subscription request is always sent specifying a secret
• only the hub and the particular subscriber know the secret for that particular subscription
• signature validation is done automatically for any and all content delivered

IMO, this should be OK.

[sanjiva]

OK that's great .. so then we'll go ahead encouraging SSL but not requiring it.

[sanjiva]

We need to make sure that each org who is registering is only registering one subscriber per org.

They will be given a username/pass to send as part of subscription process, but how do we control so only one sub per that user is done?

One option is to fail the auth if there's a registered subscriber for that user already. Alternatively, we can replace an existing subscriber with the latest one for a given user - that means we need to keep the subscription ID in a table against the user and execute the logic. Is that possible with the hub APIs? Or is there an alternative solution?

[MaryamZi]

Do we expect them to subscribe with the same callback URL? If it is the same callback URL, a new subscription request renews an old subscription (by replacing it). So for a single topic, there can only be one subscription for a particular callback URL.

If the callbacks are different, we'll have to consider an alternative as suggested.

By failing auth, do you mean something like writing a custom inbound auth handler that maintains the already subscribed users (along with the topics + callbacks maybe) and identifies subscriptions from the same user?

The hub object only allow retrieving all the topics and all the subscribers per topic.

Assuming that we know the topic + callback ([x, y]) that is now outdated, we can send an unsubscription request for [x, y] to the hub. However, an intent verification request will be sent to the callback before removing the subscription - which would require that to still be accessible.

[sanjiva]

I imagine some of the media guys will be using tools like ngrok to open a port to their machines. That means the subscription URL can change each time they reconnect! And we need to remove the old ones or the hub will keep trying to deliver to them, wasting resources.

Can we have some kind of subscriber ID? Is that part of the WebSub spec?

Maybe we can do that at the app level - we can use auth for example to get a username and use that as the key to making sure there's only one subscriber URL for that username.

BTW ref this comment: "an intent verification request will be sent to the callback before removing the subscription". If intent verification fails what happens? What happens if it passes? Basically we want to forcefully remove the subscriber.

[MaryamZi]

I imagine some of the media guys will be using tools like ngrok to open a port to their machines. That means the subscription URL can change each time they reconnect! And we need to remove the old ones or the hub will keep trying to deliver to them, wasting resources.

Can we have some kind of subscriber ID? Is that part of the WebSub spec?

It does not, the topic, callback tuple is expected to be unique, which takes away the requirement for a separate ID I guess.

Maybe we can do that at the app level - we can use auth for example to get a username and use that as the key to making sure there's only one subscriber URL for that username.

Yeah, this seems to be the only option to identify subscribers uniquely atm. However we do have the issue with intent verification.

BTW ref this comment: "an intent verification request will be sent to the callback before removing the subscription". If intent verification fails what happens? What happens if it passes? Basically we want to forcefully remove the subscriber.

We only remove the subscription if the intent verification succeeds, because anyone can send an unsubscription request, for an arbitrary subscriber, if they know the topic and callback.

The only other time we remove a subscription is if content delivery fails with a 410 Gone status code.

Do you think the hub should have an option to forcefully remove subscriptions? This can be allowed via the websub:Hub object only, to make sure only the owner can do so. However the spec does say
"The hub MUST keep the subscription active until the end of the lease duration, and if a new update is published to the topic, MUST continue to retry delivery to the previously-failing subscriber.".

[sanjiva]

Yes please the hub should always be able to throw someone out! So lets have a way to forcefully remove a subscriber directly in the code.

Let me give an example. Suppose the lease runs out and they need to resub. Suppose the user auth fails for that (because subscription period has finished). Now they can't resub and we should not ask them "hey are you ok to unsub".

[sanjiva]

Short term plan:

• issue every subscribed media org a single user ID/password
• use Ballerina's built-in basic auth filter to control access to all services
• keep record of subscription endpoint being registered by a given user ID
• if user tries to register a second subscriber one kill the first one (forcibly) and register new one
• use same creds for result display website