Allow for higher sync rate for subsciption calendars

[miaulalala]

@tcitworld wrote:

The Nextcloud server caches the subscriptions (stores data in the database) and exposes them as regular calendars. The default refresh rate for this is of one week, which is appropriate for things like public holiday calendars, but might not be for other things, for instance subscribing to a public calendar from another Nextcloud user, and expecting real-time sync.

This high refresh rate makes sense because on servers with a lot of users and subscriptions, the cron would keep wasting time refreshing calendars for nothing, and there could be a risk of the subscription source blocking the server's IP. For instance, as subscriptions can't be shared (or provided by the admin), if all 10 000 users from a company need to subscribe to the holiday calendar from their local government, that's 10 000 requests you need to make every X minutes/hours/days when the original calendar data only changes once or twice a year.

Nextcloud supports properties available for the subscription source to tell what's the appropriate refresh rate (X-PUBLISHED-TTL, REFRESH-INTERVAL), but in practice no-one provides them.

I think there's an issue about providing the setting to select an appropriate refreshment rate for each subscription to end-users in calendar, which would kinda solve most of the issue, though there's a risk users could abuse it, so a minimal rate setting for the admin would be nice as well. In any case, that's not the current state.

So on the other hand, clients have no issue with refreshing the local subscription every 15 minutes/hour/day or even each time the app is opened, as it's only doing it for a single user and from a dedicated IP address.

Therefore, people who would have been forced to manually add the subscription on their device before this change and will now just used the exposed subscriptions from the NC server may have the impression that subscriptions don't get updated when they expect it.

---

We should take a delta of calendar events and only update those events that have changed.

We should also update all calendars that have the same subscription URI (ex: holiday calendars) at the same time so we only need one run for these calendars.

Work packages

• Use Delta for subscription calendars instead of deleting and reinserting all events #43541
• Adaptive/configurable sync intervals for subscription calendars calendar#6187
• Admin docs, if applicable
• Developer docs, if applicable
• User docs, if applicable

Follow-ups

• Subscription Calendars: Bulk sync for identical URIs #46687

[ChristophWurst]

@miaulalala can this be broken down into manageable chunks? E.g.

1. Efficient subscription sync (delta sync)
2. Adaptive/configurable sync intervals
3. Bulk sync for identical URIs

[miaulalala]

@miaulalala can this be broken down into manageable chunks? E.g.

1. Efficient subscription sync (delta sync)

2. Adaptive/configurable sync intervals

3. Bulk sync for identical URIs

Yes, that sounds reasonable! Shall I open work package tickets for each?

[ChristophWurst]

yes please

[ChristophWurst]

Efficient subscription sync (delta sync)

Tracked in #43541 more or less

[miaulalala]

I was wondering if we should only allow setting a sync rate from the user side if the calendar itself doesn't already provide one?

I guess it's polite but not a must:

This property specifies a positive duration that gives a suggested minimum polling interval for checking for updates to the calendar data. The value of this property SHOULD be used by calendar user agents to limit the polling interval for calendar data updates to the minimum interval specified.

[ChristophWurst]

If the service exposes a sync interval, use that.
Otherwise allow the user to set the interval.

This part of the feature is tracked in nextcloud/calendar#6187.

[miaulalala]

I don't think we need #46687 any more - it's a case of YAGNI

[ChristophWurst]

Removed from the work packages

[tcitworld]

In the meantime until #46723 is achieved, maybe we can set the default for calendarSubscriptionRefreshRate to one day instead of one week?

[ChristophWurst]

That sounds reasonable 👍

[tcitworld]

See #49396