Telemetry could be aggregated at a configurable interval, or at configurable times

[kennsippell]

Is your feature request related to a problem? Please describe.
The Muso-Mali project team is anecdotally reporting that the upgrade from v3.5 to v3.6.1 caused a performance regression. v3.5 was deployed on July 5th and the upgrade to v3.6.1 occurred within the same month. @senseibara attempted to analyse the telemetry data to support or refute the claim that performance regressed, but can only look at aggregate date for the full month.

A build rollout is a significant event and tracking telemetry metrics before/after would be extremely useful for understanding the performance impacts of our code changes.

Describe the solution you'd like
Currently we aggregate all telemetry data into a single monthly document. Perhaps we could aggregate by month-build so if a deployment rolls out mid-month, we can compare data before/after the rollout.

[SCdF]

I think it's worth building up these learnings about things we'd like and channeling them into an investigation of "real" (ie third party) tools to replace the existing telemetry work we've done.

This gives us the opportunity to either perform that replacement (and not spend time improving something we've ultimately decided to throw away), or decide that our situation is unique enough to warrant custom work and then treat the existing telemetry as first class (and so then start caring about UIs and proper support and the like).

[kennsippell]

Although a core-upgrade specific workflow is an improvement. Reflecting on this need more generally, it would be great if telemetry could be aggregated at a configurable interval, or at configurable times.

Essentially what we want is to be able to look at telemetry before an event (core upgrade, config rollout, training, etc), and look at telemetry after an event to make comparisons.

Some projects have lots of air time, strong connectivity, newer phones and they can easily accomodate a higher rate of telemetry reporting - daily probably makes sense here. Other projects maybe do need to aggregate less frequently and the configuration should be more conservative - like monthly (or disabled).

[mrsarm]

@kennsippell ,

I'm going to close this ticket taking into account that #6915 supersedes it, but feel free to re-open it if you think #6915 doesn't cover all the needs explained in this ticket, or open a new one that covers the missed bits.