Test result analytics

[howardjohn]

It would be useful to be able to run queries against historical test rules. The easy way to do this is to download all the logs and use grep and what not, but all the test logs add up to 500gb+ last I checked.

Some example use cases:

• Search for a specific error message in logs
• Find all times a test has failed
• View pass rate of tests over time
• View test duration over time

[clarketm]

The k8s equivalent for test results analytics is probably triage: https://storage.googleapis.com/k8s-gubernator/triage/index.html. Unfortunately, it is very k8s specific and not reusable. With that being said, we can achieve a similar result with a database, lightweight frontend, and a job to "refresh" the db. But I agree an application for these use cases would be valuable.

[scottilee]

Elasticsearch + Kibana might be too heavy weight, could we use Grafana logs?