aggregation modifier for the <aggregation>_over_time() functions

[yuanbohan]

what's the scenario

query the average value of the last 5 minutes series of prometheus_http_requests_total, and group by code

what's the issue

The <aggregation>_over_time functions accept range vector, but do not support aggregation modifier by or without, while aggregation-operators only accept instant vector.

examples

the actual dataset

avg_over_time(prometheus_http_requests_total[5m])

Series	Value
{code="200", handler="/-/ready", instance="prometheus:9090", job="prometheus"}	25529.25
{code="200", handler="/alerts", instance="prometheus:9090", job="prometheus"}	13
{code="200", handler="/api/v1/alertmanagers", instance="prometheus:9090", job="prometheus"}	4
{code="200", handler="/api/v1/format_query", instance="prometheus:9090", job="prometheus"}	6
{code="200", handler="/api/v1/label/:name/values", instance="prometheus:9090", job="prometheus"}	5455
{code="200", handler="/api/v1/labels", instance="prometheus:9090", job="prometheus"}	993.5
{code="200", handler="/api/v1/metadata", instance="prometheus:9090", job="prometheus"}	1994
{code="200", handler="/api/v1/query", instance="prometheus:9090", job="prometheus"}	31570.699999999997
{code="200", handler="/api/v1/query_exemplars", instance="prometheus:9090", job="prometheus"}	394
{code="200", handler="/api/v1/query_range", instance="prometheus:9090", job="prometheus"}	89148
{code="200", handler="/api/v1/rules", instance="prometheus:9090", job="prometheus"}	37
{code="200", handler="/api/v1/series", instance="prometheus:9090", job="prometheus"}	756
{code="200", handler="/api/v1/status/buildinfo", instance="prometheus:9090", job="prometheus"}	127
{code="200", handler="/api/v1/status/config", instance="prometheus:9090", job="prometheus"}	34
{code="200", handler="/api/v1/status/flags", instance="prometheus:9090", job="prometheus"}	13
{code="200", handler="/api/v1/status/runtimeinfo", instance="prometheus:9090", job="prometheus"}	4
{code="200", handler="/api/v1/status/tsdb", instance="prometheus:9090", job="prometheus"}	14
{code="200", handler="/api/v1/targets", instance="prometheus:9090", job="prometheus"}	45
{code="200", handler="/config", instance="prometheus:9090", job="prometheus"}	6
{code="200", handler="/favicon.ico", instance="prometheus:9090", job="prometheus"}	241
{code="200", handler="/flags", instance="prometheus:9090", job="prometheus"}	18
{code="200", handler="/graph", instance="prometheus:9090", job="prometheus"}	353
{code="200", handler="/manifest.json", instance="prometheus:9090", job="prometheus"}	137
{code="200", handler="/metrics", instance="prometheus:9090", job="prometheus"}	100774.5
{code="200", handler="/rules", instance="prometheus:9090", job="prometheus"}	5
{code="200", handler="/service-discovery", instance="prometheus:9090", job="prometheus"}	8
{code="200", handler="/static/*filepath", instance="prometheus:9090", job="prometheus"}	778
{code="200", handler="/status", instance="prometheus:9090", job="prometheus"}	6
{code="200", handler="/targets", instance="prometheus:9090", job="prometheus"}	9
{code="200", handler="/tsdb-status", instance="prometheus:9090", job="prometheus"}	16
{code="200", handler="/version", instance="prometheus:9090", job="prometheus"}	4
{code="204", handler="/api/v1/*path", instance="prometheus:9090", job="prometheus"}	1
{code="302", handler="/", instance="prometheus:9090", job="prometheus"}	278
{code="400", handler="/api/v1/format_query", instance="prometheus:9090", job="prometheus"}	10
{code="400", handler="/api/v1/label/:name/values", instance="prometheus:9090", job="prometheus"}	12
{code="400", handler="/api/v1/query", instance="prometheus:9090", job="prometheus"}	141
{code="400", handler="/api/v1/query_range", instance="prometheus:9090", job="prometheus"}	224
{code="400", handler="/api/v1/series", instance="prometheus:9090", job="prometheus"}	1
{code="404", handler="/debug/*subpath", instance="prometheus:9090", job="prometheus"}	2
{code="404", handler="/static/*filepath", instance="prometheus:9090", job="prometheus"}	1
{code="422", handler="/api/v1/query", instance="prometheus:9090", job="prometheus"}	27
{code="422", handler="/api/v1/query_range", instance="prometheus:9090", job="prometheus"}	7
{code="503", handler="/api/v1/query", instance="prometheus:9090", job="prometheus"}	31
{code="503", handler="/api/v1/query_range", instance="prometheus:9090", job="prometheus"}	79

what's the expected query

avg_over_time(prometheus_http_requests_total[5m]) by (code)

Error: Expression incomplete or buggy: 1:51: parse error: unexpected <by>

what's the actual query

avg(avg_over_time(prometheus_http_requests_total[5m])) by (code)

Series	Value
{code="200"}	8337.940322580642
{code="204"}	1
{code="302"}	278
{code="400"}	77.6
{code="404"}	1.5
{code="422"}	17
{code="503"}	55

reference

you can try these promql queries at https://demo.promlens.com/