Use variable rate interval for blob thoughput metrics based on time interval #621
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nterval DA internal dashboard blob throughput does not match blob explorer throughput graphs because dataapi query uses 2m rate interval and DA dashboard uses auto `$__rate_interval` See https://grafana.com/blog/2020/09/28/new-in-grafana-7.2-__rate_interval-for-prometheus-rate-queries-that-just-work/ The problem is that `$__rate_interval` is a grafana only feature and can not be used for raw prometheus queries. This change converts dataAPI metrics to use the auto variable rate based on query interval length and changes the default rate from 2m to 4m to match what `$__rate_interval` does under the covers. Use `4m` for < 7d time interval Use `11m` for >= 7d time interval
pschork
force-pushed
the
pschork/metrics_v2
branch
from
3aa6744 to
d1484e7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DA internal dashboard blob throughput does not match blob explorer throughput graphs because dataapi query uses 2m rate interval and DA dashboard uses auto
$__rate_intervalBlob explorer show 5MiB spikes, whereas DA dashboard show 1.5MiB throughput.
Comparison overlay of 5MiB
2mrate vs 1.5MiB4mrate aka$__rate_intervalBlob explorer throughput breaks when you expand to 24h & 7d because of the 2m rate used.
See https://grafana.com/blog/2020/09/28/new-in-grafana-7.2-__rate_interval-for-prometheus-rate-queries-that-just-work/
The problem is that
$__rate_intervalis a grafana only feature and can not be used for raw prometheus queries. This change converts dataAPI metrics to use the auto variable rate based on query interval length and changes the default rate from 2m to 4m to match what$__rate_intervaldoes under the covers.Use
4mfor < 7d time intervalUse
11mfor >= 7d time interval10m (note
$__rate_intervalmatches 4m perfectly)30m (note
$__rate_intervalmatches 4m perfectly)1h (note
$__rate_intervalmatches 4m perfectly)12h (note
$__rate_intervalmatches 4m perfectly)24h (note
$__rate_intervalmatches 4m perfectly)7d (note
$__rate_intervalmatches 11m perfectly)Why are these changes needed?
Checks