Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[deltatocumulativeprocessor] Introduce an upper bound for exp histogram buckets #36874

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

euroelessar
Copy link

Description

This PR introduces a limit on the maximum number of exponential histogram buckets within the deltatocumulativeprocessor. Previously, when merging delta metrics into cumulative metrics, the resulting exponential histograms could grow very large, potentially causing excessive memory usage and processing overhead. By capping the number of buckets at 160 and dynamically downscaling histograms when necessary, this change ensures that the processor remains efficient and stable even when handling large, merged exponential histograms.

Link to tracking issue

Fixes #33277

Testing

Added unit tests for edge cases.

Documentation

Updated changelog.

@euroelessar
Copy link
Author

@sh0rez Please have a look, it's an alternative take on #34157, it avoids any extra memory allocations and could be easier to follow.
All changes are in ExpHistogram.Add (and Merge called from it), as it's the only place which can grow the histogram.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

deltatocumulative: Number of buckets in exponential histograms should be capped
2 participants