Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add code to check for staleness in labelstore #5970

Merged
merged 6 commits into from
Dec 15, 2023
Merged

Conversation

mattdurham
Copy link
Collaborator

PR Description

This wires up the staleness check from the labelstore that will ideally reduce memory usage in high cardinality or high churn environments.

Which issue(s) this PR fixes

Closes #5585

Notes to the Reviewer

PR Checklist

  • CHANGELOG.md updated
  • Tests updated

@mattdurham mattdurham requested a review from thampiotr December 13, 2023 16:58
@mattdurham mattdurham marked this pull request as ready for review December 13, 2023 16:58
Comment on lines +25 to +26
totalIDs *prometheus.Desc
idsInRemoteWrapping *prometheus.Desc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could these be constants in the package?

service/labelstore/service.go Outdated Show resolved Hide resolved
component/prometheus/interceptor.go Outdated Show resolved Hide resolved
// Run starts a Service. Run must block until the provided
// context is canceled. Returning an error should be treated
// as a fatal error for the Service.
func (s *service) Run(ctx context.Context, host agent_service.Host) error {
<-ctx.Done()
return nil
staleCheck := time.NewTicker(10 * time.Minute)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably don't need to do this now, but do you think there may be a need to configure this in the future?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since staleness markers arent configurable and this was meant to be double the staleness marker length I dont think it will need to be configurable. Though that is famous last words.

@mattdurham mattdurham enabled auto-merge (squash) December 15, 2023 14:20
@mattdurham mattdurham merged commit fd12c48 into main Dec 15, 2023
10 checks passed
@mattdurham mattdurham deleted the add_staleness_check branch December 15, 2023 14:43
BarunKGP pushed a commit to BarunKGP/grafana-agent that referenced this pull request Feb 20, 2024
* Add code to check for staleness which touched a ton of tests since I also added metrics.

* Update metric

* Add explicit _ for register

* pr feedback on changing to gauges

* updated comment with test results
@github-actions github-actions bot added the frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed. label Feb 21, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GlobalRefMap/LabelStore should check staleness markers and/or other garbage collections
3 participants