Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(eviction): Tune eviction threshold in cache mode #4142

Merged
merged 13 commits into from
Dec 5, 2024
Merged

fix(eviction): Tune eviction threshold in cache mode #4142

merged 13 commits into from
Dec 5, 2024

Conversation

BagritsevichStepan
Copy link
Contributor

@BagritsevichStepan BagritsevichStepan commented Nov 18, 2024 •
edited
Loading

fixes #4139

In this PR:

  1. Deprecate the oom_deny_ratio flag.
  2. Tune eviction logic: RSS memory usage is now considered when calculating the amount of memory to evict.
  3. Introduce a new flag, eviction_memory_budget_threshold. Eviction begins when the available memory (or RSS memory) falls below eviction_memory_budget_threshold * max_memory_limit.

@BagritsevichStepan BagritsevichStepan changed the title cache(cache_mode): Tune eviction threshold in cache mode fix(cache_mode): Tune eviction threshold in cache mode Nov 18, 2024
@BagritsevichStepan BagritsevichStepan force-pushed the memory/tune-eviction-threshold-in-cache-mode branch from cdd7fc0 to b5e1611 Compare November 18, 2024 10:14
@BagritsevichStepan BagritsevichStepan changed the title fix(cache_mode): Tune eviction threshold in cache mode fix(eviction): Tune eviction threshold in cache mode Nov 18, 2024
@BagritsevichStepan BagritsevichStepan force-pushed the memory/tune-eviction-threshold-in-cache-mode branch 5 times, most recently from 5e6c873 to 2d7dd27 Compare November 25, 2024 10:12
@BagritsevichStepan BagritsevichStepan self-assigned this Nov 25, 2024
@BagritsevichStepan
Copy link
Contributor Author

@adiholden I believe I've identified the issue: I'm calculating rss_threshold for each shard individually, but the current rss_memory value is a global metric for all shards

@BagritsevichStepan BagritsevichStepan force-pushed the memory/tune-eviction-threshold-in-cache-mode branch 3 times, most recently from 9bd58cc to fe22e03 Compare November 26, 2024 18:05
@BagritsevichStepan BagritsevichStepan force-pushed the memory/tune-eviction-threshold-in-cache-mode branch 3 times, most recently from 692a3fa to 051412e Compare December 2, 2024 09:44
@BagritsevichStepan BagritsevichStepan force-pushed the memory/tune-eviction-threshold-in-cache-mode branch 3 times, most recently from 2d407b7 to 42f45ad Compare December 5, 2024 09:51
adiholden
adiholden reviewed Dec 5, 2024
View reviewed changes
adiholden
adiholden reviewed Dec 5, 2024
View reviewed changes
@BagritsevichStepan
Copy link
Contributor Author

🤕

@BagritsevichStepan
Copy link
Contributor Author

Can not reproduce it

@BagritsevichStepan
fix(eviction): Tune eviction threshold in cache mode
41b025a 
fixes dragonflydb#4139

Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
refactor: small fix in tiered_storage_test
17f8952 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
refactor: address comments
6ce8857 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
chore(dragonfly_test): Remove ResetService
b3e6013 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
refactor: fix test_cache_eviction_with_rss_deny_oom test
12f237c 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
refactor: address comments
2b634e1 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
fix(dragonfly_test): Fix DflyEngineTest.Bug207
c8b8acc 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
fix(dragonfly_test): Increase string size in the test Bug207
116adc2 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
refactor: address comments 3
d20bef0 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
refactor: address comments 4
edf9eb2 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
fix: Fix failing tests
bf3bdc8 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
refactor: address comments 5
5327542 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan
refactor: resolve conficts
fa10a4e 
Signed-off-by: Stepan Bagritsevich <[email protected]>
@BagritsevichStepan BagritsevichStepan force-pushed the memory/tune-eviction-threshold-in-cache-mode branch from d0b3a5b to fa10a4e Compare December 5, 2024 11:43
@BagritsevichStepan BagritsevichStepan enabled auto-merge (squash) December 5, 2024 12:17
@BagritsevichStepan BagritsevichStepan merged commit 5483d1d into dragonflydb:main Dec 5, 2024
9 checks passed
@BagritsevichStepan BagritsevichStepan deleted the memory/tune-eviction-threshold-in-cache-mode branch December 5, 2024 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adiholden adiholden adiholden approved these changes

@BorysTheDev BorysTheDev BorysTheDev left review comments

Assignees

@BagritsevichStepan BagritsevichStepan

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

remove oom_deny_ratio flag
3 participants
@BagritsevichStepan @BorysTheDev @adiholden