Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RIP-75] Supports timer message on RocksDB #9141

Open
1 task done
3424672656 opened this issue Jan 17, 2025 · 0 comments · May be fixed by #9142
Open
1 task done

[RIP-75] Supports timer message on RocksDB #9141

3424672656 opened this issue Jan 17, 2025 · 0 comments · May be fixed by #9142

Comments

@3424672656
Copy link
Contributor

Before Creating the Enhancement Request

  • I have confirmed that this should be classified as an enhancement rather than a bug/feature.

Summary

We will use rocksdb's time wheel support to implement arbitrary delayed messages, solve the problem of cold read and metric imprecision caused by large-scale delayed messages, and support timely deletion and more accurate latency levels and other features

Motivation

We have encountered some problems in the actual production environment, and have been troubled by these problems. In large-scale timed message scenarios, frequent reading of old files reduces the read rate, increases the delay of delayed messages, and causes problems such as inaccurate observation and delay. Meanwhile, functions such as preheating of file version suffer large losses. Therefore, the introduction of rocksdb can avoid the impact of cold read, provide more accurate index calculation, and support longer timed messages. Provide more precise latency levels, but we still need to ensure message write performance and read performance and latency, etc.

Detailed design: https://shimo.im/docs/913JMO8ZoJugwEAE

Describe the Solution You'd Like

The proposal addresses two main issues:

  1. Cold read problem caused by large-scale timed messages
  2. Delete messages and metric inaccuracies when messages are sent out

Extra can support more optimizations

  1. Delete messages in a timely manner
  2. Messages can be preheated and more precise timing can be supported, such as supporting the millisecond level

Describe Alternatives You've Considered

no

Additional Context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant