Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc review fixes part 2 #155

Merged
merged 10 commits into from
Jul 8, 2024
31 changes: 11 additions & 20 deletions topics/latency.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ or setup.

We call this kind of latency **intrinsic latency**, and `valkey-cli`
is able to measure it. This is an example run
under Linux 3.11.0 running on an entry level server.
under Linux 3.11.0 running on an entry level server around 2014.
(This is old, but it illustrates how to measure this.)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this for now, but would it be good to illustrate what this looks like on modern linux/hardware sometime in the future with an issue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I tried on the laptop but I got pretty bad numbers. It needs a real machine. I opened #156.

Another option is to just not mention which hardware and kernel it was running on. The numbers could still make sense today, depending on hardware, OS and virtualized environment. A machine with lots of other loads can very well have these numbers today. What it does is actually "it will just try to measure the largest time the kernel does not provide CPU time to run to the valkey-cli process itself" and its on the microseconds scale. (I'd expect the difference to depend more on the scheduler, whether it's a real-time kernel, than anything else.)

The comment for the second run further down "The following is a run on a Linode 4096
instance running Valkey and Apache" we can change to something more generic like "in a virtualized environment sharing hardware with other loads".


Note: the argument `100` is the number of seconds the test will be executed.
The more time we run the test, the more likely we'll be able to spot
Expand Down Expand Up @@ -233,24 +234,13 @@ of a large memory chunk can be expensive.
Fork time in different systems
------------------------------

Modern hardware is pretty fast at copying the page table, but Xen is not.
The problem with Xen is not virtualization-specific, but Xen-specific. For instance using VMware or Virtual Box does not result into slow fork time.
The following is a table that compares fork time for different Valkey instance
size. Data is obtained performing a BGSAVE and looking at the `latest_fork_usec` filed in the `INFO` command output.
Modern hardware is pretty fast at copying the page table.
So are modern hardware-assisted virtualized environments,
but fork can be really slow in older virtualized environments without hardware support.
As of 2024, this is hardly a problem.

However the good news is that **new types of EC2 HVM based instances are much
better with fork times**, almost on par with physical servers, so for example
using m3.medium (or better) instances will provide good results.

* **Linux beefy VM on VMware** 6.0GB RSS forked in 77 milliseconds (12.8 milliseconds per GB).
* **Linux running on physical machine (Unknown HW)** 6.1GB RSS forked in 80 milliseconds (13.1 milliseconds per GB)
* **Linux running on physical machine (Xeon @ 2.27Ghz)** 6.9GB RSS forked into 62 milliseconds (9 milliseconds per GB).
* **Linux VM on 6sync (KVM)** 360 MB RSS forked in 8.2 milliseconds (23.3 milliseconds per GB).
* **Linux VM on EC2, old instance types (Xen)** 6.1GB RSS forked in 1460 milliseconds (239.3 milliseconds per GB).
* **Linux VM on EC2, new instance types (Xen)** 1GB RSS forked in 10 milliseconds (10 milliseconds per GB).
* **Linux VM on Linode (Xen)** 0.9GBRSS forked into 382 milliseconds (424 milliseconds per GB).

As you can see certain VMs running on Xen have a performance hit that is between one order to two orders of magnitude. For EC2 users the suggestion is simple: use modern HVM based instances.
You can measure the fork time for a Valkey instance by
performing a BGSAVE and looking at the `latest_fork_usec` filed in the `INFO` command output.
zuiderkwast marked this conversation as resolved.
Show resolved Hide resolved

Latency induced by transparent huge pages
-----------------------------------------
Expand Down Expand Up @@ -584,7 +574,7 @@ This is how this feature works:
* The user enables the software watchdog using the `CONFIG SET` command.
* Valkey starts monitoring itself constantly.
* If Valkey detects that the server is blocked into some operation that is not returning fast enough, and that may be the source of the latency issue, a low level report about where the server is blocked is dumped on the log file.
* The user contacts the developers writing a message in the Valkey Google Group, including the watchdog report in the message.
* The user contacts the developers by opening an issue on GitHub, including the watchdog report in the message.

Note that this feature cannot be enabled using the valkey.conf file, because it is designed to be enabled only in already running instances and only for debugging purposes.

Expand Down Expand Up @@ -618,4 +608,5 @@ The following is an example of what you'll see printed in the log file once the

Note: in the example the **DEBUG SLEEP** command was used in order to block the server. The stack trace is different if the server blocks in a different context.

If you happen to collect multiple watchdog stack traces you are encouraged to send everything to the Valkey Google Group: the more traces we obtain, the simpler it will be to understand what the problem with your instance is.
If you happen to collect multiple watchdog stack traces you are encouraged to post everything in a GitHub issue.
The more traces we obtain, the simpler it will be to understand what the problem with your instance is.
36 changes: 19 additions & 17 deletions topics/lru-cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ The following policies are available:
* **noeviction**: New values aren’t saved when memory limit is reached. When a database uses replication, this applies to the primary database
* **allkeys-lru**: Keeps most recently used keys; removes least recently used (LRU) keys
* **allkeys-lfu**: Keeps frequently used keys; removes least frequently used (LFU) keys
* **volatile-lru**: Removes least recently used keys with the `expire` field set to `true`.
* **volatile-lfu**: Removes least frequently used keys with the `expire` field set to `true`.
* **volatile-lru**: Removes least recently used keys with a time-to-live (TTL) set.
* **volatile-lfu**: Removes least frequently used keys with a TTL set.
* **allkeys-random**: Randomly removes keys to make space for the new data added.
* **volatile-random**: Randomly removes keys with `expire` field set to `true`.
* **volatile-ttl**: Removes keys with `expire` field set to `true` and the shortest remaining time-to-live (TTL) value.
* **volatile-random**: Randomly removes keys with a TTL set.
* **volatile-ttl**: Removes keys with a TTL set, the keys with the shortest remaining time-to-live value first.

The policies **volatile-lru**, **volatile-lfu**, **volatile-random**, and **volatile-ttl** behave like **noeviction** if there are no keys to evict matching the prerequisites.

Expand All @@ -68,7 +68,7 @@ In general as a rule of thumb:

The **volatile-lru** and **volatile-random** policies are mainly useful when you want to use a single instance for both caching and to have a set of persistent keys. However it is usually a better idea to run two Valkey instances to solve such a problem.

It is also worth noting that setting an `expire` value to a key costs memory, so using a policy like **allkeys-lru** is more memory efficient since there is no need for an `expire` configuration for the key to be evicted under memory pressure.
It is also worth noting that setting a TTL value to a key costs memory, so using a policy like **allkeys-lru** is more memory efficient since there is no need for a TTL configuration for the key to be evicted under memory pressure.

## How the eviction process works

Expand All @@ -88,11 +88,9 @@ Valkey LRU algorithm is not an exact implementation. This means that Valkey is
not able to pick the *best candidate* for eviction, that is, the key that
was accessed the furthest in the past. Instead it will try to run an approximation
of the LRU algorithm, by sampling a small number of keys, and evicting the
one that is the best (with the oldest access time) among the sampled keys.

However, since Redis OSS 3.0 the algorithm was improved to also take a pool of good
candidates for eviction. This improved the performance of the algorithm, making
it able to approximate more closely the behavior of a real LRU algorithm.
one that is the best (with the oldest access time) among the sampled keys, while
also managing a pool of good candidates for eviction.
This algorithm consumes less memory than an exact LRU algorithm.

What is important about the Valkey LRU algorithm is that you **are able to tune** the precision of the algorithm by changing the number of samples to check for every eviction. This parameter is controlled by the following configuration directive:

Expand All @@ -105,20 +103,23 @@ the LRU approximation used by Valkey with true LRU.

![LRU comparison](lru_comparison.png)

The test to generate the above graphs filled a Valkey server with a given number of keys. The keys were accessed from the first to the last. The first keys are the best candidates for eviction using an LRU algorithm. Later more 50% of keys are added, in order to force half of the old keys to be evicted.
The test to generate the above graphs filled a server with a given number of keys. The keys were accessed from the first to the last. The first keys are the best candidates for eviction using an LRU algorithm. Later more 50% of keys are added, in order to force half of the old keys to be evicted.

You can see three kind of dots in the graphs, forming three distinct bands.

* The light gray band are objects that were evicted.
* The gray band are objects that were not evicted.
* The green band are objects that were added.

In a theoretical LRU implementation we expect that, among the old keys, the first half will be expired. The Valkey LRU algorithm will instead only *probabilistically* expire the older keys.
In a theoretical LRU implementation we expect that, among the old keys, the first half will be evicted.
The Valkey LRU algorithm will instead only *probabilistically* evicts the older keys.

As you can see Redis OSS 3.0 does a better job with 5 samples compared to Redis OSS 2.8, however most objects that are among the latest accessed are still retained by Redis OSS 2.8. Using a sample size of 10 in Redis OSS 3.0 the approximation is very close to the theoretical performance of Redis OSS 3.0.
As you can see, Redis OSS 3.0 does a reasonable job with 5 samples.
Using a sample size of 10, the approximation is very close to an exact LRU implementation.
(The LRU algorithm hasn't changed considerably since this test was performed, so the performance of Valkey is similar in this regard.)

Note that LRU is just a model to predict how likely a given key will be accessed in the future. Moreover, if your data access pattern closely
resembles the power law, most of the accesses will be in the set of keys
resembles the power law; most of the accesses will be in the set of keys
the LRU approximated algorithm can handle well.

In simulations we found that using a power law access pattern, the difference between true LRU and Valkey approximation were minimal or non-existent.
Expand All @@ -130,16 +131,17 @@ difference in your cache misses rate.
To experiment in production with different values for the sample size by using
the `CONFIG SET maxmemory-samples <count>` command, is very simple.

## The new LFU mode
## The LFU mode

Starting with Redis OSS 4.0, the [Least Frequently Used eviction mode](http://antirez.com/news/109) is available. This mode may work better (provide a better
The [Least Frequently Used eviction mode](http://antirez.com/news/109) is available as an alternative to LRU.
This mode may work better (provide a better
hits/misses ratio) in certain cases. In LFU mode, Valkey will try to track
the frequency of access of items, so the ones used rarely are evicted. This means
the keys used often have a higher chance of remaining in memory.

To configure the LFU mode, the following policies are available:

* `volatile-lfu` Evict using approximated LFU among the keys with an expire set.
* `volatile-lfu` Evict using approximated LFU among the keys with a time-to-live (TTL) set.
* `allkeys-lfu` Evict any key using approximated LFU.

LFU is approximated like LRU: it uses a probabilistic counter, called a [Morris counter](https://en.wikipedia.org/wiki/Approximate_counting_algorithm) to estimate the object access frequency using just a few bits per object, combined with a decay period so that the counter is reduced over time. At some point we no longer want to consider keys as frequently accessed, even if they were in the past, so that the algorithm can adapt to a shift in the access pattern.
Expand Down
Loading