Concurrency level?

[brent-holland]

Howdy sir,

I was looking at this page:

https://github.com/ben-manes/caffeine/wiki/Specification

I noted there was no section for 'concurrency level'? We have this in our current configuration for guava cache, and I'm betting it might be related to the use of a concurrent hash map? Maybe the implementation of caffeine doesn't use this data structure and this no longer applies, but is there something similar i should be setting?

[ben-manes]

tldr; it is a no-op option since Java 8, so it is not exposed

In Java 5's ConcurrentHashMap, the table was statically divided into segments based on the concurrencyLevel (adjusted to a power-of-two). This was used to lock the entire segment when writing an entry that hashed into it, which led to coarse grained locking. A read is lock-free by volatile reads of a immutable snapshot linked list, so it could walk that safely. This resulted in good performance in general, but writes did not scale very well and it did not offer rich per-entry computations as it would lock too broadly. Guava adapted that by (effectively) storing a future as the entry value so that the hash table operations stay immediate and the load is scoped under the future's lock so that it does not block other writers. As Guava is a fork of the Java 5 version, this setting is still relevant for their code.

In Java 8's rewrite of ConcurrentHashMap, it will dynamically allocate hash bins which are smaller sets of entries. The number of bins increase when the map grows, so a larger map will have more available. A write will lock a bin like before, but now the table capacity determines the likelihood of a collision. As the chance is lowered, the compute methods were added to run foreign code under the bin's lock. That does mean that a long running compute can block siblings, so it is disfavored and Guava's future-style approach is available explicitly via our AsycnCache for per-entry locking. This hash table design means that a small, write-heavy map will suffer more contention so an alternative to futures is to increase the initialCapacity to influence locks and reduce collisions. The faq briefly discusses these types of problems at a high level. For backwards compatibility ConcurrentHashMap still accepts a concurrencyLevel constructor argument, but it is ignored.

Caffeine's own data structures don't require a concurrencyLevel, so that configuration option was dropped entirely. There are cases where striping helps reduce contention, but that is resized dynamically based on compare-and-set conflicts. An example of this is our striped ring buffer or counters for statistics. Therefore we dropped this knob, but be aware of the initialCapacity or AsyncCache tricks for solving related but different write contention problems.