When used with Redis cluster, redisClient doesn't get subscribed to all of the shards within the cluster.

[barbarosalp]

Questions

How can we get Vertx Redis client to subscribe to the keyspace events of all the shards of a Redis cluster(AWS Elasticache Redis Cluster)?

Version

Which version(s) did you encounter this bug ?
3.9.3

Context

Using the configuration endpoint of AWS Elasticache Redis cluster that consists of 3 shards and when subscribed to keyspace events by consuming the io.vertx.redis.__keyevent@0__:expired event bus, the consumer receives messages only from one of the shards instead of all of them. In our case, it only gets messages from Shard3.

Do you have a reproducer?

No

Steps to reproduce

1. Make sure that there is a Redis Cluster up and running with 3 shards on AWS.
2. Use configuration endpoint as the connection string and set type as RedisClientType.CLUSTER
3. With redis-cli, make sure you are connected to 3 of the shards in separate terminals and set a key in each of the shards.
4. In the Vertx Application, subscribe to this eventbus address io.vertx.redis.__keyevent@0__:expired
5. You will only receive the keyspace events from one of the shards instead of all of them.

Extra

RxJava2
AWS Elasticache
Java 8

Redis Client

    clientOptions.setConnectTimeout(connectTimeout);
    clientOptions.setReconnectAttempts(maxRetry);

    final RedisOptions redisOptions = new RedisOptions()
        .setConnectionString(String.format("redis://%s:%s", host, port))
        .setType(RedisClientType.CLUSTER)
        .setNetClientOptions(clientOptions);

    final Redis client = Redis.createClient(vertx, redisOptions);
    return RedisAPI.api(client);```

[pmlopes]

@barbarosalp can you, verify that the issue still happens on master? we're getting close to a 4.0.0 release, so backporting fixes is getting hard as the codebases are quite appart right now.

[cristian-com]

I don't think you example will work. To be able to receive notification of all shards you would have to be connected to all of them.

You can find it in the official doc:

Events in a cluster
Every node of a Redis cluster generates events about its own subset of the keyspace as described above. However, unlike regular Pub/Sub communication in a cluster, events' notifications are not broadcasted to all nodes. Put differently, keyspace events are node-specific. This means that to receive all keyspace events of a cluster, clients need to subscribe to each of the nodes.

There is also a limitation of clustering mode not supporting SUB commands here

Note | Pooling is not compatible with SUBSCRIBE, UNSUBSCRIBE, PSUBSCRIBE or PUNSUBSCRIBE because these commands will modify the way the connection operates and the connection cannot be reused.

A solution that haven't tested could probably be operating on connection mode see io.vertx.redis.client.impl.RedisStandaloneConnection

But I guess you would have to create and subscribe one by one and and will sure find more issues since the client wouldn't have idea about the cluster e.g receiving double messages from master and replica.