java.util.concurrent.ExecutionException: Provider ExceptionBug? Expecting status code for size/count

[jabley]

Environment:

RHEL5
Sun Java "1.6.0_20"
Redis 1.2.6
JRedis 1.0RC2

I intermittently see this problem in my application logs. Once this starts to happen, I usually have to restart the app server.

Unable to get value back from redis under key <jmhahqy> Provider ExceptionBug?Expecting status code for size/count
java.util.concurrent.ExecutionException: Provider ExceptionBug?  Expecting status code for size/count
    at org.jredis.ri.alphazero.connection.PendingRequest.checkStatus(PendingRequest.java:112)
    at org.jredis.ri.alphazero.connection.PendingRequest.get(PendingRequest.java:146)
    at org.jredis.ri.alphazero.connection.PendingRequest.get(PendingRequest.java:26)
    at org.jredis.ri.alphazero.JRedisFutureSupport$FutureByteArray.get(JRedisFutureSupport.java:1391)
    at org.jredis.ri.alphazero.JRedisFutureSupport$FutureByteArray.get(JRedisFutureSupport.java:1379)

Telnetting to the redis server shows that I can get the value of the key just fine.

info
$404
redis_version:1.2.6
arch_bits:64
multiplexing_api:epoll
uptime_in_seconds:1980869
uptime_in_days:22
connected_clients:4
connected_slaves:1
used_memory:4908261
used_memory_human:4.68M
changes_since_last_save:4310
bgsave_in_progress:0
last_save_time:1294229241
bgrewriteaof_in_progress:0
total_connections_received:45
total_commands_processed:66962953
role:master
db0:keys=221,expires=219

get jmhahqy
$6
border

Java client code:

/* initialisation code */
ConnectionSpec connectionSpec = DefaultConnectionSpec.newSpec(host, port, 0, null);
connectionSpec.isReliable(true);
redis = new JRedisAsynchClient(connectionSpec);
...
/* Usage */
Future<byte[]> result = redis.get(key);

try {
    return new String(result.get(timeout, TimeUnit.MILLISECONDS), "utf-8");
} catch (UnsupportedEncodingException e) {
    log.fatal("Unable to convert to UTF-8?");
} catch (TimeoutException e) {
    log.warn("Unable to get value back from redis under key <" + key + "> in " + timeout + "ms");
} catch (InterruptedException e) {

    /* Presumably the application is unloading? */
    log.warn("Unable to get value back from redis under key <" + key + "> " + e.getMessage());
} catch (ExecutionException e) {

    /* Something bad happened. */
    log.error("Unable to get value back from redis under key <" + key + "> " + e.getMessage(), e);
}

return null;

[jabley]

Please let me know if I can provide any further information which will help me understand why I'm seeing this problem. Currently, I'm having to restart each app server multiple times a day.

[alphazero]

Hi Jabley,

First thought was that the client and server were mismatched. But looks like a concurrent use of the connection. In general, unless you have client/server version mismatch, that sort of un-expected token in the response is generally due to 2 or more thread using a non-threadsafe connector.

1. Try using a pipeline connection instead.

2. Alternatively, see what happens if you comment out "connectionSpec.isReliable(true);"

[jabley]

Just to be explicit: the JRedisAsyncClient reference is being used as a shared resource across all request threads in my webapp, with no external mutex acting as a barrier. Is that a safe way of using it?

I'm going to switch this to using a pipeline connection instead and see what happens, as you suggest.

[jabley]

So my understanding of your suggestion is to change the JRedisFuture implementation that I am using; i.e. from
redis = new JRedisAsynchClient(connectionSpec);
to
redis = new JRedisPipeline(connectionSpec);

I haven't found any documentation showing how to use this class (wiki / Javadoc). Are you familiar with the JSR-305 or JCIP annotations regarding thread-safety? I've found them useful from a documentation perspective at least.

I'll make the above change to my code and try to test. Unfortunately, I'm not able to reliably reproduce the problem, so I'll just have to see if the current situation improves at all.

Cheers,

James

[alphazero]

Its not thread safe.

Semantics are defined: JRedsAsynch interface. Its identical to any implementation of JRedisAsynch.

Its trivial to reproduce the problem: the heartbeat is sending PING and expecing +Pong. That's a status response. Just simply do a tight loop with INCR (expecting a number response) and you'll get your exception.