You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
I'm using cassandra-exporter-standalone-0.9.10.jar with cassandra 3.11.10 and suddenly the exporter for some of the nodes in one DC and two different clusters, started crashing with the errors below.
What is strange is, using the same jar from a different machine and connecting to one of the troubled cassandra nodes, the exporter doesn't crash.
16:52:47.688 [main] INFO com.datastax.driver.core.GuavaCompatibility - Detected Guava < 19 in the classpath, using legacy compatibility layer
16:52:47.827 [main] INFO com.datastax.driver.core.ClockFactory - Using native clock to generate timestamps.
16:52:47.910 [main] INFO com.datastax.driver.core.NettyUtil - Found Netty's native epoll transport in the classpath, using it
16:52:48.374 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.10.1:9042 added
16:52:48.375 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.10.2:9042 added
16:52:48.375 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.11.1:9042 added
16:52:48.375 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.11.2:9042 added
16:52:48.569 [main] INFO com.zegelin.cassandra.exporter.netty.Server - cassandra-exporter server has started. Listening on http://0:0:0:0:0:0:0:0:9500.
16:52:59.996 [prometheus-netty-pool-0] WARN io.netty.channel.DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
java.lang.IllegalStateException: Unable to compute when histogram overflowed
at org.apache.cassandra.metrics.DecayingEstimatedHistogramReservoir$EstimatedHistogramReservoirSnapshot.getValue(DecayingEstimatedHistogramReservoir.java:376)
at com.codahale.metrics.Snapshot.getMedian(Snapshot.java:38)
at org.apache.cassandra.metrics.CassandraMetricsRegistry$JmxTimer.get50thPercentile(CassandraMetricsRegistry.java:505)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1445)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:639)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
The text was updated successfully, but these errors were encountered:
Hi,
I'm using cassandra-exporter-standalone-0.9.10.jar with cassandra 3.11.10 and suddenly the exporter for some of the nodes in one DC and two different clusters, started crashing with the errors below.
What is strange is, using the same jar from a different machine and connecting to one of the troubled cassandra nodes, the exporter doesn't crash.
16:52:47.688 [main] INFO com.datastax.driver.core.GuavaCompatibility - Detected Guava < 19 in the classpath, using legacy compatibility layer
16:52:47.827 [main] INFO com.datastax.driver.core.ClockFactory - Using native clock to generate timestamps.
16:52:47.910 [main] INFO com.datastax.driver.core.NettyUtil - Found Netty's native epoll transport in the classpath, using it
16:52:48.374 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.10.1:9042 added
16:52:48.375 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.10.2:9042 added
16:52:48.375 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.11.1:9042 added
16:52:48.375 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.11.2:9042 added
16:52:48.569 [main] INFO com.zegelin.cassandra.exporter.netty.Server - cassandra-exporter server has started. Listening on http://0:0:0:0:0:0:0:0:9500.
16:52:59.996 [prometheus-netty-pool-0] WARN io.netty.channel.DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
java.lang.IllegalStateException: Unable to compute when histogram overflowed
at org.apache.cassandra.metrics.DecayingEstimatedHistogramReservoir$EstimatedHistogramReservoirSnapshot.getValue(DecayingEstimatedHistogramReservoir.java:376)
at com.codahale.metrics.Snapshot.getMedian(Snapshot.java:38)
at org.apache.cassandra.metrics.CassandraMetricsRegistry$JmxTimer.get50thPercentile(CassandraMetricsRegistry.java:505)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1445)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:639)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
The text was updated successfully, but these errors were encountered: