Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to compute when histogram overflowed #104

Open
tomushu opened this issue Apr 6, 2022 · 0 comments
Open

Unable to compute when histogram overflowed #104

tomushu opened this issue Apr 6, 2022 · 0 comments

Comments

@tomushu
Copy link

tomushu commented Apr 6, 2022

Hi,
I'm using cassandra-exporter-standalone-0.9.10.jar with cassandra 3.11.10 and suddenly the exporter for some of the nodes in one DC and two different clusters, started crashing with the errors below.
What is strange is, using the same jar from a different machine and connecting to one of the troubled cassandra nodes, the exporter doesn't crash.
16:52:47.688 [main] INFO com.datastax.driver.core.GuavaCompatibility - Detected Guava < 19 in the classpath, using legacy compatibility layer
16:52:47.827 [main] INFO com.datastax.driver.core.ClockFactory - Using native clock to generate timestamps.
16:52:47.910 [main] INFO com.datastax.driver.core.NettyUtil - Found Netty's native epoll transport in the classpath, using it
16:52:48.374 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.10.1:9042 added
16:52:48.375 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.10.2:9042 added
16:52:48.375 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.11.1:9042 added
16:52:48.375 [main] INFO com.datastax.driver.core.Cluster - New Cassandra host /192.168.11.2:9042 added
16:52:48.569 [main] INFO com.zegelin.cassandra.exporter.netty.Server - cassandra-exporter server has started. Listening on http://0:0:0:0:0:0:0:0:9500.
16:52:59.996 [prometheus-netty-pool-0] WARN io.netty.channel.DefaultChannelPipeline - An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
java.lang.IllegalStateException: Unable to compute when histogram overflowed
at org.apache.cassandra.metrics.DecayingEstimatedHistogramReservoir$EstimatedHistogramReservoirSnapshot.getValue(DecayingEstimatedHistogramReservoir.java:376)
at com.codahale.metrics.Snapshot.getMedian(Snapshot.java:38)
at org.apache.cassandra.metrics.CassandraMetricsRegistry$JmxTimer.get50thPercentile(CassandraMetricsRegistry.java:505)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1445)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:639)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant