Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failures in sub-type MessageProcessor constructor calls cause open loop retries #10

Open
smoyer64 opened this issue Apr 27, 2018 · 1 comment

Comments

@smoyer64
Copy link
Contributor

When the MessageHandler attempts to spawn a new MessageProcessor sub-type, if there's an error in the constructor the MessageHandler attempts to build another without rate limiting. There are two types of errors that classes implementing MessageProcessor should differentiate between:

  1. Errors due to misconfiguration shouldn't include retries as they'll never succeed. In this case the MessageHandler should itself exit with an error.
  2. Errors due to temporary loss of connectivity or other transient problems may eventually resolve. in this case, the MessageHandler should "gracefully" retry. Exponential backoff would be nice but even a timer to limit the rate at which the MessageHandler tries to instantiate a MessageProcessor would work.

An example of the open loop behavior is as follows:

20:00:56.732 [] [main] INFO  edu.psu.activemq.MessageHandler - Processed: 0
20:00:56.732 [] [main] TRACE edu.psu.activemq.MessageHandler - Checking thresholds, count = 178 threshold = 10
20:00:56.732 [] [main] TRACE edu.psu.activemq.MessageHandler - Constructing a new Message Processor
20:00:56.732 [] [main] ERROR edu.psu.activemq.MessageHandler - Error in message handler
java.lang.InstantiationException: null
        at sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at edu.psu.activemq.MessageHandler.buildNewMessageProcessor(MessageHandler.java:246)
        at edu.psu.activemq.MessageHandler.startMonitor(MessageHandler.java:198)
        at edu.psu.activemq.MessageHandler.init(MessageHandler.java:96)
        at edu.psu.activemq.MessageHandlerFactory.build(MessageHandlerFactory.java:66)
        at edu.psu.service.academic.o365.Main.start(Main.java:29)
        at edu.psu.service.academic.o365.Main.main(Main.java:14)
20:00:56.747 [] [main] INFO  edu.psu.activemq.MessageHandler - Processed: 0
20:00:56.747 [] [main] TRACE edu.psu.activemq.MessageHandler - Checking thresholds, count = 178 threshold = 10
20:00:56.747 [] [main] TRACE edu.psu.activemq.MessageHandler - Constructing a new Message Processor
20:00:56.747 [] [main] ERROR edu.psu.activemq.MessageHandler - Error in message handler
java.lang.InstantiationException: null
        at sun.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at edu.psu.activemq.MessageHandler.buildNewMessageProcessor(MessageHandler.java:246)
        at edu.psu.activemq.MessageHandler.startMonitor(MessageHandler.java:198)
        at edu.psu.activemq.MessageHandler.init(MessageHandler.java:96)
        at edu.psu.activemq.MessageHandlerFactory.build(MessageHandlerFactory.java:66)
        at edu.psu.service.academic.o365.Main.start(Main.java:29)
        at edu.psu.service.academic.o365.Main.main(Main.java:14)
@smoyer64
Copy link
Contributor Author

Shawn has proposed that:

  1. When RuntimeException is thrown, sub-type MessageProcessor should log the specifics of the error and the MessageHandler should exit .
  2. When a checked exception is thrown, the sub-type MessageProcessor should log the specifics of the temporary problem and retry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant