InvalidHeaderException #42
Comments
|
There does seem to be an intermittent problem here with this client, see #41 and others. If you don't need any specific functionality of the MQTT 5 protocol consider using the mqtt_client package, this is an API compatible client for the MQTT 3 protocol. You should just be able to substitute mqtt_client for mqtt5_client. I will look at all this as soon as I can. |
|
I would like to use at least UserProperties, in our system some work was done on them =( |
|
Yes you are correct, in that case you need the mqtt5_client. |
|
Thank you for your work. I look forward to. |
|
May have fixed this please try version 3.3.5 |
|
Yes, with the header, in my opinion, the error went away, but I got another one (I also received it earlier, but I thought that this was a consequence of a header error) Bad state: MqttByteBuffer::clear - attempt to clear a byte buffer where postion is not zero, it is 4 |
|
but no, I jumped to conclusions, the next time I started the application, I got it again =( mmqtt-client::InvalidHeaderException: The header being processed contained an invalid size byte pattern. Message size must take a most 4 bytes, and the last byte must have bit 8 set to 0. |
|
Ok thanks, the error your seeing is a different exception to the one the other issues are reporting, Ill carry on looking at this. |
|
Thank you very much |
|
This is now effectively being handles on #44 |
|
Version 3.3.6 released, please check if this fixes the problem. |
|
At first glance, the problem is fixed. I will continue to try to use your library. Thank you for your job! |
|
i got this error again =(( �[48;5;199mmqtt-client::InvalidHeaderException: The header being processed contained an invalid size byte pattern. Message size must take a most 4 bytes, and the last byte must have bit 8 set to 0. |
|
Ok Ill need a log to go any further |
|
I was finally able to catch the error with logs turned on |
|
Quickly looking shows that the first failure occurs when a partial message has been received, when we next receive data the buffer length seems incorrect - |
|
I would suggest making a custom error handling method in such a situation. If the handler successfully handled the error, for example, clear the buffer and continue working from scratch. |
|
Yes, handling the error, cleaning up and carrying on is definitely a strategy you can adopt if you don't mind some data loss, this is something to consider here however I think disconnecting may be a bit strong. I'll consider this along with any fix I find. |
|
Shutdown as an option. Otherwise, an unhandled exception is now obtained, which cannot be caught, and further operation of the service is impossible. |
|
In general one would just like to either catch the exception or trigger a reconnect. The main thing is that the work of the service does not stop. |
|
OK, I think we are getting there, the first error is this - Lets assume for now that this is a real error, i.e. a corrupt bytestream, when this is raised the incoming buffer is cleared and reset. Problem with this is that it may not be a complete set of messages, it may be a partial set, i.e. we are still waiting for some bytes to arrive for the next complete message. When these arrive in the now cleared buffer we will interpret this as a header as we know no different and incorrectly process it, this will continue until we eventually do re sync the stream, throwing exceptions as we go. So although my initial thoughts on disconnecting may be too extreme I can't see now how we can recover successfully from this without disconnecting, reconnecting and re establishing the protocol at a known point. To this end I'll update the code to do this on any exception thrown in the onData code. Any further thoughts? |
|
Disabling is the last option. Honestly, I don't know the whole mqtt specification, but somehow I did the processing of another protocol. And there, too, could come "garbage". |
|
Yes, in some cases depending on the protocol this might work, for MQTT however the header contains the length of the message, so we receive the header, decode it and when we have received at least the length bytes then we can try and decode the message. If it just so happens that the first byte of the header decodes OK(this is just flags so probably will) and it just happens that the next bytes decode into a valid header length then we will wait for this length to arrive, we don't know that the length is 'wrong' if you see what I mean, there's no checksum or anything to check the integrity of the header. I'll do some more research, someone must have come across this somewhere. |
|
Yes, I understand what you mean. I had a checksum in the protocol. Then yes, if there is no other way out, you will have to disconnect and reconnect. If your search turns up nothing. |
|
OK, I've updated the client to send a disconnect message to the broker if any irrecoverable exceptions are raised on data reception, I've not written any unit tests for this yet but it would be useful if you could test this on repo branch issue42. In theory when the client sends a disconnect message the broker should gracefully close the connection, this will cause the client to pick this up in the normal manner(onDone) and proceed as set by the user, i.e. normal disconnect callback or autoreconnect as selected. |
|
I copied the git for myself and switched to the issue42 branch, I'll see how it behaves, I'll unsubscribe with the results. |
|
It looks like the issue is still reproducible and I'm getting intermittent disconnects and reconnects (I have my own autoreconnect) |
|
And I have a question for you, how can I understand that the message came as Retain? you have an indicator that the message is Retain=True, it means the message is saved on the broker, but how can I understand if the broker sent this message from its "saved" ones or is it a "fresh" message sent by another client? |
|
From the MQTT 5 spec - So a message received as retained is the last reported state of the sensor, not necessarily the latest state of the sensor. Its difficult to tell whats going on from the logs, if you assume the client code is correct then it is indeed receiving incomplete messages sometimes. It may be worth starting to log other factors such as what time it occurs, are you on wireless/wired/gsm at the time, does the same happen when you change brokers, does the same broker on a different endppoint give different results etc. |
|
Perhaps you did not understand me. When I wrote in C # in the MQTTNet library, the Retain=true flag showed that the proker sent me a message "from previously saved", if Retain=false came, then this was an indicator that the message was "fresh" (sent at a given time by another client ). So the question is, can you somehow determine in your library that the message is "fresh"? |
|
"Its difficult to tell whats going on from the logs, if you assume the client code is correct then it is indeed receiving incomplete messages sometimes. It may be worth starting to log other factors such as what time it occurs, are you on wireless/wired/gsm at the time, does the same happen when you change brokers, does the same broker on a different endppoint give different results etc." I'm testing the app on a WiFi network, so connection issues can be ruled out. I don’t understand how this is possible, but the broker seems to be to blame, but I can’t determine the cause, I can’t find the root of the problem. If I clear all broker messages, then everything works for a while, but then the same thing starts again =( |
|
OK, this is progress, obvious things which may or may not be possible of course are to change brokers, move the same broker to a different endpoint etc. |
|
I've merged the changes applied in this branch as I believe it has been thoroughly tested by yourself. Package re published at version 3.4.0 |
|
Thank you! Update and use. I think the question can be closed |
|
Ok thanks, closing. |
Hello. When connecting to a broker, such an error occurs randomly and the application refuses to work further, can you tell me why it could be or how to get around this problem so that the application does not freeze?
InvalidHeaderException: The header being processed contained an invalid size byte pattern. Message size must take a most 4 bytes, and the last byte must have bit 8 set to 0.
MqttHeader.readFrom (package:mqtt5_client/src/messages/mqtt_header.dart:89:7)
new MqttHeader.fromByteBuffer (package:mqtt5_client/src/messages/mqtt_header.dart:19:5)
MqttByteBuffer.isMessageAvailable (package:mqtt5_client/src/utility/mqtt_byte_buffer.dart:66:29)
MqttServerConnection._onData (package:mqtt5_client/src/connectionhandling/server/mqtt_server_connection.dart:60:26)
_rootRunUnary (dart:async/zone.dart:1399:47)
_CustomZone.runUnary (dart:async/zone.dart:1300:19)
_CustomZone.runUnaryGuarded (dart:async/zone.dart:1209:7)
_BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
The text was updated successfully, but these errors were encountered: