Exception thrown when deserializing... buffer overrun
I have implemented custom functions for an embedded platform to have publishers and subscribers, where I am using standard sockets over tcp/ip. It works fine when the message definition is short (geometry_msgs point, pose, etc). However, when the message_definition is larger such as a sensor_msgs/Image, the TCP packet to send the response from publisher to subscriber for a topicRequest exceed the standard 1446 ETH length, it is sent in two separate packages (1446 + ~800) bytes but I get the error:
[ERROR] [1586366100.643160261]: Exception thrown when deserializing message of length [72] from [/MyNODE]: Buffer Overrun
is then imperative that the responses are not broken into packages?
Thanks for the help
EDIT: Just to clarify...the response for requestTopic from a publisher to a subscriber is sent to the PC from the embedded device. The message is exactly the same as if I send it from another PC running the publisher. They match 100%. The issue is that from the embedded device it is split in two packages which leads to the error message. And the question should be if there is an issue when an xmlrpc message is broken into several packages
TCP/IP stacks typically implement framing and reconstition/reassembly. Are you doing this manually/yourself? If that's the case, you may not have implemented it correctly.
I am not. I am using the lwIP so I just use standard read/write functions. Does it mean then that I should configure it so it does not split the buffers? That is for XMLRPC, what about for TCPROS? If I have to send 640x480 rgba8 image, that is 1228800 bytes and must be split, right?
No, you should not need to do anything. If this is caused by framing in TCP/IP. Any decent IP stack should do this properly. It would not be able to function otherwise.
I would suggest to debug where this message actually comes from.
so the error message you show is printed whenever using XML-RPC? That's even stranger, as that would not even involve
tcpros. I would really suggest to make sure basic TCP/IP is working correctly on your platform.I'm not claiming anything "in ROS" is bug free, but it has been in use for quite some time now, so I would be inclined to suggest to thoroughly check your additions, as they are new.
Could this be an endianness problem fi? Perhaps an off-by-one or more? If you've implemented
tcprosyourself, you may want to validate it against a known-good ...(more)And something I just thought of: can the network traffic between your embedded device and the ROS PC successfully be dissected using Wireshark? Wireshark has dissectors for both XML-RPC as well TCPROS. That could be very insightful.
What I meant for xmlrpc is that the issue that started this is when the publisher (embedded) replies to the subscriber the topicRequest. It cases the problem as the message_definition for the image message is large. I am confident that the implementation works as I have other nodes running on my device combined with the PC. Their message definitions are shorter (geometry_msgs/Point and Pose and a custom one with 3 uint16). About the large message, I compared it by having a publisher with same node, topic name on another PC and checking the packages with wireshark. There, the total package length is ~2400 if I remember correctly. Then, I compared that "tcp payload" with the one sent by the embedded device in two packages. Combining both packages, the lengths equal the one from the PC and the content is the same. The only difference is when I run the ...(more)
Just one more thing to clarify. I quickly checked cros, it is based on Linux and I don't have an OS on the embedded device. Also, it is single thread and if I understand correctly, it needs one thread per node. I have on the other hand an RTOS running but I cannot have unlimited new tasks, so I split publishers on one and subscribers in another. I will check on to run it anyways but I don't believe that will solve the issue.
I did not link to
crosto get you to use it and drop your own work.It is a known working implementation of the same protocol, which should allow you to compare implementations.
That is all.
Yes yes, I know and I will see how I can use it to improve what I implemented already. The issue is that at the point when the xmlrpc buffer is sent, it is always writing on a socket and my issue comes when that string is larger than 1446 that it gets split.