Ask Your Question
0

How to troubleshoot: protocol synchronization is lost

asked 2018-03-08 10:14:10 -0600

knxa gravatar image

updated 2018-03-09 01:33:02 -0600

Suddenly I start seeing this error:

[ERROR] [1520525542.826204079] [/rosout]: a message of over a gigabyte was predicted in tcpros. that seems highly unlikely, so I'll assume protocol synchronization is lost.

[ERROR] [1520525542.826297310] [/rosout]: Exception thrown when deserializing message of length [0] from [/task_runner]: Buffer Overrun

[ERROR] [1520525542.826355406] [/rosout]: Exception thrown when deserializing message of length [11596] from [/task_runner]: Buffer Overrun

Details:

  • Seems to appear randomly
  • The node /task_runner is running a large smach state machine, with a large number or topic subscriptions.
  • There is only one PC involved.
  • Kinetic, Ubuntu 16.04

In the log for task_runner I found this error message at the same point in time:

[rospy.rosout][ERROR] 2018-03-09 08:28:55,914: Unable to report rosout: Set changed size during iteration
Traceback (most recent call last):
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/rosout.py", line 91, in _rosout
    _rosout_pub.publish(l)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/topics.py", line 882, in publish
    self.impl.publish(data)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/topics.py", line 1066, in publish
    serialize_message(b, self.seq, message)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/msg.py", line 152, in serialize_message
    msg.serialize(b)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosgraph_msgs/msg/_Log.py", line 153, in serialize
    for val1 in self.topics:
RuntimeError: Set changed size during iteration

So I conclude that the rosout logging breaks because I perform logging whiIe another thread is adding a topic to the same node. If so, how would I solve this in a generic way?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2018-03-09 05:24:26 -0600

knxa gravatar image

updated 2018-03-09 06:05:06 -0600

I believe this is the problem:

The combination of logging to rosout and subscribing to topics is not threadsafe. I have created a new question.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

2 followers

Stats

Asked: 2018-03-08 10:14:10 -0600

Seen: 217 times

Last updated: Mar 09 '18