Hi,

Yesterday, one of our test robots stopped working. I wasn't onsite, so I only got the logs.

It seems that it was the motor controler node that crashed, but the logs are quitte strange, and seem to be to point to the communication layer rather than to my python2 node :

[rosout][WARNING] 2021-02-08 11:58:51,476: running
[rosout][WARNING] 2021-02-08 11:58:51,576: running
[rosout][WARNING] 2021-02-08 11:58:51,676: running
[rosout][WARNING] 2021-02-08 11:58:51,776: running
[rosout][WARNING] 2021-02-08 11:58:51,876: running
[rosout][WARNING] 2021-02-08 11:58:51,976: running
[rosout][WARNING] 2021-02-08 11:58:52,076: running
[rosout][WARNING] 2021-02-08 11:58:52,176: running
[rosout][WARNING] 2021-02-08 11:58:52,276: running
[rosout][WARNING] 2021-02-08 11:58:52,376: running
[rospy.internal][ERROR] 2021-02-08 11:58:52,387: Unable to initiate TCP/IP socket to xavier0-desktop:33351 (http://xavier0-desktop:35049/): Traceback (most recent call last):
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 562, in connect
    self.read_header()
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 657, in read_header
    self._validate_header(read_ros_handshake_header(sock, self.read_buff, self.protocol.buff_size))
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 600, in _validate_header
    raise TransportInitError("remote error reported: %s"%header['error'])
TransportInitError: remote error reported: node shutting down

[rosout][WARNING] 2021-02-08 11:58:52,476: running
[rosout][WARNING] 2021-02-08 11:58:52,576: running
[rosout][WARNING] 2021-02-08 11:58:52,676: running
[rosout][WARNING] 2021-02-08 11:58:52,776: running
[rosout][WARNING] 2021-02-08 11:58:52,876: running
[rosout][WARNING] 2021-02-08 11:58:52,976: running
[rosout][WARNING] 2021-02-08 11:58:53,076: running
[rosout][WARNING] 2021-02-08 11:58:53,176: running
[rosout][WARNING] 2021-02-08 11:58:53,276: running
[rosout][WARNING] 2021-02-08 11:58:53,376: running
[rospy.internal][WARNING] 2021-02-08 11:58:53,389: Unknown error initiating TCP/IP socket to xavier0-desktop:33351 (http://xavier0-desktop:35049/): Traceback (most recent call last):
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 560, in connect
    self.socket.connect((dest_addr, dest_port))
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 111] Connection refused

[rospy.internal][INFO] 2021-02-08 11:58:53,390: topic[/cmd_vel_joystick] removing connection to http://xavier0-desktop:35049/
[rosout][WARNING] 2021-02-08 11:58:53,476: running
[rosout][WARNING] 2021-02-08 11:58:53,576: running
[rosout][WARNING] 2021-02-08 11:58:53,676: running
[rosout][WARNING] 2021-02-08 11:58:53,776: running
[rosout][WARNING] 2021-02-08 11:58:53,876: running
[rosout][WARNING] 2021-02-08 11:58:53,976: running
[rosout][WARNING] 2021-02-08 11:58:54,076: running
[rosout][WARNING] 2021-02-08 11:58:54,176: running
[rosout][WARNING] 2021-02-08 11:58:54,276: running
[rosout][WARNING] 2021-02-08 11:58:54,376: running
[rosout][WARNING] 2021-02-08 11:58:54,476: running
[rospy.internal][INFO] 2021-02-08 11:58:54,560: topic[/cmd_vel] adding connection to [http://xavier0-desktop:37239/], count 0
[rosout][WARNING] 2021-02-08 11:58:54,576: running
[rosout][WARNING] 2021-02-08 11:58:54,676: running
[rosout][WARNING] 2021-02-08 11:58:54,776: running
[rosout][WARNING] 2021-02-08 11:58:54,876: running
[rosout][WARNING] 2021-02-08 11:58:54,976: running
[rosout][WARNING] 2021-02-08 11:58:55,076: running
[rosout][WARNING] 2021-02-08 11:58:55,176: running
[rosout][WARNING] 2021-02-08 11:58:55,276: running
[rosout][WARNING] 2021-02-08 11:58:55,376: running
[rosout][WARNING] 2021-02-08 11:58:55,476: running
[rospy.core][INFO] 2021-02-08 11:58:55,574: signal_shutdown [atexit]
[rospy.internal][INFO] 2021-02-08 11:58:55,584: topic[/vx] removing connection to /record_1612780591184879318
[rospy.internal][INFO] 2021-02-08 11:58:55,585: topic[/vx] removing connection to /record_1612780591184895478
[rospy.internal][INFO] 2021-02-08 11:58:55,586: topic[/vth] removing connection to /record_1612780591184879318
[rospy.internal][INFO] 2021-02-08 11:58:55,587: topic[/vth] removing connection to /record_1612780591184895478
[rospy.internal][INFO] 2021-02-08 11:58:55,588: topic[/rosout] removing connection to /record_1612780591184895478
[rospy.internal][INFO] 2021-02-08 11:58:55,588: topic[/rosout] removing connection to /rosout
[rospy.internal][INFO] 2021-02-08 11:58:55,590: topic[/battery_motors_voltage] removing connection to /record_1612780591184879318
[rospy.internal][INFO] 2021-02-08 11:58:55,591: topic[/battery_motors_voltage] removing connection to /record_1612780591184895478
[rospy.internal][INFO] 2021-02-08 11:58:55,591: topic[/cmd_vel] removing connection to http://xavier0-desktop:37239/
[rospy.internal][INFO] 2021-02-08 11:58:55,594: topic[/capteur_proximite] removing connection to http://xavier0-desktop:46589/
[rospy.impl.masterslave][INFO] 2021-02-08 11:58:55,595: atexit

Please note that all my nodes are running on the same computer (a Jetson AGX developper kit, running ubuntu 18.04), so there is no "physical" network involved (but from my understanding, TCP/IP is still used for nodes on the same computer).

Do you have any idea where the problem can come from (it's the first time I notice this bug, and I have't changed the code recently)? Any idea how to solve it?

Thanks a lot in advance

Felix

PS : I think it is not related to the my code itself, but I'm not sure : if you think it is usefull, I can post it