ROS2 messages stop 90s after network drops

asked 2019-06-25 17:12:46 -0500

clyde
1247 ●27 ●54 ●51

I'm running ROS2 Dashing with Fast-RTPS on a single-board computer running Ubuntu 18.04 server. It's mounted inside a waterproof enclosure. When the enclosure is in the air I can talk to it via wifi, and when it goes underwater the wifi connection drops. All ROS message traffic stops ~90 seconds after the network connection is lost.

I can easily reproduce this on my desktop (Ubuntu 18.04 desktop):

in the 1st terminal run ros2 run demo_nodes_cpp talker
in the 2nd terminal run ros2 run demo_nodes_cpp listener -- starts reporting
unplug the Ethernet cable, after ~90 seconds the listener stops reporting
plug in the Ethernet cable, after a few seconds the listener starts back up

I've tried DHCP and static IP & DNS, but the behavior is the same. I don't see anything surprising in journalctl -f.

Interestingly, if I start sending messages while the network is down, it appears to ignore the state of the network:

unplug the Ethernet cable
in the 1st terminal run ros2 run demo_nodes_cpp talker
in the 2nd terminal run ros2 run demo_nodes_cpp listener -- starts reporting
plug in the Ethernet cable, no change
unplug the Ethernet cable, no change

Any ideas?

Thanks.

edit retag flag offensive close merge delete

Comments

Related: #q325872.

gvdhoorn ( 2019-06-26 03:01:47 -0500 )edit

add a comment

Comments

Thx... I'm investigating this now

clyde ( 2019-06-26 09:39:19 -0500 )edit

You were right, loopback was not configured for multicast. I configured it using route add -net 224.0.0.0 netmask 240.0.0.0 dev lo and ifconfig lo multicast. This works great if I boot up with no network, enable multicast on loopback, then run the talker and listener. I can plug / unplug the Ethernet and it continues to work.

But if I boot up with the Ethernet plugged in, enable multicast on loopback, then start the talker and listener, it still stops 90 seconds after I unplug the Ethernet. The only way to restart communications is to kill all ROS nodes then restart them in the network-starved situation.

clyde ( 2019-06-27 11:58:05 -0500 )edit

This might be a bug. Would you mind opening an issue on https://github.com/ros2/rmw_fastrtps/... ?