ROS2 and network usage

asked 2019-01-24 04:24:55 -0500

alsora gravatar image

updated 2019-02-05 09:31:35 -0500

Hi,

I'm using ROS2 Bouncy and Crystal and I noticed a very strange behavior.

I'm running several nodes on the same machine (my laptop). The nodes are standard talkers and listeners. The DDS implementation is FastRTPS. My laptop is connected to the internet through an ethernet connection and wi-fi

When I start several nodes (one immediately after the other), it's common to see the ethernet connection disconnecting. This happens more frequently when the number of nodes that I run is high (10 15 nodes).

EDIT: linked to this github issue https://github.com/ros2/rmw_fastrtps/...

EDIT:

When the ethernet gets disconnected I observe the following logs:

dmesg

e1000e: enp0s31f6 NIC Link is Down

syslog

Jan 24 11:26:15 asoragna kernel: [6536.084261] e1000e: enp0s31f6 NIC Link is Down
Jan 24 11:26:17 asoragna ntpd[3727]:Deleting interface #25 enp0s31f6,10.102.1.49#123, interface stats: received=0, sent=5, dropped=0,active_time=72 secs
Jan 24 11:26:17 asoragna ntpd[3727]: Deleting interface #26 enp0s31f6, fe80::a49b:ede8:e675:83c7%2#123, interface stats: received=0, sent=0, dropped=0, active_time=72 secs
Jan 24 11:26:19 asoragna NetworkManager[785]: <info>  [1548329179.3922] device (enp0s31f6): link disconnected (calling deferred action)
Jan 24 11:26:19 asoragna NetworkManager[785]: <info>  [1548329179.3928] device (enp0s31f6): state change: activated -> unavailable (reason 'carrier-changed') [100 20 40]
Jan 24 11:26:19 asoragna NetworkManager[785]: <info>  [1548329179.4094] dhcp4 (enp0s31f6): canceled DHCP transaction, DHCP client pid 21312
Jan 24 11:26:19 asoragna NetworkManager[785]: <info>  [1548329179.4095] dhcp4 (enp0s31f6): state changed bound -> done
Jan 24 11:26:19 asoragna avahi-daemon[772]: Withdrawing address record for 10.102.1.49 on enp0s31f6.
Jan 24 11:26:19 asoragna avahi-daemon[772]: Leaving mDNS multicast group on interface enp0s31f6.IPv4 with address 10.102.1.49.
Jan 24 11:26:19 asoragna avahi-daemon[772]: Interface enp0s31f6.IPv4 no longer relevant for mDNS.
Jan 24 11:26:19 asoragna NetworkManager[785]: <info>  [1548329179.4164] manager: NetworkManager state is now CONNECTED_LOCAL
Jan 24 11:26:19 asoragna avahi-daemon[772]: Withdrawing address record for fe80::a49b:ede8:e675:83c7 on enp0s31f6.
Jan 24 11:26:19 asoragna avahi-daemon[772]: Leaving mDNS multicast group on interface enp0s31f6.IPv6 with address fe80::a49b:ede8:e675:83c7.
Jan 24 11:26:19 asoragna avahi-daemon[772]: Interface enp0s31f6.IPv6 no longer relevant for mDNS.

Do you have any hint about what could be causing this?

edit retag flag offensive close merge delete

Comments

it's common to see the ethernet connection disconnecting.

is it really disconnecting, or can you just not get any other traffic to be passed through the connection? Those are different things.

gvdhoorn gravatar image gvdhoorn  ( 2019-01-24 05:12:41 -0500 )edit

It's really disconnecting, I get the popup message from Ubuntu showing it.

alsora gravatar image alsora  ( 2019-01-24 05:15:13 -0500 )edit

Does dmesg show something related at that time? syslog lines?

gvdhoorn gravatar image gvdhoorn  ( 2019-01-24 05:16:08 -0500 )edit

Updated with dmesg output and partial syslog

alsora gravatar image alsora  ( 2019-01-24 05:37:18 -0500 )edit

What sort of switch / router do you have this connected to?

Simple(r) consumer routers can crash when they are bombarded with too much traffic.

Not saying this is the cause, but something to look at.

gvdhoorn gravatar image gvdhoorn  ( 2019-01-24 06:41:36 -0500 )edit

My laptop is connected to the internet through an ethernet connection and wi-fi

This is a bad idea and causes a lot of problems. Use one link only and your system will run stable. Desktops are not prepared as routers do. 2 default routes via 2 interfaces will cause a lot of pain.

ChriMo gravatar image ChriMo  ( 2019-01-24 11:08:06 -0500 )edit

BTW: how handles DDS dual homed hosts ???

ChriMo gravatar image ChriMo  ( 2019-01-24 11:11:22 -0500 )edit

As long as one interface does not have a default route things should be fine. And even then, if the metric is different for the two routes, one will almost never be used.

It doesn't make much sense to me such a setup, that I agree with.

gvdhoorn gravatar image gvdhoorn  ( 2019-01-24 11:44:43 -0500 )edit