Ask Your Question
2

ros2 docker multiple hosts

asked 2020-07-31 15:36:34 -0500

dawonn_haval gravatar image

updated 2020-08-16 09:08:41 -0500

How can I get ROS2 to run in docker across multiple hosts reliably? I have two hosts, on an ethernet network:

  • Bridge networks do not support multicast
  • Overlay networks do not support multicast
  • macvlan networks cannot be accessed from the host machine (Workaround below)
  • Host networks partially work, but I only see a random subset of the nodes on the network. (Workaround below)
  • Weave network performance is very poor. (A simple topic that typically runs natively at 270Hz only manages to get up to 20Hz)
edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
1

answered 2020-08-12 16:55:53 -0500

dawonn_haval gravatar image

updated 2020-08-16 09:08:34 -0500

There appear to be two potential workarounds:

HOST:

If you want to use host networking, you also much share the pid space so that each DDS node can generate a unique GUID. ( All thanks go to @ruffsl, more detail here )

docker run --rm -d --net=host --pid=host <IMAGE>

MACVLAN:

This workaround seems to work more often than not but sometimes the container needs to be restarted before it will see all nodes on the network.

docker network create -d macvlan \
    --subnet=192.168.1.0/24 \
    --gateway=192.168.1.1 \
    --ip-range=192.168.1.64/27 \
    --attachable \
    -o parent=enp8s0 \
    macnet

sudo ip link add macnet-shim link eth0 type macvlan  mode bridge
sudo ip addr add 192.168.1.64/27 dev macnet-shim
sudo ip link set macnet-shim up

docker run --rm -d --net=macnet <IMAGE>
edit flag offensive delete link more
0

answered 2020-08-12 16:33:06 -0500

ruffsl gravatar image

It looks like multicast support has been a long open issue for docker:

https://github.com/moby/libnetwork/is...
https://stackoverflow.com/questions/5...

As suggested in one of the stackoverflow answers, If you need multicast support, you could use the host network driver, as ROS2/DDS is capable of running multiple nodes/participants per host/container and could share the network interface. If the docker host's aren't on the same subnet then that would be a separate issue for DDS discovery and NAT translation over a LANs that still supports multicast. In any case, I'd also recommend enabling SROS2 if the hosts are operating over an unsecure network.

Related:
https://answers.ros.org/question/3584...

edit flag offensive delete link more

Comments

1

The hosts are on the same subnet, and I initially started with 'host' networking but it seems to be very unreliable. Each node only sees a random subset of the nodes on the network, as if some of the multicast messages are not being routed correctly. Once a pub/sub connection is established, the connection is reliable however.

Also interesting, the ros2 multicast sent|receive tool almost always works with host networking.

As a brief status update; I am having some limited success with a macvlan network configuration. I was able to create a bridge between the docker macvlan subnet and the host machine. It's not super reliable yet though, I am often unable to establish a connection to some topics even though ros2 topic list & ros2 nodes list look good. Usually I am able to restart a container a couple of times to get things connected.

dawonn_haval gravatar image dawonn_haval  ( 2020-08-12 16:45:23 -0500 )edit
1

It may not be the case here, but some DDS venders may fall back to shared memory transport if they think another participant is from the same host, or colliding GUID, thus complicating the use of the host network driver for such scenarios. See this question/answer for more details: https://answers.ros.org/question/2968...

ruffsl gravatar image ruffsl  ( 2020-08-13 19:12:53 -0500 )edit

@dawonn_haval were you able to resolve this? I'm currently observing an identical problem in terms of random subsets of visible nodes with 'host networking'. Many thanks!

aaron98 gravatar image aaron98  ( 2020-12-02 16:12:00 -0500 )edit
1

Kinda sorta, yes: https://answers.ros.org/question/3584...

After assigning each docker container to use --net=host --pid=host I can now establish connections reliably. Both are necessary for the RMW to generate a unique GUID.

dawonn_haval gravatar image dawonn_haval  ( 2020-12-03 08:49:21 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

3 followers

Stats

Asked: 2020-07-31 15:36:34 -0500

Seen: 1,049 times

Last updated: Aug 16 '20