ROS2 Docker development practice (volume mounting)
Hi everyone, I am now working on a project, to be specified, autoware and multiple sensors. I would like to come up with a Docker environment so that out team could share the same environment during development, and also for easy deployment as well. However when I constructed the Dockerfile, I faced below questions: (assuming docker run --rm is used)
- Should I
rosdep-install && colcon build
inside the Dockerfile? If I do so, those source code would be inside Docker, any changes/config would be disappeared after exiting the docker container. This is not ideal as we still under development, the code will be changed quite often. However if I not doing so, this come to question 2: - If I git clone the source code and mount volume to the container, then run
rosdep-install && colcon build
inside docker container, of course those changes can be saved in host, however, will all the dependence lost when I exit the docker? Whererosdep-install
actaully install?
What is the best practice for building workspace in Docker? Should I use --rm
flag for minimizing the chance installed extra packages/dependence that not included in Dockerfile (redeploy issue)?
I hope the concern behind can be delivered clearly. I look forward to here from you guys, I dont have much experience working ROS with Docker.
Asked by ThomasL624 on 2022-10-19 21:14:45 UTC
Answers
I can recommend using remote development in vscode using this template: https://github.com/athackst/vscode_ros2_workspace
This is only for development, but I think it can be easily converted to a deployable docker container. This dev environment will be mounting onto your work directories root folder and thereby the development is "local" while you are building inside the docker container.
The only issue I have encountered so far is having to bind hardware(USB ports, etc.) to the docker container before running the dev environment. But in general, it works pretty nicely.
This might not be a direct answer to 1. or 2. but I think it is a better way of developing as you do not have to rebuild your container every time you want to recompile.
Asked by grontved on 2022-10-20 04:16:14 UTC
Comments
I just tested method2, the problem is that when I exit the conatiner, all the dependency downloaded from rosdep install
is gone, so the next time I docker run this image, the ROS2 launch cannot be launched.
What if I COPY
the packages in the Dockerfile, and only run the rosdep install
? Actually where these dependency be installed at? https://docs.ros.org/en/humble/Tutorials/Intermediate/Rosdep.html The document hasnt stated where will be...
Then colcon build
in a running container, as colcon build
should only build inside the workspace. I dont know will this work or not...
Asked by ThomasL624 on 2022-10-20 05:23:08 UTC
We have a docker environment working well for us. As you suggested, we mount volumes from the host into docker with the -v option. Typically we have a workspace folder which has all our ros2 workspaces and we map a workspace folder from the host into the docker. The rosdep output is written to your home $HOME/.ros folder. So we create a .ros folder on the host and map it to $HOME/.ros folder. This way the rosdep output is not lost when we exit the docker. There are probably other approaches as well but this is what we do and it works for us. Hope this helps.
Asked by thejeeb on 2022-10-20 09:57:14 UTC
Comments
In a normal Docker setup without adding a new user, $HOME would be /root. So, the path would be /root/.ros folder. I understand your approach, will try it out. As not many talk about this, if this approach works, then I will take it.
Asked by ThomasL624 on 2022-10-20 10:05:13 UTC
In our case, we do add a new user. But you can try the root user as well.
Asked by thejeeb on 2022-10-20 11:22:13 UTC
To clarify, do i have to rebuild my ros image? As the underlay ros2 is already build inside the base image, so if I mount a fresh .ros volume into container, will the underlay ros2 basic package be destroyed?
Asked by ThomasL624 on 2022-10-20 14:51:40 UTC
rosdep
installs system dependencies and ROS package dependencies (technically, it doesn't install anything itself. See #q215059).
Those don't go into ~/.ros
. The only data rosdep
writes to ~/.ros
is the cache of the database it uses to resolve keys.
I'm not sure how mounting .ros
will help here.
Asked by gvdhoorn on 2022-10-21 03:16:26 UTC
yes i just checked today, copy out .ros seem doesn’t help at all.
Asked by ThomasL624 on 2022-10-21 05:42:20 UTC
@ThomasL624 I have found it best to separate Dockerized development from Dockerized deployment. It is usually best to have a minimal image for deployment that contains just the install space (binaries, libraries, configuration, etc.) required to execute your application without any source code or git files.
I have used a customized version of https://github.com/athackst/vscode_ros2_workspace for Dockerized development with success, but the linked template does not include functionality for resolving workspace dependencies using rosdep
. The solution I have used to address this limitation is to include resolving the rosdep
s in the Dockerfile (used by devcontainer.json) as one of the last layers. The user is required to manually rebuild the vscode container periodically. If the commands utilized in the Dockerfile are efficient rebuilding the container will utilize Docker cache when there are no changes to the installed rosdep
dependencies.
https://hub.docker.com/r/arm64v8/ros/ provides an example of using a muli-stage Docker build for caching the rosdeps, the key content is:
# copy manifests for caching
WORKDIR /opt
RUN mkdir -p /tmp/opt && \
find ./ -name "package.xml" | \
xargs cp --parents -t /tmp/opt && \
find ./ -name "COLCON_IGNORE" | \
xargs cp --parents -t /tmp/opt || true
That copies the just the non-ignored package.xml files (including the folder structure) in a temporary directory that can be copied by a different stage to run rosdep
on. Then the dependencies are installed:
COPY --from=cacher /tmp/$OVERLAY_WS/src ./src
RUN . /opt/ros/$ROS_DISTRO/setup.sh && \
apt-get update && rosdep install -y \
--from-paths \
src/ros2/demos/demo_nodes_cpp \
src/ros2/demos/demo_nodes_py \
--ignore-src \
&& rm -rf /var/lib/apt/lists/*
Obviously, you should tailor the --from-paths
argument to either specify your entire src folder or specific packages.
Once the developer rebuilds the container based on a specific branch of the workspace repository all the necessary dependencies will be installed.
The same link can be used as a template to create a separate Docker image that is just used for deployment.
Asked by danambrosio on 2023-06-06 11:21:21 UTC
Comments