ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question
2

Dependency management in ROS2: CMakeLists.txt, package.xml, colcon build, make ...?

asked 2020-08-26 09:23:23 -0500

eloquent-fox gravatar image

updated 2020-08-28 03:05:34 -0500

I'm currently trying to understand the build system used by ROS2 and one thing I can't wrap my head around is the dependency management. Following this tutorial, dependencies are added using <depend> tags in package.xml. But for building the code, cmake's find_package() is used? Why do I need both? And what benefit is colcon build adding here, can't I just build the package using make?

Second, what's the prefered way to install dependencies? Take for example this package. According to its package.xml file it requires - among others - the package camera_info_manager. How do I install it? Does ROS2 ship with a package manager that allows installation of packages from remote source (like npm install or vcpkg install)? Can rosdep do the job or do I have to run sudo apt-get install ros-eloquent-camera-info-manager and cross my fingers that all packages I need are in the dpkg source?

edit retag flag offensive close merge delete

Comments

1

Why do I need both?

Related I believe (although for ROS 1, but the same/similar infrastructure is used): #q217475 and #q215059.

gvdhoorn gravatar image gvdhoorn  ( 2020-08-26 11:35:12 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
3

answered 2020-08-26 11:10:01 -0500

Dirk Thomas gravatar image

dependencies are added using <depend> tags in package.xml. But for building the code, cmake's find_package() is used?

In order to be able to automate packaging as well as determine inter-package dependencies those need to be declared in a machine readable format. That is why they are present in the manifest files package.xml.

In CMake you need to find other packages. If the names of your build dependency names align perfectly with the names of the CMake config files you might be able to use convenience functions like ament_auto_find_build_dependencies. But in reality there are often mismatches between rosdep key names (the ones in the package.xml file) and the CMake config name. Therefore it is very common for them to be reiterated in CMake.

And what benefit is colcon build adding here, can't I just build the package using make?

Any ROS package (which in ROS 2 could any build system like CMake, Python setuptools, etc.) can be build using its native build tool. So yes, you can invoke just cmake && make && make install on a ROS package using CMake (even if it uses ament_cmake or in ROS 1 catkin).

The "problem" with this approach is scalability. First, if you want to build multiple packages you need to manually determine in which order you need to build them. Second, if you want to build more than one package it will require a lot of manual labor. Pretty much all colcon is doing for you is figuring out the dependency graph and invoke the necessary commands to build each package based on the build system it uses (while also leveraging parallelism where possible to speed up the process).

According to its package.xml file it requires - among others - the package camera_info_manager. How do I install it? Does ROS2 ship with a package manager that allows installation of package from remote source?

If there are binary packages for the platform you are interested in (e.g. Ubuntu) you can install those. E.g. for Ubuntu we create debs which you can install using apt. For Windows Microsoft builds chocolatey packages which you can install with choco. So ROS uses existing package managers rather than inviting its own.

Beside that you can always build dependencies from source if either binary packages aren't available on your platform or you want a different version. When adding one dependency to your workspace it might require additional recursive dependencies. rosinstall_generator is a tool which helps you to get the source of all recursive dependencies so that you can build them from source.

edit flag offensive delete link more

Comments

Maybe offtopic, but I am curious.

So ROS uses existing package managers rather than inviting its own.

With rosinstall_generator, vcstool, colcon and rosdep one can build its own package manager for ROS2. A primitive package manager for ROS2 could look something like this:
https://gist.github.com/lukicdarkoo/d...

Is there is any specific reason it is not done already? Of course, there is apt, but with ROS' package manager, one could install packages on unsupported operating systems.

lukicdarkoo gravatar image lukicdarkoo  ( 2020-08-27 04:56:55 -0500 )edit
1

There are many many corner cases in package managers. How do you recover from errors? How do you check versions and make sure it's reproducible on two different machines? And that requires you to build from source for every package on every deployment which can take a long time for a large installation. And how do you upgrade a specific package?

There are a lot of package management tools out there and they have evolved and grown. We have support for apt, gentoo, openembedded, chocolaty, conda, snaps, docker images, and soon rpms. And support building from source on almost any platform. We do this by having the low level meta information available and tools that can leverage that and build on each other. A package manager is not something that's specific to robotics, and consequently we shouldn't reinvent the wheel, but learn from the many decades of ...(more)

tfoote gravatar image tfoote  ( 2020-08-27 05:12:08 -0500 )edit

I am not arguing that ROS' package manager supposed to replace the apt and similar but to complement them. There are many packages that are not published to apt, or there is no specific package version available, or the package is available for a new ROS distro even though it is compatible.

And how do you upgrade a specific package?

Use vcstool to pull a new code and rebuild it with colcon (maybe also re-run rosinstall_generator and rosdep). Or, delete the package and install it again.

How do you check versions

rosinstall_generator generates .repo with the branch names, maybe one could specify hashes instead.

How do you recover from errors?

But I see your point. The error recovery may be the hardest to implement. It is very hard to guarantee the packages can be compiled smoothly on different operating systems.

lukicdarkoo gravatar image lukicdarkoo  ( 2020-08-27 06:15:43 -0500 )edit

The build-pkgs-from-source helper is a nice idea @lukicdarkoo, but tbh, I'd rather we avoid having people build packages from source as much as possible. Depending on rosdep to install dependencies is possible, but it's not sufficiently robust, as many pkgs don't state their dependencies properly. In addition, too many users have development environments which are not clean or sane, leading to all sorts of problems.

Installing pkgs using apt is much more of a hardened process. As @tfoote mentions, platform pkg managers have decades of development behind them, which especially covers all sorts of corner cases and tries to maintain transactional application of updates (at the package level, not across perhaps). We shouldn't want to replicate all of that.

gvdhoorn gravatar image gvdhoorn  ( 2020-08-28 05:41:04 -0500 )edit

Question Tools

3 followers

Stats

Asked: 2020-08-26 09:23:23 -0500

Seen: 3,560 times

Last updated: Aug 28 '20