Robotics StackExchange | Archived questions

Find and clone all package dependencies before source build

I've been fumbling my way through trying to get a source build of humble up and running on Ubuntu 22.10 (which I'm stuck on for other reasons) and it's been exceedingly painful.

Installing the ROS core was relatively straight forward, but now I'm trying to get navigation up and running. I've found myself in an excruciatingly tedious cycle of:

I feel like I must be missing some tool to just find and clone the dependencies that aren't already available.

Asked by jleibs on 2023-02-21 11:27:10 UTC

Comments

Usually you would have this tool called vcstool to import/clone any dependency repositories. You should also think about the Docker Container, perhaps it can be easier to work it.

Asked by ljaniec on 2023-02-23 08:25:31 UTC

I would second the suggestion to look into using containers. That would be the go-to solution for running ROS on unsupported OS.

Asked by gvdhoorn on 2023-02-23 11:06:12 UTC

Yeah, I briefly looked at docker but immediately ran into issues with gazebo and gpu access from inside docker.

Sadly I also followed the recommended instructions from the gazebo documentation itself to install ignition, only to later discover that almost all of the existing ROS packages still use gazebo classic and almost no example launch scripts that I have found play nicely with the ignition-based gazebo.

Quite frankly this whole process has been pretty frustrating / unforgiving. The usage of the word support in the claim: "We support building ROS 2 from source on the following platforms: Ubuntu Linux" is quite misleading.

For now I've simply given up and won't be trying again until I can procure an Ubuntu 22.04 laptop and just use the debs in a properly supported manner.

Asked by jleibs on 2023-02-23 12:20:30 UTC

The usage of the word support in the claim: "We support building ROS 2 from source on the following platforms: Ubuntu Linux" is quite misleading.

Where did you read this? All I can find in the various build-from-source tutorials are warnings that building from source typically works on the supported Ubuntu versions, but other Linux distributions and versions not officially supported will be (much) more of a challenge.

Yeah, I briefly looked at docker but immediately ran into issues with gazebo and gpu access from inside docker.

Personally I use Singularity, not Docker when I want to containerise UI applications.

I've not run into issues with RViz nor Gazebo, and GPU acceleration works pretty well (best with nvidia).

Asked by gvdhoorn on 2023-02-23 13:02:02 UTC

Where did you read this?

https://docs.ros.org/en/humble/Installation.html#building-from-source

Asked by jleibs on 2023-02-23 13:31:50 UTC

On that page, if you follow the Ubuntu Linux link, the first thing it lists are the supported OS versions:

image description

22.10 is not one of those.

Asked by gvdhoorn on 2023-02-23 13:35:19 UTC

I don't see the relevance of 22.04 vs 22.10 here other than the fact that I'm trying to do a source build because I'm on 22.10. If I were on a "supported" platform I would just use the debs and this wouldn't be an issue, so this seems a bit circular.

That said, if for some reason, I felt compelled to do a from-source build on 22.04, I don't see how I would be having any easier of a time there. The problems that I'm running into aren't related to things like system-dependencies or toolchain versions. The problem I've been running into are generally poor support within the docs / ecosystem for managing source-only builds. (Though the rosinstall invocation you provided helps quite a bit with that, so thank you.)

Either way, your point is heard loud and clear: don't bother with source builds. Just use 22.04 (native or docker) if you want to use ROS.

Asked by jleibs on 2023-02-23 14:15:37 UTC

That wasn't my point at all, but if that's what you got from all of this, that's ok.

Asked by gvdhoorn on 2023-02-23 14:28:50 UTC

Answers

colcon is having an issue in finding behaviortree_cpp_v3 package, needless to say.

I found it in a tag https://github.com/BehaviorTree/BehaviorTree.CPP/blob/v3.8/package.xml, although on the default branch on this repo the package name no longer contains "_v3". No rationale found for this to be the right reference yet though.

Asked by 130s on 2023-02-23 07:43:38 UTC

Comments

The main issue of OP seems to be how to build a set of packages (plus dependencies) from source on an (officially) unsupported OS.

The "ROS core" was probably "relatively straight forward" because there was a convenient .repos file available.

For other packages, that doesn't exist.

The behaviortree_cpp_v3 error is just an example of a missing dependency I believe.

Perhaps rosinstall_generator could be used? The navigation2 build instructions document a few options. I believe "for released distributions" option could be used, together with the ROS 2 install OP already built, and a suitable rosinstall_generator command to create a .repos file for just the nav2 packages (and their dependencies).


Edit:

Except ROS_PACKAGE_PATH does not exist as a concept in ros2 so seems like I'll have to jump through some more hoops to filter out the packages that are already installed.

I believe the following starts a build of Nav2 for Humble, including dependencies, but excluding all packages already installed in /path/to/your/humble_install/share:

$ rosinstall_generator \
    navigation2 \
    --rosdistro=humble \
    --deps \
    --tar \
    --format=repos \
    --exclude-path=/path/to/your/humble_install/share \
  > $HOME/nav2_pkgs.repos
$ mkdir -p $HOME/nav2_ws/src
$ vcs import --input=$HOME/nav2_pkgs.repos $HOME/nav2_ws/src
$ rosdep update --rosdistro=humble
$ rosdep install --os=ubuntu:jammy --from-paths $HOME/nav2_ws/src -i -y --rosdistro=humble
$ colcon build --mixin release --mixin ninja

The --exclude-path=/path/to/your/humble_install/share part is basically the replacement for RPP in your original attempt.

rosdep is used to install system dependencies (so non-ROS package dependencies). The --os=ubuntu:jammy is necessary on 22.10 as otherwise rosdep won't 'recognise' the OS. Note this is not a guarantee this will work, as dependencies may have different names on Ubuntu 22.10 compared to 22.04.

I've tested this in a ros:humble Docker container and it starts a build and does not complain. I'll update this when it completes, or errors out.

I have not tested whether building this results in a functional Nav2 setup.

Asked by gvdhoorn on 2023-02-23 11:05:27 UTC

Comments

Thanks for the pointer. rosinstall_generator does look quite close to what I was imagining... trying to find documentation for how to actually use it in this case led me to: https://github.com/ros2/ros2_documentation/issues/1470

Sadly, the only documentation I can find still seems to be ros1 oriented. I was hoping to do something like: rosinstall_generator nav2_bringup --deps --rosdistro humble --exclude RPP Except ROS_PACKAGE_PATH does not exist as a concept in ros2 so seems like I'll have to jump through some more hoops to filter out the packages that are already installed.

I'm shocked that this is missing though -- I guess most folks just aren't working with source packages if they aren't actively developing the package.

Asked by jleibs on 2023-02-23 12:06:23 UTC

I'm shocked that this is missing though [..]

what is missing? rosinstall_generator is fully usable with ROS 2. You just can't use ROS 1-only features.

Note: I'm not saying it's all perfect and unicorns and rainbows. Far from it. But the base functionality is there.

It's certainly under-documented.

And technically, rosinstall_generator isn't even a "ROS utility". It just happens to be used for ROS workspace management a lot, but the metadata it consumes is what makes it "ROS".

Except ROS_PACKAGE_PATH does not exist as a concept in ros2 so seems like I'll have to jump through some more hoops to filter out the packages that are already installed.

I haven't checked, but --exclude RPP isn't the only way to exclude packages you already have. IIRC, there are also args which take a path fi.

Asked by gvdhoorn on 2023-02-23 12:15:34 UTC

It's certainly under-documented.

Sure, more specifically what I meant is that I'm shocked that what I would have expected was a common workflow is so undocumented. My memory (which is certainly fallible) is that a from-source workflow used to be much better-supported on ROS1.

But maybe most of the users working with ROS are just happily running 22.04 and using debs for their dependency management. If nobody else is asking the questions then there's no reason to worry about the lack of documentation.

Asked by jleibs on 2023-02-23 12:50:11 UTC

Humble officially only supports 22.04.

I'm not trying to gold plate things, but building something as complex as ROS 2 from source on an unsupported OS is just never going to be easy.

My memory (which is certainly fallible) is that a from-source workflow used to be much better-supported on ROS1.

I agree with you the ROS 1 workflow was better documented and more common, but in principle, the same workflow can be used with ROS 2.

If you're still interested, see my edited answer.

Asked by gvdhoorn on 2023-02-23 12:53:03 UTC

This answer works great.

This would be a great addition to the ROS2 documentation. Right now there is almost no mention of rosinstall_generator outside of a passing mention in the rqt build instructions: https://docs.ros.org/en/humble/search.html?q=rosinstall_generator&check_keywords=yes&area=default

Asked by jleibs on 2023-02-23 13:35:15 UTC

This would be a great addition to the ROS2 documentation.

I can only suggest to comment on ros2/ros2_documentation#1470, which you already found, and/or write a more extensive version of what I posted here and submit it as a PR against ros2/ros2_documentation.

As it was with ROS 1 (at least after the WillowGarage times), ROS 2 is a community project.

Asked by gvdhoorn on 2023-02-23 14:30:52 UTC