Best way to "release" closed source ROS2 packages
Hello fellow robo-enthusiasts,
our closed (~private) source project uses ROS2 packages arranged into on-site GIT repositories. We have some basic automation going on (local Jenkins instance, fragile bits of ros_buildfarm stitched together ages ago). Over time, the project logically hit limitations of our current infrastructure. From top of my head:
- arranging GIT repositories using
git submodulefails in a scenario, where the same package would be brought in by more than one submodule.colconthen simply refuses to cope with this duplicity and manual "pre-build" intervention is needed. Devs hate that. Cleaning up architecture quickly leads to this problem (single shared interface repo with Actions, Services etc.) - assembling "releases" (to try on a live hardware for example) is cumbersome, partially because of things mentioned above, as well as the need to handle live source code (creating security concerns when 3rd party hardware is used, for example when doing a demo)
Having experience with different software projects and domains, my first thought was that we need packaging system to meet our needs. Then, we would just apt-get our packages by version, the same way we get ROS2 stock ones.
In reality, so far I have spent more than a week finding out how to get there. Now I kinda fear there is no easy way. So this is my cry for help and guidance - what is the best path to take? What am I missing?
Specifically, we only need to build ROS 2 packages for foxy, ubuntu:focal targets. Because I tend to stumble into ROS 1 specific docs, not applicable for ROS 2 without detailed knowledge.
What I tried
PATH A: manual APT / DEB package creation.
According to a note found in ROS2 documentation, I tried bloom-generate rosdebian => fakeroot debian/rules binary route. After struggling with rosdep configuration with packages depending on another private packages, I managed to get to the fakeroot step, but it fails for some of our packages in a weird way. I managed some progress, but it feels painful. Specifically, I see errors about pthread library missing (while CMake colcon build ends up being fine)
PATH B: creating full local ros buildfarm clone
Because we cannot easily use the "public" open source buildfarm, I thought I'll try to recreate private one. So far, this has also been a rough ride, while trying to follow existing documentation. Because I must not tinker with the existing Jenkins instance to not disrupt rest of the team, I created my own locally in a Docker container. However, the ros_buildfarm scripting gives me all kinds of weird responses, leading me to believe I won't be able to force foxy releases this way. My other gut feeling tells me the scripting probably expects older versions of Jenkins than I am providing.
Anyone can help me to choose the right path? Then I can probably ask specific questions about the errors I see, but maybe I just plainly chose a bad way altogether.
Not an answer, but:
IIRC, you cannot deploy a
ros_buildfarminstance using Docker. It "needs" a bare metal OS / VM ("needs" in quotes, as it's obviously possible to get it to run inside a container, but if you're just starting out, that may not be the most efficient way to get it going).