How to get the build order for Bloom automation of a multi-package project?
Guys,
When using catkin build
or colcon build
, I can see that it build all the packages in the correct order.
However, I actually need to build and package everything using the Debian pipeline, with Bloom but, it doesn't support multiple packages.
When with Bloom, I need to manually figure out the build order, then, build a package, upload to an APT repo, and then, figure out the next package, and so on...
I can see that colcon graph
shows the dependency tree! The rqt_dep
is also helping to visualize the dependencies, which is cool!
The catkin list --deps
is also interesting!
Thing is, how to automate the Bloom build process of a huge project (like a Git repo that contains multiple package.xml
)?
The catkin|colcon build
is nice but I actually don't need it. Because my goal is to obtain the Debian packages...
Should I investigate, and maybe build my own, ROS build farm?
Or maybe I'll need to develop something (in Python) that uses the catkin|colcon
APIs and glue together the Workspace and Bloom to generate all the Debian packages in order (like catkin|colcon
already does)?
EDIT: Guys, anyone?! :-P
Thanks!
Asked by Thiago Martins on 2021-04-05 18:59:00 UTC
Answers
Once you have run bloom and upload the sourcedebs, this is entirely a pure debian packaging pipeline. There are many systems to do this for debian systems. And yes, this is exactly the sort of logic that the ROS buildfarm provides: https://github.com/ros-infrastructure/ros_buildfarm/blob/master/doc/index.rst
I strongly recommend against trying to develop your own system beyond maybe a quick bash script to build debian packages. There's a reason that there's large projects and infrastructure setup to do this. You start out with a simple solution you're likely to keep adding features and features. There are other solutions for this too like https://github.com/mikeferguson/buildbot-ros and if you're not deploying at a scale which warrents the overhead of many debian packages such as bundling https://github.com/mikepurvis/ros-bundling or https://aws.amazon.com/blogs/opensource/building-bundling-ros-app-aws-robomaker/
Asked by tfoote on 2021-04-21 16:23:59 UTC
Comments
Thank you for your answer!
Actually, I'm planning to deploy my own private ROS Build Farm and I'm researching how to do it / if it's a good fit.
I'm familiar with the Debian Pipeline, I even have a few Ubuntu PPAs... :-)
So far, when I run bloom-generate
, I go straight to dpkg-buildpackage -rfakeroot -uc -us -b -d
, then I can have the Debian artifact right away (since I still don't have a ROS-based build farm).
This means that I'm skipping the sourcedebs
generation... Any tips/docs about how to properly build them?
BTW, today I do have a "quick-and-dirty" Bash script that runs bloom-generate
and dpkg-buildpackage
BUT, it does something really bad... It changes the debian/rules
to point CMAKE_PREFIX_PATH
to a previously catkin build
workspace (that /devel/
subdir), to use it as a "build-depends", since the packages don't exist on Ubuntu itself... And I want to fix this, and more, by deploying an awesome ROS Build Farm / Jenkins on-premise!
Enjoy your weekend!
Asked by Thiago Martins on 2021-04-22 17:28:46 UTC
You can invoke dpkg-buildpackage
with -s
to make sourcedebs. Good luck with getting things setup for your own on premises. Getting it setup is not a small task, but we're happy to answer questions as you go. Make sure to tag questions with ros_buildfarm
Asked by tfoote on 2021-04-22 19:06:11 UTC
Comments