Ask Your Question
3

Releasing private catkin packages with bloom in legacy codebase

asked 2016-04-28 07:51:28 -0500

demmeln gravatar image

updated 2016-04-29 05:56:16 -0500

I have the following scenario. We have a legacy codebase, all in one git-repository, with different modules and a legacy build system (in fact there are 4 different build systems already, 2 of them in daily used, qmake on Linux and vs in Windows). I want to create catkin packages for the individual modules and release them into a private rosdistro.

Because the code base will continue to be used with the legacy build system in non-ROS context, I have limited freedom to restructure and change things. I would like to be as unintrusive to the current layout and structure as possible, in order to not be faced with acceptance problems in parts of our team that are not using ROS (yet).

I have gotten as far as creating the catkin packages, making sure they work with the install space (see also a previous question) and releasing them with bloom into a private rosdistro. Now I want to build debians for the released packages (first manually, later with buildbot-ros or eventually with a private buildfarm setup). I am confronted with some issue that I will describe in the following.

The repository layout is as follows:

foo-project
├── config.h
├── external-libraries
│   └── libbar
│       └── bar.h
├── module1
│   ├── package.xml
│   ├── CMakeLists.txt
│   └── ...
├── module2
│   ├── package.xml
│   ├── CMakeLists.txt
│   ├── module2.h
│   └── ...
├── module3
│   ├── package.xml
│   ├── CMakeLists.txt
│   └── ...
├── ...
...

1. Header files outside the package directory

The first issue is that there is a header file config.h in the project root. It is included with #include "../config" in the headers of module1 (which are in turn included by other modules). I install the config.h together with the headers of module1, but I still need it to be in-reach at build-time of module1, which is not the case for the bloom-released branches, that contain only the module folder, not its parent. How could I inject this file into the bloom-released branch?

2. External library files at build time

In module1, the third party library libbar is included. It needs to be only there at build time, and not installed / exported. In the current module build-definition there is a include_directories(../external-libraries/libbar) and then in the source files of module1 it is referenced simply as #include "bar.h". Similar to the previous issue, I need to make these header files available at build time. But the difference is, that I don't need install them, and that they are not in the repository root, but in a parallel subfolder. This would make it possible to wrap them up in a foo_external_libs-dev catkin package.

3. Interdependencies between modules

Mostly the dependencies between the different modules can be mapped to a tree structure, with module1 at the root. This is reflected in the dependencies as declared in the package.xml files. However, to build the individual modules, there are unfortunately cyclic interdependencies. E.g. to build module1, module2.h is needed. When building in a catkin workspace, this works because the CMakeLists.txt of module1 has a include_directories(../module2). It is also ... (more)

edit retag flag offensive close merge delete

1 Answer

Sort by » oldest newest most voted
3

answered 2016-04-28 18:27:34 -0500

William gravatar image

updated 2016-05-26 02:25:37 -0500

For issue 1 (and possibly issue 2), I'd say your best option is to preprocess the repository and copy the required files into the packages which need them, i.e. copy the top level config.h into each of the packages and change how the config file is referenced. For example, rather than headers doing #include "../config.h" do #include "config.h" and add the top level directory to the include paths for each module with -I.

For issue 2, you can also install the file that would normally be referenced with ../other_module/header.h either to a folder off the normal include path, e.g. <prefix>/include/impl/other_module/header.h and pass -I<prefix>/include/impl to modules which need it, and also change the include statement to be #include "other_module/header.h".

For issue 3, I think you have to break at least one of the packages into two packages so that you can break the cycle. For example, you could have module1 install it's headers in a module1_headers package. Since that package would not need to build anything it doesn't need to build depend on module2, but only run depend (or technically build_export_depend on it. Then you can have module2 depend on module1_headers, and module1 depend on module1_headers and module2.

There is currently no way to run a script on import with bloom, but that would be easy to implement and I'd consider a pr that adds that functionality. Another option is that bloom can import from any vcs branch or tag and even take archive files directly. So you could write a script which preprocesses your original repository (moving and copying files, adding package.xml and CMakeLists.txt if needed, etc) and then creates an archive of the result. Then you could pass that to bloom without having to extend bloom at all.

I think I attempted to address all of your questions, so hopefully there is something helpful in there. Let me know if you have more questions.

EDIT:

This is in response to @demmeln's edited question.

As you say, one could imagine incorporating the custom "prepare-release-script" into the bloom import-upstream step quite easily. However, since I am not very familiar with bloom, I'm not 100% sure how this would look like. In particular, what remains a mystery for me so far is which of the branches in the release repository are meant to be updated manually, and which are meant to be updated only by bloom? I would imagine that the branches with patches are meant to be changed manually to include the needed patches (is there documentation describing this? If so, I didn't find it yet.), so that seems to be a natural place where one might put custom scripts that would be called on top of applying the given patches. Does that make sense?

So I think that adding a "prepare-release-script" into the release repository that bloom runs is ok, so long as that ... (more)

edit flag offensive delete link more

Comments

I've been looking for sometime for a detailed overview of what exactly bloom does and how, so this answer was really enlightening. Shouldn't an explanation like this be part of the bloom wiki?

Arthur Queiroz gravatar imageArthur Queiroz ( 2017-01-24 10:25:33 -0500 )edit

The same goes for the basic usage example (i.e. generating deb packing files for a single ros package) described on https://github.com/ros-infrastructure... . This simple example would make bloom easier to understand upon the first contact with it.

Arthur Queiroz gravatar imageArthur Queiroz ( 2017-01-24 11:38:21 -0500 )edit

Yeah both answers by William are really great. It would be great to add those to the wiki, or at least some links to the answers here (anyone can edit the wiki ofc)...

demmeln gravatar imagedemmeln ( 2017-01-30 18:46:57 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

4 followers

Stats

Asked: 2016-04-28 07:51:28 -0500

Seen: 354 times

Last updated: May 26 '16