Ask Your Question
0

Why the ROS 2 Buildfarm package blacklisting is not configurable per-repository or per-package?

asked 2021-09-23 02:03:42 -0600

updated 2021-09-23 07:48:00 -0600

I am getting an email every morning saying that the ARM64 build has failed. This is expected since we are developing a robot simulator that currently supports AMD64 only (and there is not much interest in porting it to ARM64). This is somehow related to Blacklist ROS2 package for ARM build, but having such configuration file per repository would be much more convenient.

edit retag flag offensive close merge delete

Comments

1

This is two question. I'm going to answer the disable packages part. For clarity for future readers please edit your question to remove the other question and ask it in a standalone question so we can keep the discussions separate.

tfoote gravatar image tfoote  ( 2021-09-23 02:16:01 -0600 )edit
1

In addition to what @tfoote writes: please also update the title.

"ROS 2 Buildfarm Configuration" is really too generic. Before reading the question body I'd assumed this was going to be about setting up your own instance of the buildfarm.

gvdhoorn gravatar image gvdhoorn  ( 2021-09-23 06:18:45 -0600 )edit

@tfoote, my intention was to ask about a convenient way to configure Jenkins (ROS 2 Buildfarm) builds in general. The two problems I described were just concrete challenges I was facing. However, if you believe it is cleaner for other users like this then it is fine to me.

lukicdarkoo gravatar image lukicdarkoo  ( 2021-09-23 07:17:49 -0600 )edit

Technically, your updated question now seems like it's a duplicate of (the already mentioned) #q325881.

Wouldn't the more interesting question be why blacklisting is not configured per-repository or per-package? That's also what @tfoote seems to have answered.

gvdhoorn gravatar image gvdhoorn  ( 2021-09-23 07:23:22 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2021-09-23 02:26:55 -0600

tfoote gravatar image

How to disable package builds for ARM64?

You're right that to disable them on a specific platform you need to blacklist the packages. The question you link to tells you exactly how to do that.

The reason for the blacklist being where it is is because of the separation of concerns. All of the artifacts which are generated up until the binary jobs are architecture independent and at the point that we generate the source debs there is nothing architecture dependent. And at that point we're only using the debian packages and the debian packaging pipeline and there's nothing directly connected to the original repository anymore. Each repository may become N packages, and after bloom has completed the packages are no longer associated with the repository they're independent. Furthermore, grouping at the repository level makes the assumption that all packages in the repository are similarly compatible or incompatible with an architecture. It could easily be that a repository has an architecture independent package with messages and configuration files, and maybe launch files which is next to the package with an architecture dependent library.

We could embed the metadata into the package.xml or somewhere else in the repository and have the buildfarm keep the association and look it up that way. However that requires the upstream package maintainer to know and care about any and all architectures. With the current system it's quite possible that a company may want to turn on an alternative architecture that isn't on the main buildfarm (say i386 that we dropped recently) It doesn't make sense to require them to submit upstream patches with flags to the source repositories for the projects versus configuring their specific buildfarm's instructions. Furthermore, there may be other dimensions of the build environment that effects buildability of the package. For example you may choose to build specifically for the older RaspberryPi with it's odd hybrid of arm architectures. And again this is not something you necessarily want to bother the upstream maintainers with unless you want to submit a patch which extends the compatibility.

edit flag offensive delete link more

Comments

Thank for the detailed explanation! Yes, specifying supported architectures in package.xml is probably a better place. It could be opt-out/opt-in. For example, I can say <disable_architecture>arm64</disable_architecture>. Then, ROS 2 Buildfarm can add/remove support for architectures, but I claim that my package doesn't work for arm64.

lukicdarkoo gravatar image lukicdarkoo  ( 2021-09-23 07:35:32 -0600 )edit

Does that scale though? What about architectures you don't list? Say mips? If a hypothetical buildfarm adds support for that, we now either have to override the fact you didn't list it (because you didn't know it was something to list), or retrospectively update all package manifests of potentially compatible projects.

gvdhoorn gravatar image gvdhoorn  ( 2021-09-23 07:37:32 -0600 )edit

If you add a mips support then I will start receiving emails and I will update my packages.

Or, it could be the same as for Arduino. If nothing is specified then you build for all architectures (same as now). If architectures are specified then you build only for the specified architectures. Both approaches are backward compatible.

lukicdarkoo gravatar image lukicdarkoo  ( 2021-09-23 08:02:39 -0600 )edit

If the production farm adds mips support you will start receiving emails. But if we setup a testing farm, or someone with a specific use case sets up a private instance of the farm to run on an unsupported architecture you as the maintainer should not be bothered. And with several thousand packages touching and requiring a rerelease of any number of them just to declare compatibility is not feasible.

There's also issues that the most common reason for failure on a non-standard architecture is not that the package itself which we're building from source is incompatible, but that the target platform is missing a dependency. And you want to blacklist that package only for the specific platform until that dependency can be resolved. (often never) This is not uncommon when we target packages across multiple base platforms, it's not uncommon for dependencies to be at different ...(more)

tfoote gravatar image tfoote  ( 2021-09-23 12:34:21 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

2 followers

Stats

Asked: 2021-09-23 02:03:42 -0600

Seen: 38 times

Last updated: Sep 23