Disable hardware specific tests on bloom

asked 2020-03-05 06:59:43 -0600

Hey Forum

I'm working on submitting a sensor driver package to bloom for the first time. My package (https://github.com/ifm/ifm3d-ros) has unit tests which require the presence of my sensor hardware to execute. The tests are currently optional under the CATKIN_ENABLE_TESTING flag.

My suspicion is that bloom will automatically run these tests (and thus fail due to no hardware), correct? If so, I suppose I need to either:

Wrap the testing clause inside another opt-in custom CMAKE flag which won't be set inside of the bloom system
Identify some other bloom mechanism to explicitly opt-out of testing for my package

Is one or the other option preferred?

Thanks!

edit retag flag offensive close merge delete

add a comment

answered 2020-03-05 07:24:39 -0600

gvdhoorn
86574 ●283 ●1432 ●1054 http://cor.tudelft.nl/

updated 2020-03-05 09:44:53 -0600

My suspicion is that bloom will automatically run these tests (and thus fail due to no hardware), correct?

No, not really.

This could be me being pedantic, but Bloom does not run any tests. Its "only" responsibility is to create source archives of your packages and upload those (+ some metadata) to your release repository (it does a few more things of course).

The buildfarm will then use those source archives and will run jobs that will -- in the end -- create .deb and .rpm (depending on how its configured).

~~Afaik the release jobs do not actually run any tests~~ The tests are run in release jobs, but the results of the tests do not influence the success of a release build (thanks @nuclearsandwich ). That would be the responsibility of the devel jobs.

Assuming this is your package (or one of them), the devel jobs appear to have been failing so far, but not because of the tests.

edit flag offensive delete link

Comments

That was my original understanding, but then I ran a bloom Prerelease Test for the core driver package (ifm3d_core) on which this ROS package (ifm3d) depends, it included the ifm3d dependency and ran tests, which failed.

theseankelly ( 2020-03-05 07:28:33 -0600 )edit

A prerelease test is not part of Bloom. The process is just documented there as it's good to do one as part of the release cycle. Bloom is not involved.

The command you run is executed in a docker container on your local machine. If that container doesn't have access to the required hw, the tests will fail -- as expected.

Again, IIRC, the release jobs do not run the tests.

gvdhoorn ( 2020-03-05 07:32:10 -0600 )edit

@gvdhoorn is correct that bloom itself won't run tests. It does generate RPM and debian package metadata for building your packages (which will be performed on the ROS build farms for officially released packages). As part of the package build process, tests are run but the results are discarded and the package is packaged regardless of test status.

You can see how this is implemented in the template for a debian/rules Makefile for building an ament_cmake package: https://github.com/ros-infrastructure...

The devel / pr jobs do run and process tests and if those tests rely on hardware to be available then they can likely never pass on our build farm. Making the tests off by default or only running them if the required hardware is available are both ways I can think of to make the tests work on the official build farm.