Using unreleased test dependencies on the ROS build farm

asked 2020-06-02 05:18:13 -0500

jeroendm
168 ●3 ●7 ●11

Is there a recommended way to use unreleased dependencies in tests executed on the ROS build farm?

In my specific case we use MoveIt robot configuration packages to test an inverse kinematics plugin. I had to leave this dependency out of the package.xml to make the release work. Some solutions I consider, but am not happy about, are:

Writing new tests that only depend on released packages.
Creating a metapackage and adding all the dependencies to the repository.
Try to add it to moveit_resources. (see this issue)

@gavanderhoorn mentioned another possible solution in a Github comment:

Devel jobs should however have access to test_depends and I believe that's actually possible by specifying additional repositories as dependencies/prerequisites somehow in the distribution.yaml.

Notice that the CI setup with Travis works fine using a .travis.rosinstall file.

Related questions that do not provide an answer to this one.

edit retag flag offensive close merge delete

Comments

Notice that the CI setup with Travis works fine using a .travis.rosinstall file.

Note: that's an industrial_ci specific bit of functionality.

gvdhoorn ( 2020-06-02 06:09:51 -0500 )edit

@gavanderhoorn mentioned another possible solution in a Github comment:
Devel jobs should however have access to test_depends and I believe that's actually possible by specifying additional repositories as dependencies/prerequisites somehow in the distribution.yaml.

Seems I was thinking of the depends tag which is a:

list of repository names. Other repositories to perform cross referencing in the documentation. This is only necessary if the packages from the dependent repositories are not released. (default: [])

But that is only supported for doc jobs right now (according to REP-141).

gvdhoorn ( 2020-06-02 06:23:48 -0500 )edit

add a comment

answered 2020-06-02 06:09:22 -0500

tfoote

58457 ●128 ●550 ●526 http://www.ros.org/

Devel jobs should however have access to test_depends

Before the devel jobs are run their test depends are installed from released packages. If you need another package to run your tests you can add the test_depend and it will be available.

There's only currently support for having test dependencies on released packages. Adding support for arbitrary source checkouts significantly increases the complexity and burden on the buildfarm and also makes it much harder to reproduce for a new developer/contributor who then has to checkout arbitrary other content to run the tests. Extending the buildfarm to support this is something that could be contributed, but at the same time extending the build/test tools to detect and give good warnings/errors if the developer fails to checkout the appropriate source trees would also want to be done in parallel.

The other workaround is that the test build actually fetches the resources that it needs while it builds the tests. This is not generally preferred as it no longer leaves your builds self contained and may not work if your offline but it will work for CI and most developers.

edit flag offensive delete link

Comments

@tfoote: REP-141 seems to imply that doc jobs do support expressing dependency on unreleased packages.

Could that support not be reused for devel jobs?

gvdhoorn ( 2020-06-02 06:38:39 -0500 )edit

The other workaround is that the test build actually fetches the resources

Aah, I never though of that, it seems CMake can download a git repo and add it to the project.

jeroendm ( 2020-06-02 07:43:15 -0500 )edit

On the topic of should this be possible @v4nh notes (on discord) that:

From my perspective test dependencies should also work from released packages, otherwise this is still a bug. I (as a user) would expect to have tests succeed if I download a release and all released dependencies

jeroendm ( 2020-06-02 07:51:11 -0500 )edit

How would a user run tests with a released package?

The build script -- which contains/sets up the test driver -- is not included in a binary package.

gvdhoorn ( 2020-06-02 10:47:48 -0500 )edit

REP-141 seems to imply that doc jobs do support expressing dependency on unreleased packages.
Could that support not be reused for devel jobs?

I'm not saying it could be added but see my points about reproducibility and ease of use. Of course we can make it happen on the buildfarm, magically populating content into a magic overlay with all the declared extra content. But if you're building locally and want to run a test, where should all these extra checkouts go? Into you workspace? Tests shouldn't be modifying your workspace. Into a magic temp directory? How would you debug? Does it overlay any content in your workspace does it detect an implementation in your workdpace or underlay? And similarly when should those special packages be built? They need to be available at build time for the tests whereas a released package or another package in the workspace ...(more)

tfoote ( 2020-06-02 12:37:47 -0500 )edit

How would a user run tests with a released package?

You are always running tests on a package from source. But they can rely on the installed artifacts of any other package. For example if you need to play a rosbag for your tests you can add a test_depend on rosbag and then you can know you can invoke it.

tfoote ( 2020-06-02 12:37:59 -0500 )edit

We added the extra capability to the doc jobs for two reasons. First is that they don't have a special doc_depend tag. And secondly because the doc jobs are not actually building it's much simpler to just make sure to check out the contents before running the documentation rather than requiring extra overlay logic. This is not something that users typically do and will work if they just have the packages in their workspace when they run the documentation for a package, if they don't it won't cross link but that's not a blocker.

tfoote ( 2020-06-02 12:38:53 -0500 )edit

[..] magic [..] magic [..] magic [..]

Aren't you being a bit dramatic?

I agree that things need to be reproducable, but really, being able to specify additional source dependencies for devel jobs specifically, configured to run on the ROS buildfarm is essentially similar to how people use Travis CI.

And additional source dependencies could very easily be managed using a .rosinstall or .repos file. That would again be following standard usage patterns of those files and I would assume would not be surprising or invisible/magic to users.

Users who are interested in running tests locally are probably also quite capable of following some instructions, and if those instructions include an additional vcstool import < my_unreleased_test_depends.rosinstall I doubt it will scare or confuse them.

In the end we are talking about a CI config. Not a generic "how to seed your workspace with my packages" workflow.

gvdhoorn ( 2020-06-02 13:02:15 -0500 )edit

see more comments

Using unreleased test dependencies on the ROS build farm

Comments

1 Answer

Comments

Question Tools

Stats

Related questions

Using unreleased test dependencies on the ROS build farm edit

Comments

1 Answer

Comments

Question Tools

Stats

Related questions

Using unreleased test dependencies on the ROS build farm