ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question
0

Bloom an ament_python package with Pip Dependencies

asked 2021-06-14 15:42:35 -0500

David Lu gravatar image

I'm having trouble with blooming a rolling package, possibly because of its pip dependencies.

Key error:

fatal: Remote branch debian/ros-rolling-tf-transformations_1.0.0-1_focal not found in upstream origin
...
subprocess.CalledProcessError: Command '['git', 'clone', '--branch', 'debian/ros-rolling-tf-transformations_1.0.0-1_focal', '--depth', '1', '--no-single-branch', 'https://github.com/DLu/tf_transformations_release.git', '/tmp/sourcedeb/source']' returned non-zero exit status 128.

That branch doesn't exist in my release repo. I'm guessing its because of this line of the bloom execution:

==> git-bloom-generate -y rosdebian --prefix release/rolling rolling -i 1 --os-name debian --os-not-required
No platforms defined for os 'debian' in release file for the 'rolling' distro. This os was not required; continuing without error.

I bring up the pip dependencies because I got warnings about my pip dependency:

Key 'python-transforms3d-pip' resolved to '['transforms3d']' with installer 'pip', which does not match the default installer 'apt'.
Failed to resolve python-transforms3d-pip on ubuntu:focal with: Error running generator: The Debian generator does not support dependencies which are installed with the 'pip' installer.
python-transforms3d-pip is depended on by these packages: ['tf_transformations']
<== Failed
Some of the dependencies for packages in this repository could not be resolved by rosdep.
<== The following generator action reported that it is missing one or more
    rosdep keys, but that the key exists in other platforms:
'['/usr/bin/git-bloom-generate', '-y', 'rosdebian', '--prefix', 'release/rolling', 'rolling', '-i', '1', '--os-name', 'ubuntu']'

If you are absolutely sure that this key is unavailable for the platform in
question, the generator can be skipped and you can proceed with the release.
Skip generator action and continue with release [y/N]?

So how do I bloom my Python-y package correctly? I get that Native packages are preferred but there's got to be some way to use pip, otherwise why even support it in the rosdistro?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2021-06-15 02:20:41 -0500

gvdhoorn gravatar image

updated 2021-06-15 02:30:20 -0500

Bloom an ament_python package with Pip Dependencies

unless something has changed recently, packages with pip dependencies can't be released through the ROS buildfarm, afaik because the dpkg-buildpackage pipeline doesn't allow / support it.

but there's got to be some way to use pip, otherwise why even support it in the rosdistro?

you're referring to the -pip entries in the rosdep db? Those are there to facilitate builds from-source in a Catkin/Colcon workspace (installing dependencies using rosdep).


Edit: well, in ROS 1 we had catkin_virtualenv, which sort-of made it possible to release Python pkgs with pip dependencies. But that really only worked well for leaf packages, which I don't believe tf_transformations is.

edit flag offensive delete link more

Comments

What if I add it as an install_requires entry in my setup.py?

David Lu gravatar image David Lu  ( 2021-06-15 15:15:00 -0500 )edit

I don't believe that changes anything. What I've seen other people do is vendor in their dependencies. That's basically what catkin_virtualenv automated.

It will mean users will get your dependencies "for free" with your own package, but without those dependencies being known to their package managers. That can lead to some interesting situations with multiple installations of those dependencies and the potential resulting conflicts.

A venv avoids that, but as I wrote above, I don't believe that would be possible for a non-leaf pkg.

gvdhoorn gravatar image gvdhoorn  ( 2021-06-16 00:20:34 -0500 )edit

Perhaps @nuclearsandwich can add something here.

gvdhoorn gravatar image gvdhoorn  ( 2021-06-16 00:21:07 -0500 )edit

What if I add it as an install_requires entry in my setup.py?

Then building that package will likely fail on the buildfarm when dh_python executes the installation and one of the modules listed in install_requires is missing.

nuclearsandwich gravatar image nuclearsandwich  ( 2021-11-05 17:06:59 -0500 )edit

@gvdhoorn is exactly right about pip packages being in rosdep to facilitate workspace builds that use them and bloom warns you that they aren't supported exactly because the deb and rpm pipelines cannot use pypi to satisfy system package dependencies.

If the pip packages are required for the package to function, that package can't be released. It would be possible to release other packages in the same repository by ignoring the pip-dependent package, but of course if the other packages depend on pip transitively that won't work.

ROS packages do not have any way to represent optional or opportunistic dependencies so if the pip package is used during testing (sometimes the case) and not needed during runtime then it's possible to patch the package.xml at bloom time to omit the pip package but that relies on it being a test-only dependency.

nuclearsandwich gravatar image nuclearsandwich  ( 2021-11-05 17:24:43 -0500 )edit

Question Tools

1 follower

Stats

Asked: 2021-06-14 15:42:35 -0500

Seen: 400 times

Last updated: Jun 15 '21