Bloom an ament_python package with Pip Dependencies

asked 2021-06-14 15:42:35 -0500

10932 ●123 ●219 ●250 http://MetroRobots.com/

I'm having trouble with blooming a rolling package, possibly because of its pip dependencies.

Key error:

fatal: Remote branch debian/ros-rolling-tf-transformations_1.0.0-1_focal not found in upstream origin
...
subprocess.CalledProcessError: Command '['git', 'clone', '--branch', 'debian/ros-rolling-tf-transformations_1.0.0-1_focal', '--depth', '1', '--no-single-branch', 'https://github.com/DLu/tf_transformations_release.git', '/tmp/sourcedeb/source']' returned non-zero exit status 128.

That branch doesn't exist in my release repo. I'm guessing its because of this line of the bloom execution:

==> git-bloom-generate -y rosdebian --prefix release/rolling rolling -i 1 --os-name debian --os-not-required
No platforms defined for os 'debian' in release file for the 'rolling' distro. This os was not required; continuing without error.

I bring up the pip dependencies because I got warnings about my pip dependency:

Key 'python-transforms3d-pip' resolved to '['transforms3d']' with installer 'pip', which does not match the default installer 'apt'.
Failed to resolve python-transforms3d-pip on ubuntu:focal with: Error running generator: The Debian generator does not support dependencies which are installed with the 'pip' installer.
python-transforms3d-pip is depended on by these packages: ['tf_transformations']
<== Failed
Some of the dependencies for packages in this repository could not be resolved by rosdep.
<== The following generator action reported that it is missing one or more
    rosdep keys, but that the key exists in other platforms:
'['/usr/bin/git-bloom-generate', '-y', 'rosdebian', '--prefix', 'release/rolling', 'rolling', '-i', '1', '--os-name', 'ubuntu']'

If you are absolutely sure that this key is unavailable for the platform in
question, the generator can be skipped and you can proceed with the release.
Skip generator action and continue with release [y/N]?

So how do I bloom my Python-y package correctly? I get that Native packages are preferred but there's got to be some way to use pip, otherwise why even support it in the rosdistro?

answered 2021-06-15 02:20:41 -0500

gvdhoorn
86574 ●283 ●1432 ●1054 http://cor.tudelft.nl/

updated 2021-06-15 02:30:20 -0500

Bloom an ament_python package with Pip Dependencies

unless something has changed recently, packages with pip dependencies can't be released through the ROS buildfarm, afaik because the dpkg-buildpackage pipeline doesn't allow / support it.

but there's got to be some way to use pip, otherwise why even support it in the rosdistro?

you're referring to the -pip entries in the rosdep db? Those are there to facilitate builds from-source in a Catkin/Colcon workspace (installing dependencies using rosdep).

Edit: well, in ROS 1 we had catkin_virtualenv, which sort-of made it possible to release Python pkgs with pip dependencies. But that really only worked well for leaf packages, which I don't believe tf_transformations is.

edit flag offensive delete link

Comments

What if I add it as an install_requires entry in my setup.py?

David Lu ( 2021-06-15 15:15:00 -0500 )edit

I don't believe that changes anything. What I've seen other people do is vendor in their dependencies. That's basically what catkin_virtualenv automated.

It will mean users will get your dependencies "for free" with your own package, but without those dependencies being known to their package managers. That can lead to some interesting situations with multiple installations of those dependencies and the potential resulting conflicts.

A venv avoids that, but as I wrote above, I don't believe that would be possible for a non-leaf pkg.

gvdhoorn ( 2021-06-16 00:20:34 -0500 )edit

Perhaps @nuclearsandwich can add something here.

gvdhoorn ( 2021-06-16 00:21:07 -0500 )edit

What if I add it as an install_requires entry in my setup.py?

Then building that package will likely fail on the buildfarm when dh_python executes the installation and one of the modules listed in install_requires is missing.

nuclearsandwich ( 2021-11-05 17:06:59 -0500 )edit

@gvdhoorn is exactly right about pip packages being in rosdep to facilitate workspace builds that use them and bloom warns you that they aren't supported exactly because the deb and rpm pipelines cannot use pypi to satisfy system package dependencies.

If the pip packages are required for the package to function, that package can't be released. It would be possible to release other packages in the same repository by ignoring the pip-dependent package, but of course if the other packages depend on pip transitively that won't work.

ROS packages do not have any way to represent optional or opportunistic dependencies so if the pip package is used during testing (sometimes the case) and not needed during runtime then it's possible to patch the package.xml at bloom time to omit the pip package but that relies on it being a test-only dependency.

nuclearsandwich ( 2021-11-05 17:24:43 -0500 )edit

add a comment

Bloom an ament_python package with Pip Dependencies

1 Answer

Comments

Question Tools

Stats

Related questions

Bloom an ament_python package with Pip Dependencies edit

1 Answer

Comments

Question Tools

Stats

Related questions

Bloom an ament_python package with Pip Dependencies