Does Tox get along with catkin's setup.py?

asked 2015-01-24 01:21:04 -0500

396 ●8 ●15 ●26 http://roboticslab.uc3...

updated 2015-01-24 09:50:45 -0500

Hi all,

I am trying to automate the testing of my package using Tox but not without problems. It seems that Tox doesn't like catkin's setup.py config file, at least the barebones setup.py from catkin's documentation.

Here's my tox.ini config file:

[tox]
envlist = py27, py34

[testenv]
deps =
    catkin_pkg
    rospkg
    coverage
    flake8
    coveralls
    nose
    nose-cov
    nose-cover3
commands = nosetests --with-coverage --cover-package=rospy_utils

It basically tries to execute the tests in a Python 2.7 and 3.4 environments so And here's my setup.pyfile:

#!/usr/bin/env python

from distutils.core import setup
from catkin_pkg.python_setup import generate_distutils_setup

d = generate_distutils_setup(
    # #  don't do this unless you want a globally visible script
    # scripts=['bin/myscript'],
    packages=['rospy_utils'],
    package_dir={'': 'src'}
)

setup(**d)

Both seem ok but, when I run toxon my package root folder I get the following trace from tox:

viki@vROS:[~/devel/mona...src/rospy_utils]$ tox
GLOB sdist-make: /home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/setup.py
py27 inst-nodeps: /home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/.tox/dist/rospy_utils-0.2.0.zip
ERROR: invocation failed (exit code 1), logfile: /home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/.tox/py27/log/py27-4.log
ERROR: actionid=py27
msg=installpkg
cmdargs=[local('/home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/.tox/py27/bin/pip'), 'install', '--pre', '-U', '--no-deps', '/home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/.tox/dist/rospy_utils-0.2.0.zip']
env={'ROS_MAVEN_REPOSITORY': 'https://github.com/rosjava/rosjava_mvn_repo/raw/master', 'ROS_DISTRO': 'hydro', 'GNOME_DESKTOP_SESSION_ID': 'this-is-deprecated', 'PYTHONHASHSEED': '1092722124', 'LASR_INSTALL_PATH': '/opt/Loquendo/LASR', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0', 'ROS_MAVEN_DEPLOYMENT_REPOSITORY': '/home/viki/devel/rosbuild_ws/devel/share/maven', 'CPATH': '/home/viki/devel/rosbuild_ws/devel/include:/home/viki/devel/monarch/code/trunk/catkin_ws/devel/include:/home/viki/devel/catkin_ws/devel/include:/home/viki/devel/monarch/code/trunk/devel/include:/opt/ros/hydro/include', 'LOGNAME': 'viki', 'USER': 'viki', 'PATH': '/home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/.tox/py27/bin:/home/viki/devel/rosbuild_ws/devel/bin:/home/viki/devel/monarch/code/trunk/catkin_ws/devel/bin:/home/viki/devel/catkin_ws/devel/bin:/home/viki/devel/monarch/code/trunk/devel/bin:/opt/ros/hydro/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games', 'GNOME_KEYRING_CONTROL': '/tmp/keyring-RJVCk2', 'CMAKE_PREFIX_PATH': '/home/viki/devel/rosbuild_ws/devel:/home/viki/devel/monarch/code/trunk/catkin_ws/devel:/home/viki/devel/catkin_ws/devel:/home/viki/devel/monarch/code/trunk/devel:/opt/ros/hydro', 'LD_LIBRARY_PATH': '/home/viki/devel/rosbuild_ws/devel/lib:/home/viki/devel/monarch/code/trunk/catkin_ws/devel/lib:/home/viki/devel/catkin_ws/devel/lib:/home/viki/devel/monarch/code/trunk/devel/lib:/opt/ros/hydro/lib', 'SSH_AGENT_PID': '2442', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm', 'SHELL': '/bin/bash', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session0', 'XDG_SESSION_COOKIE': 'cf0fccd607b7ba15bad9beb400000011-1420981571.218005-898442704', 'SESSION_MANAGER': 'local/vROS:@/tmp/.ICE-unix/2203,unix/vROS:/tmp/.ICE-unix/2203', 'SHLVL': '1', 'MANDATORY_PATH': '/usr/share/gconf/ubuntu-2d.mandatory.path', 'DISPLAY': ':0', 'WINDOWID': '67108869', 'LTTS7_DEFAULTSESSION': '/opt/Loquendo/LTTS7/bin/default.session', 'LTTS_INSTALL_PATH': '/opt ...

(more)

edit retag flag offensive close merge delete

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2015-01-24 10:03:53 -0500

Victor Gonzalez-Pacheco

396 ●8 ●15 ●26 http://roboticslab.uc3...

updated 2015-01-25 11:04:05 -0500

EDIT:

I've found a cleaner way to solve it.

Ideally setup.py and package.xml should be in the same dir. However, this does not happens inside a tox venv unless you tell it what files should be included. To do so you need to create a MANIFEST.in in the same dir as setup.py and list there all the files you want to include in your source distribution. Here is what I put in my MANIFEST.in:

include package.xml

And my setup.py now is the barebones setup.py listed in the Catkin documentation:

from distutils.core import setup
from catkin_pkg.python_setup import generate_distutils_setup

d = generate_distutils_setup(
    # #  don't do this unless you want a globally visible script
    # scripts=['bin/myscript'],
    packages=['rospy_utils'],
    package_dir={'': 'src'}
)

adminstrosetup(**d)

It seems that every ROS-related file that you don't include in the MANIFEST.in won't be included in the source distribution that is put into the tox virtual env. I guess that, if you need to do some rostests, use some ROS msgs, etc., you might need to include them here as well.

Just for future reference, I leave the old answer here:

As @William points out, it seems that the generate_distutils_setup included in the setup.py needs to know the location of the package.xml file. The call has an optional parameter that defaults to os.path.curdir which is usually ok since setup.py is included in the same dir as package.xml.

However, it seems that tox runs the setup.py in a virtual environment located in /tmp/. Therefore, if you don't tell generate_distutils_setup that you are running in a separate dir it will fail.

I've modified my setup.py to tell it where to find the real path of the package.xml file and now I can run my tox commands perfectly. That's how my setup.py looks like now:

#!/usr/bin/env python

from distutils.core import setup
from catkin_pkg.python_setup import generate_distutils_setup

import rospkg
THIS_PKG = 'rospy_utils'
rospack = rospkg.RosPack()
pkg_path = rospack.get_path(THIS_PKG)

d = generate_distutils_setup(
    # #  don't do this unless you want a globally visible script
    # scripts=['bin/myscript'],
    packages=['rospy_utils'],
    package_dir={'': 'src'},
    package_xml_path=pkg_path
)

adminstrosetup(**d)

edit flag offensive delete link

Comments

Another option would be to assume that the package.xml and the setup.py are always next to each other (a pretty safe bet) and use the directory containing the setup.py as the package.xml path rather than using rospkg to find it. rospkg will require the ROS_PACKAGE_PATH to be set.

William ( 2015-01-24 14:05:36 -0500 )edit

Yes, you are right. The problem was that tox did not include the package.xml file. I've updated my answer with a cleaner solution.

Victor Gonzalez-Pacheco ( 2015-01-25 11:03:32 -0500 )edit

What is "adminstrosetup"?

Cerin ( 2018-03-08 15:01:20 -0500 )edit

add a comment

answered 2015-01-24 02:46:06 -0500

William

17335 ●101 ●182 ●262 http://wjwwood.io/

Yeah, unfortunately the setup.py while trying to be conventional, in practice that setup.py is sort of a crutch we use to get Python code to install to the correct place. Rather then trying to figure out where your Python code should go on every different system and variation we rely on the setuptools infrastructure to give us the right answer.

Obviously this is not working for you out of the box, but looking at the error it seems mostly confused about where the package.xml is. You might be able to modify your setup.py to make this work by changing to the directory of the setup.pybefore running the generate_distutils_setup function. At best I would call this a hack.

I don't really have any experience with tox, but my understanding is that many people use it to install into a virtualenv for several versions of Python before doing testing in each. It should be possible to install catkin packages into a virtualenv, but I haven't tried that lately. Unfortunately at this point I wouldn't know how to proceed with integrating tox into this workflow.

Here are some links which might help you to decide what to look at next:

This is a script that catkin runs to extract information from the setup.py : https://github.com/ros/catkin/blob/in...
This is the CMake code which handles installing Python code: https://github.com/ros/catkin/blob/in...

edit flag offensive delete link

Comments

Thanks @William, you have pointed me to the right direction. Indeed, it seems that generate_distutils_setup needs to know where is the package.xml dir. If you don't tell it, he'll look for it in os.curdir() and fail since we are in a venv. In a while I'll post an answer with the details

Victor Gonzalez-Pacheco ( 2015-01-24 09:48:05 -0500 )edit

add a comment

Does Tox get along with catkin's setup.py?

2 Answers

Comments

Comments

Question Tools

Stats

Related questions

Does Tox get along with catkin's setup.py? edit

2 Answers

Comments

Comments

Question Tools

Stats

Related questions

Does Tox get along with catkin's setup.py?