ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | Q&A answers.ros.org

# Does Tox get along with catkin's setup.py?

Hi all,

I am trying to automate the testing of my package using Tox but not without problems. It seems that Tox doesn't like catkin's setup.py config file, at least the barebones setup.py from catkin's documentation.

Here's my tox.ini config file:

[tox]
envlist = py27, py34

[testenv]
deps =
catkin_pkg
rospkg
coverage
flake8
coveralls
nose
nose-cov
nose-cover3
commands = nosetests --with-coverage --cover-package=rospy_utils


It basically tries to execute the tests in a Python 2.7 and 3.4 environments so And here's my setup.pyfile:

#!/usr/bin/env python

from distutils.core import setup
from catkin_pkg.python_setup import generate_distutils_setup

d = generate_distutils_setup(
# #  don't do this unless you want a globally visible script
# scripts=['bin/myscript'],
packages=['rospy_utils'],
package_dir={'': 'src'}
)

setup(**d)


Both seem ok but, when I run toxon my package root folder I get the following trace from tox:

viki@vROS:[~/devel/mona...src/rospy_utils]\$ tox
GLOB sdist-make: /home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/setup.py
py27 inst-nodeps: /home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/.tox/dist/rospy_utils-0.2.0.zip
ERROR: invocation failed (exit code 1), logfile: /home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/.tox/py27/log/py27-4.log
ERROR: actionid=py27
msg=installpkg
cmdargs=[local('/home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/.tox/py27/bin/pip'), 'install', '--pre', '-U', '--no-deps', '/home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/.tox/dist/rospy_utils-0.2.0.zip']
env={'ROS_MAVEN_REPOSITORY': 'https://github.com/rosjava/rosjava_mvn_repo/raw/master', 'ROS_DISTRO': 'hydro', 'GNOME_DESKTOP_SESSION_ID': 'this-is-deprecated', 'PYTHONHASHSEED': '1092722124', 'LASR_INSTALL_PATH': '/opt/Loquendo/LASR', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0', 'ROS_MAVEN_DEPLOYMENT_REPOSITORY': '/home/viki/devel/rosbuild_ws/devel/share/maven', 'CPATH': '/home/viki/devel/rosbuild_ws/devel/include:/home/viki/devel/monarch/code/trunk/catkin_ws/devel/include:/home/viki/devel/catkin_ws/devel/include:/home/viki/devel/monarch/code/trunk/devel/include:/opt/ros/hydro/include', 'LOGNAME': 'viki', 'USER': 'viki', 'PATH': '/home/viki/devel/monarch/code/trunk/catkin_ws/src/rospy_utils/.tox/py27/bin:/home/viki/devel/rosbuild_ws/devel/bin:/home/viki/devel/monarch/code/trunk/catkin_ws/devel/bin:/home/viki/devel/catkin_ws/devel/bin:/home/viki/devel/monarch/code/trunk/devel/bin:/opt/ros/hydro/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games', 'GNOME_KEYRING_CONTROL': '/tmp/keyring-RJVCk2', 'CMAKE_PREFIX_PATH': '/home/viki/devel/rosbuild_ws/devel:/home/viki/devel/monarch/code/trunk/catkin_ws/devel:/home/viki/devel/catkin_ws/devel:/home/viki/devel/monarch/code/trunk/devel:/opt/ros/hydro', 'LD_LIBRARY_PATH': '/home/viki/devel/rosbuild_ws/devel/lib:/home/viki/devel/monarch/code/trunk/catkin_ws/devel/lib:/home/viki/devel/catkin_ws/devel/lib:/home/viki/devel/monarch/code/trunk/devel/lib:/opt/ros/hydro/lib', 'SSH_AGENT_PID': '2442', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm', 'SHELL': '/bin/bash', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session0', 'XDG_SESSION_COOKIE': 'cf0fccd607b7ba15bad9beb400000011-1420981571.218005-898442704', 'SESSION_MANAGER': 'local/vROS:@/tmp/.ICE-unix/2203,unix/vROS:/tmp/.ICE-unix/2203', 'SHLVL': '1', 'MANDATORY_PATH': '/usr/share/gconf/ubuntu-2d.mandatory.path', 'DISPLAY': ':0', 'WINDOWID': '67108869', 'LTTS7_DEFAULTSESSION': '/opt/Loquendo/LTTS7/bin/default.session', 'LTTS_INSTALL_PATH': '/opt ...
edit retag close merge delete

Sort by » oldest newest most voted

EDIT:

I've found a cleaner way to solve it.

Ideally setup.py and package.xml should be in the same dir. However, this does not happens inside a tox venv unless you tell it what files should be included. To do so you need to create a MANIFEST.in in the same dir as setup.py and list there all the files you want to include in your source distribution. Here is what I put in my MANIFEST.in:

include package.xml


And my setup.py now is the barebones setup.py listed in the Catkin documentation:

from distutils.core import setup
from catkin_pkg.python_setup import generate_distutils_setup

d = generate_distutils_setup(
# #  don't do this unless you want a globally visible script
# scripts=['bin/myscript'],
packages=['rospy_utils'],
package_dir={'': 'src'}
)



It seems that every ROS-related file that you don't include in the MANIFEST.in won't be included in the source distribution that is put into the tox virtual env. I guess that, if you need to do some rostests, use some ROS msgs, etc., you might need to include them here as well.

Just for future reference, I leave the old answer here:

As @William points out, it seems that the generate_distutils_setup included in the setup.py needs to know the location of the package.xml file. The call has an optional parameter that defaults to os.path.curdir which is usually ok since setup.py is included in the same dir as package.xml.

However, it seems that tox runs the setup.py in a virtual environment located in /tmp/. Therefore, if you don't tell generate_distutils_setup that you are running in a separate dir it will fail.

I've modified my setup.py to tell it where to find the real path of the package.xml file and now I can run my tox commands perfectly. That's how my setup.py looks like now:

#!/usr/bin/env python

from distutils.core import setup
from catkin_pkg.python_setup import generate_distutils_setup

import rospkg
THIS_PKG = 'rospy_utils'
rospack = rospkg.RosPack()
pkg_path = rospack.get_path(THIS_PKG)

d = generate_distutils_setup(
# #  don't do this unless you want a globally visible script
# scripts=['bin/myscript'],
packages=['rospy_utils'],
package_dir={'': 'src'},
package_xml_path=pkg_path
)


more

Another option would be to assume that the package.xml and the setup.py are always next to each other (a pretty safe bet) and use the directory containing the setup.py as the package.xml path rather than using rospkg to find it. rospkg will require the ROS_PACKAGE_PATH to be set.

( 2015-01-24 14:05:36 -0500 )edit

Yes, you are right. The problem was that tox did not include the package.xml file. I've updated my answer with a cleaner solution.

( 2015-01-25 11:03:32 -0500 )edit

( 2018-03-08 15:01:20 -0500 )edit

Yeah, unfortunately the setup.py while trying to be conventional, in practice that setup.py is sort of a crutch we use to get Python code to install to the correct place. Rather then trying to figure out where your Python code should go on every different system and variation we rely on the setuptools infrastructure to give us the right answer.

Obviously this is not working for you out of the box, but looking at the error it seems mostly confused about where the package.xml is. You might be able to modify your setup.py to make this work by changing to the directory of the setup.pybefore running the generate_distutils_setup function. At best I would call this a hack.

I don't really have any experience with tox, but my understanding is that many people use it to install into a virtualenv for several versions of Python before doing testing in each. It should be possible to install catkin packages into a virtualenv, but I haven't tried that lately. Unfortunately at this point I wouldn't know how to proceed with integrating tox into this workflow.

more

Thanks @William, you have pointed me to the right direction. Indeed, it seems that generate_distutils_setup needs to know where is the package.xml dir. If you don't tell it, he'll look for it in os.curdir() and fail since we are in a venv. In a while I'll post an answer with the details

( 2015-01-24 09:48:05 -0500 )edit