catkin_add_nosetests runs with inconsistent pythonpath (depending whether it's a clean build or a rebuild)

asked 2018-05-15 19:34:05 -0500

Evan Gridley gravatar image

updated 2018-05-16 13:30:43 -0500

I seem to have extremely similar (almost identical) symptoms to an old closed question that got closed with no feedback ( ). Hopefully this thread doesn't suffer the same fate!

It's a relatively simple python package with some nosetests. The tests fail sometimes and pass sometimes. It seems to be directly correlated with clean build vs rebuild. Clean always fails, non-clean rebuild always passes.

I think it may be a bug in ROS/catkin where the python path is getting set inconsistently on build/rebuild. Here's what I've documented so far:

Scenario A: clean build. Tests fail.

  • run rm -r build devel
  • run catkin_make
  • run source devel/setup.bash
  • run catkin_make run_tests or catkin_make run_tests_<package-name>
  • Result: errors on including python files from the module this package wraps, e.g. ImportError: No module named <module>
  • Without altering any files that would cause a re-build, re-running the same run_tests command produces the same results
  • (similar errors for any attempts to include from nested sub-modules)

Scenario B: a dirty re-build, no changes to the files besides last-modified date. Tests pass.

  • Setup: Do Scenario A, or just don't wipe build & devel after doing whatever else.
  • touch the CMakeLists.txt file in the project in question
  • run catkin_make run_tests_<package-name>
  • Result: The 'broken' tests files now load & pass successfully.

Additionally, the tests pass if I run them manually by calling nosetests on the command line. My actual PYTHONPATH after sourcing setup.bash is what I would expect (/home/evan/<repo-name>/devel/lib/python2.7/dist-packages:/opt/ros/kinetic/lib/python2.7/dist-packages)

I did a little work to verify python path during test execution and here's the steps+results:

  • Added the following to the top of my test file: sys.stderr.write('PYTHONPATH=' + os.environ['PYTHONPATH'] + '\n')
  • Re-ran Scenario A
    • The debug/stderr output was PYTHONPATH=/opt/ros/kinetic/lib/python2.7/dist-packages
  • Re-ran Scenario B
    • The debug/stderr output was PYTHONPATH=/home/evan/<repo-name>/devel/lib/python2.7/dist-packages:/opt/ros/kinetic/lib/python2.7/dist-packages

PYTHONPATH seems like something that should be the same after building & running setup.bash no matter how (nor how many times) I build/run it, not something that changes on a re-build. Is this in fact a ROS/catkin bug of some kind as it seems from the above, or have I somehow misconfigured something?

my is straightforward/simple:

from distutils.core import setup
from catkin_pkg.python_setup import generate_distutils_setup

# fetch values from package.xml
setup_args = generate_distutils_setup(
    package_dir={'': 'scripts'}


My directory structure is


sample test file structure:

import unittest
from mock import Mock
from mock import MagicMock

import os
from packagename.class1 import blah
from packagename.submodule.class3 import blah3


<?xml version="1.0"?>
  <version>0.0.0</version ...
edit retag flag offensive close merge delete


Is your the real one, and is the directory structure you show an example, or the other way around?

Additional note: your CMakeLists.txt contains quite some unnecessary things for a Python pkg. catkin_package() can be empty, and rospy does not need to be find_package()-ed. Also ..

gvdhoorn gravatar image gvdhoorn  ( 2018-05-16 01:43:45 -0500 )edit

.. you don't have any include_directories(..), so remove that.

gvdhoorn gravatar image gvdhoorn  ( 2018-05-16 01:44:07 -0500 )edit

Corrected inline above. was the "real" one. (they're all "real", and do properly reference each other in our repo. I just changed the directory/package/file names to something generic but apparently I did it inconsistently) Thanks for the notes.

Evan Gridley gravatar image Evan Gridley  ( 2018-05-16 11:31:26 -0500 )edit

I don't think packagename can be in the scripts directory. At least not if you want it to be seen as a module.

gvdhoorn gravatar image gvdhoorn  ( 2018-05-16 11:49:25 -0500 )edit

Hmm... It works fine after a dirty rebuild as mentioned above. The python path is only wrong during the execution of catkin_make run_tests. Our application using this code has always worked fine.

Also, this failure showed up suddenly after an unrelated commit. Tests all passed on CI 2 days ago.

Evan Gridley gravatar image Evan Gridley  ( 2018-05-16 13:15:44 -0500 )edit

In general I would recommend not to source the setup file in the same shell where that workspace is being built. A repeated build with the setup file sourced will certainly have a different environment than the first build. Does the same problem exist when you don't source the setup file?

Dirk Thomas gravatar image Dirk Thomas  ( 2019-01-02 13:32:21 -0500 )edit