Ask Your Question
1

Colcon build and source install/setup.bash not finding packages

asked 2019-08-06 22:06:48 -0500

simon-t4 gravatar image

updated 2019-08-08 20:55:31 -0500

Hi

after using colcon build to compile Autoware 1.12 on Ubuntu 16.04 with ros kinetic , some times when running the script

 source autoware_dir/install/setup.bash

I receive an error that the packages are not found, and of therefore roslaunch or rosrun fails to find any nodes in the packages. The error is:

 user@locahost:~/autoware.ai$ source install/setup.bash
 not found: "/home/user/autoware.ai/install/adi_driver
 autoware_bag_tools
 autoware_build_flags
 autoware_can_msgs
 ..."

It turns out the problem is the way bash script tokenizes string variables (see below). My question is regarding the likely cause of the problem.

Question

Is it a problem with my environment? I may have messed up some environment variable or installed the wrong version of something. I would think if it is a problem with the colcon scripts themselves many other people would have similar problems.

Does anyone have any suggestions about what might be wrong?

I List the out put of various version info below.

EDIT: bash IFS is being overwritten by rosrun autocomplete. See below.

Cause of Problem

The error occurs because the script file install/local_setup.bash generates a list of package names which is a string with package names separated by (I think) line breaks. When the script then tries to add a prefix (path) to each package name, it treats the list of names as one single string.and of course can't find the file named after all packages. The relevant part of the script (install/local_setup.bash, from line 93) is

# get all packages in topological order                                                            
_colcon_ordered_packages="$($_colcon_python_executable "$_colcon_prefix_bash_COLCON_CURRENT_PREFIX_local_setup_util.py")"                                                                           
unset _colcon_python_executable                                                                    

# source package specific scripts in topological order                                             
for _colcon_package_name in $_colcon_ordered_packages; do                                          
      # setting COLCON_CURRENT_PREFIX avoids relying on the build time prefix of the sourced script    
     COLCON_CURRENT_PREFIX="${_colcon_prefix_bash_COLCON_CURRENT_PREFIX}/${_colcon_package_name}"     
     _colcon_prefix_bash_source_script "$COLCON_CURRENT_PREFIX/share/${_colcon_package_name}/package.bash"                                                                                              
done

The ordered package string generated by the python script is iterated over in the for loop. In this case the string _colcon_ordered_packages is not tokenised and only one giant string is processed within the loop.

Work Around

By using a while read -r _colcon_package_name; do .... done <<< "$_colcon_ordered_packages to iterate the string can be successfully tokenised and then sourced using the appropriate package.bash script

# source package specific scripts in topological order                                             
while read -r _colcon_package_name; do                                                                                                 
      # setting COLCON_CURRENT_PREFIX avoids relying on the build time prefix of the sourced script    
      COLCON_CURRENT_PREFIX="${_colcon_prefix_bash_COLCON_CURRENT_PREFIX}/${_colcon_package_name}"     
      _colcon_prefix_bash_source_script "$COLCON_CURRENT_PREFIX/share/${_colcon_package_name}/package.bash"                                                                                              
done <<< "$_colcon_ordered_packages"

Edit: fixed quotation marks

Edit: add

Persistent Work Around

Using _colcon_pthon_executable variable in above script, to find out what version of python colcon is using (in my case python3), and depending on what shell you are using (bash) edit the file :

 /usr/lib/python3/dist-packages/colcon_bash/shell/template/prefix.bash.em

This file is used by the colcon python module to generate the relevant part of install/local_setup.bash. Making the same change to the string tokenizer as above results in a workaround that will be persistent between builds.

Note: line numbers are slightly different in the ... (more)

edit retag flag offensive close merge delete

Comments

Added a persistent (between builds) work around

simon-t4 gravatar imagesimon-t4 ( 2019-08-08 00:16:27 -0500 )edit

My guess would be that you override the IFS in your bash configuration. Can you double check it and post the value you are using?

Dirk Thomas gravatar imageDirk Thomas ( 2019-08-08 10:37:47 -0500 )edit

Thanks, you are right. Open new terminal:

  echo -n "$IFS" | od -abc
  0000000  sp  ht  nl
                040 011 012
                  \t  \n
  0000003

Run source install/setup.bash, gives the original error, then

  echo -n "$IFS" | od -abc
  0000000

Which I guess means that an overwrite occurs and there are no IFS set.

The overwrite also seems to occurs when using the workaround, but the sourcing works. Somewhere in the setup.bash script IFS is being overwritten, I'll try to pinpoint where.

simon-t4 gravatar imagesimon-t4 ( 2019-08-08 20:01:17 -0500 )edit

The auto-complete function on rosrun seems to overwriting the IFS value. See original question.

simon-t4 gravatar imagesimon-t4 ( 2019-08-08 20:31:22 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2019-08-26 19:14:45 -0500

simon-t4 gravatar image

To officially answer:

Cause of problem

Thanks to Dirk Thomas's comment I think the problem is using rosrun autocompletion is causing the bash shell IFS to be overwritten. If I start a new terminal:

 user@locahost:~$ echo -n "$IFS" | od -abc
  0000000  sp  ht  nl
           040 011 012
                         \t  \n
   0000003
   user@localhost:~$ rosrun m
   map_msgs            message_generation  move_base_msgs
  ...
  user@localhost:~$ echo -n "$IFS" | od -abc
  0000000
  user@localhost:~$

Persistent Work Around

Using _colcon_pthon_executable variable in local_setup.bash script to find out what version of python colcon is using (in my case python3), and depending on what shell you are using (bash) edit the file :

  /usr/lib/python3/dist-packages/colcon_bash/shell/template/prefix.bash.em

This file is used by the colcon python module to generate the relevant part of install/local_setup.bash. When IFS is overwritten the loop to construct package names fails because string is not tokensied properly.

By using a while read -r _colcon_package_name; do .... done <<< "$_colcon_ordered_packages to iterate the string can be successfully tokenised and then sourced using the appropriate package.bash script

 # source package specific scripts in topological order                                             
 while read -r _colcon_package_name; do                                                                                                 
       # setting COLCON_CURRENT_PREFIX avoids relying on the build time prefix of the sourced script    
       COLCON_CURRENT_PREFIX="${_colcon_prefix_bash_COLCON_CURRENT_PREFIX}/${_colcon_package_name}"     
       _colcon_prefix_bash_source_script "$COLCON_CURRENT_PREFIX/share/${_colcon_package_name}/package.bash"                                                                                              
 done <<< "$_colcon_ordered_packages"
edit flag offensive delete link more

Comments

I tried to create a patch to restore the IFS in rosrunafter it is modifying it: https://github.com/ros/ros/pull/227 It would be great if you could give it a try and report back if it addresses the problem for you.

Dirk Thomas gravatar imageDirk Thomas ( 2019-08-28 16:06:41 -0500 )edit

Thanks for your work. Unfortunately (if I have applied the patch correctly) the problem still persists with the new version of rosrun.

simon-t4 gravatar imagesimon-t4 ( 2019-08-30 03:26:30 -0500 )edit
1

Thanks for giving it a try. I have since then updated the PR and double checked that it works for me. It would be great if you could give it another try and report back again.

Dirk Thomas gravatar imageDirk Thomas ( 2019-09-04 13:57:25 -0500 )edit

Sorry for the late reply. Unfortunately there is still the same behavior. I am simply editing /opt/ros/bin/rosrun script to reflect the changes you suggest. So I might not be testing correctly.

simon-t4 gravatar imagesimon-t4 ( 2019-09-17 02:53:38 -0500 )edit

You also need to patch the rosbash file since that is doing the completion.

Dirk Thomas gravatar imageDirk Thomas ( 2019-09-17 10:51:22 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

2 followers

Stats

Asked: 2019-08-06 22:06:48 -0500

Seen: 154 times

Last updated: Aug 26