Colcon build and source install/setup.bash not finding packages

Hi

after using colcon build to compile Autoware 1.12 on Ubuntu 16.04 with ros kinetic , some times when running the script

 source autoware_dir/install/setup.bash


I receive an error that the packages are not found, and of therefore roslaunch or rosrun fails to find any nodes in the packages. The error is:

 user@locahost:~/autoware.ai$source install/setup.bash not found: "/home/user/autoware.ai/install/adi_driver autoware_bag_tools autoware_build_flags autoware_can_msgs ..."  It turns out the problem is the way bash script tokenizes string variables (see below). My question is regarding the likely cause of the problem. Question Is it a problem with my environment? I may have messed up some environment variable or installed the wrong version of something. I would think if it is a problem with the colcon scripts themselves many other people would have similar problems. Does anyone have any suggestions about what might be wrong? I List the out put of various version info below. EDIT: bash IFS is being overwritten by rosrun autocomplete. See below. Cause of Problem The error occurs because the script file install/local_setup.bash generates a list of package names which is a string with package names separated by (I think) line breaks. When the script then tries to add a prefix (path) to each package name, it treats the list of names as one single string.and of course can't find the file named after all packages. The relevant part of the script (install/local_setup.bash, from line 93) is # get all packages in topological order _colcon_ordered_packages="$($_colcon_python_executable "$_colcon_prefix_bash_COLCON_CURRENT_PREFIX_local_setup_util.py")"
unset _colcon_python_executable

# source package specific scripts in topological order
for _colcon_package_name in $_colcon_ordered_packages; do # setting COLCON_CURRENT_PREFIX avoids relying on the build time prefix of the sourced script COLCON_CURRENT_PREFIX="${_colcon_prefix_bash_COLCON_CURRENT_PREFIX}/${_colcon_package_name}" _colcon_prefix_bash_source_script "$COLCON_CURRENT_PREFIX/share/${_colcon_package_name}/package.bash" done  The ordered package string generated by the python script is iterated over in the for loop. In this case the string _colcon_ordered_packages is not tokenised and only one giant string is processed within the loop. Work Around By using a while read -r _colcon_package_name; do .... done <<< "$_colcon_ordered_packages to iterate the string can be successfully tokenised and then sourced using the appropriate package.bash script

# source package specific scripts in topological order
# setting COLCON_CURRENT_PREFIX avoids relying on the build time prefix of the sourced script
COLCON_CURRENT_PREFIX="${_colcon_prefix_bash_COLCON_CURRENT_PREFIX}/${_colcon_package_name}"
_colcon_prefix_bash_source_script "$COLCON_CURRENT_PREFIX/share/${_colcon_package_name}/package.bash"
done <<< "$_colcon_ordered_packages"  Edit: fixed quotation marks Edit: add Persistent Work Around Using _colcon_pthon_executable variable in above script, to find out what version of python colcon is using (in my case python3), and depending on what shell you are using (bash) edit the file :  /usr/lib/python3/dist-packages/colcon_bash/shell/template/prefix.bash.em  This file is used by the colcon python module to generate the relevant part of install/local_setup.bash. Making the same change to the string tokenizer as above results in a workaround that will be persistent between builds. Note: line numbers are slightly different in the ... edit retag close merge delete Comments Added a persistent (between builds) work around ( 2019-08-08 00:16:27 -0500 )edit My guess would be that you override the IFS in your bash configuration. Can you double check it and post the value you are using? ( 2019-08-08 10:37:47 -0500 )edit Thanks, you are right. Open new terminal:  echo -n "$IFS" | od -abc
0000000  sp  ht  nl
040 011 012
\t  \n
0000003


Run source install/setup.bash, gives the original error, then

  echo -n "$IFS" | od -abc 0000000  Which I guess means that an overwrite occurs and there are no IFS set. The overwrite also seems to occurs when using the workaround, but the sourcing works. Somewhere in the setup.bash script IFS is being overwritten, I'll try to pinpoint where. ( 2019-08-08 20:01:17 -0500 )edit The auto-complete function on rosrun seems to overwriting the IFS value. See original question. ( 2019-08-08 20:31:22 -0500 )edit 1 Answer Sort by » oldest newest most voted To officially answer: Cause of problem Thanks to Dirk Thomas's comment I think the problem is using rosrun autocompletion is causing the bash shell IFS to be overwritten. If I start a new terminal:  user@locahost:~$ echo -n "$IFS" | od -abc 0000000 sp ht nl 040 011 012 \t \n 0000003 user@localhost:~$ rosrun m
map_msgs            message_generation  move_base_msgs
...
user@localhost:~$echo -n "$IFS" | od -abc
0000000
user@localhost:~$ Persistent Work Around Using _colcon_pthon_executable variable in local_setup.bash script to find out what version of python colcon is using (in my case python3), and depending on what shell you are using (bash) edit the file :  /usr/lib/python3/dist-packages/colcon_bash/shell/template/prefix.bash.em  This file is used by the colcon python module to generate the relevant part of install/local_setup.bash. When IFS is overwritten the loop to construct package names fails because string is not tokensied properly. By using a while read -r _colcon_package_name; do .... done <<< "$_colcon_ordered_packages to iterate the string can be successfully tokenised and then sourced using the appropriate package.bash script

 # source package specific scripts in topological order
# setting COLCON_CURRENT_PREFIX avoids relying on the build time prefix of the sourced script
COLCON_CURRENT_PREFIX="${_colcon_prefix_bash_COLCON_CURRENT_PREFIX}/${_colcon_package_name}"
_colcon_prefix_bash_source_script "$COLCON_CURRENT_PREFIX/share/${_colcon_package_name}/package.bash"
done <<< "\$_colcon_ordered_packages"

more

I tried to create a patch to restore the IFS in rosrunafter it is modifying it: https://github.com/ros/ros/pull/227 It would be great if you could give it a try and report back if it addresses the problem for you.

( 2019-08-28 16:06:41 -0500 )edit

Thanks for your work. Unfortunately (if I have applied the patch correctly) the problem still persists with the new version of rosrun.

( 2019-08-30 03:26:30 -0500 )edit
1

Thanks for giving it a try. I have since then updated the PR and double checked that it works for me. It would be great if you could give it another try and report back again.

( 2019-09-04 13:57:25 -0500 )edit

Sorry for the late reply. Unfortunately there is still the same behavior. I am simply editing /opt/ros/bin/rosrun script to reflect the changes you suggest. So I might not be testing correctly.

( 2019-09-17 02:53:38 -0500 )edit

You also need to patch the rosbash file since that is doing the completion.

( 2019-09-17 10:51:22 -0500 )edit