catkin_make CMake cross-compile not finding libs: Librt, LibThreads
Hi,
I need help configuring my setup to get a usable IDE working environment for ROS cross-compilation development.
The setup is as following:
- Raspberry Zero: compiled ROS kinetic successfully from source
- 64Bit Ubuntu x86: able to cross-compile and remote debug Raspberry Zero (within Eclispe). Installed ROS via bin packages successfully
I want to crosscompile a ROS node (via make target in Eclipse CDT) on the Ubuntu machine, then remote debug ROS node on the Zero. I was able to generate an Eclipse project as well as building ROS node for x86. Via ssh I can build the ROS node on the zero, as well.
But it is impossible for me to figure out the root cause of what’s wrong with the environment configuration. I cannot convince catkin_make to build my node with the ARM compiler and environment.
Here's what I did:
First I created a toolchainfile /home/ruben/catkinrpiws/rostoolchain.cmake to achieve cross-compile:
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_C_COMPILER /home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-gcc)
set(CMAKE_CXX_COMPILER /home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++)
set(CMAKE_FIND_ROOT_PATH /home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/)
# Have to set this one to BOTH, to allow CMake to find rospack
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM BOTH)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY_CMAKE_FIND_ROOT_PATH)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
Then I initialised environment
source /opt/ros/kinetic/setup.bash
I pulled a ROS node implementation from Git-Hub (I was able to build & exectue this on the Zero itself)
git clone https://github.com/rumble-on/ros-hc-sr04-node.git src/ros-hc-sr04-node
Klone nach 'src/ros-hc-sr04-node' ...
remote: Counting objects: 54, done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 54 (delta 22), reused 54 (delta 22), pack-reused 0
Entpacke Objekte: 100% (54/54), Fertig.
Prüfe Konnektivität ... Fertig
Finally generated workspace
catkin_init_workspace src/
Creating symlink "/home/ruben/catkin_rpi_ws/src/CMakeLists.txt" pointing to "/opt/ros/kinetic/share/catkin/cmake/toplevel.cmake"
Then I give it a try to build it
catkin_make -DCMAKE_TOOLCHAIN_FILE=/home/ruben/catkin_rpi_ws/rostoolchain.cmake
Base path: /home/ruben/catkin_rpi_ws
Source space: /home/ruben/catkin_rpi_ws/src
Build space: /home/ruben/catkin_rpi_ws/build
Devel space: /home/ruben/catkin_rpi_ws/devel
Install space: /home/ruben/catkin_rpi_ws/install
####
#### Running command: "cmake /home/ruben/catkin_rpi_ws/src -DCMAKE_TOOLCHAIN_FILE=/home/ruben/catkin_rpi_ws/rostoolchain.cmake -DCATKIN_DEVEL_PREFIX=/home/ruben/catkin_rpi_ws/devel -DCMAKE_INSTALL_PREFIX=/home/ruben/catkin_rpi_ws/install -G Unix Makefiles" in "/home/ruben/catkin_rpi_ws/build"
####
-- The C compiler identification is GNU 4.8.3
-- The CXX compiler identification is GNU 4.8.3
-- Check for working C compiler: /home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-gcc
-- Check for working C compiler: /home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++
-- Check for working CXX compiler: /home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/arm-linux-gnueabihf-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: /home/ruben/catkin_rpi_ws/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/kinetic
-- This workspace overlays: /opt/ros/kinetic
-- Found PythonInterp: /usr/bin/python (found version "2.7.12")
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/ruben/catkin_rpi_ws/build/test_results
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found gtest sources under '/usr/src/gtest': gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
CMAKE_LIBRARY_PATH:
CMAKE_LIBRARY_ARCHITECTURE: arm-linux-gnueabihf
CMAKE_SYSTEM_LIBRARY_PATH: /usr/lib/w32api;/usr/X11R6/lib;/usr/lib/X11;/usr/pkg/lib;/opt/csw/lib;/opt/lib;/usr/openwin/lib
CMAKE_VERSION=3.5.1
CMake Error at /opt/ros/kinetic/share/catkin/cmake/assert.cmake:17 (message):
Assertion failed: check for file existence, but filename
(RT_LIBRARY-NOTFOUND) unset. Message: RT Library
Call Stack (most recent call first):
/opt/ros/kinetic/share/catkin/cmake/tools/rt.cmake:42 (assert_file_exists)
/opt/ros/kinetic/share/catkin/cmake/all.cmake:147 (include)
/opt/ros/kinetic/share/catkin/cmake/catkinConfig.cmake:20 (include)
CMakeLists.txt:54 (find_package)
-- Configuring incomplete, errors occurred!
See also "/home/ruben/catkin_rpi_ws/build/CMakeFiles/CMakeOutput.log".
See also "/home/ruben/catkin_rpi_ws/build/CMakeFiles/CMakeError.log".
Invoking "cmake" failed
First issue I ran into was that cmake wasn’t able to find librt. That is little confusing for me because I am able to find it within the path I configured in the toolcahin.cmake file:
find /home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/ | grep librt.so
returned
/home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/arm-linux-gnueabihf/libc/lib/arm-linux-gnueabihf/librt.so.1
/home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/arm-linux-gnueabihf/libc/usr/lib/arm-linux-gnueabihf/librt.so
This is the snap from the toolchain.cmake
set(CMAKE_FIND_ROOT_PATH /home/ruben/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/)
After hours of try & error to convince CMake for using the right path to check for that lib I found this thread , where @103s reports same Issue.
While remaining not so sure, I modified a line of rt.cmake as follows that bypass the error above on QNX. And no build-time error occurs on Ubuntu (Quantal) too (I don't know if there's any runtime misbehavior)
Due to fact that the thread promises compilation, I gave up and commented most of the rt.cmake check, just to get a step further to immediately facing the next blocker:
catkin_make -DCMAKE_TOOLCHAIN_FILE=/home/ruben/catkin_rpi_ws/rostoolchain.cmake
Base path: /home/ruben/catkin_rpi_ws
Source space: /home/ruben/catkin_rpi_ws/src
Build space: /home/ruben/catkin_rpi_ws/build
Devel space: /home/ruben/catkin_rpi_ws/devel
Install space: /home/ruben/catkin_rpi_ws/install
####
#### Running command: "make cmake_check_build_system" in "/home/ruben/catkin_rpi_ws/build"
####
-- Using CATKIN_DEVEL_PREFIX: /home/ruben/catkin_rpi_ws/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/kinetic
-- This workspace overlays: /opt/ros/kinetic
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/ruben/catkin_rpi_ws/build/test_results
-- Found gtest sources under '/usr/src/gtest': gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.7.4
-- BUILD_SHARED_LIBS is on
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- ~~ traversing 1 packages in topological order:
-- ~~ - hc_sr04
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- +++ processing catkin package: 'hc_sr04'
-- ==> add_subdirectory(ros-hc-sr04-node)
CMake Error at /opt/ros/kinetic/share/roscpp/cmake/roscppConfig.cmake:141 (message):
Project 'hc_sr04' tried to find library 'pthread'. The library is neither
a target nor built/installed properly. Did you compile project 'roscpp'?
Did you find_package() it before the subdirectory containing its code is
included?
Call Stack (most recent call first):
/opt/ros/kinetic/share/catkin/cmake/catkinConfig.cmake:76 (find_package)
ros-hc-sr04-node/CMakeLists.txt:7 (find_package)
-- Configuring incomplete, errors occurred!
See also "/home/ruben/catkin_rpi_ws/build/CMakeFiles/CMakeOutput.log".
See also "/home/ruben/catkin_rpi_ws/build/CMakeFiles/CMakeError.log".
Makefile:290: die Regel für Ziel „cmake_check_build_system“ scheiterte
make: *** [cmake_check_build_system] Fehler 1
Invoking "make cmake_check_build_system" failed
When catkin tries to compile link(???), it is not able to find pthread lib. As matter of fact, CMake seems to find it in initial configuration inspection:
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
UPDATE 20/12/2016
running CMake on its own unveils interesting message
CMake Warning at ros-hc-sr04-node/CMakeLists.txt:130 (add_executable):
Cannot generate a safe runtime search path for target hc_sr04_node because files in some directories may conflict with libraries in implicit directories:
runtime library [libpthread.so] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
/home/ruben/rpi_libs/kinetic/lib
Some of these libraries may not be found correctly.
Looking forward for intputs. Thanks in advance.
Asked by _ruben_ on 2016-12-16 12:34:48 UTC
Comments
Not to discourage you, but I think you have an issue people generally try to avoid by either compiling on target, or by building entire root fs using OpenEmbedded or similar tools.
I can't help you I'm afraid, other than saying that I'd try to first get things working without Eclipse ..
Asked by gvdhoorn on 2016-12-20 06:17:53 UTC
..: your last update shows that that also doesn't work (yet).
And finally (but I think you already have that clear): catkin is basically CMake. It does use some trickery here and there, but it should leave most things alone.
Have you tried
catkin_tools?Asked by gvdhoorn on 2016-12-20 06:22:27 UTC
I'll have a look with Yocto and OpenEmbedded. In parallel I try to get at least the crosscompile of that node done. It turns out that I need to apply changes in nodes CMakeLists, which I guess is way far from reusability, e.g. link_directories(...), include_directories(...).
Asked by _ruben_ on 2016-12-21 01:54:14 UTC
I'm not suggesting that you should just move to Yocto & friends per se. I just wanted to give you a possible rationale for why you (can) find little information on this: cross-compilation can be difficult to get right, so users typically tend to avoid it and use something 'easier'.
Asked by gvdhoorn on 2016-12-21 03:12:51 UTC
Having to change paths manually is definitely not something that should be needed. For a typical cross-compilation setup you'd only want to change the sysroot and that should be it. I've seen people use Docker images for this.
Asked by gvdhoorn on 2016-12-21 03:14:36 UTC