rtabmap crash, RealSense D435i, Assertion px != 0 failed
Environment is a Raspberry Pi 4, Raspbian Buster, kernel 4.19.66-v71+, ROS Kinetic Kame, rtabmap v0.19.3, rtabmap_ros v0.19.3 , RealSense D435i camera
Almost everything was installed from source
librealsense works, I can use realsense-viewer, realsense2camera works, I can visualize both RGB and depth with Rviz. The problem happens when I try to use rtabmap. I am using a modified version of the rsrtabmap.launch file (use a single camera, disable gyro, accel, and enable depth alignment and sync).
If I try to launch with rviz, I end up with
rtabmap: /usr/include/boost/smart_ptr/shared_ptr.hpp:728: typename boost::detail::sp_dereference<T>::type boost::shared_ptr<T>::operator*() const [with T = boost::signals2::mutex; typename boost::detail::sp_dereference<T>::type = boost::signals2::mutex&]: Assertion `px != 0' failed.
[rtabmap/rtabmap-4] process has died [pid 2287, exit code -6, cmd /home/pi/catkin_ws/devel/lib/rtabmap_ros/rtabmap --delete_db_on_start rgb/image:=/D435i/color/image_raw depth/image:=/D435i/aligned_depth_to_color/image_raw rgb/camera_info:=/D435i/color/camera_info rgbd_image:=rgbd_image_relay left/image_rect:=/stereo_camera/left/image_rect_color right/image_rect:=/stereo_camera/right/image_rect left/camera_info:=/stereo_camera/left/camera_info right/camera_info:=/stereo_camera/right/camera_info scan:=/scan scan_cloud:=/scan_cloud user_data:=/user_data user_data_async:=/user_data_async gps/fix:=/gps/fix tag_detections:=/tag_detections odom:=/D435i/odom/sample imu:=/imu/data __name:=rtabmap __log:=/home/pi/.ros/log/5f6c9b08-d651-11e9-827c-dca632195a3e/rtabmap-rtabmap-4.log].
If I try to launch with rtabmarviz, I get
* /D435i/realsense2_camertabmap: /usr/include/boost/smart_ptr/shared_ptr.hpp:728: typename boost::detail::sp_dereference<T>::type boost::shared_ptr<T>::operator*() const [with T = boost::signals2::mutex; typename boost::detail::sp_dereference<T>::type = boost::signals2::mutex&]: Assertion `px != 0' failed.
[rtabmap/rtabmap-4] process has died [pid 32379, exit code -6, cmd /home/pi/catkin_ws/devel/lib/rtabmap_ros/rtabmap --delete_db_on_start rgb/image:=/D435i/color/image_raw depth/image:=/D435i/aligned_depth_to_color/image_raw rgb/camera_info:=/D435i/color/camera_info rgbd_image:=rgbd_image_relay left/image_rect:=/stereo_camera/left/image_rect_color right/image_rect:=/stereo_camera/right/image_rect left/camera_info:=/stereo_camera/left/camera_info right/camera_info:=/stereo_camera/right/camera_info scan:=/scan scan_cloud:=/scan_cloud user_data:=/user_data user_data_async:=/user_data_async gps/fix:=/gps/fix tag_detections:=/tag_detections odom:=/D435i/odom/sample imu:=/imu/data __name:=rtabmap __log:=/home/pi/.ros/log/8b64ca02-d64f-11e9-ac7d-dca632195a3e/rtabmap-rtabmap-4.log].
log file: /home/pi/.ros/log/8b64ca02-d64f-11e9-ac7d-dca632195a3e/rtabmap-rtabmap-4*.log
qt5ct: using qt5ct plugin
rtabmapviz: /usr/include/boost/smart_ptr/shared_ptr.hpp:728: typename boost::detail::sp_dereference<T>::type boost::shared_ptr<T>::operator*() const [with T = boost::signals2::mutex; typename boost::detail::sp_dereference<T>::type = boost::signals2::mutex&]: Assertion `px != 0' failed.
[rtabmap/rtabmapviz-5] process has died [pid 32389, exit code -6, cmd /home/pi/catkin_ws/devel/lib/rtabmap_ros/rtabmapviz -d ~/.ros/rtabmap_gui.ini rgb/image:=/D435i/color/image_raw depth/image:=/D435i/aligned_depth_to_color/image_raw rgb/camera_info:=/D435i/color/camera_info rgbd_image:=rgbd_image_relay left/image_rect:=/stereo_camera/left/image_rect_color right/image_rect:=/stereo_camera/right/image_rect left/camera_info:=/stereo_camera/left/camera_info right/camera_info:=/stereo_camera/right/camera_info scan:=/scan scan_cloud:=/scan_cloud odom:=/D435i/odom/sample __name:=rtabmapviz __log:=/home/pi/.ros/log/8b64ca02-d64f-11e9-ac7d-dca632195a3e/rtabmap-rtabmapviz-5.log].
One of the things I noticed was that these messages are happening async to the launch, notice how it cut off a message. I also don't know how to turn on debug during launch, only after, since I only know how to use the GUI rqtloggerlevel
rtabmapviz simply won't show up. rviz will, but not with the data correctly routed, I get no images and no point cloud. HOWEVER, I know my camera works, if I manually add the RGB and depth image from the published camera I can at least see the output from the camera just fine! I just need rtabmap to work
Researching the issue, I found https://github.com/introlab/rtabmap_ros/issues/99 , and it said the solution was to install PCL 1.7 , I looked on my system and see I have 1.9.1 installed. I tried to run "sudo apt-get install libpcl-dev=1.7" and it said it can't find it. "apt-cache madison libpcl-dev" shows only 1.9.1 available from raspbian.
Or maybe I should back off on the rtabmap version? Go back to v0.18.0 full-release instead of the latest pre-release? This takes hours to do, I'd like to ask you guys before going through with this effort
(NOTE: libusb will not work with accel and gyro, something about not being able to claim the interface, so I've disabled accel and gyro in the launch file)
Please help, let me know what I can try or what information you need
Asked by frank26080115 on 2019-09-13 13:39:22 UTC
Answers
PCL >= 1.8 is built by default with -march=native
, so every dependencies of rtabmap using Eigen (e.g., g2o) should be also built with this flag, otherwise rtabmap nodes are crashing on start. See also https://github.com/introlab/rtabmap/issues/343#issuecomment-443476732
You can launch the nodes in gdb to see more info about the crash. See http://wiki.ros.org/roslaunch/Tutorials/Roslaunch%20Nodes%20in%20Valgrind%20or%20GDB
Example:
$ roslaunch rtabmap_ros rtabmap.launch args:="-d" launch_prefix:="xterm -e gdb --args"
cheers,
Mathieu
Asked by matlabbe on 2019-09-13 22:32:30 UTC
Comments
I've been rebuilding everything, starting from a blank Raspbian Buster image. No ROS installed at all, I just need RTAB-Map
GCC and G++ 6.5 used
vtk6.3, gtk3, qt5, qt5svg5 installed from apt-get, eigen3, tf2, protobuf, tbb2, tesseract, suitesparse, installed from apt-get
boost 1.58.0 installed from source
PCL 1.9 installed from source, explicitly set PCL_ENABLE_SSE:BOOL=ON
to have -march=native
g2o installed from source, BUILD_WITH_MARCH_NATIVE:BOOL=ON
gtsam-4.0.0-alpha, installed from source, I don't think there was an option that changes -march
, maybe I should add it to CMAKE_CXX_FLAGS?
cvsba-1.0.0 installed from source, I don't think there was an option that changes -march
, maybe I should add it to CMAKE_CXX_FLAGS?
OpenCV 3.4 installed from source, OPENCV_ENABLE_NONFREE=ON, with contrib modules, again, I didn't have anything explicitly setting -march
continuing in next comment because of character limit
Asked by frank26080115 on 2019-09-18 23:25:54 UTC
So with the above setup, I built rtabmap 0.19.3-kinetic, and ran it. No GUI launched, no splash screen, got an error that says
terminate called after throwing an instance of 'std::length_error'
what(): vector::_M_fill_insert
Aborted
Previously I installed ROS first (and none of the optional gtsam g2o cvsba opencv), and I at least got rtabmap's gui to show up when I tried to launch it. (but with a crap ton of other non fatal related errors, and that fatal error I originally mentioned)
What am I doing wrong? Is it simply the -march=native
thing? Or is there something else? I'll run another build with more CXX_FLAGS tonight
EDIT: I also had to enable -std=c++11
on a lot of builds, does that matter?
Asked by frank26080115 on 2019-09-18 23:29:48 UTC
I would try to keep only the core dependencies, like g2o and opencv. Can you try with:
cd rtabmap/build
cmake -DWITH_CVSBA=OFF -DWITH_GTSAM=OFF ..
make
If this minimal configuration doesn't work, launch in gdb to see where it crashes. GTSAM requires setting manually -march=native
in the CMAKE_CXX_FLAGS
. -std=c++11
is fine.
Asked by matlabbe on 2019-09-19 07:42:34 UTC
I did a build with g2o off, cvsba off, and gtsam off, plus Debug... got this out of gdb
#6 0xa5f76354 in std::__throw_length_error(char const*) () at /usr/lib/arm-linux-gnueabihf/libstdc++.so.6
#7 0xb1d4d7d8 in std::vector<long long, std::allocator<long long> >::_M_fill_insert(__gnu_cxx::__normal_iterator<long long*, std::vector<long long, std::allocator<long long> > >, unsigned int, long long const&) ()
at /usr/local/lib/libopencv_dnn.so.3.4
#8 0xa157b64c in () at /usr/lib/arm-linux-gnueabihf/libvtkCommonCore-6.3.so.6.3
#9 0xa1448630 in () at /usr/lib/arm-linux-gnueabihf/libvtkCommonCore-6.3.so.6.3
hmm VTK is installed via apt-get, OpenCV I built myself, also, I swear right now I should have VTK 7 and not 6
I checked my CMakeCache for OpenCV and it was built with VTK6... could that be it? I'll start another OpenCV build again...
EDIT: tried again with VTK7, same result, now suspecting it's libopencv_dnn's fault, rebuilding without the dnn module...
Asked by frank26080115 on 2019-09-19 15:44:20 UTC
Can you build opencv without VTK? libopencv_dnn is not used directly by rtabmap (not sure what is this library).
Asked by matlabbe on 2019-09-22 16:52:05 UTC
I went back to a fresh raspbian image, and reran my installer script (I've automated almost everything). This time I'm putting ROS Kinetic back on too. I got to a point where rtabmap launches but doesn't bring up a GUI, and GDB pointed it at a .so file that's a part of GTK 2, which is weird since I'm pretty sure I didn't need it and I had GTK 3 installed too.
After getting rid of GTK 2, rtabmap worked.
But right now I'm battling a new problem, it seems like if I don't build libpcl from source, it pulls in libboost 1.67 and having both 1.58 and 1.67 seems to be causing a problem inside of rtabmap, I can get 3 frames from the camera before it complains about https://github.com/introlab/rtabmap/issues/382
I'm going to spend another night, this time building libpcl from scratch on top of libboost 1.58 instead of having apt pull in libboost 1.67 , I'll get rid of VTK 7 too (giving warnings)
Since I'm using the ROS Kinetic build, there's no libopencv_dnn since opencv is at 3.3, not 3.4
Asked by frank26080115 on 2019-09-23 12:12:01 UTC
Thanks for all the help so far, consider this topic kinda closed since the error I'm getting now is kind of already known. If I need more help with it, I'll probably open a new topic
Asked by frank26080115 on 2019-09-23 12:13:12 UTC
Comments