rtabmap stops recording images after 7-10 seconds

asked 2019-04-30 11:14:34 -0500

pratikparwatwar gravatar image

I am having on ROS Kinetic on Ubuntu 16.04. I am trying to run Kinect 360 on ROS to get regular as well as depth images. I have installed freenect and rtabmap using following commands.

sudo apt-get install -y ros-kinetic-freenect-camera ros-kinetic-freenect-launch
sudo apt-get install -y ros-kinetic-freenect-stack ros-kinetic-libfreenect
sudo apt-get install ros-kinetic-rtabmap-ros

Now, in CASE 1. running roscore in 1st terminal then launching freenect in 2nd terminal.

roscore
roslaunch freenect_launch freenect.launch depth_registration:=true

In 3rd terminal launching rtabmap.

roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="--delete_db_on_start"enter code here

the rtabmap window opens but it doesn't record any images and in the freenect terminal window the following oputput repeats.

[ INFO] [1556638758.135309207]: Stopping device RGB and Depth stream flush.
[ INFO] [1556638780.594793009]: Device timed out. Flushing device.
[ INFO] [1556638780.595096944]: Starting a 3s RGB and Depth stream flush.

and in rtabmap terminal the following warnings repeat.

[WARN] [1556639251.666465373]: /rtabmap/rgbd_odometry: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. 
/rtabmap/rgbd_odometry subscribed to (approx sync):
   /camera/rgb/image_rect_color,
   /camera/depth_registered/image_raw,
   /camera/rgb/camera_info
[ WARN] [1556639252.656543038]: /rtabmap/rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=10).
/rtabmap/rtabmap subscribed to (approx sync):
   /rtabmap/odom,
   /camera/rgb/image_rect_color,
   /camera/depth_registered/image_raw,
   /camera/rgb/camera_info,
   /rtabmap/odom_info

Now in CASE2

running roscore in 1st terminal

roscore

and launching rtabmap 2nd.

roslaunch rtabmap_ros rtabmap.launch rtabmap_args:="--delete_db_on_start"

and then in 3rd terminal launching freenect

roslaunch freenect_launch freenect.launch depth_registration:=true

rtabmap window opens up and starts recording images for approximately 7-10 seconds and then stops reconding new images. Now in freenect terminal the following error occurs

[INFO] [1556639771.038578426]: Stopping device RGB and Depth stream flush.
terminate called after throwing an instance of 'std::runtime_error'
  what():  freenect_process_events error
[camera/camera_nodelet_manager-1] process has died [pid 7765, exit code -6, cmd /opt/ros/kinetic/lib/nodelet/nodelet manager __name:=camera_nodelet_manager __log:=/home/pratik/.ros/log/68acbe86-6b59-11e9-a105-70778130b1b9/camera-camera_nodelet_manager-1.log].
log file: /home/pratik/.ros/log/68acbe86-6b59-11e9-a105-70778130b1b9/camera-camera_nodelet_manager-1*.log

and the output of rtabmap terminal

[ INFO] [1556639780.054831440]: rtabmap (8): Rate=1.00s, Limit=0.000s, RTAB-Map=0.0499s, Maps update=0.0008s pub=0.0004s (local map=1, WM=1)
[ INFO] [1556639780.158713000]: Odom: quality=83, std dev=0.006837m|0.057880rad, update time=0.127539s
[ INFO] [1556639780.256065956]: Odom: quality=77, std dev=0.008145m|0.062481rad, update time=0.094164s
[ INFO] [1556639780.362580483]: Odom: quality=90, std dev=0.004738m|0.057249rad, update time=0.100875s
[ INFO] [1556639780.462297168]: Odom: quality=75, std dev=0.005483m|0.050833rad, update time=0.096905s
[ INFO] [1556639780.565689327]: Odom: quality=74, std dev=0.007174m|0.072406rad, update time ...
(more)
edit retag flag offensive close merge delete

Comments

USB problem? Try another USB. In both cases it seems that freenect is not able to publish steadily the image stream, even crash. Verify freenect stream is stable at ~30Hz with: $ rostopic hz /camera/rgb/image_rect_color /camera/depth_registered/image_raw /camera/rgb/camera_info

matlabbe gravatar imagematlabbe ( 2019-04-30 11:43:18 -0500 )edit

output for

$ rostopic hz /camera/rgb/image_rect_color /camera/depth_registered/image_raw /camera/rgb/camera_info

is

    topic                   rate   min_delta   max_delta   std_dev    window
======================================================================================
/camera/rgb/image_rect_color         30.03   0.02242     0.04972     0.004047   120   
/camera/depth_registered/image_raw   29.97   0.02196     0.04733     0.003585   120   
/camera/rgb/camera_info              30.03   0.02141     0.04624     0.00346    120

for several times and then stops and following output repeats.

no new messages
no new messages
no new messages
no new messages
no new messages

This happens exactly after the case2 error occurs. I have tried each USB port but same error occurs.

pratikparwatwar gravatar imagepratikparwatwar ( 2019-04-30 12:10:34 -0500 )edit

You could also try with openni_launch package instead of freenect_launch to see if it more stable on your system.

matlabbe gravatar imagematlabbe ( 2019-04-30 12:15:24 -0500 )edit

with openni the rtabmap window ran for 1 second and then in openni terminal the following error occured

Image: PrimeSense/SensorV2/5.1.0.41: Got a timeout while waiting for a network command to complete!

[FATAL] [1556644847.738970767]: Failed to load nodelet '/camera/disparity_registered_sw` of type `depth_image_proc/disparity` to manager `camera_nodelet_manager'
[FATAL] [1556644847.739080679]: Failed to load nodelet '/camera/points_xyzrgb_hw_registered` of type `depth_image_proc/point_cloud_xyzrgb` to manager `camera_nodelet_manager'
[FATAL] [1556644847.739516981]: Failed to load nodelet '/camera/disparity_depth` of type `depth_image_proc/disparity` to manager `camera_nodelet_manager'
[FATAL] [1556644847.739584269]: Failed to load nodelet '/camera/depth_registered_rectify_depth` of type `image_proc/rectify` to manager `camera_nodelet_manager'

And this error repeats for several times.

pratikparwatwar gravatar imagepratikparwatwar ( 2019-04-30 12:26:27 -0500 )edit

and case1 error happens in rtabmap terminal

[ WARN] [1556645297.540155135]: /rtabmap/rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). If topics are not published at the same rate, you could increase "queue_size" parameter (current=10).
/rtabmap/rtabmap subscribed to (approx sync):
   /rtabmap/odom,
   /camera/rgb/image_rect_color,
   /camera/depth_registered/image_raw,
   /camera/rgb/camera_info,
   /rtabmap/odom_info
pratikparwatwar gravatar imagepratikparwatwar ( 2019-04-30 12:29:58 -0500 )edit

The problem is the camera driver, you should debug this before trying rtabmap. I think you can reproduce the problem without starting rtabmap, just by doing the following and wait a couple of seconds:

$ roslaunch freenect_launch freenect.launch depth_registration:=true
$ rostopic hz /camera/rgb/image_rect_color /camera/depth_registered/image_raw /camera/rgb/camera_info

If it is the case, you could open a new question on why freenect and openni fail with your camera.

matlabbe gravatar imagematlabbe ( 2019-04-30 19:14:09 -0500 )edit

Yes you are right. the above commands reproduce the same issue. when you are saying camera driver issue. So what can the issue be with Kinect driver? or USB?. Also I''ll be putting up a question regarding 'why freenect and openni fail with my camera'. Thanks for showing patience.

pratikparwatwar gravatar imagepratikparwatwar ( 2019-05-01 01:51:12 -0500 )edit

At this point, it could be the kinect version (see under the kinect), usb, or something else about the camera driver.

matlabbe gravatar imagematlabbe ( 2019-05-01 10:55:23 -0500 )edit