Revision history [back]

OK, I think I sorted this, for anyone else in this situation. Basically the intrinsics between the depth and rgb cameras does not match, resulting in misalignment between depth and rgb images. I'm using the Realsense Gazebo plugin in ROS. Normally you are supposed to set the flag align_depth=true for the real camera, however the simulated one does not have this option as far as I can see. So instead, I went in and set the FOV of the RGB camera to the same as the depth cameras in the realsense's model.sdf:

<!--horizontal_fov>1.211</horizontal_fov-->
 <horizontal_fov>1.571</horizontal_fov>

Then I set the rtab_map's camera info topic to the depth cam info topic:

<arg name="camera_info_topic" value="/camera/camera/depth/camera_info"/>

Now the depth and color images are aligned and I have a much nicer result:

Note that I am using the mavros provided odom topic (calculated by the FCU/PX4). I have never seen anyone else use thei topic for RTAB_MAP, but then again, there are really not many examples using RTAB_MAP and PX4 that I can find. Ultimately I want to simulate flight in an indoor, non-gps environment, but I had to turn on the GPS in the simulation, otherwise the drone will not hold position (even though I have an optical flow sensor on the model). In real life I can hold position ok without GPS. Maybe someone out there has the solution for this.

OK, I think I sorted this, for anyone else in this situation. Basically the intrinsics between the depth and rgb cameras does not match, resulting in misalignment between depth and rgb images. I'm using the Realsense Gazebo plugin in ROS. Normally you are supposed to set the flag align_depth=true for the real camera, however the simulated one does not have this option as far as I can see. So instead, I went in and set the FOV of the RGB camera to the same as the depth cameras in the realsense's model.sdf:

<!--horizontal_fov>1.211</horizontal_fov-->
 <horizontal_fov>1.571</horizontal_fov>

Then I set the rtab_map's camera info topic to the depth cam info topic:

<arg name="camera_info_topic" value="/camera/camera/depth/camera_info"/>

Now the depth and color images are aligned and I have a much nicer result:

Note that I am using the mavros provided odom topic (calculated by the FCU/PX4). I have never seen anyone else use thei this odom topic for RTAB_MAP, but then again, there are really not many examples using RTAB_MAP and PX4 that I can find. Ultimately I want to simulate flight in an indoor, non-gps environment, but I had to turn on the GPS in the simulation, otherwise the drone will not hold position (even though I have an optical flow sensor on the model). In real life I can hold position ok without GPS. Maybe someone out there has the solution for this.

OK, I think I sorted this, for anyone else in this situation. Basically the intrinsics between the depth and rgb cameras does not match, resulting in misalignment between depth and rgb images. I'm using the Realsense Gazebo plugin in ROS. Normally you are supposed to set the flag align_depth=true for the real camera, however the simulated one does not have this option as far as I can see. So instead, I went in and set the FOV of the RGB camera to the same as the depth cameras in the realsense's model.sdf:

<!--horizontal_fov>1.211</horizontal_fov-->
 <horizontal_fov>1.571</horizontal_fov>

Then I set the rtab_map's camera info topic to the depth cam info topic:

<arg name="camera_info_topic" value="/camera/camera/depth/camera_info"/>

Now the depth and color images are aligned and I have a much nicer result:

Note that I am using the mavros provided odom topic (calculated by the FCU/PX4). I have never seen anyone else use this odom topic for RTAB_MAP, but then again, there are really not many examples using RTAB_MAP and PX4 that I can find. Ultimately I want to simulate flight in an indoor, non-gps environment, but I had to turn on the GPS in the simulation, otherwise the drone will not hold position (even though I have an optical flow sensor on the model). In real life I can hold position ok without GPS. Maybe someone out there has the solution for this.