Robotics StackExchange | Archived questions

Intel Realsense D400 in Gazebo simulation

Im trying to use intel D400 with gazebo simulation on ROS Kinetic / Ubuntu 16.04. So far I have been using the OpenNI Kinect plugin (libgazeborosopenni_kinect.so).

I found there is a Realsense plugin for Gazebo (librealsensegazeboplugin.so). I am not sure how to replace the openni_kinect plugin with it in my URDF file, considering that the Realsense has two infrared cameras and an RGB camera.

I want to experiment with the Realsense in simulation before shifting to the real camera. I would like to work with depth images and point clouds for my application. First by using the depthimagetolaserscan package and later the point clouds for obstacle avoidance. Thanks in advance.

Here is how I currently use the OpenNI Kinect plugin in URDF:

            <link name="camera_link">
          <visual>
            <origin xyz="0.0 0.0 0.004" rpy="0 0 0"/>
            <geometry>
              <mesh filename="package://evarobotmodel_description/meshes/kinect.dae" scale="0.9 0.9 0.9"/>
            </geometry>
          </visual>
          <collision>
            <origin xyz="0.0 0.0 0.0" rpy="0 0 0" />
            <geometry>
                <box size="0.0730 0.2760 0.0720"/>
            </geometry>
          </collision>
          <inertial>
            <mass value="0.170" />
            <origin xyz="0 0 0" />
            <inertia ixx="0.001152600" ixy="0.0" ixz="0.0"
                     iyy="0.000148934" iyz="0.0"
                     izz="0.001154654" />
          </inertial>
        </link>
        <joint name="camera_rgb_joint" type="fixed">
          <origin xyz="0.0 -0.02 0.0" rpy="0.0 0.0 0.0"/>
          <parent link="camera_link"/>
          <child link="camera_rgb_frame" />
        </joint>
        <gazebo reference="camera_link">
            <material>Gazebo/FlatBlack</material>
        </gazebo>
        <link name="camera_rgb_frame"/>
        <joint name="camera_rgb_optical_joint" type="fixed">
          <origin xyz="0 0 0" rpy="-1.57079632679 0 -1.57079632679" />
          <parent link="camera_rgb_frame" />
          <child link="camera_rgb_optical_frame" />
        </joint>
        <link name="camera_rgb_optical_frame"/>
        <joint name="camera_joint" type="fixed">
          <origin xyz="0.17 0.0 0.1913" rpy="0.0 0.0 0.0"/>
          <parent link="base_link"/>
          <child link="camera_link"/>
        </joint>
        <joint name="camera_depth_joint" type="fixed">
          <origin xyz="0 -0.045 0" rpy="0 0 0" />
          <parent link="camera_link" />
          <child link="camera_depth_frame" />
        </joint>
        <link name="camera_depth_frame"/>
        <joint name="camera_depth_optical_joint" type="fixed">
          <origin xyz="0 0 0" rpy="-1.57079632679 0 -1.57079632679" />
          <parent link="camera_depth_frame" />
          <child link="camera_depth_optical_frame" />
        </joint>
        <link name="camera_depth_optical_frame"/>
         <!--  Asus mount  -->
        <joint name="mount_asus_xtion_pro_joint" type="fixed">
          <origin xyz="0.1685 -0.0050 0.1563" rpy="1.57079632679 0 -1.57079632679"/>
          <parent link="base_link"/>
          <child link="mount_asus_xtion_pro_link"/>
        </joint>  
        <link name="mount_asus_xtion_pro_link">
<!--          <visual>
            <origin xyz="0 0 0" rpy="0 0 0"/>
            <geometry>
              <mesh filename="package://evarobotmodel_description/meshes/asus_base.dae"/>
            </geometry>
            <material name="Black"/>
          </visual>  -->
          <collision>
            <origin xyz="0.0 0.0 0.0" rpy="0 0 0" />
          <geometry>
            <box size="0.2760 0.0330 0.0120"/>
          </geometry>
          </collision>
          <inertial>
            <mass value="0.170" />
            <origin xyz="0 0 0" />
            <inertia ixx="0.001152600" ixy="0.0" ixz="0.0"
                     iyy="0.000148934" iyz="0.0"
                     izz="0.001154654" />
          </inertial>
        </link>
        <gazebo reference="mount_asus_xtion_pro_link">
            <material>Gazebo/FlatBlack</material>
        </gazebo>
        <gazebo reference="camera_link">  
          <sensor type="depth" name="camera">
            <always_on>true</always_on>
            <update_rate>20.0</update_rate>
            <camera>
              <horizontal_fov>1.0471975512</horizontal_fov>
              <image>
                <format>R8G8B8</format>
                <width>640</width>
                <height>480</height>
              </image>
              <clip>
                <near>0.05</near>
                <far>8.0</far>
              </clip>
            </camera>
            <plugin name="kinect_camera_controller" filename="libgazebo_ros_openni_kinect.so">
              <robotNamespace>/</robotNamespace>
              <cameraName>camera</cameraName>
              <alwaysOn>true</alwaysOn>
              <updateRate>10</updateRate>
              <imageTopicName>rgb/image_raw</imageTopicName>
              <depthImageTopicName>depth/image_raw</depthImageTopicName>
              <pointCloudTopicName>depth/points</pointCloudTopicName>
              <cameraInfoTopicName>rgb/camera_info</cameraInfoTopicName>
              <depthImageCameraInfoTopicName>depth/camera_info</depthImageCameraInfoTopicName>
              <frameName>camera_depth_optical_frame</frameName>
              <baseline>0.1</baseline>
              <pointCloudCutoff>0.4</pointCloudCutoff>
              <distortionK1>0.00000001</distortionK1>
              <distortionK2>0.00000001</distortionK2>
              <distortionK3>0.00000001</distortionK3>
              <distortionT1>0.00000001</distortionT1>
              <distortionT2>0.00000001</distortionT2>
              <CxPrime>0</CxPrime>
              <Cx>0</Cx>
              <Cy>0</Cy>
              <focalLength>0</focalLength>
             <hackBaseline>0</hackBaseline>
            </plugin>
          </sensor>

Asked by q8wwe on 2020-03-02 07:46:28 UTC

Comments

I cleaned your post, but I didn't understand everything. You should link the packages that you are using now, and where you are stuck with what you have tried. What about the second Google hit for "Realsense gazebo plugin"? Did you try it? Have you searched the questions on Gazebosim, like this one?

Asked by fvd on 2020-03-04 04:42:52 UTC

Thank you for the support , after many attempt I was able to make the simulation work with following the link :https://github.com/pal-robotics/realsense_gazebo_plugin/tree/kinetic-devel

after success I tested it with depthimage _to_laser but it didn't work "Could not convert depth image to laserscan: Cannot call rectifyPoint when distortion is unknown."

issue for the dpthimage_to_laser : https://github.com/SyrianSpock/realsense_gazebo_plugin/issues/51

for the problems I faced with realsense plugin I had to shift back to Kinect plugin.

if any solution please advice.

Asked by q8wwe on 2020-03-21 10:56:13 UTC

It would be great if you could post and accept what you wrote as an answer.

Asked by fvd on 2020-03-23 10:45:13 UTC

Answers

Hey i found out how to at least use D435. First download this package from https://github.com/pal-robotics/realsense_gazebo_plugin and put it into your catkinws src folder and then compile the catkin package. This will generate a shared library called librealsense_gazebo_plugin.so for you which is used in the files you gonna download in the next steps. you need to download the two xacro files https://github.com/pal-robotics-forks/realsense/tree/upstream/realsense2_description/urdf _d435.gazebo.xacro and _d435.urdf.xacro put those two files in the package inside the urdf folder where you want your realsense to be simulated then in the file _d435.urdf.xacro change the line 13 from to packagename with being the name of the package Then in the urdf file where you want to use the simulated realsense add the following code, while again replacing packagename with the name of the package you are using. and baselink which the link where you want the realsense camera to be joint on.

Afterwards when you launch file which uses this urdf model you will find the simulate realsense and you should see rostopics like /camera/color/image_raw

Furthermore in the file urdf _d435.gazebo.xacro file in line 139 you can change the parameter to true so you will get the point cloud you want.

Asked by arminkazim on 2020-04-04 09:59:51 UTC

Comments

Hi there.

I'm new about ROS and Gazebo. i'm trying to simulate the intel Realsense D435i into Gazebo for my master thesis, but following what you have said, I'm not able to obtain the camera model and to go on with the simulation. Can I ask you more datails about how you have solved this problem? I'm pretty sure that the problem in my case is at this step:

"Then in the urdf file where you want to use the simulated realsense add the following code, ... /xacro:include/xacro:include/xacro:include (why here there are 3 tags to close the xacro:include ?)"

The URDF file that you are talking about is the urdf file that contains also the desciption about the robot or is another urdf file? The camera model should appear in the list of available models of Gazebo or somewhere else?

I hope to have pointed out my doubts.

Thanks for you reply!

Asked by AndreSpa on 2020-04-29 17:42:35 UTC

Hello, is there something similar for D415 camera? I could not find any _d415 gazebo xacro file? From the intel specs, they seem to be quite different.

Asked by ThreeForElvenKings on 2020-06-01 05:49:49 UTC

Install https://github.com/pal-robotics/realsense_gazebo_plugin (melodic-devel branch), then you may use this sdf file https://gist.github.com/GPrathap/044fb6d601ccfad61e98fab6f4171e9a , Here I have some more stuff other than a just camera, hope you will able to figure it out. If not let's know, will help you..

Asked by GPrathap on 2020-06-25 09:09:51 UTC

Comments

Hi, I am able to add realsense camera to the iris drone but when i run the command

roslaunch realsense2_camera rs_camera.launch

it displays

[ WARN] [1628580407.165804390]: No RealSense devices were found!

I am not able to get what is the problem. And when i run the command rostopic list. I am able to see following topic

/camera/fisheye1/image_raw /camera/fisheye2/image_raw /camera/odom/sample /camera/odom/sample_throttled

but when i run rostopic echo /camera/odom/sample it shows

WARNING: topic [/camera/odom/sample] does not appear to be published yet

Asked by MWK on 2021-08-10 02:43:36 UTC