Robotics StackExchange | Archived questions

Astra Pro registered point cloud

Hi all. I am using Ubuntu 16.04 with ROS kinetic. Fresh install with upgrades for both.

I have purchased an Astra Pro unit (unknowingly that it has completely separate RGB and depth units inside.... its a shame they don't tell that before you buy one).

I have installed their driver as per instructed, with udev rules and everything. When I launch the astrapro.launch file I can see the depth point cloud and depth image in Rviz. The Astra driver does not publish a color image but I am able to watch it through the usbcam node so it is operational. The thing is, that the astra driver has a "/camera/depthregistered/points" topic, but echoing it results in nothing. The terminal running the launch pops up the message:

[ INFO] [1544185697.502299836]: Starting depth stream. [ INFO] [1544185697.698366710]: camera calibration URL: file:///home/steve/catkinws/src/camerainfo/depthAstraOrbbec.yaml

I did make a calibration file for both the rgb and the depth, put them in the default ${ROSHOME}/camerainfo/${NAME.yaml}, and added the paths to the astrapro launch file.

I have tried various launch argument combinations and solutions from the internet but with no avail. Orbbec did not answer mails as well.

Am I missing something here? Do I need a third node to match the RGB and D? All I want is to be able to view a registered pointcloud in Rviz using the Astra Pro, even if a workaround is needed. Don't care about synchronization delay.

Hope I'm not the only one having this exact problem. Thank you all in advance.

Asked by StevenCoral on 2018-12-07 07:49:28 UTC

Comments

Answers

Sorry to be answering my own question but I found the solution, and in the absence of other instructions I want to make it available to other users ever encountering the same problem.

The Astra Pro launch file loads the Astra driver (publishing the depth and IR images only). It uses a third party node called rgbd_launch found here along with partial reference info. This node does most of the real job - listening to depth and RGB images to create point clouds and registered point clouds.

In order to do so, everything has to be aligned. In the astrapro launch file, the nodes are published under the /camera namespace. The default namespaces for the depth, IR and RGB publications are as depicted, and are the same for both the astra driver and rgbd_launch nodes. What needs to be done (in addition to the short instructions given at the orbbec ros astra wrapper git ) is as follows:

  1. Create both an RGB and depth calibration files because rgbd_launch has to rectify the data before registering. Default location for these files, so I've heard, is in the ${ROS_HOME}/camera_info/${NAME.yaml}, so I actually created and empty package named camera_info and put the files in the root directory (so that I could use $(find camera_info) in the launch file. In these files, see to it that the name of the cameras are: depth_Astra_Orbbec / rgb_Astra_Orbbec respectively, I don't remember why anymore but I got an error about it from one of the launched nodes / nodelets.

  2. Launch a node that would publish an image topic named /rgb/image_raw and the camera calibration info (taken from the calibration files) into /rgb/camera_info, both have to be in the same namespace where the rgbd_launch node is running (because it's listening to it). The images must be raw, not rectified. Also, the frame_id in the header must match the one used in the camera driver. In our case, the usb_camera node publishes the topics as /camera/rgb/image_raw and /camera/rgb/camera_info ("camera" because of the namespace they are all running at, and "rgb" because thats what we chose for the rgb image namespace). The frame id is the default one, "camera_rgb_optical_frame".

  3. Make sure the following parameters for rgbd_launch are true: "rgb_processing", "depth_processing", "sw_registered_processing" and "depth_registered_processing". Also make sure that "hw_registered_processing" is false, I received errors stating that the device is not calibrated. Perhaps I'll dig into it, maybe it is possible to upload a calibration file to the device flash? maybe this is unsupported? who knows. Also not sure how the pipeline would work if it did. BTW, other processing flags in rgbd_launch seems to have little to no connection with the publishing registered pointclouds.

This creates a namespace where /depth/camera_info, /depth/image_raw, /rgb/camera_info, and /rgb/image_raw are all published together along with a running rgbd_launch node, listening to them and publishing (among others) the /detph/points and /depth_registered/points topics.

EDIT: This modified-default launch file should give an example of how to implement:

<launch>

  <!-- "camera" should uniquely identify the device. All topics are pushed down
       into the "camera" namespace, and it is prepended to tf frame ids. -->
  <arg name="camera" default="camera" />
  <arg name="rgb_frame_id"   default="$(arg camera)_rgb_optical_frame" />
  <arg name="depth_frame_id" default="$(arg camera)_depth_optical_frame" />

  <!-- device_id can have the following formats:
         "#n"            : the nth device found, starts from 1
         "2@n"           : the nth device on USB bus 2, n starts from 1
         "2bc5/0401@1/6" : uri in format <vendor ID>/<product ID>@<bus number>/<device number>
         "15120410023"   : serial number -->
  <arg name="device_id" default="#1" />
  <arg name="bootorder" default="0" />
  <arg name="devnums" default="1" />

  <!-- By default, calibrations are stored to file://${ROS_HOME}/camera_info/${NAME}.yaml,
       where ${NAME} is of the form "[rgb|depth]_[serial#]", e.g. "depth_B00367707227042B".
       See camera_info_manager docs for calibration URL details. -->
  <arg name="rgb_camera_info_url"   default="file://$(find camera_info)/rgb_Astra_Orbbec.yaml" />
  <arg name="depth_camera_info_url" default="file://$(find camera_info)/depth_Astra_Orbbec.yaml" />

  <!-- Hardware depth registration -->
  <arg name="depth_registration" default="false" />

  <!-- Driver parameters -->
  <arg name="color_depth_synchronization"     default="false" />
  <arg name="auto_exposure"                   default="true" />
  <arg name="auto_white_balance"              default="true" />

  <!-- Arguments for remapping all device namespaces -->
  <arg name="rgb"              default="rgb" />
  <arg name="ir"               default="ir" />
  <arg name="depth"            default="depth" />

  <!-- Optionally suppress loading the driver nodelet and/or publishing the default tf
       tree. Useful if you are playing back recorded raw data from a bag, or are
       supplying a more accurate tf tree from calibration. -->
  <arg name="load_driver" default="true" />
  <arg name="publish_tf" default="true" />
  <!-- Processing Modules -->
  <arg name="rgb_processing"                  default="true"  />
  <arg name="debayer_processing"              default="false" />
  <arg name="ir_processing"                   default="false" />
  <arg name="depth_processing"                default="true" />
  <arg name="depth_registered_processing"     default="true" />
  <arg name="disparity_processing"            default="false" />
  <arg name="disparity_registered_processing" default="false" />
  <arg name="hw_registered_processing"        default="true" if="$(arg depth_registration)" />
  <arg name="sw_registered_processing"        default="false" if="$(arg depth_registration)" />
  <arg name="hw_registered_processing"        default="false" unless="$(arg depth_registration)" />
  <arg name="sw_registered_processing"        default="true" unless="$(arg depth_registration)" />

  <!-- Disable bond topics by default -->
  <arg name="respawn" default="false" />

  <!-- Worker threads for the nodelet manager -->
  <arg name="num_worker_threads" default="4" />

  <!-- Push down all topics/nodelets into "camera" namespace -->
  <group ns="$(arg camera)">

        <!-- Start nodelet manager -->
        <arg name="manager" value="$(arg camera)_nodelet_manager" />
        <arg name="debug" default="false" /> <!-- Run manager in GDB? -->
        <include file="$(find rgbd_launch)/launch/includes/manager.launch.xml">
          <arg name="name" value="$(arg manager)" />
          <arg name="debug" value="$(arg debug)" />
          <arg name="num_worker_threads"  value="$(arg num_worker_threads)" />
        </include>

    <!-- Load driver -->
    <include file="$(find astra_launch)/launch/includes/device.launch.xml" if="$(arg load_driver)">
      <arg name="manager"                         value="$(arg manager)" />
      <arg name="device_id"                       value="$(arg device_id)" />
      <arg name="bootorder"                       value="$(arg bootorder)" />
      <arg name="devnums"                         value="$(arg devnums)" />
      <arg name="rgb_frame_id"                    value="$(arg rgb_frame_id)" />
      <arg name="depth_frame_id"                  value="$(arg depth_frame_id)" />
      <arg name="rgb_camera_info_url"             value="$(arg rgb_camera_info_url)" />
      <arg name="depth_camera_info_url"           value="$(arg depth_camera_info_url)" />
      <arg name="rgb"                             value="$(arg rgb)" />
      <arg name="ir"                              value="$(arg ir)" />
      <arg name="depth"                           value="$(arg depth)" />
      <arg name="respawn"                         value="$(arg respawn)" />
      <arg name="depth_registration"              value="$(arg depth_registration)" />
      <arg name="color_depth_synchronization"     value="$(arg color_depth_synchronization)" />
      <arg name="auto_exposure"                   value="$(arg auto_exposure)" />
      <arg name="auto_white_balance"              value="$(arg auto_white_balance)" />
    </include>

        <node name="rgb" pkg="usb_cam" type="usb_cam_node" output="screen" >
          <param name="video_device" value="/dev/video1" />
          <param name="image_width" value="640" />
          <param name="image_height" value="480" />
          <param name="camera_frame_id" value="$(arg rgb_frame_id)" />
          <param name="framerate" value="30"/>
            <param name="pixel_format" value="yuyv"/>
            <param name="camera_info_url" value="$(arg rgb_camera_info_url)"/>
            <param name="camera_name" value="rgb_Astra_Orbbec"/>
        </node>

    <!-- Load standard constellation of processing nodelets -->
    <include file="$(find rgbd_launch)/launch/includes/processing.launch.xml">
      <arg name="manager"                         value="$(arg manager)" />
      <arg name="rgb"                             value="$(arg rgb)" />
      <arg name="ir"                              value="$(arg ir)" />
      <arg name="depth"                           value="$(arg depth)" />
      <arg name="respawn"                         value="$(arg respawn)" />
      <arg name="rgb_processing"                  value="$(arg rgb_processing)" />
      <arg name="debayer_processing"              value="$(arg debayer_processing)" />
      <arg name="ir_processing"                   value="$(arg ir_processing)" />
      <arg name="depth_processing"                value="$(arg depth_processing)" />
      <arg name="depth_registered_processing"     value="$(arg depth_registered_processing)" />
      <arg name="disparity_processing"            value="$(arg disparity_processing)" />
      <arg name="disparity_registered_processing" value="$(arg disparity_registered_processing)" />
      <arg name="hw_registered_processing"        value="$(arg hw_registered_processing)" />
      <arg name="sw_registered_processing"        value="$(arg sw_registered_processing)" />
    </include>

  </group> <!-- camera -->

  <!-- Load reasonable defaults for the relative pose between cameras -->
  <include file="$(find astra_launch)/launch/includes/astra_frames.launch" if="$(arg publish_tf)">
    <arg name="camera" value="$(arg camera)" />
  </include>

</launch>

Cheers, Steve

Asked by StevenCoral on 2018-12-08 09:59:02 UTC

Comments

No need to apologise for answering your own question: it's perfectly ok to do that.

Asked by gvdhoorn on 2018-12-08 11:30:08 UTC

I'm trying to attach my launch file as a reference but the xml syntax mixes with the forum's..... advice any1?

Paste the launch file in your answer, select all lines of xml and press the Preformatted Text button (the one with 101010 on it). Or press ctrl+k. Done.

Asked by gvdhoorn on 2018-12-08 11:30:59 UTC

One time I did that and someone stated that I should write a comment instead... not really enough space for that. Thanks for the help, I was trying the quote thing and it didn't work. All done.

Asked by StevenCoral on 2018-12-08 13:32:18 UTC

Thank guy, you solve my problem which confuse me constantly.

Asked by huoxingwen on 2019-12-26 23:02:48 UTC