Astra Pro registered point cloud

edit

Sorry to be answering my own question but I found the solution, and in the absence of other instructions I want to make it available to other users ever encountering the same problem.

The Astra Pro launch file loads the Astra driver (publishing the depth and IR images only). It uses a third party node called rgbd_launch found here along with partial reference info. This node does most of the real job - listening to depth and RGB images to create point clouds and registered point clouds.

In order to do so, everything has to be aligned. In the astrapro launch file, the nodes are published under the /camera namespace. The default namespaces for the depth, IR and RGB publications are as depicted, and are the same for both the astra driver and rgbd_launch nodes. What needs to be done (in addition to the short instructions given at the orbbec ros astra wrapper git ) is as follows:

1. Create both an RGB and depth calibration files because rgbd_launch has to rectify the data before registering. Default location for these files, so I've heard, is in the ${ROS_HOME}/camera_info/${NAME.yaml}, so I actually created and empty package named camera_info and put the files in the root directory (so that I could use $(find camera_info) in the launch file. In these files, see to it that the name of the cameras are: depth_Astra_Orbbec / rgb_Astra_Orbbec respectively, I don't remember why anymore but I got an error about it from one of the launched nodes / nodelets.

2. Launch a node that would publish an image topic named /rgb/image_raw and the camera calibration info (taken from the calibration files) into /rgb/camera_info, both have to be in the same namespace where the rgbd_launch node is running (because it's listening to it). The images must be raw, not rectified. Also, the frame_id in the header must match the one used in the camera driver. In our case, the usb_camera node publishes the topics as /camera/rgb/image_raw and /camera/rgb/camera_info ("camera" because of the namespace they are all running at, and "rgb" because thats what we chose for the rgb image namespace). The frame id is the default one, "camera_rgb_optical_frame".

3. Make sure the following parameters for rgbd_launch are true: "rgb_processing", "depth_processing", "sw_registered_processing" and "depth_registered_processing". Also make sure that "hw_registered_processing" is false, I received errors stating that the device is not calibrated. Perhaps I'll dig into it, maybe it is possible to upload a calibration file to the device flash? maybe this is unsupported? who knows. Also not sure how the pipeline would work if it did. BTW, other processing flags in rgbd_launch seems to have little to no connection with the publishing registered pointclouds.

This creates a namespace where /depth/camera_info, /depth/image_raw, /rgb/camera_info, and /rgb/image_raw are all published together along with a running rgbd_launch node, listening to them and publishing (among others) the /detph/points and /depth_registered/points topics.

EDIT: This modified-default launch file should give an example of ... (more)