ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question

How to assign an image_message to a transformation

asked 2021-11-10 10:35:05 -0500

max_ros gravatar image

Hi dear TF community.

I am publishing two image messages, "depth_image" and "infared_image" from two cameras. My goal is to register the depth image on the ir img. I have made the external calibration, and do publish it using the static_transform_publisher. The internal calibration parameters are published from the camera nodes. I am visualizing the images in rviz, and do see the two frames I created (depth_camera and ir_camera). Now if I switch in rviz between the frames, nothing happens to the images. My guess is, that I haven't assigned the frames to the cameras.

So... If my guess is right, how would I assign a frame to a camera node? Or if not, do you maybe have an idea what is missing for the registration process?

Thanks, Josh.

edit retag flag offensive close merge delete


Hi @max_ros take a look at prior question:

Yes you will need to assign transform for each camera

osilva gravatar image osilva  ( 2021-11-12 12:19:25 -0500 )edit

Hi @osilva. Thanks. I followed the hints in the answer you linked and am now on the following status:

  • all images are published with their corresponding frame_id
  • The transformation between the frames is published as static transform
  • In rviz, when opening tf, there are no errors or warnings as all transformations are known.
  • Really I have two cameras (ir camera and realsense d435) but three images, as thre realsenes camera is an RGB-D camera.

However I still haven't archieved my goal, which is to overlay the ir_image on the rgb image (or on the pointcloud, doesn't matter). Do you have a hint for me how to archive this? Do i need to apply the tranfsormations to the images manually? (I though rviz takes care of that for me?).

max_ros gravatar image max_ros  ( 2021-12-22 03:34:53 -0500 )edit

Hi @max_ros check if this is what you are trying to do:

osilva gravatar image osilva  ( 2021-12-22 04:57:34 -0500 )edit
osilva gravatar image osilva  ( 2021-12-22 05:02:21 -0500 )edit

Hi @osilva - yes this was again a very good hint, thanks!

However I still haven't archieved my goal :-D I now use the depth_image_proc register node, which is supposed to map my depth map on my infrared image. From there on I could also project both on a pointcloud. This in general works fine, except that I only do receive black images. So I strongly guess something is wrong with my calibration. However I do have troubles to debug this. Do you by chance also have a hint how to proceed (is is the internal calibration of the infrared camera, is it the external calibration, ...)?

max_ros gravatar image max_ros  ( 2021-12-28 08:32:29 -0500 )edit

Hi @max_ros I suggest we wrap this question with a partial answer based on the progress you made so far. Document your effort so others may benefit and start a new question with the new things that are still unresolved. This way more people will have visibility to help as well. Also with a new question, you could more pictures to help visualize what you are dealing with.

osilva gravatar image osilva  ( 2021-12-28 08:41:34 -0500 )edit
max_ros gravatar image max_ros  ( 2021-12-29 11:31:59 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted

answered 2021-12-29 11:16:02 -0500

max_ros gravatar image

updated 2021-12-29 11:16:52 -0500

Ok, as @osilva suggested, here a summary how I solved it:

I have the following topics published:

  • depth/image_raw
  • depth/camera_info (conatining the internal calbiration parameter and the frame id of the depth camera)
  • ir/camera_info (conatining the internal calbiration parameter and the frame id of the ir camera)
  • tf2 static tranform between the two frames ids.

Using depth_image_proc package regsiter, I can now map the depth data in the ir frame. My launch file looks like this:

  <arg name="namespace_ir" default="gas_camera"/>
  <arg name="namespace_vis" default="realsense"/>

  <!-- gas_cam --> 
  <include file="$(find gas_cam)/launch/gas_cam.launch"> 
        <arg name="namespace" value="$(arg namespace_ir)" />

  <!-- realsense --> 
    <include file="$(find realsense2_camera)/launch/rs_camera.launch"  ns="$(arg namespace_vis)"> 
        <arg name="filters" value="pointcloud" />
        <arg name="tf_prefix" value="rs" />

  <!-- tf2 -->   
    <node pkg="tf2_ros" type="static_transform_publisher" name="rs_2_ir" args="-0.062249 -0.083060 0.006821 0.00465763 0.052355 0.02046292  rs_link ir_kam_frame" />

  <!-- stereo proc for image registration -->
    <node pkg="nodelet" type="nodelet" name="nodelet_manager" args="manager" />
    <node pkg="nodelet" type="nodelet" name="nodelet1" args="load depth_image_proc/register nodelet_manager">
        <remap from="depth/camera_info" to="/$(arg namespace_vis)/camera/depth/camera_info"/>
        <remap from="rgb/camera_info" to="/$(arg namespace_ir)/camera_info"/>
        <remap from="depth/image_rect" to="/$(arg namespace_vis)/camera/depth/image_rect_raw"/>

    <!-- RVIZ -->
    <node pkg="rviz" type="rviz" name="rviz" args="-d $(find gas_cam)/rviz/default_visualization.rviz"/>
edit flag offensive delete link more

Question Tools


Asked: 2021-11-10 10:35:05 -0500

Seen: 119 times

Last updated: Dec 29 '21