Syncing/Merging camera and lidar data for a detected object

asked 2020-09-15 23:27:07 -0600

PKGosal
5 ●1 ●5 ●5 https://prabhjotkaurgosal.com/

I would like to use a camera to detect and classify an object. Further, I would like to use a lidar to get the distance to the detected object. How can I ensure that the distance I am getting from the lidar is for the object of interest detected/classified by the camera. Meaning how do I do the time syncing of the data published by these two separate sensors.

Btw, I do have a working robot that has a camera and lidar attached to it and I can see the data being published correctly.

Any direction would be helpful.

answered 2020-09-16 00:45:59 -0600

mgruhler

12390 ●32 ●193 ●173 https://github.com/mgruhler

Assuming the time-stamping in the drivers is correct, both are running on sufficient frequency (whatever that means for your application, i.e. any delays between the two nodes don't lead to significant errors; this is usually the case for high speed cases like drones) and you have your URDF/SDF set up properly so you can actually directly associated a detected object from the camera with the physical location w.r.t. the lidar, you can use the message_filters package to get this done. I suggest to use the ApproximateTime Policy.

edit flag offensive delete link

Comments

@mgruhler Thank you for pointing me to a source. In my case, everything implemented so far only exists in the simulation - I am using a gazebo simulator and I use the opensource camera and lidar plugins. I looked into the message_filters package and concur that ApproxiteTime Policy maybe the one I should start with since the messages published on the camera topic do not have a header from where time information could be extracted.... There is statement on the ROS wiki that says ..... "If some messages are of a type that doesn't contain the header field, ApproximateTimeSynchronizer refuses by default adding such messages. However, its Python version can be constructed with allow_headerless=True, which uses current ROS time in place of any missing header.stamp field ....... I take this means I will have to code in Python.

PKGosal ( 2020-09-16 22:34:07 -0600 )edit

Could you provide a link to the camera plugin you use? The default one should have a header, AFAIK.

mgruhler ( 2020-09-17 01:07:15 -0600 )edit

The plugin I am using is called "libgazebo_ros_camera.so". I believe this is the correct location of the file, (https://github.com/ros-simulation/gaz...) . . . Upon further investigation, it looks like the default plugin does publish the time stamp. I checked it with rosmsg show sensor_msgs/CameraInfo and then echoed the CameraInfo topic.

Next, I will work on using the message_filters package. I appreciate your help.

PKGosal ( 2020-09-17 18:51:17 -0600 )edit

great. If this answer was helpful, please consider marking it as solved by clicking the checkmark next to it. Thanks.

mgruhler ( 2020-09-18 01:46:26 -0600 )edit

add a comment

Syncing/Merging camera and lidar data for a detected object

1 Answer

Comments

Question Tools

Stats

Related questions

Syncing/Merging camera and lidar data for a detected object edit

1 Answer

Comments

Question Tools

Stats

Related questions

Syncing/Merging camera and lidar data for a detected object