Ask Your Question
0

How to synchronize image and marker display between two ROS nodes

asked 2011-06-10 02:51:00 -0500

Pi Robot gravatar image

updated 2011-06-10 13:21:47 -0500

I have two ROS nodes, a "display node" and a "detector node", that both use cv_bridge and subscribe to the same image topic. The detector node publishes an array of ROI boxes around the objects it is designed to detect (e.g. faces). While I could draw these boxes on an OpenCV image window using the same node, I would like the boxes display along with potentially other markers from other detector nodes.

So I have a display node that listens on both the raw image topic and the detector topics and draws the appropriate boxes on a marker image that it then merges with the raw image topic and updates an OpenCV image display window.

The problem I am having is that the markers (e.g. boxes) significantly lag behind the raw image. For instance, if my detector node uses the Haar face detector, and I move my head, the face boxes trail my head movement by a very noticeable amount. (Maybe half of a second?)

I am guessing this is because the display node can update the raw image display much faster than the detector node can update the face detections. But I'm not sure what I can do about it or if this approach is even a good idea.

UPDATE: Well, I've run into a snag. I have no trouble with the CameraInfo Python example found here. However, I cannot get synchronization to work when trying to synchronize /camera/rgb/image_color (I'm using a Kinect and the openni_camera node) and a test publisher I wrote.

Here are the two test scripts I am using. The first script simply publishes a DetectionArray message about 30 times per second with nothing but the header.stamp field set to rospy.Time.now(). The second script subscribes to this topic as well as /camera/rgb/image_color. While the second script successfully prints out the value of header.stamp using the regular rospy.Subscriber callback, I never see the message "Synchronized" from the TimeSynchronizer callback. Looking at the timestamps manually using rostopic echo on the /camera/rgb/image_color and /detections topics shows that they are very close to being synchronized already so I thought this should work.

I must be missing something simple I can't see so any tips would be appreciated!

Here is the test_pub.py publisher script:

#!/usr/bin/env python

""" test_pub.py - Version 0.1 2011-06-10  
"""

import roslib
roslib.load_manifest('pi_rein')
import rospy
import sys
from std_msgs.msg import String
from pi_rein.msg import *
from pi_rein.srv import *
import message_filters

class TestSync:
    def __init__(self):
        rospy.init_node("test_pub")

        self.test_pub = rospy.Publisher("detections", DetectionArray)

        while not rospy.is_shutdown():
            msg = DetectionArray()
            msg.header.stamp = rospy.Time.now()
            self.test_pub.publish(msg)
            rospy.sleep(0.033)     

def main(args):   
    try:
        TestSync()
    except KeyboardInterrupt:
        print "Shutting down test pub node."

if __name__ == '__main__':
    main(sys.argv)

Here is the test_sync.py script:

#!/usr/bin/env python

""" test_sync.py - Version 0.1 2011-06-10  
"""

import roslib
roslib.load_manifest('pi_rein')
import ...
(more)
edit retag flag offensive close merge delete

Comments

Hi Patrick, do you know about this: http://www.ros.org/wiki/rein ? It seems to be a good framework for what you're doing.
Ugo gravatar imageUgo ( 2011-06-10 03:24:53 -0500 )edit
Thanks Ugo--in fact I am trying to do something similar to REIN using Python. (See my posting on ros-users at http://code.ros.org/lurker/message/20110609.184757.957cc335.en.html). And even in the exiting C++/Nodelet REIN framework, I don't see a way to display markers from multiple detector nodes on the same image. If I am just missing it, hopefully someone will point it out as it might help me solve my synchronization issue.
Pi Robot gravatar imagePi Robot ( 2011-06-10 03:54:53 -0500 )edit
My bad, sorry (in fact I discovered the REIN framework thanks to your post on the mailing list :) ).
Ugo gravatar imageUgo ( 2011-06-10 03:59:43 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2011-06-10 03:52:13 -0500

dornhege gravatar image

updated 2011-06-10 13:59:12 -0500

You should obey the timestamps. Your detector node sends out the appropriate boxes with the image timestamp.

The display node only displays image+markers from the same timestamp. A Time Synchronizer filter should be applicable for this.

Update: To make it work, try to set the timestamps exactly the same. I.e. for your application: The detector subscribes the image and sends out its detections with the image timestamp (not ros::Time::now()). The display subscribes image and detections for synchronization.

edit flag offensive delete link more

Comments

Thanks dornhege--this looks like exactly what I need!
Pi Robot gravatar imagePi Robot ( 2011-06-10 04:00:19 -0500 )edit
That was it! With the timestamp set to the image timestamp and the synchronizer queue set to 100, it works like a charm. Thanks again!
Pi Robot gravatar imagePi Robot ( 2011-06-10 14:07:47 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2011-06-10 02:51:00 -0500

Seen: 2,569 times

Last updated: Jun 10 '11