Smach and OpenCV (passing image data between states)

asked 2017-11-07 09:17:13 -0500

glukon gravatar image

updated 2017-11-07 09:30:26 -0500

Hi guys,

currently I'm working with the smach tutorial here:

I added a new class called getNewFrame where I subscribe to the camera's topic and save image files to filesystem. Now I want to add a new state for detecting a Marker within the frame. By using smach this means that I have to create a new class for the new state - done. Inside the class I need to do all the OpenCV stuff (load last frame, processing). Finally I want to have a state for determing the outcome like if is_marker_found == True: return outcome1 and so on. I think you got it.

But after trying I got a lot of errors... I can not pass the frame from one to another class like you would normally do in python. I'm not really into smach so I hope the community can tell me IF and HOW I can use smach to pass OpenCV Image data so a state called "get new frame" can pass the frame to the next state "detectMarker".

Thanks and hope you can help me :-) Have a nice day!

This code will not be executable without editing the placeholder #####

# define state getNewFrame
class getNewFrame(smach.State):
    def __init__(self):
        smach.State.__init__(self, outcomes=['haveNewFrame', 'haveNotFrame'])

        self.image_topic = "/camera/image_raw"
        rospy.Subscriber(self.image_topic, Image, self.image_callback)

        self.is_newFrame = False
        # Instantiate CvBridge
        self.bridge = CvBridge()
        lst_frame = []

    def image_callback(self, msg):
            # Convert your ROS Image message to OpenCV2
            lst_frame = self.bridge.imgmsg_to_cv2(msg, "bgr8")
            self.is_newFrame = True
        except CvBridgeError, e:
            self.is_newFrame = False
            # Save your OpenCV2 image as a jpeg 
            cv2.imwrite('#####', lst_frame)

    def execute(self, userdata):
        rospy.loginfo('Executing state GNF')
        if self.is_newFrame == True:          
            return 'haveNewFrame'
            return 'haveNotFrame'

class #####(object):
    def __init__(self):
        self.i = 0

    def run(self):
        sm = smach.StateMachine(outcomes=['#####'])

        # Open the container
        with sm:

            smach.StateMachine.add('FMIF', findMarkerInFrame(), transitions={'markerFound':'GMPTS', 'markerNotFound':'GNF'})
            smach.StateMachine.add('GNF', getNewFrame(), transitions={'haveNewFrame':'FMIF', 'haveNotFrame':'GNF'})
            smach.StateMachine.add('GMPTS', getMarkerPts(), transitions={'haveMarkerPts':'AKB', 'haveNoMarkerPts':'GNF'})
            smach.StateMachine.add('AKB', alignKobukiToMarker(), transitions={'isAligned':'DF', 'isNotAligned':'GNF'})
            smach.StateMachine.add('DF', driveBaseForward(), transitions={'is_finishedDrive':'AKB', 'stop':'GNF'})

        sis = smach_ros.IntrospectionServer('server_name', sm, '/SM_ROOT')

        # Execute SMACH plan
        outcome = sm.execute()


if __name__ == '__main__':
     ##### = #####()
edit retag flag offensive close merge delete



It's completely up to you (as in: how you design your software), but from your example I get the impression that you are mixing computation (processing of data) with coordination (when processing should happen, and with what data). From an SE perspective (reuse, maintainability, ..

gvdhoorn gravatar image gvdhoorn  ( 2017-11-07 09:36:12 -0500 )edit

.. comprehensibility (and a load of other '-bilities')) that is not recommended.

If you reduce the responsibilities of your state machine to just pub/sub and calling the right services / actions at the right time, your states become much simpler, removing the need for all sorts of sub classes of ..

gvdhoorn gravatar image gvdhoorn  ( 2017-11-07 09:38:17 -0500 )edit

.. smach.State as it already has classes for pub/sub, service invocation and action-server interaction.

This is obviously not an answer, but perhaps something to consider if/when you start a new project.

gvdhoorn gravatar image gvdhoorn  ( 2017-11-07 09:41:19 -0500 )edit

hey, i already deleted this question cause I noticed that it was my fault - in more detail it was a syntax problem. I need to access between classes by doing frame = getNewFrame() and then frame.lst_frame in class #2 to access lst_frame of class #1

glukon gravatar image glukon  ( 2017-11-07 09:54:10 -0500 )edit

And well, I am learning ros on my own for a couple of month now. Hope I will be better soon :)

glukon gravatar image glukon  ( 2017-11-07 09:56:19 -0500 )edit

I would appreciate it if instead of deleting your question, you could just post what you discovered as an answer and accept your own answer. That way future readers can benefit from your experience just as you've benefitted from posters before you.

gvdhoorn gravatar image gvdhoorn  ( 2017-11-07 09:58:25 -0500 )edit

Irrespective of whether you've solved your immediate issue, my comment about mixing coordination with computation remains applicable.

gvdhoorn gravatar image gvdhoorn  ( 2017-11-07 09:59:03 -0500 )edit

Well you are right, with respect I undeleted this question. Can you give me some advice or tipps how to manage the coordination and the computation separately? From what you already said I get that it's needed to work only with pub/sub and services/ actions? Where can I find more to read?

glukon gravatar image glukon  ( 2017-11-08 03:48:03 -0500 )edit