ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question

An alternative to data processing inside callback functions?

asked 2016-02-24 16:28:23 -0500

ros_geller gravatar image

I want to perform calculations on the data from my subscriptions the instant it arrives. I could of course just do this inside the callback function, but i would insted like to perform these calculations outside of the callback function, right after it has returned. The callback function will then just pass the data to a queue which another function will use for processing.

My question is then: How can this be achieved?.

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted

answered 2016-02-25 08:13:45 -0500

kramer gravatar image

updated 2018-07-15 18:43:23 -0500

Edit: now I see that I can edit my answer, oops. Please make sure and read mitch722 's answer below! Original answer follows:

By moving the calculations outside of the callback (recommended, of course!), you're explicitly decoupling calculations from the callback mechanism, thereby necessarily losing the ability to begin processing the "instant it arrives". With that said, what you probably desire is an event system (that operates with its own event queue, not simply a subsidiary function callback). That's beyond the scope of this answer.

What you could do instead -- much easier, but likely uses a bit more CPU and is a bit less responsive -- is to use a combination of ROS Timer and your own data queue. In your data callback, put the data in your queue; in the timer callback, process an (likely all) element that is in the queue (if none, just return).

Note that by using a Timer, you're utilizing the ROS callback system. Assuming single threaded spinning, there's no need to worry about data synchronization. But note also that a typical spin() will process all messages in the callback queue, so special measures would need to be taken to process one message at a time (e.g., guarantee that there will always be a timer event between queued data messages).

edit flag offensive delete link more


Wouldn't simply pushing the incoming message into an application/node level queue and then assigning that queue to a (pool of) worker threads give you that "event system"? That isn't too complicated, and you also don't need to resort to implementing a polling system as you describe.

gvdhoorn gravatar image gvdhoorn  ( 2016-02-25 08:46:02 -0500 )edit

Yes, as you say, that's effectively an event system as I used the term. With no language specified, I assume C++, so shy away from recommending multi-threading. Plus, it still seems to me that a Timer of high enough frequency is the easiest method, albeit not the most efficient/timely.

kramer gravatar image kramer  ( 2016-02-26 10:43:04 -0500 )edit

Sorry for the late response: I am using C++, yes. After thinking about it, what I think i need here is something like a producer-consumer design. So I'm now wondering if there are ROS native functionalities to do this or if i have to do it from scratch? I should prob go read up on the design pattern

ros_geller gravatar image ros_geller  ( 2016-03-01 04:20:31 -0500 )edit

What I described is a producer-consumer. The publisher is the producer, the timer callback is the consumer. The issues are the delay and interleaving message reception/processing. Y'know, if polling is a concern, you could add a one-shot timer in the message callback.

kramer gravatar image kramer  ( 2016-03-02 07:50:05 -0500 )edit

answered 2018-06-29 14:44:45 -0500

I know this is a late answer but I'm adding it after doing a review of someones code and found they had implemented it blindly because it is "recommended, of course!" by kramer.

Why do you need to move calculations outside the callback? The only case it's really valid is if you are filtering messages. Ie you expect more messages than your module can/wants to process so you need to throw away some of them, in which case the subscriber message queue will fill up so you may not be processing the latest message.

In that case either add more AsyncSpinner threads to process your callback or have a calculation thread running to process your calculations. If you use multiple AsyncSpinners then use mutexes for the data accessed across mutliple threads or make your callback stateless (only depend on the variables passed into the callback) - which is a good idea anyway.

You can also have the callback post a message into a queue (or post a semaphore) which is exactly what you did to get the message into the callback in the first place and have another thread blocking on a queue receive (or a semaphore pend) then process your data. This is the correct implementation of the producer - consumer method.

If you are running in any normal system where you aren't spamming your modules with messages and you want to process all the messages, then handle it in the callback. Handling all the processing in the callback makes the code easier to read and maintain and actually REDUCES the processing load because you are not needlessly passing data around. The data has to be processed somewhere, the more it's passed around the more overhead is added. The above answer by kramer suggests moving the processing to a Timer thread. This is the worst method of all. It means that you are now adding a polling loop into your code. It's just a pain - don't do it.

  1. You have an event driven architecture given to you by ros and are using it! Don't use it to then create a polling architecture.

  2. Unless you implement some checking, you won't know if you're dropping messages. Ie if 2 messages come in before the timer goes off.

  3. You're adding a delay between receiving the message and processing it. If you're passing the message between many module then you might be adding a ton of latency into your system and if you aren't adding delay then you're adding a ton of processing because of the fast polling loop.

  4. It's extremely difficult to profile your code and work out how much load you're inducing. If you're not sending any messages your loops are all still running checking for messages. If you're sending around lots of messages you can still only process them as fast as your polling loop so you'll likely be dropping messages and not ...

edit flag offensive delete link more


1st, IIRC, I can only comment on answers, not post another, each comment limited to ~250 chars. Let me say that your response contains many valid points. Thank you for taking the time and effort to respond. Honestly, it's much appreciated.

kramer gravatar image kramer  ( 2018-07-15 18:32:19 -0500 )edit

2nd, I have to point out that my answer acknowledged both increased computation and time requirements. I think the key word in your 1st paragraph is "blindly" sounds like your code reviews are beneficial.

kramer gravatar image kramer  ( 2018-07-15 18:32:58 -0500 )edit

3rd, the above said, my assumption is that the OP is not asking their question blindly, but has a specific need or desire. To which I provided an answer, whereas you did not. Rather, you ignored and then overrode the explicit request.

kramer gravatar image kramer  ( 2018-07-15 18:33:59 -0500 )edit

4th, you claim that message filtering is the only really valid case for moving processing out of callbacks. If that's not hyperbole, I question whether you have put enough thought into the issue to make the claim at all.

kramer gravatar image kramer  ( 2018-07-15 18:34:59 -0500 )edit

5th, the time, computation, and memory penalty from copying data in a callback for later processing is often relatively and situationally cost-free. But not always; while I still say it's recommended (for most non-trivial nodes), one should not do it blindly.

kramer gravatar image kramer  ( 2018-07-15 18:36:03 -0500 )edit

6th, IMO, the choice of using an AsyncSpinner is not to be taken lightly. In my experience, a node complex enough to benefit from an AsyncSpinner also benefits from moving calculations out of callbacks, allowing one to separate the data and control paths (i.e., the algorithm from its fodder).

kramer gravatar image kramer  ( 2018-07-15 18:36:50 -0500 )edit

7th, it seems to me that if you're not using mutexes with an AyncSpinner, then either: 1) you don't actually need an AsyncSpinner and should be using a single thread, or 2) your application just hasn't blown up yet. :)

kramer gravatar image kramer  ( 2018-07-15 18:37:39 -0500 )edit

8th, perhaps I haven't put enough thought into this, but it seems to me that using multiple AsyncSpinners in a single node indicates structural issues that might be resolved with a code refactor. Or a lack of understanding of the ROS callback system.

kramer gravatar image kramer  ( 2018-07-15 18:39:49 -0500 )edit

Question Tools



Asked: 2016-02-24 16:28:23 -0500

Seen: 1,346 times

Last updated: Jul 15 '18