How to implement offline data synchronization on several nodes?

asked 2014-02-27 21:46:12 -0500

Nicolas Vignard
11 ●2 ●3 ●3

updated 2014-02-28 12:28:12 -0500

ahendrix

47576 ●179 ●367 ●662 http://namniart.com/

Hi,

I asked a first question here http://answers.ros.org/question/12290... but I did not have the right answer of my requirement.

In order to clarify my need, I would like to use the same call graph as in online mode (publish/subscribe mechanism) to do quality test of one algorithm or several. For a better understanding, I draw an example of a ROS nodes architecture (sorry for the link below, my karma is insufficient to publish links):

http://www.dropbox.com/s/tow3fsidlsx4...

With this architecture in mind, I want to test the quality of each algorithm for each message. Typically, if the Algorithm 3 is too slow compare to the Algorithm 2, Algorithm2 need to wait the end of the Algorithm 3 process.

Have you already thought about a synchronize solution inside ROS? Are you going to implement this king of feature, in ROS, in a near future? Is it impossible to do that in the core?

Nicolas

edit retag flag offensive close merge delete

add a comment

0

answered 2014-02-28 12:32:49 -0500

ahendrix

47576 ●179 ●367 ●662 http://namniart.com/

Ecto is a synchronous processing pipeline that can be tied into ROS. It's aimed primarily at image processing, but I suspect it can be used with other types of data as well.

edit flag offensive delete link

add a comment

2

answered 2018-11-03 10:44:44 -0500

Matias
122 ●7 ●11 ●15

Hi, this might help you: https://github.com/lrse/batch_ros

edit flag offensive delete link

add a comment

0

answered 2014-02-28 11:47:26 -0500

demmeln
4306 ●32 ●45 ●69

Have you considered using subscribers with infinite message queue and possibly message_filters [1] (in particular time synchonizer) for nodes with multiple inputs? This way you can make sure that all data is processed.

[1] http://wiki.ros.org/message_filters

edit flag offensive delete link

Comments

The purpose of a time synchronizer is to synchronize several topics in one call back. If this call back takes a long time, you do not know if a message has been lost during this call back.

Nicolas Vignard ( 2014-03-02 21:39:59 -0500 )edit

That's why i suggest using infinite message queues. You might not need message filters. However it does depend on the exact scenario if time synchronizer is applicable, but you can also write your own message filter if it doesn't suite you.

demmeln ( 2014-03-02 23:36:56 -0500 )edit

Sure, we can use the infinite queue with small bag files but the size of my bag files is around 200Go. I do not have 200 Go of memory.

Nicolas Vignard ( 2014-03-03 01:12:07 -0500 )edit

Then AFAICT your only options are setting an appropriate rate for rosbag play and using infinite queues such that you do not run out of memory, or not using ROS in the way you would like to, but rather something else.

demmeln ( 2014-03-03 05:47:54 -0500 )edit

I'm not in the ROS maintainer team, but to answer your question, I believe what you are asking for is not on the agenda for ROS. However, I am sure if you can come up with a concept of how exactly it should work or maybe even some form of proof of concept, you will be supported by the community.

demmeln ( 2014-03-03 05:50:35 -0500 )edit

add a comment

How to implement offline data synchronization on several nodes?

3 Answers

Comments

Question Tools

Stats

Related questions

How to implement offline data synchronization on several nodes? edit

3 Answers

Comments

Question Tools

Stats

Related questions

How to implement offline data synchronization on several nodes?