How do Publisher/Subscriber Message Queues Work?

asked 2016-09-15 15:57:04 -0500

593 ●12 ●18 ●24

Hi all,

I can't seem to find any documentation on this, but it I want to understand more deeply how messaging queues work.

My understanding is that each node has its own publisher and subscriber queues, and it is up the ROS backend to pull messages from publisher queues and push them onto subscriber queues.

Let's say I have a publisher that publishes int32s at a fast rate into a queue of size 10. When my node runs the publish() function, it will push values onto the end of the queue, [1, 2, 3, 4, 5, ...]. If the queue grows too large, queue_size will drop the values at the front of the queue leaving me with [2, 3, 4, 5, 6, ...]. Similarly, if ROS sends the first message in the queue over the wire that automatically pops it and leaves us with the same message queue of [2, 3, 4, 5, 6, ...].

Now on the subscriber end it seems like the logic is when the callback is called the first value is popped, similarly to the above, if the subscriber queue_size is exceeded, the backend is the one that pops the first values and will append the new one to the end in a FIFO manner.

Is this somewhat correct? Thanks!

edit retag flag offensive close merge delete

add a comment

16

answered 2016-09-16 00:13:40 -0500

alienmon
582 ●21 ●26 ●35

updated 2016-09-16 00:20:28 -0500

Publisher Subscriber Overview gives a good explanation on this:

When you create a publisher

ros::Publisher advertise(const std::string& topic, uint32_t queue_size, bool latch = false);

The queue_size there is the publisher/outgoing message queue (B) .. If you are publishing faster than roscpp can send the messages over the wire, roscpp will start dropping OLD messages.

When you create a subscriber

ros::Subscriber subscribe(const std::string& topic, uint32_t queue_size, <callback, which may involve multiple arguments>, const ros::TransportHints& transport_hints = ros::TransportHints());

The queue_size there is the incoming message/subscriber queue (C) size roscpp will use for your callback. If messages are arriving too fast and you are unable to keep up (e.g. you do not spin/ the rate of spin is slow ), roscpp will start throwing away OLD messages.

------------.

I believe that everything is FIFO. When queue is full, the oldest message will be thrown away.

------------.

From reading "publish() behavior and queueing" from the link that I put. What I understand is :

publish() is asynchronous.
When you publish, messages are pushed into a queue (A) for later processing. This queue is immediately pushed into the outgoing/publisher queue (B) . PS: If no one subscribes to the topic, the end is here.
When a subscriber subscribes to that topic. Messages will be sent/pushed from the corresponding outgoing/publisher queue (B) to the incoming/subscriber queue (C).--> this is done by internal thread
When you spin/ callback, the messages handled are from the incoming/subscriber queue (C).

-----------.

That is what I know, feel free to give any correction/ addition.

edit flag offensive delete link

Comments

Hello,

Does it mean that if you have two subscribers then the queue is duplicated? once per each subscriber?

Thank you, Octavio

opo ( 2017-10-18 13:44:36 -0500 )edit

If you have two subscribers, there is still a single queue, but instead of executing one callback on each message in the queue, both callbacks are executed on each message series.

Ed Venator ( 2017-11-05 17:16:41 -0500 )edit

add a comment

2

answered 2020-11-06 15:46:37 -0500

skpro19

310 ●35 ●65 ●58 https://twitter.com/skpro19

updated 2020-11-06 15:49:54 -0500

Role of queue_size in Publishers:

This parameter is needed because, in most cases, the message must be transmitted to another node. This communication process can be time consuming, especially compared to the time needed to create messages.

ROS mitigates this delay by having the publish method store the message in an outbox queue and return right away. A separate thread behind the scenes actually transmits the message.

The (integer) value of the queue_size is the number of messages that the message queue can hold.

Interestingly, the ROS client library is smart enough to know when the publisher and subscriber nodes are part of the same underlying process. In these cases, the message is delivered directly to the subscriber, without using anty network transport.

Role of queue_size in Subscribers:

When new messages arrive, they are stored in a queue until ROS gets a chance to execute your callback function. This parameter establishes the maximum number of messages that ROS will store in that queue at one time. If new messages arrive when the queue is full, the oldest unprocesses messages will be dropped to make room.

Difference betwen the cases of Publisher and Subscriber

The rate at which ROS can empty a publishing queue depends on the time taken to actually transmit the messages to subscribers, and is largely out of our control.

In contrast, the speed with which ROS empties a subscribing queue depends on how quickly we process callbacks. Thus, we can reduce the likelihood of a subscriber queue overflowing by a) ensuring that we allow callbacks to occur via ros::spin or ros::SpinOnce. frequently, and b) reducing the amount of time consumed by each callback.

Source - 'A Gentle Introduction to ROS' by Jason M. O' Kane

edit flag offensive delete link

add a comment

-1

answered 2020-08-06 05:47:17 -0500

cascais
35 ●1 ●3 ●4

updated 2020-08-06 13:31:12 -0500

"ROS Spinning, Threading, Queuing Effective use of multi spinner threads, different queues in ROS" https://levelup.gitconnected.com/ros-...

edit flag offensive delete link

Comments

1

@cascais: could I ask you to please not spam links like this?

The blog post does seem like a good resource, but if it ever disappears (and it will, as all web pages eventually disappear), the link will 404 and the answer will lose all its value.

Please update your answer with the relevant parts quoted from the blog post.

gvdhoorn ( 2020-08-07 02:31:41 -0500 )edit

add a comment

How do Publisher/Subscriber Message Queues Work?

3 Answers

Comments

Comments

Question Tools

Stats

Related questions

How do Publisher/Subscriber Message Queues Work? edit

3 Answers

Comments

Comments

Question Tools

Stats

Related questions

How do Publisher/Subscriber Message Queues Work?