Throttle publish rate based on topic queue size and fill level
I've seen questions asked about how to change the subscriber's message receive rate, and filters seem to be the solution to this problem. However, this assumes that it is ok if the subscriber does not receive every published message.
My question is similar: is it possible for a publishing node to query the fill level of the topic queue to determine if there is space left? Generally speaking, is there a way to throttle an upstream publisher node's publish rate based consumption rates further down the pipeline? The key here is that I cannot afford to have any messages dropped.
The scenario: an upstream node publishes messages at a high rate and a downstream node needs to process EVERY message published. I know I could simply increase the pub/sub queue sizes, but this solution is not ideal for extended tests.
Ideally, I would like to check how much space is left in the queue before publishing, but as far as I can tell, there's no way to do that.
Any suggestions would be greatly appreciate!
