Messages drop on slow subscriber although queue not full / empty
Problem
Setup: ROS noetic, Ubuntu20.
I have a slow pose estimation node which subscribes to incoming images with a low frequency of only 2Hz. The images are published by a rosbag with 20Hz. The bag contains about 800 images (colored, 1280x1024px) , so they should easily fit into the 10k elements queue of the publisher (rosbag play --queue=10000) and subscriber (message_filters.TimeSynchronizer([image_sub, info_sub],10000)
I would expect that the subscriber queues the incoming images so that every single image can be processed one after another. But instead, already after 3 image messages the dropping of messages starts so that in the end ~60% of the messages are dropped.
While debugging the message_filter.TimeSynchronizer (synchronizes CameraInfo and Image subscriber) I could see that the
image queue is never longer than 1, while the camera info queue grew proportional to the published messages.
posting this here is perfect, so +100 for that, but could you please post the solution as an answer instead of a section in your question?
If you then accept the answer, it will be completely clear the issue was resolved.
Right now we only see a new and unanswered question on the index page.
Added an answer but was not able to accept it as correct since I have < 10 points.