Ask Your Question

Why is message processing in sporadic bursts ?

asked 2019-06-05 12:34:53 -0600

elpidiovaldez gravatar image

updated 2019-06-05 12:54:14 -0600

I am trying to increase the responsiveness of a control loop. I am controlling the pan/tilt servos of a camera (attached to a Raspberry Pi 3) based on the center of a bounding box found by YOLO vision system (running on a desktop pc with GTX 1080 ti GPU). I am observing very jumpy servo movements.


 Yolo(on pc)   <----wifi------ PiCam(Pi3)

 PID(on pc)    -----wifi-----> ServoController(Pi3)

The desired servo positions are sent as messages by the PID controller on the pc at 10 Hz (this is confirmed by the messages seen on rqt_console and rostopic Hz). The message queue size of the publisher is 1.

On the Pi3 I have set the loop rate of the servo controller to values between 10 and 50 Hz. I have set the subscriber queue size to 1 and 10.

  • With a ServoController queue size of 1, rqt_console shows only about one in 10 of the control messages are processed. A control signal sent to the servos at 1 or 2 Hz resulting in very jumpy movements.
  • With a ServoController queue size of 10, rqt_console shows that all the control messages are processed, but they are processed very quickly in blocks of about 10, then nothing happens for almost a second, then another block of 10 are processed. The servos remain very jumpy because all the small incremental movements are processed at the same time and the largest step dominates.

My conclusion is that the controller on the Pi3 is processing in bursts, but why ? Changing the loop rate from 10 to 50 did not improve matters. I checked the cpu utilisation on the Pi3 but it is never more than 7%.

Here is output from rqt_console which shows the bursts. 'I heard' denotes a message processed by the servo controller. PID denotes a message sent from the PID loop. ```

"I heard: [14, 99]",/PiServoNode,1559747410.29711
"I heard: [14, 101]",/PiServoNode,1559747410.29413
"I heard: [14, 124]",/PiServoNode,1559747410.29118
"I heard: [14, 113]",/PiServoNode,1559747410.2879
"PID X: pan=99, err=84, cumErr=204, P=92.400000 I=12.240000, clamp=10000
"PID X: pan=101, err=91, cumErr=120, P=100.100000 I=7.200000, clamp=10000
"PID X: pan=124, err=118, cumErr=29, P=129.800000 I=1.740000, clamp=10000
"PID X: pan=113, err=114, cumErr=-89, P=125.400000 I=-5.340000, clamp=10000
"I heard: [14, 101]",/PiServoNode,1559747409.1169
"I heard: [14, 91]",/PiServoNode,1559747409.11382
"I heard: [14, 78]",/PiServoNode,1559747409.11077
"I heard: [14, 65]",/PiServoNode,1559747409.10774
"I heard: [14, 54]",/PiServoNode,1559747409.10459
"I heard: [14, 43]",/PiServoNode,1559747409.1008
"I heard: [14, 31]",/PiServoNode,1559747409.09732
"I heard: [14, 20]",/PiServoNode,1559747409.09406
"I heard: [14, 13]",/PiServoNode,1559747409.09102
"I heard: [14, 18]",/PiServoNode,1559747409.0879
"PID X: pan=101, err=109, cumErr=-203, P=119.900000 I=-12.180000, clamp ...
edit retag flag offensive close merge delete



My conclusion is that the controller on the Pi3 is processing in bursts, but why ?

I believe there are many older Q&As already discussing this. See #q303464, #q237592, #q272375 and #q290122. Did you not find those when searching for "messages bursts"?

The first thing to check is whether this is due to Nagle.

gvdhoorn gravatar image gvdhoorn  ( 2019-06-05 13:56:57 -0600 )edit

No I did not find those, but my original searches (and the original title of my post) used the word 'chunks' instead of 'bursts'. I edited it later when I thought of a better word ! Thanks for a great pointer, I had never come across any of that material in the ROS tutorials that I have read.

elpidiovaldez gravatar image elpidiovaldez  ( 2019-06-05 20:01:18 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted

answered 2019-06-05 20:08:35 -0600

elpidiovaldez gravatar image

updated 2019-06-05 20:17:38 -0600

I am closing the post because I have now found a bug in the ServoController code on the Pi3 that was reducing its rate to 2 Hz, so the problem was nothing to do with ROS. I am also now using the 'tcpNoDelay' and 'unreliable' transport options which are described in the documents linked by gvdhoorn. These are just what I need, so some good came out of this.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower


Asked: 2019-06-05 12:34:53 -0600

Seen: 104 times

Last updated: Jun 05 '19