How to profile message activity?

asked 2016-10-13 09:49:29 -0500

Cerin
940 ●138 ●160 ●169

Are there any tools for profiling or collecting statistics on the types of message activity per node?

I'm trying to troubleshoot some nodes consuming excessive amounts of CPU, and to help diagnose the problem, I'd like to get a histogram of the types of messages they publish over a given time range.

e.g. If I find something like:

topic                     messages/min
/mynode/encoders          23
/mynode/button            1
/mynode/imu               239423847887
/mynode/gps               56

then I'll immediately know it's publish rate for IMU messages is too high.

Obviously, I could do this manually by checking every single topic, but it's be nice to have a tool to automate this, and I don't want to reinvent the wheel.

edit retag flag offensive close merge delete

add a comment

1

answered 2016-10-13 11:57:42 -0500

Mani
1694 ●9 ●30 ●36 https://mani.im/

updated 2016-10-13 12:57:07 -0500

Since _ROS Indigo_ there is builtin support for topic statistics in ROS. Based on the documentation, these are the metrics that are collected:

Period of messages by all publishers (average, maximum, standard deviation)
Age of messages, based on header timestamp (average, maximum, standard deviation)
Number of dropped messages
Traffic volume (Bytes)

To enable this feature, the /enable_statistics parameter needs to be set to true.

There are some tools that enable higher level analysis, visualisation and diagnostics based on topic statistics:

rqt_graph (Visualisation of topic statistics)
rosprofiler (Profiler and Visualisation based on topic statistics)
arni (Visualisation and Fault Detection/Mitigation based on topic statistics and a custom process and host level monitor)

There is also a tool called Drums that collects fine-grained statistics about hosts, processes and network sockets that are part of a running ROS computation graph independent of the builtin data from ROS topic statistics. (Disclaimer: I am the author of Drums and the documentation is far from perfect at the moment).