CPU Issues on ROS2 multiple robots

asked 2021-06-09 03:40:17 -0600

Rinkeeee96 gravatar image

Hey all,

I am currently working on a ROS2 project and i am having some big performance issues. I am using 3 raspberry Pi 4 4gb controlled robots.
They are running a navigation, cartographer, diff drive, encoder and motorkit. And some small other nodes. This didnt seem as an issue before. But when combining them all, they are heavily CPU loaded and will not reach their timings. When checking the CPU usage, it shows that cartographer is using a big chunk of it. Which i understand. After some configuration it dropped a bit. But the biggest issue is the ros2_daemon itself. When checking on a Raspberry that was not even running the application. It showed that 1 core would run at a 100% just from the daemon itself. See picture. This image shows the cpu usage of a running Robot https://ibb.co/pfbrbcG

This image shows the CPU usga of a robot which is not running, but while the other robot is. https://ibb.co/G9Sc8CC

It seems that the amount of messages and topics on the robot that are all sent to the idle one is creating a very big CPU load. And will be even bigger when that second and 3rd robot will be running.

Any fix for this?

The reason the robots are running on the same network is because they need to communicate.

Thanks in advance!

Rinke

edit retag flag offensive close merge delete

Comments

IIUC, the ROS 2 daemon should not be using so much CPU time. I remember an earlier Q&A about something tangential, which turned out to be something related to the network configuration or something like that.

I'll see if I can find it again.


Edit: and ignoring the daemon: seeing as you mention rpis: was everything compiled with optimisations enabled? The default build commands will not do that. On lower specced systems, that can make a huge difference.

gvdhoorn gravatar image gvdhoorn  ( 2021-06-09 03:52:52 -0600 )edit

And please attach your screenshots directly to your question. I've given you sufficient karma.

Note: normally screenshots of terminals are not allowed, but in this case it's the "graph" from htop, which seems like it's in a bit of a grey area.

gvdhoorn gravatar image gvdhoorn  ( 2021-06-09 03:54:45 -0600 )edit

I remember an earlier Q&A about something tangential, which turned out to be something related to the network configuration or something like that. I'll see if I can find it again.

Unfortunately I can't seem to find it. It was a Q&A about something not related to the daemon, but in the end turned out to be (partially) caused by it, and was solved by some bit configuration somewhere (yes, not very helpful, I know).

In the meantime: it might be insightful to see what the daemon is (attempting to) do(ing). If you bring down your system, kill the daemon and then start it in the foreground (as in #q373411, make sure to specify the correct RMW) and keep that running while you start the rest of your system, it may show informative logging output.

gvdhoorn gravatar image gvdhoorn  ( 2021-06-09 04:10:42 -0600 )edit