CPU Issues on ROS2 multiple robots
Hey all,
I am currently working on a ROS2 project and i am having some big performance issues.
I am using 3 raspberry Pi 4 4gb controlled robots.
They are running a navigation, cartographer, diff drive, encoder and motorkit. And some small other nodes.
This didnt seem as an issue before. But when combining them all, they are heavily CPU loaded and will not reach their timings.
When checking the CPU usage, it shows that cartographer is using a big chunk of it. Which i understand. After some configuration it dropped a bit. But the biggest issue is the ros2_daemon itself.
When checking on a Raspberry that was not even running the application. It showed that 1 core would run at a 100% just from the daemon itself.
See picture.
This image shows the cpu usage of a running Robot
https://ibb.co/pfbrbcG
This image shows the CPU usga of a robot which is not running, but while the other robot is. https://ibb.co/G9Sc8CC
It seems that the amount of messages and topics on the robot that are all sent to the idle one is creating a very big CPU load. And will be even bigger when that second and 3rd robot will be running.
Any fix for this?
The reason the robots are running on the same network is because they need to communicate.
Thanks in advance!
Rinke
IIUC, the ROS 2 daemon should not be using so much CPU time. I remember an earlier Q&A about something tangential, which turned out to be something related to the network configuration or something like that.
I'll see if I can find it again.
Edit: and ignoring the daemon: seeing as you mention rpis: was everything compiled with optimisations enabled? The default build commands will not do that. On lower specced systems, that can make a huge difference.
And please attach your screenshots directly to your question. I've given you sufficient karma.
Note: normally screenshots of terminals are not allowed, but in this case it's the "graph" from
htop, which seems like it's in a bit of a grey area.Unfortunately I can't seem to find it. It was a Q&A about something not related to the daemon, but in the end turned out to be (partially) caused by it, and was solved by some bit configuration somewhere (yes, not very helpful, I know).
In the meantime: it might be insightful to see what the daemon is (attempting to) do(ing). If you bring down your system, kill the daemon and then start it in the foreground (as in #q373411, make sure to specify the correct RMW) and keep that running while you start the rest of your system, it may show informative logging output.