ROS2 Real-time computing pendulum demo performing worse on a PREEMPT_RT kernel
I have tested running pendulum demo according to these instructions with a normal Linux kernel (5.14.0-1048-oem) and a real-time kernel (PREEMPTRT, 5.15.55-rt48), but for some reason both the latency and the standard deviation is worse for the PREEMPTRT kernel. Having a worse latency makes sense as the throughput should be worse, but the standard deviation should be smaller as far as I understand.
Both kernels report 0 minor and major page faults during the run. On both kernels, I ran the code with 10000000 iterations.
I'm running the code on a docker container with a user that has RTPRIO 98.
Normal kernel
Mean: 25500.82 ns
STD: 25511.86 ns
Max: 2023204 ns
Min: 1746 ns
PREEMPT_RT
Mean: 31894.42 ns
STD: 34334.73 ns
Max: 2551278 ns
Min: 1796 ns
System information:
ROS2 Galactic
Ubuntu 20.04.4 (kernels 5.14.0-1048-oem and 5.15.55-rt48)
x86_64
Are these results correct? Am I understanding something fundamentally wrong about PREEMPT_RT?
Asked by alqio on 2022-08-24 00:27:30 UTC
Answers
I would guess you either do not have your kernel configured correctly (you can check with the cyclictest tool or, more probable, the rt-priority is not propagated through the docker correctly. Anyway, the performance indicate you do not have the PREEMPT_RT set up correctly. This is what I get on
$ uname -a
Linux testbox 5.10.113-rt61-yocto-preempt-rt #1 SMP PREEMPT_RT Wed May 4 02:32:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
And cpu is Intel(R) Core(TM) i7-8665UE CPU @ 1.70GHz
rttest statistics for results:
- Minor pagefaults: 0
- Major pagefaults: 0
Latency (time after deadline was missed):
- Min: 3878 ns
- Max: 45969 ns
- Mean: 4468.777000 ns
- Standard deviation: 1322.342240
Asked by HorstLocal on 2022-08-24 09:51:49 UTC
Comments
As gvdhoorn suggests, you could check if the system is correctly configured and verify this by using cyclictest.
In case you are interested in having an additional reference, here you have the results for a RPI4 and Galactic:
- https://ros-realtime.github.io/ros2_realtime_benchmarks/benchmark_results/galactic/pendulum_control/rpi4-rt/idle/rmw_cyclonedds_cpp/pendulum_results.html
- rttest options used: https://ros-realtime.github.io/ros2_realtime_benchmarks/benchmark_tools/pendulum_control.html#setup
- RPI4 image: https://github.com/ros-realtime/ros-realtime-rpi4-image/releases/tag/rpi4_ros2_galactic_v5.4.106-rt54-raspi-arm64-lttng
Asked by carlossvg on 2022-09-27 05:33:00 UTC
Comments