ROS performance in different machines

asked 2020-08-09 09:01:17 -0500

azerila
25 ●30 ●35 ●38

updated 2020-08-09 09:02:40 -0500

I have a ros application that when I simulate I see difference between it's performance in one machine compared to another.

My understanding of ros so far has been that non-determinism of ros applications in general can originate from ros topics.

My questions are:

1_Can there be other reasons why I see difference between the performance of the same application between two machines?

2_Are there ways to reduce this difference, or make the performance more deterministic?

3_Are there other sources of non-determinism in ros applications than ros topics?

P.S.: the main difference between my two machines are that one is a desktop computer with Intel® Xeon® E-2144G Processor and other one a server using Intel® Xeon® Gold 5218 Processor. The performance is worse in the server one and I suspect that a part of my application may be running faster inside it while not waiting enough for some subscribers get the message and finish their callbacks. But as there may also be other reasons I have posted this question.

edit retag flag offensive close merge delete

add a comment

answered 2020-08-09 10:04:02 -0500

Tahir M.
213 ●16 ●35 ●28

I think you are simulating some robot. The simulation actually is running a physics engine at back-end. So that can be one thing which actually can slowdown if the world is quite complex.

1_ Can there be other reasons why I see difference between the performance of the same application between two machines?

Up to my knowledge simulation is one of the main reason to slowdown the process. There can also be some big computations going on in your program as well. You can debug this more by checking the memory consumption of your PC by running simulation alone and may be your software running it.

2_ Are there ways to reduce this difference, or make the performance more deterministic?

If you don't need the visual part of simulation you should disable it. And to get the visuals just run only RVIZ and see the necessary part if needed.

For example for Gazebo the other way could be to run Server and Client on different machines. You can also run your software on a different machine.

3_ Are there other sources of non-determinism in ros applications than ros topics?

In a simulation if you are taking some data for example camera feed, that will of-course be advertised in form of ROS topics so rostopics are the one which will consume the resources more. As mentioned above if you are having some huge computations than other components can take time as well.

edit flag offensive delete link

Comments

Your guess about the involvement of a physics engine is correct and I use Mujoco, which is completely deterministic to my knowledge, and I'm not using its GUI. However, the problem is not the amount of computation or the speed as you mentioned. In general, I see the robot performing a task with a lower success rate which is shown by a curve. The question is: what is causing the difference between one machine to the other one, or what difference between one machine and the other one is changing the stochasticity of the application.

azerila ( 2020-08-09 10:13:10 -0500 )edit

As @Tahir M. said you should check the CPU consumption when running only the simulation. Since the server has 16 cores instead of 4 in your desktop computer which is a big difference.

What is probably happenning is that running the simulation and your application consume 100 % of your memory in your desktop computer but not in the server causing everything to be slower, especially the callback functions getting the data from your simulator. Now it can't be determined which processes would be slowed down but if the simulation publishes data faster than your callbacks function can handle, you can end up with overflowing the calbacks buffer and loose some data which will result to your non-deterministic behavior.

Delb ( 2020-08-10 05:31:33 -0500 )edit

Another result of the big performance gap between your two machines is that even if you don't consume 100% of memory on your desktop computer, it has less cores and threads than the server. You should check how many threads are created with your simulation and application. If you have more than 8 (the number of threads of your desktop computer) it will mean that if you have 12 threads, there will be 4 of them that won't be executed independently but sequentially on the same core. This will also result in a non-deterministic behavior.

Delb ( 2020-08-10 05:34:47 -0500 )edit

add a comment

ROS performance in different machines

1 Answer

Comments

Question Tools

Stats

Related questions

ROS performance in different machines edit

1 Answer

Comments

Question Tools

Stats

Related questions

ROS performance in different machines