# care-o-bot simulation extremely slow

I'm trying to simulate care-o-bot in gazebo running electric with ubuntu 10.4.

Although not all of the cob_dashboard buttons seem to work, the ones that do are _extremely_ slow. Simply extending the tray takes almost 2 minutes. Is this to be expected? Also, can anyone suggest why the other buttons (particularly, the base motions) don't seem to do anything?

Thanks, Paul.

edit retag close merge delete

can you provide more details as to what you are running, what commands were used to launch/control the simulation? thanks.
( 2011-09-18 10:03:02 -0500 )edit
Hi Hsu. I'm running (verbatim) the commands given at: http://www.care-o-bot-research.org/software/tutorials/move-the-robot-in-simulation. I've tried running with both the dashboard and with the teleop keyboard commands (but not at the same time). I get the similarly slow response in both cases. Would appreciate any help you can give me.
( 2011-09-19 02:36:43 -0500 )edit

Sort by » oldest newest most voted

In electric, I am getting about ~0.35Xreal-time on my desktop (i7 quad), if you are getting 0.03xreal-time, something is wrong. Can you try profiling it with your favorite profiler (e.g. valgrind) and post some results?

The 0.35Xreal-time should be improvable too, but let start with the 0.03X problem?

I tried turning off the kinect camera by commenting out following line in the urdf

<xacro:cob_kinect_gazebo_v0 name="${name}" ros_topic="${ros_topic}"/>


and I am getting about 1.4Xreal-time. Taking a closer look, the current code was using gazebo_ros_block_laser plugin with 160X160 configuration. A better way to implement this is to use the gazebo_ros_openni_kinect plugin, for example, update your kinect.gazebo.xacro file to look something like this:

<?xml version="1.0"?>
<root xmlns:sensor="http://playerstage.sourceforge.net/gazebo/xmlschema/#sensor"
xmlns:controller="http://playerstage.sourceforge.net/gazebo/xmlschema/#controller"
xmlns:interface="http://playerstage.sourceforge.net/gazebo/xmlschema/#interface"
xmlns:xacro="http://ros.org/wiki/xacro">

<xacro:macro name="cob_kinect_gazebo_v0" params="name ros_topic">

<gazebo reference="${name}_frame"> <sensor:camera name="${name}_sensor">
<imageFormat>R8G8B8</imageFormat>
<imageSize>320 240</imageSize>
<hfov>57</hfov>
<nearClip>0.01</nearClip>
<farClip>5</farClip>
<updateRate>1.0</updateRate>
<baseline>0.2</baseline>
<controller:gazebo_ros_openni_kinect name="${name}_controller" plugin="libgazebo_ros_openni_kinect.so"> <alwaysOn>true</alwaysOn> <updateRate>1.0</updateRate> <imageTopicName>/${ros_topic}/image_raw</imageTopicName>
<pointCloudTopicName>/${ros_topic}/points</pointCloudTopicName> <cameraInfoTopicName>/${ros_topic}/camera_info</cameraInfoTopicName>
<frameName>/${name}_frame</frameName> <pointCloudCutoff>0.5</pointCloudCutoff> <distortion_k1>0.00000001</distortion_k1> <distortion_k2>0.00000001</distortion_k2> <distortion_k3>0.00000001</distortion_k3> <distortion_t1>0.00000001</distortion_t1> <distortion_t2>0.00000001</distortion_t2> </controller:gazebo_ros_openni_kinect> </sensor:camera> <material value="Gazebo/Red" /> <turnGravityOff>true</turnGravityOff> </gazebo> </xacro:macro> </root>  and you should see drastic improvements in speed. Aside from performance gains, fundamentally, a depth camera like Kinect is better modeled by a gazebo_ros_camera/gazebo_ros_openni_kinect than gazebo_ros_block_laser. In gazebo_ros_block_laser, the scan angles are constant as you make a horizontal or vertical scan, but for kinect, you actually want equal size pixels for each ray trace, which means the angle between rays as you scan vertically or horizontally is not constant. more ## Comments Thanks a lot! On my laptop commenting out the kinect line gains same factor than for you, now running 0.15. I'll try your xacro file and do a oprofile soon, but need a lot of things get done before leaving to iros. So maybe that will take a while. ( 2011-09-19 23:29:28 -0500 )edit Commenting out the kinect line give me (initially) about 0.8 real time. However, this decreases to about 0.5 after a few seconds. Using the kinect plugin gives me about 0.5. If I also add 2dnav, performance degrades to about 0.3. All these cases are better than I was previously experiencing... but 2dnav is still slow to look at. ( 2011-09-24 04:40:02 -0500 )edit We do have the same problem here on several machines. Even a powerful 4 core machine with nvidia graphics (ubuntu 10.4 64bit, electric) runs only at 0.03xrealtime. It was at least 10 times faster with diamondback. Interestingly enough the same speed is shown on a i5 laptop with intel graphics. So one could speculate that it is somehow controlled to that speed, not limited by the hardware. I talked to some people at IPA and they have seen this also, but only sporadicly. They can get rid of that by starting again. One person remembers to have seen this under diamondback also, but feels that occurecne increased under electric. For us it keeps beeing slow for every start. Any ideas how to attac this and narrow down the reason? For the base motion: you need to start cob_2dnav. Only then commanding base motion works. You can then move the base also from rviz by setting target goals. more ## Comments Thanks for the feedback RobMichael and for the tip on starting base motion. I'm curious as to why gazebo/ROS is such a slow simulator. Is it generally true that ROS/gazebo do not produce real time simulations? ( 2011-09-19 09:10:31 -0500 )edit Does the 0.03Xreal-time issue happen with the plugin substitution suggested above? Also, can you try profiling a run using valgrind? e.g. update your gazebo node to look like this:  <node alaunch-prefix="valgrind --tool=callgrind --dump-instr=yes --simulate-cache=yes --collect-jumps=yes " name="gazebo" pkg="gazebo" type="gazebo" args="$(optenv SIMX) \$(find cob_gazebo_worlds)/common/worlds/empty.world" respawn="false" output="screen"/>


roslaunch cob simulation, let it run for a while. In your ros log directory (usually ~/.ros) there should be a file that looks like calgrind.out.[pid], run kcachegrind on it to see some nice plots.

more

Here some new (quite confusing) results (still on laptop, hope to do some desktop tests tomorow):

• After it looked like a good step forward yesterday, today I was back at 0.03 - without changing any configuration file. After quite a while of search I found out I can reproduce the following (after 3 tests each sequence, I don't think thats by accident): it makes a big difference if I start roscore and cob_bringup sim.launch (nothing else) before or after firefox !?????. With firefox first I get 0.03 otherwise almost 0.2. If firefox is already running gazebo also starts with 0.2 but then goes down to 0.03 in the first 2-3 seconds.

• Some oprofile results for both states (I tried to go more deeply into python and kernel but did'n manage for now):

    almost 0.2xrealtime (startet before firefox)
samples|      %|
------------------
145994 66.5676 python2.6
51055 23.2791 vmlinux-2.6.32-33-generic-pae
6636  3.0258 oprofiled
4748  2.1649 aes_i586
3148  1.4354 Xorg
1950  0.8891 oprofile
1260  0.5745 gazebo
1129  0.5148 joint_trajectory_action
499  0.2275 compiz
461  0.2102 cob_base_drive_chain_sim_node
419  0.1910 xts
391  0.1783 bash
379  0.1728 state_publisher
255  0.1163 cob_undercarriage_ctrl_node
210  0.0958 cob_scan_filter
128  0.0584 gf128mul
118  0.0538 dm_crypt
117  0.0533 fix_gazebo_pointcloud
111  0.0506 pr2_mechanism_diagnostics
108  0.0492 point_cloud_converter
45  0.0205 i915
...

ca 0.04xrealtime (startet after firefox)
CPU_CLK_UNHALT...|
samples|      %|
------------------
151393 43.6056 python2.6
140351 40.4252 vmlinux-2.6.32-33-generic-pae
17351  4.9976 oprofiled
11419  3.2890 Xorg
9592  2.7628 aes_i586
4739  1.3650 oprofile
2276  0.6556 gazebo
2239  0.6449 compiz
1975  0.5689 joint_trajectory_action
1311  0.3776 bash
824  0.2373 xts
673  0.1938 opreport
478  0.1377 cob_base_drive_chain_sim_node
348  0.1002 state_publisher
306  0.0881 gf128mul
256  0.0737 cob_scan_filter
255  0.0734 cob_undercarriage_ctrl_node
227  0.0654 i915
219  0.0631 dm_crypt
152  0.0438 fix_gazebo_pointcloud
140  0.0403 pr2_mechanism_diagnostics
136  0.0392 point_cloud_converter
...
25  0.0072 firefox-bin



Note that firefox-bin does not eat up the time, but somehow influences and brings up the kernel percentage.

To see python and kernel to use almost all the time is also something I didn't expect at all. Is this something cob specific? Is anybody of the cob people following this thread?

strange!

@PaulOnc: how is your realtime factor (shown at bottom line of gazebo)? Can you reproduce the influence of firefox?

Result after some more experiments:

• The difference is made on startup of gazebo GUI. Speed depends on weather firefox is running at that point in time. It keeps slow even after closing firefox! It keeps fast when firefox is opened after gazeboo start.
• For a headless gazebo there is no influence from firefox
• The effect is deterministic and reproducable every time.
more

can you try running with rendering disabled and check with rviz to see how fast simulation is running? i.e. launch gazebo node with the -r option (no windows will come up, but gazebo will run in background without any image sensor generation). You can use rviz to check wall-clock time vs. sim time
( 2011-09-20 06:09:19 -0500 )edit
With headless: 0.75xrealtime - firefox makes no difference. Afterwards checked again with visualisation. Firefox effect reproducable every time.
( 2011-09-20 06:38:57 -0500 )edit
it would appear the gpu on your laptop is having a hard time keeping up with rendering images. One thing you can try is to go through all the camera/kinect-depth sensors in your robot, and try reducing the resolution & update-rate to see if it helps.
( 2011-09-20 06:43:11 -0500 )edit
Do you have any idea why firefox would have such a big influence - even with minimized window, and even after firefox is closed?! There should be no difference for the gpu??? Maybe there is any graphics mode setting that firefox prevents the gazeboo GUI to go into on startup?
( 2011-09-20 06:47:26 -0500 )edit
@RobMichael: not sure why firefox is hogging up resources. have you tried either disabling flash or using chrome?
( 2011-09-20 09:46:52 -0500 )edit
I am unable to reproduce the firefox symptom. Rather, my performance level (0.03 or 0.3) seems to be completely intermittent regardless of firefox. If I invoke the cob simulation, sometimes it comes up with 0.03 and sometimes its at 0.3. Whereever the performance is on startup is generally where it stays.
( 2011-09-21 03:18:23 -0500 )edit