Ask Your Question
2

care-o-bot simulation extremely slow

asked 2011-09-18 05:05:05 -0500

Paul0nc gravatar image

I'm trying to simulate care-o-bot in gazebo running electric with ubuntu 10.4.

Although not all of the cob_dashboard buttons seem to work, the ones that do are _extremely_ slow. Simply extending the tray takes almost 2 minutes. Is this to be expected? Also, can anyone suggest why the other buttons (particularly, the base motions) don't seem to do anything?

Thanks, Paul.

edit retag flag offensive close merge delete

Comments

can you provide more details as to what you are running, what commands were used to launch/control the simulation? thanks.
hsu gravatar imagehsu ( 2011-09-18 10:03:02 -0500 )edit
Hi Hsu. I'm running (verbatim) the commands given at: http://www.care-o-bot-research.org/software/tutorials/move-the-robot-in-simulation. I've tried running with both the dashboard and with the teleop keyboard commands (but not at the same time). I get the similarly slow response in both cases. Would appreciate any help you can give me.
Paul0nc gravatar imagePaul0nc ( 2011-09-19 02:36:43 -0500 )edit

4 Answers

Sort by ยป oldest newest most voted
4

answered 2011-09-19 13:11:07 -0500

hsu gravatar image

updated 2011-09-19 14:56:07 -0500

In electric, I am getting about ~0.35Xreal-time on my desktop (i7 quad), if you are getting 0.03xreal-time, something is wrong. Can you try profiling it with your favorite profiler (e.g. valgrind) and post some results?

The 0.35Xreal-time should be improvable too, but let start with the 0.03X problem?

I tried turning off the kinect camera by commenting out following line in the urdf

<xacro:cob_kinect_gazebo_v0 name="${name}" ros_topic="${ros_topic}"/>

and I am getting about 1.4Xreal-time. Taking a closer look, the current code was using gazebo_ros_block_laser plugin with 160X160 configuration. A better way to implement this is to use the gazebo_ros_openni_kinect plugin, for example, update your kinect.gazebo.xacro file to look something like this:

<?xml version="1.0"?>
<root xmlns:sensor="http://playerstage.sourceforge.net/gazebo/xmlschema/#sensor"
      xmlns:controller="http://playerstage.sourceforge.net/gazebo/xmlschema/#controller"
      xmlns:interface="http://playerstage.sourceforge.net/gazebo/xmlschema/#interface"
      xmlns:xacro="http://ros.org/wiki/xacro">

<xacro:macro name="cob_kinect_gazebo_v0" params="name ros_topic">

  <gazebo reference="${name}_frame">

    <sensor:camera name="${name}_sensor">
      <imageFormat>R8G8B8</imageFormat>
      <imageSize>320 240</imageSize>
      <hfov>57</hfov>
      <nearClip>0.01</nearClip>
      <farClip>5</farClip>
      <updateRate>1.0</updateRate>
      <baseline>0.2</baseline>
      <controller:gazebo_ros_openni_kinect name="${name}_controller" plugin="libgazebo_ros_openni_kinect.so">
        <alwaysOn>true</alwaysOn>
        <updateRate>1.0</updateRate>
        <imageTopicName>/${ros_topic}/image_raw</imageTopicName>
        <pointCloudTopicName>/${ros_topic}/points</pointCloudTopicName>
        <cameraInfoTopicName>/${ros_topic}/camera_info</cameraInfoTopicName>
        <frameName>/${name}_frame</frameName>
        <pointCloudCutoff>0.5</pointCloudCutoff>
        <distortion_k1>0.00000001</distortion_k1>
        <distortion_k2>0.00000001</distortion_k2>
        <distortion_k3>0.00000001</distortion_k3>
        <distortion_t1>0.00000001</distortion_t1>
        <distortion_t2>0.00000001</distortion_t2>
      </controller:gazebo_ros_openni_kinect>
    </sensor:camera>

    <material value="Gazebo/Red" />
    <turnGravityOff>true</turnGravityOff>
  </gazebo>

</xacro:macro>

</root>

and you should see drastic improvements in speed.

Aside from performance gains, fundamentally, a depth camera like Kinect is better modeled by a gazebo_ros_camera/gazebo_ros_openni_kinect than gazebo_ros_block_laser. In gazebo_ros_block_laser, the scan angles are constant as you make a horizontal or vertical scan, but for kinect, you actually want equal size pixels for each ray trace, which means the angle between rays as you scan vertically or horizontally is not constant.

edit flag offensive delete link more

Comments

Thanks a lot! On my laptop commenting out the kinect line gains same factor than for you, now running 0.15. I'll try your xacro file and do a oprofile soon, but need a lot of things get done before leaving to iros. So maybe that will take a while.
RobMichael gravatar imageRobMichael ( 2011-09-19 23:29:28 -0500 )edit
Commenting out the kinect line give me (initially) about 0.8 real time. However, this decreases to about 0.5 after a few seconds. Using the kinect plugin gives me about 0.5. If I also add 2dnav, performance degrades to about 0.3. All these cases are better than I was previously experiencing... but 2dnav is still slow to look at.
Paul0nc gravatar imagePaul0nc ( 2011-09-24 04:40:02 -0500 )edit
1

answered 2011-09-19 04:50:56 -0500

RobMichael gravatar image

updated 2011-09-19 06:11:38 -0500

We do have the same problem here on several machines. Even a powerful 4 core machine with nvidia graphics (ubuntu 10.4 64bit, electric) runs only at 0.03xrealtime. It was at least 10 times faster with diamondback. Interestingly enough the same speed is shown on a i5 laptop with intel graphics. So one could speculate that it is somehow controlled to that speed, not limited by the hardware.

I talked to some people at IPA and they have seen this also, but only sporadicly. They can get rid of that by starting again. One person remembers to have seen this under diamondback also, but feels that occurecne increased under electric. For us it keeps beeing slow for every start.

Any ideas how to attac this and narrow down the reason?

For the base motion: you need to start cob_2dnav. Only then commanding base motion works. You can then move the base also from rviz by setting target goals.

edit flag offensive delete link more

Comments

Thanks for the feedback RobMichael and for the tip on starting base motion. I'm curious as to why gazebo/ROS is such a slow simulator. Is it generally true that ROS/gazebo do not produce real time simulations?
Paul0nc gravatar imagePaul0nc ( 2011-09-19 09:10:31 -0500 )edit
1

answered 2011-09-21 06:13:33 -0500

hsu gravatar image

Does the 0.03Xreal-time issue happen with the plugin substitution suggested above?

Also, can you try profiling a run using valgrind? e.g. update your gazebo node to look like this:

   <node alaunch-prefix="valgrind --tool=callgrind --dump-instr=yes --simulate-cache=yes --collect-jumps=yes " name="gazebo" pkg="gazebo" type="gazebo" args="$(optenv SIMX) $(find cob_gazebo_worlds)/common/worlds/empty.world" respawn="false" output="screen"/>

roslaunch cob simulation, let it run for a while. In your ros log directory (usually ~/.ros) there should be a file that looks like calgrind.out.[pid], run kcachegrind on it to see some nice plots.

edit flag offensive delete link more
0

answered 2011-09-20 05:31:57 -0500

RobMichael gravatar image

updated 2011-09-20 07:04:27 -0500

Here some new (quite confusing) results (still on laptop, hope to do some desktop tests tomorow):

  • After it looked like a good step forward yesterday, today I was back at 0.03 - without changing any configuration file. After quite a while of search I found out I can reproduce the following (after 3 tests each sequence, I don't think thats by accident): it makes a big difference if I start roscore and cob_bringup sim.launch (nothing else) before or after firefox !?????. With firefox first I get 0.03 otherwise almost 0.2. If firefox is already running gazebo also starts with 0.2 but then goes down to 0.03 in the first 2-3 seconds.

  • Some oprofile results for both states (I tried to go more deeply into python and kernel but did'n manage for now):

    almost 0.2xrealtime (startet before firefox)
  samples|      %|
------------------
   145994 66.5676 python2.6
    51055 23.2791 vmlinux-2.6.32-33-generic-pae
     6636  3.0258 oprofiled
     4748  2.1649 aes_i586
     3148  1.4354 Xorg
     1950  0.8891 oprofile
     1260  0.5745 gazebo
     1129  0.5148 joint_trajectory_action
      499  0.2275 compiz
      461  0.2102 cob_base_drive_chain_sim_node
      419  0.1910 xts
      391  0.1783 bash
      379  0.1728 state_publisher
      255  0.1163 cob_undercarriage_ctrl_node
      210  0.0958 cob_scan_filter
      128  0.0584 gf128mul
      118  0.0538 dm_crypt
      117  0.0533 fix_gazebo_pointcloud
      111  0.0506 pr2_mechanism_diagnostics
      108  0.0492 point_cloud_converter
       45  0.0205 i915
      ...

ca 0.04xrealtime (startet after firefox)
CPU_CLK_UNHALT...|
  samples|      %|
------------------
   151393 43.6056 python2.6
   140351 40.4252 vmlinux-2.6.32-33-generic-pae
    17351  4.9976 oprofiled
    11419  3.2890 Xorg
     9592  2.7628 aes_i586
     4739  1.3650 oprofile
     2276  0.6556 gazebo
     2239  0.6449 compiz
     1975  0.5689 joint_trajectory_action
     1311  0.3776 bash
      824  0.2373 xts
      673  0.1938 opreport
      478  0.1377 cob_base_drive_chain_sim_node
      348  0.1002 state_publisher
      306  0.0881 gf128mul
      256  0.0737 cob_scan_filter
      255  0.0734 cob_undercarriage_ctrl_node
      227  0.0654 i915
      219  0.0631 dm_crypt
      152  0.0438 fix_gazebo_pointcloud
      140  0.0403 pr2_mechanism_diagnostics
      136  0.0392 point_cloud_converter
      ...
       25  0.0072 firefox-bin

Note that firefox-bin does not eat up the time, but somehow influences and brings up the kernel percentage.

To see python and kernel to use almost all the time is also something I didn't expect at all. Is this something cob specific? Is anybody of the cob people following this thread?

strange!

@PaulOnc: how is your realtime factor (shown at bottom line of gazebo)? Can you reproduce the influence of firefox?

Result after some more experiments:

  • The difference is made on startup of gazebo GUI. Speed depends on weather firefox is running at that point in time. It keeps slow even after closing firefox! It keeps fast when firefox is opened after gazeboo start.
  • For a headless gazebo there is no influence from firefox
  • The effect is deterministic and reproducable every time.
edit flag offensive delete link more

Comments

can you try running with rendering disabled and check with rviz to see how fast simulation is running? i.e. launch gazebo node with the -r option (no windows will come up, but gazebo will run in background without any image sensor generation). You can use rviz to check wall-clock time vs. sim time
hsu gravatar imagehsu ( 2011-09-20 06:09:19 -0500 )edit
With headless: 0.75xrealtime - firefox makes no difference. Afterwards checked again with visualisation. Firefox effect reproducable every time.
RobMichael gravatar imageRobMichael ( 2011-09-20 06:38:57 -0500 )edit
it would appear the gpu on your laptop is having a hard time keeping up with rendering images. One thing you can try is to go through all the camera/kinect-depth sensors in your robot, and try reducing the resolution & update-rate to see if it helps.
hsu gravatar imagehsu ( 2011-09-20 06:43:11 -0500 )edit
Do you have any idea why firefox would have such a big influence - even with minimized window, and even after firefox is closed?! There should be no difference for the gpu??? Maybe there is any graphics mode setting that firefox prevents the gazeboo GUI to go into on startup?
RobMichael gravatar imageRobMichael ( 2011-09-20 06:47:26 -0500 )edit
@RobMichael: not sure why firefox is hogging up resources. have you tried either disabling flash or using chrome?
hsu gravatar imagehsu ( 2011-09-20 09:46:52 -0500 )edit
I am unable to reproduce the firefox symptom. Rather, my performance level (0.03 or 0.3) seems to be completely intermittent regardless of firefox. If I invoke the cob simulation, sometimes it comes up with 0.03 and sometimes its at 0.3. Whereever the performance is on startup is generally where it stays.
Paul0nc gravatar imagePaul0nc ( 2011-09-21 03:18:23 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2011-09-18 05:05:05 -0500

Seen: 1,933 times

Last updated: Sep 21 '11