Catkin-compiled Code Runs 3x slower

edit

I've been running a slate of tests on the new hydro navigation. I was trying to figure out why updating the costmap in the new hydro version takes so much longer than in the old 'Fuerte' version. (Note that both of these are tested in Groovy)

I searched and searched to figure out how the new code was different. But it wasn't the code at all. It was the compiler.

The colors of each segment of the bar are irrelevant in this case. The first row shows how long the catkin-compiled version of the Hydro code takes to update the costmap once. The second bar shows the same operation in Fuerte, which is compiled using Rosbuild. The third is the same exact code from the first bar, but compiled using Rosbuild.

My question is: what accounts for this 3x slowdown when compiling with Catkin?

Links:

• Catkin CMake
• Rosbuild CMake
• Other

Edit: The benchmark was performed using locally compiled packages and the timing was measured with repeated gettimeofday calls (since that infrastructure was already in the code for measuring update-time.)

Edit 2: Thanks to William's comment, I figured it out.

Explanation: The update loop timing is usually only available in the ROS_DEBUG information. Printing the DEBUG logs takes a bit of time, which you can see with the red bars. For the packages I compiled, I also changed it to be printed quicker without ROS_DEBUG. However, I couldn't do that for the debian package.

The key result is that Rosbuild code (which William mentions, already was built in Release mode) runs as fast as the catkin code with the release flag set.

I don't know why the Debian is as slow as it is, but its not going to bother me immediately.

(@William: Put your answer below for some sweet sweet karma)