Stefan-
I spent some time early this year working configurating Gazebo to run faster with the PR2 model. Basically, my approach was to downsample sensors as much as possible, downsample physics, and then patch Gazebo as needed where the profile showed huge hits. Like you, my application is teleoperation. With a powerful machine, Gazebo can run at real time or faster (I run at 1.4x real-time using an Intel i7 2600K with a GT 560 Ti GPU). I would guesstimate it to run about 5x faster with the PR2 model.
If you use the "kt2_simulator" branch of Gazebo, (https://code.ros.org/svn/wg-ros-pkg/branches/trunk_electric/kt2_simulator), you'll find the changes to the URDF of the PR2, controller gains, and the world file. A rosinstall file will allow you pull down the branch of Gazebo we're using. These changes should go out in Electric.
The biggest gains I found found were from lowering the physics update rate from 1KHz to 250Hz. Even if you have to increase the number of quick step iterations per physics loop, decreasing the update rate is still a win because you don't have to do collision checks as often. Obviously, this can cause problems in physics quality, and objects/robots can "explode". You'll have to tune controller gains for a lower update rate. The modified gains for the PR2 are in the "kt2_pr2_defs" package in the above stack.
One major improvement to PR2 physics to a "fixed joint reduction" which John put in place. The PR2 URDF calls out over 40 fixed joints (usually to connect a sensor to a body). In Diamondback, Gazebo models these joints as revolute joints. In the branch, the URDF is reduced to only bodies that are connected by revolute or prismatic joints. This increases physics-only simulation by about 50%. If you run the "urdf2model" or "spawn_model" utilities, fixed joints in your URDF will be reduced.
A large performance hit to the simulator is laser scan generation. The laser data is generated with ODE collision checking, and you can see a very noticeable difference in speed when you are subscribing to the laser data. For the PR2, I cut the update rate for the lasers in half, and reduced the number of rays from 640 to 180 for each laser. These are changes to the URDF, but ideally they will be dynamically reconfigurable at some point.
Camera rendering is also expensive. In the branch, I implemented a reconfigurable update rate for each camera to allow you to throttle down the render rate (this feature exists in Diamondback, but does not actually change the render rate of the camera, so does not affect the simulation speed). Slow down the camera update rates as much as you can. If you don't subscribe to a camera topic, the data won't be generated, so make sure you have lazy subscription in your pipeline.
In the new branch, Gazebo cameras can generate depth data, like a Kinect. In ... (more)