robot_localization: sensor frame in differential mode

edit

I believe your question is answered a few lines above. We compute the delta between the two poses (in this case, in the camera frame), then rotate that using the rotation between the sensor frame and the world frame. That should put the delta into your base_link_frame.

However, this might not actually help you, especially if you have your camera's transform defined relative to your base_link_frame (and you probably do). r_l is not good at handling differential poses for sensors whose transforms are defined relative to the base_link_frame.

You have two options, I think:

1. Modify the VIO package so that it outputs pose and velocity, and then fuse the velocity directly.
2. As long as you are _only_ going to use the sensor in differential mode, I _think_ you could get away with defining the transform for the camera relative to your world_frame, rather than your base_link_frame. So if you create a static transform publisher and make it publish a rotation-only transform from your world_frame to your camera_frame (the contents of said transform should be the rotation of the camera VO data relative to your robot base_link_frame). The one downside I can see for doing this is that you won't be able to counter the effects of false linear velocity resulting from turning in place, assuming the sensor is not placed at centre of rotation.