Handle lost visual odometry with IMU sensor fusion

edit

Hi,

I am trying to fuse data from an IMU and a visual odometry node with robot_localization in order to have a better localization for my robot. The IMU is a Tinkerforge IMU Brick 2.0; the visual odometry node is the one provided in the package rtabmap_ros.
Both nodes work well, and the extended kalman filter does what it's supposed to do.

I am using ROS kinetic on Ubuntu 16.04.
The only issue that I'm facing is when the visual odometry gets lost: in this case the node publishes a message like that:

header:
   seq: 769
    stamp:
       secs: 1522407457
       nsecs: 640873928
    frame_id: "odom"
child_frame_id: "kinect2_base_link"
pose:
    pose:
       position:
          x: 0.0
          y: 0.0
          z: 0.0
      orientation: 
          x: 0.0
          y: 0.0
          z: 0.0
          w: 0.0
    covariance: [9999.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9999.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9999.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9999.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9999.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9999.0]
twist:
    twist:
       linear:
          x: 0.0
          y: 0.0
          z: 0.0
       angular: 
          x: 0.0
          y: 0.0
          z: 0.0
     covariance: [9999.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9999.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9999.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9999.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9999.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9999.0]

In this case the kalman filter still continues to work, and this results in the estimante odometry having an erroneous translational velocity. How can I face with this? I was thinking about a way to "pause" the kalman filter until the visual odometry returns working but actually I can't figure it out.

Thank you in advance for all your support!