Your tree should look like map->odom_filtered->base_link->... not base_link->odomX. Take the odometry message of source1 (I asume its camera centered, so frame_ID would be "camera1_base") fuse it in robot_localization ekf (link text with odom2 and output the fused data (ekf output) in odom_filtered frame.

@Dragonslayer Hi. I fell asleep before I could tag you and you beat me to it. So welcome to the post. I am starting from the basics. My config file for robot_localization is below. But I am still confused what needs to go in for odom_frame, base_link_frame, map_frame, world_frame ?

For the sake of simplicity, the frame names for camera1 are cam1_odom->cam1_base->cam1_optical... and the frame names for camera2 are cam2_odom->cam2_base->cam2_optical... and the PC is the base_link

frequency: 5
two_d_mode: true
publish_tf: true
predict_to_current_time: true
print_diagnostics: true

**odom_frame: ???
base_link_frame: ???
#map_frame: ???
world_frame: ???**

# xyz, rpy, vxyz, vrpy, axyz
odom0: /camera1/odom
odom0_config: [true,...
               ...false]
odom0_differential: false

odom1: /camera2/odom
odom1_config: [false,... 
               ..., false]
odom1_differential: true ...

• odom_frame: odom #published by ekf itslef.
• base_link_frame: base_link #the robot base link/chassis of your system. Or projected base_footprint frame depending on robot model. Published to tf from urdf robot model (usually) via robot_state_publisher.
• map_frame: map #The map frame the localization or slam package publishes.
• world_frame: odom #defaults to odom if unspecified for fusing continous data like in your case this is the way to go.

@Dragonslayer: I get the following warning messages repeatedly when I run the localization node with the frames set to

odom_frame: odom
base_link_frame: base_link
map_frame: map
world_frame: odom

Warnings:

[ WARN] [1599935167.107900670]: Could not obtain transform from cam2_base to base_link. Error was Could not find a connection between 'base_link' and 'cam2_base' because they are not part of the same tree.Tf has two or more unconnected trees.

[ WARN] [1599935169.111119120]: Could not obtain transform from cam1_odom to odom. Error was Could not find a connection between 'odom' and 'cam1_odom' because they are not part of the same tree.Tf has two or more unconnected trees.

...

In a way I would expect these errors because there are no tf which would establish a relation between cam1 tree, cam2 tree, and base_link. :/

You dont need the cam1 and cam2 tree. just a urdf for the robot model, base_link up to the frame the cams odometries are in reference to (camera_linkX or cam1_base or whatever its called).

@Dragonslayer So, I turned off the tf between the camX_odom --> camX_base. I then modified the robot urdf to publish base_link --> camX_base urdfs. Now my tf tree looks like this.

And now when I try to run robot_localization, I get the following warn message repeatedly:

[ WARN] [1599967026.507911316]: Could not obtain transform from cam1_odom to odom. Error was "cam1_odom" passed to lookupTransform argument source_frame does not exist

This I would expect too, since there is no tf to relate cam1_odom to any other frames. Do I need to manually publish the inverse tf from camX_base --> camX_odom?

Are you sure its ekf node that sends the warn? What does your /camera1/odom message look like (frame_ID)? Likely somewhere in your setup some node is looking for cam1_odom but your odom frame now is odom or odom_filtered whatever you put in the ekf parameter odom_frame. You dont get a cam2_odom warning?

@Dragonslayer The warning messages only start when I start the ekf node and in the same window where ekf is running. The odom_frame in ekf config is set to odom. Here is the config-file for ekf node. If I change the order of odom in config i.e. put odom0: /camera2/odom and odom1: /camera1/odom, the warning message change to cam2_odom warnings instead of cam1_odom warnings:

[ WARN] [1600008042.140231369]: Could not obtain transform from cam2_odom to odom. Error was "cam2_odom" passed to lookupTransform argument source_frame does not exist

This is what the /camera1/odom message looks like:

header: 
  seq: 1252
  stamp: 
    secs: 1600008164
    nsecs: 680465541
  frame_id: "cam1_odom"
child_frame_id: "cam1_base"
pose: 
  pose: 
    position: ...

Get /odom to be the frame_id in the odom messages, I would suggest. Or for a quick try set odom_frame in ekf to cam1_odom and see what happens.

@Dragonslayer I tried the latter suggestion. I changed the odom_frame: cam1_odom and the warnings went away. The data /odometry/filtered has the frame_id: cam1_odom, child_frame_id: base_link

But this means that the ekf output is in cam1_odom. The cam1_odom originates at the cam1_base start location. This would be different from the odom frame which should originate from the base_link start location?

Yes, to make it really work you have to get odom to be frame_ID in the cam1 odom message. My first suggestion so to speak.

Iam not sure how your odometry node works. You should visualize the transforms in rviz to see if this is really the case, after all ekf should place its odom frame (which in the test is cam1_odom) in a the base_link relation.

@Dragonslayer The cam node does allow me to change the name of the odometry frame via. param file. But I don't think just changing the name from "cam1_odom" to "odom" would be the right way to go about. I would need to somehow add a odom->cam1_odom tf to indicate the relation between the two odometry frames. I have been trying to figure out if that is a possibility but it doesn't seems so.

The other way I can think of is to keep using the cam1_odom frame as the fusion frame. And then use a custom node to transform the fused odometry data from cam1_odom frame to odom frame and publish both the tf and the odometry msg. Both cam1_odom and odom are stationary, so doing the above should make sense. What do you think ?

What in the code makes you think this again and again, I absolutely dont understand? You never prove your point. Why dont you just try?

@Dragonslayer Sorry at times I get confused myself. My understanding is that, I should not directly change the odometry frame name in the camera param file from "cam1_odom" to "odom" because cam1_odom represents the odometry frame that has its origin located at the starting location of the cam1 (i.e. cam1_base) in 3D space. Whereas "odom" is supposed to be the odometry frame with its origin located at the starting physical location of the robot/pc (i.e. the base_link). Now, if I were to change the name of the camera odometry frame in the param file directly from "cam1_odom" to "odom", it would mean that the camera is outputting odometry messages in the "odom" frame and that the odom frame's origin is located at the physical starting point of camera1. This contradict the notion that odom needs to have its origin located at the physical start point of the robot/pc (i.e. the base_link).

Moreover if I do the same for camera2, it becomes even more contradictory. Is my understanding valid?

@Dragonslayer Also, sorry for the delay in trying out things. I am working on it remotely and don't have access to the setup over the weekend. Therefore, before I tend to make modifications, I try to reason it out in my head. Really sorry about it all and I really appreciate your patience with me.

Have you tried it?

@Dragonslayer Not yet. Apparently my fellow student lab mates think that I was hogging the system over the weekend so I had to relinquish it for a bit. But I have it reserved for tonight. Will update as soon as I have tried it out!

@Dragonslayer I was able to run it. Both camera1 and camera2 frame_id are set to "odom" to match the odom_frame = world_frame = "odom" in robot_localizationAnd it works without any warnings or errors. Thank you for the help getting here! I have been able to test it out while the setup was stationary. I will be doing further tests by moving the setup around to check the estimate.

However, can you help me understand why this worked? It seems my understanding of the basics is wrong for odometry frames. I thought that since camera1 and camer2 generate their own VO independently, they have their own odom frames, centered at the camera start position. Whereas the odom frame has base_link at its origin. Something like this. So when we set both camera1 and camera2 odometry frame_id to "odom", doesn't it imply that they are publishing in the same "odom" frame, whereas their original frames were separate?

Glad to hear. The issue is that tf takes care of the relation of the frames to each other (frames are simply said joint-link "pairs"). Odometry just says "I moved forward 1 meter", seen from frame_ID/child_frame_ID. What this means for the robot in map space is "figured out" by tf, via its knowledge of the relation of the frames to each other (tf free, kinematics). It takes in data and frame_ID (where the data is in reference to) and outputs pose. Pose is taking all other related frames into account. The problem with your cam-odometry is that its already a running system including tf-publishing and odome_frame and all, but then you tried to put it as is into another system. There might actually be a way to get this to work...it would only be work for nothing.

@Dragonslayer sorry for the delay in getting back to you. I had a couple of exams to get through which kept my mind tied up. Just want to thank you for your help and explanation: Thank you! I will be giving the system a try while moving it around. I have one more exam to get through and then I should be able to play with the system again. Stationary it was working fine. Will let you know if I run into something that confuses me regarding this issue! Thanks once again!