Ask Your Question
3

Recommendations for multi-camera fusion for 360 deg image?

asked 2020-11-05 09:29:53 -0500

msmcconnell gravatar image

Looking for recommendations for ROS nodes or C++ libraries that can take multiple independent camera image streams and combine them into a single 360 deg image stream. I’m looking for something that handles stitching the images together rather than just overlaying them based on the URDF file. Any recommendations?

edit retag flag offensive close merge delete

Comments

Is the calibration between the cameras known and static? Or do you need to also do some feature matching and pose optimization before the stitching?

peci1 gravatar image peci1  ( 2020-11-09 08:26:11 -0500 )edit

You can also give a try to panotools, but I've never used it for anything else than stitching panoramas from my outdoor trips... However, it's opensource, high quality, and it should have all the features you might need.

peci1 gravatar image peci1  ( 2020-11-09 08:37:51 -0500 )edit

camera positions are static relative to each other but the system is on a moving base.

msmcconnell gravatar image msmcconnell  ( 2020-11-09 14:29:32 -0500 )edit

1 Answer

Sort by » oldest newest most voted
2

answered 2020-11-09 08:35:36 -0500

peci1 gravatar image

For the case of static and known camera calibration, we've created a package called virtual_camera that stitches a spherical panorama from a configurable number of input cameras (we use 5 horizontal cams and 1 ceiling-looking). It hasn't been properly published anywhere, but you can access its docs and code from http://cmp.felk.cvut.cz/nifti/sw/virt... . The code in there is a little old, I think it was last tested on Groovy or so, but it shouldn't be that difficult to get it running on modern ROS1 distros. We have an unpublished version for Melodic - if you were interested, PM me on ROS discourse (peci1).

Example input (all cameras stitched next to each other for the visualization, but normally it's 6 distinct images): image description

Example output:

image description

edit flag offensive delete link more

Comments

Thanks @peci1 ! This is exactly the kind of thing I am looking for. I will message you on discourse and I'll probably mark this as the answer, but I'll give some more time for others to respond if there is additional tooling out there.

msmcconnell gravatar image msmcconnell  ( 2020-11-09 14:12:50 -0500 )edit

I came here looking for the same thing. Can I get the source code?

nyxrobotics gravatar image nyxrobotics  ( 2021-09-13 04:32:59 -0500 )edit

We're now in the process of rationalizing our codebase and open-sourcing as much as possible. The virtual camera package should be released in a month or two. In the meantime, have a look at https://github.com/tu-darmstadt-ros-p... (maybe we'll merge our implementation with this one if we find the two close enough).

peci1 gravatar image peci1  ( 2021-10-05 04:55:30 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

4 followers

Stats

Asked: 2020-11-05 09:29:53 -0500

Seen: 295 times

Last updated: Nov 09 '20