ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange
Ask Your Question

Recommendations for multi-camera fusion for 360 deg image?

asked 2020-11-05 09:29:53 -0600

msmcconnell gravatar image

Looking for recommendations for ROS nodes or C++ libraries that can take multiple independent camera image streams and combine them into a single 360 deg image stream. I’m looking for something that handles stitching the images together rather than just overlaying them based on the URDF file. Any recommendations?

edit retag flag offensive close merge delete


Is the calibration between the cameras known and static? Or do you need to also do some feature matching and pose optimization before the stitching?

peci1 gravatar image peci1  ( 2020-11-09 08:26:11 -0600 )edit

You can also give a try to panotools, but I've never used it for anything else than stitching panoramas from my outdoor trips... However, it's opensource, high quality, and it should have all the features you might need.

peci1 gravatar image peci1  ( 2020-11-09 08:37:51 -0600 )edit

camera positions are static relative to each other but the system is on a moving base.

msmcconnell gravatar image msmcconnell  ( 2020-11-09 14:29:32 -0600 )edit

1 Answer

Sort by » oldest newest most voted

answered 2020-11-09 08:35:36 -0600

peci1 gravatar image

For the case of static and known camera calibration, we've created a package called virtual_camera that stitches a spherical panorama from a configurable number of input cameras (we use 5 horizontal cams and 1 ceiling-looking). It hasn't been properly published anywhere, but you can access its docs and code from . The code in there is a little old, I think it was last tested on Groovy or so, but it shouldn't be that difficult to get it running on modern ROS1 distros. We have an unpublished version for Melodic - if you were interested, PM me on ROS discourse (peci1).

Example input (all cameras stitched next to each other for the visualization, but normally it's 6 distinct images): image description

Example output:

image description

edit flag offensive delete link more


Thanks @peci1 ! This is exactly the kind of thing I am looking for. I will message you on discourse and I'll probably mark this as the answer, but I'll give some more time for others to respond if there is additional tooling out there.

msmcconnell gravatar image msmcconnell  ( 2020-11-09 14:12:50 -0600 )edit

I came here looking for the same thing. Can I get the source code?

nyxrobotics gravatar image nyxrobotics  ( 2021-09-13 04:32:59 -0600 )edit

We're now in the process of rationalizing our codebase and open-sourcing as much as possible. The virtual camera package should be released in a month or two. In the meantime, have a look at (maybe we'll merge our implementation with this one if we find the two close enough).

peci1 gravatar image peci1  ( 2021-10-05 04:55:30 -0600 )edit

Hi @peci1. I'm also facing the same thing. Have you released the virtual camera package? if so, can you share it with me?

chwbo gravatar image chwbo  ( 2022-04-06 04:47:54 -0600 )edit

I know I've already said several times it's around the corner, but a lot of things are happening now and this is not at the very top of our work list. But it is still there! I'll post here once the release is done!

peci1 gravatar image peci1  ( 2022-04-06 09:27:19 -0600 )edit

Hey, is there any update to this?

Pran-Seven gravatar image Pran-Seven  ( 2023-02-07 02:58:54 -0600 )edit

Time is scarcer than I though. This is still close to the top of my list, but never made it to the very top. But I still want to make the release!

peci1 gravatar image peci1  ( 2023-03-05 03:55:00 -0600 )edit

Hi @peci1 will appreciate if you share when it's released!

I'm new to all these and doing them as a hobby, but what started from an idle search about how 360 cameras work turned into a fascination on how complex all these stuff are from taking pictures to stitching them to turning them into 360 pictures (still not sure if 360 refers to spherical or not). Grappling with a lot of new concepts to me from the lens all the way to the software, so this will be very interesting to me as well. Thanks in advance!

KL_Newt gravatar image KL_Newt  ( 2023-04-02 02:27:11 -0600 )edit

Question Tools



Asked: 2020-11-05 09:29:53 -0600

Seen: 1,078 times

Last updated: Nov 09 '20