External camera as SLAM sensor/input for navigation

asked 2023-05-16 05:13:09 -0500

Theodoro Cardoso
20 ●3 ●6 ●10

Hi everyone!

If this question has been asked somewhere else, please point me there.

Consider the following challenge: onboard navigation is not accurate enough. We'd like to enhance it using outboard cameras (fixed in the environment)

Given the external camera transform to a global frame is known and we can estimate with some variance the transform from the camera to the robot, would it be possible to use that as an input to the robot's navigation algorithm?

For illustration purposes, let's imagine the image below was obtained by the external camera and we're interested in navigating a single robot...

aruco_robots

How one would go about integrating that on nav2? Any hints or resources would be appreciated.

Comments

Do you understand how the Transform Tree is used to keep track of the pose of a robot?
Are you familiar with how odometry drift-correction is implemented (for example, by AMCL)?

Mike Scheutzow ( 2023-05-16 07:51:37 -0500 )edit

Mike, thanks for your comment.

My understanding is that you can find the robot's pose in any arbitrary frame given there's a connection between them in the Transform Tree.
I'm familiar with the motion, sensing, and resampling loop on the MCL algorithm, but not how it is implemented in ROS.

Theodoro Cardoso ( 2023-05-16 08:57:12 -0500 )edit

add a comment

answered 2023-05-16 13:41:22 -0500

Mike Scheutzow
4918 ●2 ●430 ●45

updated 2023-05-18 08:13:06 -0500

would it be possible to use that as an input to the robot's navigation algorithm?

Yes, using global localization as a component of navigation is very common. You will either use the value directly (if it is frequent enough), or you use it to fine-tune the robot's odometry data when it becomes available. if your robot's transform tree looks like this:

map -> odom -> base_link

then you use the new data to calculate a new map->odom transform. You ask specifically about "SLAM Navigation": what I describe here is relevant only to the "Localization" portion of SLAM.

Update: What you describe is 99% not a "nav2" problem, it's a image processing problem. You need to find an object within an image, and estimate the location of the object relative to the camera lens. There is not a standard ros package that does this for you (because the simplifying assumptions are different for each environment.) A web search should give you a large number of hits of other people doing this, but make sure you understand what assumptions they made.

edit flag offensive delete link

Comments

Sounds good!

Are there any specific packages (e.g. 'robot_localization') or other resources/tutorials that come to mind when we talk about using global localization in concert with nav2?

If it helps to narrow it down, the application consists of an indoor environment (mapped in advance) with dynamic obstacles like people and boxes and multiple outboard cameras that see the robot when it's not occluded.

Theodoro Cardoso ( 2023-05-16 13:57:31 -0500 )edit

the application consists of an indoor environment (mapped in advance)

Let's review: you put "SLAM" in the question when you're not doing SLAM, and you included an outdoor image in your question when the application is indoor. Got it.

Mike Scheutzow ( 2023-05-18 08:02:57 -0500 )edit

Well, I'm sure I could have done a better job explaining it and I apologize for the lack of clarity.

I believed that updating the base map with obstacles would turn the localization-only problem into SLAM but that's apparently not the case.

The point of this loosely formulated question was to get a better understanding of how to use sensors that are not in the robot to aid navigation results, hopefully getting a hint on where to start (packages, repositories, or any helpful material on the topic)

Theodoro Cardoso ( 2023-05-18 17:40:13 -0500 )edit

add a comment

External camera as SLAM sensor/input for navigation

Comments

1 Answer

Comments

Question Tools

Stats

Related questions

External camera as SLAM sensor/input for navigation edit

Comments

1 Answer

Comments

Question Tools

Stats

Related questions

External camera as SLAM sensor/input for navigation