Ask Your Question

Converting Pixel Coordinates to spatial coordinates: Python, Ros

asked 2016-06-17 14:14:45 -0500

laurent gravatar image


So currently, i am reading an image from my robot's camera, located on its hand. I want to obtain the set of coordinates of a certain object, from both the camera's perspective and the robot's base perspective, given a certain pixel coordinate from the camera i.e. 340x200 (location of my object on the camera image)

I have been using the equation on the second line from this link

But i haven't obtained any good results.(I have inverted the equation,, since i want a set of 3d coordinates from a pair of pixel coordinates and not the opposite as the original equation would give)

So because i didnt get any good result, i tried breaking down the equation. The matrix [R|T] converts the [x,y,z,1] coordinates to the base frame right?? (assuming [R|T] is with respect to the base frame)

So if I wanted the coordinate WRT to the camera's frame, could I just use the equation, without the [R|T] matrix? i.e [u,v,1] = A[X,Y,Z]?

If not, how should i proceed to convert the pixel coordinates to spatial coordinates

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2016-06-17 17:27:12 -0500

I didn't really read the details of your question very closely, nor did I look at the link. It seems to me that there are so many different conventions and coordinate systems used for camera geometry it often leads to confusion. If you are using a camera calibrated with camera_calibration, and the camera information is available as a CameraInfo topic, then I'd recommend using the image_geometry package. The two packages are specifically designed to work together, and the image_geometry package contains a number of useful functions. Check out the Python API and the C++ API.

edit flag offensive delete link more


I tried using image_geometry, but i get a 'NoneType' object has no attribute '__getitem__' error when using it. Here's what i did

self.img_geo = PinholeCameraModel()
(r_x,r_y,r_z) = self.img_geo.projectPixelTo3dRay((359,28))

any idea whats going on?

laurent gravatar image laurent  ( 2016-06-20 08:34:11 -0500 )edit

I obviously imported the packages beforehand

laurent gravatar image laurent  ( 2016-06-20 08:34:40 -0500 )edit

You need to read in the camera calibration parameters

cam_info = rospy.wait_for_message("/camera1/camera_info", CameraInfo, timeout=None)
img_proc = PinholeCameraModel()
n = img_proc.projectPixelTo3dRay((30,30))
jarvisschultz gravatar image jarvisschultz  ( 2016-06-20 10:01:23 -0500 )edit

So I added that bit to my code, but it seems that the code stops at rospy.wait_for_message and doesnt compute the rest

laurent gravatar image laurent  ( 2016-06-20 10:23:31 -0500 )edit

So I figrued that that cam_info was basically just subscribing to a camera, so i replcaed the wait_for_message with directly subscribing to the camera. and the error that i get says that 'Subscriber' object has no attribute 'K'

laurent gravatar image laurent  ( 2016-06-20 10:48:38 -0500 )edit

Your first comment indicates, most likely, that you weren't publishing a CameraInfo message on the /camera1/camera_info topic. Your second comment indicates that you are passing the created subscriber into the fromCameraInfo method. That is not correct.

jarvisschultz gravatar image jarvisschultz  ( 2016-06-20 11:37:44 -0500 )edit

The wait_for_message function creates a subscriber, waits until the topic is published, and returns a copy of the message that was received. My code doesn't return a subscriber, it returns a CameraInfo message that has been filled out with information from camera calibration.

jarvisschultz gravatar image jarvisschultz  ( 2016-06-20 11:40:16 -0500 )edit

So given that I revert back to wait_for_message, how do i publish a CameraInfo message? You can't publish to the camera right, you can only subscribe to it

laurent gravatar image laurent  ( 2016-06-20 11:56:29 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower


Asked: 2016-06-17 14:14:45 -0500

Seen: 2,271 times

Last updated: Jun 17 '16