# Converting Pixel Coordinates to spatial coordinates: Python, Ros

Hi,

So currently, i am reading an image from my robot's camera, located on its hand. I want to obtain the set of coordinates of a certain object, from both the camera's perspective and the robot's base perspective, given a certain pixel coordinate from the camera i.e. 340x200 (location of my object on the camera image)

I have been using the equation on the second line from this link

But i haven't obtained any good results.(I have inverted the equation,, since i want a set of 3d coordinates from a pair of pixel coordinates and not the opposite as the original equation would give)

So because i didnt get any good result, i tried breaking down the equation. The matrix [R|T] converts the [x,y,z,1] coordinates to the base frame right?? (assuming [R|T] is with respect to the base frame)

So if I wanted the coordinate WRT to the camera's frame, could I just use the equation, without the [R|T] matrix? i.e [u,v,1] = A[X,Y,Z]?

If not, how should i proceed to convert the pixel coordinates to spatial coordinates

edit retag close merge delete

Sort by ยป oldest newest most voted

I didn't really read the details of your question very closely, nor did I look at the link. It seems to me that there are so many different conventions and coordinate systems used for camera geometry it often leads to confusion. If you are using a camera calibrated with camera_calibration, and the camera information is available as a CameraInfo topic, then I'd recommend using the image_geometry package. The two packages are specifically designed to work together, and the image_geometry package contains a number of useful functions. Check out the Python API and the C++ API.

more

I tried using image_geometry, but i get a 'NoneType' object has no attribute '__getitem__' error when using it. Here's what i did

self.img_geo = PinholeCameraModel()
(r_x,r_y,r_z) = self.img_geo.projectPixelTo3dRay((359,28))


any idea whats going on?

( 2016-06-20 08:34:11 -0500 )edit

I obviously imported the packages beforehand

( 2016-06-20 08:34:40 -0500 )edit

You need to read in the camera calibration parameters

cam_info = rospy.wait_for_message("/camera1/camera_info", CameraInfo, timeout=None)
img_proc = PinholeCameraModel()
img_proc.fromCameraInfo(cam_info)
n = img_proc.projectPixelTo3dRay((30,30))

( 2016-06-20 10:01:23 -0500 )edit

So I added that bit to my code, but it seems that the code stops at rospy.wait_for_message and doesnt compute the rest

( 2016-06-20 10:23:31 -0500 )edit

So I figrued that that cam_info was basically just subscribing to a camera, so i replcaed the wait_for_message with directly subscribing to the camera. and the error that i get says that 'Subscriber' object has no attribute 'K'

( 2016-06-20 10:48:38 -0500 )edit

Your first comment indicates, most likely, that you weren't publishing a CameraInfo message on the /camera1/camera_info topic. Your second comment indicates that you are passing the created subscriber into the fromCameraInfo method. That is not correct.

( 2016-06-20 11:37:44 -0500 )edit

The wait_for_message function creates a subscriber, waits until the topic is published, and returns a copy of the message that was received. My code doesn't return a subscriber, it returns a CameraInfo message that has been filled out with information from camera calibration.

( 2016-06-20 11:40:16 -0500 )edit

So given that I revert back to wait_for_message, how do i publish a CameraInfo message? You can't publish to the camera right, you can only subscribe to it

( 2016-06-20 11:56:29 -0500 )edit