ccny_rgbd is unable to detect features

asked 2013-02-26 11:13:03 -0500

32 ●5 ●6 ●10

Hi everyone,

Consider these pioneer datasets: http://vision.in.tum.de/data/datasets/rgbd-dataset/download#freiburg2_pioneer_slam2

They represent the specialties of my target application nicely, so I'm keen on testing some VO engines with them.

I've had a problem with CCNY RGBD: it works well with Kinect, but when I try to run it with Pioneer dataset it produces the following errors for each input frame:

[VO 147] Fr: 0.0 GFT[400][0]: 15.0 ICPProbModel[1]  0.1 TOTAL 15.2
[ WARN] [1361918845.707617958]: No features detected.
[ WARN] [1361918845.707702069]: Could not estimate motion from RGBD data, using Identity transform.
[VO 148] Fr: 0.0 GFT[400][0]: 13.9 ICPProbModel[1]  0.2 TOTAL 14.1
[ WARN] [1361918845.752454258]: [ICP] Not enough correspondences (0 of 15 minimum). Leacing ICP loop
[ WARN] [1361918845.752498383]: Could not estimate motion from RGBD data, using Identity transform.

I'm using the default settings. Color images are in BGR8 format, depth in 32FC1 or MONO16 (have tried both of them).

It is really strange that ccny_rgbd is unable to find any features on these images, I suppose there is some general issue, wrong image format maybe?

answered 2013-02-26 11:26:15 -0500

Ivan Dryanovski

4954 ●52 ●70 ●86 http://robotics.ccny.cuny.edu/

updated 2013-02-27 02:37:26 -0500

ccny_rgbd currently requires depth images in 16UC1 format, in mm. A ticket was opened today to allow 32FC1 support.

Can you post the output of when you feed in the MONO16 depth topic?

This might be an issue with the depth image. Features that fall on a depth value of 0 are discarded as invalid. To test this, you can try running feature_viewer on your dataset. Set the show_keypoints flag to true. It should show you the keypoints detected on the RGB image, before the invalid ones are discarded.

Also maybe post your launch file so I can take a look how the topics are getting remapped.

Edit:

I just added support for 32FC1, and also clearer instructions in all launch files how to remap topics coming directly from OpenNI when rgbd_image_proc is not used. Passing in 16UC1 is still (slightly) more efficient. See ticket.

edit flag offensive delete link

Comments

Ivan, thanks it's working now, I converted the depth images into 16UC1 in millimeters. But anyway, in some parts of these datasets VO is unable to find enough features to track [http://pastebin.com/xFRg1V4c]. Maybe I need to switch to another kind of features?

Spym ( 2013-02-26 21:54:18 -0500 )edit

Is MONO16 different from 16UC1? I'm not clear on this. I'm working on 32FC1 support so the datasets can be used directly.

Ivan Dryanovski ( 2013-02-27 02:01:29 -0500 )edit

As for the features, try changing the type of feature detector. If you use SURF, you might need to switch to QVGA, depending on your machine. If it still doesnt work well, let me know which dataset it is, and I'll see if I can implement some fallback strategies when features arent present

Ivan Dryanovski ( 2013-02-27 02:03:41 -0500 )edit

I suppose MONO16 is just alias for 16UC1. When I tried MONO16, I used it with resolution of 0.2mm per LSB, thereby VO discarded almost entire frame because depth values were above max_range threshold. I just changed the scale to 1mm/LSB and it that's it.

Spym ( 2013-02-27 02:14:37 -0500 )edit

About features: thanks, I'll give it a try a bit later today, stay tuned.

Spym ( 2013-02-27 02:16:15 -0500 )edit

See edit - added 32FC1 support. What's LSB?

Ivan Dryanovski ( 2013-02-27 02:39:13 -0500 )edit

Yep, I already started following your repository on github, so I've seen an update on that issue. LSB stands for Least Significant Bit, i.e. amount of depth per one bit in our case.

Spym ( 2013-02-27 02:41:34 -0500 )edit

Ivan, sorry for the offtopic, but I have a minor suggestion: it would be great to avoid direct stdout access from rgbd applications. Consider this: http://goo.gl/Kj39y, I suggest to replace printf() with ROS_DEBUG() here and everywhere else, to avoid the log pollution.

Spym ( 2013-02-27 03:02:14 -0500 )edit

see more comments

ccny_rgbd is unable to detect features

1 Answer

Comments

Question Tools

Stats

Related questions

ccny_rgbd is unable to detect features edit

1 Answer

Comments

Question Tools

Stats

Related questions

ccny_rgbd is unable to detect features