Face Recognition with Kinect

asked 2011-02-24 05:41:41 -0500

Scott
693 ●33 ●45 ●53

updated 2016-10-24 09:00:49 -0500

ngrennan
1 ●1 ●1

Update: Patrick Goebel has posted a great new package and tutorial for robust face detection. I have posted an answer to this question below (See Pi Robot example of robust face detection with Kinect:) if you find Patrick's work helpful, please vote up the answer to move it to the top of this question. (Scott)

I am moving this thread to the Answers site. It was started on ros-kinect before Answers was released, so trying to be a good ROS citizen. :)

Hi Radu I am able to pretty easily do face detection via the kinect using the built in recognition python package that comes inside the vision_opencv stack. For those on this thread here are the steps... roscd to openni_camera go into the launch directory, edit the openni_kinect.launch file and add the following line to remap the default video output topic from kinect to the topic that the face recognition listens node listens on.

Run the face recognition node with.. rosrun opencv_tests rosfacedetect.py (located in vision_opencv stack)

If performance lags a bit, use rosrun dynamic_reconfigure reconfigure_gui to adjust settings (I tend to use the QVGA options)

For any newbies reading this, you can use rosrun rviz rviz and add a pointcloud2 option that listens on /camera/depth/points2 topic to see the point cloud.

OK....So where does this get us? I now have the face found and boxes to a region of interest and can visualize it. I can capture and see the point cloud data of the larger scene. What I want to do to feed this into your recognition flow Radu is to then use the detected face ROI to crop the point cloud down to just the points that represent the face.

...and that is where I am stuck... Any tips from anyone on how to use ROI info to crop a point cloud image on the fly would be greatly appreciated.

And for those that are interested in facial recognition, hopefully this helps you get started.

Best Regards, -Scott

edit retag flag offensive close merge delete

Comments

FYI: you can edit an existing answer instead of reposting. I'm going to delete your duplicate question with they typo in the name.

Tully ( 2011-02-24 05:59:39 -0500 )edit

Is this a question?

Asomerville ( 2011-05-11 11:03:30 -0500 )edit

add a comment

4

answered 2011-02-27 07:49:00 -0500

watts
584 ●2 ●7 ●10

Scott, I've been keeping an eye on your thread on ros-users.

I've written a small package "face_builder" to detect faces using OpenCV and return a clustered point cloud of the face. It works as a stand-alone node, but it should also work as a library. It publishes clustered faces on the "faces" topic (type: sensor_msgs/PointCloud2) and publishes a "detected_faces" image for debugging.

It builds against trunk of PCL right now and ROS Diamondback. If you need to patch it to get it to build, let me know.

Not having a great place to put it, I uploaded it to the "PR" wiki: http://pr.willowgarage.com/wiki/Useful_general_references Look for "face_builder.tar"

Let me know what you find with face recognition.

edit flag offensive delete link

Comments

Has any more work been done on your face_builder? I'm going to hack on it some.

Murph ( 2011-04-07 08:06:11 -0500 )edit

Hi Murph, glad to see your post. I have been off in rgbdslam/kinect/octomap land and my progress there has hit a bit of a wall. So in the interim, I am coming back to put in some more cycles on this as well. Please post your progress and tips and I will do the same as I try to follow after you.

Scott ( 2011-05-17 03:36:54 -0500 )edit

add a comment

4

answered 2011-06-04 05:10:49 -0500

Scott
693 ●33 ●45 ●53

updated 2011-06-05 01:31:15 -0500

Pi Robot example of robust face detection with Kinect:

Patrick Goebel has just announced a fantastic new package for face detection. Please go to the following link to try this out yourself. It is very well documented and robust. If you find Patrick's work to be helpful, please vote up this answer to move it to the top of this very popular question.

Link to Pi Face Tracker work: http://www.ros.org/wiki/pi_face_tracker

edit flag offensive delete link

add a comment

2

answered 2012-04-10 07:16:38 -0500

Scott
693 ●33 ●45 ●53

2d-face-recognition-basic-implementation

Preface: Face recognition is moving fast these days, this entry on answers.ros alone has had over 2000 hits since I first posted it. This answer is my attempt at a simple, yet effective approach to face recognition that I hope some find useful. It is very much a proof of concept and I hope those that give it a try will extend it and share their improvements with me and others.

Overview: The 2d face recognition methodology I present here is an easy and straightforward implementation that combines a modified version of the pi_face_tracker (pi_vision package) with the objects_of_daily_use package.

The pi_face_tracker code is used to “find” a persons head, crop it down to an roi (region of interest) then publish that image over a rostopic (ie the modified ros2opencv.py file in the pi_face_tracker package).

Setup: Link to my modified files for pi_face_tracker and objects_of_daily_use_finder: http://code.google.com/p/2d-face-recognition-basic/downloads/list

1st install the pi_vision package (gives you pi_face_tracker and ros2opencv), install the objects_of_daily_use package and install any dependencies they require. http://www.ros.org/wiki/pi_face_tracker http://www.ros.org/wiki/objects_of_daily_use_finder

Backup the ros2opencv.py file located in the ros2opencv/src folder of the pi_vision package Download my modified version of the file and put it in the src folder, replacing the original version

Download the two launch files and put them in the launch folder of the objects_of_daily_use_finder package.

Create two directories in the objects_of_daily_use_finder directory. faces_database faces_images

Edit both the launch files to give the FULL path to both of these files (relative path with ~ will not work)

Acquisition and Training of face images: Run : roslaunch ros2opencv openni_node.launch This will launch the driver for the kinect

cd to the faces_images folder_recognize] example mkdir scott cd to that persons named directory (VERY Important otherwise you will have to move all of you images) Note: you will repeat this step for each new person you want to learn and recognize You do not need to rename the images, as they will all be referenced by the parent folder that has their name

Run: rosrun image_view image_view image:=/camera/rgb/image_color This will open image view which lets us view raw images from the kinect Have a person face the kinect While the image_view screen is active, right click your mouse. This will save a jpg image to the folder you started the image_view from Have the person turn to different angles and make different facial expressions and each time right click the mouse again to get a new image

Cropping the images: Install an image editing program (I used GIMP) Open each file one by one, select a rectangle around the head region, then select crop to selection. Resave the image

Training: Run: roslaunch objects_of_daily_use_finder 2d_build_faces_db.launch This will do the training and will place the files needed for recognition in the faces_database directory you created earlier (as long as you properly edited the path in the launch files before running this ... (more)

edit flag offensive delete link

Comments

For 2D face_recognition, you can also use the following package http://www.ros.org/wiki/face_recognition

Pouyan ( 2012-04-12 23:32:31 -0500 )edit

add a comment

-1

answered 2011-05-17 10:50:45 -0500

Xi
1

Watts and Scott,

I have installed Ubuntu 10.04 and ROS and make Watts' face_builder compiled successfully. The executable i got in the bin is kfbuilder. Then i make a new terminal, and run roscore. Then i go back to the original terminal, and type "rosrun face_builder kfbuilder". However, the prompt just stay there and nothing happens. So i was wondering if this program is face recognition in an still image or real-time video stream from a camera?

Thanks!

Xi

edit flag offensive delete link

Comments

Xi, check that the topics are named correctly using "rxgraph". I think the default topic names changed since I posted the code.

watts ( 2011-05-17 13:47:52 -0500 )edit

Watts, I cannot use rxgraph and run kfbuilder at the same time. When i just run rxgraph, there is just /rsout in the graph. So what should the results look like when i run your package?

Xi ( 2011-05-18 06:20:08 -0500 )edit

Xi there is a check box in rxgraph that says list all topics. Try that.

Scott ( 2011-05-18 23:41:00 -0500 )edit

add a comment

-1

answered 2011-02-27 15:37:28 -0500

Scott
693 ●33 ●45 ●53

Thanks Kevin, I was able to fetch the code and will try it out this week and post a reply. Very excited to try it out. I was getting reasonably adept at OpenCV prior to ROS days, but still have much to learn about bridging the two, even beyond the tutorials on the ROS site, and being able to learn from some examples would really be great.

Question for Tully or the other site Admin's if they see this... When people correspond on this site and they want to share code, is there a common location or way to do that? Thanks, -Scott

edit flag offensive delete link

add a comment

Face Recognition with Kinect

Comments

5 Answers

Comments

Comments

Comments

Question Tools

Stats

Related questions

Face Recognition with Kinect edit

Comments

5 Answers

Comments

Comments

Comments

Question Tools

Stats

Related questions

Face Recognition with Kinect