Kinect for Windows

edit

"Kinect for Windows" does not yet work in ROS. The official OpenNI software supports the ASUS Xtion and PrimeSense's own devices. We apply a patch (basically just adding the Kinect's USB ids) to OpenNI to support Kinect for Xbox. AFAIK no one has yet produced a similar patch for K4W.

I don't believe libfreenect officially supports K4W out-of-the-box yet either, but Drew Fisher did manage to identify the USB protocol differences (thread):

Now: it turns out the K4W device is (from a protocol level) practically identical. You have to set the K4W to use an alternate USB configuration, and the productIds are different, but other than that, all the rest of the code works as-is. The patch to get data streaming, registration and all, was about 50 lines. It was somewhat hackish, but for a proof of concept: very promising.

So, in case anyone feels like taking this on :), here's how you could go about getting K4W support in ROS (via OpenNI):

• See if Drew's libfreenect branch with experimental K4W support works with your device.
• Examine the relevant patch to understand the changes in opening the device at the USB level.
• Our build scripts and patches for OpenNI are located in the openni/drivers project. Examine our patch for the ps-engine device driver to see where Kinect for Xbox support is added - the key file is XnDeviceSensorIO.cpp.
• Modify the patch to add K4W support, following Drew's example.

Finally, an entirely different approach if you're on Windows: there's an experimental module that uses Microsoft's Kinect SDK to communicate with the device, then relays the data through OpenNI.

Since some days I can stream video data from the "Kinect for Windows" in ROS. It is working not optimal, but it works.

After launching the openni drivers (with roslaunch openni_launch openni.launch) I get this Warnings:

[ INFO] [1339081254.638412112]: Number devices connected: 1
[ INFO] [1339081254.638652716]: 1. device on bus 002:05 is a Microsoft Kinect Camera (2bf) from Microsoft (45e) with serial id '0000000000000000'
[ INFO] [1339081254.639833104]: Searching for device with index = 1
[ INFO] [1339081254.697212118]: Opened 'Microsoft Kinect Camera' on bus 2:5 with serial number '0000000000000000'
[ INFO] [1339081254.729447364]: rgb_frame_id = '/camera_rgb_optical_frame' 
[ INFO] [1339081254.729658424]: depth_frame_id = '/camera_depth_optical_frame'

[ WARN] [1339081254.735901259]: Camera calibration file /home/test/.ros/camera_info/rgb_0000000000000000.yaml not found.
[ WARN] [1339081254.736156947]: Using default parameters for RGB camera calibration.
[ WARN] [1339081254.736304452]: Camera calibration file /home/test/.ros/camera_info/depth_0000000000000000.yaml not found.
[ WARN] [1339081254.736458941]: Using default parameters for IR camera calibration.

What could I do to solve the warnings?

I just tried it with the openni_camera driver (electric/.debs/Natty/64bit) and it keeps telling me:

[ INFO] [1330948479.141759180]: No devices connected.... waiting for devices to be connected
[ INFO] [1330948482.157149616]: No devices connected.... waiting for devices to be connected

So for the moment, it seems like the Kinect for Windows does not work out of the box with openni_camera (and the system specs I posted above).

@GermanUser: Can you please give a detailed method of how you solved this?? Cant seem to get it working.

A quite dumb way to resolve the problem.

# Prepare the catkin workspace for the libfreenect
source /opt/ros/hydro/setup.bash
mkdir -p ~/libfreenect_catkin_ws/src
cd ~/libfreenect_catkin_ws/src
catkin_init_workspace
cd ~/libfreenect_catkin_ws/
catkin_make
source devel/setup.bash

# Get the latest sources from the github repository
cd ~/libfreenect_catkin_ws/src/ 
git clone https://github.com/OpenKinect/libfreenect.git

# Grab the source of the freenect_stack
git clone https://github.com/ros-drivers/freenect_stack.git tmp
mv tmp/freenect_stack .
mv tmp/freenect_camera .
mv tmp/freenect_launch .
rm -rf tmp/

# catkinize the "vanilla" libfreenect sources
cd ~/libfreenect_catkin_ws/src/libfreenect/
wget -q https://raw.githubusercontent.com/ros-drivers-gbp/libfreenect-release/master/patches/package.xml

replace :{version} in package.xml file with the 0.4.3 or the up-to-date version to the moment of accessing this page.

Original File

<?xml version="1.0"?>
<package>
  <name>libfreenect</name>
  <version>:{version}</version>

Updated file

<?xml version="1.0"?>
<package>
  <name>libfreenect</name>
  <version>0.4.3</version>

After that install the required packages using rosdep

# Installing package dependencies
rosdep update
rosdep install libfreenect
sudo aptitude install ros-hydro-camera-info-manager ros-hydro-diagnostic-msgs ros-hydro-diagnostic-updater ros-hydro-dynamic-reconfigure

Now you can build the packages using catkin_make_isolated

# use sudo if needed
catkin_make_isolated
# change hydro if you are using different distribution
catkin_make_isolated --install --install-space=/opt/ros/hydro/

I tried to use Kinect for Windows for weeks and always get "No device connected" error. Once I changed to Kinect for Xbox everything works... so my suggestion is just to get a Kinect for Xbox or wait until the driver comes out...