Kinect microphone array (linux), HARK in combination with OpenNI.

asked 2012-11-20 07:58:28 -0500

WG-

109 ●7 ●7 ●11

updated 2016-10-24 09:00:39 -0500

ngrennan
1 ●1 ●1

I am working on sound-source localization for our robot. For our robot vision we use OpenNI.

In my search for software for reading the Kinect microphone array I did not found anything usefull at OpenNI. The old decrepated page says however that OpenNI supports the Kinect microphone array, http://www.ros.org/wiki/kinect.

Offtopic question: Can one read the Kinect microphone array using OpenNI? If so how? And why is there no documentation for it (I looked really hard)?

Now because I could not figure out if OpenNI supported the Kinect microphone array I looked for other solutions. I came in contact with HARK-kinect. Which is in fact a kind of API for ROS to communicate with HARK. HARK comes with a driver for the Kinect to read out the microphone array. Now I have already build some schematics in HARK and tested it with the Kinect.

This morning when bicycling to the university, it came up with me that all the work I am doing now is totally pointless. HARK namely has a own driver for communicating with the Kinect, so does OpenNI (I presume).

Main question: Does anybody know if both HARK and OpenNI can communicate with the Kinect at the same time?

I could imaging namely that both drivers would conflict with each other?

I hope someone can say something clever about this.

answered 2012-11-20 23:52:04 -0500

Ben_S

2500 ●14 ●29 ●53

updated 2012-11-21 03:47:59 -0500

With hours of pain and suffering we managed to install hark with the Kinect ALSA-drivers on our robot and i can confirm, that audio over ALSA and depth-/imagedata over OpenNI work simultaneously.

We had the whole thing, including the sound-localization demo, set up properly under electric, but now have serious problems with hark under fuerte. The documentation for hark is quite poor in my opinion and it is a lot of try and error to get it all right...

Followup because of the question in the comments: The setup was several months ago and sadly isnt fully functional anymore. However the hark kinect audio driver is still working set up by this instructions: KINECT Installation

That should be sufficient to run audio_capture and audio_play from the audio_common package. However in our case we had to alter the audio_capture/src/audio_capture.cpp by adding the following line:

  _source = gst_element_factory_make("alsasrc", "source");
+ g_object_set(G_OBJECT(_source), "device", "hw:1", NULL);
  _convert = gst_element_factory_make("audioconvert", "convert");

The hw:1 part may change, depending on the index of your kinect soundcard, seen by cat /proc/asound/cards.

For the ROS part of hark we followed this tutorial, but parts of that dont seem to work with fuerte anymore. I were not able to get it running again till now. But since its not my main focus of research, i think i will not dig any deeper in trying to fix this. If anyone else manages to get it working under fuerte, i would be very pleased to read about the steps. :)

edit flag offensive delete link

Comments

Hello @Ben_S. It would be awesome if you would post a quick HOW-TO that finally worked for you under ROS Electric. You could save the rest of us more hours of pain and suffering! :-)