Ask Your Question
1

Kinect microphone array (linux), HARK in combination with OpenNI.

asked 2012-11-20 07:58:28 -0500

WG- gravatar image

updated 2016-10-24 09:00:39 -0500

ngrennan gravatar image

I am working on sound-source localization for our robot. For our robot vision we use OpenNI.

In my search for software for reading the Kinect microphone array I did not found anything usefull at OpenNI. The old decrepated page says however that OpenNI supports the Kinect microphone array, http://www.ros.org/wiki/kinect.

Offtopic question: Can one read the Kinect microphone array using OpenNI? If so how? And why is there no documentation for it (I looked really hard)?

Now because I could not figure out if OpenNI supported the Kinect microphone array I looked for other solutions. I came in contact with HARK-kinect. Which is in fact a kind of API for ROS to communicate with HARK. HARK comes with a driver for the Kinect to read out the microphone array. Now I have already build some schematics in HARK and tested it with the Kinect.

This morning when bicycling to the university, it came up with me that all the work I am doing now is totally pointless. HARK namely has a own driver for communicating with the Kinect, so does OpenNI (I presume).

Main question: Does anybody know if both HARK and OpenNI can communicate with the Kinect at the same time?

I could imaging namely that both drivers would conflict with each other?

I hope someone can say something clever about this.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
6

answered 2012-11-20 23:52:04 -0500

Ben_S gravatar image

updated 2012-11-21 03:47:59 -0500

With hours of pain and suffering we managed to install hark with the Kinect ALSA-drivers on our robot and i can confirm, that audio over ALSA and depth-/imagedata over OpenNI work simultaneously.

We had the whole thing, including the sound-localization demo, set up properly under electric, but now have serious problems with hark under fuerte. The documentation for hark is quite poor in my opinion and it is a lot of try and error to get it all right...


Followup because of the question in the comments: The setup was several months ago and sadly isnt fully functional anymore. However the hark kinect audio driver is still working set up by this instructions: KINECT Installation

That should be sufficient to run audio_capture and audio_play from the audio_common package. However in our case we had to alter the audio_capture/src/audio_capture.cpp by adding the following line:

  _source = gst_element_factory_make("alsasrc", "source");
+ g_object_set(G_OBJECT(_source), "device", "hw:1", NULL);
  _convert = gst_element_factory_make("audioconvert", "convert");

The hw:1 part may change, depending on the index of your kinect soundcard, seen by cat /proc/asound/cards.

For the ROS part of hark we followed this tutorial, but parts of that dont seem to work with fuerte anymore. I were not able to get it running again till now. But since its not my main focus of research, i think i will not dig any deeper in trying to fix this. If anyone else manages to get it working under fuerte, i would be very pleased to read about the steps. :)

edit flag offensive delete link more

Comments

Hello @Ben_S. It would be awesome if you would post a quick HOW-TO that finally worked for you under ROS Electric. You could save the rest of us more hours of pain and suffering! :-)

Pi Robot gravatar imagePi Robot ( 2012-11-21 03:20:54 -0500 )edit
1

I have added some information to the answer. :)

Ben_S gravatar imageBen_S ( 2012-11-21 03:49:24 -0500 )edit

@Ben_S Fantastic--thanks!

Pi Robot gravatar imagePi Robot ( 2012-11-21 04:33:13 -0500 )edit

Thanks for posting more information about this! I was trying to get it working too: http://answers.ros.org/question/43143/capturing-kinect-audio/

georgebrindeiro gravatar imagegeorgebrindeiro ( 2012-11-21 04:35:19 -0500 )edit

Thank you very much for your indept information @Ben_S!

WG- gravatar imageWG- ( 2012-11-26 10:45:47 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

7 followers

Stats

Asked: 2012-11-20 07:58:28 -0500

Seen: 4,048 times

Last updated: Nov 21 '12