ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | Q&A
Ask Your Question

Natural language processing in ROS

asked 2014-07-07 04:13:32 -0500

Mehdi. gravatar image

I am interested in decomposing a user's sentence and recognizing the subject, the object and what the user is saying (declarative sentence or asking for time, place, person etc...)

I found this :

It works decently but the project is dead and the last commit was 3 years ago. I wanted to ask if there is already something as a well maintained ROS package. I heard about KnowRob and checked the wiki but they really use a complicated vocabulary and I couldn't understand if KnowRob has this ability of natural language processing or not.

edit retag flag offensive close merge delete


I and my co-researchers have developed chat-bot like scripts using NLTK, the prospects are to develop an interactive robot HEAD (using ROS), have a look here -- -- may not be much help though !

Arkapravo gravatar image Arkapravo  ( 2014-07-07 16:16:09 -0500 )edit

Thanks for sharing this, do you have any demo code?

Mehdi. gravatar image Mehdi.  ( 2014-07-09 21:54:21 -0500 )edit

@Mehdi, this is a sample we developed the rest of our work on this !

Arkapravo gravatar image Arkapravo  ( 2014-07-10 11:31:19 -0500 )edit

3 Answers

Sort by ยป oldest newest most voted

answered 2014-07-07 15:32:53 -0500

fergs gravatar image

updated 2014-07-07 15:33:58 -0500

Completely open ended NLP is still a fairly open problem. Most NLP systems tend to still be application specific.

You might want to take a look at the Natural Language Toolkit (NLTK) ( ) which is a Python toolkit for building NLP applications. While not integrated with ROS, since it is Python, it would be very easy for you to connect your application using rospy.

If you are trying to get audio input, this gets even harder. Right now, the only speech recognition package in ROS that I am aware of is my simple wrapper around pocketsphinx-gstreamer ( )

edit flag offensive delete link more

answered 2014-07-07 15:50:50 -0500

d gravatar image

updated 2014-07-07 15:53:03 -0500

+1 for @fergs answer.

in addition to nltk, i would also look at stanford nlp

to have perfect nlp (i.e. a dialog between user and robot) is really hard. but some simple keyword spotting algorithms + some simple grammars could do most of the interaction with the robot. looking at the use cases you described above (asking for time, askint for place, person, etc). that could be done with a few grammars and keyword spottings (i.e. find the words "what", "who" in the context a simple grammar such as ( what | who | where) ( is | are ) <subject>.

on speech recognition, you can use pocketsphinx, but the accuracy is really low, unless you have to train your own pocketsphinx, which could be very expensive. i would recommend to use sphinx4 (since it comes with a large language model and dictionary already).

otherwise, you can hack siri or android speech recognition and call those services via http for your robot.

at&t and nuance also provide great speech recognition services. they cost a little once you go to production, but for development & research, the usage volume is usually low, and it's almost free. the good thing is that they're very good, so you have a COTS node that you don't have to worry about and focus on your core problem.

edit flag offensive delete link more



I am currently working on a Nuance VoCon speech recognition node and the recognition performance is really great. We use Nuance because we have to have German language recognition and the German voice models for pocketsphinx are very bad. Yet, we don't know if we are allowed to publish our code.

BennyRe gravatar image BennyRe  ( 2014-07-08 01:13:59 -0500 )edit

VoCon also supports some kind of semantics parsing, but I have not had the time to take a look at it.

BennyRe gravatar image BennyRe  ( 2014-07-08 01:53:27 -0500 )edit

answered 2014-07-10 01:23:36 -0500

moritz gravatar image

KnowRob does not focus on natural-language processing nor dialog with humans, but rather provides a semantic platform for integrating knowledge sources and reasoners. We have investigated the parsing and understanding of natural-language sentences in instructions from the Web a few years ago, the code is available in the comp_ehow package in . Note, however, that this work aimed at a deeper semantic understanding and the alignment of the resulting descriptions with the knowledge base. Therefore, it may be too slow for fluent natural interaction.

edit flag offensive delete link more


By the way, what do you mean with "complicated vocabulary" -- the language on the wiki or the language used within the system? The latter is the same that ORO also uses, it's the commonly used Cyc ontology.

moritz gravatar image moritz  ( 2014-07-10 01:24:35 -0500 )edit

I meant the language in the wiki, it uses a lot of technical words not so obvious for somebody totally new to this field

Mehdi. gravatar image Mehdi.  ( 2014-07-10 20:41:45 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools



Asked: 2014-07-07 04:13:32 -0500

Seen: 2,779 times

Last updated: Jul 10 '14