ROS Speech Recognition with Pocketsphinx and Kinect

Speech recognition is one of the most important abilities for a robot because it allows you to control the robot by voice. You can give the robot a simple command “Bring beer from the fridge” and with sufficient skills the robot can perform all the necessary operations related to removing the beer from the fridge and delivering it to the right place.

In this article, I want to talk about installing and configuring all the necessary drivers and libraries for voice recognition using Pocketsphinx and Kinect Xbox 360 sensor as a microphone. I chose the Pocketsphinx package because it is one of the most popular, has an official ROS package and has good recommendations.

I performed all the steps described on the Ubuntu 12.04 system.

Install OpenKinect

First, to use the Kinect sensor as a microphone, you must install the OpenKinect library (Libfreenect): http://openkinect.org/wiki/Main_Page . Fortunately, it is available as binary packages in Ubuntu, so there shouldn't be any problems here:
')

sudo apt-get install freenect 

Together with the driver will be installed and demo applications.

When installing libfreenect in Ubuntu 12.04, it is also advised to remove the gspca modules, since they do not allow libfreenect to work in user mode (user-mode). Follow the recommendations:

 $ sudo modprobe -r gspca_kinect $ sudo modprobe -r gspca_main 

To test the driver, you can run the demo command:

 freenect-glview 

A camera image and a depth map will appear.

Kinect Connection Test

Secondly, for Kinect to function correctly as a sound receiver, the Kinect sensor must be connected to a computer via USB only version 2.0 or 1.0, and will not work via USB 3.0. You can verify that the Kinect audio device is available by typing:

 cat /proc/asound/cards 

You should see a similar output from the command:

 0 [AudioPCI ]: ENS1371 - Ensoniq AudioPCI Ensoniq AudioPCI ENS1371 at 0x2080, irq 16 1 [Audio ]: USB-Audio - Kinect USB Audio Microsoft Kinect USB Audio at usb-0000:02:03.0-1, high speed … 

If you have not seen the Kinect device record, then Kinect may be connected via USB 3.0 port. Also check that the Kinect microphone is available as an audio input device in the Ubuntu sound settings.

Kinect is now ready for use and you can begin to configure Pocketsphinx.

Installing Pocketsphinx

Installing Pocketsphinx is very easy. You need to install the package for ROS:

 sudo apt-get install ros-"groovy  hydro"-pocketsphinx 

For users of other systems (not Ubuntu), the installation will be as follows:

 git clone https://github.com/mikeferguson/pocketsphinx sudo apt-get install gstreamer0.10-pocketsphinx 

There are several manuals for using this library. They can be found here and here . I tried the first one. The manuals are very simple and you will have no difficulties with them.

Good luck in speech recognition with Pocketsphinx!