FreeSWITCH + mod_unimrcp + VoiceNavigator. Synthesis and recognition of Russian speech in FreeSWITCH

In the continuation of a series of articles on the integration of Asterisk and VoiceNavigator I will describe the integration of VoiceNavigator with FreeSWITCH .

On Habré there are several articles and mentions of FS, but it’s not as popular with Asterisk))
')
FreeSWITCH, being a continuation and development of ideas embedded in Asterisk, has a truly modular architecture, where all the basic functionality, as far as possible, is rendered into separate modules. This allows for greater flexibility in creating solutions scalable from softphone to softswitch. FreeSWITCH can be used as a switch, PBX, media gateway or media server for IVR applications that use XML files to control the call processing algorithm.

For FreeSWITCH, unlike Asterisk, there is no need to separately compile UniMRCP. UniMRCP support has already been added and works out of the box as a module asr_tts / mod_unimrcp.

The following describes installing FreeSWITCH, connecting VoiceNavigator to it, and creating a test voice application.

Install FreeSWITCH

The installation of FS was described both on Habré and in other sources, therefore I’ll dwell on it briefly. Fortunately, everything is simple and affordable. The following describes the installation on CentOS.

Download the distribution from www.freeswitch.org . At the time of writing, the latest version of the distribution is 1.0.6.

Before compiling, we install the necessary packages for FreeSWITCH and its modules:
yum install autoconf automake libtool gcc-c++ ncurses-devel make expat-devel zlib zlib-devel libjpeg-devel unixODBC-devel openssl-devel gnutls-devel libogg-devel libvorbis-devel curl-devel gdbm gdbm-devel

To add a UniMRCP module in the modules.conf file, uncomment the line asr_tts / mod_unimrcp.
You also need to uncomment languages ​​/ mod_perl for the ability to work with a dialplan from Perl.

Next, the standard compilation procedure:

  ./configure
 make
 make check
 make install
 make cd-sounds-install
 make cd-moh-install
 make cd-sounds-ru-install
 make samples 

After installation, FreeSWITCH is already configured and ready to use.
To enable the use of the Russian language as the default language, you need to add to /usr/local/freeswitch/conf/vars.xml

  <X-PRE-PROCESS cmd = "set" data = "default_language = en" /> 

To run, use the / usr / local / freeswitch / bin / freeswitch command
To test the operation, set up the softphone with user parameters - 1000, password - 1234 and register with the server, call 5000. If FreeSWITCH is set correctly, you will hear a test IVR.

Connecting VoiceNavigator to FreeSWITCH

1) Add the mod_unimrcp module load.
To do this, add the following file to the /usr/local/freeswitch/conf/autoload_configs/modules.conf.xml file

  <load module = "mod_unimrcp" /> 

2) In the / usr / local / freeswitch / conf / mrcp_profiles folder, create a vn.xml file with the profile of our MRCP server.

  <include>
   <! - VoiceNavigator ->
   <profile name = "vn" version = "1">
     <param name = "server-ip" value = "192.168.2.106" /> <! - IP address of the computer with VoiceNavigator ->
     <param name = "server-port" value = "8000" /> <! - Connection port to the MRCP server ->
     <param name = "resource-location" value = "" />
     <param name = "speechsynth" value = "tts" />
     <param name = "speechrecog" value = "asr" />
     <param name = "rtp-ip" value = "192.168.3.161" /> <! - IP address of the computer with FreeSWITCH ->
     <param name = "rtp-port-min" value = "32768" />
     <param name = "rtp-port-max" value = "33268" />
     <param name = "codecs" value = "PCMU PCMA L16 / 96/8000" />
    </ profile>
 </ include> 

3) Make the newly created profile used by default.
To do this, in the file /usr/local/freeswitch/conf/autoload_configs/unimrcp.conf.xml, we will correct the lines indicating the default profiles.

  <! - UniMRCP profile to use for TTS ->
 <param name = "default-tts-profile" value = "vn" />
 <! - UniMRCP profile to use for ASR ->
 <param name = "default-asr-profile" value = "vn" /> 

4) To allow FreeSWITCH interaction with other nodes, edit the file
/usr/local/freeswitch/conf/autoload_configs/acl.conf.xml, adding the IP address of the machine with VoiceNavigator to the allowed nodes.

  <list name = "domains" default = "deny">		
	 <node type = "allow" domain = "$$ {domain}" />
	 <! - Allow for interaction the node with the address 192.168.2.106 ->
	 <node type = "allow" cidr = "192.168.2.106/32" />	
 </ list> 

This completes the setup of connecting the server from VoiceNavigator to FreeSWITCH.

Creating a test voice application

1) Create a digits.gram grammar file in the / usr / local / freeswitch / grammar / folder

  <? xml version = "1.0"?>
 <grammar xmlns = "http://www.w3.org/2001/06/grammar" xml: lang = "en-ru" version = "1.0" mode = "voice" root = "test">
     <rule id = "test">
         <one-of>
             <item> one </ item>
             <item> two </ item>
             <item> three </ item>
             <item> four </ item>
             <item> five </ item>
             <item> six </ item>
             <item> seven </ item>
             <item> eight </ item>
             <item> nine </ item>
             <item> ten </ item>
         </ one-of>
     </ rule>
 </ grammar> 

2) Create a dialing plan in the test_vn.xml file and put it in / usr / local / freeswitch / conf / dialplan /

  <? xml version = "1.0" encoding = "utf-8"?>
 <include>
   <context name = "context_vn">
     <extension name = "welcome">
         <condition field = "destination_number" expression = "^ 777 $">
             <action application = "answer" />
             <action application = "set" data = "tts_engine = unimrcp: vn" />
             <action application = "set" data = "tts_voice = Maria8000" />
             <action application = "speak" data = "Give the number from one to ten!" />
             <action application = "detect_speech" data = "unimrcp / usr / local / freeswitch / grammar / digits vn" />
             <action application = "sleep" data = "10000" />
             <action application = "hangup" />
         </ condition>
     </ extension>
   </ context>
 </ include> 

Do you still think that FS dialplans are inconvenient and unreadable? In my opinion it is quite simple and understandable))

3) To be able to call this application, add to the /usr/local/freeswitch/conf/dialplan/default.xml file

  <extension name = "context_vn">
       <condition field = "destination_number" expression = "^ 777 $">
         <action application = "transfer" data = "777 XML context_vn" />
       </ condition>
 </ extension> 

FreeSWITCH is configured and ready to test the operation of the synthesis and recognition functions.
Restart FS to apply the changes.

Now you can call on number 777 to check the work of synthesis and recognition. The system asks for a number from one to ten and displays the recognition result in the console.

But then the most interesting. How to get this result and process? If in Asterisk it was enough to parse the $ {RECOG_RESULT} variable, then FreeSWITCH uses the more complex Event Socket mechanism for this. Or rather the event DETECTED_SPEECH with Speech-Type: detected-speech.
The topic of working with sockets FreeSWITCH is beyond the scope of this article and deserves a separate large description.
For those interested, you can see the details here and here .