View unanswered posts | View active topics It is currently Thu Apr 26, 2018 9:08 pm






Reply to topic  [ 5 posts ] 
Speech Recognition with Arduino and BitVoicer Server 
Author Message
Rookie

Joined: Sun Jan 06, 2013 11:11 am
Posts: 11
Post Speech Recognition with Arduino and BitVoicer Server
In this post I am going to show how to use an Arduino board and BitVoicer Server to control a few LEDs with voice commands. I will be using the Arduino Micro in this post, but you can use any Arduino board you have at hand.

Image



The following procedures will be executed to transform voice commands into LED activity:

  1. Audio waves will be captured and amplified by the Sparkfun Electret Breakout board;
  2. The amplified signal will be digitalized and buffered in the Arduino using its analog-to-digital converter (ADC);
  3. The audio samples will be streamed to BitVoicer Server using the Arduino serial port;
  4. BitVoicer Server will process the audio stream and recognize the speech it contains;
  5. The recognized speech will be mapped to predefined commands that will be sent back to the Arduino;
  6. The Arduino will identify the commands and perform the appropriate action.

The video above shows the final result of this post. Note in the video that BitVoicer Server also provides synthesized speech feedback. This speech feedback is defined in the server and reproduced by the server audio adapter, but the synthesized audio could also be sent to the Arduino and reproduced using a digital-to-analog converter (DAC). In my next post, I am going to show how to use the Arduino DUE, one amplified and one speaker to reproduce the synthesized speech using the Arduino itself.

List of Materials:



Last edited by msb4180 on Fri Dec 04, 2015 1:25 pm, edited 2 times in total.



Fri Dec 04, 2015 12:57 pm
Profile
Rookie

Joined: Sun Jan 06, 2013 11:11 am
Posts: 11
Post Speech Recognition with Arduino and BitVoicer Server - STEP1
STEP 1: Wiring

The first step is to wire the Arduino and the breadboard with the components as shown in the pictures below.

Image

Image

Image

Image

The most important detail here refers to the analog reference provided to the Arduino ADC. In my tests, I got better results using 3.3V with the Sparkfun Electret Breakout. That is why I added a jumper between the 3.3V pin and the AREF pin. If you decide to use the analogRead funcion (for any reason) while 3.3V is being applied to the AREF pin, you MUST call analogReference(EXTERNAL) before you use the analogRead function. Otherwise, you will short together the active reference voltage (internally generated) and the AREF pin, possibly damaging the microcontroller on your Arduino board.


Fri Dec 04, 2015 1:05 pm
Profile
Rookie

Joined: Sun Jan 06, 2013 11:11 am
Posts: 11
Post Speech Recognition with Arduino and BitVoicer Server - STEP2
STEP 2: Uploading the code to the Arduino

Now you have to upload the code below to your Arduino. You can also download the Arduino sketch from the link at the bottom. Before you upload the code, you must properly install the BitVoicer Server libraries into the Arduino IDE (Importing a .zip Library).

Code:
#include <BVSP.h>
#include <BVSMic.h>

// Defines the Arduino pin that will be used to capture audio
#define BVSM_AUDIO_INPUT 5

// Defines the LED pins
#define RED_LED_PIN 6
#define YELLOW_LED_PIN 9
#define GREEN_LED_PIN 10

// Defines the constants that will be passed as parameters to
// the BVSP.begin function
const unsigned long STATUS_REQUEST_TIMEOUT = 1000;
const unsigned long STATUS_REQUEST_INTERVAL = 2000;

// Defines the size of the audio buffer
const int AUDIO_BUFFER_SIZE = 64;

// Defines the size of the receive buffer
const int RECEIVE_BUFFER_SIZE = 2;

// Initializes a new global instance of the BVSP class
BVSP bvsp = BVSP();

// Initializes a new global instance of the BVSMic class
BVSMic bvsm = BVSMic();

// Creates a buffer that will be used to read recorded samples
// from the BVSMic class
byte audioBuffer[AUDIO_BUFFER_SIZE];

// Creates a buffer that will be used to read the commands sent
// from BitVoicer Server.
// Byte 0 = pin number
// Byte 1 = pin value
byte receiveBuffer[RECEIVE_BUFFER_SIZE];

void setup()
{
  // Sets up the pin modes
  pinMode(RED_LED_PIN, OUTPUT);
  pinMode(YELLOW_LED_PIN, OUTPUT);
  pinMode(GREEN_LED_PIN, OUTPUT);
 
  // Starts serial communication at 115200 bps
  Serial.begin(115200);
 
  // Sets the Arduino serial port that will be used for
  // communication, how long it will take before a status request
  // times out and how often status requests should be sent to
  // BitVoicer Server.
  bvsp.begin(Serial, STATUS_REQUEST_TIMEOUT, STATUS_REQUEST_INTERVAL);
   
  // Defines the function that will handle the frameReceived
  // event
  bvsp.frameReceived = BVSP_frameReceived;
 
  // Prepares the BVSMic class timer
  bvsm.begin();
}

void loop()
{
  // Checks if the status request interval has elapsed and if it
  // has, sends a status request to BitVoicer Server
  bvsp.keepAlive();
 
  // Checks if there is data available at the serial port buffer
  // and processes its content according to the specifications
  // of the BitVoicer Server Protocol
  bvsp.receive();

  // Checks if there is one SRE available. If there is one,
  // starts recording.
  if (bvsp.isSREAvailable())
  {
    // If the BVSMic class is not recording, sets up the audio
    // input and starts recording
    if (!bvsm.isRecording)
    {
      bvsm.setAudioInput(BVSM_AUDIO_INPUT, EXTERNAL);
      bvsm.startRecording();
    }

    // Checks if the BVSMic class has available samples
    if (bvsm.available)
    {
      // Makes sure the inbound mode is STREAM_MODE before
      // transmitting the stream
      if (bvsp.inboundMode == FRAMED_MODE)
        bvsp.setInboundMode(STREAM_MODE);
       
      // Reads the audio samples from the BVSMic class
      int bytesRead = bvsm.read(audioBuffer, AUDIO_BUFFER_SIZE);
     
      // Sends the audio stream to BitVoicer Server
      bvsp.sendStream(audioBuffer, bytesRead);
    }
  }
  else
  {
    // No SRE is available. If the BVSMic class is recording,
    // stops it.
    if (bvsm.isRecording)
      bvsm.stopRecording();
  }
}

// Handles the frameReceived event
void BVSP_frameReceived(byte dataType, int payloadSize)
{
  // Checks if the received frame contains binary data
  // 0x07 = Binary data (byte array)
  if (dataType == DATA_TYPE_BINARY)
  {
    // If 2 bytes were received, process the command.
    if (bvsp.getReceivedBytes(receiveBuffer, RECEIVE_BUFFER_SIZE) ==
      RECEIVE_BUFFER_SIZE)
    {
      analogWrite(receiveBuffer[0], receiveBuffer[1]);
    }
  }
}


This sketch has four major parts:

  • Library references and variable declaration: The first two lines include references to the BVSP and BVSMic libraries. These libraries are provided by BitSophia and can be found in the BitVoicer Server installation folder. The other lines declare constants and variables used throughout the sketch. The BVSP class is used to communicate with BitVoicer Server and the BVSMic class is used to capture and store audio samples.
  • Setup function: This function initializes serial communication, the BVSP class, the BVSMic class and sets the “event handler” (it is actually a function pointer) for the frameReceived event.
  • Loop function: This function performs three important actions: requests status info to the server (keepAlive() function), checks if the server has sent any data and processes the received data (receive() function), and controls the recording and sending of audio streams (isSREAvailable(), startRecording(), stopRecording() and sendStream() functions).
  • BVSP_frameReceived function: This function is called every time the receive() function identifies that one complete frame has been received. Here I run the command sent from BitVoicer Server. The command contains 2 bytes. The first byte indicates the pin and the second byte indicates the pin value. I use the analogWrite() function to set the appropriate value to the pin.

Arduino Sketch: BVS_Demo1.ino


Fri Dec 04, 2015 1:20 pm
Profile
Rookie

Joined: Sun Jan 06, 2013 11:11 am
Posts: 11
Post Speech Recognition with Arduino and BitVoicer Server - STEP3
STEP 3: Importing BitVoicer Server Solution Objects

Now you have to set up BitVoicer Server to work with the Arduino. BitVoicer Server has four major solution objects: Locations, Devices, BinaryData and Voice Schemas.

Locations represent the physical location where a device is installed. In my case, I created a location called Home.

Devices are the BitVoicer Server clients. I created a Mixed device, named it ArduinoMicro and entered the communication settings. NOTE ABOUT ARDUINO MICRO: it uses RTS and DTR so you have to enable these settings in the communication tab. I also created a SystemSpeaker device to synthesize speech using the server audio adapter.

BinaryData is a type of command BitVoicer Server can send to client devices. They are actually byte arrays you can link to commands. When BitVoicer Server recognizes speech related to that command, it sends the byte array to the target device. I created one BinaryData object to each pin value and named them ArduinoMicroGreenLedOn, ArduinoMicroGreenLedOff and so on. I ended up with 18 BinaryData objects in my solution, so I suggest you download and import the objects from the VoiceSchema.sof file below.

Voice Schemas are where everything comes together. They define what sentences should be recognized and what commands to run. For each sentence, you can define as many commands as you need and the order they will be executed. You can also define delays between commands. That is how I managed to perform the sequence of actions you see in the video.

You can import (Importing Solution Objects) all solution objects I used in this post from the files below. One contains the Devices and the other contains the Voice Schema and its Commands.

Solution Object Files:
Devices.sof
VoiceSchema.sof


Fri Dec 04, 2015 1:33 pm
Profile
Rookie

Joined: Sun Jan 06, 2013 11:11 am
Posts: 11
Post Speech Recognition with Arduino and BitVoicer Server - STEP4
STEP 4: Conclusion

That is it! I hope you liked it.

You have everything you need to run the demo shown in the video. Note that in the video I started by enabling the ArduinoMicro device in the BitVoicer Server Manager. As soon as it gets enabled, the Arduino identifies an available Speech Recognition Engine and starts streaming audio to BitVoicer Server.

You can follow the recognition results in the Server Monitor tool available in the BitVoicer Server Manager.

In my next post I will show how you can reproduce synthesized speech using an Arduino DUE.



Fri Dec 04, 2015 1:38 pm
Profile
Display posts from previous:  Sort by  
Reply to topic   [ 5 posts ] 

Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  



Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software for PTF.