Hello,
I use arduino uno and for some reason sometimes the program
hangs-up on a line getShieldResponse().indexOf(“PONG”) in a bool MOVI::isReady() method.
It simply does not return and nothing works. What should I do.
Thank you,
Benny.
Benny,
First of all make sure you are using the latest MOVI library. There was a bug in an earlier version just at that point. You want to use MOVI™ Library Version 1.13.
If you are using the newest version, the bug hasn’t been fixed completely. The issue though is that I need to be able to reproduce it. Therefore, if you can, it would be good if you could somehow send me a repeatable trace (serial console output, your detailed configuration, program that you are using).
Thanks,
Gerald
There is no output to console.
I attached the content of the ino file to next post.
I don’t see an attachment…
/****************************************************************************
* This is an example for the use of Audeme's MOVI(tm) Voice Control Shield *
* ----> http://www.audeme.com/MOVI/ *
* This code is inspired and maintained by Audeme but open to change *
* and organic development on GITHUB: *
* ----> https://github.com/audeme/MOVIArduinoAPI *
* Written by Gerald Friedland for Audeme LLC. *
* Contact: fractor@audeme.com *
* BSD license, all text above must be included in any redistribution. *
****************************************************************************
*
* Sometimes, especially with more complex dialogs, it can be desirable
* to have MOVI compare the recognized words only against a subset of the sentences
* rather than against all of them. MOVI does not support this feature natively.
* However, Arduino is fast enough so that these function can be implemented
* in a sketch.
*
* CAUTION:
* This example requires advanced programming skills and knowledge in
* computer science. It is meant as a guidance for larger projects with MOVI.
* Do not start learning about MOVI using this example.
*
* Using the example of a simple phone-number storage device, this sketch shows
* two different type of matchings:
* - Sentence matching (similar to what's internally implemented in MOVI)
* - Word-by-word correction
*
* The functions of this sketch are as follows:
* - "Store number": asks for a phone number, the individual numbers are then word-by-word corrected eg., "on to" will be corrected to "one two".
* - "Recall number": Reads the stored number or complains that no number was stored.
* - "Scratch number": Asks for confirmation via "yes" or "no". "Yes" and "no" are sentence matched against the second set only. If 'yes', the phone number is deleted.
*
* Just like the security question, the three basic commands are also manually matched. So an uttered "one two three" will result in "store number". In order to avoid this behavior,
* a threshold against the Levenshtein matching function can be used to filter sentences that are "too far away".
*
* Circuitry:
* Arduino UNO R3, MEGA2560 R3, or Arduino Leonardo R3.
* Connect speaker to MOVI.
* IMPORTANT: Always use external power supply with MOVI.
*
* Other Arduino-compatible boards:
* Consult MOVI's User Manual before connecting MOVI.
*
*/
#include "MOVIShield.h" // Include MOVI library, needs to be *before* the next #include
#if defined(ARDUINO_ARCH_AVR) || defined(ARDUINO_ARCH_PIC32)
#include <SoftwareSerial.h> // This is nice and flexible but only supported on AVR and PIC32 architecture, other boards need to use Serial1
#endif
#define SENTENCE_MATCH_THRESHOLD 2
MOVI recognizer(false); // Get a MOVI object, true enables serial monitor interface, rx and tx can be passed as parameters for alternate communication pins on AVR architecture
// Define sentence sets. Sentences must be defined without any punctuation and in UPPERCASE as MOVI's raw results are in this format.
const String GeneralMenuSentenceSet[]=
{
"OPTIONS"
};
#define GENERAL_MENU__TELL_MENU 2000
const String MainMenuSentenceSet[]=
{
"SET TIME",
"MANUAL SETUP",
"WHAT TIME IS IT"
};
const String TimeSetupSentenceSet[]=
{
"INCREASE",
"DECREASE",
"BACK",
"NEXT",
"FINISH"
};
const String HoursSentenceSet[]=
{
"ZERO",
"ONE",
"TWO",
"THREE",
"FOUR",
"FIVE",
"SIX",
"SEVEN",
"EIGHT",
"NINE",
"TEN",
"ELEVEN",
"TWELVE",
"THIRTEEN",
"FOURTEEN",
"FIFTEEN",
"SIXTEEN",
"SEVENTEEN",
"EIGHTEEN",
"NINETEEN",
"TWENTY"
};
const String MinutesSentenceSet[]=
{
HoursSentenceSet[ 0],
HoursSentenceSet[ 1],
HoursSentenceSet[ 2],
HoursSentenceSet[ 3],
HoursSentenceSet[ 4],
HoursSentenceSet[ 5],
HoursSentenceSet[ 6],
HoursSentenceSet[ 7],
HoursSentenceSet[ 8],
HoursSentenceSet[ 9],
HoursSentenceSet[10],
HoursSentenceSet[11],
HoursSentenceSet[12],
HoursSentenceSet[13],
HoursSentenceSet[14],
HoursSentenceSet[15],
HoursSentenceSet[16],
HoursSentenceSet[17],
HoursSentenceSet[18],
HoursSentenceSet[19],
HoursSentenceSet[20],
"THIRTY",
"FORTY",
"FIFTY"
};
const String CommonSentenceSet[]=
{
"UP",
"DOWN",
"NICE",
"O"
};
// OPTIONAL but useful: Bind the sets together for easier handling (they appear as 2D array).
const String *sentence_sets[] = { GeneralMenuSentenceSet, MainMenuSentenceSet, TimeSetupSentenceSet, HoursSentenceSet, MinutesSentenceSet, CommonSentenceSet };
const int set_sizes[] = { 1, 3, 5, 21, 24, 4 };
enum {
GENERAL_MENU_SENTENCE_SET = 0,
MAIN_MENU_SENTENCE_SET = 1,
TIME_MANUAL_SETUP_SENTENCE_SET = 2,
HOURS_SENTENCE_SET = 3,
MINUTES_SENTENCE_SET = 4,
COMMON_SENTENCE_SET = 5,
NUMBER_OF_SENTENCE_SETS = 6
};
enum {
READ_MENU,
READ_TIME_MANUAL_HOURS,
READ_TIME_MANUAL_MINUTES,
READ_TIME_HOURS,
READ_TIME_MINUTES,
READ_TIMER_MINUTES,
READ_TIMER_SECONDS,
READ_ALARM_HOURS,
READ_ALARM_MINUTES
} clock_states;
enum {
UPDATE_HOURS,
UPDATE_MINUTES,
UPDATE_SECONDS
};
int clock_state = READ_MENU;
String str=""; // Stores user string.
int time_h = 0;
int time_m = 0;
int manual_time_setup_state = UPDATE_HOURS;
void setup()
{
Serial.begin(9600);
Serial.println("Enter setup");
recognizer.init(); // Initialize MOVI (waits for it to boot)
Serial.println("MOVI init finished");
/*
// Usual note: training can only be performed in setup().
// The training functions are "lazy" and only do something if there are changes.
// They can be commented out to save memory and startup time once training has been performed.
recognizer.callSign("ARDUINO"); // Train callsign Arduino (may take 20 seconds)
for (int i = 0; i < NUMBER_OF_SENTENCE_SETS; i++) {
for (int j = 0; j < set_sizes[i]; j++) { // Add all sentences from set 0
recognizer.addSentence(sentence_sets[i][j]);
}
}
recognizer.train(); // Train (may take 20seconds)
Serial.println("MOVI finished learning sentences");
//*/
recognizer.responses(false); // Since we are not using MOVI's built-in sentence matcher, we don't want to hear from it either...
Serial.println("Disable MOVI's built-in sentence matcher");
recognizer.setThreshold(10); // uncomment and set to a higher value (valid range 2-95) if you have a problems due to a noisy environment.
Serial.println("Set MOVI's noisy environment threshhold");
tell_menu(-1);
Serial.println("Finished telling user basic options");
}
void loop() // run over and over
{
String result; // Stores MOVI's raw result
boolean processed = false; // Make sure we react to a result only once
signed int event = recognizer.poll(); // Get result from MOVI, 0 denotes nothing happened, negative values denote events (see docs)
// If words are available...
if (event == RAW_WORDS)
{
result = recognizer.getResult(); // ... read them.
if (result=="---") // Ignore silence
{
clock_state = READ_MENU;
processed = true;
}
if (result=="?") // Ignore unknown sentences
{
processed = true;
}
// basic commands
if (clock_state == READ_MENU && !processed)
{
Serial.println("Enter READ_MENU state");
int res = matchsentence(MAIN_MENU_SENTENCE_SET, result); // get the match for the basic commands
if (res == 0) { // sentence ("set time")
processed=true; // Mark result as processed
recognizer.ask(F("Please, say hours")); // ask for hours
clock_state = READ_TIME_HOURS;
}
if (res == 1) { // sentence ("set manual time")
processed = true; // Mark sentence as processed
recognizer.ask(F("Manual time setup mode. Update hours"));
clock_state = READ_TIME_MANUAL_HOURS;
manual_time_setup_state = UPDATE_HOURS;
}
if (res == 2) { // sentence ("tell time")
processed = true; // Mark sentence as processed
recognizer.say("The time is " + String(time_h, DEC) + " hours and " + String(time_m, DEC) + " minuts");
}
if (res == GENERAL_MENU__TELL_MENU) { // sentence ("tell menu")
processed = true; // Mark result as processed
tell_menu(MAIN_MENU_SENTENCE_SET);
}
}
if (clock_state == READ_TIME_HOURS && !processed) { // Mode after "read hours"
Serial.println("Enter READ_TIME_HOURS state");
processed = true; // Mark result as processed
str = correctwords(HOURS_SENTENCE_SET, result); // Match the individual numbers from set hours.
time_h = get_numbers(HOURS_SENTENCE_SET, str);
recognizer.ask(F("Please, say minuts")); // ask for minuts
clock_state = READ_TIME_MINUTES;
}
if (clock_state == READ_TIME_MINUTES && !processed) { // Mode after "read minutes"
Serial.println("Enter READ_TIME_MINUTES state");
processed = true; // Mark result as processed
str = correctwords(MINUTES_SENTENCE_SET, result); // Match the individual numbers from set hours.
time_m = get_numbers(MINUTES_SENTENCE_SET, str);
recognizer.say(F("Time setup complete")); // inform user
clock_state = READ_MENU;
}
if (clock_state == READ_TIME_MANUAL_HOURS && !processed) {
Serial.println("Enter READ_TIME_MANUAL_HOURS state");
int res = matchsentence(TIME_MANUAL_SETUP_SENTENCE_SET, result);
if (res == 0) { // Mode after "up"
processed = true;
if (manual_time_setup_state == UPDATE_HOURS) {
time_h++;
if (time_h > 23) {
time_h = 0;
}
} else if (manual_time_setup_state == UPDATE_MINUTES) {
time_m++;
if (time_m > 59) {
time_m = 0;
}
}
recognizer.ask("The time is " + String(time_h, DEC) + " hours and " + String(time_m, DEC) + " minutes");
}
if (res == 1) { // Mode after "down"
processed = true;
if (manual_time_setup_state == UPDATE_HOURS) {
time_h--;
if (time_h < 0) {
time_h = 23;
}
} else if (manual_time_setup_state == UPDATE_MINUTES) {
time_m--;
if (time_m < 0) {
time_m = 59;
}
}
recognizer.ask("The time is " + String(time_h, DEC) + " hours and " + String(time_m, DEC) + " minutes");
}
if (res == 2 || res == 3) { // sentences "back" & "next"
processed = true;
if (manual_time_setup_state == UPDATE_HOURS) {
manual_time_setup_state = UPDATE_MINUTES;
recognizer.ask("Update minutes");
} else {
manual_time_setup_state = UPDATE_HOURS;
recognizer.ask("Update hours");
}
}
if (res == 4) { // sentence "finish"
processed = true;
recognizer.say(F("Time setup complete"));
clock_state = READ_MENU;
}
if (res == GENERAL_MENU__TELL_MENU) { // sentence ("tell menu")
processed = true; // Mark result as processed
tell_menu(TIME_MANUAL_SETUP_SENTENCE_SET);
}
}
}
}
void tell_menu(int setno) {
bool is_tell_specific_menu = setno >= 0 && setno < NUMBER_OF_SENTENCE_SETS;
if (is_tell_specific_menu) {
const String *sentences = sentence_sets[setno]; // chose the sentences from the set
recognizer.say(F("The menu is:"));
for (int i = 0; i < set_sizes[setno]; i++) {
recognizer.say(sentences[i]);
}
recognizer.say(F("You can allways ask for:"));
} else {
recognizer.say(F("If you don't know what to do, ask for:"));
}
for (int i = 0; i < set_sizes[GENERAL_MENU_SENTENCE_SET]; i++) {
recognizer.say(sentence_sets[GENERAL_MENU_SENTENCE_SET][i]);
}
if (is_tell_specific_menu) {
recognizer.ask(""); // do it inorder to re-read the option
}
}
int get_number(int setno, String number_str)
{
int res = matchsentence(setno, number_str); // and then matches it
if (setno == HOURS_SENTENCE_SET) {
return res;
} else if (setno == MINUTES_SENTENCE_SET) {
if (res >= 0 && res <= 20) {
return res;
} else if (res == 21) {
return 30;
} else if (res == 22) {
return 40;
} else if (res == 23) {
return 50;
}
}
return 0;
}
int get_numbers(int setno, String numbers_str)
{
int res = 0;
int index = 0;
String oneword;
String matchedstring = "";
do { // this loop
oneword = getword(numbers_str, index); // extracts each word
if (oneword != "") {
int num = get_number(setno, oneword);
res += num;
}
index++;
} while (oneword != ""); // until no words are left in the input
Serial.print("number read: ");
Serial.println(res);
return res;
}
/*
* The following function uses the Levenshtein distance to match a given sentence against a result set.
* The minimum distance wins. If there's more than one minimum, the first minimum wins.
* This is where an ambiguity detector and/or a threshold could be implemented to force increased robustness.
* The Levenshtein distance is a common metric in the speech recognition community for this kind of task.
*/
int matchsentence(int setno, String result)
{
int matchscore=30000; // unachievably high number
int matchindex=0;
const String *sentences = sentence_sets[setno]; // chose the sentences from the set
for (int i = 0; i < set_sizes[setno]; i++) { // Find the minimally distant sentence.
int score = levenshtein(sentences[i], result);
if (score < matchscore) {
matchscore = score;
matchindex = i;
}
}
for (int i = 0; i < set_sizes[GENERAL_MENU_SENTENCE_SET]; i++) {
int score = levenshtein(sentence_sets[GENERAL_MENU_SENTENCE_SET][i], result);
if (score < matchscore) {
matchscore = score;
if (i == 0) {
matchindex = GENERAL_MENU__TELL_MENU;
}
}
}
Serial.print("Input word: '");
Serial.print(result);
Serial.print("' Recognized sentence: '");
if (matchindex == GENERAL_MENU__TELL_MENU) {
Serial.print(sentence_sets[GENERAL_MENU_SENTENCE_SET][0]);
} else {
Serial.print(sentences[matchindex]);
}
Serial.print("' Distance: ");
Serial.println(matchscore);
if (matchscore > SENTENCE_MATCH_THRESHOLD) {
Serial.println("Fail match");
matchindex = -1;
recognizer.ask(F("Please, say again!"));
} else {
Serial.println("Match OK");
}
return matchindex;
}
/*
* This function uses matchsentence() to replace each word in the given raw result string with the closest match in a sentence set, thereby returning a corrected result.
*/
String correctwords(int setno, String rawresult)
{
int index=0;
String oneword;
String matchedstring="";
const String *sentences = sentence_sets[setno]; // chose the sentences from the set
do { // this loop
oneword=getword(rawresult,index); // extracts each word
if (oneword!="") {
int res=matchsentence(setno,oneword); /// and then matches it
matchedstring=matchedstring+sentences[res]+" "; // the matching result is then added to the string, thereby correcting whatever input was with the closest match
}
index++;
} while (oneword!=""); // until no words are left in the input
return matchedstring; // return the result
}
/*********** HELPER FUNCTIONS ********************/
/*
* This function returns a single string that is separated by a space at a given index.
*/
String getword(String str, int index)
{
int found = 0;
int strIndex[] = {0, -1};
int maxIndex = str.length()-1;
char separator = ' ';
for(int i=0; i<=maxIndex && found<=index; i++){
if(str.charAt(i)==separator || i==maxIndex){
found++;
strIndex[0] = strIndex[1]+1;
strIndex[1] = (i == maxIndex) ? i+1 : i;
}
}
return found>index ? str.substring(strIndex[0], strIndex[1]) : "";
}
/*
* This function calculates the Levenshtein, or edit-distance, counting the number of insertions, deletions, and substitutions needed to go from string s1 to string s2.
* More information about the function can be found here: https://en.wikipedia.org/wiki/Levenshtein_distance
* This particular implementation is optimized to use O(min(m,n)) space instead of O(mn) as memory on the Arduino is scarce.
* More information on this function can be found from the original source it was adopted from: https://en.wikibooks.org/wiki/Algorithm_Implementation/Strings/Levenshtein_distance
*/
#define MIN3(a, b, c) ((a) < (b) ? ((a) < (c) ? (a) : (c)) : ((b) < (c) ? (b) : (c)))
int levenshtein(String s1, String s2) {
unsigned int s1len, s2len, x, y, lastdiag, olddiag;
s1len=s1.length();
s2len=s2.length();
unsigned int column[s1len+1];
for (y = 1; y <= s1len; y++)
column[y] = y;
for (x = 1; x <= s2len; x++) {
column[0] = x;
for (y = 1, lastdiag = x-1; y <= s1len; y++) {
olddiag = column[y];
column[y] = MIN3(column[y] + 1, column[y-1] + 1, lastdiag + (s1[y-1] == s2[x-1] ? 0 : 1));
lastdiag = olddiag;
}
}
return(column[s1len]);
}I use Arduino UNO with Atmel 328p from EBAY.
Most likely you run out of memory. The Uno has very limited string memory. And a couple Serial.println() can do it.
Try using the F() function. For example: Serial.println(F("Finished telling user basic options"));
This is described here:
https://playground.arduino.cc/Learning/Memory
Let me know if that helps,
Gerald
I did what you said. New code will be attached to the next message.
When the arduino loads it should say: “If you don’t know what to do, ask for: options”.
but it stuck on "If you don’t know what to do, ask for:"
Please, help.
Thank you,
Benny.
#include "MOVIShield.h" // Include MOVI library, needs to be *before* the next #include
#if defined(ARDUINO_ARCH_AVR) || defined(ARDUINO_ARCH_PIC32)
#include <SoftwareSerial.h> // This is nice and flexible but only supported on AVR and PIC32 architecture, other boards need to use Serial1
#endif
#define SENTENCE_MATCH_THRESHOLD 2
MOVI recognizer(false); // Get a MOVI object, true enables serial monitor interface, rx and tx can be passed as parameters for alternate communication pins on AVR architecture
// Define sentence sets. Sentences must be defined without any punctuation and in UPPERCASE as MOVI's raw results are in this format.
const String GeneralMenuSentenceSet[] PROGMEM =
{
"OPTIONS"
};
#define GENERAL_MENU__TELL_MENU 2000
const String MainMenuSentenceSet[] PROGMEM =
{
"SET TIME",
"MANUAL SETUP",
"WHAT TIME IS IT"
};
const String TimeSetupSentenceSet[] PROGMEM =
{
"INCREASE",
"DECREASE",
"BACK",
"NEXT",
"FINISH"
};
const String HoursSentenceSet[] PROGMEM =
{
"ZERO",
"ONE",
"TWO",
"THREE",
"FOUR",
"FIVE",
"SIX",
"SEVEN",
"EIGHT",
"NINE",
"TEN",
"ELEVEN",
"TWELVE",
"THIRTEEN",
"FOURTEEN",
"FIFTEEN",
"SIXTEEN",
"SEVENTEEN",
"EIGHTEEN",
"NINETEEN",
"TWENTY"
};
const String MinutesSentenceSet[] PROGMEM =
{
HoursSentenceSet[ 0],
HoursSentenceSet[ 1],
HoursSentenceSet[ 2],
HoursSentenceSet[ 3],
HoursSentenceSet[ 4],
HoursSentenceSet[ 5],
HoursSentenceSet[ 6],
HoursSentenceSet[ 7],
HoursSentenceSet[ 8],
HoursSentenceSet[ 9],
HoursSentenceSet[10],
HoursSentenceSet[11],
HoursSentenceSet[12],
HoursSentenceSet[13],
HoursSentenceSet[14],
HoursSentenceSet[15],
HoursSentenceSet[16],
HoursSentenceSet[17],
HoursSentenceSet[18],
HoursSentenceSet[19],
HoursSentenceSet[20],
"THIRTY",
"FORTY",
"FIFTY"
};
const String CommonSentenceSet[] PROGMEM =
{
"UP",
"DOWN",
"NICE",
"O"
};
// OPTIONAL but useful: Bind the sets together for easier handling (they appear as 2D array).
const String *sentence_sets[] = { GeneralMenuSentenceSet, MainMenuSentenceSet, TimeSetupSentenceSet, HoursSentenceSet, MinutesSentenceSet, CommonSentenceSet };
const int set_sizes[] = { 1, 3, 5, 21, 24, 4 };
enum {
GENERAL_MENU_SENTENCE_SET = 0,
MAIN_MENU_SENTENCE_SET = 1,
TIME_MANUAL_SETUP_SENTENCE_SET = 2,
HOURS_SENTENCE_SET = 3,
MINUTES_SENTENCE_SET = 4,
COMMON_SENTENCE_SET = 5,
NUMBER_OF_SENTENCE_SETS = 6
};
enum {
READ_MENU,
READ_TIME_MANUAL_HOURS,
READ_TIME_MANUAL_MINUTES,
READ_TIME_HOURS,
READ_TIME_MINUTES,
READ_TIMER_MINUTES,
READ_TIMER_SECONDS,
READ_ALARM_HOURS,
READ_ALARM_MINUTES
} clock_states;
enum {
UPDATE_HOURS,
UPDATE_MINUTES,
UPDATE_SECONDS
};
int clock_state = READ_MENU;
String str=""; // Stores user string.
int time_h = 0;
int time_m = 0;
int manual_time_setup_state = UPDATE_HOURS;
void setup()
{
Serial.begin(9600);
Serial.println(F("Enter setup"));
recognizer.init(); // Initialize MOVI (waits for it to boot)
Serial.println(F("MOVI init finished"));
/*
// Usual note: training can only be performed in setup().
// The training functions are "lazy" and only do something if there are changes.
// They can be commented out to save memory and startup time once training has been performed.
recognizer.callSign("ARDUINO"); // Train callsign Arduino (may take 20 seconds)
for (int i = 0; i < NUMBER_OF_SENTENCE_SETS; i++) {
for (int j = 0; j < set_sizes[i]; j++) { // Add all sentences from set 0
recognizer.addSentence(sentence_sets[i][j]);
}
}
recognizer.train(); // Train (may take 20seconds)
Serial.println("MOVI finished learning sentences");
//*/
recognizer.responses(false); // Since we are not using MOVI's built-in sentence matcher, we don't want to hear from it either...
Serial.println(F("Disable MOVI's built-in sentence matcher"));
recognizer.setThreshold(10); // uncomment and set to a higher value (valid range 2-95) if you have a problems due to a noisy environment.
Serial.println(F("Set MOVI's noisy environment threshhold"));
tell_menu(-1);
Serial.println(F("Finished telling user basic options"));
}
void loop() // run over and over
{
String result; // Stores MOVI's raw result
boolean processed = false; // Make sure we react to a result only once
signed int event = recognizer.poll(); // Get result from MOVI, 0 denotes nothing happened, negative values denote events (see docs)
// If words are available...
if (event == RAW_WORDS)
{
result = recognizer.getResult(); // ... read them.
if (result=="---") // Ignore silence
{
clock_state = READ_MENU;
processed = true;
}
if (result=="?") // Ignore unknown sentences
{
processed = true;
}
// basic commands
if (clock_state == READ_MENU && !processed)
{
Serial.println(F("Enter READ_MENU state"));
int res = matchsentence(MAIN_MENU_SENTENCE_SET, result); // get the match for the basic commands
if (res == 0) { // sentence ("set time")
processed=true; // Mark result as processed
recognizer.ask(F("Please, say hours")); // ask for hours
clock_state = READ_TIME_HOURS;
}
if (res == 1) { // sentence ("set manual time")
processed = true; // Mark sentence as processed
recognizer.ask(F("Manual time setup mode. Update hours"));
clock_state = READ_TIME_MANUAL_HOURS;
manual_time_setup_state = UPDATE_HOURS;
}
if (res == 2) { // sentence ("tell time")
processed = true; // Mark sentence as processed
recognizer.say("The time is " + String(time_h, DEC) + " hours and " + String(time_m, DEC) + " minuts");
}
if (res == GENERAL_MENU__TELL_MENU) { // sentence ("tell menu")
processed = true; // Mark result as processed
tell_menu(MAIN_MENU_SENTENCE_SET);
}
}
if (clock_state == READ_TIME_HOURS && !processed) { // Mode after "read hours"
Serial.println(F("Enter READ_TIME_HOURS state"));
processed = true; // Mark result as processed
str = correctwords(HOURS_SENTENCE_SET, result); // Match the individual numbers from set hours.
time_h = get_numbers(HOURS_SENTENCE_SET, str);
recognizer.ask(F("Please, say minuts")); // ask for minuts
clock_state = READ_TIME_MINUTES;
}
if (clock_state == READ_TIME_MINUTES && !processed) { // Mode after "read minutes"
Serial.println(F("Enter READ_TIME_MINUTES state"));
processed = true; // Mark result as processed
str = correctwords(MINUTES_SENTENCE_SET, result); // Match the individual numbers from set hours.
time_m = get_numbers(MINUTES_SENTENCE_SET, str);
recognizer.say(F("Time setup complete")); // inform user
clock_state = READ_MENU;
}
if (clock_state == READ_TIME_MANUAL_HOURS && !processed) {
Serial.println(F("Enter READ_TIME_MANUAL_HOURS state"));
int res = matchsentence(TIME_MANUAL_SETUP_SENTENCE_SET, result);
if (res == 0) { // Mode after "up"
processed = true;
if (manual_time_setup_state == UPDATE_HOURS) {
time_h++;
if (time_h > 23) {
time_h = 0;
}
} else if (manual_time_setup_state == UPDATE_MINUTES) {
time_m++;
if (time_m > 59) {
time_m = 0;
}
}
recognizer.ask("The time is " + String(time_h, DEC) + " hours and " + String(time_m, DEC) + " minutes");
}
if (res == 1) { // Mode after "down"
processed = true;
if (manual_time_setup_state == UPDATE_HOURS) {
time_h--;
if (time_h < 0) {
time_h = 23;
}
} else if (manual_time_setup_state == UPDATE_MINUTES) {
time_m--;
if (time_m < 0) {
time_m = 59;
}
}
recognizer.ask("The time is " + String(time_h, DEC) + " hours and " + String(time_m, DEC) + " minutes");
}
if (res == 2 || res == 3) { // sentences "back" & "next"
processed = true;
if (manual_time_setup_state == UPDATE_HOURS) {
manual_time_setup_state = UPDATE_MINUTES;
recognizer.ask(F("Update minutes"));
} else {
manual_time_setup_state = UPDATE_HOURS;
recognizer.ask(F("Update hours"));
}
}
if (res == 4) { // sentence "finish"
processed = true;
recognizer.say(F("Time setup complete"));
clock_state = READ_MENU;
}
if (res == GENERAL_MENU__TELL_MENU) { // sentence ("tell menu")
processed = true; // Mark result as processed
tell_menu(TIME_MANUAL_SETUP_SENTENCE_SET);
}
}
}
}
void tell_menu(int setno) {
bool is_tell_specific_menu = setno >= 0 && setno < NUMBER_OF_SENTENCE_SETS;
if (is_tell_specific_menu) {
const String *sentences = sentence_sets[setno]; // chose the sentences from the set
recognizer.say(F("The menu is:"));
for (int i = 0; i < set_sizes[setno]; i++) {
recognizer.say(sentences[i]);
}
recognizer.say(F("You can allways ask for:"));
} else {
recognizer.say(F("If you don't know what to do, ask for:"));
}
for (int i = 0; i < set_sizes[GENERAL_MENU_SENTENCE_SET]; i++) {
recognizer.say(sentence_sets[GENERAL_MENU_SENTENCE_SET][i]);
}
if (is_tell_specific_menu) {
recognizer.ask(""); // do it inorder to re-read the option
}
}
int get_number(int setno, String number_str)
{
int res = matchsentence(setno, number_str); // and then matches it
if (setno == HOURS_SENTENCE_SET) {
return res;
} else if (setno == MINUTES_SENTENCE_SET) {
if (res >= 0 && res <= 20) {
return res;
} else if (res == 21) {
return 30;
} else if (res == 22) {
return 40;
} else if (res == 23) {
return 50;
}
}
return 0;
}
int get_numbers(int setno, String numbers_str)
{
int res = 0;
int index = 0;
String oneword;
String matchedstring = "";
do { // this loop
oneword = getword(numbers_str, index); // extracts each word
if (oneword != "") {
int num = get_number(setno, oneword);
res += num;
}
index++;
} while (oneword != ""); // until no words are left in the input
Serial.print(F("number read: "));
Serial.println(res);
return res;
}
/*
* The following function uses the Levenshtein distance to match a given sentence against a result set.
* The minimum distance wins. If there's more than one minimum, the first minimum wins.
* This is where an ambiguity detector and/or a threshold could be implemented to force increased robustness.
* The Levenshtein distance is a common metric in the speech recognition community for this kind of task.
*/
int matchsentence(int setno, String result)
{
int matchscore=30000; // unachievably high number
int matchindex=0;
const String *sentences = sentence_sets[setno]; // chose the sentences from the set
for (int i = 0; i < set_sizes[setno]; i++) { // Find the minimally distant sentence.
int score = levenshtein(sentences[i], result);
if (score < matchscore) {
matchscore = score;
matchindex = i;
}
}
for (int i = 0; i < set_sizes[GENERAL_MENU_SENTENCE_SET]; i++) {
int score = levenshtein(sentence_sets[GENERAL_MENU_SENTENCE_SET][i], result);
if (score < matchscore) {
matchscore = score;
if (i == 0) {
matchindex = GENERAL_MENU__TELL_MENU;
}
}
}
Serial.print(F("Input word: '"));
Serial.print(result);
Serial.print(F("' Recognized sentence: '"));
if (matchindex == GENERAL_MENU__TELL_MENU) {
Serial.print(sentence_sets[GENERAL_MENU_SENTENCE_SET][0]);
} else {
Serial.print(sentences[matchindex]);
}
Serial.print(F("' Distance: "));
Serial.println(matchscore);
if (matchscore > SENTENCE_MATCH_THRESHOLD) {
Serial.println(F("Fail match"));
matchindex = -1;
recognizer.ask(F("Please, say again!"));
} else {
Serial.println(F("Match OK"));
}
return matchindex;
}
/*
* This function uses matchsentence() to replace each word in the given raw result string with the closest match in a sentence set, thereby returning a corrected result.
*/
String correctwords(int setno, String rawresult)
{
int index=0;
String oneword;
String matchedstring="";
const String *sentences = sentence_sets[setno]; // chose the sentences from the set
do { // this loop
oneword=getword(rawresult,index); // extracts each word
if (oneword!="") {
int res=matchsentence(setno,oneword); /// and then matches it
matchedstring=matchedstring+sentences[res]+" "; // the matching result is then added to the string, thereby correcting whatever input was with the closest match
}
index++;
} while (oneword!=""); // until no words are left in the input
return matchedstring; // return the result
}
/*********** HELPER FUNCTIONS ********************/
/*
* This function returns a single string that is separated by a space at a given index.
*/
String getword(String str, int index)
{
int found = 0;
int strIndex[] = {0, -1};
int maxIndex = str.length()-1;
char separator = ' ';
for(int i=0; i<=maxIndex && found<=index; i++){
if(str.charAt(i)==separator || i==maxIndex){
found++;
strIndex[0] = strIndex[1]+1;
strIndex[1] = (i == maxIndex) ? i+1 : i;
}
}
return found>index ? str.substring(strIndex[0], strIndex[1]) : "";
}
/*
* This function calculates the Levenshtein, or edit-distance, counting the number of insertions, deletions, and substitutions needed to go from string s1 to string s2.
* More information about the function can be found here: https://en.wikipedia.org/wiki/Levenshtein_distance
* This particular implementation is optimized to use O(min(m,n)) space instead of O(mn) as memory on the Arduino is scarce.
* More information on this function can be found from the original source it was adopted from: https://en.wikibooks.org/wiki/Algorithm_Implementation/Strings/Levenshtein_distance
*/
#define MIN3(a, b, c) ((a) < (b) ? ((a) < (c) ? (a) : (c)) : ((b) < (c) ? (b) : (c)))
int levenshtein(String s1, String s2) {
unsigned int s1len, s2len, x, y, lastdiag, olddiag;
s1len=s1.length();
s2len=s2.length();
unsigned int column[s1len+1];
for (y = 1; y <= s1len; y++)
column[y] = y;
for (x = 1; x <= s2len; x++) {
column[0] = x;
for (y = 1, lastdiag = x-1; y <= s1len; y++) {
olddiag = column[y];
column[y] = MIN3(column[y] + 1, column[y-1] + 1, lastdiag + (s1[y-1] == s2[x-1] ? 0 : 1));
lastdiag = olddiag;
}
}
return(column[s1len]);
}Have you double checked that a Serial.println(sentence_sets[GENERAL_MENU_SENTENCE_SET])[i]) shows “OPTIONS” in the console? It’s an awful lot of sentences and indirections and I would hate for there to still be a memory problem. Another question is: What happens if you replace recognizer.say(F("If you don't know what to do, ask for:")); temporarily with recognizer.say(sentence_sets[GENERAL_MENU_SENTENCE_SET][i]);?
My bet is that your issue has nothing to do with MOVI but rather your use of PROGMEM and the indirection you have with pointers.
const String MainMenuSentenceSet[] PROGMEM =
and then you do
const String *sentence_sets[] = { GeneralMenuSentenceSet, MainMenuSentenceSet, TimeSetupSentenceSet, HoursSentenceSet, MinutesSentenceSet, CommonSentenceSet };
and then
for (int i = 0; i < set_sizes[GENERAL_MENU_SENTENCE_SET]; i++) {
recognizer.say(sentence_sets[GENERAL_MENU_SENTENCE_SET][i]);
}
It’s been some time since I have really focused on an Arduino program, but I have some foggy memory of being able to confuse it and get it good and locked up if it gets lost on a pointer when using PROGMEM and not being super careful about it.
Try removing all keywords PROGMEM and see if your program gets any further. If it does then you can try building back up the use of PROGMEM.
Good luck with it!
Dylan.