Speech Recognition via Android Mode Questions

Hello everybody, I have been developing a software for android for home automation via Speech Recognition. I have been using Processing and following this topics:

Link 1: Utilizes the easy way to call back speech recognition with the Google Button and Field

I have used this code to get the basic know-how on the code operates, and how to call back the results

Link: 2: utilizes RecognizerIntent from its raw form to call back speech recognition without the google button and field

I am using link 2 most of my operations but i am having problems when i’m importing sound files.

import java.util.*;
import android.content.Intent;
import android.media.AudioManager;
import android.content.pm.PackageManager;
import android.content.pm.ResolveInfo;
import android.os.Bundle;
import android.speech.RecognizerIntent;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.view.View;
import android.app.Activity;
import android.os.Handler;
import android.os.Message;
import android.widget.TextView;
import android.view.View.OnClickListener;
import android.widget.Button;
import java.util.ArrayList;
import android.util.Log;
import android.content.Context;
import android.os.Vibrator;///let us take problems one after the other!!!!!
import android.os.VibrationEffect;
import android.service.voice.AlwaysOnHotwordDetector;
import android.widget.Toast;
import java.lang.Throwable;
import java.lang.Exception;
import java.lang.RuntimeException;
import android.content.ActivityNotFoundException;
import java.util.Locale;
import android.view.ViewGroup.LayoutParams;
import android.os.Build;
import android.Manifest;

import processing.sound.*;

//SOUND objects
SoundFile powerup;
SoundFile preset;
SoundFile entermode;
SoundFile exitmode;
SoundFile entertheme;
SoundFile exittheme;
SoundFile confirm;
SoundFile cancel;
SoundFile select;


Activity act;
Intent intent;
Context context;
AudioManager amanager;
public SpeechRecognizer sr;

String heard = new String();
boolean hotword=false;

 long previousMillis = 0;
long interval = 1000; //save values
boolean isConfiguring=true;


public void onStart(){

}

void setup() {
  act = this.getActivity();
  context = act.getApplicationContext();
  //select = new SoundFile(this, "ui_vats_move.wav");

 //amanager=(AudioManager)context.getSystemService(Context.AUDIO_SERVICE); 
  size(displayWidth, displayHeight);
  orientation(PORTRAIT);
  background(210,210,210);
}
 

void draw() {
  if(isConfiguring)
  {
      
//  powerup = new SoundFile(this, "ui_vats_ready.wav");
 // entermode = new SoundFile(this, "ui_pipboy_lighton.wav");
  //exitmode = new SoundFile(this, "ui_pipboy_lightoff.wav");
  //entertheme = new SoundFile(this, "ui_perkmenu_cards_equipup_04.wav");
  //exittheme = new SoundFile(this, "ui_perkmenu_cards_equipdown_01.wav");
 // confirm = new SoundFile(this, "ui_player_vending_success_seller_01.wav");
 // cancel = new SoundFile(this, "ui_targetingcomputermap_targetcancel_01.wav");

}
  
   long currentMillis = millis();  
       if (currentMillis - previousMillis > interval) 
    {
      previousMillis = currentMillis;   

    }
    
          
             if(heard.equals("confirm"))
                  {
                    confirm.play();
                    heard="";
                  }
                  
                       if(heard.equals("cancel"))
                  {
                    cancel.play();
                     heard="";
                  }
                  
                     if(heard.equals("select"))
                  {
                    select.play();
                     heard="";
                  }
                  
                         if(heard.equals("power up"))
                  {
                    powerup.play();
                     heard="";
                  }
                  
                               if(heard.equals("enter mode"))
                  {
                    entermode.play();
                     heard="";
                  }
                  
                  if(heard.equals("exit mode"))
                  {
                    exitmode.play();
                     heard="";
                  }
                  
                           if(heard.equals("enter theme"))
                  {
                    entertheme.play();
                     heard="";
                  }
                  
                    if(heard.equals("exit theme"))
                  {
                    exittheme.play();
                     heard="";
                  }
}

void mousePressed(){
  
   runOnUiThread(new Runnable() {
    @ Override
    public void run() {
      //Initialize the recognizer on the UI thread
      initRecognizer();
    }
  });
 
}
 

 
void initRecognizer() {

  //  amanager.setStreamVolume(AudioManager.STREAM_MUSIC, 0,  0);
 sr = SpeechRecognizer.createSpeechRecognizer(getActivity());
          sr.setRecognitionListener(new listener());
 
                Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);        
                intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
                intent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE,"voice.recognition.test");
               // intent.putExtra(RecognizerIntent.EXTRA_MAX_RESULTS,5);             
                     sr.startListening(intent);
                    
                      //    amanager.setStreamVolume(AudioManager.STREAM_MUSIC, amanager.getStreamMaxVolume(AudioManager.STREAM_MUSIC),  0);
}
  
 
/*************RECOGNITION LISTENER CLASS*************************/
 
 
  public class listener implements RecognitionListener          
   {
            public void onReadyForSpeech(Bundle params)
            {
                     //println( "onReadyForSpeech");
            }
            public void onBeginningOfSpeech()
            {
                   // println( "onBeginningOfSpeech");
            }
            public void onRmsChanged(float rmsdB)
            {
                   //  println( "onRmsChanged");
                    //detect dB change
            }
            public void onBufferReceived(byte[] buffer)
            {
                   //  println( "onBufferReceived");
            }
            public void onEndOfSpeech()
            {
                  //   println( "onEndofSpeech");
            }
            public void onError(int error)
            {
                     println( "error " +  error);
                   //  mText.setText("error " + error);
            }
            public void onResults(Bundle results)                   
            {       
                     String str = new String();
                   //  println( "onResults " + results);
                     ArrayList data = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
                   
                  // println("result " + data.get(0));
                  str+=data.get(0);
                  heard=str;
                  str="";
                  println(heard);
             
                  /*   for (int i = 0; i < data.size(); i++)
                     {
                               println("result " + data.get(i));
                               str += data.get(i);
                     }*/
                    // mText.setText("results: "+String.valueOf(data.size()));        
            }
            public void onPartialResults(Bundle partialResults)
            {
                    // println( "onPartialResults");
            }
            public void onEvent(int eventType, Bundle params)
            {
                    // println( "onEvent " + eventType);
            }
   }
   
   

The above code should play a sound file after recognizing the phrase, it requires a mousePressed callback to run the Speech Recognizer

I have commented out the sound file codes above, but when you try to import the sound files like so:

“powerup = new SoundFile(this, “ui_vats_ready.wav”);”

the Speech Recognizer sends out a Error 8, and it feels like the the speech recognizer goes into a loop error. But when i remove the sound files import code the program runs smoothly.

Also another addition to further improve the code, Does anyone know a tutorial for AlwaysOnHotwordDetector voice Service? I would want to call my Speech recognizer after a hotword trigger so my automation code will be total hands free for android…

1 Like

@Erald ===

  • I have posted these 2 codes (which are quite similar) and i know that they run.
  • As for the error i suppose that it s a conflict between P5 Sound and speech recognizer (which uses record audio classes); yet i ll give a look ASAP
  • As for AlwaysOnHotwordDetector API i dont think that “normal” apps can use that: only system apps have this permission.
  • Why posting twice?

*@akenaton thank you for your response, yes i have read your works as well. Thanks to you have an in depth understanding of speech recognition

*Yes your codes run as i have tried myself , the code from [DanielJay (https://forum.processing.org/two/profile/62/DanielJay) is the one that i had to rework the code to adapt to latest Processing.

*The sample code you gave @josephh worked like a charm and i have adapted it in fact in my other automation.

  • Regarding the Conflict between P5 Sound and Speech Recognizer, yes i am also trying to figure out why Speech Recognizer goes into Error code #8 when there is an import from P5 Sound. Please let me know if you have any leads.

  • By system apps, you mean apps that tied to the Android System? Our normal apps wont be able to utilize AlwaysOnHotwordDetector i guess… so i’ll have to wait till there is a good update for this feature in the future.

  • sorry for posting twice i thought that the AlwaysOnHotwordDetector was a different topic as i have indeed read that it was used for System apps…