# Speech Recognition via Android Mode Questions

**URL:** <https://discourse.processing.org/t/speech-recognition-via-android-mode-questions/22291>\
**Category:** Processing for Android\
**Created:** [July 1, 2020, 6:04am UTC](https://discourse.processing.org/t/speech-recognition-via-android-mode-questions/22291 "2020-07-01T06:04:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Erald](https://avatars.discourse-cdn.com/v4/letter/e/919ad9/32.png) [@Erald](https://discourse.processing.org/u/Erald)\
**Post date:** [July 1, 2020, 6:04am UTC](https://discourse.processing.org/t/speech-recognition-via-android-mode-questions/22291/1 "2020-07-01T06:04:48Z")

</div>

Hello everybody, I have been developing a software for android for home automation via Speech Recognition. I have been using Processing and following this topics:

Link 1: Utilizes the easy way to call back speech recognition with the Google Button and Field

> [@Speech recognition for android mode](https://discourse.processing.org/t/speech-recognition-for-android-mode/463):
>
> Hello, Does someone know a speech recognition (speech to text) library for Android Mode in processing? Thanks

I have used this code to get the basic know-how on the code operates, and how to call back the results

Link: 2: utilizes RecognizerIntent from its raw form to call back speech recognition without the google button and field

> **[Processing 2.x and 3.x Forum](https://forum.processing.org/two/discussion/2773/android-speechrecognition-class-within-processing)**
>
> Processing is an electronic sketchbook, a language and a worldwide community. This is its forum.

I am using link 2 most of my operations but i am having problems when i’m importing sound files.

```auto
import java.util.*;
import android.content.Intent;
import android.media.AudioManager;
import android.content.pm.PackageManager;
import android.content.pm.ResolveInfo;
import android.os.Bundle;
import android.speech.RecognizerIntent;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.view.View;
import android.app.Activity;
import android.os.Handler;
import android.os.Message;
import android.widget.TextView;
import android.view.View.OnClickListener;
import android.widget.Button;
import java.util.ArrayList;
import android.util.Log;
import android.content.Context;
import android.os.Vibrator;///let us take problems one after the other!!!!!
import android.os.VibrationEffect;
import android.service.voice.AlwaysOnHotwordDetector;
import android.widget.Toast;
import java.lang.Throwable;
import java.lang.Exception;
import java.lang.RuntimeException;
import android.content.ActivityNotFoundException;
import java.util.Locale;
import android.view.ViewGroup.LayoutParams;
import android.os.Build;
import android.Manifest;

import processing.sound.*;

//SOUND objects
SoundFile powerup;
SoundFile preset;
SoundFile entermode;
SoundFile exitmode;
SoundFile entertheme;
SoundFile exittheme;
SoundFile confirm;
SoundFile cancel;
SoundFile select;

Activity act;
Intent intent;
Context context;
AudioManager amanager;
public SpeechRecognizer sr;

String heard = new String();
boolean hotword=false;

 long previousMillis = 0;
long interval = 1000; //save values
boolean isConfiguring=true;

public void onStart(){

}

void setup() {
  act = this.getActivity();
  context = act.getApplicationContext();
  //select = new SoundFile(this, "ui_vats_move.wav");

 //amanager=(AudioManager)context.getSystemService(Context.AUDIO_SERVICE); 
  size(displayWidth, displayHeight);
  orientation(PORTRAIT);
  background(210,210,210);
}
 

void draw() {
  if(isConfiguring)
  {
      
// powerup = new SoundFile(this, "ui_vats_ready.wav");
 // entermode = new SoundFile(this, "ui_pipboy_lighton.wav");
  //exitmode = new SoundFile(this, "ui_pipboy_lightoff.wav");
  //entertheme = new SoundFile(this, "ui_perkmenu_cards_equipup_04.wav");
  //exittheme = new SoundFile(this, "ui_perkmenu_cards_equipdown_01.wav");
 // confirm = new SoundFile(this, "ui_player_vending_success_seller_01.wav");
 // cancel = new SoundFile(this, "ui_targetingcomputermap_targetcancel_01.wav");

}
  
   long currentMillis = millis();  
       if (currentMillis - previousMillis > interval) 
    {
      previousMillis = currentMillis;   

    }
    
          
             if(heard.equals("confirm"))
                  {
                    confirm.play();
                    heard="";
                  }
                  
                       if(heard.equals("cancel"))
                  {
                    cancel.play();
                     heard="";
                  }
                  
                     if(heard.equals("select"))
                  {
                    select.play();
                     heard="";
                  }
                  
                         if(heard.equals("power up"))
                  {
                    powerup.play();
                     heard="";
                  }
                  
                               if(heard.equals("enter mode"))
                  {
                    entermode.play();
                     heard="";
                  }
                  
                  if(heard.equals("exit mode"))
                  {
                    exitmode.play();
                     heard="";
                  }
                  
                           if(heard.equals("enter theme"))
                  {
                    entertheme.play();
                     heard="";
                  }
                  
                    if(heard.equals("exit theme"))
                  {
                    exittheme.play();
                     heard="";
                  }
}

void mousePressed(){
  
   runOnUiThread(new Runnable() {
    @ Override
    public void run() {
      //Initialize the recognizer on the UI thread
      initRecognizer();
    }
  });
 
}
 

 
void initRecognizer() {

  // amanager.setStreamVolume(AudioManager.STREAM_MUSIC, 0, 0);
 sr = SpeechRecognizer.createSpeechRecognizer(getActivity());
          sr.setRecognitionListener(new listener());
 
                Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);        
                intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
                intent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE,"voice.recognition.test");
               // intent.putExtra(RecognizerIntent.EXTRA_MAX_RESULTS,5);             
                     sr.startListening(intent);
                    
                      // amanager.setStreamVolume(AudioManager.STREAM_MUSIC, amanager.getStreamMaxVolume(AudioManager.STREAM_MUSIC), 0);
}
  
 
/ *************RECOGNITION LISTENER CLASS************************* /
 
 
  public class listener implements RecognitionListener          
   {
            public void onReadyForSpeech(Bundle params)
            {
                     //println( "onReadyForSpeech");
            }
            public void onBeginningOfSpeech()
            {
                   // println( "onBeginningOfSpeech");
            }
            public void onRmsChanged(float rmsdB)
            {
                   // println( "onRmsChanged");
                    //detect dB change
            }
            public void onBufferReceived(byte[] buffer)
            {
                   // println( "onBufferReceived");
            }
            public void onEndOfSpeech()
            {
                  // println( "onEndofSpeech");
            }
            public void onError(int error)
            {
                     println( "error " + error);
                   // mText.setText("error " + error);
            }
            public void onResults(Bundle results)                   
            {       
                     String str = new String();
                   // println( "onResults " + results);
                     ArrayList data = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
                   
                  // println("result " + data.get(0));
                  str+=data.get(0);
                  heard=str;
                  str="";
                  println(heard);
             
                  /* for (int i = 0; i < data.size(); i++)
                     {
                               println("result " + data.get(i));
                               str += data.get(i);
                     }*/
                    // mText.setText("results: "+String.valueOf(data.size()));        
            }
            public void onPartialResults(Bundle partialResults)
            {
                    // println( "onPartialResults");
            }
            public void onEvent(int eventType, Bundle params)
            {
                    // println( "onEvent " + eventType);
            }
   }
   
   

```

The above code should play a sound file after recognizing the phrase, it requires a mousePressed callback to run the Speech Recognizer

I have commented out the sound file codes above, but when you try to import the sound files like so:

“powerup = new SoundFile(this, “ui\_vats\_ready.wav”);”

the Speech Recognizer sends out a Error 8, and it feels like the the speech recognizer goes into a loop error. But when i remove the sound files import code the program runs smoothly.

Also another addition to further improve the code, Does anyone know a tutorial for AlwaysOnHotwordDetector voice Service? I would want to call my Speech recognizer after a hotword trigger so my automation code will be total hands free for android…

---

<div class="post-metadata">

**Author:** ![akenaton](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@akenaton](https://discourse.processing.org/u/akenaton)\
**Post date:** [July 1, 2020, 12:46pm UTC](https://discourse.processing.org/t/speech-recognition-via-android-mode-questions/22291/2 "2020-07-01T12:46:43Z")

</div>

@Erald ===

- I have posted these 2 codes (which are quite similar) and i know that they run.
- As for the error i suppose that it s a conflict between P5 Sound and speech recognizer (which uses record audio classes); yet i ll give a look ASAP
- As for AlwaysOnHotwordDetector API i dont think that “normal” apps can use that: only system apps have this permission.
- Why posting twice?

---

<div class="post-metadata">

**Author:** ![Erald](https://avatars.discourse-cdn.com/v4/letter/e/919ad9/32.png) [@Erald](https://discourse.processing.org/u/Erald)\
**Post date:** [July 7, 2020, 4:22am UTC](https://discourse.processing.org/t/speech-recognition-via-android-mode-questions/22291/3 "2020-07-07T04:22:37Z")

</div>

\*@akenaton thank you for your response, yes i have read your works as well. Thanks to you have an in depth understanding of speech recognition

\*Yes your codes run as i have tried myself , the code from [DanielJay ([https://forum.processing.org/two/profile/62/DanielJay](https://forum.processing.org/two/profile/62/DanielJay)) is the one that i had to rework the code to adapt to latest Processing.

\*The sample code you gave @josephh worked like a charm and i have adapted it in fact in my other automation.

- Regarding the Conflict between P5 Sound and Speech Recognizer, yes i am also trying to figure out why Speech Recognizer goes into Error code #8 when there is an import from P5 Sound. Please let me know if you have any leads.

- By system apps, you mean apps that tied to the Android System? Our normal apps wont be able to utilize AlwaysOnHotwordDetector i guess… so i’ll have to wait till there is a good update for this feature in the future.

- sorry for posting twice i thought that the AlwaysOnHotwordDetector was a different topic as i have indeed read that it was used for System apps…