Speech recognition for android mode

josephh · May 28, 2018, 11:18am

Hello,

Does someone know a speech recognition (speech to text) library for Android Mode in processing?

Thanks

akenaton · May 29, 2018, 9:42am

@josephh===
no need for a lib for that; you only have to use the android API speechRecognizer. Guidelines:

you create an instance of this service:
createSpeechRecognizer(Context context)
then set a listener for callbacks:
setRecognitionListener(RecognitionListener listener)
then start listening:
startListening(Intent recognizerIntent)

you must have permissions for micro
more details here:
https://developer.android.com/reference/android/speech/SpeechRecognizer

josephh · May 29, 2018, 3:58pm

Thanks for your answer but I’m not really experimented in JavaScript and I don’t know how to instance the speech recognizer service.
Is it :
SpeechRecognizer sp = createSpeechRecognizer(context); ?

And what is the Context context ?

Could you post the basic code for speech recognition ?

Thanks a lot.

josephh · May 29, 2018, 5:40pm

I tried to do :

import java.lang.Object;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.content.Context;

SpeechRecognizer sp;
RecognitionListener listener;

void setup(){
  fullScreen();
  
  setRecognitionListener(listener);
  sp = createSpeechRecognizer(getContext());
}


void draw(){
  background(0);
}

but it says :

The function "setRecognitionListener(Recognitionlistener)" does not exist
&&
The function "createSpeechRecognizer(Context)" does not exist

akenaton · May 30, 2018, 7:27am

@josephh===

you are in a fragment not in an activity; so you have to create an instance of Activity and make your calls from it.

josephh · May 30, 2018, 7:52am

Please I don’t understand a word of what I am doing :

import java.lang.Object;
import android.content.ContextWrapper;
import android.view.ContextThemeWrapper;
import android.app.Activity;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.content.Context;
import android.os.BaseBundle;
import android.os.Bundle;
import android.content.Intent;

RecognitionListener listener;
Activity act;
Bundle savedInstanceState;
Intent intent;

void setup(){
  fullScreen();
  
  savedInstanceState = new Bundle();
  
  onCreate(savedInstanceState);
  setRecognitionListener(listener);
  
  SpeechRecognizer sp = createSpeechRecognizer(getApplicationContext());
}


void draw(){
  background(0);
  
}

The Android documentation is so confuse.

Thanks

akenaton · May 30, 2018, 10:07am

@josephh===

you are making a strange mixed code, android…processing && yes, i can see that you dont understand what you are doing! - When trying to make things like “notification”, “speechToText” and everything of this kind you are leaving the processing world and have to know some basics about android native because processing is mainly a (good) graphics library. More: you are not in an activity, you are in a fragment (and the android code you are using is for an activity…); first thing is to understand what is a fragment and how you can call for methods present in the main activity but not in a fragment. Example for your sp you have to write

sp= SpeechRecognizer.createSpeechRecognizer(getActivity());

Anyway i am afraid that you cannot do as for now what you are trying. Supposing that you have absolutely to do that, in this case take another way, use the already implemented speechToText android, which is easy to code with 2 methods && some imports( import java.util.Locale and so on)

first one: startSpeech () ---- call it as you want!

private void startSpeech {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);///standard intent
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,
                RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT,
                "nous vous écoutons...");///show a message to the user
        try {
            this.getActivity().startActivityForResult(intent, 666);///start Activity for result with some "code" (what you want) to "identify" your call
        } catch (ActivityNotFoundException a) {
            Toast.makeText(this.getActivity().getApplicationContext(),
                    "desolé votre téléphone ne supporte pas cette fonction",
                    Toast.LENGTH_SHORT).show();///error message in case that your phone cannot offer SpeechToText
        }
    };

second methhod: callbacks and using the returned data::::

protected void onActivityResult(int requestCode, int resultCode, Intent data) {
       
        switch (requestCode) {
            case 666: {
                if (resultCode == Activity.RESULT_OK && null != data) {//data are returned && the phone can use speechToText
                    ArrayList<String> result = data
                            .getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                  ///here you add yor code for using the data
                   // jecoute = true;some boolean could be used here
                }
                break;
            }
        }
    }

Try; but dont forget to add permissions (audio_record) and (according to SDK ) add checkPermissions on run time. Make a button for starting the firts method; add a text to your app. And if you cannot do that, leave this project!

josephh · May 30, 2018, 10:40am

Well thanks for answering my questions. It’s true that I don’t really know some basics about android native but I think that it is too complicated for common people (like me even if I know how to code in Processing) to use SpeechRecognition. Someone should implement a simple library for Android Mode.
By the way I’m french !

Bonne journée

I was able to do :

import java.lang.Object;
import android.content.ContextWrapper;
import android.view.ContextThemeWrapper;
import android.app.Activity;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.speech.RecognizerIntent;
import android.content.Context;
import android.content.Intent;
import android.os.Vibrator;
import android.widget.Toast;
import java.lang.Throwable;
import java.lang.Exception;
import java.lang.RuntimeException;
import android.content.ActivityNotFoundException;
import java.util.Locale;


SpeechRecognizer sp;
Activity act;
Intent intent;

void setup(){
  fullScreen();
  
  act = this.getActivity();
  
  sp = SpeechRecognizer.createSpeechRecognizer(getActivity());
}


void draw(){
  background(0);
  
  onActivityResult(666,act.RESULT_OK,intent);
  
}

void mousePressed(){
  Vibrator vibrer = (Vibrator) act.getSystemService(Context.VIBRATOR_SERVICE);
  vibrer.vibrate(100);
}

private void startSpeech() {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);///standard intent
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT,
                "nous vous écoutons...");///show a message to the user
        try {
            this.getActivity().startActivityForResult(intent, 666);///start Activity for result with some "code" (what you want) to "identify" your call
        } catch (ActivityNotFoundException a) {
            Toast.makeText(this.getActivity().getApplicationContext(),"desolé votre téléphone ne supporte pas cette fonction",
            Toast.LENGTH_SHORT).show();///error message in case that your phone cannot offer SpeechToText
        }
}

void onActivityResult(int requestCode, int resultCode, Intent data) {
       
        switch (requestCode) {
            case 666: {
              if (resultCode == Activity.RESULT_OK && null != data) {//data are returned && the phone can use speechToText
                  ArrayList<String> result = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                  background(255);
                  text(result.get(0),width/2,height/2);r
              }
              break;
            }
    }
}

But it didn’t work ahah

akenaton · May 30, 2018, 11:06am

@josephh===
ok, me too i’m french but for others we have to speak english… Try this code and try to understand it; you can simplify it because here i am creating a button in the android way; you can (onStart()) forget it and create a bouton with the P5 way, rect, mouseX, mouseY and so on…


import android.view.View;
import android.content.Context;
import android.widget.Button;
import android.app.Activity;
///import android.speech.SpeechRecognizer;//in the "easy way that is useless
import android.view.Gravity;//for my layout
import android.graphics.Color;//for the color of the button
///import android.speech.RecognitionListener;///useless for the "easy way"
import android.speech.RecognizerIntent;

import android.content.Intent;
//import android.os.Vibrator;///let us take problems one after the other!!!!!
import android.widget.Toast;
import java.lang.Throwable;
import java.lang.Exception;
import java.lang.RuntimeException;
import android.content.ActivityNotFoundException;
import java.util.Locale;
import android.view.ViewGroup.LayoutParams;
import android.widget.FrameLayout;
import android.view.View.OnClickListener;
import android.content.Context;
import android.speech.SpeechRecognizer;

///////////////////////////////////////////////////////////////////////////////////////

Activity act;
Intent intent;
Button bouton;
private static final int MY_BUTTON1 = 9000;
 FrameLayout fl;
 Context context;
void settings(){
fullScreen();
};

public void onStart(){
   act = this.getActivity();
 context = act.getApplicationContext();

 bouton = new Button(act);
bouton.setText("startspeech");
 bouton.setBackgroundColor(Color.WHITE);
  bouton.setId(MY_BUTTON1);
      
       OnClickListener oclMonBouton = new OnClickListener() {
       public void onClick(View v) {
         println("on m'a cliqué");
         startSpeech();///here the call for method 1;
         
     }
       };
     
    bouton.setOnClickListener(oclMonBouton);

      ///adding the button to the frameLayout (processing);

    fl = (FrameLayout)act.getWindow().getDecorView().getRootView();
       getActivity().runOnUiThread(new Runnable() {
     //@Override
     public void run() {
    
     FrameLayout.LayoutParams params1 = new FrameLayout.LayoutParams(LayoutParams.WRAP_CONTENT,LayoutParams.WRAP_CONTENT,Gravity.TOP); 
                  // FrameLayout.LayoutParams params2 = new FrameLayout.LayoutParams(LayoutParams.WRAP_CONTENT,LayoutParams.WRAP_CONTENT,Gravity.BOTTOM); //uncomment if you want change the button position (which can also be set with setX());
            
            fl.addView(bouton,params1);
   
    }
});

}

void setup(){
  
  act = this.getActivity();

////create a button and add a listener to it; if you want you can also use the rect and mouseX "standard" p5 way but i prefer the android way with onClick!: that 's is made while on create();
  
  
}


void draw(){
  background(0);
  
 
  
}



private void startSpeech() {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);///standard intent
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT,
                "nous vous écoutons...");///show a message to the user
        try {
            this.getActivity().startActivityForResult(intent, 666);///start Activity for result with some "code" (what you want) to "identify" your call
        } catch (ActivityNotFoundException a) {
            Toast.makeText(this.getActivity().getApplicationContext(),"désolé votre téléphone ne supporte pas cette fonction",
            Toast.LENGTH_SHORT).show();///error message in case that your phone cannot offer SpeechToText
        }
}

void onActivityResult(int requestCode, int resultCode, Intent data) {
       
        switch (requestCode) {
            case 666: {
              if (resultCode == Activity.RESULT_OK && null != data) {//data are returned && the phone can use speechToText
                  ArrayList<String> result = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                  background(255);
                  text(result.get(0),width/2,height/2);
              }
              break;
            }
    }
}

PERMISSIONS:
audio recording && internet

josephh · May 30, 2018, 12:44pm

@akenaton

Thanks. There’s still some code that I don’t understand but I think I understand the structure and the logic.
I have questions :

How do you run the speech recognition in background without the Google window?

When do I have to use the void onActivityResult(int requestCode, int resultCode, Intent data) function to (for example) print the text result? Do I call it like this ? :

onActivityResult(666,Activity.RESULT_OK,intent);

and add :

println(result.get(0));

Thanks again

akenaton · May 30, 2018, 1:32pm

for hiding the android button and field you have to use the other way i indicated at the beginning…
for getting the result, you have only to add some code inside the callback (not in the draw()!!!)

josephh · May 30, 2018, 3:54pm

Is this ? :

sp = SpeechRecognizer.createSpeechRecognizer(getActivity());

I think that this is going to be too complicated for me (because I don’t know android JavaScript) and I don’t want you to write me all the code. Thanks

akenaton · May 30, 2018, 3:57pm

@josephh===

yes it 's the very beginning…
BTW= that is NOT javaScript!

josephh · May 30, 2018, 4:00pm

What is it then ?

There’s so many syntax for doing simple things (like creating a button or making speechRecognition lol).

akenaton · May 30, 2018, 4:02pm

@josephh==

let’say some kind of continent inside the java world…

josephh · May 30, 2018, 4:03pm

@akenaton

Thanks a lot for all you answers !

kfrajer · July 18, 2018, 7:24pm

@josephh I just noticed this post. I suggest you have a look at the Processing Android’s websites. There are some tutorials there. You can also fin the first three chapters of Andres’ book there where he talks about Processing and the Android mode. Andres is the creator and leader of this project.

You are also very welcome to explore previous android posts to understand what you can do with P3 Android. For basic things, you do not need to know the Android API as Processing Android will cover all the technicalities. When you start using Android API’s features, then it is important to check under the hood and understand how Android does things, and specially, how Processing manages the Android part. A good way to start in Android is by checking the life cycle of an activity.

@akenaton Thxs for sharing this post.

Kf

noel · July 18, 2018, 11:02pm

@akenaton
Thanks for this nice code.
I have some trouble with the language choise.
I outcommented the folowing lines

//  intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
// intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);

Still the local (portuguese Brazil) AND the english language are recognized.
I want to work with the dutch and german language so I added the folowing lines:

intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_PREFERENCE, "nl-NL");

But that does not work.
Any idea?

akenaton · July 19, 2018, 8:30am

@noel===

EXTRA_LANGUAGE_PREFERENCE gets the user preference: have you set it to nl??? -is this language present in the list on your phone (you can see it or get it with code, using LANGUAGE_DETAILS, which returns this list and also the language preference set:: https://developer.android.com/reference/android/speech/RecognizerIntent.html#ACTION_GET_LANGUAGE_DETAILS
then if everything is ok try this code:
intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,
RecognizerIntent.EXTRA_LANGUAGE, “nl-NL”);

juanjoc · January 13, 2020, 8:13am

Hi guys

Is it any function that activate the speech recognition with a keyword? E.G. “OK GOOGLE” and any keyword to finalize the speech recognition? or the limit time to begin the speech recognition

Topic		Replies	Views
Creating speech recognition Processing for Android	5	1041	February 4, 2020
Speech Recognition via Android Mode Questions Processing for Android	2	471	July 7, 2020
Is there a text to speech for Android? Processing for Android	22	952	May 1, 2022
AlwaysOnHotwordDetector Examples for Android Mode Processing for Android	2	729	September 4, 2020
How to add speech to text in processing android Processing for Android	8	527	July 1, 2020

Speech recognition for android mode

Related Topics