# Speech recognition for android mode

**URL:** <https://discourse.processing.org/t/speech-recognition-for-android-mode/463>\
**Category:** Processing for Android\
**Created:** [May 28, 2018, 11:18am UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463 "2018-05-28T11:18:30Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 28, 2018, 11:18am UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/1 "2018-05-28T11:18:30Z")

</div>

Hello,

Does someone know a speech recognition (speech to text) library for Android Mode in processing?

Thanks

---

<div class="post-metadata">

**Author:** ![akenaton](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@akenaton](https://discourse.processing.org/u/akenaton)\
**Post date:** [May 29, 2018, 9:42am UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/2 "2018-05-29T09:42:24Z")

</div>

@josephh===  
no need for a lib for that; you only have to use the android API speechRecognizer. Guidelines:

- you create an instance of this service:  
createSpeechRecognizer(Context context)
- then set a listener for callbacks:  
setRecognitionListener(RecognitionListener listener)
- then start listening:  
startListening(Intent recognizerIntent)

you must have permissions for micro  
more details here:  
[https://developer.android.com/reference/android/speech/SpeechRecognizer](https://developer.android.com/reference/android/speech/SpeechRecognizer)

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 29, 2018, 3:58pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/3 "2018-05-29T15:58:53Z")

</div>

Thanks for your answer but I’m not really experimented in JavaScript and I don’t know how to instance the speech recognizer service.  
Is it :  
`SpeechRecognizer sp = createSpeechRecognizer(context);` ?

And what is the `Context context` ?

Could you post the basic code for speech recognition ?

Thanks a lot.

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 29, 2018, 5:40pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/4 "2018-05-29T17:40:21Z")

</div>

I tried to do :

```auto
import java.lang.Object;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.content.Context;

SpeechRecognizer sp;
RecognitionListener listener;

void setup(){
  fullScreen();
  
  setRecognitionListener(listener);
  sp = createSpeechRecognizer(getContext());
}

void draw(){
  background(0);
}

```

but it says :

```auto
The function "setRecognitionListener(Recognitionlistener)" does not exist
&&
The function "createSpeechRecognizer(Context)" does not exist

```

---

<div class="post-metadata">

**Author:** ![akenaton](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@akenaton](https://discourse.processing.org/u/akenaton)\
**Post date:** [May 30, 2018, 7:27am UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/5 "2018-05-30T07:27:09Z")

</div>

@josephh===

you are in a fragment not in an activity; so you have to create an instance of Activity and make your calls from it.

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 30, 2018, 7:52am UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/6 "2018-05-30T07:52:23Z")

</div>

Please I don’t understand a word of what I am doing :

```auto
import java.lang.Object;
import android.content.ContextWrapper;
import android.view.ContextThemeWrapper;
import android.app.Activity;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.content.Context;
import android.os.BaseBundle;
import android.os.Bundle;
import android.content.Intent;

RecognitionListener listener;
Activity act;
Bundle savedInstanceState;
Intent intent;

void setup(){
  fullScreen();
  
  savedInstanceState = new Bundle();
  
  onCreate(savedInstanceState);
  setRecognitionListener(listener);
  
  SpeechRecognizer sp = createSpeechRecognizer(getApplicationContext());
}

void draw(){
  background(0);
  
}

```

The Android documentation is so confuse.

Thanks

---

<div class="post-metadata">

**Author:** ![akenaton](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@akenaton](https://discourse.processing.org/u/akenaton)\
**Post date:** [May 30, 2018, 10:07am UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/7 "2018-05-30T10:07:25Z")

</div>

@josephh===

you are making a strange mixed code, android…processing && yes, i can see that you dont understand what you are doing! - When trying to make things like “notification”, “speechToText” and everything of this kind you are leaving the processing world and have to know some basics about android native because processing is mainly a (good) graphics library. More: you are not in an activity, you are in a fragment (and the android code you are using is for an activity…); first thing is to understand what is a fragment and how you can call for methods present in the main activity but not in a fragment. Example for your sp you have to write

`sp= SpeechRecognizer.createSpeechRecognizer(getActivity());`

Anyway i am afraid that you cannot do as for now what you are trying. Supposing that you have absolutely to do that, in this case take another way, use the already implemented speechToText android, which is easy to code with 2 methods && some imports( import java.util.Locale and so on)

first one: startSpeech () ---- call it as you want!

```auto
private void startSpeech {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);///standard intent
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,
                RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT,
                "nous vous écoutons...");///show a message to the user
        try {
            this.getActivity().startActivityForResult(intent, 666);///start Activity for result with some "code" (what you want) to "identify" your call
        } catch (ActivityNotFoundException a) {
            Toast.makeText(this.getActivity().getApplicationContext(),
                    "desolé votre téléphone ne supporte pas cette fonction",
                    Toast.LENGTH_SHORT).show();///error message in case that your phone cannot offer SpeechToText
        }
    };

```

second methhod: callbacks and using the returned data::::

```auto
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
       
        switch (requestCode) {
            case 666: {
                if (resultCode == Activity.RESULT_OK && null != data) {//data are returned && the phone can use speechToText
                    ArrayList<String> result = data
                            .getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                  ///here you add yor code for using the data
                   // jecoute = true;some boolean could be used here
                }
                break;
            }
        }
    }

```

Try; but dont forget to add permissions (audio\_record) and (according to SDK ) add checkPermissions on run time. Make a button for starting the firts method; add a text to your app. And if you cannot do that, leave this project!

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 30, 2018, 10:40am UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/8 "2018-05-30T10:40:56Z")

</div>

> [@akenaton](#):
>
> And if you cannot do that, leave this project!

Well thanks for answering my questions. It’s true that I don’t really know some basics about android native but I think that it is too complicated for common people (like me even if I know how to code in Processing) to use SpeechRecognition. Someone should implement a simple library for Android Mode.  
By the way I’m french !

Bonne journée 😅

I was able to do :

```auto
import java.lang.Object;
import android.content.ContextWrapper;
import android.view.ContextThemeWrapper;
import android.app.Activity;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.speech.RecognizerIntent;
import android.content.Context;
import android.content.Intent;
import android.os.Vibrator;
import android.widget.Toast;
import java.lang.Throwable;
import java.lang.Exception;
import java.lang.RuntimeException;
import android.content.ActivityNotFoundException;
import java.util.Locale;

SpeechRecognizer sp;
Activity act;
Intent intent;

void setup(){
  fullScreen();
  
  act = this.getActivity();
  
  sp = SpeechRecognizer.createSpeechRecognizer(getActivity());
}

void draw(){
  background(0);
  
  onActivityResult(666,act.RESULT_OK,intent);
  
}

void mousePressed(){
  Vibrator vibrer = (Vibrator) act.getSystemService(Context.VIBRATOR_SERVICE);
  vibrer.vibrate(100);
}

private void startSpeech() {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);///standard intent
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT,
                "nous vous écoutons...");///show a message to the user
        try {
            this.getActivity().startActivityForResult(intent, 666);///start Activity for result with some "code" (what you want) to "identify" your call
        } catch (ActivityNotFoundException a) {
            Toast.makeText(this.getActivity().getApplicationContext(),"desolé votre téléphone ne supporte pas cette fonction",
            Toast.LENGTH_SHORT).show();///error message in case that your phone cannot offer SpeechToText
        }
}

void onActivityResult(int requestCode, int resultCode, Intent data) {
       
        switch (requestCode) {
            case 666: {
              if (resultCode == Activity.RESULT_OK && null != data) {//data are returned && the phone can use speechToText
                  ArrayList<String> result = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                  background(255);
                  text(result.get(0),width/2,height/2);r
              }
              break;
            }
    }
}

```

But it didn’t work ahah 😁

---

<div class="post-metadata">

**Author:** ![akenaton](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@akenaton](https://discourse.processing.org/u/akenaton)\
**Post date:** [May 30, 2018, 11:06am UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/9 "2018-05-30T11:06:08Z")

</div>

@josephh===  
ok, me too i’m french but for others we have to speak english… Try this code and try to understand it; you can simplify it because here i am creating a button in the android way; you can (onStart()) forget it and create a bouton with the P5 way, rect, mouseX, mouseY and so on…

```auto

import android.view.View;
import android.content.Context;
import android.widget.Button;
import android.app.Activity;
///import android.speech.SpeechRecognizer;//in the "easy way that is useless
import android.view.Gravity;//for my layout
import android.graphics.Color;//for the color of the button
///import android.speech.RecognitionListener;///useless for the "easy way"
import android.speech.RecognizerIntent;

import android.content.Intent;
//import android.os.Vibrator;///let us take problems one after the other!!!!!
import android.widget.Toast;
import java.lang.Throwable;
import java.lang.Exception;
import java.lang.RuntimeException;
import android.content.ActivityNotFoundException;
import java.util.Locale;
import android.view.ViewGroup.LayoutParams;
import android.widget.FrameLayout;
import android.view.View.OnClickListener;
import android.content.Context;
import android.speech.SpeechRecognizer;

///////////////////////////////////////////////////////////////////////////////////////

Activity act;
Intent intent;
Button bouton;
private static final int MY_BUTTON1 = 9000;
 FrameLayout fl;
 Context context;
void settings(){
fullScreen();
};

public void onStart(){
   act = this.getActivity();
 context = act.getApplicationContext();

 bouton = new Button(act);
bouton.setText("startspeech");
 bouton.setBackgroundColor(Color.WHITE);
  bouton.setId(MY_BUTTON1);
      
       OnClickListener oclMonBouton = new OnClickListener() {
       public void onClick(View v) {
         println("on m'a cliqué");
         startSpeech();///here the call for method 1;
         
     }
       };
     
    bouton.setOnClickListener(oclMonBouton);

      ///adding the button to the frameLayout (processing);

    fl = (FrameLayout)act.getWindow().getDecorView().getRootView();
       getActivity().runOnUiThread(new Runnable() {
     //@Override
     public void run() {
    
     FrameLayout.LayoutParams params1 = new FrameLayout.LayoutParams(LayoutParams.WRAP_CONTENT,LayoutParams.WRAP_CONTENT,Gravity.TOP); 
                  // FrameLayout.LayoutParams params2 = new FrameLayout.LayoutParams(LayoutParams.WRAP_CONTENT,LayoutParams.WRAP_CONTENT,Gravity.BOTTOM); //uncomment if you want change the button position (which can also be set with setX());
            
            fl.addView(bouton,params1);
   
    }
});

}

void setup(){
  
  act = this.getActivity();

////create a button and add a listener to it; if you want you can also use the rect and mouseX "standard" p5 way but i prefer the android way with onClick!: that 's is made while on create();
  
  
}

void draw(){
  background(0);
  
 
  
}

private void startSpeech() {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);///standard intent
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT,
                "nous vous écoutons...");///show a message to the user
        try {
            this.getActivity().startActivityForResult(intent, 666);///start Activity for result with some "code" (what you want) to "identify" your call
        } catch (ActivityNotFoundException a) {
            Toast.makeText(this.getActivity().getApplicationContext(),"désolé votre téléphone ne supporte pas cette fonction",
            Toast.LENGTH_SHORT).show();///error message in case that your phone cannot offer SpeechToText
        }
}

void onActivityResult(int requestCode, int resultCode, Intent data) {
       
        switch (requestCode) {
            case 666: {
              if (resultCode == Activity.RESULT_OK && null != data) {//data are returned && the phone can use speechToText
                  ArrayList<String> result = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                  background(255);
                  text(result.get(0),width/2,height/2);
              }
              break;
            }
    }
}

```

PERMISSIONS:  
audio recording && internet

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 30, 2018, 12:44pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/10 "2018-05-30T12:44:21Z")

</div>

@akenaton

Thanks. There’s still some code that I don’t understand but I think I understand the structure and the logic.  
I have questions :

- How do you run the speech recognition in background without the Google window?

 ![Screenshot_2018-05-30-14-39-21%5B1%5D](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/1X/f6668ec99be77a7892085d251cf9059a178f7405.png)

- When do I have to use the `void onActivityResult(int requestCode, int resultCode, Intent data)` function to (for example) print the text result? Do I call it like this ? :

```auto
onActivityResult(666,Activity.RESULT_OK,intent);

```

and add :

```auto
println(result.get(0));

```

Thanks again

---

<div class="post-metadata">

**Author:** ![akenaton](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@akenaton](https://discourse.processing.org/u/akenaton)\
**Post date:** [May 30, 2018, 1:32pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/11 "2018-05-30T13:32:11Z")

</div>

for hiding the android button and field you have to use the other way i indicated at the beginning…  
for getting the result, you have only to add some code inside the callback (not in the draw()!!!)

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 30, 2018, 3:54pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/12 "2018-05-30T15:54:19Z")

</div>

> [@akenaton](#):
>
> use the other way i indicated at the beginning…

Is this ? :

```auto
sp = SpeechRecognizer.createSpeechRecognizer(getActivity());

```

I think that this is going to be too complicated for me (because I don’t know android JavaScript) and I don’t want you to write me all the code. Thanks

---

<div class="post-metadata">

**Author:** ![akenaton](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@akenaton](https://discourse.processing.org/u/akenaton)\
**Post date:** [May 30, 2018, 3:57pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/13 "2018-05-30T15:57:19Z")

</div>

@josephh===

yes it 's the very beginning…  
BTW= that is NOT javaScript!

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 30, 2018, 4:00pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/14 "2018-05-30T16:00:14Z")

</div>

> [@akenaton](#):
>
> BTW= that is NOT javaScript!

What is it then ? 😂

> [@akenaton](#):
>
> yes it 's the very beginning…

There’s so many syntax for doing simple things (like creating a button or making speechRecognition lol).

---

<div class="post-metadata">

**Author:** ![akenaton](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@akenaton](https://discourse.processing.org/u/akenaton)\
**Post date:** [May 30, 2018, 4:02pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/15 "2018-05-30T16:02:33Z")

</div>

@josephh==

let’say some kind of continent inside the java world…

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 30, 2018, 4:03pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/16 "2018-05-30T16:03:47Z")

</div>

@akenaton

Thanks a lot for all you answers !

---

<div class="post-metadata">

**Author:** ![kfrajer](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/kfrajer/32/196_2.png) [@kfrajer](https://discourse.processing.org/u/kfrajer)\
**Post date:** [July 18, 2018, 7:24pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/17 "2018-07-18T19:24:33Z")

</div>

@josephh I just noticed this post. I suggest you have a look at the Processing Android’s websites. There are some tutorials there. You can also fin the first three chapters of Andres’ book there where he talks about Processing and the Android mode. Andres is the creator and leader of this project.

You are also very welcome to explore previous android posts to understand what you can do with P3 Android. For basic things, you do not need to know the Android API as Processing Android will cover all the technicalities. When you start using Android API’s features, then it is important to check under the hood and understand how Android does things, and specially, how Processing manages the Android part. A good way to start in Android is by checking the life cycle of an activity.

@akenaton Thxs for sharing this post.

Kf

---

<div class="post-metadata">

**Author:** ![noel](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/noel/32/213_2.png) [@noel](https://discourse.processing.org/u/noel)\
**Post date:** [July 18, 2018, 11:02pm UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/18 "2018-07-18T23:02:01Z")

</div>

@akenaton  
Thanks for this nice code.  
I have some trouble with the language choise.  
I outcommented the folowing lines

```auto
// intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
// intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);

```

Still the local (portuguese Brazil) AND the english language are recognized.  
I want to work with the dutch and german language so I added the folowing lines:

`intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_PREFERENCE, "nl-NL");`

But that does not work.  
Any idea?

---

<div class="post-metadata">

**Author:** ![akenaton](https://avatars.discourse-cdn.com/v4/letter/a/a88e4f/32.png) [@akenaton](https://discourse.processing.org/u/akenaton)\
**Post date:** [July 19, 2018, 8:30am UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/19 "2018-07-19T08:30:55Z")

</div>

@noel===

EXTRA\_LANGUAGE\_PREFERENCE gets the user preference: have you set it to nl??? -is this language present in the list on your phone (you can see it or get it with code, using LANGUAGE\_DETAILS, which returns this list and also the language preference set:: [https://developer.android.com/reference/android/speech/RecognizerIntent.html#ACTION\_GET\_LANGUAGE\_DETAILS](https://developer.android.com/reference/android/speech/RecognizerIntent.html#ACTION_GET_LANGUAGE_DETAILS)  
then if everything is ok try this code:  
intent.putExtra(RecognizerIntent.EXTRA\_LANGUAGE\_MODEL,  
RecognizerIntent.EXTRA\_LANGUAGE, “nl-NL”);

---

<div class="post-metadata">

**Author:** ![juanjoc](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@juanjoc](https://discourse.processing.org/u/juanjoc)\
**Post date:** [January 13, 2020, 8:13am UTC](https://discourse.processing.org/t/speech-recognition-for-android-mode/463/20 "2020-01-13T08:13:25Z")

</div>

Hi guys

Is it any function that activate the speech recognition with a keyword? E.G. “OK GOOGLE” and any keyword to finalize the speech recognition? or the limit time to begin the speech recognition