Speech recognition for android mode


#1

Hello,

Does someone know a speech recognition (speech to text) library for Android Mode in processing?

Thanks


#2

@josephh===
no need for a lib for that; you only have to use the android API speechRecognizer. Guidelines:

  • you create an instance of this service:
    createSpeechRecognizer(Context context)
  • then set a listener for callbacks:
    setRecognitionListener(RecognitionListener listener)
  • then start listening:
    startListening(Intent recognizerIntent)

you must have permissions for micro
more details here:
https://developer.android.com/reference/android/speech/SpeechRecognizer


#3

Thanks for your answer but I’m not really experimented in JavaScript and I don’t know how to instance the speech recognizer service.
Is it :
SpeechRecognizer sp = createSpeechRecognizer(context); ?

And what is the Context context ?

Could you post the basic code for speech recognition ?

Thanks a lot.


#4

I tried to do :

import java.lang.Object;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.content.Context;

SpeechRecognizer sp;
RecognitionListener listener;

void setup(){
  fullScreen();
  
  setRecognitionListener(listener);
  sp = createSpeechRecognizer(getContext());
}


void draw(){
  background(0);
}

but it says :

The function "setRecognitionListener(Recognitionlistener)" does not exist
&&
The function "createSpeechRecognizer(Context)" does not exist

#5

@josephh===

you are in a fragment not in an activity; so you have to create an instance of Activity and make your calls from it.


#6

Please I don’t understand a word of what I am doing :

import java.lang.Object;
import android.content.ContextWrapper;
import android.view.ContextThemeWrapper;
import android.app.Activity;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.content.Context;
import android.os.BaseBundle;
import android.os.Bundle;
import android.content.Intent;

RecognitionListener listener;
Activity act;
Bundle savedInstanceState;
Intent intent;

void setup(){
  fullScreen();
  
  savedInstanceState = new Bundle();
  
  onCreate(savedInstanceState);
  setRecognitionListener(listener);
  
  SpeechRecognizer sp = createSpeechRecognizer(getApplicationContext());
}


void draw(){
  background(0);
  
}

The Android documentation is so confuse.

Thanks


#7

@josephh===

you are making a strange mixed code, android…processing && yes, i can see that you dont understand what you are doing! - When trying to make things like “notification”, “speechToText” and everything of this kind you are leaving the processing world and have to know some basics about android native because processing is mainly a (good) graphics library. More: you are not in an activity, you are in a fragment (and the android code you are using is for an activity…); first thing is to understand what is a fragment and how you can call for methods present in the main activity but not in a fragment. Example for your sp you have to write

sp= SpeechRecognizer.createSpeechRecognizer(getActivity());

Anyway i am afraid that you cannot do as for now what you are trying. Supposing that you have absolutely to do that, in this case take another way, use the already implemented speechToText android, which is easy to code with 2 methods && some imports( import java.util.Locale and so on)

first one: startSpeech () ---- call it as you want!

private void startSpeech {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);///standard intent
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,
                RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT,
                "nous vous Ă©coutons...");///show a message to the user
        try {
            this.getActivity().startActivityForResult(intent, 666);///start Activity for result with some "code" (what you want) to "identify" your call
        } catch (ActivityNotFoundException a) {
            Toast.makeText(this.getActivity().getApplicationContext(),
                    "desolé votre téléphone ne supporte pas cette fonction",
                    Toast.LENGTH_SHORT).show();///error message in case that your phone cannot offer SpeechToText
        }
    };

second methhod: callbacks and using the returned data::::

protected void onActivityResult(int requestCode, int resultCode, Intent data) {
       
        switch (requestCode) {
            case 666: {
                if (resultCode == Activity.RESULT_OK && null != data) {//data are returned && the phone can use speechToText
                    ArrayList<String> result = data
                            .getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                  ///here you add yor code for using the data
                   // jecoute = true;some boolean could be used here
                }
                break;
            }
        }
    }

Try; but dont forget to add permissions (audio_record) and (according to SDK ) add checkPermissions on run time. Make a button for starting the firts method; add a text to your app. And if you cannot do that, leave this project!


#8

Well thanks for answering my questions. It’s true that I don’t really know some basics about android native but I think that it is too complicated for common people (like me even if I know how to code in Processing) to use SpeechRecognition. Someone should implement a simple library for Android Mode.
By the way I’m french !

Bonne journée :sweat_smile:

I was able to do :

import java.lang.Object;
import android.content.ContextWrapper;
import android.view.ContextThemeWrapper;
import android.app.Activity;
import android.speech.SpeechRecognizer;
import android.speech.RecognitionListener;
import android.speech.RecognizerIntent;
import android.content.Context;
import android.content.Intent;
import android.os.Vibrator;
import android.widget.Toast;
import java.lang.Throwable;
import java.lang.Exception;
import java.lang.RuntimeException;
import android.content.ActivityNotFoundException;
import java.util.Locale;


SpeechRecognizer sp;
Activity act;
Intent intent;

void setup(){
  fullScreen();
  
  act = this.getActivity();
  
  sp = SpeechRecognizer.createSpeechRecognizer(getActivity());
}


void draw(){
  background(0);
  
  onActivityResult(666,act.RESULT_OK,intent);
  
}

void mousePressed(){
  Vibrator vibrer = (Vibrator) act.getSystemService(Context.VIBRATOR_SERVICE);
  vibrer.vibrate(100);
}

private void startSpeech() {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);///standard intent
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT,
                "nous vous Ă©coutons...");///show a message to the user
        try {
            this.getActivity().startActivityForResult(intent, 666);///start Activity for result with some "code" (what you want) to "identify" your call
        } catch (ActivityNotFoundException a) {
            Toast.makeText(this.getActivity().getApplicationContext(),"desolé votre téléphone ne supporte pas cette fonction",
            Toast.LENGTH_SHORT).show();///error message in case that your phone cannot offer SpeechToText
        }
}

void onActivityResult(int requestCode, int resultCode, Intent data) {
       
        switch (requestCode) {
            case 666: {
              if (resultCode == Activity.RESULT_OK && null != data) {//data are returned && the phone can use speechToText
                  ArrayList<String> result = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                  background(255);
                  text(result.get(0),width/2,height/2);r
              }
              break;
            }
    }
}

But it didn’t work ahah :grin:


#9

@josephh===
ok, me too i’m french but for others we have to speak english… Try this code and try to understand it; you can simplify it because here i am creating a button in the android way; you can (onStart()) forget it and create a bouton with the P5 way, rect, mouseX, mouseY and so on…


import android.view.View;
import android.content.Context;
import android.widget.Button;
import android.app.Activity;
///import android.speech.SpeechRecognizer;//in the "easy way that is useless
import android.view.Gravity;//for my layout
import android.graphics.Color;//for the color of the button
///import android.speech.RecognitionListener;///useless for the "easy way"
import android.speech.RecognizerIntent;

import android.content.Intent;
//import android.os.Vibrator;///let us take problems one after the other!!!!!
import android.widget.Toast;
import java.lang.Throwable;
import java.lang.Exception;
import java.lang.RuntimeException;
import android.content.ActivityNotFoundException;
import java.util.Locale;
import android.view.ViewGroup.LayoutParams;
import android.widget.FrameLayout;
import android.view.View.OnClickListener;
import android.content.Context;
import android.speech.SpeechRecognizer;

///////////////////////////////////////////////////////////////////////////////////////

Activity act;
Intent intent;
Button bouton;
private static final int MY_BUTTON1 = 9000;
 FrameLayout fl;
 Context context;
void settings(){
fullScreen();
};

public void onStart(){
   act = this.getActivity();
 context = act.getApplicationContext();

 bouton = new Button(act);
bouton.setText("startspeech");
 bouton.setBackgroundColor(Color.WHITE);
  bouton.setId(MY_BUTTON1);
      
       OnClickListener oclMonBouton = new OnClickListener() {
       public void onClick(View v) {
         println("on m'a cliqué");
         startSpeech();///here the call for method 1;
         
     }
       };
     
    bouton.setOnClickListener(oclMonBouton);

      ///adding the button to the frameLayout (processing);

    fl = (FrameLayout)act.getWindow().getDecorView().getRootView();
       getActivity().runOnUiThread(new Runnable() {
     //@Override
     public void run() {
    
     FrameLayout.LayoutParams params1 = new FrameLayout.LayoutParams(LayoutParams.WRAP_CONTENT,LayoutParams.WRAP_CONTENT,Gravity.TOP); 
                  // FrameLayout.LayoutParams params2 = new FrameLayout.LayoutParams(LayoutParams.WRAP_CONTENT,LayoutParams.WRAP_CONTENT,Gravity.BOTTOM); //uncomment if you want change the button position (which can also be set with setX());
            
            fl.addView(bouton,params1);
   
    }
});

}

void setup(){
  
  act = this.getActivity();

////create a button and add a listener to it; if you want you can also use the rect and mouseX "standard" p5 way but i prefer the android way with onClick!: that 's is made while on create();
  
  
}


void draw(){
  background(0);
  
 
  
}



private void startSpeech() {
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);///standard intent
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());//choose the local language
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT,
                "nous vous Ă©coutons...");///show a message to the user
        try {
            this.getActivity().startActivityForResult(intent, 666);///start Activity for result with some "code" (what you want) to "identify" your call
        } catch (ActivityNotFoundException a) {
            Toast.makeText(this.getActivity().getApplicationContext(),"désolé votre téléphone ne supporte pas cette fonction",
            Toast.LENGTH_SHORT).show();///error message in case that your phone cannot offer SpeechToText
        }
}

void onActivityResult(int requestCode, int resultCode, Intent data) {
       
        switch (requestCode) {
            case 666: {
              if (resultCode == Activity.RESULT_OK && null != data) {//data are returned && the phone can use speechToText
                  ArrayList<String> result = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                  background(255);
                  text(result.get(0),width/2,height/2);
              }
              break;
            }
    }
}

PERMISSIONS:
audio recording && internet


#10

@akenaton

Thanks. There’s still some code that I don’t understand but I think I understand the structure and the logic.
I have questions :

  • How do you run the speech recognition in background without the Google window?

  • When do I have to use the void onActivityResult(int requestCode, int resultCode, Intent data) function to (for example) print the text result? Do I call it like this ? :
onActivityResult(666,Activity.RESULT_OK,intent);

and add :

println(result.get(0));

Thanks again


#11

for hiding the android button and field you have to use the other way i indicated at the beginning…
for getting the result, you have only to add some code inside the callback (not in the draw()!!!)


#12

Is this ? :

sp = SpeechRecognizer.createSpeechRecognizer(getActivity());

I think that this is going to be too complicated for me (because I don’t know android JavaScript) and I don’t want you to write me all the code. Thanks


#13

@josephh===

yes it 's the very beginning…
BTW= that is NOT javaScript!


#14

What is it then ? :joy:

There’s so many syntax for doing simple things (like creating a button or making speechRecognition lol).


#15

@josephh==

let’say some kind of continent inside the java world…


#16

@akenaton

Thanks a lot for all you answers !