# Capturing game controller button events in Processing for Android

**URL:** <https://discourse.processing.org/t/capturing-game-controller-button-events-in-processing-for-android/40073>\
**Category:** Processing for Android\
**Created:** [December 8, 2022, 6:41pm UTC](https://discourse.processing.org/t/capturing-game-controller-button-events-in-processing-for-android/40073 "2022-12-08T18:41:05Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sableraph](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/sableraph/32/251_2.png) [@sableraph](https://discourse.processing.org/u/sableraph)\
**Post date:** [December 8, 2022, 6:41pm UTC](https://discourse.processing.org/t/capturing-game-controller-button-events-in-processing-for-android/40073/1 "2022-12-08T18:41:05Z")

</div>

Hello everyone!

I’m trying to write a sketch in Processing for Android that gets inputs from a physical game controller.

With help from the [documentation on controller input](https://developer.android.com/develop/ui/views/touch-and-input/game-controllers/controller-input#java) I managed to make a sketch that uses `android.view.InputDevice` and makes a list of connected controllers and outputs it to the console.

Now I am trying to capture button presses and I am finding this challenging.

Here’s the code I have cobbled together, focusing on the directional keys for now.

```auto
Dpad dpad = new Dpad();

public boolean onKeyDown(int keyCode, KeyEvent event) {
    int direction = dpad.getDirectionPressed(event);
    // if the event comes from a dpad
    if (direction != -1) {
        // check which direction was pressed
        switch (direction) {
            case Dpad.UP:
                System.out.println("UP button pressed");
                break;
            case Dpad.LEFT:
                System.out.println("LEFT button pressed");
                break;
            case Dpad.RIGHT:
                System.out.println("RIGHT button pressed");
                break;
            case Dpad.DOWN:
                System.out.println("DOWN button pressed");
                break;
            default:
                break;
        }
    }
    return true;
}

```

_Note: The `Dpad` class comes from [this example](https://developer.android.com/develop/ui/views/touch-and-input/game-controllers/controller-input#dpad) in the documentation._

The code snippet below comes from the doc as well. It shows that the `onKeyDown` method should be an Override inside of a class that extends the View 😵‍💫

```auto
public class GameView extends View {
    ...

    @Override
    public boolean onKeyDown(int keyCode, KeyEvent event) {
         ...

```

This is where I’m getting stumped as these concepts don’t really fit with my understanding of Processing.

How can I capture these gamepad events in Processing for Android?

---

<div class="post-metadata">

**Author:** ![GoToLoop](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/gotoloop/32/86_2.png) [@GoToLoop](https://discourse.processing.org/u/GoToLoop)\
**Post date:** [December 8, 2022, 11:17pm UTC](https://discourse.processing.org/t/capturing-game-controller-button-events-in-processing-for-android/40073/2 "2022-12-08T23:17:19Z")

</div>

> [@sableraph](#):
>
> … as these concepts don’t really fit with my understanding of Processing.

Disclaimer: I’ve never dealt w/ Android coding. This is purely blind advice! 🕶

- In Java everything must be inside a class or interface.
- Processing hides that basic truth from us when our code is inside a “.pde” file.
- But if we check the “.java” file which got transpiled out of our “.pde” files we’ll realize everything ends up wrapped inside a PApplet subclass after all.
- When we need to define a callback function in Java, that also ends up belonging to some class or interface.
- So even callbacks gotta belong to some datatype and all of its inherited datatypes.
- And consequently we need to pass an instance of a callback’s class instead of the callback itself.
- Unless a callback belongs to a [@FunctionalInterface](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/FunctionalInterface.html); in which case we can alternatively pass a lambda expression instead.
- According to Android’s documentation, we’re advised to create a helper class w/ suggested name Dpad.
- Plus a subclass which [`extends`](https://processing.org/reference/extends.html) class View w/ suggested name GameView, in which we can [@Override](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Override.html) View’s callback functions.
- Basically that’s where you should define your customized **onKeyDown()** callback event method aKa listener.

> [@sableraph](#):
>
> How can I capture these gamepad events in Processing for Android?

- Unfortunately that’s where I can’t help you anymore! 😢
- It seems like parent class View is merely a placeholder for listener callback methods.
- Instantiating a `new` View doesn’t magically activate its listeners at all.
- You’d still need to pass its instance to some “unknown” Android method which will then register & start listening to the View’s callbacks within your sketch app.

---

<div class="post-metadata">

**Author:** ![sableraph](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/sableraph/32/251_2.png) [@sableraph](https://discourse.processing.org/u/sableraph)\
**Post date:** [December 9, 2022, 11:12am UTC](https://discourse.processing.org/t/capturing-game-controller-button-events-in-processing-for-android/40073/3 "2022-12-09T11:12:52Z")

</div>

Thanks for the pointers @GoToLoop 🤝 Sounds like I need to read up on Java callbacks.

There is an [example about sensors](https://android.processing.org/tutorials/sensors/index.html) in the Processing for Android documentation that shows how to get accelerometer data. It creates a `android.hardware.SensorManager` called `manager` and registers the listener with `manager.registerListener(...)`.

I did a bit more research and it looks like Android has a method called `android.hardware.input.InputManager.registerInputDeviceListener()` and I’m guessing that’s the “unknown” Android method you hinted at?

Either way I have my work cut out for me! Thanks for your help 🤗

---

<div class="post-metadata">

**Author:** ![sableraph](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/sableraph/32/251_2.png) [@sableraph](https://discourse.processing.org/u/sableraph)\
**Post date:** [December 9, 2022, 10:46pm UTC](https://discourse.processing.org/t/capturing-game-controller-button-events-in-processing-for-android/40073/4 "2022-12-09T22:46:18Z")

</div>

I am still investigating this, but it is a challenge for me since I have no prior experience with the Android framework 🥲

In the meantime I have made a feature request on the Processing for Android repository:

> <https://github.com/processing/processing-android/issues/715>
>
> It would be great if Processing for Android could wrap the \[APIs for detecting a…nd processing user input from game controllers\](https://developer.android.com/develop/ui/views/touch-and-input/game-controllers/) that are already included in the Android framework. They are not currently easily accessible in Processing for Android. Making these APIs accessible in Processing would enable developers to create interactive experiences using game controllers on Android devices.
> 
> References:
> \- https://developer.android.com/develop/ui/views/touch-and-input/game-controllers/
> \- https://discourse.processing.org/t/capturing-game-controller-button-events-in-processing-for-android/40073/2

---

<div class="post-metadata">

**Author:** ![jafal](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jafal/32/19112_2.png) [@jafal](https://discourse.processing.org/u/jafal)\
**Post date:** [December 10, 2022, 2:07am UTC](https://discourse.processing.org/t/capturing-game-controller-button-events-in-processing-for-android/40073/5 "2022-12-10T02:07:54Z")

</div>

Hi Sir  
Maybe I am far about your topic  
But Easy way to do it

1- If the controller connect via Bluetooth I think it’s easy to capture the key’s code as array and use it in sketch

2-You can read Bluetooth key code if you want using Serial Bluetooth Terminal App for Android

 ![screen-2](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/3X/e/5/e5d307c4691f3e8640434eca8bf07db72de02933.webp)

3 -for USB I found one library support USB for Android make connection with Arduino (not all kinds of Arduino) I don’t know if it’s support game controller

Here is example to receive Bluetooth

> [@Android Bluetooth](https://discourse.processing.org/t/android-bluetooth/27140):
>
> The Ketai Bluetooth Class method onBluetoothDataEvent() does not work on the latest Android modes. Also, it has no readStringUntil() method which makes it difficult to read fast incoming data. So I wrote a native Android Bluetooth code to make this possible. I tested it on a MultiLaser Lollipop 5.1 device, as well as on a Samsung Android 10, using an HC-05 module set to a baud rate of 115200, without losing a single reading byte. The first code is the simplest reading code, it will display a sin…

---

<div class="post-metadata">

**Author:** ![sableraph](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/sableraph/32/251_2.png) [@sableraph](https://discourse.processing.org/u/sableraph)\
**Post date:** [December 11, 2022, 7:33pm UTC](https://discourse.processing.org/t/capturing-game-controller-button-events-in-processing-for-android/40073/6 "2022-12-11T19:33:58Z")

</div>

Note: I have posted [an updated version of this question](https://stackoverflow.com/questions/74763835/handle-inputs-from-a-gamepad-with-processing-for-android) on Stackoverflow.

---

<div class="post-metadata">

**Author:** ![sableraph](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/sableraph/32/251_2.png) [@sableraph](https://discourse.processing.org/u/sableraph)\
**Post date:** [December 12, 2022, 1:24pm UTC](https://discourse.processing.org/t/capturing-game-controller-button-events-in-processing-for-android/40073/7 "2022-12-12T13:24:33Z")

</div>

For Reference I’ll put the [response](https://stackoverflow.com/questions/74763835/handle-inputs-from-a-gamepad-with-processing-for-android#comment131953187_74763835) I got from StackOverflow user Morrison Chang:

> [@Morrison Chang](#):
>
> Seems similar to an [older closed issue request](https://github.com/processing/processing-android/issues/86). I took a quick look at [PApplet.java](https://github.com/processing/processing-android/blob/master/core/src/processing/core/PApplet.java) which is probably where some/any changes would be. You’ll have to fork the repo for customization. Given that gamepads could have multiple buttons pressed at the same time I’m not sure if they fit in the existing event framework: [github.com/processing/processing-android/tree/master/core/src/…](https://github.com/processing/processing-android/tree/master/core/src/processing/event) Good luck.

So it looks like there is no way around this that don’t involve customising the core of Processing for Android. Based on the 2014 [issue mentioned above](https://github.com/processing/processing-android/issues/86), It looks like Andrés Colubri (who maintains Processing for Android) was open to a pull request adding gamepad support:

> [@Andrés Colubri](#):
>
> We have limited resources to implement and support additional functionality in the android mode at this moment. Pull requests are welcome though, and I will be very happy to discuss and eventually incorporate a PR extending the handling of input events.

I’ll see if I can wrap my head around the Android framework well enough to make that pull request. Wish me luck (or if you are able to help, please let me know!)