Inserting your sketch in a Jetpack Compose Screen

Processing for Android
1

Hi, I’m working on a small app for Android that started as a processing for Android sketch. At some point I wanted to add some UI but I felt stuck with the current offering of the processing GUI libraries (Which is a subject I would love to talk about with interested people).

So I exported my sketch as an Android Studio Project and copy pasted the generated PApplet in a new Android Studio project. Now Google promotes this new Jetpack Compose framework to do GUI, which is not exactly the same technology as the View/Fragment system used by Processing Android. But I wanted to mix compose GUI and my sketch underneath it.

I’m not experienced in Android Development so I struggled a bit but I found a solution that I will describe here.
All the new code is in kotlin.

  1. Your activity must extends AppCompatActivity
class MainActivity : AppCompatActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContent {
            MainMenu(activity = this)
        }
    }

Here my main activity will show the sketch, so it’s an AppCompatActivity. You can also see that my MainMenu composable function receives this as an argument, we’ll get back on this later.

  1. Using Box to overlay GUI and Sketch

Box (Compose layout basics  |  Jetpack Compose  |  Android Developers) seems to be the way to make things overlap in Jetpack Compose. So I called my SketchView composable inside a Box and added other UI elements after it. Notice that we pass the activity to the SketchView composable.

@Composable
fun MainMenu(activity: AppCompatActivity) {
    MyTheme {
        // A surface container using the 'background' color from the theme
        Surface(modifier = Modifier.fillMaxSize(), color = MaterialTheme.colorScheme.background)) {
                Box {
                    SketchView(activity)
                    // Custom UI
                    Column (
                        horizontalAlignment = Alignment.CenterHorizontally,
                        verticalArrangement = Arrangement.Center
                    ) {
                       // ....
                    }
                }

            }
    }
}
  1. Creating the SketchView composable

The official doc for interoperability is a bit scarce and does not talk about view or fragments but I found this stackoverflow answer that helped me insert a view without using XML inflation or whatever. We needed to pass the activity down for the call to frag.setView().

@Composable
fun SketchView(activity : AppCompatActivity) {
    AndroidView(
        modifier = Modifier.fillMaxSize(), // Occupy the max size in the Compose UI tree
        factory = cb@{ context ->
            val view = FragmentContainerView(context)
            view.id = ViewCompat.generateViewId()
            val sketch = MySketch()
            val frag = PFragment(sketch)
            frag.setView(view,activity)
            return@cb view
        }
    )
}

A last note on theses snippet :

In my case I needed the sketch to be an animated background for my menu so I did not research how to trigger things inside the sketch from the Compose UI. I’m only giving arguments to the sketch through the sketch constructor (not shown here). I guess it could be accomplished by adding methods to the sketch and keeping a reference to the sketch in the activity where you’ll call those methods with your onclick callbacks.

Cheers ++
(I had more links to documentations but as a new user I’m limited in the number of links I can add).