Visuals AND music with Processing?


#1

Hi all. First time poster here. I teach MS computer science and am looking for a platform my classes can use to combine visuals and music to make multimedia performances, preferably live code. Since it’s middle school, it has to be not too onerous in terms of syntax, etc. I am already using Sonic Pi for live music coding, and I know that I can output OSC messages from SP into Processing. Just not exactly sure what to do on the Processing end.

I’m hoping some awesome educator has played with this a bit and can help me out. Even links to other stuff is welcome.

TIA!
Bob Irving
Porter-Gaud School
Charleston, SC
@birv2


#2

Hi CrouchingPython,

You can probably have a look at praxis live for live coding performance.

It is made by @neilcsmith here on the forum.


#3

Thanks! I’ve dl’ed PraxisLive, am following Neil and Praxis on Twitter and am researching starting points for learning it. Does it connect with Processing?


#4

It is made using processing but I’ll let @neilcsmith tell you more :slight_smile:


#5

I’ve played a little bit (had a spare period this morning). It’s really amazing!

Having said that, it may not work for my situation exactly. I’ll keep exploring. I want my students to be able to write code. I’m sure they can handle Sonic Pi and Processing. Not so sure about Java! And using the gui means you don’t have to do text-based coding. Which is not something I’m looking for.

Or maybe I’m missing something?


#6

Well, Processing is Java! :wink: PraxisLIVE is a code editor with nodes, built on top of Apache NetBeans, so you’ve got a much more powerful code editor than PDE and support for live coding. You can mostly ignore the node interface if you want. Connect a video:gl:p2d or video:gl:p3d node to the video output, right click and edit code, and you’ve effectively got a Processing sketch that updates every time you hit save.

There’s also built in OSC support that you can easily map to any field you expose.

Thanks @jb4x :smile:


#7

Yep, I know that Processing is Java, but maybe kinda Java Lite? Not too OO-crazy, which is where most middle schoolers would drop out… They can handle Processing.

What you say about hooking up Processing is what intrigues me, but we’ll chat more after your holiday. Thanks again!


#8

AFAIK there’s a REPL mode for Processing, but it’s Java-only, so that can be sufficient for live coding. As for something not too OOP-heavy, processing by itself isn’t like that! All of Java’s OOP elements are handled by the internal parser, but if you really want something easier, check out Python mode.


#9

It is up to you to use OOP concept or not. We have a saying in french: “Qui peut le plus, peut le moins” roughly meaning “He who can do the most can do the least”. And this way, student who feel comfy with progrmming will be able to play around with OOP concept and the other one won’t :slight_smile:


#10

I’ve never tried Python mode, but I’m definitely interested. Is that a library that I import?


#11

It’s not a library, it’s a mode. You can download modes by clicking on the mode selector (dropdown bow in the upper right corner) and “Install modes”. Here you can install Python mode as well as many others.


#12

Looks pretty amazing. I’m a huge Python fan. However, I’m not seeing any audio stuff. Can it take MIDI or OSC and do something visually with that data?


#13

The PraxisCORE runtime is in some ways simpler, but a lot more powerful, than hacks like the REPL mode or anything using HotSwap. It’s also a runtime bringing elements of Smalltalk and Erlang to Java. Nothing wrong with real OOP, as opposed to the typical C++ / Java version! :wink: It’s sometimes said that the most OOP language on the JVM is Clojure. Which does bring to mind Quil for an alternative approach to live coding Processing.


#14

Processing.py is extremely cool – and one of the cool things about it is that, in addition to importing pure Python libraries, many of the Processing(Java) libraries can simply be imported and used. This is because it uses Jython and runs on JVM. So you could just import the OSC library and see how it works. Or, for audio, import minim – there are examples of working with sound in the Processing.py example set that installs into PDE when you install the mode.

One downside is that it is Python 2.7 – so your students wouldn’t be learning Python 3. Another is that, due to running on JVM in emulation, only pure Python libraries will work – anything relying on CPython won’t.


#15

Super cool stuff here!

I’m using Sonic Pi and doing live coding with that. It can output OSC messages. So what I’m trying to do it capture those messages in Processing and generate visuals. I’m happy to start super simple and have ANYTHING appear in response to beat, volume, pitch, etc. Once I get that down, I can start to build on it.

So rather than grab an mp3, I’m trying to get live messages. Can anyone point me in the right direction for this? TIA!!

Bob


#16

@CrouchingPython I can certainly tell you how to connect OSC messages to a Processing node inside PraxisLIVE if that’s the way you want to do it?

What’s the format of the OSC messages from Sonic Pi, or is that up to you?


#17

I’d love to do it in straight Processing, if I could. AFAIK, SonicPi sends messages in “regular” OSC, but maybe I shouldn’t talk about what I really don’t know for sure. Noob alert. Thanks!


#18

Up to you. But PraxisLIVE makes this easy because it’s basically supported out of the box - annotate and map any field you want to control.

OSC messages have an address, like a url or path. Can Sonic Pi send any address, or does it send predefined ones?


#19

Thanks again for the response. I do want to use Processing (or something like it) because my purpose is teaching students to code. PraxisLive is amazing, but it doesn’t have the code aspect that I need for middle school students. Processing (yes, I know it’s Java) is still way more accessible than straight Java for the great majority of 13-year-olds.

As for how Sonic Pi sends OSC messages, here’s a link to the docs: https://github.com/samaaron/sonic-pi/blob/master/etc/doc/tutorial/12.2-Sending-OSC.md

Hope that helps!
Bob


#20

er, yes it does! It’s still primarily a code-based IDE. I’m currently teaching a course on creative coding with Processing using it.

I’ll have a look through those docs, and see if I can tell, or if not I’ll ask the developer.