Making an object react to sound amplitude of live audio feed using minim library or sound library

Hello!
I’m currently working on something i feel should be pretty simple but i’m really struggling to do it :sweat:
Basically, i have a png image and i would like to make it move position on the canvas by reacting to live sound from the device’s mic. The effect i would like to create is an oscillating iris moving within the eye socket. I have managed to get some code working and it kind of looks like what im aiming for but so far it only reacts to sound no matter how loud or quiet it is.

So what i’m trying to do is have the iris move positions more erratically depending on the amplitude of sound rather than just when it detects sound. so like… it would go: “up, left, right, down” rather than just: “down, right” like what i have right now…

import ddf.minim.*;

Minim minim;
AudioInput in;

int reactLast;
int reactBetween = 10;
float soundIntensity = 0.05;

PImage constraint;
PImage eyeris;
PImage background;

void setup() {
  size(620, 310);
  strokeWeight(3);
  smooth();
  noFill();

  constraint = loadImage ("constraint.png");
  eyeris = loadImage ("eyeris.png");
  background = loadImage("background.png");
  

  minim = new Minim(this);
  in = minim.getLineIn(Minim.MONO, 64);

  reactLast = millis();
}

void draw() {

  image(background, 0, 0);
  
 float sound = 0;  
  for (int i=0; i<in.bufferSize()-1; i++) {
    
    sound += in.left.get(i);
     
  }
  if (abs(sound)>soundIntensity && millis()-reactLast>reactBetween) {
    reactLast = millis();
    
    image(eyeris, random(-soundIntensity, soundIntensity+5 ), random(-soundIntensity, soundIntensity+5 ));
    
  } else {
    image(eyeris, 0, 0);
  }
 

  image(constraint, 0, 0);
  
}

void stop() {
  in.close();
  minim.stop();
  super.stop();
}

i got the code from here

this is probably where i have to make the modifs

void draw() {

  image(background, 0, 0);
   
   //i feel like i should try mapping the soundIntensity to make it a value 
   //that can control the position of the iris but
   // i don't know how exactly or if it's even necessary...
   //float amplitude = map(soundIntensity, soundIntensity - 100, 0, 100, 0); 
   
  float sound = 0;  
  for (int i=0; i<in.bufferSize()-1; i++) {
    
    sound += in.left.get(i);
     
  }
  if (abs(sound)>soundIntensity && millis()-reactLast>reactBetween) {
    reactLast = millis();
    
    image(eyeris, random(-soundIntensity, soundIntensity+5 ), random(-soundIntensity, soundIntensity+5 ));
    
  } else {
    image(eyeris, 0, 0);
  }
 

  image(constraint, 0, 0);
  
}

i’ve tried looking at different codes and resources but so far what i’ve found seems pretty linear and/or i just can’t understand how to implement them in the code…
this site should be helpful for understanding the minim library but again i’m confused on how and where to integrate the lines of code given…
any help would be greatly appreciated :bowing_woman:

ps: would it be possible to use the same code on video files with minimal modification?

i also wish i could make a codepen to make it easier for anyone to test it out but i’m sorry i don’t know how :frowning: so if that’s okay i’ll leave the image files down here if necessary

background
background

constraint
constraint

eyeris
eyeris

1 Like

Hello,

I am listing some of the related Processing sound library resources:

Check out the examples that come with Processing.
These are in the menu under File > Examples…> :

You may already have these. It will be helpful to others visiting the topic.

:)

oh nice! i didn’t know there were such examples from processing that could be used, thank you!

1 Like