Movie pause and loop triggered by amplitude instead of mousePressed

blindschleiche · July 17, 2023, 11:32am

the idea is, that the movie should show in a paused situation. when there is some noise recognised by the microphone, the movie should play and a red line shows up in front of the movie. as soon as there is no more noise recognised, the movie should pause. obviousley i am doing something wrong, but i don’t have any idea, why the moviepicture stucks but the sound of the movie works without getting stuck and why the moviepicture does not start where it was stopped but jumps to somewhere.
here is my code - can anybody help me?

import processing.sound.*;
Amplitude amp;
AudioIn in;
float angleRotate = 0.0;

import processing.video.*;
Movie myMovie;

void setup() {
  size(1080, 1080);
  frameRate(30);
  //mikrofon aktivieren 
  amp = new Amplitude(this);
  in = new AudioIn(this, 0);
  in.start();
  amp.input(in);
  
  myMovie = new Movie(this, "kurzkurzversion.mov");
  myMovie.loop();
   myMovie.pause();
}

void draw() {
  pushMatrix();
  translate(width/2, height/2);
  rotate(radians(angleRotate));
  stroke(255,0,100,200);//linienfarbe
  line(0, 0, width, 0);
  popMatrix();
   
  angleRotate += 0.5; //rotationsgeschwindigkeit
  
  if (amp.analyze() < 0.02){

    background(0,0,20,0);
   image(myMovie, width/2, height/2);  
     myMovie.pause();

  }
  
   if (amp.analyze() >= 0.02){
    myMovie.loop();  
  }
      
      println(amp.analyze());
}

void movieEvent(Movie m) {
  m.read();
}

mnse · July 18, 2023, 5:24am

Hi @blindschleiche, (witziger Name )

You mean s.th like this ?

Cheers
— mnse

import processing.video.*;
import processing.sound.*;

Movie myMovie;
Amplitude amp;
AudioIn in;

float angleRotate = 0.0;

void setup() {
  size(800, 800);
  frameRate(30);
  //mikrofon aktivieren
  amp = new Amplitude(this);
  in = new AudioIn(this, 0);
  in.start();
  amp.input(in);

  myMovie = new Movie(this, "kurzkurzversion.mov");
  myMovie.loop();
  myMovie.pause();
}

void draw() {
  background(0, 0, 20);
  if (myMovie.isLooping() && amp.analyze() > 0.02) {
    myMovie.pause();
  } else if (myMovie.isPaused() && amp.analyze() <= 0.02) {
    myMovie.loop();
    angleRotate=0;
  }

  if (!myMovie.isPaused()) {
    tint(255);
    image(myMovie, 0, 0, width, height);
    translate(width/2, height/2);
    stroke(255, 0, 100, 200);//linienfarbe
    for (float f = 0; f < angleRotate; f += 0.5) {
      pushMatrix();
      rotate(radians(f));
      line(0, 0, width, 0);
      popMatrix();
    }
    angleRotate = (angleRotate+0.5) % 360;
  } else {
    tint(255, 128);
    image(myMovie, 0, 0, width, height);
  }
  println(amp.analyze());
}

void movieEvent(Movie m) {
  m.read();
}

blindschleiche · July 18, 2023, 6:37pm

hi mnse (geheimnisvoller name ;o))
thank you so much for your script - it is close to what i try to do. i need it the opposite around. the movie starts, when there is some noise and stops when it is quiet.
and the red line should go over the movie when it is playing and has to disapear, when the movie stops. i changed your script so that it plays the right way around - but i could not find out, how i have to do it, that the red line goes over the movie - maybe you also know how i could do that?

import processing.video.*;
import processing.sound.*;

Movie myMovie;
Amplitude amp;
AudioIn in;

float angleRotate = 0.0;

void setup() {
  size(600, 600);
  //mikrofon aktivieren
  amp = new Amplitude(this);
  in = new AudioIn(this, 0);
  in.start();
  amp.input(in);
  
  myMovie = new Movie(this, "kurzkurzversion.mov");
  myMovie.loop();
  myMovie.pause();
}

void draw() {
    
  if (myMovie.isPaused() && amp.analyze() > 0.02) {
    myMovie.loop();
       background(0,0,20,0);
 
  } else if (myMovie.isLooping() && amp.analyze() <= 0.02) {
    myMovie.pause();
    background(0,0,20,0);
  }

  if (!myMovie.isPaused()) {
    image(myMovie, width/4, height/4, width/2, height/2);
   
    translate(width/2, height/2);
    stroke(255, 0, 100, 200);//linienfarbe
      pushMatrix();
     rotate(radians(angleRotate));    
      line(0, 0, width, 0);
      popMatrix();

  } else {
     image(myMovie, width/4, height/4, width/2, height/2);
  }
   angleRotate += 0.5; //rotationsgeschwindigkeit
  
  println(amp.analyze());
}

void movieEvent(Movie m) {
  m.read();
}

mnse · July 19, 2023, 6:03am

Hi @blindschleiche,

try this…
Please note that I’ve added some kind of threshold to prevent a too fast switch from on to off…
Also not understand if you want just a single line to rotate or if you want to draw s.th. implemented below. if you want just a single red line to rotate remove the for loop…

Hope that helps!

Cheers
— mnse

import processing.video.*;
import processing.sound.*;

Movie myMovie;
Amplitude amp;
AudioIn in;

float angleRotate = 0.0;
float startRotate = 0.0;
int   threshold   = 0;

void setup() {
  size(600, 600);
  //mikrofon aktivieren
  amp = new Amplitude(this);
  in = new AudioIn(this, 0);
  in.start();
  amp.input(in);

  myMovie = new Movie(this, "kurzkurzversion.mov");
  myMovie.loop();
  myMovie.pause();
}

void draw() {
  background(0, 0, 20, 0);
  if (myMovie.isPaused() && isAudio()) {
    myMovie.loop();
    startRotate = angleRotate;
    println("on");
  } else if (!myMovie.isPaused() && !isAudio()) {
    myMovie.pause();
    println("off");
  }

  image(myMovie, width/4, height/4, width/2, height/2);
  if (!myMovie.isPaused()) {
    translate(width/2, height/2);
    stroke(255, 0, 100, 200);//linienfarbe
    for (float f=startRotate; f < angleRotate; f++) {
      pushMatrix();
      rotate(radians(f));
      line(0, 0, width, 0);
      popMatrix();
    }
  }
  angleRotate += 0.5; //rotationsgeschwindigkeit
}

void movieEvent(Movie m) {
  m.read();
}

// introduce some threshold to smooth check if noise or not
// prevent of flickering
boolean isAudio() {
  if (amp.analyze() > 0.02) {
    threshold = constrain(threshold+1, 0, 30);
  } else {
    threshold = constrain(threshold-1, 0, 30);
  }
  //println(threshold + ":" + nf(amp.analyze(),0,4));
  return threshold > 0;
}

blindschleiche · July 20, 2023, 5:03pm

hi mnse
thank you so much - now it does what it should - there ist just one problem left - when there is nois over a longer time, the moving of the line ist getting slower and slower - and the computers fan starts to make noise - which ist not so good for the noiseeffect - that means, it doesn’t stop anymore, because of the fan… it must be something, that is in a buffer…
maybe it could be possible to solve the problem by slowly fading out the lines, which are already drawn?

mnse · July 21, 2023, 6:27pm

Hi @blindschleiche,

Sorry! Didn’t notice you wrote back…
I’m not at my box but looking at the code the issue is obvious. If the sound is running a longer time the for loop for the line draws more and more lines because the delta between startRotate and angleRotation is getting bigger and bigger. To solve this you need to constrain the value in a proper bounds…
If the delta is bigger than 360 you should reset it, resp update the startAngle.or use a proper modulo…

Hope that helps…

Cheers
— mnse

PS: If you stuck I can change it when I’m at my box. But could be maybe Monday…

mnse · July 22, 2023, 7:09am

Hi @blindschleiche,

there are many solutions for this, here just two examples.
Just put one of this right after the for loop…

// version 1 start over rotation
if (angleRotate - startRotate >= 360.) {
  startRotate = angleRotate;     
}

// version 2 max to a single turn
angleRotate = constrain(angleRotate,startRotate,startRotate + 360 - 0.5);

Cheers
— mnse

blindschleiche · July 22, 2023, 4:03pm

hy mnse
it helps a little bit, but i’m not sure if i put it in the right place. it still slows down and slows down and when the 360 degrees are done, it gets stuck.
maybe i should let the movie finish playing before it starts again when a new noise impulse comes. but that doesn’t work either because i don’t know how to rebuild the loop so that the movie runs through till it’s end and then starts again. i replaced loop with play. but then it doesn’t start again at the next noise. but i couldn’t get it to work with the change threshold either because it takes too long to count down. if it’s loud too many times in a row, then it doesn’t work at all.
the solution seems to be in this loop, but i can’t figure out how to…

  if (myMovie.isPaused() && isAudio()) {
    myMovie.loop();
    startRotate = angleRotate;
    println("on");
  } else if (!myMovie.isPaused() && !isAudio()) {
    myMovie.pause();
    println("off");
  }

mnse · July 23, 2023, 7:28am

Hi @blindschleiche,

Need to see your complete code to analyse the issue…

Cheers
— mnse

blindschleiche · July 23, 2023, 12:48pm

hi @mnse
here is the complete code.

import processing.video.*;
import processing.sound.*;

Movie myMovie;
Amplitude amp;
AudioIn in;

PFont s;
float angleRotate = 0.0;
float startRotate = 0.0;
int   threshold   = 0;

void setup() {
  //size(600, 600);
   size(900, 900);
  //fullScreen();
  
  //mikrofon aktivieren
  amp = new Amplitude(this);
  in = new AudioIn(this, 0);
  in.start();
  amp.input(in);
  
     // Create the font from the .ttf file in the data folder
    s = createFont("SourceCodePro-Regular.ttf", 125);
    textFont(s);
    textAlign(CENTER, CENTER);


    myMovie = new Movie(this, "auge_kurz.mov");
    
  myMovie.pause();
}

void draw() {
  background(0, 0, 20, 0);
  textSize(125);
   fill(100,150,255,30); //schriftfarbe und transparenz - hellblau
  
  if (myMovie.isPaused() && isAudio()) {
    myMovie.loop();
    startRotate = angleRotate;
    println("on");
  } else if (!myMovie.isPaused() && !isAudio()) {
    myMovie.pause();
    println("off");
  }

  image(myMovie, 0, 0, 900, 900);
  if (!myMovie.isPaused()) {
    translate(width/2, height/2);
    stroke(255, 0, 100, 200);//linienfarbe
    for (float f=startRotate; f < angleRotate; f++) {
      pushMatrix();
      rotate(radians(f));
      text("WAS WAR SAW", 0, -20);//so ist der rote punkt in der mitte
      line(0, 0, width, 0);
      popMatrix();
    }
    
     angleRotate = constrain(angleRotate,startRotate,startRotate + 360 - 0.5);
  }
  angleRotate += 0.5; //rotationsgeschwindigkeit
}

void movieEvent(Movie m) {
  m.read();
}

// introduce some threshold to smooth check if noise or not
// prevent of flickering
boolean isAudio() {
  //if (amp.analyze() > 0.02) {
    if (amp.analyze() > 0.012) {
    threshold = constrain(threshold+1, 0, 30);
  } else {
    threshold = constrain(threshold-1, 0, 30);
  }
  //println(threshold + ":" + nf(amp.analyze(),0,4));
  return threshold > 0;
}

and i wonder, if there is a possibility to play two movies - one after the other?
cheers

mnse · July 24, 2023, 6:35am

Hi @blindschleiche,

I would like to leave the creative part to you … you need to work out what’s good for you and what is not…

Cheers
— mnse

Yes! Of course it is possible …

import processing.video.*;
import processing.sound.*;

ArrayList<Movie> myMovies;
int movieIndex = 0;
Movie currentMovie;

Amplitude amp;
AudioIn in;

PFont s;
float angleRotate = 0.0;
int   threshold   = 0;

void setup() {
  size(900, 900);

  //mikrofon aktivieren
  amp = new Amplitude(this);
  in = new AudioIn(this, 0);
  in.start();
  amp.input(in);

  // Create the font from the .ttf file in the data folder
  s = createFont("SourceCodePro-Regular.ttf", 125);
  textFont(s);
  textSize(125);
  textAlign(CENTER, CENTER);

  myMovies = new ArrayList<>();
  myMovies.add(new Movie(this, "movie1.mp4"));
  myMovies.add(new Movie(this, "movie2.mp4"));
  for (Movie m : myMovies)
    m.pause();
  currentMovie = myMovies.get(movieIndex);
}

void draw() {
  // background(0, 0, 20, 0); // makes no sense
  background(0);

  handleMovie();

  if (currentMovie.isPaused() && isAudio()) {
    currentMovie.play();
    println("on");
  } else if (!currentMovie.isPaused() && !isAudio()) {
    currentMovie.pause();
    println("off");
  }

  image(currentMovie, 0, 0, width, height);
  if (currentMovie.isPlaying()) {
    doCreativePart();
  }
  angleRotate += 0.5; //rotationsgeschwindigkeit
}

void movieEvent(Movie m) {
  m.read();
}

void handleMovie() {
  if (currentMovie.time() >= currentMovie.duration()) {
    currentMovie.pause();
    currentMovie.jump(0);
    //println("movie switch from: " + movieIndex);
    movieIndex = (movieIndex + 1) % myMovies.size();
    //println("movie switch to: " + movieIndex);
    currentMovie = myMovies.get(movieIndex);
    currentMovie.pause();
  }
  //surface.setTitle("currentMovie="+movieIndex+",isPaused="+currentMovie.isPaused()+",ti="+currentMovie.time()+" of " + currentMovie.duration());
}

void doCreativePart() {
// do some fancy stuff here 
/*
  pushMatrix();
  translate(width/2, height/2);
  rotate(radians(angleRotate));
  fill(100, 150, 255, 30); //schriftfarbe und transparenz - hellblau
  stroke(255, 0, 100, 200);//linienfarbe
  text("WAS WAR SAW", 0, -20);//so ist der rote punkt in der mitte
  line(-width/2, 0, width/2, 0);
  popMatrix();
*/
}

// introduce some threshold to smooth check if noise or not
// prevent of flickering
boolean isAudio() {
  //if (amp.analyze() > 0.02) {
  if (amp.analyze() > 0.012) {
    threshold = constrain(threshold+1, 0, 30);
  } else {
    threshold = constrain(threshold-1, 0, 30);
  }
  //println(threshold + ":" + nf(amp.analyze(),0,4));
  return threshold > 0;
}```

blindschleiche · July 24, 2023, 10:24pm

hi @mnse
thank you so much for your script to play two movies. and it is good to see the fancy stuff in a seperate funktion. much better overview and easy for tests to uncomment.
i tried the script out with two movies with name 1movie.mov and 2movie.mov. unfortunately, it did not play the fist one for the whole size but only for the probably defined section in the threshold = constrain(threshold+1, 0, 30);
after several noises the first movie finally got to the end but got stuck there. i mad a new noise again but whether the first nor the second movie started. nothing happend anymore. i have no idea why. could you please help me again?
cheers

mnse · July 25, 2023, 7:01am

Hi @blindschleiche,

sometimes this failed because of several possible issues (ie. VFR, etc)

if (currentMovie.time() >= currentMovie.duration())

You can try to fix it by many methods, ie introduce a stuck counter if not isPaused and time index not changing. Assuming the video is not corrupt, it stucks only a view milliseconds before real end.

You can try this first…

if (floor(currentMovie.time()*10.) >= floor(currentMovie.duration()*10.))

Cheers
— mnse

blindschleiche · July 26, 2023, 3:53pm

hi @mnse
did i understand it right - changing the if command within the function handleMovie?
i did so - but nothing different happend - its the same as before…
the mistake is probabely somewhere else?
cheers

blindschleiche · July 26, 2023, 11:46pm

hi @mnse
now i found one reason, why it didn’t work - when the movie is at the end there comes a text in the console - Element: [avdec_prores1]: No valid frames decoded before end of stream
i made the movies with AfterEffects. I had to export it as a mov because mp4 doesn’t work with the loop in processing. when i test the movie in processing with the duration() it says, that it is 10 seconds. maybe it is the way i exported the movies?
the other problem, that the movies don’t play through still exists.
i would like to play the first movie when there is noise. all noises while the movie is playing should be ingnored. when the movie is at the end, the next noise should start the second movie, when the next noise is comming, the first movie should play again - and so on…
cheers

mnse · July 27, 2023, 8:56am

Hi @blindschleiche,

didn’t understand why you can’t work with mp4 ?
I’ve tested my code with valid mp4 movies and it worked Try to use valid video files, so it works without any issues to be sure it does what you want. Afterwards you can handle special cases like broken videos, etc.
…

That’s a good exercise for you to get further with your programming experiences…
Think about the logic and try to implement it, if you are close and have specific issues, come back and open a new topic for the specific issue/question …

Cheers
— mnse

blindschleiche · July 29, 2023, 3:55pm

hi @mnse
the problem is, that there are so many different ways to export a mp4 or a mov. my mp4 had one picture to much and i had to convert it with handbrake until it reacted the way it should. why this is like this, i don’t know. but i read somewhere, that only mov work with loop. it seams, that mp4 with all the corrections works now anyway. so this problem seems to be solved.
the other problem, i don’t really understand, why i am not able to play the movie and when it is done, that i can restart it again. this seems to be impossible unless i let it play with the loop. but then it doesn’t stop at the end… i thought, that this would be simple, but it seems to be not?

mnse · July 29, 2023, 4:54pm

Hi @blindschleiche,

I don’t know. As said I’ve done this with two of my mp4 videos. It runs like a charm, it loops and had no issues like you described. So, I can’t reproduce

If you use play and find a way to detect the end you can use jump to rewind to 0.

Cheers
— mnse

blindschleiche · July 30, 2023, 5:14pm

it is really strange, that i doesn’t work with certain mp4. but this problem is solved. i have now 2 mp4 which work as wished. but as soon as the trigger is the microphone, the second movie stops if there is not noise the whole time. it jumps alsways to the off-position. with the simple script below it works with the mousepressed and released, but not with the microphone.

import processing.video.*;
Movie myMovie;
Movie myMovie2;

void setup() {
  size(1000, 500);
  myMovie = new Movie(this, "test_auge_1080_conv.mp4");
  myMovie2 = new Movie(this, "auge_verzerren_2_conv.mp4");
  myMovie.loop();
  myMovie2.loop();
  myMovie.pause();
  myMovie2.pause();
}
void draw() {
  image(myMovie, 0, 0,500,500);
  image(myMovie2, 200, 0,500,500);
}
void movieEvent(Movie m) {
  m.read();
}
void mousePressed() {
  myMovie.play();
  myMovie2.pause();
}
void mouseReleased() {
  myMovie.pause();
  myMovie2.play();
}

if i could detect a certain moment in the movie would be interesting anyway. the jump just jumps but doesn’t play until a certain moment. is there another possibilitiy to play to a certain point of the movie and then do someting else?

Topic		Replies	Views
Play two movies with the microphone as trigger instead of mousepressed and released Libraries	6	118	August 1, 2023
Sound in Processing Libraries	3	341	October 23, 2018
Sound cuts off, then plays again Libraries homework	1	500	November 23, 2020
Slight pause when reversing video playback speed Libraries	0	220	August 12, 2022
Using a webcam to trigger off sound Libraries	1	587	October 24, 2018

Movie pause and loop triggered by amplitude instead of mousePressed

Related Topics