Help needed to code videos play with processing by receiving data from Arduino

Processing Electronics (Arduino, etc.)
1

Hello everyone!

I am an artist and I am trying to make an interactive artwork using Arduino and Processing. I am really new at all this,.I know very little about programming but I have to complete this artwork as it is the final task of an intensive program I took part in, which introduced me to interactive arts using Arduino.

So, for this artwork, I am using an ultrasonic distance sensor to capture the existence of a person in front of it, an Arduino Uno, and of course Processing. Firstly, I want to play a video that only has sound with Processing. When the sensor detects a person at a maximum distance of 50cm, then the first video stops, and a second video starts playing without looping. Right after the second video ends, then the first video starts looping again until the sensor detects another person standing in front.

So far I have found a code for Arduino and written a code in Processing. The problem is that the first video only plays a few seconds and then the second video starts playing without interacting with the sensor. I have checked the sensor and it functions properly. I have also tested the communication between Arduino and Processing with a premade code and it also works. But I still haven’t found what I do wrong. I would really appreciate any given help!

Thank you in advance!

This is the code I run with Arduino, which I found online:

#include <NewPing.h>

#define TRIGGER_PIN 10 // Arduino pin tied to trigger pin on the ultrasonic sensor.
#define ECHO_PIN 9 // Arduino pin tied to echo pin on the ultrasonic sensor.
#define MAX_DISTANCE 50 // Maximum distance we want to ping for (in centimeters). Maximum sensor distance is rated at 400-500cm.

NewPing sonar(TRIGGER_PIN, ECHO_PIN, MAX_DISTANCE); // NewPing setup of pins and maximum distance.

void setup() {
Serial.begin(9600); // Open serial monitor at 115200 baud to see ping results.
}

void loop() {
delay(100); // Wait 50ms between pings (about 20 pings/sec). 29ms should be the shortest delay between pings.
//Serial.print("Ping: ");
Serial.write(sonar.ping_cm()); // Send ping, get distance in cm and print result (0 = outside set distance range)
//Serial.println(“cm”);
}

And this is a code that I did myself at processing:

import processing.serial.; //importing serial lib
import processing.video.;

Serial arduino;

int serialIn;
int val = 49; // Data received from the serial port
int THRESHOLD = 50;

Movie sound;
Movie video;

void setup() {

size(500,500);

printArray(Serial.list());
arduino = new Serial(this,Serial.list()[1],9600);

frameRate(25);

sound= new Movie(this,“keys.mp4”);
video= new Movie(this,“cross.mov”);

sound.loop();
}
void movieEvent(Movie m) {
m.read();
}

void draw() {
if(arduino.available()>0)
{
serialIn=arduino.read();
println(serialIn);

if (val <= THRESHOLD) { //If the sensor detect object at less 10cm
  sound.stop();
  video.play();
  image(video,0,0);
 }

}
}

2

hello,

and welcome to the forum!

Great to have you here!

Essentially, you have 2 videos.

You load both in setup(). Good. (Not sure, if mp4 are ok, did you test?)

Now to your problem.

In my understanding you need a variable boolean useImageMovie (or whatever name) that indicates what to play.

This:

should be

if (val <= THRESHOLD) { //If the sensor detect object at less 10cm
  sound.stop();
  video.play();
  useImageMovie= true; 
 }

if(useImageMovie) {
  image(video,0,0);
}
else {
  image(sound,0,0);
}

Because, when you say image(video,0,0); in the first if-clause it will only play during the if (val <= THRESHOLD) and not to its end.

More

when video is over, set useImageMovie to false:

useImageMovie = false;

Declaration

declare boolean useImageMovie = false; before setup()

I hope this helps!

Warm regards,

Chrisir

3

Hello and thanks a lot for your immediate reply!

You said that when the video is over, I should set useImageMovie to false. This means that I should write this line of code at the end of the if, else- clause?

I run the code with 2 mov-type videos but I still get the same result. The first video plays for one second and almost immediately the second video starts playing without placing any object in front of the sensor.

4

No.

remember that draw() runs 60 times per second during the movie plays.

So the execution doesn’t wait in the if-clause but runs on.

Therefore, this line of code must be outside the if-clause.

Does the movie has something like .isPlaying property?

5

Ok so, here’s what my draw() looks like

void draw() {
if(arduino.available()>0)
{
serialIn=arduino.read();
println(serialIn);

if (val <= THRESHOLD) { //If the sensor detect object at less 50cm
  sound.stop();
  video.play();
  //image(video,0,0);
  useImageMovie= true;
}
   if(useImageMovie) {
   image(video,0,0);
   useImageMovie = false;
   }
    else {
     image(sound,0,0);
   }
   useImageMovie= false;
 }

}

but still nothing. The first video stops and the second plays right after without any interaction with the sensor.

I’m sorry I dont understand the question.

6

No.

Do you have sound.play(); in setup()?

Do you have boolean useImageMovie = false; before setup()?
try:




void draw() {
  if (arduino.available()>0) {
    serialIn=arduino.read();
    println(serialIn);

    if (val <= THRESHOLD) { 
      //If the sensor detect object at less 50cm
      sound.stop();
      video.play();
      useImageMovie = true;
    }
  }

  // ------------

  if (useImageMovie) {
    image(video, 0, 0);
  } else {
    image(sound, 0, 0);
  }
}
7

I have sound.loop(); I want this one to loop until an interaction happens.

Yes.

Ok, I’ll try it.
Thanks again!

1 Like
8

I hope the video starts now properly ending the sound.

We still have to find to end the video after it played once though.

in my opinion, the movie doesn’t tell you whether its finished or not according to https://www.processing.org/reference/libraries/video/index.html

Hence, let’s use

float durationVideoMovie; int startMs; before setup()

after (!) video.play(); say
durationVideoMovie = video.duration(); // measured in seconds
startMs=millis(); // startMs is in millis, see reference

in this section

if (useImageMovie) {
    image(video, 0, 0);
  }

say

if (useImageMovie) {
    image(video, 0, 0);
    // detect if video is over
    if((millis()-startMs) / 1000 >= durationVideoMovie ) {
          // Yes, go back to default black video (sound only) 
          useImageMovie=false; 
          video.stop();
          sound.play(); 
    }
  }

not tested…

:wink: