Processing Script very slow or not running at all

Processing Libraries
1

Hello, I am trying to get my Kinect to control the framerate and opacity of a video file that is playing. The script either runs really slowly or not at all. I tried troubleshooting:

  1. Before using the Kinect, I tried using mouseX and mouseY to control the framerate and opacity and found that the script was running really slowly, causing the videos to play really jittery.

  2. After connecting the Kinect and and modifying the script to work with it, I found that it stopped working altogether. The image captured from the Kinect does not appear on the screen and even the video files refuse to play when I activate them.

  3. I tried running it again using mouseX and mouseY while drawing the Kinect captures. It was able to do this but runs really slowly with the videos playing really jittery again.

OS: Windows 10 Pro
Processor: Intel® Core™ i7-4870HQ CPU @ 2.50GHz
Installed RAM: 16.0 GB
System type: 64-bit operating system, x64-based processor
Processing v3.4

I’ve pasted the script below. Is there a way I can fix it so it runs with less latency?

Thanks very much for your help.

import org.openkinect.processing.*;
Kinect2 kinect2;

import processing.video.*;
Movie morning;
Movie afternoon;
Movie evening;

float minThresh = 480;
float maxThresh = 830;
PImage img;

PImage base;


void setup() {
  fullScreen();

  base = loadImage("Base.jpg");

  // LOAD PROJECTIONS
  morning = new Movie(this, "0930H.mp4");
  morning.loop();
  morning.read();

  afternoon = new Movie(this, "1330H.mp4");
  afternoon.loop();
  afternoon.read();

  evening = new Movie(this, "1730H.mp4");
  evening.loop();
  evening.read();


  // INITIALISE KINECT
  kinect2 = new Kinect2(this);
  kinect2.initDepth();
  kinect2.initDevice();
  img = createImage(kinect2.depthWidth, kinect2.depthHeight, RGB);

  background(0);

  imageMode(CENTER);
}



void draw() {
  //GET PIXELS WITHIN DEPTH THRESHOLD
  img.loadPixels();
  int[] depth = kinect2.getRawDepth();
  float sumX = 0;
  float sumY = 0;
  float totalPixels = 0;
  for (int x = 0; x < kinect2.depthWidth; x++) {
    for (int y = 0; y < kinect2.depthHeight; y++) {
      int offset = x + y * kinect2.depthWidth;
      int d = depth[offset];
      if (d > minThresh && d < maxThresh) {
        img.pixels[offset] = color(248, 152, 56);
        sumX += x;
        sumY += y;
        totalPixels++;
      } else {
        img.pixels[offset] = color(0);
      }
    }
  }
  img.updatePixels();

  // CALCULATE AVERAGE POINT OF HAND
  float avgX = sumX / totalPixels;
  float avgY = sumY / totalPixels;
  float floatX = map(avgX, 0, 512, 336, 1104);
  float floatY = map(avgY, 0, 424, 450, 900);
  int intX = int(floatX);
  int intY = int(floatY);


  //FRAMERATE MANIPULATION
  //float speed = map(mouseX, 0, width, 1, 100); //controlled by mouseX
  float speed = map(floatX, 0, 2560, 1, 100); //controlled by Kinect
  frameRate(speed);

  //OPACITY MANIPULATION
  //float opacity = map(mouseY, 0, height, 0, 255); //controlled by mouseY
  float opacity = map(totalPixels, 0, (img.width*img.height), 0, 255); //controlled by Kinect
  tint(255,255);
  image(base, width/2, (height/2+220), base.width*2.6, base.height*2.6);

  //PLAY MORNING VIDEO
  if (key == '1') {
    morning.read();
    tint(255, opacity);
    image(morning, width/2, (height/2+220), morning.width*2.6, morning.height*2.6);
  }

  //PLAY AFTERNOON VIDEO
  if (key == '2') {
    afternoon.read();
    tint(255, opacity);
    image(afternoon, width/2, (height/2+220), morning.width*2.6, morning.height*2.6);
  }

  //PLAY EVENING VIDEO
  if (key == '3') {
    evening.read();
    tint(255, opacity);
    image(evening, width/2, (height/2+220), morning.width*2.6, morning.height*2.6);
  }
  
  //DRAW KINECT CAPTURE
  stroke(248,152,56);
  noFill();
  image(img, width/2, (106+20), 256, 212);
  rect(1152,20,256,212);
}
1 Like
2

If you are still working on this: Can you provide the non-kinect version, since it sounds like that is the core that needs to be optimized first (and that is more accessible to more forum members).