Hello, I am trying to get my Kinect to control the framerate and opacity of a video file that is playing. The script either runs really slowly or not at all. I tried troubleshooting:
-
Before using the Kinect, I tried using mouseX and mouseY to control the framerate and opacity and found that the script was running really slowly, causing the videos to play really jittery.
-
After connecting the Kinect and and modifying the script to work with it, I found that it stopped working altogether. The image captured from the Kinect does not appear on the screen and even the video files refuse to play when I activate them.
-
I tried running it again using mouseX and mouseY while drawing the Kinect captures. It was able to do this but runs really slowly with the videos playing really jittery again.
OS: Windows 10 Pro
Processor: Intel® Core™ i7-4870HQ CPU @ 2.50GHz
Installed RAM: 16.0 GB
System type: 64-bit operating system, x64-based processor
Processing v3.4
I’ve pasted the script below. Is there a way I can fix it so it runs with less latency?
Thanks very much for your help.
import org.openkinect.processing.*;
Kinect2 kinect2;
import processing.video.*;
Movie morning;
Movie afternoon;
Movie evening;
float minThresh = 480;
float maxThresh = 830;
PImage img;
PImage base;
void setup() {
fullScreen();
base = loadImage("Base.jpg");
// LOAD PROJECTIONS
morning = new Movie(this, "0930H.mp4");
morning.loop();
morning.read();
afternoon = new Movie(this, "1330H.mp4");
afternoon.loop();
afternoon.read();
evening = new Movie(this, "1730H.mp4");
evening.loop();
evening.read();
// INITIALISE KINECT
kinect2 = new Kinect2(this);
kinect2.initDepth();
kinect2.initDevice();
img = createImage(kinect2.depthWidth, kinect2.depthHeight, RGB);
background(0);
imageMode(CENTER);
}
void draw() {
//GET PIXELS WITHIN DEPTH THRESHOLD
img.loadPixels();
int[] depth = kinect2.getRawDepth();
float sumX = 0;
float sumY = 0;
float totalPixels = 0;
for (int x = 0; x < kinect2.depthWidth; x++) {
for (int y = 0; y < kinect2.depthHeight; y++) {
int offset = x + y * kinect2.depthWidth;
int d = depth[offset];
if (d > minThresh && d < maxThresh) {
img.pixels[offset] = color(248, 152, 56);
sumX += x;
sumY += y;
totalPixels++;
} else {
img.pixels[offset] = color(0);
}
}
}
img.updatePixels();
// CALCULATE AVERAGE POINT OF HAND
float avgX = sumX / totalPixels;
float avgY = sumY / totalPixels;
float floatX = map(avgX, 0, 512, 336, 1104);
float floatY = map(avgY, 0, 424, 450, 900);
int intX = int(floatX);
int intY = int(floatY);
//FRAMERATE MANIPULATION
//float speed = map(mouseX, 0, width, 1, 100); //controlled by mouseX
float speed = map(floatX, 0, 2560, 1, 100); //controlled by Kinect
frameRate(speed);
//OPACITY MANIPULATION
//float opacity = map(mouseY, 0, height, 0, 255); //controlled by mouseY
float opacity = map(totalPixels, 0, (img.width*img.height), 0, 255); //controlled by Kinect
tint(255,255);
image(base, width/2, (height/2+220), base.width*2.6, base.height*2.6);
//PLAY MORNING VIDEO
if (key == '1') {
morning.read();
tint(255, opacity);
image(morning, width/2, (height/2+220), morning.width*2.6, morning.height*2.6);
}
//PLAY AFTERNOON VIDEO
if (key == '2') {
afternoon.read();
tint(255, opacity);
image(afternoon, width/2, (height/2+220), morning.width*2.6, morning.height*2.6);
}
//PLAY EVENING VIDEO
if (key == '3') {
evening.read();
tint(255, opacity);
image(evening, width/2, (height/2+220), morning.width*2.6, morning.height*2.6);
}
//DRAW KINECT CAPTURE
stroke(248,152,56);
noFill();
image(img, width/2, (106+20), 256, 212);
rect(1152,20,256,212);
}