Sharing Image variables between PApplets

[chuchu]

Hello Processing Discourse!

I am trying to select a portion of video from a video capture by dragging a rectangle and then display that to a second window (a new PApplet). My question is how do I pass the captured cropped image from the first PApplet to the second, I’ve done a fair bit of googling and looking on here but can’t figure it out. Unfortunately my code is based on using the kinect library, I can’t get the video library working on my machine (OSX Catalina) or I would have provided something more generic.

I think there is probably an issue with how/where I am trying to store the selected pixels to PImage crop and then also how I am trying to display that in the second PApplet, what I find interesting and frustrating is that I can pass the variables for my selection rectangle to the second PApplet and display that no problem.

Any advice or pointers in the right direction much appreciated.

Ta

import org.openkinect.freenect.*;
import org.openkinect.freenect2.*;
import org.openkinect.processing.*;

Kinect2 kinect2;

int x, y, w, h; //x,y,w,h of crop rect
PImage full;
PImage crop = null;

void settings() {

  size(960, 540);
  pixelDensity(2);

}

void setup() {

  x=0;
  y=0;
  w=0;
  h=0;

  kinect2 = new Kinect2(this);
  kinect2.initVideo();
  kinect2.initDevice();

  String[] args = {"SecondFrame"};
  SecondApplet sa = new SecondApplet();
  PApplet.runSketch(args, sa);

}

void draw() {

  full = kinect2.getVideoImage();
  background(0);
  
  //flip kinect video and scale to width/height
  pushMatrix(); 
  scale(-1, 1);
  image(full, -map(kinect2.colorWidth, 0, kinect2.colorWidth, 0, width), 0, map(kinect2.colorWidth, 0, kinect2.colorWidth, 0, width), map(kinect2.colorHeight, 0, kinect2.colorHeight, 0, height));
  popMatrix();
  
  //this breaks the code when i try to draw crop rect
  //i think it should be loading the crop rect pixels to PImage crop?
  //crop = get(x, y, x + w, y + h); 
  
  noFill();
  stroke(255, 0, 0);
  rect(x, y, w, h);

}

void mousePressed() {

  x = mouseX;
  y = mouseY;

}

void mouseDragged() {

  w = mouseX-x;
  h = mouseY-y;
  rect(x, y, w, h);

}

public class SecondApplet extends PApplet {

  public void settings() {

    size(960, 540);
    pixelDensity(2);

  }

  public void draw() {

    background(255);
    noFill();
    stroke(255, 0, 0);
    
    //and this breaks the code :(
    //image(crop, 0, 0); 
    
    rect(x, y, w, h);

  }
}

[chuchu]

I’ve stripped out the unnecessary code so that it is easier to see what I am trying to achieve and others are able to run it, below I am just trying to copy the screen of the first PApplet to the display on the second PApplet. Trying to display image(screencopy,0,0) in the second window gives me null pointer exception, but the rect variables carry across ok

Ta!

int a, b, c, d;
PImage screencopy;

void settings() {
  size(960, 540);
}

void setup() {
  frameRate(2);
  rectMode(CORNER);
  a=0;
  b=0;
  c=0;
  d=0;
  String[] args = {"TwoFrameTest"};
  SecondApplet sa = new SecondApplet();
  PApplet.runSketch(args, sa);
}

void draw() {
  background(255);
  for (int i = 0; i < 100; i++) {
    fill(random(0, 255));
    noStroke();
    ellipse(random(0, width), random(0, height), 100, 100);
  }
  screencopy = get();
  noFill();
  stroke(255, 0, 0);
  rect(a, b, c, d);
}

void mousePressed() {
  a = mouseX;
  b = mouseY;
}

void mouseDragged() {
  c = mouseX-a;
  d = mouseY-b;
}

public class SecondApplet extends PApplet {

  public void settings() {
    size(960, 540);
  }
  
  public void draw() {
    background(255);
    noFill();
    stroke(255, 0, 0);
    image(screencopy, 0, 0);//this breaks the code :(
    rect(a, b, c, d);
  }
}

[GoToLoop]

@chuchu

… gives me null pointer exception,

createImage() / Reference

PImage screencopy; → PImage screencopy = createImage(1, 1, ARGB);

[chuchu]

My bad, I didn’t realise I needed to initialise the screencopy variable rather than just load to it via get(). I have two more problems with the code, not sure if they are best placed here or in another thread, but here we go:

If I try to use the P2D renderer the program will not exit cleanly and gives me a ThreadDeath error

and

If I try to use pixelDensity(2), as soon as I start to draw my rect to crop the image I get ArrayIndexOutOfBoundsException on line 28 screencrop = get(a,b,c,d).

I am up for paying someone to work on this and the project behind it if that’s your sort of thing or know a freelancer who can help. I know what I want to achieve but my understanding is limited

Ta

Chchu

int a, b, c, d;
PImage screencrop = createImage(1,1,ARGB);

void settings() {
  size(960, 540);
  //pixelDensity(2);
}

void setup() {
  frameRate(2);
  
  a=0;
  b=0;
  c=0;
  d=0;
  String[] args = {"SecondWindow"};
  SecondApplet sa = new SecondApplet();
  PApplet.runSketch(args, sa);
}

void draw() {
  background(255);
  for (int i = 0; i < 100; i++) {
    fill(random(0, 255));
    noStroke();
    ellipse(random(0, width), random(0, height), 100, 100);
  }
  screencrop = get(a,b,c,d);
  noFill();
  stroke(255, 0, 0);
  rect(a, b, c, d);
}

void mousePressed() {
  a = mouseX;
  b = mouseY;
}

void mouseDragged() {
  c = mouseX-a;
  d = mouseY-b;
  
}

public class SecondApplet extends PApplet {

  public void settings() {
    size(960, 540);
    //pixelDensity(2);
  }
  
  public void draw() {
    background(255);
    noFill();
    stroke(255, 0, 0);
    image(screencrop, (width/2) - (c/2), (height/2) - (d/2));
  }
}

[chuchu]

Does anyone have any ideas on this or pointers in the right direction?

Ta

[jeremydouglass]

Hi @chuchu – were you able to resolve your issues?

It seems like there are many things going on here – one being not using the video library on Catalina, and another being the P2D rendered exit (please provide an example sketch if you are still having that problem), and another being the way that you are using get() with pixelDensity() (are you calling loadPixels first?). It is often easier to respond to one problem at a time, with code relating to just that specific problem.

[chuchu]

Hi @jeremydouglass

Thanks for your response!

Good thinking, lets take it step by step

Let’s ignore the video library troubles for now, from what I’ve read on github it’s a bigger problem, and also the kinect library, which is where I’ll be capturing my video, works fine.

I fixed the pixelDensity(2) issue by doubling the values used in my get(), so get(a*2,b*2,c*2,d*2), I think this works because because a, b, c, d are relative to window size but when I use get() it wants to be pixel size? Another scenario is that I have just stumbled across a solution blindly! I’m not using loadPixels() because as I understand it get() will take the pixel values from the screen, where I am displaying my image so there is no need?

I have not resolved the P2D issue.

Window 1 will capture video feed from my kinect (P2D seems to provide much better framerate) , I’ll draw a rectangle and this cropped section of the feed will display in window 2 which will be displayed on a projector, inside window 2 I’d also like to keystone the cropped image to my projection surface using deadpixels.keystone library (which requires P3D render).

tl;dr

Why doesn’t the below sketch exit cleanly and gives me ThreadDeath error?

void settings() {
  size(960, 540, P2D);
  pixelDensity(2);
}

void setup() {
  String[] args = {"SecondWindow"};
  SecondApplet sa = new SecondApplet();
  PApplet.runSketch(args, sa);
}

void draw() {
}

public class SecondApplet extends PApplet {

  public void settings() {
    size(960, 540, P3D);
    pixelDensity(2);
  }
  
  public void draw() {
  }
}

[jeremydouglass]

Okay, here are resources on the PApplet exit issue:

Prevent one Processing window from closing others

I’ve done some digging – inspired by @GoToLoop and @quark – and have found a solution. Closing sketches in P2D and JAVA2D renderer modes calls the PApplet instance’s exitActual() method, which contains the line System.exit(0) – a static method – which terminates the currently running Java virtual machine. Overriding exitActual() with nothing (i.e. preventing the call to System.exit(0)) prevents sketches from closing others, fixing the issue, but introduces another problem in the case of P2D s…

Closing a second window

I want to be able to create a second window. The problem is I want the user to be able to close the second window, but doing so closes both the second and main frame. Is there a way I can prevent this? PWindow win; public void settings() { size(320, 240); } void setup() { win = new PWindow(); } void draw() { background(255, 0, 0); fill(255); rect(10, 10, frameCount, 10); } void mousePressed() { println("mousePressed in primary window"); } class PWindow extends PApplet { PWi…

…

For sharing images between PApplets, see also:

Sharing objects between multiple PApplets - Processing in Eclipse - runSketch()

hello, I am using Processing (3.2.3) in Eclipse and I have managed to produce a second PApplet window using using runSketch(). I would like to use the second window to control what goes on in the first (and vice versa as well) - so use PApplet 2 to control what’s going on in PApplet 1. I am not new to Processing but I am new to Eclipse and using full Java. So I am wondering how I may access classes which are being used by the other applet - ie. how would different PApplets share objects? Below …

[chuchu]

Thanks for the links, I have seen these but must admit they’re a bit beyond me, I’m trying to get the ‘closing second window’ code to work with my multi window example, have replied there for help.

Ta