Program in a Program?

There isn’t a one-size-fits-all approach because the logic of many sketches does not logically coexist – for example, sketches that are using a video camera feed, or setting up peasycam, or catching MIDI events, etc. For some things, you could extend this through routing – for example, have the master sketch set up MIDI and multiple sub-sketches listen for MIDI events, or have the master sketch grab the latest data from serial and then have multiple sub-sketches react. For other things, it is trickier – like instantiating a different cam object for each peasycam. (Almost) all these things can be worked around, but they are several different work-arounds. Some libraries are strongly designed around a single-sketch paradigm. You also can’t have multiple top-level interaction functions like keyPressed() – they need to be merged, and then the order in which different client sketches intercept events could matter. A meta-input handler can register and unregister event listeners in some order, but that isn’t the simple, out-of-the-box Processing setup.

That said, many people are unaware that you can layer sketch-like objects as PGraphics into a single layout, and that they don’t have to all be the same renderer – so long as the top-level canvas is P2D or P3D, you can mix as many JAVA2D, P2D, or P3D layers in under it and treat each one like a separate sketch at the level of rendering – although not at the level of code. If you want a quick way to assemble visual sketches like legos, start by pointing each at a dedicated PGraphics with drawing the PGraphics as the last step.

If you are trying to create a dynamic gallery of sketches there are several ways to tackle that outside Processing – depending on if you want it to be run-time dynamic or compiled. You can have sketches launch other sketches. You can also format stand-alone sketches ahead of time so that a script can mash them up into one multi-object.

However, this all requires advanced planning. If you want simultaneous sketches embedded in a surface that are largely stand-alone, I would suggest using p5.js in a web browser with iframes or namespacing.


Also: If you want a framework that lets you use Processing sketches like patches or plugins then I recommend checking out the latest release of PraxisLive.

See also using Syphon to pipe Processing into VDMX: https://vdmx.vidvox.net/tutorials/vdmx-processing-syphon-osc-intro

4 Likes

For Python Mode sketches (".pyde"), you might ask its dev if there’s a way to load & run those sketches from within another “.pyde” sketch:

1 Like

You cannot literally run a sketch within a sketch, but you can definitely write something where that appears to be the case. This involves writing an alternative base class to PApplet, copying all the methods you want to support and instead making them draw to a PGraphics, and then loading the sketch code against this. It’s taking what @jeremydouglass talks about but also as much as possible making it function at “the level of code”.

This is loosely how PraxisLIVE actually works. And yes, handling events and libraries is interesting, but doable. :smile: Add the ability to live code any sketch into the mix and it’s even more “fun” to build.

Of course, if you want multiple sketches on screen at once, you also need to reason how sketches merge. What does it mean for two sketches to both call background() for example?!

@GoToLoop Python mode would still have to handle the fact the library can’t literally do this, though. Might be easier to write the delegation method to fake this.

1 Like

Isn’t this what PApplet already does?

OK, offscreen PGraphics! :smile: It’s similar, it obviously has to be, but not using the primary graphics. And there are a few other differences to make it work as if the offscreen is a primary.

Thanks for the advice! I have not heard of PApplet… Ill have to look into it. It sounds interesting! :smiley:

Every sketch you write is a PApplet. You’ll definitely need to get your head around what that is and what it does to take this idea further,

For those wishing to do so – from the Processing javadocs:

Base class for all sketches that use processing.core.
The Window Size and Full Screen page on the Wiki has useful information about sizing, multiple displays, full screen, etc.

Processing uses active mode rendering. All animation tasks happen on the “Processing Animation Thread”. The setup() and draw() methods are handled by that thread, and events (like mouse movement and key presses, which are fired by the event dispatch thread or EDT) are queued to be safely handled at the end of draw(). […]

Inspired by this thread I tried my hand on a way to run multiple Processing sketches at the same time, spawned from a “parent” sketch. Then the secondary sketches are made invisible, and the parent sketch retrieves their graphic content. Could be a start to what @Repet is after.

Downside is that mouse and keyboard events do not get forwarded to the secondary sketches. I believe that can be solved by some tinkering, but might not be thread safe.


PWindow[] win;

public void settings() {
  size(600, 600);
}

void setup() { 
  win = new PWindow[4];
  for(int i = 0;i < win.length; i++)
  {
    win[i] = new PWindow();
  }
  surface.setLocation(20,20);
}

void draw() {
  background(0);
  image(win[0].get(), 0, 0);
  image(win[1].get(), 0, 300);
  image(win[2].get(), 300, 0);
  image(win[3].get(), 300, 300);
}

void mousePressed() {
} 

class PWindow extends PApplet {
  PWindow() {
    super();
    PApplet.runSketch(new String[] {this.getClass().getSimpleName()}, this);
  }
  
  int h = (int) random(255);
  PVector position;

  void settings() {
    size(300, 300);
  }

  void setup() {
    colorMode(HSB);
    position = new PVector(random(width), random(height));
    surface.setVisible(false);
  }

  void draw() {
    background(color(h++, 255, 255));
    ellipse(position.x, position.y, random(50), random(50));
    h=h%255;
  }

  void mousePressed() {
    println("mousePressed in secondary window");
  }
}

Events are only part of the issue - your win[..].get() won’t be threadsafe either, and depending on renderer might work, work most of the time, work some of the time, or crash. Note the part in the Javadoc @jeremydouglass posted about the Processing Animation Thread - you now have five of them! :wink:

Guess I got lucky with simple sketches.

I’m not an expert on multithreading, but would this naive code make it thread safe?

For this to fully work, there should be a way to synchronise all the animation threads across all the PApplet instances.

PWindow[] win;

public void settings() {
  size(600, 600);
}

void setup() { 
  win = new PWindow[4];
  for (int i = 0; i < win.length; i++)
  {
    win[i] = new PWindow((i/2)*300, (i%2)*300);
  }
  surface.setLocation(20, 20);
}

void draw() {
  background(0);
  for (int i = 0; i < win.length; i++)
  {
    synchronized(win[i].g)
    {
      image(win[i].get(), win[i].winPos.x, win[i].winPos.y);
    }
  }
}

void mousePressed() {
  for (PWindow window : win)
  {
    if (mouseOver(window.winPos.x, window.winPos.y, window.width, window.height))
    {
      window.mouseX = (int)(mouseX-window.winPos.x);
      window.mouseY = (int)(mouseY - window.winPos.y);
      window.mousePressed();
    }
  }
} 

boolean mouseOver(float x, float y, float w, float h)
{
  return mouseX >= x && mouseX <= x + w && mouseY >= y && mouseY <= y + h;
}

class PWindow extends PApplet {
  PVector winPos;
  PWindow(int x, int y) {
    super();
    winPos = new PVector(x, y);
    PApplet.runSketch(new String[] {this.getClass().getSimpleName()}, this);
  }

  int h = (int) random(255);
  PVector position;

  void settings() {
    size(300, 300);
  }

  void setup() {
    colorMode(HSB);
    position = new PVector(random(width), random(height));
    surface.setVisible(false);
  }

  void draw() {
    background(color(h++, 255, 255));
    ellipse(position.x, position.y, 50, 50);
    h=h%255;
    fill(0);
    rect(20, 8, textWidth(frameRate+""), 20); 
    fill(255);
    text(frameRate, 20, 20);
  }

  void mousePressed() {
    println(mousePressed);
    position.x = mouseX;
    position.y = mouseY;

  }
}

The mousePressed() implementation, while working in this example, is flawed. Maybe I shouldn’t be posting this, or somebody might run off with this code and try to attempt to use it in a real scenario :stuck_out_tongue:

No! Well, almost certainly not. synchronized is useless unless both sides (threads) synchronize on the same object. There’s a vague chance that the Processing renderer might synchronize on the same PApplet throughout all parts of the animation that might break what you’re doing, but I wouldn’t count on it! :smile:

AFAIK, Processing hasn’t been synchronized its stuff at least since Processing 1.5.1! :nerd_face:

And indeed, synchronized is moot to protect any code from another Thread against the moment the PApplet enters its rendering step, where the contents of its main canvas are actually transferred to its window app. :zipper_mouth_face:

@colouredmirrorball, there’s no need to clone the PGraphics main canvas of the other PApplet by calling the PImage::get() method. You can simply use it via PApplet::getGraphics(). :wink:

Also, remove that useless synchronized () {} block.
It just makes your sketch slower w/o any concurrency guarding benefit whatsoever in this particular case. :roll_eyes:

And a plural or collective name is better for containers such as the array:

static final int WINDOWS = 4;
final PWindow[] windows = new PWindow[WINDOWS];

void draw() {
  for (final PWindow window : windows) {
    final PGraphics canvas = window.getGraphics();
    final PVector coords = window.winPos;
    set(coords.x, coords.y, canvas);
  }
}

yup like the replies prior every sketch is a PApplet
the tutorial for eclipse shows how it works
https://processing.org/tutorials/eclipse/
So i’d guess to have different sketches in a bundle, one could have different sketches (classes) inherited from PApplet and run them in different threads. I’d think they’d have their own windows

and about the threads, i think AWT or more correctly Swing still have something to do with the Event Dispatch Thread which is a java issue. One could nevertheless test to see where the limits are.

That was mostly meant as a joke! :smile: Although you could theoretically extend a renderer to did this.

You were more correct with AWT actually! :smile: The EDT is part of AWT, and you can check if your code is running on it using EventQueue.isDispatchThread(). This should mainly be in event callbacks. Neither the Java2D or JOGL renderers use this for draw().

We can type in println( Thread.currentThread() ); in order to find out the current Thread running a function: :smile_cat:

OT: about threads:
this would take it somewhat off topic, as i meddle more in java than processing itself.
i’ve tried things like rendering things in image buffers and using executor service to run them in different threads. http://tutorials.jenkov.com/java-util-concurrent/executorservice.html
after that i combine/blit them into the canvas like tiles when each thread of the executor service completes and return images to the main routine (this could be draw()). even on AWT/Swing that significantly speed up some heavy graphing/plotting work. this is especially observable (from waiting ~10 seconds to completing in a heartbeat) on the slower but multi-core cpus that’s prevalent on laptops, netbooks, tablets PCs these days.
i’d think similar strategies can be done in processing.

Yes, that’ll work if you’re CPU bound, and doing operations that Java2D can’t run on the GPU, so direct pixel stuff especially. But I would guess most people doing intensive graphical things are running one of the OpenGL renderers?! In which case they’re probably GPU bound and this approach wouldn’t help / would break.

Good to see you using ExecutorService - most attempts at threading around here are “interesting” :smile:

one of those things i like about ExecutorService is that invokeAll().
it is basically first setting up a bunch of Callables put them in a Set and invokeAll().
this has the disadvantage that it would block and wait for all the threads to complete and results to return, but this could still be a lot faster if each thread/job takes about same amount of time to complete. But i’d say invokeAll() is very convenient compared to managing the threads separately.

for openGL i’m not too sure if separate sets of GLSL scripts may be generated in different threads and then bundled into a big script say when it gets collected up in draw(), then the whole script could be ‘pushed’ to the hardware to render. but i think there may still be bottlenecks with this approach