# PGraphics - read pixels as they are drawn

**URL:** <https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837>\
**Category:** Coding Questions\
**Created:** [June 13, 2020, 11:40am UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837 "2020-06-13T11:40:30Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 11:40am UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/1 "2020-06-13T11:40:30Z")

</div>

with the PGraphics class, is there a way to read pixles as they get drawn?

for example

```auto
PGraphics c;

void settings() {
    size(100, 100, P3D);
}

void setup() {
    c = createGraphics(100,100, P3D);
    c.beginDraw();
}
void draw() {
    c.background(255);
    // read pixels
    c.background(100, 100, 100);
    // read pixels
}

```

i want to do this in order to, for example, eliminate the need to do beginDraw and endDraw every single time i want to update and display my pixels

as i am thinking that i could thus copy the live pixels from the PGraphics object into a PImage and then image that similar to how most renderers will directly access the graphics (stack?) buffer to update the screen and just grab what ever may be there even while a draw may be executing (single buffered)

as this is the only way i can think of that will speed up my rendering when

for example, rendering 2500 PGraphics objects, calling `beginDraw(); /* ... */; endDraw();` makes my fps go to about 7 FPS

NOTE: loadPixels internally calls beginDraw and endDraw so this will add additional overhead and thus be ineffective - [https://github.com/processing/processing/blob/78f7c30076de3f9a24536913249009be7e543a6b/core/src/processing/opengl/PGraphicsOpenGL.java#L5378](https://github.com/processing/processing/blob/78f7c30076de3f9a24536913249009be7e543a6b/core/src/processing/opengl/PGraphicsOpenGL.java#L5378)

example:

```auto
PGraphics g1, g2;

void settings() {
  size(400, 400, P3D);
}

void setup() {
  background(0);
  g1 = createGraphics(500,500, P3D);
  g1.beginDraw();
  g2 = createGraphics(500,500, P3D);
  g2.beginDraw();
}

void draw() {
  g2.clip(0,0,50,50);
  g2.background(0, 128, 128);
  g2.clip(0,0,25,25);
  g2.background(0);
  g2.clip(35,35,65,65);
  g2.background(0, 228, 128);
  g2.clip(0,0,50,50);
  g2.flush();
  g1.image(g2.get(0,0,500,500), 0, 0);
  g1.flush();
  image(g1.get(0,0,500,500), 0, 0);
}

```

---

<div class="post-metadata">

**Author:** ![noel](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/noel/32/213_2.png) [@noel](https://discourse.processing.org/u/noel)\
**Post date:** [June 13, 2020, 12:37pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/2 "2020-06-13T12:37:50Z")

</div>

I do not know if I ‘get’ your question, but why don’t you get the whole screen. A 100 by 100 takes only 2 milliseconds. far faster then a draw loop. Only getting a screen 8 times bigger will start to take the 60 f/s down.

```auto
PImage pi;
int start_time;

void setup() {  
  size(100, 100, P3D);
}

void draw() {
  start_time = millis();
  for (int i = 0; i < 100; i++) {
    pi = get();
  }
  println(millis()-start_time);
}

```

I’ve been following your last topic about shapes, and I think these are valid things to try out. Even if you can’t use box(), you can construct them with vertices.

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 12:38pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/3 "2020-06-13T12:38:57Z")

</div>

get() does not seem to work for a PGraphics object

```auto
  g2.flush();
  g1.image(g2.get(0,0,500,500), 0, 0);
  g1.flush();
  image(g1.get(0,0,500,500), 0, 0);

```

see example in top post for full code

---

<div class="post-metadata">

**Author:** ![noel](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/noel/32/213_2.png) [@noel](https://discourse.processing.org/u/noel)\
**Post date:** [June 13, 2020, 12:41pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/4 "2020-06-13T12:41:11Z")

</div>

> [@mgood7123](#):
>
> i am thinking that i could thus copy the live pixels from the PGraphics object into a PImage

But isn’t PImage your end goal?

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 12:42pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/5 "2020-06-13T12:42:12Z")

</div>

yes but is MUST be obtainable from a PGraphics object

---

<div class="post-metadata">

**Author:** ![noel](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/noel/32/213_2.png) [@noel](https://discourse.processing.org/u/noel)\
**Post date:** [June 13, 2020, 12:44pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/6 "2020-06-13T12:44:38Z")

</div>

Ok, I get it…

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 12:52pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/7 "2020-06-13T12:52:05Z")

</div>

i tried doing

```auto
PGraphics g1, g2;

void settings() {
  size(400, 400, P3D);
}

void setup() {
  background(0);
  g1 = createGraphics(500,500, P3D);
  //g1.beginDraw();
  g2 = createGraphics(500,500, P3D);
  g2.beginDraw();
}

void draw() {
  g2.clip(0,0,50,50);
  g2.background(0, 128, 128);
  g2.clip(0,0,25,25);
  g2.background(0);
  g2.clip(35,35,65,65);
  g2.background(0, 228, 128);
  g2.clip(0,0,50,50);
  g2.flush();
  assert(g2.isGL());
  PGraphicsOpenGL g3 = (PGraphicsOpenGL) g2;
  FrameBuffer fb = g3.getFrameBuffer();
  fb.readPixels();
  java.nio.IntBuffer ib = fb.getPixelBuffer();
  int[] intArray = ib.array();
  PImage image = createImage(500, 500, ARGB);
  image.loadPixels();
  for (int i = 0; i < image.pixels.length; i++) {
    image.pixels[i] = intArray[i];
  }
  image.updatePixels();
  g1.beginDraw();
  g1.image(image, 0, 0);
  g1.endDraw();
  image(g1, 0, 0);
}

```

how ever i do not get anything displayed

---

<div class="post-metadata">

**Author:** ![noel](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/noel/32/213_2.png) [@noel](https://discourse.processing.org/u/noel)\
**Post date:** [June 13, 2020, 1:05pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/8 "2020-06-13T13:05:00Z")

</div>

What if instead of using the pixel array make your own custom int array and keep track of them?

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 1:07pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/9 "2020-06-13T13:07:36Z")

</div>

cus i wont be able to use PGraphics calls on them, for example box(40);

since ill need to manually create the box myself using pixel data such as a library that draws shapes and stuff directly to a bitmap, which would severly limit the capabilities, such as PShape cannot be used as PShape probably does not know how to be read by the drawing library along with most other Processing specific classes

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 1:27pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/10 "2020-06-13T13:27:25Z")

</div>

this DOES work HOWEVER i need to call `endDraw()` in order for `g3.getTexture().get(image.pixels)` to get any pixels from the texture, in which want to AVOID needing to call `endDraw()` so this IS NOT a solution, however i believe it will help

and

```auto
  FrameBuffer fb = g3.getFrameBuffer();
  fb.readPixels();
  fb.getPixels(image.pixels);

```

is empty both BEFORE and AFTER `endDraw();`

```auto
PGraphics g1, g2;

void settings() {
  size(400, 400, P3D);
}

void setup() {
  background(0);
  g1 = createGraphics(500,500, P3D);
  //g1.beginDraw();
  g2 = createGraphics(500,500, P3D);
  //g2.beginDraw();
}

void draw() {
  g2.beginDraw();
  g2.clip(0,0,50,50);
  g2.background(0, 128, 128);
  g2.clip(0,0,25,25);
  g2.background(0);
  g2.clip(35,35,65,65);
  g2.background(0, 228, 128);
  g2.clip(0,0,50,50);
  g2.endDraw();
  //g2.flush();
  assert(g2.isGL());
  PGraphicsOpenGL g3 = (PGraphicsOpenGL) g2;
  PImage image = createImage(500, 500, ARGB);
  image.loadPixels();
  g3.getTexture().get(image.pixels);
  image.updatePixels();
  println("image created");
  g1.beginDraw();
  g1.image(image, 0, 0);
  g1.endDraw();
  image(g1, 0, 0);
}

```

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 1:56pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/11 "2020-06-13T13:56:02Z")

</div>

and

```auto
  //g2.endDraw();
  g2.flush();
  assert(g2.isGL());
  PGraphicsOpenGL g3 = (PGraphicsOpenGL) g2;
  PImage image = createImage(500, 500, ARGB);
  image.loadPixels();
  
  // frame buffer fails
  
  //FrameBuffer fb = g3.getFrameBuffer();
  //fb.readPixels();
  //fb.getPixels(image.pixels);
  
  // texture fails
  
  Texture t = g3.getTexture();
  t.updateTexels();
  t.get(image.pixels);
  image.updatePixels();
  println("image created");

```

the frame buffer is empty both before and after the `endDraw();`  
however the texture is only empty before the `endDraw();`

unless it draws to a framebuffer i do not yet know about

---

<div class="post-metadata">

**Author:** ![noel](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/noel/32/213_2.png) [@noel](https://discourse.processing.org/u/noel)\
**Post date:** [June 13, 2020, 2:02pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/12 "2020-06-13T14:02:51Z")

</div>

To follow this I have to learn what assert even does.  
I found [this post](https://stackoverflow.com/questions/2758224/what-does-the-java-assert-keyword-do-and-when-should-it-be-used) where it’s said that when an error is encountered it stops executing the code right when the error happens, so I don’t yet understand the use of it in your code.  
The example there really stops the code by summing too big numbers and prints the error to the console, but the second part doesn’t give a ‘wrong’ print.

```auto
void setup() {  
  // int i = sum(2147483647, 1);
  // println(i);
  int t = sum(0, 0);
  println(t);
}

int sum(int a, int b) {
  assert (Integer.MAX_VALUE - a >= b) : 
  "Value of " + a + " + " + b + " is too large to add.";
  final int result = a + b;
  assert (result - a == b) : 
  "Sum of " + a + " + " + b + " returned wrong sum " + result;
  return result;
}

```

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 2:06pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/13 "2020-06-13T14:06:28Z")

</div>

`assert(condition)` is basically

```auto
if (condition == false) abort();

```

which usually just prints the error condition and then exits the program

however this behaviour is Programming Language implementation-dependent (eg not guarenteed to do the same thing in every programming language)

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 2:08pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/14 "2020-06-13T14:08:10Z")

</div>

in my code i ensure that the graphics object is of type PGraphicsGL as that is what i am casting to and accessing methods from

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 2:12pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/15 "2020-06-13T14:12:51Z")

</div>

in your example, it might be

```auto
void setup() {  
  // int i = sum(2147483647, 1);
  // println(i);
  int t = sum(0, 0);
  println(t);
}

int sum(int a, int b) {
  if (Integer.MAX_VALUE - a >= b) {
    String msg = "Value of " + a + " + " + b + " is too large to add.";
    PrintErrorAndPauseExecution(msg);
  }
  final int result = a + b;
  if (result - a == b) {
    String msg = "Sum of " + a + " + " + b + " returned wrong sum " + result;
    PrintErrorAndPauseExecution(msg);
  }
  return result;
}

```

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 13, 2020, 2:19pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/16 "2020-06-13T14:19:58Z")

</div>

anyway, im starting to think that this question is too high-level for you to help me, as it seems like you are inexperienced with OpenGL and the internals of Processing API (specifically PGraphics/PGraphicsGL)

---

<div class="post-metadata">

**Author:** ![noel](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/noel/32/213_2.png) [@noel](https://discourse.processing.org/u/noel)\
**Post date:** [June 13, 2020, 2:30pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/17 "2020-06-13T14:30:27Z")

</div>

> [@mgood7123](#):
>
> anyway, im starting to think that this question is too high-level for you to help me

Absolutely right! I’m following this to learn, and by answering many times maybe atract someone with tho knowledge to jump in.

---

<div class="post-metadata">

**Author:** ![jeremydouglass](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jeremydouglass/32/20_2.png) [@jeremydouglass](https://discourse.processing.org/u/jeremydouglass)\
**Post date:** [June 16, 2020, 4:08pm UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/18 "2020-06-16T16:08:44Z")

</div>

I suggest studying the internals in question, @mgood7123.

> <https://github.com/processing/processing/blob/master/core/src/processing/opengl/PGraphicsOpenGL.java>

In particular, look at endDraw, since this is what you are trying to avoid.

**I am not an OpenGL expert** , but my understanding is that OpenGL rendering of commands – such as calling box, or drawing a shape – combine geometry with texture/lighting/camera. It isn’t drawing a line of pixels, it is describing a 3D scene which **later** gets rendered in a render pipeline. So: before rendering, just geometry. After, pixels. At some point / periodically that geometry gets “flushed” down the pipeline.

There is a partial illustration of that pipeline in the [Shaders tutorial](https://processing.org/tutorials/pshader/).

[![](https://processing.org/tutorials/pshader/imgs/pipeline.png) ](https://processing.org/tutorials/pshader/imgs/pipeline.png)

This is one of the reasons that pixels may notappear when you display a PGraphics before the pipeline runs – they aren’t pixels yet, they are just geometry.

You may notice that endDraw() calls flush.

You seem to want to render many discrete areas (hundreds or thousands?) using an OpenGL pipeline in a high-performance way. You have started many closely related threads on this.

As I understand it, your key problem is that you want a big object-oriented class tree with lots of encapsulation – hundreds or thousands of PGraphics. That gives you hundreds or thousands of geometry queues with no pixels, and hundreds or thousands of pipelines to run.

I outlined an alternative in another of your threads – send all your geometry to a single shared PGraphics (or a small number), then call endDraw once. This has disadvantages, because the shared pipeline is shared – camera, texture, lighting – but it reduces your pipeline flush calls.

Otherwise you may need to rewrite PGraphics and/or create your own renderer. A question is whether what you want (that many separate pipeline flush calls per second) is possible with OpenGL in general – never mind Processing.

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 17, 2020, 10:28am UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/19 "2020-06-17T10:28:03Z")

</div>

yea, i have already taken a bit of a look, as in [PGraphics framebuffer can be rendered in `noLoop` mode but cannot be rendered in `loop` mode](https://discourse.processing.org/t/pgraphics-framebuffer-can-be-rendered-in-noloop-mode-but-cannot-be-rendered-in-loop-mode/21912)

however my main goal is obtaining the pixels of a PGraphics Object, like what

```auto
graphicsObject.endDraw();
image(graphicsObject);

```

does

geometry and other stuff such as lights are not particularly important, only the pixels that get drawn, now i CAN do this in noLoop mode (looping = false) however this seems broken, and i do not know why

note that i get

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/2/274cd97caa83999824c3a23e3c43d5ecf6ef4f52.png)

if i have

```auto
void drawBox() {
  // draw a box onto the PGraphics object
  g2.background(0);
  int c = wrap(frameCount, 255);
  g2.fill(0, c, c);
  if (frameCount == 1) g2.lights();
  g2.noStroke();
  //g2.translate(width/2, height/2);
  //g2.rotateX(frameCount/100.0);
  //g2.rotateY(frameCount/200.0);
  g2.box(40);
}

void draw() {
  if (!looping) looping = true;
  
  background(0);

  drawBox();
  
  // render the PGraphic's FBO, note this fails in loop mode
  g2.imageFrameBuffer();
  
  int oldColor = getGraphics().fillColor;
  fill(255);
  textSize(16);
  text("frameCount = " + frameCount + ", FPS: " + frameRate, 10, 20);
  fill(oldColor);
}

```

which indicates that all geometry and stuff gets reset somewhere in endDraw

note that

> <https://github.com/mgood7123/Processing3_DirectR/blob/master/DirectR.pde#L17>

  
is used for `g2.imageFrameBuffer();`

in which if i do

```auto
  g2.beginDraw();
  drawBox();
  g2.endDraw();
  image(g2, 0, 0);

```

then lights gets reset in the next and subsequent draws

---

<div class="post-metadata">

**Author:** ![mgood7123](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@mgood7123](https://discourse.processing.org/u/mgood7123)\
**Post date:** [June 24, 2020, 7:57am UTC](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837/20 "2020-06-24T07:57:10Z")

</div>

i think that the primary reason that endDraw() is slow when used more than once in a draw call, is that it internally called \*swapBuffers(…) which is very expensive as, for example:

```auto
// C++ CODE
/**
 * buffer swapping is expensive if **vsync** is enabled
 *
 * if vsync is enabled, eglSwapBuffers always waits for
 * the screen to be refreshed once between swapbuffer calls
 */
EGLBoolean GLIS_SwapBuffers(class GLIS_CLASS & GLIS) {
    return eglSwapBuffers(GLIS.display, GLIS.surface);
}

```

when this is called, say 10 times per every draw, my FPS plummets to 12 FPS

[Next page](https://discourse.processing.org/t/pgraphics-read-pixels-as-they-are-drawn/21837.md?page=2)