# P2D & PGraphics Movie not displaying across render modes? + BONUS P2D performance issue

**URL:** <https://discourse.processing.org/t/p2d-pgraphics-movie-not-displaying-across-render-modes-bonus-p2d-performance-issue/31944>\
**Category:** Libraries\
**Created:** [August 26, 2021, 9:15am UTC](https://discourse.processing.org/t/p2d-pgraphics-movie-not-displaying-across-render-modes-bonus-p2d-performance-issue/31944 "2021-08-26T09:15:27Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![rapatski](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/rapatski/32/5164_2.png) [@rapatski](https://discourse.processing.org/u/rapatski)\
**Post date:** [August 26, 2021, 9:15am UTC](https://discourse.processing.org/t/p2d-pgraphics-movie-not-displaying-across-render-modes-bonus-p2d-performance-issue/31944/1 "2021-08-26T09:15:27Z")

</div>

Hi friends,

Again, sad to come back to the forum with some puzzling behaviour. Maybe someone can point me towards something I’m doing wrong or perhaps a workaround?

I currently have a sketch that draws into an off-screen buffer (PGraphics) that I use to manipulate the on-screen image. The way I do the on-screen stuff uses vertex() calls so I’m forced to use the P2D renderer for this. Until now, it seemed fine keeping the off-screen PGraphics in the standard renderer, and everything was just dandy.

Yesterday however, I decided that rather than generating some noise in the off-screen buffer, I wanted to render a video into it. Easy enough I thought, yet when I draw the video into the buffer using pg.image() I get nothing. I’ve exhaustively debugged this - any static PImage draws fine from the P2D main program into the standard rendered PGraphics, but when I use a Movie, nothing shows up. Changing the PG to P2D solves the issue…

BUT

…introduces a MASSIVE performance hit (from 60 to 7fps) caused by function that samples the brightness (bit shifted red channel deriv) from the off-screen buffer to use for the on-screen content. Indeed it should be a bit of an expensive function, as it downloads the texture and iterates over all pixels, but like I said before, the exact sketch runs at 60fps when the buffer uses the standard renderer. Writing this I’m thinking, could it be that in P2D it does not check the downloaded flag on the texture, and re-downloads the texture with every loadPixels() call? Brb!

Below two snippets of code to illustrate:

P2D Video playback example:

```auto
import processing.video.*;

PGraphics pg;
Movie movie;

void setup()
{
  size(500, 500, P2D);

  movie = new Movie(this, "launch2.mp4");
  movie.loop();

  pg = createGraphics(width, height); // video does not display
  //pg = createGraphics(width, height, P2D); // video works fine, but causes performance bug in other code
}

//void movieEvent(Movie m) {
// m.read();
//}

void draw()
{
  surface.setTitle("FPS: "+nf(frameRate, 1, 1));
  
  // update pg
  
  if (movie.available() == true) {
    movie.read();
  }
  
  pg.beginDraw();
  pg.image(movie, 0, 0, pg.width, pg.height);
  pg.endDraw();
  
  // draw
  
  image(pg, 0, 0);
}

```

Snippet that samples from the pg that seems to cause a performance hit in P2D, rather than in the standard renderer:

```auto
color sampleColour(PImage img, float normX, float normY)
{
  img.loadPixels();
  return img.pixels[int(normY * img.height) * img.width + int(normX * img.width)];
}

float sampleBrightness(PImage img, float normX, float normY)
{
  return (sampleColour(img, normX, normY) >> 16 & 0xFF);
}

```

---

<div class="post-metadata">

**Author:** ![neilcsmith](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/neilcsmith/32/144_2.png) [@neilcsmith](https://discourse.processing.org/u/neilcsmith)\
**Post date:** [August 26, 2021, 9:24am UTC](https://discourse.processing.org/t/p2d-pgraphics-movie-not-displaying-across-render-modes-bonus-p2d-performance-issue/31944/2 "2021-08-26T09:24:52Z")

</div>

> [@rapatski](#):
>
> Indeed it should be a bit of a expensive function, as it downloads the texture and iterates over all pixels, but like I said before, the exact sketch runs at 60fps when the buffer uses the standard renderer.

Why is this surprising?! The texture download, and particularly the bit manipulation done to get the pixel data in the format expected by Processing, is really non-performant. So using the standard renderer has a lot less to do.

I’d look into shaders if you want this to perform well.

No idea why the standard renderer isn’t working, but the current video library is a little borked with OpenGL! (standard answer from me, try it in PraxisLIVE instead 😉 )

---

<div class="post-metadata">

**Author:** ![rapatski](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/rapatski/32/5164_2.png) [@rapatski](https://discourse.processing.org/u/rapatski)\
**Post date:** [August 26, 2021, 9:31am UTC](https://discourse.processing.org/t/p2d-pgraphics-movie-not-displaying-across-render-modes-bonus-p2d-performance-issue/31944/3 "2021-08-26T09:31:33Z")

</div>

Found the issue… whether my assumption of the ‘checking of a download’ flag is true or not, pulling the loadPixels() from that function, and manually just downloading the texture once solved the performance issue. 🤦‍♂️

---

<div class="post-metadata">

**Author:** ![rapatski](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/rapatski/32/5164_2.png) [@rapatski](https://discourse.processing.org/u/rapatski)\
**Post date:** [August 27, 2021, 12:30pm UTC](https://discourse.processing.org/t/p2d-pgraphics-movie-not-displaying-across-render-modes-bonus-p2d-performance-issue/31944/4 "2021-08-27T12:30:34Z")

</div>

Filed them as two bugs:

> <https://github.com/processing/processing/issues/6215>
>
> \## Description
> When I try and draw a movie into a PGraphics object that does no…t share the same renderer; main sketch uses P2D renderer, PGraphics created with standard renderer, the movie frame shows up as black. When the renderers match, either both standard for the main sketch and for the created PGraphics object, or both P2D, it does work as expected.
> 
> \## Expected Behavior
> If I pass a Movie object into a pg.image() call, I expect it to be displayed, regardless of renderers ;)
> 
> \## Steps to Reproduce
> \`\`\`
> import processing.video.\*;
> 
> PGraphics pg;
> Movie movie;
> 
> 
> void setup()
> {
> size(500, 500, P2D);
> 
> movie = new Movie(this, "launch2.mp4");
> movie.loop();
> 
> pg = createGraphics(width, height); // video does not display
> //pg = createGraphics(width, height, P2D); // video works fine
> }
> 
> //void movieEvent(Movie m) {
> // m.read();
> //}
> 
> void draw()
> {
> surface.setTitle("FPS: "+nf(frameRate, 1, 1));
>   
> // update pg
>   
> if (movie.available() == true) {
> movie.read();
> }
>   
> pg.beginDraw();
> pg.image(movie, 0, 0, pg.width, pg.height);
> pg.endDraw();
>   
> // draw
>   
> image(pg, 0, 0);
> }
> \`\`\`
> 
> \## Your Environment
> \* Processing version: 3.5.4 and 4.0b1
> \* Operating System and OS version: macOS Mojave 11.4 (20F71)
> \* Other information:

> <https://github.com/processing/processing/issues/6214>
>
> \## Description
> Whenever loadPixels() is used \*\*on a PGraphics object\*\* in a P2D… context, it seems to literally download the texture every time, regardless of whether it's been called/downloaded before. In the standard renderer, it seems like subsequent loadPixels() calls do not result in numerous texture downloads.
> 
> \## Expected Behavior
> I was expecting the loadPixels() function to check the isLoaded() / loaded flag before deciding on whether the pixel array has to be recreated or not.
> 
> \## Current Behavior
> In P2D, calling loadPixels on a large PGraphics texture numerous times a frame absolutely wrecks the frame rate, as it should if it recreates/downloads the buffer every time. The standard renderer seems to ignore subsequent loadPixels() calls to the same PGraphics texture. On standard PImage files this issue does not exist. Only PGraphics objects passed into / converted to PImage objects, see example below.
> 
> \## Steps to Reproduce
> \`\`\`
> PGraphics pg;
> 
> void setup()
> {
> size(500, 500);
> frameRate(60);
>   
> PImage image = loadImage("test.jpg");
> pg = createGraphics(width, height);
> pg.beginDraw();
> pg.image(image, 0, 0, pg.width, pg.height);
> pg.endDraw();
>   
> }
> 
> void draw()
> {
> surface.setTitle("FPS: "+nf(frameRate, 1, 1));
> 
> // please ignore the 'logic' behind this example, 
> // this is purely to highlight the issue
>   
> // adjust these until your machine struggles
> int numX = 20;
> int numY = 20;
>   
> pg.loadPixels();
>   
> for(int i = 0; i \< numX; i++)
> for(int j = 0; j \< numY; j++)
> {
> float normX = i/float(numX);
> float normY = j/float(numY);
> color c = sampleColour(pg, normX, normY);
>       
> fill(c);
> rect(normX\*width, normY\*height, width/numX, height/numY);
> }
> }
> 
> color sampleColour(PImage img, float normX, float normY)
> {
> img.loadPixels(); // with this uncommented, this function wrecks the sketch in P2D - but not in the standard renderer
> return img.pixels\[int(normY \* img.height) \* img.width + int(normX \* img.width)\];
> }
> \`\`\`
> 
> \## Your Environment
> \* Processing version: Both 3.5.4 and 4.0b1
> \* Operating System and OS version: macOS Big Sur 11.4 (20F71)
> \* Other information:

---

<div class="post-metadata">

**Author:** ![rapatski](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/rapatski/32/5164_2.png) [@rapatski](https://discourse.processing.org/u/rapatski)\
**Post date:** [August 27, 2021, 12:38pm UTC](https://discourse.processing.org/t/p2d-pgraphics-movie-not-displaying-across-render-modes-bonus-p2d-performance-issue/31944/5 "2021-08-27T12:38:58Z")

</div>

> [@neilcsmith](#):
>
> Why is this surprising?! The texture download, and particularly the bit manipulation done to get the pixel data in the format expected by Processing, is really non-performant. So using the standard renderer has a lot less to do.
> 
> I’d look into shaders if you want this to perform well.

Hi Neil, I’m not sure I understand what you’re saying here…? I am aware that manipulating the pixel array is expensive, and that shaders are the way to go when looking for performance. In some cases though you’d still need to download a texture to manipulate it cpu side.

The issue described is about why loadPixels() behaves differently on a PGraphics object created with a P2D than it does with the standard renderer, eg:

```auto
  pg = createGraphics(width, height); // works fine - seems to ignore subsequent loadPixels() calls
  pg = createGraphics(width, height, P2D); // massive performance hit with multiple loadPixels() calls

```