Performance issues with PGraphics layers and image() calls

Processing Coding Questions
1

Hello! I am working on a Falling Sand + Artillery game.

TL;DR: Why is stretching images over a higher resolution significantly more expensive, even if the original image res stays the same? Is there a smart way to render something on multiple layers and then upscale that without loss or blur?

To keep it brief: I am writing the game in a way that allows it to be scaled to basically any resolution, in addition to it allowing for different scaling of the output window.

For example, you could set an internal resolution of 320 x 240 and a window scale of 2, which results in the game content being rendered in multiple layers at 320 x 240, with all of those layers then getting upscaled to 640 x 480 as images.

This is what the crucial part of my main draw() loop looks like. The drawGame(), drawSand(), etc. methods draw the respective content to the layer, the res of which matches the internal resolution, and then all of the layers are drawn on top of each other, stretched to fit the window size.

case 1:
    background(0);
    if (init) startAnim(playMode);
    heldInputs();
    simulation();
    drawGame();
    drawSand();
    drawTrails();
    drawEffects();
    drawGameUI();
    image(gameLayer, 0, 0, WIDTH, GAME_HEIGHT);
    image(sandLayer, 0, 0, WIDTH, GAME_HEIGHT);
    image(trailLayer, 0 - (cameraX*WINDOW_SCALE), -(2*VSCREEN_GAME_HEIGHT) - (cameraY*WINDOW_SCALE));
    image(effectsLayer, 0, 0, WIDTH, GAME_HEIGHT);
    image(UILayer, 0, GAME_HEIGHT, WIDTH, HEIGHT);
//etc.

This is what drawGame() looks like, for instance:

void drawGame() {
gameLayer.beginDraw();

for (int y = 0; y < VSCREEN_GAME_HEIGHT; y++)
{
  gameLayer.stroke((y+cameraY)/1.5, 15, y+cameraY+150);
  gameLayer.line(0, y, VSCREEN_WIDTH, y);
}

for (Tank t : tanks) {
  if (t.getAlive()) t.drawTank(tankWidth);
}
gameLayer.endDraw();
}

Of course, not all of these drawSomething() calls are super optimised, but something I find very odd is how changing the output res actually has a very strong effect on performance. I get around 67 FPS when using a res of 640 x 480 at double window scale, but when I reduce the window scale to 1, I get a perfect 120 FPS (the target performance), even though the computing workload for the layers doesn’t change at all. So the culprits are the image() calls, right? but why does simply upscaling an image (and without any AA at that) seemingly cost so many resources? Could someone give me a hint on how to optimise this?

Many thanks for reading until the end and have a great day :+1:

(In case you’re curious, this is what the game currently looks like. This particular shows the game running in 640 x 480 at a window scale of 1. The separate layers are, from drawn first to last: Sky and Tanks, Sand, Trails (the dotted lines in the sky), Effects (includes the coloured name tags), UI)

Unbenannt
Unbenannt640×480 7.7 KB

2

Does the grid size of your sand simulation adapt to the resolution?

3

Yeah, but once again, just the internal resolution, not the output resolution. The sand grid is defined like this:

int[] sandGrid = new int[MAP_WIDTH * VSCREEN_GAME_HEIGHT];

…with MAP_WIDTH being the internal horizontal res multiplied by the factor of how big you want the map to be.
But this grid size is not affected by the output resolution, so doubling the window scale doesn’t have any effect on that.
The sand is also only exclusively rendered within the camera view, so any sand particles out of view are ignored.

I did do a performance test where I disabled the drawLayer() calls for each of the different layers, and skipping the sand rendering did result in a moderate 14% perf boost, but that’s not related to the sand simulation (which, on that note, is only active when something has exploded, so it’s usually dormant.)

4

Would you mind creating a minimal sketch that reproduces and isolates the performance issue you’re seeing? That would help diagnose the issue, and figure out whether that’s something maintainers need to look into.

5

Hello @Braunstein,

I was inspired by your topic and shared code in the Gallery:
Processing PGraphics Image Scaling and Renderer Test

Tatooine came to mind and I found the image (classic scene) I was remembering:

I saw the first movie in the theatre! Good memories.

I have done similar in the past and have encountered the same performance and other issues.

  1. JAVA2D uses CPU rendering; image(pg, a, b, c, d) resizing is slow for a PGraphic.

  2. P2D uses GPU rendering; image(pg, a, b, c, d) resizing is faster for a PGraphic.

  3. In JAVA2D you can also copy the PGraphic canvas directly into a PImage and resize which is faster than 1:

    PImage target = pgSun.get();  
    target.resize(900, 900);
    image(target, 0, 0);

It was getting late so scrutinize and test the content provided.

Have fun!

:)

6

Hello! @glv
That’s a really cool sample sketch you made. I’m too young to have watched the original trilogy in theatres, unfortunately :^) But I’m glad my little screenshot inspired you!

As for the problem- I didn’t know P2D uses the GPU instead! It is indeed much faster now. Though, a different problem has come up due to using P2D. P2D and P3D automatically use bilinear texture filtering when an image is upscaled, independent of smooth()/noSmooth().
This can actually be seen in your sketch as well, where increasing the size of the window results in a filtered image.
But the look I am trying to achieve needs nearest-neighbor filtering, or in other words: I don’t want any AA/filtering at all, just that raw pixel goodness.

Running at 320 x 240, upscaled to 640 x 480 (P2D renderer, yikes, not pretty)

1
1642×512 25.8 KB

Running at 320 x 240, upscaled to 640 x 480 (JAVA2D renderer, very beautiful crisp pixels)

2
2642×512 4.8 KB

The performance P2D gets is amazing, but ideally, I’d like to have the best of both worlds- the untreated image of JAVA2D and the performance of P2D.

I found some resources on how to deal with that, but most of them are refering to P3D and are also quite old. This is the best one I found:

Unfortunately, that doesn’t seem to work either. I’ve heard that the way OpenGL graphics are handled depends on the precise version of Processing? I don’t really have the technical knowledge behind that. For the record, I am using 4.0.1 at the moment (guess I haven’t updated in quite a while) and using textureSampling(2) does not disable the filtering.

Would you happen to know anything about how that works or if I should use a different version?
Have a great day :smiley:

7

I experienced the same problem when I was creating the G4P (first released in 2009). In G4P text drawn using the P2D / P3D renderers (both use OPENGL) was very poor compared to JAVA2D mode (which used JAVA AWT). My solution was to create graphics buffers (PGraphics) using the JAVA2D mode and then rendering these buufers on to a P2D or P3D sketch using OPENGL using the beginShape function somewhat like this

beginShape(TRIANGLE_STRIP);
vertex(120, 80);
vertex(340, 80);
vertex(120, 300);
vertex(340, 300);
endShape(CLOSE);

Also If you are working with bitmap images it is better to create them oversize and down-scale them rather than up-scale small bitmaps which can cause pixelisation effects.

8

Take a look at my examples in https://discourse.processing.org/t/texture-sampling-nearest-in-p2d-render/48399/2