Fast image cropping in Processing

Processing Coding Questions
1

Hello !

I am currently working on a project that is quite large and I would like to optimize parts of it as soon as possible.

In the course of this project, I need to crop some quite large images (around 2K) to a size (and cropping position) that can vary from frame to frame. Of course my initial idea is to use get(int x, int y, int width, int height) in this way :

frame.image(myImage.get(x, y, w, h), x, y);

(note : here, myImage is a PGraphics object)

The issue is that commenting this line leads to a shift from around 45 FPS to 30 FPS. Is there a way to achieve this behaviour with less of a FrameRate drop ?

Thanks for the help !

UPDATE :

By switching to the following line :

frame.image(myImage.get(), x, y, w, h, x, y, x+w, x+h);

I managed to get the following FPS (Average over 167 frames) :

  • With get cropping : 33.231 FPS
  • With GPU accelerated image cropping : 30.785 FPS
  • Without image cropping (Max possible FPS) : 44.269 FPS

So this solution seems to be worse…

2

That should be -

frame.image(myImage, x, y, w, h, x, y, x+w, x+h);

A PGraphics is a PImage. Calling get() on it is going to take it off the GPU.

3

Hmmmm… Alright so I just tried that, issue is that myImage is actually a P2D PGraphics object that is rendered using a shader. I don’t know if this has anything to do with it but when I don’t use the get() function on it before feeding it onto frame.image(), nothing is drawn…

Perhaps drawing myImage on a non-P2D PGraphics object before drawing it with frame.image() could work ? I’ll try it.

Thanks for the response btw :slight_smile:

UPDATE : Tried doing this, but again, not using the get function makes the image not displayed. Are shaders calculated when get() is called ?

4

Are you making sure to call beginDraw, and particularly endDraw, on the PGraphics? EDIT - try sharing more of your setup / drawing code.

5

Well, here is the class I’m using (it’s quite long so I’ll remove setters and getters) :

public class AlphaOverlay{
	public static final float FILL_1 = 240;
	public static final float FILL_2 = 255;
	
	private final PolygonEditor context;

	public PGraphics image;

	private PShader alphaOverlayShader;

	private int scale;
	
	private int w;
	private int h;
	
	private int x0;
	private int y0;
	
	public AlphaOverlay(PolygonEditor context, int x0, int y0, int w, int h, int scale){
		this.context = context;
		
		this.x0 = x0;
		this.y0 = y0;
		
		this.scale = scale;

		updateSize(w, h);
		
		if(PolygonEditor.shaderAllowed) {
			alphaOverlayShader = this.context.loadShader("shaders/glslAlphaOverlay.glsl");
			
			float c1 = FILL_1 / 256.0f;
			alphaOverlayShader.set("fill_1", c1, c1, c1, 1.0f);
			
			float c2 = FILL_2 / 256.0f;
			alphaOverlayShader.set("fill_2", c2, c2, c2, 1.0f);
			
			alphaOverlayShader.set("scale", scale);
			alphaOverlayShader.set("initial_position", x0, y0);
		}
	}

	public void updateInitialPos(int x0, int y0){
		this.x0 = x0;
		this.y0 = y0;
		alphaOverlayShader.set("initial_position", x0, y0);
	}

	public void updateSize(int w, int h){
		this.w = w;
		this.h = h;
		if(PolygonEditor.shaderAllowed) {
			try {
				this.image = context.createGraphics(this.w, this.h, PConstants.P2D);
			}
			catch(Exception e) {
				this.image = context.createGraphics(this.w, this.h);
				System.err.println("Unable to load OpenGL Graphics Object...");
				PolygonEditor.shaderAllowed = false;
			}
		}
		else this.image = context.createGraphics(this.w, this.h);
	}

	public void render(){
		if(PolygonEditor.shaderAllowed) {
			image.beginDraw();
			image.background(0);
			image.endDraw();
			image.filter(alphaOverlayShader);
		}
		else {
			image.beginDraw();
			image.noStroke();
			image.background(FILL_1);

			for(int xPos = 0; xPos < w/scale; xPos++) {
				for(int yPos = 0; yPos < h/scale; yPos++) {
					image.fill(FILL_1);
					if((xPos+yPos) % 2 == 0) image.fill(FILL_2);

					image.rect(scale*xPos, scale*yPos, scale, scale);
				}
			}
			image.endDraw();
		}
	}
	
	public PGraphics getImage() {
		return this.image;
	}
}

Thing is, I’ve really been struggling with drawing image using my shader so on the render function, if shaders are allowed, the drawing procedure looks like :

image.beginDraw();
image.background(0);
image.endDraw();
image.filter(alphaOverlayShader);

Perhaps this is wrong, but using filter() inside beginDraw and endDraw, the image isn’t displayed, and removing the background call, the image isn’t displayed either… Furthermore, not using get to display image on a non P2D PGraphics object, the image isn’t drawn either… I’ve been stuck on this for quite a while actually :confused:

Thanks for the help

Oh and I’ll also add the shader code just in case :

// Created by Ruben JAOUI, 28/11/2020
// Basic Checkerboard Shader

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

uniform vec4 fill_1;
uniform vec4 fill_2;

uniform ivec2 initial_position;

uniform int scale;

void main() {
    gl_FragColor = mix(
		fill_1,
		fill_2, 
		float(
			(int(step(scale/2, (int(gl_FragCoord.x) - initial_position.x) % scale)) +
			 int(step(scale/2, (int(gl_FragCoord.y) - initial_position.y) % scale)))  %2));
}
6

Nothing immediately obvious jumping out, but I might have missed something. However, changing one of the shader examples to use filter on an offscreen graphics and then drawing that seems to work fine. So, I would suggest a very simple sketch with an offscreen graphics and a basic shader (no uniforms, constant colour?) and check if that works for you. If it does, then try adding in this shader and other aspects of your project to see where exactly it breaks.

1 Like
7

Alright I’ll try that, thanks :slight_smile: