Any way to write to just the R, G, or B channel of the canvas? blendMode(ADD) is not working as expected

[neel]

Hey all. I’m trying to use blendMode() in a pretty specific way, but I’m getting results I wouldn’t expect. I would like to draw random red rectangles (255,0,0) and green rectangles (0,255,0) and then use a fragment shader to assign a color to a pixel based on whether it is red, green, red and green, or neither. Here’s my processing code.

PShader testShader;

void setup() {
 size(1000,1000,P2D);
 testShader = loadShader("test.frag");
}


void draw() {
  noStroke();
  shader(testShader);
  blendMode(REPLACE);
  fill(0,0);
  rect(0,0,width,height);
  blendMode(ADD);
  fill(255,0,0);
  rect(0,0,750,750);
  fill(0,255,0);
  rect(250,250,1000,1000);
}

And here is my shader…

#ifdef GL_ES
precision mediump float;
#endif

uniform vec2 u_resolution;

varying vec4 vertColor;
varying vec4 vertTexCoord;



void main() {
    vec3 color;

    if (vertColor[0] == 1 && vertColor[1] == 1) {
        color = vec3(151,171,128);
    }
    else
    if (vertColor[0] == 1 && vertColor[1] == 0) {
        color = vec3(182,217,208);
    } else
    if (vertColor[0] == 0 && vertColor[1] == 1) {
        color = vec3(217,106,73);
    } else
    if (vertColor[0] == 0 && vertColor[1] == 0) {
        color = vec3(26,71,84);
    }

    gl_FragColor = vec4(color/255.,1.);
    gl_FragColor =  vertColor;
}

If I uncomment that last line and set gl_FragColor to vertColor, I get a red box in the top left, a green box in the bottom right, and a yellow box where they overlap. This is what I’d expect, and the yellow in the middle looks like (255,255,0).

If I comment that last line, the pixel color should be set to one of the four color values I have hard coded. The background would be a dark teal, the top left would be a light blue, the bottom right would be an orange/red, and the middle would be great. The dark teal is the only one of these colors that look the way I’d expect. The blue and red are lighter than expected, and the green color is totally white.

I can’t tell what’s going on, but I would expect vertColor to be (1,1,0) in the middle, thus resulting in a green pixel due to the logic I’ve implemented.

Alternatively, if there is another way to write to just the red channel of a canvas, I would love to know about that. I’m essentially trying to write multiple monochrome canvases so that I can do things with those multiple monochrome canvases in glsl.

[neel]

I figured out why this wasn’t working with the help of Piter Pasma (generative artist on instagram).

Fragment shaders are called at the shape level rather than the canvas level. Blending happens at a later stage. I can’t expect to operate with the colors from two separate shapes in a fragment shader.

Instead, I’ll likely create a canvas that has the ADD blend mode applied, draw my shapes, and then import that into my shader as a texture.

[neilcsmith]

Note that you can do blending in a shader though, if you pass in two or more offscreen graphics or images as textures - I realise not quite the same thing, but might still be useful depending on what you’re trying to do - see eg. https://github.com/processing/processing-docs/tree/master/content/examples/Topics/Shaders/CustomBlend

[neel]

Thanks, neilcsmith!

So I’ve actually been trying this, and I find that using get() on a pgraphics object and passing it into a shader as a texture absolutely destroys my frame rate. When I pass my first canvas through a second one in this code, my framerate goes from 60 to 30. When I do it twice, I go down to 20 fps. Any thoughts on how this process can be optimized?

pde

 PGraphics canvas0;
PGraphics canvas1;
PGraphics canvas2;
PGraphics canvas3;

PShader testShader;

void setup() {
 size(1000,1000,P2D);
 canvas0 = createGraphics(1000,1000,P2D);
 canvas1 = createGraphics(1000,1000,P2D);
 canvas2 = createGraphics(1000,1000,P2D);
 canvas3 = createGraphics(1000,1000,P2D);

 testShader = loadShader("test.frag");
}


void draw() {



  canvas0.beginDraw();
  canvas0.blendMode(REPLACE);
  canvas0.fill(0);
  canvas0.rect(0,0,width,height);
  canvas0.blendMode(ADD);
  canvas0.fill(255,0,0);
  canvas0.rect(0,0,750,750);
  canvas0.fill(0,255,0);
  canvas0.rect(250,250,1000,1000);
  canvas0.endDraw();

  canvas1.beginDraw();
  canvas1.shader(testShader);
  testShader.set("u_canvas",canvas0.get());
  canvas1.fill(0);
  canvas1.rect(0,0,width,height);
  canvas1.endDraw();

  canvas2.beginDraw();
  canvas2.shader(testShader);
  testShader.set("u_canvas",canvas1.get());
  canvas2.fill(0);
  canvas2.rect(0,0,width,height);
  canvas2.endDraw();

  image(canvas2,0,0);
  text(frameRate,10,10);
}

frag

        #ifdef GL_ES
        precision mediump float;
        #endif

        uniform vec2 u_resolution;
        uniform sampler2D u_canvas;



        void main() {
            vec2 st = gl_FragCoord.xy/u_resolution.xy;
            gl_FragColor =  texture2D(u_canvas, st);
        }

[neilcsmith]

@neel

So I’ve actually been trying this, and I find that using get() on a pgraphics object

I’m not surprised! Why are you calling get() on it?! This will cause the pixel data to be downloaded and reuploaded to the GPU. A PGraphics is a PImage - just pass it in as is.

Incidentally, also be aware of this issue - your texture coordinates might be different than you expect (upside down) Add matrix options for PImage uniforms in PShader · Issue #3437 · processing/processing · GitHub

[neel]

Wow. That made a substantial difference.

I actually made another post about this, in the hopes that it would be more identifiable for future searchers. Hopefully that wasn’t poor form for this forum.

Slow performance when passing multiple PGraphics into each other as shader textures

Hey all, I’m attempting to use multiple pgraphics objects for my code. I want to draw some shapes on the first pgraphics. I then want to use a shader on the second pgraphics object to apply blur to the red channel and to generate a feedback loop by saving the previous version of this pgraphics and feeding it in as a texture. Finally, I want to feed the output of the second pgraphics into the third pgraphics so that I can apply some color transformations. This process seems to be tanking my fra…

I’ll post an update there and credit you. Thank you again!

In my actual code (not simple enough to post), I use get() in order to save the previous image for one of my PGraphics objects and feed it back into itself for feedback. Is there some way to do that without calling get()?

[neilcsmith]

@neel

I use get() in order to save the previous image for one of my PGraphics objects and feed it back into itself for feedback. Is there some way to do that without calling get()?

Yes, use PGraphics and image(...) - just draw one on the other. Remember, a PGraphics is a PImage!

[neel]

Wow. The impact that this has had on the code’s performance is insane. Thanks again for the explanation.

Now, in order to get feedback, I’m doing the following.

    PImage previous_state;
    previous_state = canvas2;

    canvas2.shader(shader);
    shader.set("u_canvas",canvas1);
    gridlock_shader.set("u_previousr",previous_state);

    canvas2.beginDraw();
    canvas2.rect(0,0,width,height);
    canvas2.endDraw();

My whole code (which includes other elements) went from running at 14 fps to 60. I guess I was under the assumption that get() made the information usable. Now I realize that the information was always usable, but get() changes where that information lives. That’s an expensive step that I can avoid for basically all of my code.

Thank you!