Compositing of two blurred shapes drawn into PGraphics

[stoph]

Hello, everyone!
What I want to achieve is having two differently colored blurred shapes in one PGraphics. The problem is that when copying the latter shape/image into the first one the semi-transparent pixels (blurred edges) are composited with the background color of the first image and don’t keep their original color. In the screenshot below the black border around the white shape is what I want to avoid.

I know this can be done perfectly when drawing one after the other onto the main canvas, but as my actual sketch is a little more complex, I need a solution that doesn’t involve the main canvas.

I tried and read a lot already and have the stupid feeling I’m missing something very basic here, as this is something that comes natural with for example Photoshop.
Any help is appreciated! If this is already solved or explained somewhere else, I’m totally fine with a link.

// PShader taken from Examples->Topics->Shaders->SepBlur

PGraphics pg1, pg2;
PShader blur;

void setup() {
  size(500, 500, P2D);
  blur = loadShader("blur.glsl");
  blur.set("blurSize", 20);
  blur.set("sigma", 5f);
  
  pg1 = createPGfx(255);
  pg2 = createPGfx(0);
  
  pg2.beginDraw();
  pg2.image(pg1, 100, 100);
  pg2.endDraw();
}

void draw() {
  background(127);
  image(pg2, -50, -50);
}

PGraphics createPGfx(color c) {
  PGraphics pg = createGraphics(width, height, P2D);
  pg.noSmooth();
  pg.beginDraw();
  pg.noStroke();
  pg.fill(c, 255);
  pg.background(c, 0);
  pg.rectMode(CENTER);
  pg.rect(width/2, height/2, width/2, height/2);
  blur.set("horizontalPass", 0);
  pg.filter(blur);
  blur.set("horizontalPass", 1);
  pg.filter(blur);
  pg.endDraw();

  return pg;
}

[neilcsmith]

This “rant” of mine might be relevant - https://github.com/processing/processing/issues/3391

Blending is broken in Processing. It would work OK in PraxisLIVE which uses extensions of the standard renderers with workarounds for this.

[kfrajer]

@stoph

In the screenshot below the black border around the white shape is what I want to avoid.

Removing this line pg.background(c, 0); in createPGfx() does the trick, so I understand from your comment.

Kf

[neilcsmith]

Removing that line should make it even worse. It’s a hack that’s making the blur filter bit work, although only if you’re blurring a single coloured shape! This would then probably be ok if both offscreen graphics were drawn to the screen.

The problem is not that Processing uses non-premultiplied alpha over premultiplied, but that it mixes them. Processing expects input images to have non-premultiplied alpha. However, the output of any OpenGL blending is premultiplied. So when the offscreen gets drawn to the screen, the translucent colours around the blurred shape get multiplied by the alpha a second time, and you get a dark halo.

[kfrajer]

@neilcsmith The border, dark halo was not present when I removed that line in my test. I wouldn’t know how to test for premultiplied alpha. I can propose looking at the alpha channels of both PGraphics before and after the mixing and the resultant alpha values stored in the main PGraphics g. I will wait for the OP(@stoph ) to come back and confirm if the black halo is gone or further testing is required. If it is a bug, I could check for the ticket in GH and added this to it.

Kf

[neilcsmith]

@kfrajer

The border, dark halo was not present when I removed that line in my test.

True! But now you get a pale halo around the black shape instead!

@kfrajer

I wouldn’t know how to test for premultiplied alpha. I can propose looking at the alpha channels of both PGraphics before and after the mixing and the resultant alpha values stored in the main PGraphics g

Looking at the alpha channels would tell you nothing! Pre-multiplied alpha actually means that the colour values are already multiplied by the value of alpha. All blending requires doing this - see eg. slide #16 at Blend Modes for OpenGL | PPT - or in fact look at all of it, it’s useful info.

I was surprised to see the transparent colour starting to leak through though (try changing the colours to blue/red in that sketch)- that’s not what I expected just from the pre-multiplied issue alone. There also appears to be a bug at processing/core/src/processing/opengl/PGraphicsOpenGL.java at master · processing/processing · GitHub I think that should be

pgl.blendFuncSeparate(PGL.SRC_ALPHA, PGL.ONE_MINUS_SRC_ALPHA,
PGL.ONE, PGL.ONE_MINUS_SRC_ALPHA);

which brings it in line with the behaviour on that slide. Of course the output of that is pre-multiplied and nothing you can do about it.

[stoph]

Hey, thanks a lot to both of you for replying!

And yes, removing the

pg.background(c, 0);

makes BOTH layers fade to gray, which is the default background color value of transparent pixels in PGraphics. This can’t be seen in the sketch if you just remove that line, because the background of the canvas is gray, but if you’d change it to red for example (that’s at least how things made sense to me).

So that’s what you mean by this:

@neilcsmith

I was surprised to see the transparent colour starting to leak through though

…right?

And concerning this part:

@neilcsmith

It’s a hack that’s making the blur filter bit work, although only if you’re blurring a single coloured shape! This would then probably be ok if both offscreen graphics were drawn to the screen.

…yes, it works out nicely if you draw single layers consecutively to the main canvas, like I said in the beginning. But now I know why

@neilcsmith: Unfortunately I don’t really understand most of the things you wrote, but I’ll try to catch up over the weekend! Never felt the need to think about premultiplied alpha…

[stoph]

@neilcsmith: You said “there’s nothing you can do about it” but change the program (e.g. PraxisLIVE).
Well, I started to learn vvvv in the beginning of last year and now getting deeper into processing than I actually wanted to, I’m not gonna learn another programm. At least for now

But I wondered if there’s a good way to manually composit PGraphics’ maybe.
My first approach is in the code below but the result is still far from perfect, which might have a lot to do with me and my almost virgin skills in pixel manipulation…
Is there resources you can recommend which aren’t technically too avanced?
Or is it just not the right way to adress this?

PGraphics pg1, pg2;
PShader blur;
color colorFront = color(255), colorBack = color(0);

void setup() {
  size(500, 500, P2D);
  blur = loadShader("blur.glsl"); // taken from Examples->Topics->Shaders->SepBlur
  blur.set("blurSize", 20);
  blur.set("sigma", 5f);
  
  pg1 = createPGfx(100, colorFront);
  pg2 = createPGfx(0, colorBack);
  
  //"manual" compositing, 
  //draw a mix of top- & bottom-layer pixels into bottom-layer:  
  pg1.loadPixels();
  pg2.beginDraw();
  pg2.loadPixels();
  for (int x = 0; x < pg2.width; x++) {
    for (int y = 0; y < pg2.height; y++ ) {
      int loc = x + y*pg2.width;
      float alphaFront = alpha(pg1.pixels[loc]);
      float alphaBack = alpha(pg2.pixels[loc]);
      // copy opaque pixels from top-layer into bottom-layer:
      if (alphaFront == 255) {
        pg2.pixels[loc] = color(colorFront, 255);
      }
      // calculate new pixel color if front is
      // partly transparent (blurred edges & antialiasing):
      else if (alphaFront > 0) {
        color mix;
        // get the factor for lerpColor by looking at the alpha values.
        // limit alphaBack to remainder of 255-alphaFront, so front color stays dominant
        float alphaBackConstrained = constrain(alphaBack, 0, 255-alphaFront);
        // use constrained value to get relation between front and back color,
        // sorry, dunno how to explain the next line, but it kind of makes sense to me at least 
        float dif = (alphaBackConstrained - alphaFront) / (alphaBackConstrained + alphaFront);
        // map result to {0 <= x <= 1} for lerp
        float lerpFactor = map(dif, -1, 1, 0, 1);
        mix = lerpColor(colorFront, colorBack, lerpFactor);
        // find new alpha value the way Photoshop calculates it 
        int mixAlpha = round(alphaBack + alphaFront - (alphaFront*alphaBack)/255);
        pg2.pixels[loc] = color(mix, mixAlpha);
      }
    }
  }
  pg2.updatePixels();
  pg2.endDraw();  
}

void draw() {
  background(127);
  image(pg2, -50, -50);
}

PGraphics createPGfx(int posXY, color c) {
  PGraphics pg = createGraphics(width, height, P2D);
  pg.noSmooth();
  pg.beginDraw();
  pg.noStroke();
  pg.fill(c, 255);
  pg.background(c, 0);
  pg.rectMode(CENTER);
  pg.rect(width/2+posXY, height/2+posXY, width/2, height/2);
  blur.set("horizontalPass", 0);
  pg.filter(blur);
  blur.set("horizontalPass", 1);
  pg.filter(blur);
  pg.endDraw();

  return pg;
}

Result:

The slightly darker border around the white rect is just jpg-compression, in the renderer it’s not there, but there are obviously problems in the areas where black and white squares meet on the outside and I don’t understand why.
I reckon with more complex shapes and/or overlapping this can hardly be called “production quality”, which is what I want to achieve, of course

[neilcsmith]

@stoph well, PraxisLIVE is still Processing, just with nodes and code, multiple sketches, live coding … and a fixed renderer. Coming from vvvv you’d probably get on well with it!

However, you’re also free to borrow the renderer extensions - I think someone else has done that, or at least asked me about it. Ideally file a bug! I’ve offered multiple times to help upstream with this, but until it’s accepted as an issue that’s not going to happen.