Wrong blending of flat elements in WEBGL mode

Hi,

When drawing text, or images, or other flat elements, the opacity of transparent areas breaks.

See for example how the text behaves here if drawn before or after a series of rects (simulating fog): type-load-google-fonts-webgl_a copy by JuanIrache -p5.js Web Editor If drawing it before the “fog“, the fog the area around letters takes the color of the background. If drawn after the fog, letters behind any semi-transparent rect disappear. The same happens when drawing images.

I’ve seen this discussed in some other places, but it either sounds like it was fixed in some update (but it does fail with the most recent versions) or the proposed solutions don’t seem to help. See for example here: https://stackoverflow.com/questions/75723397/how-can-i-make-type-backgrounds-in-p5-js-transparent

Disabling the depth test with drawingContext.disable(drawingContext.DEPTH_TEST) makes the text fully visible, but breaks the fog effect. Is there a known correct way of doing this or is it a bug?

Should I report this in the github repo instead?

Thanks!

Drawing transparency in 3D is very complicated and not something that either p5 or Processing is likely to support. The only correct way to implement it is to render all of the opaque surfaces first and then to render every transparent triangle ordered from back to front with respect to the current camera orientation.

I give a slightly more detailed answer here: https://discourse.processing.org/t/draw-order-based-transparency-issue/22675/6

1 Like

So this is a very common problem that every 3D engine deals with in its own way. The tl;dr is, there’s no easy answer across the board, but I added some examples using a library I made to combine a few techniques, but it’ll still require you to fine tune a bit because it’s not a silver bullet.

The long answer: transparency + blending is hard to get both correct and fast in all cases, so every 3D engine chooses some compromise. The reason for this is that lots of 3D engines deal with occlusion by recording the distance of the closest thing to the camera at every pixel; when you draw a new thing, if it’s going to be farther than that distance, it doesn’t get drawn. This breaks transparency because a semitransparent thing still ends up recording that distance, preventing anything else from drawing behind it. The solution is generally to draw in back-to-front order like you would in 2D mode, or often just draw the transparent things after all the opaque things (the compromise most games make.)

To make that easier, I made a little library to help. Just loading the library looks like this: type-load-google-fonts-webgl_a copy copy by davepagurek -p5.js Web Editor

It mooostly fixes the text that you draw before the semitransparent stuff. Not fully though; what it does here is it fully discards any completely transparent pixels. The semitransparent pixels around the antialiased edges are still a problem though, leaving a bit of a border.

With this library, anything with some (semi-)transparency that you draw, wrap it in drawTransparent(() => yourCodeHere()) and the library manages the depth sorting for you. But that also sorts each drawTransparent call back-to-front for you. If you have a single object that partially intersects other things like your text does, you’ll likely have to split it up into multiple draw calls. In this case, each letter is maybe the best you can reasonably do. Even still, because your rectangles are so big, you’ll get some glitchy clipping on each letter because each rectangle will still partially go over and partially go under the letters. But at least with the library doing the back-to-front drawing for you, you can more safely disable depth testing. That looks like this: type-load-google-fonts-webgl_a copy copy copy by davepagurek -p5.js Web Editor

(edit: better screenshot)

In case you want to see what other options are available in the wild, there’s a technique called depth peeling that gets you correct transparency regardless of what order you draw in, but it involves a lot more memory (you store many layers of your image as you draw to it) and can end up affecting performance a lot. Another option is to only draw opaque pixels, so if you have a semitransparent one, you randomly assign it full transparency or discard it weighted to the amount of transparency. This also works but gives you a grainy result unless you draw at a larger resolution and then sample the result down, which also affects performance/memory in general.

Hopefully that helped, sorry for the wall of text! let me know if I can elaborate on anything!

1 Like

On the other hand, if what you really want is not transparency but fog, then that can be done much more easily with a post-processing filter() assuming you have access to the depth buffer values.

@davepagurek, what’s the easiest way to slip in a fog shader with 2.0? Can we hook in a post-processing fragment shader to use the gl_FragCoord.z? Or would it be better to get the depth buffer from a Framebuffer and apply a filter() to average in the fog color based on depth?

1 Like

A fog shader is definitely doable for regular objects! Here’s a thing that modifies the regular material shader to add fog: Fog shader by davepagurek -p5.js Web Editor

It’s harder for text because we don’t yet expose the internal text shader for modification like that. So far that you’d probably be better off using the framebuffer + depth method that you described. Although you’ll probably still need to discard the fully-transparent pixels of the text, otherwise you’ll see fog being applied to the full rectangle each glyph is drawn into, not just the glyph itself. Importing the p5.transparency addon will do that for you. Another option could be to use textToModel to turn the font into regular geometry instead of using p5’s special font shader, and then it’ll work with the fog shader above. (The downside is that you have to pick a sample rate and size and such ahead of time instead of being able to zoom in and out at runtime without seeing the triangles.)

1 Like

Thanks both for the very informative replies!

The fog example was just to demonstrate the issue, but I’m trying to solve this on a much more complex sketch, so now my question is how to load the great p5.transparency library when using instance mode.

I think as long as you add the script tag for p5.transparency after p5 itself (or if you’re using a build system, put you import 'p5.transparency' after import p5 from 'p5') and then the internal shaders get patched, and p5 instances should get p5Instance.drawTransparent(() => { ... }) and .drawTwoSided(() => { ... }) methods.

Thanks. I didn’t realise this was also an NPM module. However when doing

import p5 from ‘p5’;
import ‘p5.transparency’;

I get

Uncaught ReferenceError: p5 is not defined
at p5__transparency.js?v=26367f5b:204:4

No matter if I use p5 version 1 or 2. Appreciate any help.

Edit: I can load the library successfully with script tags, but then it crashes with

Uncaught (in promise) TypeError: Cannot read properties of undefined (reading ‘includes’) e.Shader @ p5.transparency.min.js:1

Which is line

super(e, r, t.includes(“uniform bool isClipping”) etc

I just published a new version on npm that should hopefully resolve some issues: it now publishes a UMD module and checks for p5’s existence differently, so if just importing p5 doesn’t do anything because p5 isn’t globally available, you can import transparency from 'p5.transparency' and then (in p5 2.0) call p5.registerAddon(transparency) manually (in 1.x, call transparency(p5).)

Based on the error you’re seeing, it looks like that’s in 1.x, and that new Shader(renderer, vertSrc, fragSrc) got called with fragSrc being undefined somehow. What method of loading a shader are you using? Is there a chance that that’s not getting its inputs correctly?

I’m calling loadShader like this:

shader = p.loadShader(
‘static/shader.vert’,
‘static/shader.frag’,
() => {
// do something
},
console.error
);

This works until I plug p5.transparency.

Thank you for looking into it.

Also, version 0.0.15 (with p5 1.11.10) still fails when imported like this:

import p5 from ‘p5’;
import transparency from ‘p5.transparency’;

transparency(p5);

Gives error

Uncaught ReferenceError: p5 is not defined
at p5__transparency.js?v=4810082f:216:9

Which is line

`if (p5.registerAddon) {`

I sent a couple of pull requests to handle the problems I was having with p5.transparency. Once it works, it works like a charm!