I am trying to display a 2D overlay (head-up-display) over a 3D sketch using a shader. Now I have never created a shader before but have spent the last couple of days researching and experimenting.
I am very close in that I have a
3D animated sketch (p5js example)
2D overlay with transparent areas (created programmatically)
shader program (based on code from GeeksForGeeks)
Now it all works except for one thing, although the 2D overlay has transparent areas they are appear opaque
Rather than post the code here I have created a test sketch on p5js web editor that you can copy and experiment with.
The last line in the draw() function is
displayHUD(canvas, hudBuffer);
If this is commented out then you see the 3D animation in action, but if executed it shows the HUD but the transparent areas are opaque. It is not replacing the 3D display as occasionally you get a glimpse of the purple balls in front of it.
In webgl2, textures do not need to be powers of 2. They can have any resolution you like.
filter() would, more simply, replace the shader code that you wrote.
https://editor.p5js.org/scudly/sketches/vjPx7ElaX gives a really simple example of a filter shader that looks at each pixel of the existing image, which p5 passes in as tex0, and then outputs a new pixel. In my code, I’m dimming the brightness to fade the image, but you could pass in your HUD as a second texture uniform and mix() the pixels together based on the HUD image alpha.
But also, why do you need a shader at all? If you just want a text overlay on a 3-D scene you can use createGraphics()to make a non-webgl image, render your HUD including text to it, and image() it on with no shaders involved.
https://editor.p5js.org/scudly/sketches/Y6ldVcftN uses shaders to draw a bunch of points with sphere imposters on them in 3D and then uses a 2D p5.Graphics to overlay some text with a semi-transparent background.
The example code from GeeksForGeeks used webgl hence the power of 2. When I moved to code into a p5js sketch I discovered it used webgl2 but I didn’t realise that it removed the power-of-2 restriction.
So thanks very much for the information and I have updated my test sketch to correct my ignorance.
Your examples show that p5js provides several functions to simplify the creation and use of shaders but I want the code to use native JS code only. In other words without dependency on p5js.
The reason I am doing this is that I want my p5js library canvasGUI to be available to all Javascript programmers not just p5js users. To achieve that canvasGUI must have no external dependencies and only use native Javascript code.
It seems that webgl2 does not provide 2d graphics functionality so I was looking at shaders to provide the solution.
As I am an absolute newbie to shaders so it looks like I will have to do some more research.
In your displayHUD(), change the webgl code at the bottom to
// Draw the quad with the texture applied
gl.clear(gl.DEPTH_BUFFER_BIT);
gl.enable(gl.BLEND);
gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA);
gl.drawArrays(gl.TRIANGLE_FAN, 0, 4);
gl.disable(gl.BLEND);
You’re clearing the color buffer which you don’t want to do since you want to see the previously drawn geometry. Instead, you want to clear the depth buffer so that your HUD will be drawn on top. You also need to enable alpha blending and then put it back so that p5 can do its thing.
You are currently compiling the shader every frame. Better would be to compile it once, saving the program to a variable and then only useProgram()and set the uniforms each frame. Split your function so that the calls above useProgram()are only called once and from that line down is done each frame. The scaleX and scaleY should be passed in as a uniform vec2 resolution so they can change when the HUD size changes.
Simpler than shaders, though, it seems to me that you should be able to use just a combination of html and css to overlay one canvas (2D) over another (3D). This video https://youtu.be/TKHaDC49ITY?t=1474, for instance, uses html to make a toolbar overlay for his javascript Minecraft clone.
Just for completeness, some other options I could suggest:
Use framebuffers. If you draw your 3D scene to a framebuffer, then draw your HUD to a framebuffer, you could draw both to the main canvas as flat layers. This is kind of like using two p5.Graphics in 2D mode, if that’s a familiar point of reference – the main difference is that in a WebGL-only sketch, framebuffers use less data copying and will be faster. Additionally, you can draw to a framebuffer by sandwiching code between buffer.begin() and buffer.end() so it’s a little easier to use than adding a g. prefix to every call like you would need to do for a p5.Graphics.
Use createCamera() to create a separate camera for your HUD. Then, after drawing your 3D scene, you can render your HUD with:
push()
clearDepth() // So the HUD goes on top and won't get occluded by 3D content
setCamera(hudCamera)
// Draw your HUD here
pop()
Thanks to both @scudly and @davepagurek for your replies I think this discussion provides a good synopsis of the techniques for creating 2D overlays over a 3D canvas.
In canvasGUI V1 the controls were drawn directly onto the canvas so the code was cumbersome because each control needed two render methods for 2D and WEBGL. In V2 the gui is drawn to a 2D buffer and then drawn over the sketch window. There were other issues in particular I had to include code to test the for version of p5js so I could make the library compatible to both.
It seems to me that by using the underlying native JS contexts (WebGL2RenderingContext and CanvasRenderingContext2D) directly with shaders makes the library independent of the p5js version and makes it available to none p5js users. Shaders also have the advantage of being indifferent to the current model and perspective matrices.
I am continuing with my experiments with shaders not just for canvasGUI but because they seem fun and challenging to play with.
I have updated the sketch to seperate the shader creation / use into their own functions and the keys ‘r’, ‘g’ and ‘b’ changes the HUD buffer.
