Paint an image with transparency on a quad via a fragment shader

p5.js
1

Hi I would appreciate help implementing this feature for an art project I’m working on. Functionally it would be similar to a p5.Image filter() but using my own fragment shader instead of the built in filters.

So what I have is a “sprite sheet” image, which is multiple images packed into a larger image.
I want to draw a subsection of the sprite sheet image and run it through a fragment shader so I can modify the colors on the fly. That means I need to set the texture uv components.

shader.vert:

attribute vec3 aPosition;
attribute vec2 aTexCoord;

varying vec2 vTexCoord;

void main() {
    vTexCoord = aTexCoord;
    vec4 positionVec4 = vec4(aPosition, 1.0);
    gl_Position = positionVec4;
}

shader.frag

precision mediump float;

// ranges from 0..1
varying vec2 vTexCoord;
uniform sampler2D texture;

void main() {
    vec2 uv = vTexCoord;
    uv = 1.0 - uv;
    vec4 tex = texture2D(texture, uv);
    // just invert the texture color to demonstrate the fragement shader is running
    gl_FragColor = vec4(1.0-tex.rgb, tex.a);
}

implementation 1-- the shader is ignored

p5.shader(shader);
p5.image(image, x+ss.x, y+ss.y, frame.w, frame.h, xOffset, yOffset, frame.w, frame.h);

ss and frame and xOffset(s) are values that come from the image location on the larger sprite sheet image.

implementation 2-- shader is ignored.

p5.shader(shader);
p5.beginShape();
p5.texture(image);
p5.vertex(x+ss.x, y+ss.y, xOffset, yOffset);
p5.vertex(x+ss.x + frame.w, y+ss.y, xOffset+frame.w, yOffset);
p5.vertex(x+ss.x + frame.w, y+ss.y+frame.h, xOffset+frame.w, yOffset+frame.h);
p5.vertex(x+ss.x, y+ss.y+frame.h, xOffset, yOffset+frame.h);
p5.endShape();

I have tried other more in depth implementations but none work very well. Any help would be appreciated!

2

Two issues: The first is that there is a bug in p5.js where user defined shaders are never recognized as supporting textures, so the WEBGL renderer falls back to the built in shader; The second is that your shader needs to conform to the completely undocumented conventions for attribute and uniform naming in p5.js and apply some boilerplate transformation matrices to the vertex coordinates.

Fix for issue #1:

  shader.samplerIndex = 1;

Fix for issue #2:

shader.vert

uniform mat4 uModelViewMatrix;
uniform mat4 uProjectionMatrix;

attribute vec3 aPosition;
attribute vec2 aTexCoord;

varying vec2 vTexCoord;

void main() {
  vTexCoord = aTexCoord;

  // You need to apply these matrices for your shape to show up in the correct location
  vec4 viewModelPosition = uModelViewMatrix * vec4(aPosition, 1.0);
  gl_Position = uProjectionMatrix * viewModelPosition;
}

frag.vert

precision mediump float;

// ranges from 0..1
varying vec2 vTexCoord;

// Your texture uniform must use this name
uniform sampler2D uSampler;

void main() {
    vec4 tex = texture2D(uSampler, vTexCoord);
    // just invert the texture color to demonstrate the fragement shader is running
    gl_FragColor = vec4(1.0-tex.rgb, tex.a);
}

Note: I couldn’t make heads or tails of your vertex coordinates, so I just made that part up for my example. The documentation indicates that if you want to specify UV coordinates you need to specify x y and z first (it appeared you were trying vertex(x, y, u, v) and I’m not sure if that will work).

2 Likes
3

KumuPaul, thank you very much this is very helpful. I also encountered that bug you described, now I know why the shader was toggled on and off when I comment and uncomment the texture call.

This convention ought to be documented! Well thanks for sharing it with me.

The vertex coordinates aren’t important, you understood what I was trying to do perfectly. In vertex() documentation is says vertex(x, y, [z], u, v) in which the brackets imply that z is optional. I’ll try setting the z to zero.

Thanks.

4

Oh, my bad, you’re absolutely right about that [z].

I’ve opened an issue on GitHub regarding the samplerIndex issue: User defined shaders are never used when a texture is specified because isTextureShader always returns false. · Issue #5507 · processing/p5.js · GitHub

I agree about the documentation for custom shaders, it is something I have been meaning to contribute.

5

I answered here but the short answer is that when using the shader() function, you are expected to pass your own uniforms with .setUniform().

So in this case, you could remove the call to texture() and add shdr.setUniform("uSampler", img); after shader(shdr);

6

Also FWIW, there was an issue where some work had started on documenting the existing uniforms, but it never finished up. I reached out to see if it was still ongoing but I’m unsure of the current status.

1 Like