Texture as data in shader has odd behavior

Dear community,

I’m learning how to use a shader, especially within p5js.
I’ve learned from this post how to pass a set of data as a texture and use it in the shader program.

I appreciate that @hamoid and @myselfhimself has shared their code for it :wink:

But after trying some variation with these codes, I found an odd behaviour from this.

This odd behaviour is visible at most in this sketch.

If you run this sketch, you might found that one from the 50 particles isn’t following the coordinate it gets. This one particle is somehow wandering and driven into a different way.

I’ve found that there is nothing wrong in the javascript part.
The most suspicious part for me is the shader, especially where the data is taken from the sampler2D by the texcoord. (line 29 of the sketch)

vec4 data = texture2D(tex, vec2(x, 0.0));

Could the texture coordinate be the problem, as this is somehow interpolated in a wrong way?
I honestly couldn’t find from where to start to debug this problem.
I’m still trying to write different versions of codes by using the same technique, but always resulting in the same problem.

I wonder how this problem could be sorted out.
Any tips would be appreciated! :slight_smile:

1 Like

Hi :slight_smile:

I looked into it but I can’t figure out why the oddness.

I wrote a slightly simplified version:

let img;
let fx;
let NUM_PARTICLES = 50;

function setup() {
  createCanvas(300, 300, WEBGL);  
  fx = createShader(`
    precision highp float; 
    attribute vec3 aPosition;
    void main() { 
      gl_Position = vec4(aPosition, 1.0); 
    }`, `
    precision highp float;
    uniform sampler2D tex;
    void main() {
      float bri = 0.0;
      float maxBri = 0.0009;
      vec2 pos = gl_FragCoord.xy / 300.0;
      for(int i=0; i<${NUM_PARTICLES}; i++) {
        vec2 data = texture2D(tex, vec2(float(i) / ${NUM_PARTICLES}.0, 0.0)).rg;
        bri += maxBri / distance(pos, data);
      }
      gl_FragColor = vec4(vec3(bri), 1.0);
    }`);
  noStroke();
  shader(fx);

  img = createImage(NUM_PARTICLES, 1);
}

function draw() {
  quad(-1, -1, 1, -1, 1, 1, -1, 1);
  img.loadPixels();
  let i = frameCount % NUM_PARTICLES;
  img.pixels[i*4+0] = int(255.0 * mouseX / width);
  img.pixels[i*4+1] = int(255.0 * (height-mouseY) / height);
  img.pixels[i*4+2] = 0;
  img.pixels[i*4+3] = 255; // alpha seems to affect R G and B (RGB multiplied by alpha?)
  img.updatePixels();
  fx.setUniform('tex', img);
}

It doesn’t make sense that one particle seems to move, as there’s no code to move particles: they are recycled and the mouse position is used to update one of the them. It would be interesting to rewrite it in Processing to see if the issue is still there.

The same issue is present in Processing:

color.frag

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

uniform sampler2D tex;

void main() {
  float bri = 0.0;
  float maxBri = 0.0009;
  vec2 pos = gl_FragCoord.xy / 300.0;
  for(int i=0; i<50; i++) {
    vec2 data = texture2D(tex, vec2(float(i) / 50.0, 0.0)).rg;
    bri += maxBri / distance(pos, data);
  }
  gl_FragColor = vec4(vec3(bri), 1.0);
}

sketch.pde

PShader fx;
PImage img;
int NUM_PARTICLES = 50;

void settings() {
  size(300, 300, P2D);
}
void setup() {
  fx = loadShader("color.frag");
  shader(fx);
  img = createImage(NUM_PARTICLES, 1, ARGB);
  noStroke();
}

void draw() {
  img.loadPixels();
  img.pixels[frameCount % NUM_PARTICLES] = color(
    255.0 * mouseX / width, 
    255.0 * (height-mouseY) / height, 
    0, 
    255);
  img.updatePixels();
  fx.set("tex", img);
  rect(0, 0, width, height);
}

Very interesting… a particle with personality… :slight_smile:

Appears to be texture filtering.

Using Hamoid’s example, add ((PGraphicsOpenGL)g).textureSampling(POINT); into the setup function.
I have used that before to preserve “blockyness” of an image (in other words, the texture is not filtered).

I am not sure how to do the same in p5js.
Possibly with gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR); or something like it. (I have not tried this)

1 Like

Nice! Thanks for sharing :slight_smile:

In p5.js this seems to fix the funny particle:

function setup() {
  let canvas = createCanvas(300, 300, WEBGL);
  
  // ... unrelated code not shown ...
  
  img = createImage(NUM_PARTICLES, 1);
  
  tex = canvas.getTexture(img);
  tex.setInterpolation(NEAREST, NEAREST); // <--- this
}

I found it here.

Interesting that the odd behavior is present with LINEAR but not with NEAREST (in p5.js).

1 Like

Hey, @noahbuddy @hamoid Big thanks for the help! :wink:

I didn’t knew that the interpolation mode could be adjusted :upside_down_face:

This really opens the the door wide into using shader!

1 Like