Texture as data in shader has odd behavior

Dear community,

I’m learning how to use a shader, especially within p5js.
I’ve learned from this post how to pass a set of data as a texture and use it in the shader program.

I appreciate that @hamoid and @myselfhimself has shared their code for it :wink:

But after trying some variation with these codes, I found an odd behaviour from this.

This odd behaviour is visible at most in this sketch.

If you run this sketch, you might found that one from the 50 particles isn’t following the coordinate it gets. This one particle is somehow wandering and driven into a different way.

I’ve found that there is nothing wrong in the javascript part.
The most suspicious part for me is the shader, especially where the data is taken from the sampler2D by the texcoord. (line 29 of the sketch)

vec4 data = texture2D(tex, vec2(x, 0.0));

Could the texture coordinate be the problem, as this is somehow interpolated in a wrong way?
I honestly couldn’t find from where to start to debug this problem.
I’m still trying to write different versions of codes by using the same technique, but always resulting in the same problem.

I wonder how this problem could be sorted out.
Any tips would be appreciated! :slight_smile:

1 Like

Hi :slight_smile:

I looked into it but I can’t figure out why the oddness.

I wrote a slightly simplified version:

let img;
let fx;
let NUM_PARTICLES = 50;

function setup() {
  createCanvas(300, 300, WEBGL);  
  fx = createShader(`
    precision highp float; 
    attribute vec3 aPosition;
    void main() { 
      gl_Position = vec4(aPosition, 1.0); 
    }`, `
    precision highp float;
    uniform sampler2D tex;
    void main() {
      float bri = 0.0;
      float maxBri = 0.0009;
      vec2 pos = gl_FragCoord.xy / 300.0;
      for(int i=0; i<${NUM_PARTICLES}; i++) {
        vec2 data = texture2D(tex, vec2(float(i) / ${NUM_PARTICLES}.0, 0.0)).rg;
        bri += maxBri / distance(pos, data);
      }
      gl_FragColor = vec4(vec3(bri), 1.0);
    }`);
  noStroke();
  shader(fx);

  img = createImage(NUM_PARTICLES, 1);
}

function draw() {
  quad(-1, -1, 1, -1, 1, 1, -1, 1);
  img.loadPixels();
  let i = frameCount % NUM_PARTICLES;
  img.pixels[i*4+0] = int(255.0 * mouseX / width);
  img.pixels[i*4+1] = int(255.0 * (height-mouseY) / height);
  img.pixels[i*4+2] = 0;
  img.pixels[i*4+3] = 255; // alpha seems to affect R G and B (RGB multiplied by alpha?)
  img.updatePixels();
  fx.setUniform('tex', img);
}

It doesn’t make sense that one particle seems to move, as there’s no code to move particles: they are recycled and the mouse position is used to update one of the them. It would be interesting to rewrite it in Processing to see if the issue is still there.

The same issue is present in Processing:

color.frag

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

uniform sampler2D tex;

void main() {
  float bri = 0.0;
  float maxBri = 0.0009;
  vec2 pos = gl_FragCoord.xy / 300.0;
  for(int i=0; i<50; i++) {
    vec2 data = texture2D(tex, vec2(float(i) / 50.0, 0.0)).rg;
    bri += maxBri / distance(pos, data);
  }
  gl_FragColor = vec4(vec3(bri), 1.0);
}

sketch.pde

PShader fx;
PImage img;
int NUM_PARTICLES = 50;

void settings() {
  size(300, 300, P2D);
}
void setup() {
  fx = loadShader("color.frag");
  shader(fx);
  img = createImage(NUM_PARTICLES, 1, ARGB);
  noStroke();
}

void draw() {
  img.loadPixels();
  img.pixels[frameCount % NUM_PARTICLES] = color(
    255.0 * mouseX / width, 
    255.0 * (height-mouseY) / height, 
    0, 
    255);
  img.updatePixels();
  fx.set("tex", img);
  rect(0, 0, width, height);
}

Very interesting… a particle with personality… :slight_smile:

Appears to be texture filtering.

Using Hamoid’s example, add ((PGraphicsOpenGL)g).textureSampling(POINT); into the setup function.
I have used that before to preserve “blockyness” of an image (in other words, the texture is not filtered).

I am not sure how to do the same in p5js.
Possibly with gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR); or something like it. (I have not tried this)

1 Like

Nice! Thanks for sharing :slight_smile:

In p5.js this seems to fix the funny particle:

function setup() {
  let canvas = createCanvas(300, 300, WEBGL);
  
  // ... unrelated code not shown ...
  
  img = createImage(NUM_PARTICLES, 1);
  
  tex = canvas.getTexture(img);
  tex.setInterpolation(NEAREST, NEAREST); // <--- this
}

I found it here.

Interesting that the odd behavior is present with LINEAR but not with NEAREST (in p5.js).

1 Like

Hey, @noahbuddy @hamoid Big thanks for the help! :wink:

I didn’t knew that the interpolation mode could be adjusted :upside_down_face:

This really opens the the door wide into using shader!

1 Like

Any idea why i get an error that say: cant find variable nearest ? Using the same code in my computer. It works fine in p5.editor

Hi! The forum says it was +2 years since you posted last time so welcome back :slight_smile:

So the code is identical in both cases? (same CASE?) If yes, maybe they are different versions of p5.js? The editor seems to be using v1.4.0.

Hello Hamoid! Yes! Thanks!
Im using the last version of p5. i think is 1.4.1. Any idea how can i make this work?

Could you maybe try if it works with 1.4.0? You can get it at the bottom of Release v1.4.0 · processing/p5.js · GitHub

Just to know if the version makes a difference or not. Also if you have more than one web browser you could try in all of them see if that has an effect.

I also test it in version 1.4.0 and same effect…Where i can find the reference of that function "tex.setInterpolation() ?

There seems to be no documentation for p5.Texture. I can try it here if you send me a zip file. E-mail at aBe. Contact

Thanks. I think is some problem with my computer…

but hey, look at this example

Im trying to do the same but with a pg.Graphics, but it seems that it doesnt work the nearest interpolation. Any idea? Notice that i need to get the EXACT values that i send to the shader, and i cant use arrays…

If i use the nearest interpolation i get the right values but i dont know how can i make it work with a graphic

I tried drawing to a pg.Graphics but unfortunately I couldn’t figure out why it’s all white :frowning:

1 Like

Try using texelFetch() with integer coordinates instead of texture2D() using texture coordinates. texelFetch doesn’t interpolate the texture data.

https://www.khronos.org/registry/OpenGL-Refpages/es3.0/html/texelFetch.xhtml

1 Like

I believe texelFetch() was introduced in ES 3.0 which requires WebGL 2.0 and currently p5.js uses WebGL 1.0. I don’t think setInterpolation() is going to help you in this case. There were numerous things wrong with your example, here’s what I think is a working version including the proper discrete pixel sampling:

1 Like

Thanks, i notice that it doest return the EXACT value, Although I access the array correctly, there is a slight distance between the value it sends and the value it receives. If it were a color it would be almost imperceptible, but if I send integer values ​​to receive angles for example, it becomes something visible. Does anyone know how I can correct it to receive the EXACT value?

Dealing with floating points it’s common to have to deal with small precisions errors. Checking for equality with floating points, other than perhaps 0.0, is almost always a mistake. If you are using colors to indicate discrete integer values I suggest you convert them to integers by multiplying and rounding.

int num = int(floor(255.0 * data.r + 0.5));
1 Like