Frozen Pixels in Shaders?

Processing Coding Questions
1

Hello, I’m a long time Processing user and I’m currently getting extremely strange behavior from a shader I’m developing. I’ve stripped it down to the core, and I’ve simply never seen anything similar. When using the shader as a filter, I’m getting “stuck pixels” which persist between frames. My main code is just boilerplate to display a rotating cube and apply the filter:

PShader AOShader;

void setup() {
  size(800,800,P3D);
  AOShader = loadShader("ambient.glsl"); 
}

void draw() {
  lights();
  
  float r = frameCount/100.0;
  camera(75*cos(r),100,75*sin(r),0,0,0,0,1,0);
  
  background(255);
  fill(128);
  box(50);
  
  filter(AOShader);
}

The filter itself is currently just hashing the coordinates of the pixel and using that to jitter the input (the full shader where I first observed this is a screen-space ambient occlusion implementation). To make sure the hash function wasn’t causing the issue, I’ve replaced it with a simple bad linear hash.

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

#define PROCESSING_TEXTURE_SHADER

uniform sampler2D texture;
uniform vec2 texOffset;

varying vec4 vertColor;
varying vec4 vertTexCoord;

// really using pcg3d16, but this shows the failure
float noise(int a, int b, int c) {
  return fract(float(a+b+c)*0.1);
}

void main(void) {

  int pi = int(vertTexCoord.x/texOffset.x);
  int pj = int(vertTexCoord.y/texOffset.y);

  float sf = 100.0;

  vec2 shift = sf*(2*vec2(noise(pi,pj,0),noise(pi,pj,1)-1));
  vec4 cur = texture2D(texture, vertTexCoord.st + shift*texOffset.st);

  gl_FragColor = cur;
}

The thing that makes this particularly odd is that the behavior changes as you change sf! If sf = 1.0, there are no frozen pixels and everything behaves as you’d expect. If sf = 10.0, then about 1-5% of the pixels are frozen at their value in the first frame. If sf=100.0, then all pixels are frozen in their value from the first frame. As far as I can tell, the first frame always renders as expected.

Anyone have any thoughts? I’m quite stuck. Mac OS 10.15.7, Processing 3.5.4.

2

Hi, welcome to the forum :slight_smile: I would say the likely cause is trying to read and write from the same texture during the same animation frame.

When you run the shader many instances will run in parallel processing pixels in various areas of the texture, but you can’t know in which order they will run, and the order will change depending on the GPU model. What should it do if you try copy pixel a onto pixel b and at the same time pixel b onto pixel a?

From GLSL Common mistakes:

Normally, you should not sample a texture and render to that same texture at the same time. This would give you undefined behavior. It might work on some GPUs and with some driver version but not others.

To solve this I believe you should use a ping-pong approach: have two textures and alternate: during odd frames (1, 3, 5…) read from texture A and write to texture B, on even frames (2, 4, 6…) read from B and write to A. That way you never read and write from the same texture during an animation frame.

4 Likes
3

In theory, filter(..) handles this for you.

1 Like
4

Ah really? How does it do it? Does it work internally with two textures?

5

Yes, kind of, via different layers of the FBO I think - there is a specific filterTexture field that is synced to before calling the shader - vaguely remember some odd issues - do always do the ping-pong approach myself - processing/PGraphicsOpenGL.java at master · processing/processing · GitHub (link is directly to the implementation line, even if it doesn’t look it!)

1 Like
6

Using a PGraphics I think makes it work:

PShader AOShader;
PGraphics pg;

void setup() {
  size(800,800,P2D);
  pg = createGraphics(width, height, P3D);
  AOShader = loadShader("ambient.glsl");
  shader(AOShader);
}

void draw() {
  pg.beginDraw();
  pg.lights();
  
  float r = frameCount/100.0;
  pg.camera(75*cos(r),100,75*sin(r),0,0,0,0,1,0);
  
  pg.background(255);
  pg.fill(128);
  pg.box(50);
  pg.endDraw();
  
  image(pg, 0, 0);  
}

After tweaking some values looks like this and no pixels are stuck:

image
image685×680 36.5 KB

3 Likes
7

Comments for the filter() methods in the source code linked by @neilcsmith

/**
   * This is really inefficient and not a good idea in OpenGL. Use get() and
   * set() with a smaller image area, or call the filter on an image instead,
   * and then draw that.
   */
8

You’re looking at the wrong (non-shader) filter method!

9

:man_facepalming:

Ok ignore 50% of what I write hahah :rofl:

10

Well the other 50% is gold! :grinning_face_with_smiling_eyes:

1 Like
11

Interesting, thanks! Odd that I need to run it in a PGraphics, but that does seem to fix the issue. I had assumed filter was reading and writing from different buffers, but I guess that was not the case! This also explains why I’ve been able to use this exact same trick just fine before ;).