Most efficient way of incrementing the entire sketches pixels saturation values?

p5.js Coding Questions
1

Hi All,

I’m currently trying to develop a sketch where you can rub out a colour from the screen, but it keeps fading back from black to the colour.

You can see an index of experiments here:
Sketches here

The sketch in question is here:
Screen fades back - caution - will slow your computer right down!

You can see the source code here:
GitHub link to source code

		loadPixels();
	for (let y = 0; y < height; y++) {
		for (let x = 0; x < width; x++) {
			for (let i = 0; i < d; i++) {
				for (let j = 0; j < d; j++) {
					// if the colour of the current pixel matches the given colour, then increment numberOfPixelsThatMatchGivenColour
					index = 4 * ((y * d + j) * width * d + (x * d + i));
					currentColour = color(
						pixels[index],
						pixels[index + 1],
						pixels[index + 2],
						pixels[index + 3]
					);

					if (
						pixels[index] == redTarget &&
						pixels[index + 1] == greenTarget &&
						pixels[index + 2] == blueTarget
					) {
						//do nothing, we are already there
					} else {
						//this is bad because we lerp forever, never quite getting there...
						let newColor = lerpColor(currentColour, targetColour, lerpAmount);

						pixels[index] = red(newColor);
						pixels[index + 1] = green(newColor);
						pixels[index + 2] = blue(newColor);
						pixels[index + 3] = alpha(newColor);
					}
				}
			}
		}
	}
	//https://p5js.org/reference/#/p5/updatePixels
	updatePixels();

It’s very inefficient to have to check each pixel on the screen - I was hoping to be able to get access to the pixels as an HSB rather than RGBA array - that way I could just increment each pixels saturation to fade from black to the foreground colour, but this doesn’t seem to be possible.

Should I be using a shader? Should I be drawing in a smaller offscreen buffer and then scaling to the size of the screen? All suggestions gratefully received!

Thanks,

Joel

2

Here is some code to play with that when you click and hold the mouse button, fades the image to gray within a circle around the pointer and then re-saturates when you release or move the circle.

A challenge when fading images is that the components are stored as 8-bit integers and have to change by at least 1.0 or it won’t register as a change. Consequently, if you want to fade from full white to black, you can’t have the fade take longer than 256 frames. So, I made a function called move() that walks with linear steps from one color to another but which must have the step size be at least 1/256 to work. (In shaders, colors go from 0.0 to 1.0 rather than from 0 to 255.)

Edit: it’s worse that I first thought. If you step from black to white, the vector moves along the diagonal of the cube, so your step size has to be at least sqrt(3)/256 to guarantee that the image has a change. So, 0.007 is the smallest fade increment you should use.

ppixels is a texture that Processing provides to shaders that contains the previous frame’s image. Shaders have 0,0 at the lower left while Processing frustratingly puts it at the upper left, so we have to index into ppixels by flipping the y-coordinate. Apparently texture coordinates on rect() don’t flip, so that’s why there is both uv and ppuv.

PImage img;
PShader shdr;

void setup() {
  size( 1041, 1041, P2D );
  img = loadImage("https://www.nasa.gov/sites/default/files/thumbnails/image/e-pia00104-venus-full-alt-1041.jpg");

  //image( img, 0, 0 );

  shdr = new PShader( g.parent, vertSrc, fragSrc );
  shdr.set( "texture", img );
}

void draw() {
  if( frameCount < 2 ) image( img, 0, 0 );
  else {
  shdr.set( "mouse", float(mouseX)/width, float(mouseY)/height, mousePressed?1.:0. );
  noStroke();
  shader( shdr );
  rect( 0, 0, width, height );
  resetShader();
  }
}


String[] vertSrc = { """
#version 120
uniform mat4 transformMatrix;
uniform mat4 texMatrix;
in vec4 position;
in vec2 texCoord;
varying vec4 vertTexCoord;

void main() {
  gl_Position = transformMatrix * position;
  vertTexCoord = texMatrix * vec4( texCoord, 1.0, 1.0 );
  //vertTexCoord.y = 1.0 - vertTexCoord.y;
}
""" };


String[] fragSrc = { """
#version 120
uniform sampler2D ppixels;
uniform sampler2D texture;
uniform vec3 mouse;
in vec4 vertTexCoord;

vec3 move( vec3 c0, vec3 c1, float d ) {
  vec3 v = c1 - c0;
  return length( v ) < d ? c1 : c0 + normalize(v) * d;
}

void main() {
  vec2 uv = vertTexCoord.st;
  vec2 ppuv = vec2( uv.x, 1.0-uv.y );
  vec3 col = texture2D( texture, uv ).rgb;
  float gray = 0.2126 * col.r + 0.7152 * col.g + 0.0722 * col.b;
  if( mouse.z > 0.5 && length( uv-mouse.xy ) < 0.1 ) {
    col = move( texture2D( ppixels, ppuv ).rgb, vec3( gray ), 0.05 );
  } else {
    col = move( texture2D( ppixels, ppuv ).rgb, col, 0.007 );
  }
  gl_FragColor = vec4( col, 1. );
}
""" };
3

Ah, here’s a version that works better. Create a separate image (PGraphics) that acts as the saturation map (actually DEsaturation since 0 means we show full color). It stores a value spread across both the green and blue colors so we get 16 bit values to work with – could be extended to red as well if we needed it. Its shader uses the mouse position to increase each pixel’s value, otherwise it slowly drops to zero. The image shader uses the saturation map to draw each image pixel scaled to gray by the map value.

With the greater resolution, we can have nice long slow fades.

PImage img;
PGraphics satImg;
PShader satShdr;
PShader shdr;

void settings() {
  img = loadImage("https://www.nasa.gov/sites/default/files/thumbnails/image/e-pia00104-venus-full-alt-1041.jpg");
  size( img.width, img.height, P2D );
}

void setup() {
  satImg = createGraphics( width, height, P2D );
  satShdr = new PShader( g.parent, vertSrc, satFragSrc );

  shdr = new PShader( g.parent, vertSrc, fragSrc );
  shdr.set( "img", img );
}

void draw() {
  satShdr.set( "mouse", float(mouseX)/width, 1.0-float(mouseY)/height, mousePressed?1.:0. );
  satImg.beginDraw();
  satImg.noStroke();
  satImg.shader( satShdr );
  satImg.rect( 0, 0, width, height );
  satImg.endDraw();

  shdr.set( "satImg", satImg );
  noStroke();
  shader( shdr );
  rect( 0, 0, width, height );
  resetShader();
}


String[] vertSrc = { """
#version 120
uniform mat4 transformMatrix;
uniform mat4 texMatrix;
in vec4 position;
in vec2 texCoord;
varying vec4 vertTexCoord;

void main() {
  gl_Position = transformMatrix * position;
  vertTexCoord = texMatrix * vec4( texCoord, 1.0, 1.0 );
  //vertTexCoord.y = 1.0 - vertTexCoord.y;
}
""" };


String[] satFragSrc = { """
#version 120
uniform sampler2D ppixels;
uniform vec3 mouse;
in vec4 vertTexCoord;

void main() {
  vec2 uv = vertTexCoord.st;
  vec2 ppuv = vec2( uv.x, 1.0-uv.y );
  ivec2 ivsat = ivec2( texture2D( ppixels, ppuv ).gb * 255 );
  float sat = float(ivsat.x * 256 + ivsat.y) / 65535.0;
  if( mouse.z > 0.5 && length( uv-mouse.xy ) < 0.1 ) {
    sat = min( sat + 0.1, 1. );    // de-saturation speed
  } else {
    sat = max( sat - 0.001, 0. );  // re-saturation speed
  }
  int isat = int( sat * 65536 );
  int hi = isat/256;
  int lo = isat - hi*256;
  gl_FragColor = vec4( 0, float(hi)/255, float(lo)/255, 1. );
}
""" };


String[] fragSrc = { """
#version 120
uniform sampler2D satImg;
uniform sampler2D img;
in vec4 vertTexCoord;

void main() {
  vec2 uv = vertTexCoord.st;
  vec3 col = texture2D( img, uv ).rgb;
  float gray = 0.2126 * col.r + 0.7152 * col.g + 0.0722 * col.b;
  ivec2 ivsat = ivec2( texture2D( satImg, uv ).gb * 255 );
  float sat = float(ivsat.x * 256 + ivsat.y) / 65535.0;
  col = mix( col, vec3( gray ), sat );
  gl_FragColor = vec4( col, 1. );
}
""" };
4

Hi Scudly,

Thanks so much for the speedy reply. How would this work with the interaction that I want to do? I.e. an image that is fully filled with a single RGB colour, that can be “blanked” to black with a circular shape that can be changed in size. The previously blanked part then fade up to the previously mentioned single RGB colour. Should I create a PImage out of the screen on each frame and then pass that to the shader? I don’t have much knowledge of shaders. Finally, is it possible for you to upload this sketch somewhere? Perhaps on the p5.js editor?

Best,

Joel

5

Sorry, I didn’t see the p5.js tag and so coded it in java Processing. I have much less experience with p5.js, but I think I only used code that should be able to translate over.

Instead of the PImage, do all your drawing in a p5.Renderer made with createGraphics() and then pass that to the 2nd shader each frame:

shdr.set( "satImg", satImg );
shdr.set( "img", pg );
shader( shdr );

You can adjust the size of the blanking circle by adding a “uniform float radius;” to the satShdr. Texture coordinates go from 0.0 to 1.0. I applied the desaturation within 0.1 of the mouse, but you could use that radius uniform variable instead. Ah, I’m also assuming that the drawing region is a perfect square. If it’s not, the circle will be an ellipse. Processing passes a resolution to all shaders from which you could compute the right aspect ratio. I would hope p5 does similar, but I don’t have experience with it.

I had the saturation value fade out gradually. If you want it to instantly snap to black, then just change the 'de-saturation speed" line to sat = 1.0;. In the fragSrc shader, I use the sat value in the col = mix(...) line. (mix() is a shader-provided function for lerp.) You can do anything there you might prefer, such as mixing from black (which you would write as vec3(0.) to col. The sat value is just a float from 0. to 1. for each pixel that you can interpret however you’d like.