Unable to apply filter to a capture from the webcam

Hi,

I’m trying to apply filter using the filter() ( eg: filter(GRAY) ), the problem I have is that it applies the filter to the whole Canvas instead of only the “selected” element.

So what I’m doing exactly is capturing the webcam image and drawing 2 images in front of each other ( using image() ), I would like to be able to apply a filter or multiple filters to the first image (in the background) and apply one or multiple filters to the foreground image. I know (I think) it’s doable when using loadImage() or createImage() but what in the case of capturing the webcam image.

I’m also confused about the blendMode(), I’m not really sure if it’s applying the blendMode to the whole items created or only the following items, so my second question is how to apply a different blend mode to each item, something like push() and pop() but for BlendMode and filter(), in my case I want my background image to be blended with the background color (let say MULTIPLY), and the foreground image to be blended with anything beneath with another blend mode.

Side questions: how to add contrast and brightness (like all the filters in css) to the whole canvas and to separate elements (in my case : image()), is there something like filter(CONTRAST, 1.2) and or filter(BRIGHTNESS, 2). Or could it be done using canvas stuff instead of P5?

The demo can be seen here : https://codepen.io/romaingranai/pen/eYopayK?editors=0010

You probably need to use distinct p5.Graphics to apply your filters separately examples | p5.js

I rarely use p5 but you may try this:
frontImg.filter(GRAY);

You can use;
capture.style('filter', 'brightness(50%)');
capture.style('filter', 'contrast(120%)');

However, CSS features works for only capture element itself. If you comment capture.hide(), you will get the effect. But this time, you cannot use the blendMode functions.

Another option can be using shaders to process the image like this.

@DevaPan’s answer is the most appropriate one for you, I think. But it can be expensive for the CPU.

Another trick can be playing around tint function before drawing the images.

1 Like

Hi @TiborUdvari, That’s probably one of the solution, gonna try that, I need to learn a bit how createGraphics works, but I think I understand how stuff are going to work.

@DevaPan yes that would be perfect, but as explain in the first thread, it only work if the image is brought to the canvas using loadImage() or createImage(), which is not ideal because in my case the image comes from the webcam using capture.

so :

function setup() {
  createCanvas(window.innerWidth, window.innerHeight);
  capture = createCapture(VIDEO,{ flipped:true });
  capture.size(390, 240);
	// capture.filter(GRAY);  // Not Working either
	capture.hide();
	frameRate(30);
}

function draw() {
  frontImg = image(capture, 0, 0, window.innerWidth, window.innerHeight);
  frontImg.filter(GRAY); //Not Working cause frontImg is not coming from loadImage() or createImage()
}

Doesn’t work.

Anyway, thanks for your quick answers.

1 Like

Hey @RomainGr,

I made a library (p5.FIP) for p5.js that does these kind of filters (including grayscale, blend, contrast and brightness) on the GPU for better performance. Using these filters with framebuffers (tutorial) should suit your use case. To use it in your sketch you’d just have to import the library in your index.html file:

<head>
    <!-- other head stuff -->
    <script src="https://prontopablo.github.io/p5.FIP/assets/javascripts/p5.FIP.js"></script>
    <!-- ...-->
</head>

Then in sketch.js load the shaders you want to use and create some framebuffers and apply the shaders to just the framebuffers:

function preload() {
    grayscale = createShader(fip.defaultVert, fip.grayscale); // Load the grayscale shader
    blend = createShader(fip.defaultVert, fip.blend); // Load the blend shader
    contrast = createShader(fip.defaultVert, fip.contrast); // Load the contrast shader
    brightness = createShader(fip.defaultVert, fip.brightness); // Load the brightness shader
}

function setup() {
    createCanvas(600, 600, WEBGL); // Use WEBGL mode to use the shader
     // Create framebuffers to draw onto (faster p5.js version of createGraphics())
    layer1 = createFramebuffer();
    layer2 = createFramebuffer();
}

I see. Maybe you could try something like this:

PImage frontImg = capture.get();
frontImg.filter(GRAY);
image(frontImg, 0, 0);

It works in Processing at least…

Hello @RomainGr ,

An example for you:

let capture;
let img0, img1, img2;

function setup() 
  {
  createCanvas(640, 480);
  capture = createCapture(VIDEO);
  capture.size(320, 240);
  capture.hide();
  
  img1 = createImage(320, 240);
  img2 = createImage(320, 240); 
  }

function draw() 
  {
  background(255, 0, 0);
  
  // Method 1
  img0 = capture.get();
  img0.filter(GRAY);
  image(img0, 0, 0);
  
  // Method 2
  img1.copy(capture, 0, 0, 320, 240, 0, 0, 320, 240);
  img1.filter(INVERT);
  image(img1, 320, 0);
  
  // Method 3 
  img2.loadPixels();
  for (let i = 0; i < capture.pixels.length; i++) 
    {
    img2.pixels[i] = capture.pixels[i];
    }
  img2.updatePixels();
  img2.filter(BLUR, 5);
  img2.filter(POSTERIZE, 5);
  image(img2, 0, 240);
  
  // Just for fun!
  let th = (frameCount%360)*(TAU/360);
  let x = 50*cos(th) + capture.width/2 - 25;
  let y = 50*sin(th) + capture.height/2 - 25;
  img3 = capture.get(x, y, 50, 50);
  img3.filter(THRESHOLD);
  capture.set(x, y, img3);
  image(capture, 320, 240);  
  }

:)

1 Like

Hi guys,

Many thanks for all the solutions you gave me, gonna rework on the project this week and try the different paths. I’ll let you know what’s the working solution as soon as I put my hand on it again.

Thanks again!

2 Likes

Hey @RomainGr.

I’ll add a couple things that may be helpful for you to know in combination with the great advice you already received.

  • Since p5.js version 1.8.0, the filter() function runs on the GPU by default.
  • There is a convenience function createFilterShader() to import custom pixel shaders without the need for a vertex shader.

Good luck!

3 Likes

That’s crazy cool! will def use it on some other project.
I’ve just had a quick look, and learned a lot of small tricks, thanks for that.