Alternative to get() for faster performance?

p5.js Coding Questions
1

Hello, i was wondering how it would be possible to replace the get(x,y,w,h) function with something else to enhance performance… i’m currently working on a project where i have to “degrade the quality” of a webcam image in real time. I settled on an effect like this here. i integrated the code as is and it works but it slows down the whole sketch and the webcam feedback becomes so slow it’s as if it freezed… i believe it’s because of this command set(x2, y2, get(x1,y1,w,h));, specifically the get() function?
I read that another way to access pixels faster is by directly going after the pixel array like this

 myVideo.pixels[i], //r
 myVideo.pixels[i + 1], //g
 myVideo.pixels[i + 2], //b
 myVideo.pixels[i + 3], //a

but i’m already confused at how i’m supposed to get the position or change the scale.

so far this is the code i have in draw

myVideo.loadPixels();

for (let y = 0; y < myVideo.height; y ++) {
    for (let x = 0; x < myVideo.width; x ++) {
      
      var r = getQuick(myVideo, x, y)[0];
      var g = getQuick(myVideo, x, y)[1];
      var b = getQuick(myVideo, x, y)[2];
      var a = getQuick(myVideo, x, y)[3];

if (r < effect || g < effect || b < effect || a < effect ) {

        var x1 = random(myVideo.width);
        var y1 = random(myVideo.height);

        var x2 = round(x1 + random(-10, 10));
        var y2 = round(y1 + random(-10, 10));

        var w = 10;
        var h = 10;

        set(x2, y2, get(x1,y1,w,h));
       }
    }
  }
  myVideo.updatePixels();

Can someone help me solve this? I found that using the pixel array to simply manipulate the color of the pixels worked very well and allowed speed and fluidity. is it possible to do the same for position and scale and get a result as shown in this example? i already took a look at the references for set(), get(), and pixels[] but it did not help me understand how i’m supposed to execute what im trying to do :frowning:

by the way, i created this function though i am not sure it is entirely necessary, but anyway, putting this here in case i am actually making my life harder with useless stuff :confused:

function getQuick(myVideo, x, y) {
  var i = (y * myVideo.width + x) * 4;
  return [
    myVideo.pixels[i], //r
    myVideo.pixels[i + 1], //g
    myVideo.pixels[i + 2], //b
    myVideo.pixels[i + 3], //a
  ];
}
1 Like
2

have you tried copy? if it’s still slow, you can give a shot with context.drawImage

3

---- EDIT ----
the answer below is wrong as it correspond to processing java and P5.js pixel array is different
---- EDIT ----
hi,
in pixels[], each value of the array is one pixel

so the pixel (23 ; 32) of a 100x100 image is at the address 23+100 * 32 (x+y*image.width)
(no need to multiply by 4 as in your code)
then,for this pixel, you got one integer coding the 4 channel ARGB in one value
so to copy the pixel , just copy this value to somewhere else,

the color is coded like that:this integer is 4 byte, each byte represent one of ARGB channels , so the 8 less significant bit are blue value (256 possibles value) then upper 8 bit are green… and so on
tell me if you need more information on manipulating those color

4

hi! thanks for your reply! i tried using copy but i have the problem that the function expects integers but at some point, for

copy(sx, sy, sw, sh, dx, dy, dw, dh);

i need to replace 5th and 6th parameters with “effect”

function setup(){
let effect = 0;
}
function draw() {
let amplitude = 255;
effect = constrain(effect, 0, amplitude);
}

so it returns smth like “expected integer received number instead”. and i’m not sure if that contributes to slowing down the code but it did so i guess it does?
I then tried your fix with context.drawImage but it doesn’t do anything… there are no errors though. but i have a few questions:
what exactly do these for loops do?

for (let i = 0; i < tilesX; i++) {
      for (let j = 0; j < tilesY; j++) {}
}

it doesn’t seem like it would give us access to the pixels of the video but does it still serve to replace a loop such as

for (let y = 0; y < myVideo.height; y ++) {
    for (let x = 0; x < myVideo.width; x ++) {}
}

could that be why it’s not working? must i still do something like this for it to work?

for (let y = 0; y < myVideo.height; y ++) {
    for (let x = 0; x < myVideo.width; x ++) {

      var r = getQuick(myVideo, x, y)[0];
      var g = getQuick(myVideo, x, y)[1];
      var b = getQuick(myVideo, x, y)[2];
      var a = getQuick(myVideo, x, y)[3];

      if (r < effect || g < effect || b < effect || a < effect ) {

        for (let i = 0; i < tilesX; i++) {
            for (let j = 0; j < tilesY; j++) {

        let wave = int(sin(frameCount * 0.05 + ( i * j ) * 0.07) * 100);

        let sx = i * tileW + wave;
        let sy = j * tileH;
        let sw = tileW;
        let sh = tileH;

        let dx = i * tileW;
        let dy = j * tileH;
        let dw = tileW;
        let dh = tileH;

        drawingContext.drawImage(
              pg.canvas,
              sx,
              sy,
              sw,
              sh,
              dx,
              dy,
              dw,
              dh
            );
          }
        }

i tried it like that too but now it makes it buffer and still nothing happens, no errors either…
i am sadly very lost. i hope i exposed my problem clearly enough :sweat:
in any case, thank you very much for your response!

5

hello! thank you for your reply!
i think i understand how pixels[] work in general… so you can control the position individually by using functions such as copy for example, but using a method like

      var r = getQuick(myVideo, x, y)[0];
      var g = getQuick(myVideo, x, y)[1];
      var b = getQuick(myVideo, x, y)[2];
      var a = getQuick(myVideo, x, y)[3];

can only control the colors?
then, would it be possible to get the color values from one pixel and exchange it with those of another?

6

Color values are accessed as follows

float br = brightness(pixels[i]);
float r = red(pixels[i]);
float g = green(pixels[i]);
float b = blue(pixels[i])

Where i is the index of the pixel required.

7

I’m pretty sure this is incorrect @paulgoux. the pixels array contains individual number values for the different color components. The get() function has the virtue over this of returning the p5.Color value of the pixel from which you can get the components with the functions you’ve specified. More info can be found on the p5js.org reference page for pixels.

1 Like
8

Hello,

This topic inspired me!

Something I worked on:

Example of random tiling < Click here to see code! 
// Pixel Manipulation 
// v1.0.0
// GLV 2021-05-30

let img;
let cap;

function setup() 
  {
  createCanvas(640, 480); // Set to video size
  
  img = createGraphics(width, height);

  pixelDensity(1);    
  img.pixelDensity(1);
  
  cap = createCapture(VIDEO);
  cap.hide(); 
  
  //d = pixelDensity(); // Future! See references.
  }

function draw() 
  { 
  cap.loadPixels();
  img.loadPixels();
  
  tilesX = 10;
  tilesY = 10;
  
  //samples = map(mouseX, 0, width, 0, 2000);
  samples = 200; 
  
  for (let i = 0; i<samples; i++)
    {
    x = (random(width))| 1;  // Converts to int
    y = int(random(height)); // Converts to int 
      
    let loc = x + y*cap.width;
  
  // Copy a tile from capture to pixels     
    for (let xt = x; xt < x+tilesX; xt++) 
      {
      for (let yt = y; yt < y+tilesY; yt++) 
        {        
        let loct = (xt + yt*cap.width)*4;
        
        img.pixels[loct + 0] = cap.pixels[loct + 0];  //r
        img.pixels[loct + 1] = cap.pixels[loct + 1];  //g
        img.pixels[loct + 2] = cap.pixels[loct + 2];  //b
        img.pixels[loct + 3] = cap.pixels[loct + 3];  //a
        }
      }
    }    
    
  img.updatePixels();
  image(img, 0, 0);
  
  // Display data
  fill(255);
  stroke(1);
  textSize(18);
  text(frameRate()|1, 10, 20);
  text(samples, 10, 40);
  }

image
image654×609 191 KB

A very cool effect!

:)

3 Likes
9

Wow, that’s super cool! Thank you :slight_smile: