Question about noise point reduction

baoter · September 6, 2018, 3:53am

Hi, I’m working on a project with Kinect V2 and have some questions…
I take the video from the depth camera of Kinect V2 and there are so many noise points!! I only need black and white color image. The idea is make the color black when something is in a certain depth. And all the others will be white. Is there any way that can delete or reduce the noise points?
36%20AM|612x500

This is the code, I’m using the example by Daniel Shiffman.


// Daniel Shiffman
// Depth thresholding example

// https://github.com/shiffman/OpenKinect-for-Processing
// http://shiffman.net/p5/kinect/

// Original example by Elie Zananiri
// http://www.silentlycrashing.net

import org.openkinect.processing.*;

Kinect2 kinect2;

import codeanticode.syphon.*;
SyphonServer server;

// Depth image
PImage depthImg;

// Which pixels do we care about?
int minDepth =  0;
int maxDepth =  4500; //4.5m

// What is the kinect's angle
float angle;
float frame;

void setup() {
  size(640, 480, P2D);
  server = new SyphonServer(this, "Processing Syphon");

  kinect2 = new Kinect2(this);
  kinect2.initDepth();
  kinect2.initDevice();

  frame = 60;

  // Blank image
  depthImg = new PImage(kinect2.depthWidth, kinect2.depthHeight);
}

void draw() {
  // Draw the raw image
  //image(kinect2.getDepthImage(), 0, 0);

  // Threshold the depth image
  int[] rawDepth = kinect2.getRawDepth();


  for (int i=0; i < rawDepth.length; i++) {
    if (rawDepth[i] >= minDepth && rawDepth[i] <= maxDepth) {
      //float ratio = (maxDepth - minDepth)/255.0;
      depthImg.pixels[i] = color(0);
    } else {
      depthImg.pixels[i] = color(255);
    }
  }


  // Draw the thresholded image
  depthImg.updatePixels();
  image(depthImg, 0, 0, 640, 480);
  
  //fill(0);
  //rect(0, 0, width, frame);
  //rect(0, height-frame, width, frame);
  //rect(0, 0, frame, height);
  //rect(width-frame, 0, frame, height);


  fill(255);
  text("THRESHOLD: [" + minDepth + ", " + maxDepth + "]", 10, 36);
  server.sendScreen();
}

// Adjust the angle and the depth threshold min and max
// range from 0 to 4500
void keyPressed() {
  if (key == 'a') {
    minDepth = constrain(minDepth+100, 0, maxDepth);
  } else if (key == 's') {
    minDepth = constrain(minDepth-100, 0, maxDepth);
  } else if (key == 'z') {
    maxDepth = constrain(maxDepth+100, minDepth, 1165952918);
  } else if (key =='x') {
    maxDepth = constrain(maxDepth-100, minDepth, 1165952918);
  }
}

jb4x · September 6, 2018, 5:02am

Hi,

Have you tried blur + threshold?
You should be able to reduce the noise like this.

Edit:
Is was able to get the following result with phtoshop using almost that technique.
Instead of a pure threshold though, I tweaked a bit the level of the image.

ImgLvl

For a pure threshold, the 3 sliders would be at the same position. As you can see it’s not far from it

Here’s the result:

ImgTestv2

Note that you might loose precision with this method, areas can grow or shrink a little.

baoter · September 6, 2018, 7:23am

Thanks! The effect is perfect! How can I do this in Processing?

jb4x · September 6, 2018, 7:40am

Check out this function: https://processing.org/reference/filter_.html
Both blur and threshold are in it.

baoter · September 6, 2018, 9:09am

Thanks. I tried that, but it doesn’t work like photoshop…

jb4x · September 6, 2018, 12:04pm

You should be able to get a result quite similar than the one with photoshop.
It might need some tweeking though.

I don’t have processing right now so I can’t try

jb4x · September 6, 2018, 3:43pm

I manage to get this result with processing and the 2 functions we talked about:

Result

As you can see it is pretty close to the result from phtoshop.

Here is a code to play with the parameter of the blur and the one of the threshold.

o : decrease blur value
p: increase blur value
l: decrease threshold value
m: increase threshold value

I’m sorry if the keys make no sense with your keyboard but it makes sense with a french one

PImage originalPic;
int paramBlur;
float paramThreshold;

void setup() {
  size(1275, 959);
  
  originalPic = loadImage("test.png");
  paramBlur = 2;
  paramThreshold = 0.3;
}

void draw() {
  image(originalPic, 0, 0);
  filter(BLUR, paramBlur);
  filter(THRESHOLD, paramThreshold);
  
  fill(0);
  noStroke();
  rect(0, 0, 150, 50);
  fill(255);
  text("Blur value: " + paramBlur, 10, 20);
  text("Threshold value: " + paramThreshold, 10, 40);
}

void keyPressed() {
  if (key == 'o') {
    paramBlur--;
    if (paramBlur == -1) {
      paramBlur = 0;
    }
  }
  
  if (key == 'p') {
    paramBlur++;
    if (paramBlur == 10) {
      paramBlur = 10;
    }
  }
  
  if (key == 'l') {
    paramThreshold -= 0.1;
    if (paramThreshold < 0) {
      paramThreshold = 0;
    }
  }
  
  if (key == 'm') {
    paramThreshold += 0.1;
    if (paramThreshold > 1) {
      paramThreshold = 1;
    }
  }
}

To try the code, you will need to create a “Data” folder in your sketch folder and put the image below that you will name “test.png”

kfrajer · September 7, 2018, 3:05am

You should look also into erode and dilate: https://processing.org/reference/PImage_filter_.html

Kf

baoter · September 7, 2018, 7:48am

Hi, I tried with this… it makes the noise point fewer but bigger…

baoter · September 10, 2018, 3:17am

It works great, but only on processing. My idea is to use syphon to transfer the image from processing to resolume arena. When I use the filter, the points are much more fewer than before, but when I transfer the image to arena, it is the same just like before. I also tried openCV: opencv.dilate(); opencv.erode(); opencv.blur(2); , it works but it doesn’t have parameter.

baoter · September 10, 2018, 3:23am

It is the same problem just like my reply. I can’t transfer this image to arena…

jb4x · September 10, 2018, 6:04am

Are you sure your are exporting the right image. It feels that you are exporting the original one?

Can you put here the code that you use to export your image?

baoter · September 10, 2018, 7:25am

I think so… I send the image at the very last step…


// Daniel Shiffman
// Depth thresholding example

// https://github.com/shiffman/OpenKinect-for-Processing
// http://shiffman.net/p5/kinect/

// Original example by Elie Zananiri
// http://www.silentlycrashing.net
import gab.opencv.*;
import org.openkinect.processing.*;
import codeanticode.syphon.*;

SyphonServer server;
Kinect2 kinect2;
OpenCV opencv;

// Depth image
PImage depthImg;

// Which pixels do we care about?
int minDepth =  0;
int maxDepth =  4500; //4.5m

// What is the kinect's angle
float angle;
float frame;

void setup() {
  size(640, 480, P2D);

  server = new SyphonServer(this, "Processing Syphon");

  kinect2 = new Kinect2(this);
  kinect2.initDepth();
  kinect2.initDevice();

  frame = 30;

  // Blank image
  depthImg = new PImage(kinect2.depthWidth, kinect2.depthHeight);
      opencv = new OpenCV(this, kinect2.depthWidth, kinect2.depthHeight);

}

void draw() {
  // Draw the raw image
  //image(kinect2.getDepthImage(), 0, 0);

  // Threshold the depth image
  int[] rawDepth = kinect2.getRawDepth();
  

  for (int i=0; i < rawDepth.length; i++) {
    if (rawDepth[i] >= minDepth && rawDepth[i] <= maxDepth) {
      //float ratio = (maxDepth - minDepth)/255.0;
      depthImg.pixels[i] = color(255);
    } else {
      depthImg.pixels[i] = color(0);
    }
  }


  // Draw the thresholded image
  depthImg.updatePixels();

  try {
    opencv.loadImage(depthImg);
    //opencv.dilate();
    //opencv.erode();
    opencv.blur(2);
    image(opencv.getSnapshot(), 0, 0, 640, 480);

  }
  catch(Exception e) {
    print(e);

  }


  fill(0);
  noStroke();
  //rect(0, 0, width, frame+30);
  //rect(0, height-frame, width, frame);
  //rect(0, 0, frame, height);
  //rect(width-frame, 0, frame, height);


  fill(255);
  text("THRESHOLD: [" + minDepth + ", " + maxDepth + "]", 10, 36);
  server.sendScreen();
}

// Adjust the angle and the depth threshold min and max
// range from 0 to 4500
void keyPressed() {
  if (key == 'a') {
    minDepth = constrain(minDepth+100, 0, maxDepth);
  } else if (key == 's') {
    minDepth = constrain(minDepth-100, 0, maxDepth);
  } else if (key == 'z') {
    maxDepth = constrain(maxDepth+100, minDepth, 1165952918);
  } else if (key =='x') {
    maxDepth = constrain(maxDepth-100, minDepth, 1165952918);
  }
}

baoter · September 10, 2018, 7:27am

I think so… I send the image at the very last step…


// Daniel Shiffman
// Depth thresholding example

// https://github.com/shiffman/OpenKinect-for-Processing
// http://shiffman.net/p5/kinect/

// Original example by Elie Zananiri
// http://www.silentlycrashing.net
import gab.opencv.*;
import org.openkinect.processing.*;
import codeanticode.syphon.*;

SyphonServer server;
Kinect2 kinect2;
OpenCV opencv;

// Depth image
PImage depthImg;

// Which pixels do we care about?
int minDepth =  0;
int maxDepth =  4500; //4.5m

// What is the kinect's angle
float angle;
float frame;

void setup() {
  size(640, 480, P2D);

  server = new SyphonServer(this, "Processing Syphon");

  kinect2 = new Kinect2(this);
  kinect2.initDepth();
  kinect2.initDevice();

  frame = 30;

  // Blank image
  depthImg = new PImage(kinect2.depthWidth, kinect2.depthHeight);
      opencv = new OpenCV(this, kinect2.depthWidth, kinect2.depthHeight);

}

void draw() {
  // Draw the raw image
  //image(kinect2.getDepthImage(), 0, 0);

  // Threshold the depth image
  int[] rawDepth = kinect2.getRawDepth();
  

  for (int i=0; i < rawDepth.length; i++) {
    if (rawDepth[i] >= minDepth && rawDepth[i] <= maxDepth) {
      //float ratio = (maxDepth - minDepth)/255.0;
      depthImg.pixels[i] = color(255);
    } else {
      depthImg.pixels[i] = color(0);
    }
  }


  // Draw the thresholded image
  depthImg.updatePixels();

  try {
    opencv.loadImage(depthImg);
    //opencv.dilate();
    //opencv.erode();
    opencv.blur(2);
    image(opencv.getSnapshot(), 0, 0, 640, 480);

  }
  catch(Exception e) {
    print(e);

  }


  fill(0);
  noStroke();
  //rect(0, 0, width, frame+30);
  //rect(0, height-frame, width, frame);
  //rect(0, 0, frame, height);
  //rect(width-frame, 0, frame, height);


  fill(255);
  text("THRESHOLD: [" + minDepth + ", " + maxDepth + "]", 10, 36);
  server.sendScreen();
}

// Adjust the angle and the depth threshold min and max
// range from 0 to 4500
void keyPressed() {
  if (key == 'a') {
    minDepth = constrain(minDepth+100, 0, maxDepth);
  } else if (key == 's') {
    minDepth = constrain(minDepth-100, 0, maxDepth);
  } else if (key == 'z') {
    maxDepth = constrain(maxDepth+100, minDepth, 1165952918);
  } else if (key =='x') {
    maxDepth = constrain(maxDepth-100, minDepth, 1165952918);
  }
}

baoter · September 10, 2018, 7:28am

I think so, I send the image at the very last step…


// Daniel Shiffman
// Depth thresholding example

// https://github.com/shiffman/OpenKinect-for-Processing
// http://shiffman.net/p5/kinect/

// Original example by Elie Zananiri
// http://www.silentlycrashing.net
import gab.opencv.*;
import org.openkinect.processing.*;
import codeanticode.syphon.*;

SyphonServer server;
Kinect2 kinect2;
OpenCV opencv;

// Depth image
PImage depthImg;

// Which pixels do we care about?
int minDepth =  0;
int maxDepth =  4500; //4.5m

// What is the kinect's angle
float angle;
float frame;

void setup() {
  size(640, 480, P2D);

  server = new SyphonServer(this, "Processing Syphon");

  kinect2 = new Kinect2(this);
  kinect2.initDepth();
  kinect2.initDevice();

  frame = 30;

  // Blank image
  depthImg = new PImage(kinect2.depthWidth, kinect2.depthHeight);
      opencv = new OpenCV(this, kinect2.depthWidth, kinect2.depthHeight);

}

void draw() {
  // Draw the raw image
  //image(kinect2.getDepthImage(), 0, 0);

  // Threshold the depth image
  int[] rawDepth = kinect2.getRawDepth();
  

  for (int i=0; i < rawDepth.length; i++) {
    if (rawDepth[i] >= minDepth && rawDepth[i] <= maxDepth) {
      //float ratio = (maxDepth - minDepth)/255.0;
      depthImg.pixels[i] = color(255);
    } else {
      depthImg.pixels[i] = color(0);
    }
  }


  // Draw the thresholded image
  depthImg.updatePixels();

  try {
    opencv.loadImage(depthImg);
    //opencv.dilate();
    //opencv.erode();
    opencv.blur(2);
    image(opencv.getSnapshot(), 0, 0, 640, 480);

  }
  catch(Exception e) {
    print(e);

  }


  fill(0);
  noStroke();
  //rect(0, 0, width, frame+30);
  //rect(0, height-frame, width, frame);
  //rect(0, 0, frame, height);
  //rect(width-frame, 0, frame, height);


  fill(255);
  text("THRESHOLD: [" + minDepth + ", " + maxDepth + "]", 10, 36);
  server.sendScreen();
}

// Adjust the angle and the depth threshold min and max
// range from 0 to 4500
void keyPressed() {
  if (key == 'a') {
    minDepth = constrain(minDepth+100, 0, maxDepth);
  } else if (key == 's') {
    minDepth = constrain(minDepth-100, 0, maxDepth);
  } else if (key == 'z') {
    maxDepth = constrain(maxDepth+100, minDepth, 1165952918);
  } else if (key =='x') {
    maxDepth = constrain(maxDepth-100, minDepth, 1165952918);
  }
}

Topic		Replies	Views
Silhouette Kinect V2 + Removing Background Libraries	1	1395	February 20, 2019
Depth of Field effect with a shader Processing.py	11	3215	November 26, 2018
Changing webcam-pixels to "nearest" color from color palette array Libraries	29	1863	November 17, 2019
Processing x Kinect v2: Getting depth data and tracking moving object in 3D world Libraries	0	1390	August 21, 2019
Framerate in live video delay project : Ed Tannenbaum, Recollections Project Guidance	30	1933	November 3, 2019

Question about noise point reduction

Related topics