OpenCV update, Heaviside Step Function, deriving parametric functions of shapes, Sobel filter

[quark]

I set bezier mode to true and got the exception IndexOutOfBoundsException: Index: 1, Size: 0 so I could not try that out.

@noel

The first thing that came into my mind was to programmatically move the control-point around its anchor-point in a spiral way, keeping track of the matching pixels of the original line and the formed curve.

Since you have the coordinates of the curve you want to represent you could use these to perform a Bezier curve fitting - see the Pomax (curve fitting) guide.

I am not sure when I will be releasing my Bezier library, the actual Bezier class is virtually complete and implements most of the functionality in Pomax guide but I still have a lot to do before release.

Most graphics applications use cubic or quadratic beziers but I have written the library to handle curves of any order >= 2 (i.e. degree >=1). So we have

Degree          Shape                    Order (= number of control points)
  1             straight line              2
  2             quadratic bezier           3
  3             cubic bezier               4
  4             quartic bezier             5
and so on

This means I have not implemented any functionality from the guide that is specific to a particulare order e.g. cubic.

Some of the functionality, especially for higher order curves uses matrix maths and I use EJML (Efficient Java Matrix Library) for its ease of use. This does mean that my library is for JavaMode and would need a lot of work to convert it to JS

[paulgoux]

Providing that the picture has enough contrast could you not use the marching squares algorithm to detail an outline. Sebastian league has a video on this for procedural cave generation and coding train just recently did a video on this very topic. I’ve implemented my own version and I can confirm that you can produce a set of points for vertices of an image which you can store in an array, the beauty of this is that it allows you to define the minimum amount of vertices and the maximum amount too and therefore allows you to kinda denoise the image too.

The gist of the program is this.
Retrieve an rgba value use r g b or a. Set a threshold based on the colour you are using. Anything over the threshold is 1 everything else is 0.

Then run through all points mark it as an open space if its a 1. This should give an array of array of points. Trim to you desired size.

Then do the same for all the wall points. Again trim based on desired size.

Repeat step one again. Trimming the walls will make new non wall tiles which now need to be counted.

extract edges of non walls and or walls.

Finally use marching squares to produce outline.

Note that whilst marching squares is based on the idea of retrieving one of 16 vertices based on neighbour tile variations it can be amended to give you a higher resolution, and finally I’m sure edge points with similar neighbouring gradients could be culled.

Note this is quite intensive CPU wise and on my t440p I can only run this on a grid 200 by 200 maybe a little higher but at 300x300 the program just hangs. I haven’t tried multi threading mind you which should be very easily doable.

so can confirm this will extract the edges. No I haven’t yet added the marching squares part so im just setting the color of edge tiles to a different color.

this has been done extracting blue setting threshold to 90, open space cuttof size to 10 and wall cutoff to 20.

here’s without any size threshold for comparison.

threshold 95

threshold 99

reference image
;

only downside to this is that its not very repeatable and settings have to be amended for individual images. Also its highly dependent on min max values.
passed red cutoff threshold 20

here is a picture of a car i passed through the program

passed blue same setting

passed green same settings

passed average (r + g + b )/3 threshold 10.

however something with good contrast will work fine

just noticed there is some deformation in the image, this happens at resolutions above 100 for some reason, due to the way I extract the pixel points.

[noel]

@paulgoux

Sebastian league has a video on this

I watched the serie now, and also Daniel’s latest video. I must admit that I need to re-watch them to fully understand this. But I see plenty of possibilities using this algorithm, as you also have shown with your project. Are you going to work this further out? When (if possible) I am ready with my bezier-node reducing project, I definitely will look into this. For the moment opencv is giving me a satisfying result.

[noel]

@quark

I set bezier mode to true and got the exception

What do you mean by “set to true”? I’m not getting this error. Did you use an imported image?
I know this can be improved, but first I have to learn a lot, and it’ll take some time
A lib in just JavaMode would be a great contribution. I’m looking forward to this.

[quark]

@noel

What do you mean by “set to true”?

boolean curve_mode = true, bezier_mode;

@noel

Did you use an imported image?

No forgot that LOL

[paulgoux]

That’s great. The idea is to improve image recognition hopefully by using the gradient of vertices as opposed to individual points.

This is really good stuff, I didnt think of using this for recreating images, which essentially could be used to increase resolution of images.

[paulgoux]

here’s my own take on this idea. Some differences however, this does not produce any curve just an array of points.

more precisely;
an arraylist of arraylist of wall points,
an arraylist of arraylist of non wall points,
an arraylist of arraylist of edge points,

I will add the marching squares algorithm later to produce actual lines as opposed to tiles.

Please note there are some bugs here and there and this code is provided as is.

paulgoux/image-to-vertex

It can be used on any image, but works best on images with good contrast.

[noel]

@paulgoux

here’s my own take on this idea.

I took some more time to familiarize myself a bit more with marching squares and it´s really interesting, although takes quite some processing power making it run slowly. As my next step is to convert a colored image into shapes I tried several edge detection codes. Also processing’s example with different and amplified kernel matrices with really nice results. But nothing comes close to what OpenCV’s Adaptive threshold function can do. Here is a test code and some screenshots using the car image you posted above.
Block Size is size of a pixel neighborhood that is used to calculate a threshold value for the pixel: 3, 5, 7, and so on.
C_mean is a constant subtracted from the mean or weighted mean. Normally, it is positive but may be zero or negative as well.|

import gab.opencv.*;

OpenCV opencv;
PImage  img, adaptive, gray;
float loc1 = 15, loc2 = 15;
int bs = 3, cns = -4;
boolean drag, slider_1, slider_2;

void setup() {
  surface.setTitle("Adaptive threshold");
  size(960, 660);
  background(255);
  img = loadImage("car.jpeg");
  opencv = new OpenCV(this, img);  
  gray = opencv.getSnapshot();
  image(gray, 0, 0);
  drawSlider();
  textSize(15);
  fill(0);
  text("BlockSize =  "+bs, 20, 629);
  text("C-mean = "+cns, 20, 580);
}

void draw() {
  if (drag) {
    opencv.loadImage(gray);
    opencv.adaptiveThreshold(bs, cns);
    adaptive = opencv.getSnapshot();
    image(adaptive, 0, 0);
    drawSlider();
    text("BlockSize =  "+bs, 20, 629);
    text("C-mean = "+cns, 20, 580);
  }
}

void drawSlider() {
  push();
  fill(255);
  rect(0, 540, width, 130); 
  fill(200, 200, 255);
  rect(10, 645, width-20, 5); 
  rect(10, 595, width-20, 5);
  fill(150, 150, 255);
  ellipse(loc1, 647, 15, 15);
  ellipse(loc2, 597, 15, 15);
  pop();
} 

void mouseDragged() {
  drag = true;
  if (slider_2) {
    loc1 = mouseX;
    if (loc1 < 15) loc1 = 15;
    if (loc1 > width-15) loc1 = width-15;
    int t =  int(map(loc1-10.55, 0.0, width, 3, 200));
    if (t % 2 == 0); 
    else bs = t;
  }
  if (slider_1) {
    loc2 = mouseX;
    if (loc2 < 15) loc2 = 15;
    if (loc2 > width-15) loc2 = width-15;
    cns = int(map(loc2-10.55, 0.0, width, -5, 15));
  }
}

void mousePressed() {
  if (mouseY > 580 && mouseY < 610) slider_1 = true;
  if (mouseY > 630) slider_2 = true;
}

void mouseReleased() {
  drag = slider_1 = slider_2 = false;
}

[noel]

I need to retrieve the hierarchy array (outer (parent) and inner (child) shapes) of the contours. The methods of OpenCV_processing lib do not give access to that, so I’m trying to subtract them directly from the original OpenCV 2.4.5 lib (as used in some of OpenCV_processing’s example sketches).
This is OpenCV_processing’s findContours() method in java file.

public ArrayList<Contour> findContours(boolean findHoles, boolean sort) {
  ArrayList<Contour> result = new ArrayList<Contour>();
  ArrayList<MatOfPoint> contourMat = new ArrayList<MatOfPoint>();
  try {
    int contourFindingMode = (findHoles ? Imgproc.RETR_LIST : Imgproc.RETR_EXTERNAL);
    Imgproc.findContours(getCurrentMat(), contourMat, new Mat(), contourFindingMode, Imgproc.CHAIN_APPROX_NONE);
  } 
  catch(CvException e) {
    PApplet.println("ERROR: findContours only works with a gray image.");
  }
  for (MatOfPoint c : contourMat) {
    result.add(new Contour(parent, c));
  }
  if (sort) {
    Collections.sort(result, new ContourComparator());
  }
  return result;
}

.
And this is the code and image I am using trying to get the result. But this code shows 11 contours, while when using only OpenCV_processing it gives correctly nr 2.
Can anybody point out where I’m going wrong?
@quark maybe?

import gab.opencv.*;
import org.opencv.imgproc.Imgproc;
import org.opencv.core.Mat;
import org.opencv.core.MatOfPoint;


OpenCV opencv;
Contour contour;
 
PImage  src;
ArrayList<MatOfPoint> contours;

void setup() {
  size(527, 488);
  src = loadImage("6.png");
  opencv = new OpenCV(this, src);
  background(255);
  opencv.loadImage(src);
  opencv.invert();
  opencv.threshold(50);
  Mat gray = OpenCV.imitate(opencv.getGray());
  opencv.getGray().copyTo(gray);
  Mat thresholdMat = OpenCV.imitate(opencv.getGray());
  contours = new ArrayList<MatOfPoint>();
  Mat hierarchy = new Mat();
  int[] current_hierarchy = new int[4];
  boolean findHoles = true;
  //int contourFindingMode = (findHoles ? Imgproc.RETR_LIST : Imgproc.RETR_EXTERNAL);
  Imgproc.findContours(thresholdMat, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_NONE);
  //Imgproc.findContours(thresholdMat, contours, new Mat(), contourFindingMode, Imgproc.CHAIN_APPROX_NONE);
  println("Number of contours = "+contours.size());
  for(int i = 0; i < contours.size(); i++){
    // println(hierarchy.get(0, i, current_hierarchy));
    println(hierarchy.get(0, i));
  }
  // Imgproc.drawContours(new Mat(), contours, -1, color(0, 0, 0), 2, 1, hierarchy, 2, new Point());
  image(opencv.getOutput(), 0, 0);
}

[paulgoux]

I’ve been working on the line detection on my program here are the current results. I still have yet to implement a decent way of extracting vertices though as the simple thresholding I used before would have been inappropriate, so I’ll update later. I’m fairly satisfied with the results.

and still produces a clean outline on highly contoured images.

[noel]

Wow. I really like that.
I haven’t seen an update on your Github repo for that. Would be nice to share it as a new filter.

[paulgoux]

I shall just still tinkering around.

[paulgoux]

paulgoux/imgconvolution

make sure to use the sobel operator.

[noel]

Marvelous. That house is so beautiful in sobel! Sobel, blur and mean are working well. I haven’t really looked into the code, so I guess you are still working on the others. Tomorrow I will explore the code more with the other images.

[paulgoux]

I really like the sobel! The blur is a simple box blur, I’m working on making it separable to increase speed, currently anything over 25 squares wide is really slow, a separable version, will reduce operations to 51 operations as opposed to 25*25. Also need to implement Gaussian blur which provides better results.

My current implementation isnt great at extracting lines from hand-drawn images as it will give everything a double edge, look through the data folder to try different examples.

[noel]

@paulgoux

My current implementation isnt great at extracting lines from hand-drawn images

No problem, for that we use openCV. The sobel edge detection is already an excellent result for itself.

[noel]

I’ve tried hard to get a similar result with OpenCV’s sobel filter even using the original lib directly, but it doesn’t even come close. And if you google images with their filter you really see a, may I say, poor result. This one you definitely win over OpenCV.

[paulgoux]

I think that its because of the multiplier I’m applying to the end result. If you set it to zero does it give you a similar result? In any case there’s still some more work to do the next logical step after sobel is canny and that normally produces better results than my implementation. Also invert the output of the results setting the color as 255 - k.

[noel]

@paulgoux

If you set it to zero does it give you a similar result?

Nope. I tried it all.
You might find it interesting how they treat the Sobel derivatives (and other filters) on a tutorial that is the same as the release 2.4.3 on which Processing_OpenCV is built on. It starts on page 197.

opencv_tutorials.pdf

[paulgoux]

for the house image using openCv do you get an output similar to this?

youch!! Documentation my old nemesis.

I’ll take a look at it for sure though when I have some time, but for now computerphile has been my go to Mike Pounds videos are excelent.

also I meant setting the multiplier to 1 not zero,