Can Video and Text Co-exist?

I have been using Daniel Example “sketch_11_5_AveragePixelColorTracking” for blob tracking.
I thought I could get myself a couple of text lines under the video in the sketch to show XY etc…

The text displays but does not erase with each iteration. So I got numbers on top of old numbers…

What to do ?
Thanks a lot

here is my code

import processing.video.*;
import java.awt.Font;

PFont f;
PFont font;
int w= #FFFFFF; // white
int y= #FCF442; //yellow
int g= #A0FFA3; //green
int b= #64E1FF; // blue
int m= #CC15D3; // Magenta
int o= #FF6C67;  // orange
int i= #B767FF; // indigo
int r= #FC0B03; // red
int bk = (#212121);

Capture video;

color trackColor; 
float threshold = 25;

int Wheel; // value from the mouse wheel
int WheelAcc=0; // mouse wheel accoumulated


void setup() {
  size(1800, 1100, JAVA2D);
  String[] cameras = Capture.list();
  printArray(cameras);
  video = new Capture(this, "name=iSight,size=640x480,fps=15");
  video.start();
  trackColor = color(255, 0, 0);
  font = loadFont("AgencyFB-Bold-200.vlw");
}

void captureEvent(Capture video) {
  video.read();
}

void draw() {

  video.loadPixels();
  image(video, 0, 0);

  //threshold = map(mouseX, 0, width, 0, 100);
  threshold = 30+ WheelAcc;
  threshold = constrain (threshold, 0, 100);

  // /*
  float avgX = 0;
  float avgY = 0;

  int count = 0; // nr of pixels found in 

  // Begin loop to walk through every pixel
  for (int x = 0; x < video.width; x++ ) {
    for (int y = 0; y < video.height; y++ ) {
      int loc = x + y * video.width;
      // What is current color
      color currentColor = video.pixels[loc];
      float r1 = red(currentColor);
      float g1 = green(currentColor);
      float b1 = blue(currentColor);
      float r2 = red(trackColor);
      float g2 = green(trackColor);
      float b2 = blue(trackColor);

      float d = distSq(r1, g1, b1, r2, g2, b2); 

      if (d < threshold*threshold) {
        stroke(255);
        strokeWeight(1);
        point(x, y);
        avgX += x;
        avgY += y;
        count++;
      }
    }
  }

  // We only consider the color found if its color distance is less than 10. 
  // This threshold of 10 is arbitrary and you can adjust this number depending on how accurate you require the tracking to be.
  if (count > 20) { 
    avgX = avgX / count;
    avgY = avgY / count;
    // Draw a circle at the tracked pixel
    println(avgX + " is avgX") ;
    println(avgY + " is avgY") ;
    fill(255);
    strokeWeight(4.0);
    stroke(0);
    ellipse(avgX, avgY, 24, 24);
  }
  textFont(font, 45); 
  fill (r);
  text("avgX, AvgY:", 10, 930 ); 
  text( nf(avgX, 0, 1), 240, 930);
  text( nf(avgY, 0, 1), 380, 930);
  text( "     ", 240, 930);
  text( "     ", 380, 930);
} // end of draw

float distSq(float x1, float y1, float z1, float x2, float y2, float z2) 
{
  float d = (x2-x1)*(x2-x1) + (y2-y1)*(y2-y1) +(z2-z1)*(z2-z1);
  return d;

  // */
}

void mousePressed() {
  if (mouseButton == LEFT) { // Save color where the mouse is clicked in trackColor variable
    int loc = mouseX + mouseY*video.width; // definition of location by pixel number.
    trackColor = video.pixels[loc];
    println(loc + " is loc") ;
    println(trackColor + " is trackColor") ;
  } else if (mouseButton == RIGHT) 
  {
  }
}

void mouseWheel(MouseEvent event) {
  Wheel = event.getCount();
  WheelAcc = WheelAcc + Wheel;
  //println(WheelAcc + "Acc") ;
  //println(Wheel + "Wheel") ;
  println(threshold + " is Treshold") ;
  Wheel = 0;
}
1 Like

Did you try to draw rect before your text for a background for your text ?

1 Like

Thanks a lot for the quick reply.
That worked. Why is this happening?

And one more question please: I dont understand the line:
trackColor = video.pixels[loc];

It is returning a color in the format " -6377867" for example, for red. What does the number mean?
I was used with RGB values, or Hex.

Thanks

I think it is because you miss the background(0); in the draw()

1 Like

O my God that was so simple.
I aded it and it worked, TNKS a. lot pascal

the trackColor is a number that contain the color of one pixel , it can be divided
in red , blue and green like this

    color trackColor = video.pixels[loc];
     fill(trackColor);
 
     text(str(int(red(trackColor)))+","+str(int(green(trackColor)))+","+str(int(blue(trackColor))),mouseX-160+55,mouseY-160+55);
1 Like

Here is an installation I did this year , and I have stated with the Daniel Blob Tracking tutorail.
https://vimeo.com/291156737

1 Like

Pascal,

Thanks a lot for the clarifications. I saw your presentation, very nice. Are you a veterinarian?

I am trying to make a sketch that will pick a dice from the table by a robotic arm.
I will let you know when is done,

No but I work for veterinarian at Université de Montréal, I did this art project in collaboration with a veterinairan searcher.

For your dice project will you have to detect bolb with computer vision?
If yes look this tuto, that was my starting point for my project.

Bolb detection tutorial by Daniel Shiffman
good lock and let my know

1 Like

Pascal,
Since we were inspired from the same project Daniels, sketch_11_9_BlobTrackingIDs.pde I would need your help again.
I was successful in building an interface for my robot in order to pick one blob at the time off the table.
See the video, at

Next I am trying to make it take multiple blobs so I adapted the above example.
The problem I have, I dont know how to extract the XY for each blob. Honestly, I never got good with writing Class.

So how can I write a function that will give something like:
Blob 0, X0, Y0
Blob 1, X1, Y1
Blob 2, X2, Y2

Than I would convert these values in XYZ Robot coordinates and sent it to pick them up,.
The robot is able to send a signal to the processing sketch after dropping each piece.
Then the sketch would push out the next coordinates.

Also I dont know how to make the program stop looking for blobs when it reaches 3 for instance.

I wonder if you’d want to collaborate on this,
Merci

1 Like

to stop looking for blobs:
If you stated with Daniel’s code you should have something like this:
for (Blob b : currentBlobs)
you can check if the size of the currentBlobs is <=3

for (Blob b : currentBlobs) 
    {
      //stratégie de limitation de blob 
      if(currentBlobs.size()<=3)
      if (b.isNear(x, y, 5)) {
        b.add(x, y);
        found = true;
        break;
      }
    }
    

Here a part of my blob class :
but this is a very different projet than your’s so it hard to say if it will help you.

    class Blob {
  float minx;
  float miny;
  
  float Pminx;//Stabilisation de la position de node
  float Pminy;//Stabilisation de la position de node
  
  float maxx;
  float maxy;
  int diametre=3;//15 
  int id = 0;
  color couleur=#711118;

  int lifespan = maxLife;

  boolean taken = false;

  Blob(float x, float y,color _couleur) {
    minx = x;
    miny = y;
    Pminx = x;//stabilisation
    Pminy = y;//stabilisation
    maxx = x;
    maxy = y;
    couleur=_couleur;
  }
  ...

To help you more I would need to see you code.

Ok, below is my code.
I did not change anything in the Blob file,

The Problem:
I was able to print out
println ("x= " + x, "y= " + y);
and it gives me the coordinates of each 5 red spots.

But they all = X and Y
29%20PM

How would I be able generate 5 values for X and 5 For Y.

Such as
X_0 =82; Y_0=264;
X_1=100; Y_1= 334

ETC

Here is my interface. It freezes video when 5 items are found.

void draw() {
background(bk);
stroke(w);
strokeWeight(3);
line (320, 0, 320, 500);

video.loadPixels();
image(video, 0, 0);

textFont(font, 25);
fill(w);
if (Aquire)
text("VIDEO ", 650, 40 );
else
text("FREEZE ", 650, 40 );

threshold = Slider1Output;
distThreshold=Slider2Output;
ArrayList currentBlobs = new ArrayList();

// Begin loop to walk through every pixel
for (int x = 0; x < video.width; x++ ) {
for (int y = 0; y < video.height; y++ ) {
int loc = x + y * video.width;
// What is current color
color currentColor = video.pixels[loc];
float r1 = red(currentColor);
float g1 = green(currentColor);
float b1 = blue(currentColor);
float r2 = red(trackColor);
float g2 = green(trackColor);
float b2 = blue(trackColor);

  float d = distSq(r1, g1, b1, r2, g2, b2); 

  if (d < threshold*threshold && Aquire==true) {

    boolean found = false;
    for (Blob b : currentBlobs) {
      if (b.isNear(x, y)) {
        b.add(x, y);
        found = true;
        break;
      }
    }

    if (!found) {
      Blob b = new Blob(x, y);
      currentBlobs.add(b);
      println ("x= " + x, "y= " + y);
      
    }
  }
}

}

for (int i = currentBlobs.size()-1; i >= 0; i–) {
if (currentBlobs.get(i).size() < SizeTresh) {
currentBlobs.remove(i);
}
}

// There are no blobs!

if (blobCounter<=4 && Aquire==true)
{
if (blobs.isEmpty() && currentBlobs.size() > 0 ) {
println(“Adding blobs!”);
for (Blob b : currentBlobs) {
b.id = blobCounter;
blobs.add(b);
blobCounter++;

}

} else if (blobs.size() <= currentBlobs.size()) {
// Match whatever blobs you can match
for (Blob b : blobs) {
float recordD = 1000;
Blob matched = null;
for (Blob cb : currentBlobs) {
PVector centerB = b.getCenter();
PVector centerCB = cb.getCenter();
float d = PVector.dist(centerB, centerCB);
if (d < recordD && !cb.taken) {
recordD = d;
matched = cb;
}
}
matched.taken = true;
b.become(matched);
}

// Whatever is leftover make new blobs
for (Blob b : currentBlobs) {
  if (!b.taken) {
    b.id = blobCounter;
    blobs.add(b);
    blobCounter++;
  }
}

} else if (blobs.size() > currentBlobs.size()) {
for (Blob b : blobs) {
b.taken = false;
}

// Match whatever blobs you can match
for (Blob cb : currentBlobs) {
  float recordD = 1000;
  Blob matched = null;
  for (Blob b : blobs) {
    PVector centerB = b.getCenter();
    PVector centerCB = cb.getCenter();         
    float d = PVector.dist(centerB, centerCB);
    if (d < recordD && !b.taken) {
      recordD = d; 
      matched = b;
    }
  }
  if (matched != null) {
    matched.taken = true;
    matched.become(cb);
  }
}

for (int i = blobs.size() - 1; i >= 0; i--) {
  Blob b = blobs.get(i);
  if (!b.taken) {
    blobs.remove(i);
  }
}

}

}// end of if (blobCounter<=5)

else
Aquire = false;

fill (trackColor);
rectMode(CENTER);

rect(30, 530, 40, 40, 7);
textFont(font, 15);
fill(w);
text("RIGHT CLICK ON TARGET ", 70, 500 );
textFont(font, 25);

text(“trackColor= “, 70, 540 );
text( trackColor, 240, 540);
text(“RGB= “, 380, 540 );
text(str(int(red(trackColor)))+”,”+str(int(green(trackColor)))+”,”+str(int(blue(trackColor))), 460, 540);

//text("Colour threshold= ", 10, 640 );
// text( nf(threshold, 0, 1), 240, 640);

//textAlign(LEFT);
text("Color threshold: " + nf(threshold, 0, 1), 10, 620);
fill(o);
text("Dist threshold: " + nf(distThreshold, 0, 1), 10, 680);

text("BLOB SIZE TRESH: " + SizeTresh, 10, 740);

text("blobCounter: " + blobCounter, 10, 890);

for (Blob b : blobs) {
b.show();
}

Hello, What do you mean by generate 5 values for X and Y?
Do you mean to give arbritary values , it’s not clear to me what you want to do.

Thanks for the quick reply, Pascal

As you can see in the interface picture, there are 5 red marbles on the mat.
I have a robot arm that has to pick them all, one by one.
In order to do that, it would have to know the X and Y for the 0 blob, X and Y

for the 1 blob, for the 2 blob till the last one.
How else would he know where to go?

Do I make sense?

// try to add this line
println (“X_”+id +“=”+ minx +" Y+“+id+”=" + maxy);
in the blob class

  void show() {
    stroke(0);
    fill(255, 100);
    strokeWeight(2);
    rectMode(CORNERS);
    rect(minx, miny, maxx, maxy);
    
    textAlign(CENTER);
    textSize(64);
    fill(0);
    text(id, minx + (maxx-minx)*0.5, maxy - 10);
    
    // try to add this line
    println ("X_"+id +"="+ minx +" Y+"+id+"=" + maxy);
  }

Ces’t Genial! I took french in school for a bunch of years.

Thanks a lot, Now I got to make global values out of them and convert them in robot coordinates.

I got
X_0=86.0 Y+0=281.0
X_1=99.0 Y+1=332.0
X_2=147.0 Y+2=292.0
X_3=198.0 Y+3=225.0
X_4=219.0 Y+4=292.0

I am also having trouble clearing all the values and the found blobs.
I tried to use in my CLR BLOBS button code:

public void button3_click1(GButton source, GEvent event) { //CODE:button3:521277:
Aquire=false;
blobCounter =0;
blobs.remove(0);
blobs.remove(1);
blobs.remove(2);
blobs.remove(3);
blobs.remove(4);
blobs.remove(5);
blobs.remove(6);
blobs.remove(7);

println(“REMOVE BLOBS”);
}

It does not reset all blobs.

Do you know a better way to clean up?
The idea is that after the robot pics the first 5 , a fresh reading should be made.

Thanks

I think this should be like:

println ("X_"+id +"="+ minx +" Y_"+id+"=" + maxy);

if you want this format X_0 =82; Y_0=264;
I just put a + instead of a _

Do you passe the (X,Y) to the Uarm with an Arduino?

En francais cette fois:
Si tu sort la ligne :ArrayList currentBlobs = new ArrayList();
du draw() pour en faire une variable global
Tu pourras essayer de mettre les lignes suivantes dans ton bouton:


public void button3_click1(GButton source, GEvent event) { //CODE:button3:521277:
  Aquire=false;
  blobCounter =0;`
  blobs.clear();
  currentBlobs.clear();//just if it is global
  
  println ("========================",blobs.size());
  println ("========================",currentBlobs.size());

   println(“REMOVE BLOBS”);
}

Ca Marche tres bien.

Mon fis et moa regarded le “Aventures de Elepahnt” . Tres drolle, Nous ne amusount bocoupt.

1 Like