Motion Detection of Multiple Objects

Processing Libraries
1

Hi, I’m working on a project where I’m using a top-down camera over a space and I need to be able to get multiple people’s positions and a single x and y value each based on their movement. People will be the only thing moving in the camera’s frame. I’m using these points of movement to control the gain of various sounds based on the distance between them and the areas I designated for each sound. I tried adding code from the 11.6: Computer Vision: Motion Detection - Processing Tutorial and got motion detection from one point working well with the rest of the program, but I would like to track two or more points at a time.

From the examples I’m seeing around the internet, it looks like blob detection is the way to go, but I don’t need the whole areas they outline, and they look for certain colors. Is there a way I can use blobs to detect where the centers of movement are without having to track specific colors and can I get specific variable names for each blob to use for other parts of the code?

This is the part of the code dealing with motion detection from one source:

loadPixels();
  // Begin loop to walk through every pixel
  for (int x = 0; x < video.width; x++ ) {
    for (int y = 0; y < video.height; y++ ) {
      int loc = x + y * video.width;
      // What is current color
      color currentColor = video.pixels[loc];
      float r1 = red(currentColor);
      float g1 = green(currentColor);
      float b1 = blue(currentColor);
      color prevColor = prev.pixels[loc];
      float r2 = red(prevColor);
      float g2 = green(prevColor);
      float b2 = blue(prevColor);

      float d = distSq(r1, g1, b1, r2, g2, b2); 

      if (d > threshold*threshold) {
        //stroke(255);
        //strokeWeight(1);
        //point(x, y);
        avgX += x;
        avgY += y;
        count++;
        //pixels[loc] = color(255);
      } else {
        //pixels[loc] = color(0);
      }
    }
  }
  updatePixels();

  // We only consider the color found if its color distance is less than 10. 
  // This threshold of 10 is arbitrary and you can adjust this number depending on how accurate you require the tracking to be.
  if (count > 200) { 
    motionX = avgX / count;
    motionY = avgY / count;
    // Draw a circle at the tracked pixel
  }
 
  lerpX = lerp(lerpX, motionX, 0.1); 
  lerpY = lerp(lerpY, motionY, 0.1); 
 
//take motion from 320x240 video to a 1920x1080 screen
//pos1X and pos1Y are used to set the gain of sound loops based on distance from designated coordinates
  pos1X = width-lerpX*6;
  pos1Y = lerpY*4.5;

I apologize if this question is too vague. I feel like I’m in way over my head and I’m not sure I’m looking in the right direction, thanks.

1 Like
2

are you aware of this code repository?
https://github.com/CodingTrain/website/tree/master/Tutorials/Processing/11_video ?
because you link to a p5.js video and show JAVA code…

or even look for a library? like
https://github.com/atduskgreg/opencv-processing ?

1 Like
3

It looked like he was using java, it works for me :man_shrugging:. The repository is where I got the code from. The background subtraction from OpenCV looks interesting, but I don’t think there is any advantage over looking at the previous frame instead. By the way, my program is not displaying the video and I would rather have a “press play and it works” kind of setup.

4

Sorry for the double post, it doesn’t look like I can just edit the last one.

I made some progress on this code but I still need some help. I’ve got blob detection going based on the difference between the previous frame and current and its kinda jittery but it works for now. Is there a way I can get variables that get the center of each blob from minx, miny, maxx, and maxy so I can use those x and y values in void draw() and smooth out the movement?

import processing.video.*;

import blobDetection.*;

class Blob {
  float minx;
  float miny;
  float maxx;
  float maxy;
  int id = 0;

  boolean taken = false;

  Blob(float x, float y) {
    minx = x;
    miny = y;
    maxx = x;
    maxy = y;
  }

  void show() {
    stroke(0);
    fill(255, 100);
    strokeWeight(2);
    rectMode(CORNERS);
    rect(minx, miny, maxx, maxy);
    textAlign(CENTER);
    textSize(12);
    fill(0);
    text(id, minx + (maxx-minx)*0.5, maxy - 10);
  }


  void add(float x, float y) {
    minx = min(minx, x);
    miny = min(miny, y);
    maxx = max(maxx, x);
    maxy = max(maxy, y);
  }

  void become(Blob other) {
    minx = other.minx;
    maxx = other.maxx;
    miny = other.miny;
    maxy = other.maxy;
  }

  float size() {
    return (maxx-minx)*(maxy-miny);
  }

  PVector getCenter() {
    float x = (maxx - minx)* 0.5 + minx;
    float y = (maxy - miny)* 0.5 + miny;    
    return new PVector(x, y);
  }

  boolean isNear(float x, float y) {

    float cx = max(min(x, maxx), minx);
    float cy = max(min(y, maxy), miny);
    float d = distSq(cx, cy, x, y);

    if (d < distThreshold*distThreshold) {
      return true;
    } else {
      return false;
    }
  }
}

int blobCounter = 0;

int maxLife = 200;

Capture video;

PImage prev;


float threshold = 75;
float distThreshold = 20;

ArrayList<Blob> blobs = new ArrayList<Blob>();

void setup() {
  size(640, 360);
  String[] cameras = Capture.list();
  printArray(cameras);
  video = new Capture(this, 320, 240);
  video.start();
  prev = createImage(320, 240, RGB);
  // trackColor = color(255, 0, 0);
}

void captureEvent(Capture video) {
  prev.copy(video, 0, 0, video.width, video.height, 0, 0, prev.width, prev.height);
  prev.updatePixels();
  video.read();
}

void keyPressed() {
  if (key == 'a') {
    distThreshold+=5;
  } else if (key == 'z') {
    distThreshold-=5;
  }
  if (key == 's') {
    threshold+=5;
  } else if (key == 'x') {
    threshold-=5;
  }


  println(distThreshold);
}

void draw() {
  video.loadPixels();
  prev.loadPixels();
  image(video, 0, 0);

  blobs.clear();

  loadPixels();

  // Begin loop to walk through every pixel
  for (int x = 0; x < video.width; x++ ) {
    for (int y = 0; y < video.height; y++ ) {
      int loc = x + y * video.width;
      // What is current color
      color currentColor = video.pixels[loc];
      float r1 = red(currentColor);
      float g1 = green(currentColor);
      float b1 = blue(currentColor);
      color prevColor = prev.pixels[loc];
      float r2 = red(prevColor);
      float g2 = green(prevColor);
      float b2 = blue(prevColor);

      float d = distSq(r1, g1, b1, r2, g2, b2); 

      if (d > threshold*threshold) {

        boolean found = false;
        for (Blob b : blobs) {
          if (b.isNear(x, y)) {
            b.add(x, y);
            found = true;
            break;
          }
        }

        if (!found) {
          Blob b = new Blob(x, y);
          blobs.add(b);
        }
      }
    }
  }

  for (Blob b : blobs) {
    if (b.size() > 500) {
      b.show();
    }
  }


  textAlign(RIGHT);
  fill(0);
  textSize(12);
  text("distance threshold: " + distThreshold, width-10, 25);
  text("color threshold: " + threshold, width-10, 50);
}


// Custom distance functions w/ no square root for optimization
float distSq(float x1, float y1, float x2, float y2) {
  float d = (x2-x1)*(x2-x1) + (y2-y1)*(y2-y1);
  return d;
}


float distSq(float x1, float y1, float z1, float x2, float y2, float z2) {
  float d = (x2-x1)*(x2-x1) + (y2-y1)*(y2-y1) +(z2-z1)*(z2-z1);
  return d;
}
1 Like
5

Your code lists import blobDetection.*; – but you also wrote your own Blob class (I think?). Did you look at the examples for that library, for example bd_image.pde?

For that, you just use b.x and b.y. All those values are in the range 0-1, giving the locations of the blob bounding box and centroid on a 1x1 unit square. To draw them, multiply by the image width and height:

Blob b;
// ...
ellipse(b.x*width, b.y*height, 5, 5);   // mark center of blob b

Also, if you type:

Blob b;
b.

…and wait, after a moment PDE will pop up a list all the available methods and properties of Blob. That depends on you not defining your own Blob class on top of the library, of course.

Example sketch with bounding boxes and centroids:

import blobDetection.*;

BlobDetection blobs;
PImage img;
float thresh=0.6;

void setup() {
  size(512, 512);
  // load image
  img = loadImage("https://upload.wikimedia.org/wikipedia/commons/thumb/8/89/Yellow-greenholes.svg/480px-Yellow-greenholes.svg.png");
  // configure detector
  blobs = new BlobDetection(img.width, img.height);
  blobs.setPosDiscrimination(false);
  blobs.setThreshold(thresh);
  blobs.computeBlobs(img.pixels);
}

void draw() {
  image(img, 0, 0, width, height);
  drawBlobs(true, true);
}

void drawBlobs(boolean drawBounds, boolean drawCenters) {
  Blob b;
  noFill();
  stroke(255, 0, 0);
  for (int n=0; n<blobs.getBlobNb(); n++) {
    b=blobs.getBlob(n);
    if (drawBounds && b!=null) {
      rect( b.xMin*width, b.yMin*height, b.w*width, b.h*height);
    }
    if (drawCenters && b!=null) {
      ellipse( b.x*width, b.y*height, 5, 5);
    }
  }
  fill(0);
  text(thresh + "\ndrag mouse up/down to adjust", 5, 15);
}

void mouseDragged() {
  thresh=mouseY/float(height);
  blobs.setThreshold(thresh);
  blobs.computeBlobs(img.pixels);
}