Display OpenCV face detection controlled by MIDI

Processing Coding Questions
1

Hello everybody,

Im a artist and new to processing en coding in general.

Im trying to make a code that captures a image of a face and use that to make a dynamic Image composed of the faces that watches it.
it uses Opencv face detection to detect the face. and it uses a midi controller to output to a external program for sound.

Ive been merging different liberaries en trying to make the code work. So far ive managed to place copies of faces on top of the image.

I want the images to stack on top of each other. The entire screen should fill up with different images of faces depending on who watching where in the screen. I tried switching places in the draw section, but cant get it to work. please take a look at the code and help me to fix it! Or maybe i should approach this in a other way. Any feedback will help!

import gab.opencv.*;
import processing.video.*;
import java.awt.*;
import themidibus.*;
 PImage img;
  PImage saveImage; 
PImage partialSave;
 
 
MidiBus myBus;
Capture video;
OpenCV opencv;



void setup() {
  size (640, 480);
  
  frameRate(15);
  MidiBus.list(); // List all available Midi devices on STDOUT. This will show each device's index and name.
  myBus = new MidiBus(this, -1, 3);  
  video = new Capture(this, 640, 480);
  opencv = new OpenCV(this, 640, 480);
  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);  
 
 loadPixels();
  updatePixels();
img = loadImage("FaceSnap.jpg");

  
  video.start();
 
 
}

void draw() {
 
 
  opencv.loadImage(video);
   Rectangle[] faces = opencv.detect();
  println(faces.length);
 

  

  for (int i = 0; i < faces.length; i++) {
    println(faces[i].x + "," + faces[i].y);
                
  
    float valuex = faces[i].x;
        myBus.sendControllerChange(1, 20,int(valuex)); // Send a controllerChange
    float valuey = faces[i].y;
        myBus.sendControllerChange(1, 21,int(valuey)); // Send a controllerChange
    
    float valuez = faces[i].width; //extra controller
      
    float valueq = faces[i].height; //extra controller

 partialSave = get(faces[i].x, faces[i].y, faces[i].width, faces[i].height);  

  partialSave.save("FaceSnap.jpg");



background(0,0,0);
   image(video,0,0);
   
image (img, valuex, valuey);
img = loadImage("FaceSnap.jpg");
   

  
  
}
  
}

void captureEvent(Capture c) {
  c.read();
}
2

Can you post the console error code?

3

well, the code works so far , but i need to know where to look next… the part of not working is that i cant keep the created face image in front of every thing. I think i only need the video for making the facesnap. im trying to make something like this. but that with a camvideo as input

Voorbeeldbeeld
Voorbeeldbeeld.jpg1600Ă—900 534 KB

4

Do you want to do this ? :

import gab.opencv.*;
import processing.video.*;
import java.awt.*;
PImage img;
 
Capture video;
OpenCV opencv;


void setup() {
  size (640, 480);
  frameRate(15); 
  video = new Capture(this, 640, 480);
  opencv = new OpenCV(this, 640, 480);
  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);  
 
  loadPixels();
  updatePixels();
  img = createImage(100,100,RGB);
  video.start();
}

void draw() {
  opencv.loadImage(video);
  Rectangle[] faces = opencv.detect();
  //println(faces.length);
  
  for (int i = 0; i < faces.length; i++) {
    //println(faces[i].x + "," + faces[i].y);
    img = video.get(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
    //background(0,0,0);
    //image(video,0,0);
    image (img, random(width), random(height));
  }
  
}

void captureEvent(Capture c) {
  c.read();
}
1 Like
5

yea, you simplified it nicely! I was making things to complicated… This is a good base for me so i can work from this code now, thx for the help!

Do you also know somewhere i can look for some code for midi control output en stuff. I cant get more than 1 signal at the time for mine. thx again!

6

If you want to use The MidiBus library, you should check :

I never used it so I can’t give you advices. :smile:

1 Like
7

Interesting use of face detection. in the future, i would make your subject line more descriptive to help make your post stand out:-) This is a great project. Good luck!

8

I’d create a class called Face that contains a PImage and any other information you’d like to keep track of (like a PVector for position). Then I’d use an ArrayList of Face objects and add a new one for each new face image found. Look at Shiffman’s particle examples for how this would work.

The face detection methods are expensive so I would wrap them in an if (video.available()) statement. That way they only run when the camera sends a new frame, instead of every draw() loop.

Keep going, you’re almost there!