# Display OpenCV face detection controlled by MIDI

**URL:** <https://discourse.processing.org/t/display-opencv-face-detection-controlled-by-midi/505>\
**Category:** Coding Questions\
**Created:** [May 29, 2018, 6:27pm UTC](https://discourse.processing.org/t/display-opencv-face-detection-controlled-by-midi/505 "2018-05-29T18:27:11Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![snaphaan](https://avatars.discourse-cdn.com/v4/letter/s/e36b37/32.png) [@snaphaan](https://discourse.processing.org/u/snaphaan)\
**Post date:** [May 29, 2018, 6:27pm UTC](https://discourse.processing.org/t/display-opencv-face-detection-controlled-by-midi/505/1 "2018-05-29T18:27:11Z")

</div>

Hello everybody,

Im a artist and new to processing en coding in general.

Im trying to make a code that captures a image of a face and use that to make a dynamic Image composed of the faces that watches it.  
it uses Opencv face detection to detect the face. and it uses a midi controller to output to a external program for sound.

Ive been merging different liberaries en trying to make the code work. So far ive managed to place copies of faces on top of the image.

I want the images to stack on top of each other. The entire screen should fill up with different images of faces depending on who watching where in the screen. I tried switching places in the draw section, but cant get it to work. please take a look at the code and help me to fix it! Or maybe i should approach this in a other way. Any feedback will help!

```auto
import gab.opencv.*;
import processing.video.*;
import java.awt.*;
import themidibus.*;
 PImage img;
  PImage saveImage; 
PImage partialSave;
 
 
MidiBus myBus;
Capture video;
OpenCV opencv;

void setup() {
  size (640, 480);
  
  frameRate(15);
  MidiBus.list(); // List all available Midi devices on STDOUT. This will show each device's index and name.
  myBus = new MidiBus(this, -1, 3);  
  video = new Capture(this, 640, 480);
  opencv = new OpenCV(this, 640, 480);
  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);  
 
 loadPixels();
  updatePixels();
img = loadImage("FaceSnap.jpg");

  
  video.start();
 
 
}

void draw() {
 
 
  opencv.loadImage(video);
   Rectangle[] faces = opencv.detect();
  println(faces.length);
 

  

  for (int i = 0; i < faces.length; i++) {
    println(faces[i].x + "," + faces[i].y);
                
  
    float valuex = faces[i].x;
        myBus.sendControllerChange(1, 20,int(valuex)); // Send a controllerChange
    float valuey = faces[i].y;
        myBus.sendControllerChange(1, 21,int(valuey)); // Send a controllerChange
    
    float valuez = faces[i].width; //extra controller
      
    float valueq = faces[i].height; //extra controller

 partialSave = get(faces[i].x, faces[i].y, faces[i].width, faces[i].height);  

  partialSave.save("FaceSnap.jpg");

background(0,0,0);
   image(video,0,0);
   
image (img, valuex, valuey);
img = loadImage("FaceSnap.jpg");
   

  
  
}
  
}

void captureEvent(Capture c) {
  c.read();
}

```

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 29, 2018, 6:43pm UTC](https://discourse.processing.org/t/display-opencv-face-detection-controlled-by-midi/505/2 "2018-05-29T18:43:57Z")

</div>

Can you post the console error code?

---

<div class="post-metadata">

**Author:** ![snaphaan](https://avatars.discourse-cdn.com/v4/letter/s/e36b37/32.png) [@snaphaan](https://discourse.processing.org/u/snaphaan)\
**Post date:** [May 29, 2018, 8:02pm UTC](https://discourse.processing.org/t/display-opencv-face-detection-controlled-by-midi/505/3 "2018-05-29T20:02:09Z")

</div>

well, the code works so far , but i need to know where to look next… the part of not working is that i cant keep the created face image in front of every thing. I think i only need the video for making the facesnap. im trying to make something like this. but that with a camvideo as input

 ![Voorbeeldbeeld](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/1X/301318f06adc6466e7824595d0200f04b3b4b63b.jpg)

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 30, 2018, 7:29am UTC](https://discourse.processing.org/t/display-opencv-face-detection-controlled-by-midi/505/4 "2018-05-30T07:29:27Z")

</div>

Do you want to do this ? :

```auto
import gab.opencv.*;
import processing.video.*;
import java.awt.*;
PImage img;
 
Capture video;
OpenCV opencv;

void setup() {
  size (640, 480);
  frameRate(15); 
  video = new Capture(this, 640, 480);
  opencv = new OpenCV(this, 640, 480);
  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);  
 
  loadPixels();
  updatePixels();
  img = createImage(100,100,RGB);
  video.start();
}

void draw() {
  opencv.loadImage(video);
  Rectangle[] faces = opencv.detect();
  //println(faces.length);
  
  for (int i = 0; i < faces.length; i++) {
    //println(faces[i].x + "," + faces[i].y);
    img = video.get(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
    //background(0,0,0);
    //image(video,0,0);
    image (img, random(width), random(height));
  }
  
}

void captureEvent(Capture c) {
  c.read();
}

```

---

<div class="post-metadata">

**Author:** ![snaphaan](https://avatars.discourse-cdn.com/v4/letter/s/e36b37/32.png) [@snaphaan](https://discourse.processing.org/u/snaphaan)\
**Post date:** [May 30, 2018, 3:50pm UTC](https://discourse.processing.org/t/display-opencv-face-detection-controlled-by-midi/505/5 "2018-05-30T15:50:09Z")

</div>

yea, you simplified it nicely! I was making things to complicated… This is a good base for me so i can work from this code now, thx for the help!

Do you also know somewhere i can look for some code for midi control output en stuff. I cant get more than 1 signal at the time for mine. thx again!

---

<div class="post-metadata">

**Author:** ![josephh](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/josephh/32/210_2.png) [@josephh](https://discourse.processing.org/u/josephh)\
**Post date:** [May 30, 2018, 3:59pm UTC](https://discourse.processing.org/t/display-opencv-face-detection-controlled-by-midi/505/6 "2018-05-30T15:59:26Z")

</div>

If you want to use The MidiBus library, you should check :

- [https://github.com/sparks/themidibus](https://github.com/sparks/themidibus) for examples
- [http://www.smallbutdigital.com/docs/themidibus/themidibus/package-summary.html](http://www.smallbutdigital.com/docs/themidibus/themidibus/package-summary.html) The documentation

I never used it so I can’t give you advices. 😄

---

<div class="post-metadata">

**Author:** ![dan850](https://avatars.discourse-cdn.com/v4/letter/d/e9c0ed/32.png) [@dan850](https://discourse.processing.org/u/dan850)\
**Post date:** [May 30, 2018, 11:36pm UTC](https://discourse.processing.org/t/display-opencv-face-detection-controlled-by-midi/505/7 "2018-05-30T23:36:13Z")

</div>

Interesting use of face detection. in the future, i would make your subject line more descriptive to help make your post stand out:-) This is a great project. Good luck!

---

<div class="post-metadata">

**Author:** ![n1ckfg](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/n1ckfg/32/119_2.png) [@n1ckfg](https://discourse.processing.org/u/n1ckfg)\
**Post date:** [May 31, 2018, 3:14pm UTC](https://discourse.processing.org/t/display-opencv-face-detection-controlled-by-midi/505/8 "2018-05-31T15:14:10Z")

</div>

I’d create a class called Face that contains a PImage and any other information you’d like to keep track of (like a PVector for position). Then I’d use an ArrayList of Face objects and add a new one for each new face image found. Look at Shiffman’s particle examples for how this would work.

The face detection methods are expensive so I would wrap them in an if (video.available()) statement. That way they only run when the camera sends a new frame, instead of every draw() loop.

Keep going, you’re almost there!