A portrait that watches you (beginner project)

Processing for Pi
1

Hi!
I’m new to Processing and have very limited coding skills in general, please bear with me.

I’m trying to create a portrait whose “eyes follow you around the room” like the painting cliché, but in a very literal sense with the help of a webcam and facetracking with opencv.

I have a managed to put together a (mostly) working sketch, for now just with a single eye. A black ellipse (the “pupil”) is drawn over the nearest (largest) detected face, then a .png image is drawn of an eye socket that is either open or closed, depending on whether or not a face is detected.

eyecontact
eyecontact1117×600 135 KB

I would like to duplicate that sketch (so as to have a pair of eyes) and compose them within a larger drawing to create the full portrait. I’m really not sure how to go about this and would greatly appreciate any advice!

Before that though, the code needs to be optimised. Currently the sketch runs at 100% CPU load if I run it above 7fps, which seems extremely high for the resolution (640x480) and my machine (Windows 10, i5 9300H, GTX 1050). I’ve tried the standard renderer, P2D and P3D, but it makes no difference. The sketch doesn’t cap at 7fps, it runs smoothly all the way up to 60fps, but the CPU is maxed out and it sounds like I’m next to a jet engine. This makes me think that I have made a mistake somewhere in my sketch that is causing an unnecessary load. I was hoping to run the final project on a Raspberry Pi, but unless I can fix this issue that’s not going to be possible.

Also, the facetracking is not particularly reliable. This could be the quality of my webcam, lighting conditions, opencv and so on. I’m not sure if this can be improved, but any suggestions are welcome!

Here’s the sketch:

import gab.opencv.*; 
import processing.video.*; 
import java.awt.Rectangle;
PImage closed;
PImage open;
Capture cam; 
OpenCV opencv; 
Rectangle[] faces;
 
void setup() { 
  frameRate(30);
  size(640, 480, P2D); 
  background (0, 0, 0); 
  cam = new Capture( this, 640, 480, 30); 
  cam.start(); 
  opencv = new OpenCV(this, cam.width, cam.height); 
  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);
  closed = loadImage("closed.png");
  open = loadImage("open.png");
}
 
void draw() { 
  background (255, 255, 255); 
  opencv.loadImage(cam); 
  faces = opencv.detect();
  println(faces.length);
  

//find biggest/closest face
    int biggest = 0;
    int biggest_index = -1;
    if (faces!=null) {

      for (int i=0; i< faces.length; i++) {
        // print (faces[i]);       
        if (faces[i].width > biggest)
        {
  
          biggest_index = i;
          biggest = faces[i].width;
        }
      }
    }
    
  //pupil placement
  if (biggest_index >= 0) {          

 
    fill(0, 0, 0); 
    ellipseMode(CORNER);
    int i  = biggest_index;
    ellipse(-faces[i].x+640-faces[i].width, faces[i].y, faces[i].width, faces[i].height);
    image(open,-0,0);
  }
  
  //blinking and resting
  if (faces.length<=0) { 
    image(closed,-0,0);
  }
}
 
void captureEvent(Capture cam) { 
  cam.read();
}