Live Face Tracking (sans OpenCV)

Gallery
1

Hello everyone,

This technique is for anyone wanting to do some pretty decent face tracking with their webcam. I was not happy with the accuracy of OpenCV, so I came up with another method, it involves processing and two separate, open source programs that are used to control flight simulators.

The first program is AITrack, which uses neural networks to track faces. This data is sent and picked up via UDP to opentrack, which parses that data and can output it via UDP (or other methods) to the address and port of your choosing. After that you can pickup the data stream in processing and convert it to int’s or float’s and do what you want with it.

import hypermedia.net.*;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;

UDP udp;
int PORT_RX= 7000; // set port in opentrack
String HOST_IP="192.168.86.28";
String receivedFromUDP = "";
int x,y,z;

void setup() {
  size(200, 100);
  udp= new UDP(this, PORT_RX, HOST_IP);
  //udp.log(true);
  udp.listen(true);
  super.start();
}

void draw() {
  background(0);
  textSize(20);
  text(x, 20, 30);
  text(y, 20, 50);
  text(z, 20, 70);
}

void receive(byte[] data) {    
  ByteBuffer bb = ByteBuffer.allocate(data.length);
  bb.order(ByteOrder.LITTLE_ENDIAN);
  bb.put(data);
  bb.rewind();
  x = (int) bb.getDouble();
  y = (int) bb.getDouble();
  z = (int) bb.getDouble();
  //println("x,y,z: " + xx + ", " + yy + ", " + zz);
}

float byteArrayToFloat(byte[] data) {
  int intBits = 
    data[3] << 24 | (data[2] & 0xFF) << 16 | (data[1] & 0xFF) << 8 | (data[0] & 0xFF) |
    data[4] << 32 | (data[5] & 0xFF) << 40 | (data[6] & 0xFF) << 48 | (data[7] & 0xFF) << 56;
  return Float.intBitsToFloat(intBits);
}

NOTE: To see the codes’ evolution and it’s contributors, see the original thread here. Thank you to those that helped with this.