How to modify the program? Sound detection and face tracking

Hi again,

Sorry I misspelled mouth with mouse, my bad! :hugs:

In the above code, this is how you play a sound with the oscillator :

// Start the oscillator
osc.start();

// Then set its frequency and amplitude
osc.freq(freq, 0.1);
osc.amp(amp, 0.1);

Then to know when the mouth is open or not, you have the mouthDist variable that gives you the distance between two points on the mouth.

The mouth is open when this value is greater than something like so :

if (mouthDist > minimumMouthDistance) {
  // Then do something
}

where minimumMouthDistance is the minimum threshold distance where the mouth is considered open (here 3 for example but it depends on the size of the face related to the camera).

For this check this pseudo code that resume the previous points :

const minimumMouthDistance = 3;

let osc;

function setup() {
  // Create your oscillator
  osc = new p5.Oscillator("sawtooth");

  // Start it
  osc.start();
}

function draw() {
  // Get the distance of the opening of the mouth
  mouthDist = mouthTop.sub(mouthBottom).mag();

  // Check if the 
  if (mouthDist > minimumMouthDistance) {
    // Play some sound
    osc.freq(freq, 0.1);
    osc.amp(amp, 0.1);
  } else {
    // Otherwise mute the oscillator
    osc.amp(0, 0.5);
  }
}
1 Like