Color from FFT center frequency and its amplitude

Processing Coding Questions
1

Hello

This is my first post. I am trying to do a project with a variant of “sound painting” with a theremin. Let me explain a bit. The basic idea is to acquire sound from a theremin (with a microphone), then perform a FFT on the signal, calculate average frequency band and amplitude of the most dominant frequency. I want to do coloring with basic colors, so for that I would need to acquire a few frequencies from the theremin. For example I would generate 8 sound tones that have 8 specific frequencies with their amplitudes. To each of those colors I would assign a different color. In the end I would paint a rectangle with the paint that is “played” with the theremin.

To perform a FFT analysis I did no reinvent any thing. I took an example from Minim library, SoundSpectrum.

I changed the code to this:

import ddf.minim.analysis.*;
import ddf.minim.*;

Minim minim;  
//AudioPlayer jingle;
AudioInput jingle;
FFT fftLin;
FFT fftLog;

float height3;
float height23;
float spectrumScale = 4;

PFont font;

void setup()
{
  size(500, 500);
  height3 = height/3;
  height23 = 2*height/3;

  minim = new Minim(this);
  //jingle = minim.loadFile("test.mp3", 1024);
  jingle = minim.getLineIn(Minim.MONO, 1024, 44100);
  
  // loop the file
  //jingle.loop();
  
  // create an FFT object that has a time-domain buffer the same size as jingle's sample buffer
  // note that this needs to be a power of two 
  // and that it means the size of the spectrum will be 1024. 
  // see the online tutorial for more info.
  fftLin = new FFT( jingle.bufferSize(), jingle.sampleRate() );
  
  // calculate the averages by grouping frequency bands linearly. use 30 averages.
  fftLin.linAverages( 100 );
  
  // create an FFT object for calculating logarithmically spaced averages
  fftLog = new FFT( jingle.bufferSize(), jingle.sampleRate() );
  
  // calculate averages based on a miminum octave width of 22 Hz
  // split each octave into three bands
  // this should result in 30 averages
  fftLog.logAverages( 22, 3 );
  
  rectMode(CORNERS);
  font = loadFont("ArialMT-12.vlw");
  
  println(jingle.bufferSize());
  println(jingle.sampleRate());
}

void draw()
{
  background(0);
  
  textFont(font);
  textSize( 18 );
 
  float centerFrequency = 0;
  float centerFrequency1 = 0;
  float amplitudeFrequency = 0;
  
  // perform a forward FFT on the samples in jingle's mix buffer
  // note that if jingle were a MONO file, this would be the same as using jingle.left or jingle.right
  fftLin.forward( jingle.mix );
  fftLog.forward( jingle.mix );
 
  // draw the linear averages
  {
    // since linear averages group equal numbers of adjacent frequency bands
    // we can simply precalculate how many pixel wide each average's 
    // rectangle should be.
    int w = int( width/fftLin.avgSize() );
    for(int i = 0; i < fftLin.avgSize(); i++)
    {
      centerFrequency = fftLin.getAverageCenterFrequency(i);
      amplitudeFrequency = fftLin.getFreq(i);
      // if the average center frequency is between a specified boundries and its amplitude is above specified level than write that frequency
      if (centerFrequency >= 4000 && centerFrequency <= 4500 && amplitudeFrequency > 30)
      {
        //if (amplitudeFrequency > 30)
        //  {
            println(centerFrequency);
            fill(255, 128);
            text("Linear Average Center Frequency: " + centerFrequency, 5, height23 - 25);
          //}
      }
       rect(i*w, height, i*w + w, height - fftLin.getAvg(i)*spectrumScale);
    }
  }
  
  //float x1 = map(centerFrequency, 200, 800, 0, 255);
  //float x2 = map(centerFrequency, 200, 800, 0, 150);
  //float x3 = map(centerFrequency, 200, 800, 0, 125);
  //fill(x1,x2,x3);
  //rect(0, 0, 150, 150);
}

So you can see, that I used the part where the linear average is calculated. I included getFreq() class to extract the amplitude from the average center frequency.

For testing I would just like to write to the console and to the screen when the criteria for the right frequency is met.
The main problem I have is the speed of the console output. It is laggy i.e. the response is not so quick in comparison to the original example where I change the input (so I am always acquiring the sound through the mic.

So what I would like to achieve is a smother (more fluent, more “real time” feeling) output of the center frequency.

Obviously the problem lies on my part in understanding and writing the code. I dont believe that the PC is to slow, my understanding is :smile: :blush:

So I am asking you kind, more experienced users to help a friend in need.

Best regards

2

[ONE MONTH LATER…]
Sorry for replying that late.
I’m not sure if I’m understanding your problem.
So if I’m right, you want to have a color based on the frequency and a color based on the amplitude of the current sound input. Then you would combine them to another color and display this color on the screen.
So if you’d have red for the frequency and green for the amplitude, you want the sketch to display a yellow rectangle.
Please correct me if I’m wrong.

3

For real-time data inspection I would recommend using text() and drawing the values each frame to the screen – and or writing to a log file.

For color outputs that are binned (like a crayon box) or that fade continuously into one another (like a multi-value gradient) you will have slightly different mapping methods. For continuous color transitions between 8 tone-colors, use lerpColors.