Audio Visualisation

Hi FOLKS…

I am super sad because I am struggling a lot with a program I am trying to code it since long…but I cannot figure out how to do it!!!

So what I want to do is:
When I press the R (keypressed) it records the audio frequency and divide them into BASS & HIGH until I release the R. With those data I want create some points in the canvas and connect them together with a line drawing a shape. For example with b1&h1 / b2&h2 / b3&h3 / b4&h4 / b5&h5 etc [how many it records during the keypressed]. I know that I need two ArrayList but I cannot understand how it really works, I saw like 7 tutorials, readed a lot of stuff but I cannot manage all these things!

I paste my code but I don’t know if it can help you, I hope you can help ME! Thankssss

import ddf.minim.*; 
import ddf.minim.analysis.*;  
Minim minim;   
FFT fft;  
AudioInput in;   
float amp = 15; // used to make signal stronger/weaker 
float ampWave = 10*amp; 
float avgAudio; // store avg volume globally 

float bass; 
float high;

BassTones = bass;
HighTones = high;

float BassTones = bass;
float HighTones = high;

ArrayList<BassTones> b;
ArrayList<HighTones> h;

void setup(){
  minim = new Minim(this); // initalize in setup 
  in = minim.getLineIn(Minim.STEREO, 512); // audio in + bufferSize 512 or 1024 
  fft = new FFT(in.bufferSize(), in.sampleRate());  
  fft.logAverages(22, 3); // 3 = 30, 4 = 40, slices of frequency

  size(800,500);
  colorMode(HSB, 100);
  background(0,0,100); 
  
}

void draw(){
  println(bass+" / "+high);
}


void keyPressed (){
  
  fft.forward(in.mix); // FFT anaylsis 
  
  bass = fft.calcAvg(20,800)*amp; // fft.calcAvg(minFreq, maxFreq)  
  high = fft.calcAvg(800,10000)*amp;  
  
  b = new ArrayList<BassTones>();
  h = new ArrayList<HighTones>();
  
  b.add(new BassTones());
  h.add(new HighTones());
  
?
  
}

Sorry @rioma7 your program is all wrong and very vague. It won’t compile. If you can clearly state your problem, perhaps I can help you, when I find time.

I know because I don’t understand how to use the ArrayList and also I don’t know how to tell him to use it to create points.

your arraylist are half mashed, the way you declare an arraylist is like this:
ArrayList<ClassType> ListName;
its not compiling as BassTones and HighTones are not classes
Also in your void keyPressed() you are initializing each time you are pressing a key which you don’t want
I also dont see why you are using an ArrayList for this.
ArrayLists are good for making a new object in a class when you want to have more than 3 of that object, if you want to be able to remove an object. I would also suggest looking at the reference for ArrayLists.

if you are going to make it so you can press R multiple times and show them all at the same time, I would suggest an ArrayList but at the same time make a Class to record and display and test it with one first


Doge

You need to break your problem down. Before you go into using ArrayList or any other container, you need to work with extracting the information from your FFT related to your frequencies of interest. Are you confident you can do this? Are you able to do it now? it is not clear from the code that you posted if you are able to do this core task in your challenge.

To do this first part, go back to the basic program in minim that extracts the frequency bands from your sample sound file. From this code, you need to generate the code that allows you to pick the BASS and the HIGH from the F spectrum. How are you choosing these values? Do you have a threshold value that separates low from the high frequency? After you get this code done, we can go to a second step which is saving these value pairs when you press a key. A third task is Saving the value pairs in a container like an ArrayList. Finally, one works in the visualization part.

Kf

1 Like

Thanks guys… now I am going step by step and for the moment I coded this, which is the first step…using Bass of FFT as X and High as Y to create a point. What I would like to do is an beginShape but instead of the vertex I would like to use those X&Y…and I thought that I probably need an ArrayList because I don’t know each time how many data I record!

  import ddf.minim.*; 
  import ddf.minim.analysis.*;  
  Minim minim;   
  FFT fft;  
  AudioInput in;   
  float amp = 15; // used to make signal stronger/weaker 
  float ampWave = 10*amp; 
  float avgAudio; // store avg volume globally 
  
  float bass; 
  float high; 
  
  float x;
  float y;

void setup(){
  minim = new Minim(this); // initalize in setup 
  in = minim.getLineIn(Minim.STEREO, 512); // audio in + bufferSize 512 or 1024 
  fft = new FFT(in.bufferSize(), in.sampleRate());  
  fft.logAverages(22, 3); // 3 = 30, 4 = 40, slices of frequency
  
  size(640,360);
  background(255);
}

void keyPressed(){
  fft.forward(in.mix); // IMPORTANT! -update for FFT anaylsis
  
  bass = fft.calcAvg(20,800)*amp; // fft.calcAvg(minFreq, maxFreq)  
  high = fft.calcAvg(800,10000)*amp;
  
  println(bass+" / "+high);
}

void draw(){
  x = bass;
  y = high;
  
  strokeWeight(2);
  point(x, y);
}

any input about the last code @kfrajer?

If you look at the output from your println, you will see that you are getting very small values. You need to adjust your amplitude from 15 to 1500 so to be able to have a responsive sketch. Now, you will need to visualize the frequency of your incoming data before you start making decisions about what data to plot, what to select and what to discard. You will need to study the fft of a single sound sample, become familiar with it, and then extract the information from it before you start testing across different sound sources.

Kf


//REFERENCE: https://forum.processing.org/two/discussion/23097/how-can-i-determine-which-spectrum-is-generated-from-which-file-on-mouseclick#latest



import ddf.minim.analysis.*;
import ddf.minim.*;

Minim minim;
AudioInput in;
FFT fft;
AudioPlayer ap;
// how many individual peak bands we have (dep. binsperband)
float gain = 0; // in dB
float dB_scale = 2.0;  // pixels per dB
int buffer_size = 1024;  // also sets FFT size (frequency resolution)
float sample_rate = 44100;

int spectrum_height = 200; // determines range of dB shown
int legend_height = 20;
int spectrum_width = 512; // determines how much of spectrum we see
int legend_width = 40;
int freqOffset = 0;

final color colSelected=color(255,0,0);
final color colPlain=color(255);

void setup()
{
  size(552, 220, P2D);
  textMode(SCREEN);
  //textFont(createFont("Georgia", 12));

  minim = new Minim(this);

  //in = minim.getLineIn(Minim.MONO,buffer_size,sample_rate);
  ap = minim.loadFile("da.mp3");
  // create an FFT object that has a time-domain buffer 
  // the same size as line-in's sample buffer
  fft = new FFT(ap.bufferSize(), ap.sampleRate());

  // Tapered window important for log-domain display
  fft.window(FFT.HAMMING);

  //println("buffer:"+in.bufferSize());
  //println("sampleRate:"+in.sampleRate());
  //println("peaksize:"+peaksize);
}


void draw()
{
  // clear window
  background(0);

  // perform a forward FFT on the samples in input buffer
  fft.forward(ap.mix);
  ap.play();

  // now draw current spectrum in brighter blue
  stroke(64, 255, 255);
  noFill();
  for (int i = 0; i < fft.specSize(); i++) {

    // draw the line for frequency band i using dB scale
    float val = dB_scale*(20*((float)Math.log10(fft.getBand(i))) + gain);

    if (fft.getBand(i) == 0) {   
      val = -200;
    }  // avoid log(0)

    int y = spectrum_height - Math.round(val);

    if (y > spectrum_height) { 
      y = spectrum_height;
    }

    float x1=legend_width+i+freqOffset;
    float y1=spectrum_height;
    float x2=x1;
    float y2=y;

    line(x1, y1, x2, y2);

    pushStyle();
    if (dist(mouseX, mouseY, x2, y2)<20) 
      fill(colSelected);
    else
      fill(colPlain);
    noStroke();
    ellipse(x2, y2, 3, 3);
    popStyle();

    // update the peak record
    // which peak bin are we in?
    //int peaksi = i/binsperband;
    //if (val > peaks[peaksi]) {
    //  peaks[peaksi] = val;
    //  // reset peak age counter
    //  peak_age[peaksi] = 0;
    //}
  }
  // add legend
  // frequency axis
  fill(255);

  stroke(255);

  int y = spectrum_height;

  line(legend_width, y, legend_width+spectrum_width, y); // horizontal line

  // x,y address of text is immediately to the left of the middle of the letters 
  textAlign(CENTER, TOP);

  int spFreq=0; //for spacing

  for (float freq = 0.0; freq < ap.sampleRate(); freq += 2000.0) {
    int x = legend_width+spFreq+freqOffset; // which bin holds this frequency
    //println(freq+"->"+fft.freqToIndex(freq));
    line(x, y, x, y+4); // tick mark
    text(Math.round(freq/1000) +"kHz", x, y+5); // add text label
    spFreq+=45;
  }

  // DBlevel axis
  int x = legend_width;

  line(x, 0, x, spectrum_height); // vertictal line

  textAlign(RIGHT, CENTER);

  for (float level = -100.0; level < 100.0; level += 20.0) {
    y = spectrum_height - (int)(dB_scale * (level+gain));

    line(x, y, x-3, y);

    text((int)level+" dB", x-5, y);
  }
}

void keyReleased()
{
  // +/- used to adjust gain on the fly
  if (key == '+' || key == '=') {
    gain = gain + 5.0;
  } else if (key == '-' || key == '_') {
    gain = gain - 5.0;
  }
  //(.)/(/) used to adjust frequency axis
  else if (key == '/')
  {
    freqOffset = freqOffset-4;
  } else if ( key == '.')
  {
    freqOffset = freqOffset+4;
  }
}

void stop()
{
  // always close Minim audio classes when you finish with them
  in.close();
  minim.stop();

  super.stop();
}