Fluctuating Signals - Filtering

Processing Libraries
1

I’m receiving a realtime continuous set of float values from a measurement, I need to be able to detect large changes however there are several small fluctuations too that aren’t due to any underlying pattern other than noise and bad measurement. What would be the best way to stabilise the set of values I’m receiving in Processing?

1 Like
2

An infinite impulse response (IIR) filter would likely do the job.

Very condensed description ahead:
Record a weighted sum of the current sample, the previous sample, and your previous recording.
Adjust the weights to get the filter response you are after.

Weights for a basic low-pass (to remove noise) might be “a = 0.15” and “b = 0.85” and

// previous recording also contains information from the previous sample
record[now] = a*sample + b*record[previous];

The similar FIR filter (finite impulse response) might also work. That will only look at previous samples, not storing filtered values, aka “feedback”.

You can find more about calculating the weights here: http://www.dspguide.com/ch19.htm (easier to follow, in my opinion, than the wiki article)

1 Like
3

I’ve used the simplest of these filters many times. The formula I use is output = input / tor + output * (tor - 1) / tor. Tor is the filter time constant in units of frameRate periods. If tor = 6.66 it gives the numbers as in @noahbuddy’s example. (Thanks I’d forgotten they’re called IIRs.) Recently I had to build a filter in a place with no floating point so I wrote the same thing in Processing to test the effects of integer arithmetic. The input blue line is mouseY, the white line is the output. (Gain is only to reduce the effect of the integer truncation, with floats it can be 1 or not there.)

Filter

/**
* Filter
* 
* 
* Chart by Andreas Schlegel, 2014
* www.sojamo.de/libraries/controlp5
*
*/
import controlP5.*;

ControlP5 cp5;

Chart myChart;

void setup() 
{
  size(800, 400);
  frameRate(5);
  cp5 = new ControlP5(this);
  
  myChart = cp5.addChart("dataflow")
               .setPosition(50, height / 4)
               .setSize(400, height /2)
               .setRange(-height /4 , height / 4)
               .setView(Chart.LINE)
               .setStrokeWeight(1.5)
               .setColorCaptionLabel(color(40))
               ;

  myChart.addDataSet("in_");
  myChart.addDataSet("out");
  myChart.setData("in_", new float[100]);
  myChart.setData("out", new float[100]);

  myChart.setColors("out", color(230,230,230));
}

int tor  = 10;
int gain = 10;
int store = 0;

void draw() 
{
  int pv;
  int op;
  background(20);
  //myChart.push("incoming", (sin(frameCount*0.1)*10));
  pv = 0 - (mouseY - height / 2);
    
  store = pv * gain / tor + store * (tor - 1) / tor;
  op = store    /gain;
  
  myChart.push("in_", pv);
  myChart.push("out", op);
    
  fill(0, 255, 0);
  textSize(18);

  textAlign(LEFT);
  text("Tor"  , 200, 20);
  text("Gain" , 200, 40);
  text("In"   , 200, 60);
  text("Store", 200, 80);
  text("Out"  , 200,100);
  
  textAlign(RIGHT);
  text(  tor, 300, 20);
  text( gain, 300, 40);
  text(   pv, 300, 60);
  text(store, 300, 80);
  text(   op, 300,100);
}
1 Like
4

Hi, thanks so much. Seems like a simple algorithm but the thing is it doesn’t do enough. Perhaps taking a look at my project will help better. Am I doing it correctly? Perhaps my signal fluctuates at far larger amounts that this filter doesn’t do much? It’s definitely improved slightly, but there are still some large fluctuations.

FloatList record;
float amtao; //INITIAL RECEIVED SIGNAL
float amta; //VALUE OF INTEREST - THE FINAL SIGNAL
int count;
void setup(){
   size(1000, 700, JAVA2D);
  record = new FloatList();
  count = 0;
  amta = 0;
  amtao = 0;
//RECEIVE REALTIME DATA FROM PORT 8338
 oscP5 = new OscP5(this, 8338);
oscP5.plug(this, "rawReceived", "/fil");
}
void draw(){
//SIGNAL FILTERING
if (count ==0){
 record.set(0, amtao);
 count++;
} else if (count >0){
    record.append(amtao + 0.85*record.get(count-1)); //AMTAO = 0.15*INCOMING SIGNAL
    amta = record.get(count);
    count++;
}

}

public void rawReceived(float prob) {
  //ATTEMPTED THRESHOLDING AND CONTROL BASED ON OBSERVATION
  if(prob>0.55){
    prob = 0.55;
  }
  if (prob >0.15){
  amtao = 0.15*(prob+0.45); //0.15 is the basic low-pass "a" value
  }else{
    amtao = 0;
  }
}
1 Like
5

Something to try:

Change the rawReceived function to:

public void rawReceived(float prob) {
  amtao = prob; // here, I am assuming that prob _is_ the data...
}

and the recording line to

record.append(0.15*amtao + 0.85*record.get(count-1)); //AMTAO = 0.15*INCOMING SIGNAL

I expect that to reduce the noise on the signal.
You can always clip or scale the result after.

Another toy filter:

FloatList record;
float amtao; //INITIAL RECEIVED SIGNAL
float amta; //VALUE OF INTEREST - THE FINAL SIGNAL
int count;

float t=0, dt=0.03;
float M = 40; // draw scale
float NOISE_AMT = 0.2;

float lastSampleVal=0, lastFilterVal=0;

float a = 0.10;
float b = 1 - a;

void setup(){
  size(1000, 700, JAVA2D);
  record = new FloatList();
  count = 0;
  amta = 0;
  amtao = 0;
  fill(0);
}

void draw(){
  // Read values
  amtao = sampling();
  t += dt;
  
  // Filter
  if (count ==0){
   record.set(0, amtao);
   count++;
  } else if (count >0){
      record.append(a*amtao + b*record.get(count-1)); //AMTAO = 0.15*INCOMING SIGNAL
      amta = record.get(count);
      count++;
  }
  
  // Show
  translate(0, height/4);
  stroke(0,255,0);
  line(count-1,M*lastSampleVal, count,M*amtao);
  text("Raw", 20,0);
  
  translate(0, height/4);
  stroke(0,0,255);
  line(count-1,M*lastFilterVal, count,M*amta);
  text("Result", 20,0);
  
  lastSampleVal = amtao;
  lastFilterVal = amta;
}

float sampling()
{
  return cos(t) + random(-NOISE_AMT,NOISE_AMT);
}
2 Likes