Sound Input Considerable latency

Processing Libraries
1

Hi, Im using the Sound library in processing (I think it comes form the Jsyn source?) anyways,I’m using it to make an audio visualizer to be used in a live music set. Currently everything is working properly with peak amplitude analysis, except there is a large noticeable latency between my Audio Input and it reaching Processing. It really disrupts the experience, as it is terribly asynchronous with the music (which is the idea)
One of the messages I get is " INFO: JSyn: default output latency set to 80 msec for Windows 10"
This would seem to be the obvious culprit.
What tweaks can I make to minimize this latency??
Your help is greatly appreciated.

2

That info log you are seeing is actually referring to output latency and not input latency. The input latency will come from your sound-card / audio interface. If you could post some code that would make it much easier to solve the problem. Could be that you need to apply a smoothing factor to your analysis, it could need a separate Thread. Hard to say without any code to analyze.

3

Thanks for pointing that out PHuzzeyMusic. I am currently routing my audio internally straight to my input so there should be virtually no latency there, and I have tested with other (even web) tools, and the sound visualizations do not have any latency when reacting to my audio input, leading me to believe it is something wrong with the routing into processing.

Here is a very condensed version of the program (the original is very heavy)

import processing.sound.*;
FFT fft;
AudioIn in;

color strokeCol = color(30, 255, 255);
int cols, rows;
int w = 500, h = 500;
int space = 15;
float[][] z;
float sound;
float speed = 0.01;
float density = 0.1;
float amp = 100;
float move = 0;

int bands = 16;
float[] spectrum = new float[bands];
float sum = 0;
float smooth = .2;

void setup() {
  fullScreen(P3D);
  colorMode(HSB, 255, 255, 255);
  noFill();

  cols = w / space;
  rows = h / space;

  z = new float[cols][rows];

  in = new AudioIn(this, 0);
  in.start();

  fft = new FFT(this, bands);
  fft.input(in);
}

void draw(){
 background(0);
 soundCalc();
 
 rotateX(1);
 
 depthCalc();
  
  translate(width / 2, height / 2);
  translate(-w / 2, -h / 2, -200); 
  
  showPlane();
  
}

void depthCalc() { //goes through array of points and assigns each a depth according to Perlin Noise function
  float noiseY = move;
  for (int y = 0; y < rows; y++) {
    float noiseX = 0;
    for (int x = 0; x < cols; x++) {
      z[x][y] = map(noise(noiseX, noiseY), 0, 1, -amp, amp);
      noiseX += density;
    }
    noiseY += density;
  }

  move -= speed;
}

void showPlane() { //places vertices and creates POINTS shaoe with them
  for (int y = 0; y < rows - 1; y++) {
    beginShape(POINTS);
    for (int x = 0; x < cols; x++) {
      vertex(x * space, y * space, z[x][y]);
      vertex(x * space, (y + 1) * space, z[x][y + 1]);
    }
    endShape();
  }
}

void soundCalc(){
  fft.analyze(spectrum);
    sound += (spectrum[0] - sound) * smooth; //use spectrum[0] for bass frequancies

  strokeCol = color(map(sound, 0, 1, 80, -180), 255, 255);

  stroke(strokeCol);
  strokeWeight(map(sound, 0, 1, 2, 8));
  amp = map(sound, 0, 1, 0, 1000); 
  speed = map(sound, 0, 1, 0.1, .8);
}

I used FFT in this program but I have tried Amplitude with the same latency results.
Any help for minimizing latency is appreciated TIA

4

The first sentence is correct, the second is not! The input latency is also controlled by the software that opens the card. By default it looks like JSyn has a 100ms input latency?! I’m not sure if the Sound library gives you access to change the underlying input parameters. If not, you’d be better looking into another sound library, perhaps Beads? Or perhaps use JSyn directly?

5

100ms!? Silly implementation :man_facepalming: but I’ll accept it, I’ll see if the other libraries have more acceptable latency.
Thank you Neil for the info, I’ll look into everything youve mentioned

6

I’ve tried and tried to make a change to the input latency and it seems the 100ms, even running JSyn directly is the best you can hope for. I tried minim , got some return values I wasn’t expecting and also it wasn’t too keen on me making my own threads to run evaluations. Since this is an ongoing problem in my main project, I’ve tried switching to cinder. So far, it has been really frustrating, but if it cuts down on latency for audio, as they promise it will, I will keep you posted. Hopefully one day Java will make a better solution for dealing with audio, because I really like Java/Processing a lot more than C++.

7

There are multiple good options for dealing with audio with Java, just not necessarily packaged for Processing. However Beads is a pretty good option latency wise, and can make use of JACK which potentially (with another piece of software) gives you ASIO routing. Beads uses an older version of the JAudioLibs audio backend I originally wrote for PraxisLIVE.

Also I’m confused why you need your own threads for evaluation? What does this mean exactly? Threads and low latency don’t necessarily go well together.

1 Like
8

Cool, I will look into those options and try and get something figured out. I am using Timer and TimerTask to update the fft so that it happens on its own Timer and isn’t called from Processing’s draw method.

9

That might be an issue - hard to visualise what you have. Two of the audio examples in PraxisLIVE do FFT visualization in a Processing node, although with generated audio not live input, but the latency will be about the same. Both are doing the FFT calculations in the audio thread itself.

10

Sorry I meant to reply sooner. Downloaded PraxisLIVE, and played around with some of the example projects, mainly the funky origami one. It is very, very cool, really enjoy the patch bay style interface for connecting components together. I am going to start trying to re-create my project with it in the near future. Here is a link to a video I made with the Windows 10 gamebar, it looks pretty dumb at this point but, just to give you some context, this is what I got so far.
https://drive.google.com/open?id=1zqNNpk6_1tVDcmEH0thn2ABF89qFlI95

11

Love it! :heart: Glad you’re liking the PraxisLIVE UI too. Let me know if you fancy putting something in as an example project sometime - actively looking to get some third-party contributions in there at the moment.