FrameRate hiccups

I have experienced hiccups in my animations at the start of my animations and trying to understand why this happens. Is it Processing, Java or other?

I have reduced it down to a simple example that displays frameRates below a threshold of 54 fps:

float threshold = 54;  // fps
int y = 100;           // spacing for text

void setup() 
  {
  size(700, 750, P3D);
  background(0);
  textSize(24);
  }

void draw() 
  {
//  background(0);
  float temp = frameRate;
  if (temp < threshold)
    {
    print(frameCount + "\t");
    println(temp);
    text(frameCount, 10, y);
    text(temp, 160, y);
    y += 30;
    }

Output to console (sames as display window) is:
121 53.569176
122 51.614117
123 52.08391
124 52.479534
125 52.76374
126 53.006912
127 53.192307
128 53.490566
129 53.920532
251 53.79264
252 53.38054
253 53.67791
254 53.97975

I made 121 and 251 bold because these are frameCounts where the framerates are below 54.

UPDATE I was rushing out the door and did not “frame” that last statement well…

I made 121 and 251 bold because these are frameCounts where the framerates start dropping below 54 and repeat at these values consistently with minor variation.
Sometimes it starts at 121 or 251 or both.

Any thoughts on this?

1 Like

You will get variations because the OS is multitasking so the sketch has to share the CPU with other programs. Your results are not unusual.

1 Like

Agreed, pretty normal. You can reduce these effects by closing other applications, but this may still leave services that run in the background – email checking, cloud syncing of files like gdrive dropbox etc, antivirus scanning, active browser content or push notifications, software update checking or pulling, os indexing for file search, et cetera.

1 Like

I updated my post…

I do understand that other applications may affect this; this is a very specific example at start of code execution.

Frame rates slow down consistently and starting at the two points mentioned.
I have had “hiccups” at the start of code execution since working with Processing and have workarounds for the first 5 seconds or so until frame rates settle down. I am now exploring why this happens.

After 5 seconds frame rates are more stable and I do notice occasional hiccups; these are not a concern and some are related to Java garbage collection and other activity on PC.

I have tried using VisualVM to monitor performance. This is a great tool to monitor performance and helped me debug some code once where the “heap” get growing out of control.

[https://visualvm.github.io/?VisualVM_1.4.2]

I did observe that frame rates drop when a “garbage clean” is performed by Java and if I do a manual “Perform GC” with the VisualVM software while monitoring. Since I have to “run” my Processing app, wait for it to show up in VisualVM and then “monitor” I miss the activity at start of code; there may be a better way and I will work on this.

I will continue to explore this…

GLV

Why are you concerned about this? How does it affect your program?

1 Like

In order to diminish, and even eliminate, GC pauses, you need to refactor your code to reuse objects rather than discard them. :wastebasket:

1 Like

Can you say more about what your specific goals are? Could you drop your fps to frameRate(50) so that your sketch always hits the minimum? Do you need periodic ticks that are exactly on a certain time? Can you drop frames to reinforce alignment?

I once wrote a syncing class for a library that was meant to make simultaneous sketch output robustly similar when running on different hardware with unreliable frame rates. You might find it helpful. Keep in mind though, the main thing you can do within sketch code to make frame times more reliable or aligned is make them slower. Or you can buy faster hardware, or change your software environment, et cetera.

Syncer.java

package com.jeremydouglass.toolboxing.time;

/**
 * A Syncer gives ways to measure lag and remediate it.
 *
 * A Sync lets sketches share an ideal set of target reference
 * times or frames, and also lets sketches attempt to conform
 * to these times / frames. It computes a target time for a
 * given frame number, or a target frame number for a given time.
 *
 * For sketches that are able to maintain the given frame rate,
 * Sync provides a stable set of references that can be provided
 * as random seeds to produce synchronized pseudo-random behavior.
 * 
 * When sketches drift or fall behind the target frameRate, Sync
 * provides two mechanisms of enforcing alignment: alignFrame,
 * which skips drawing frames to catch up, and alignMillis(),
 * which also uses millisecond delays to further align a late
 * frame to the next target.
 
 * down a given frame (aligning it to the next target
 * time) or skip one or more frames (catching up to a
 * target frame). 

 * Processing's frameRate() assigns a *minimum* frame duration.
 * This mean that frames run late, not early. This running late
 * is called lag, and it may involve a slow drift over time,
 * or a periodic long lag due to infrequent demanding operations,
 * or consistent inability to hit the target speed every frame.
 * 
 * Lag can be expressed in two ways -- frame lag and time lag.
 *
 * Frame lag:
 * In frame lag, the current frame is less than expected.
 * For example, drawing frame 8 while scheduled to draw frame 10.
 *
 * Time lag:
 * In time lag, the current time is greater than expected.
 * For example, drawing frame 10 at 210 when scheduled for 167.
 *
 * Lag may be remediated via alignment.
 * Skip the sketch forward to the next scheduled frame.
 * This has the effect of skipping frames.
 * 
 * Sync may also reset the schedule to the current late frame.
 */
 
public interface Syncer {
  public int startMillis = 0;
  public long startTime = 0;
  public int targetFrameRate = 60;
  public int alignFrame();
  //int alignMillis();
  public int targetFrame();
  public int targetMillis();
  public long targetTime();
}

Sync.java

package com.jeremydouglass.toolboxing.time;

import processing.core.PApplet;

/**
 * 
 */
public class Sync implements Syncer {
  private PApplet pa;
  public long startTime;
  public long syncStartTime;
  public int targetFrameRate;
  public int step;

  /**
   * Constructor
   *
   * Default is to use the sketch current `frameRate`.
   * Note that after setup this value is dynamic, and 
   * may not be equal to the setting specified with
   * `frameRate()`.
   *
   * @param  pa               Processing sketch PApplet ('this')
   * @param  targetFrameRate  The target frame rate for sync to describe / enforce
   */
  public Sync(PApplet pa, int targetFrameRate) {
    this.pa = pa;
    this.targetFrameRate = targetFrameRate;
    this.step = (int)(1000/targetFrameRate);
    this.startTime = System.currentTimeMillis();
    this.syncStartTime = startTime + step - startTime%step;
    PApplet.println("tfr", targetFrameRate, "step", step, "sync-start", syncStartTime);
    PApplet.println();
  }
  /**
   * If the current time is later than a threshold value,
   * skip to the next aligned frame. When alignMillis is
   * true, then delay until milliseconds are also aligned.
   *
   * Default threshold is half the step value. For example,
   * a frameRate of 10fps has a step value of 100 per frame.
   * Half the value is 50, so calling align when a frame is
   * more than 50 milliseconds late will trigger a frame align
   * (skipping one or more frame numbers) and millisecond align
   * in the form of a short delay.
   * 
   * Millisecond alignments are only approximate.
   *
   * @param  pa             Processing sketch PApplet ('this')
   * @param  minLateMillis  Threshold lateness in milliseconds for triggering alignment
   * @param  alignMillis    Set true to delay frame until milliseconds are better aligned. Default false.
   */
   
  //public int alignMillis() {
  //  return this.alignMillis(this.step/2);
  //}
  
  //public int alignMillis(int minLateMillis) {
  //  int diff = this.diffMillis();
  //  if (diff > minLateMillis) {
  //    int framesSkipped = this.alignFrame();
  //    // this.skip();
  //    diff = this.diffMillis();
  //    if (diff > minLateMillis) {
  //      delay(diff);
  //      return diff + framesSkipped * step;
  //    }
  //    return framesSkipped * step;
  //  }
  //  return 0;
  //}
  
  //public int alignMillis(int minLateMillis) {
  //  int diff = this.diffMillis();
  //  if (diff > minLateMillis) {
  //    int framesSkipped = this.alignFrame();
  //    diff = this.diffMillis();
  //    if (diff > minLateMillis) {
  //      delay(diff);
  //      return diff + framesSkipped * step;
  //    }
  //    return framesSkipped * step;
  //  }
  //  return 0;
  //}
  
  /**
   * If the current time is later than a threshold value,
   * skip to the next aligned frame. When alignMillis is
   * true, then delay until milliseconds are also aligned.
   */
  public int alignFrame() {
    int target = this.targetFrame();
    if (pa.frameCount < target) {
      return this.skip(target - pa.frameCount);
    }
    return 0;
  }
  
  /**
   * Increment sketch frameCount by an amount. Defaults to 1.
   * @return  Count of frames skipped.
   */
  public int skip() {  // internal
    return this.skip(1);
  }
  
  /**
   * Increment sketch frameCount by an amount. Defaults to 1.
   * 
   * @param count  Count of frames to skip.
   * @return       Count of frames skipped.
   */
  private int skip(int count) {
    pa.frameCount += count;
    return count;
  }
  
  /**
   * Returns the difference between the current sketch frameCount and the target frame.
   * @return  Difference from target frame.
   */
  public int diffFrame() {  // internal
    return this.targetFrame() - pa.frameCount;
  }
  
  /**
   * Returns the difference between the current sketch millis and the target millis.
   * @return  Difference from target millis.
   */
  public int diffMillis() {  // internal
    int diff = pa.millis() - targetMillis();
    return diff;
  }
  
  /**
   * Gives the target frame count for a given time in milliseconds.
   * Defaults to the current sketch millis().
   * 
   * @return  The target frame count.
   */
  public int targetFrame() {
    return pa.millis()/this.step;
    //return this.targetMillis()/this.targetFrameRate;
  }
  
  /**
   * Gives the target millisecond time for a given frame.
   * Defaults to the current sketch frameCount().
   * 
   * @return  The target millisecond time.
   */
  public int targetMillis() {
    return pa.frameCount * this.step;
    //int millis = pa.millis();
    //return millis - millis%step;
  }
  
  /**
   * The target time, offset by the start time, given a current frame.
   * 
   * @return  The target millisecond time.
   */
  public long targetTime() {
    return syncStartTime + this.targetMillis();
  }
}

Run sketch twice side-by-side to test:

import com.jeremydouglass.toolboxing.time.Sync;

Sync sync;

void setup() {
  int fps = 10;
  frameRate(fps);
  sync = new Sync(this, fps);
}

void draw() {
  // log frame skipping
  int skip = sync.alignFrame();
  if(skip>0){
    print("  ", skip);
  }
  
  // fade old lines
  fill(255,8);
  rect(-2,-2,width+4,height+4);

  // draw random line
  randomSeed(sync.targetTime());
  line(random(0, width), random(0, height), 
    random(0, width), random(0, height));

  // simulate load (parallel across sketches)
  delay((int)random(0, 100));

  // simulate other lag (not parallel)
  delay((int)random(0, 100));
}

I threw together some code to visualize the hiccups; they are within the first 15 seconds (repeatable) and after that it is a rock solid 59 to 60 fps (green).

The bottom row is first 15 seconds (up to vertical line) and then it keeps plotting framerate and moves up a row at a time:


The glitch on top row is when I used “Snipping Tool”.
I went through many of my apps to see what cause hiccups (frame rate drops) and was very surprised! Some had no impact and others significant.
I’ll clean up code and post later.

glv

1 Like

Very interesting. Looking forward to what you learn.

There has sometimes been a significant lag on the first frame that calls text() (lazy loading) – although your repeating pattern looks more like object / garbage – or something like large ArrayList resizing as something scales up? – rather than such on-time lags.

As well as text, there are a few other things that initialize and cache things at sketch start-up that will cause hiccups, and also first use of a function can trigger a lot of classloading. However, one of the primary causes of performance issues at start-up will be JIT compilation as various hot spots in the code switch from interpreted Java bytecode to compiled code.

The Perform GC button in VisualVM works quite differently to how normal GC in your application works (stop the world vs incrementally). Be careful also that VisualVM changes how your application works because the JVM has to stop performing some optimizations. As such it can make the monitored application slower and perform more GC.

This is almost always the wrong advice that will be counterproductive and make your application slower. You need to be very clear why you’re pooling particular objects, and actually measure the results rather than trusting intuition.

3 Likes

Reusing objects is not a bad scenario. If for example we can make use of an object which exists and can ‘fool’ the user that it feels like a new instance of the object exists I would see that as a win to maximize resources efficiently. Looking deeper into why this would be advantageous one could look at Big O notation and space omplexity. By nature more objects means more resources are being used and toggling down the hardware on the pc. But keeping it as cute as possible allows you to use those resources posiblly for a different scenario. That is why reusing objects / calling things in the most effective manner can pose a more effective outcome.

There also is time complexity. So taking for example a 2D loop which fills a 2D array for the size of the array where the size is N the loops will always run O(n^2) times. Fixing in particular parts to minimize this complexity. Can also save resources!

Intuitively that sounds correct, but in reality you’re just fighting against how the JVM works, and will in most cases make your code slower. If an object utilizes a lot of memory or instantiation is particularly complex, then caching may make sense. However, the clearing to make an object appear like a new one is often more resource intensive than allocating a new one.

Short-lived, lightweight objects are actually quite efficient, almost free, for the garbage collector - all the GCs mostly don’t collect garbage, they instead collect all the live objects and move them elsewhere before clearing that region of memory.

Finally, if you cache an object when you don’t need to you’re likely reducing the optimizations the JIT compiler can make. eg. escape analysis. If you allocate a new PVector every frame for calculations for example, and it doesn’t escape the draw() method, there’s a good chance that the fields that make up the PVector will be inlined on the stack, as will your calculations, and no PVector will actually be created. Even if caching removes the allocation, it makes the reading/writing code less efficient. Never second guess the JVM unless you can prove there is a problem.

As it was mentioned in this thread, never use VisualVM either to check for escape analysis / object allocation - it will disable this feature. Try JMC instead.

1 Like

I wrote some code to store the frameRate to a file and plotted with Excel.
This eliminated any elements in draw() that draw to screen and only has file I/O to an SDD.

Code:

PrintWriter output;

void setup() 
  {
  size(1024, 768, P3D);
  frameRate(60);
  output = createWriter("frameRate.csv"); 
  }

void draw() 
  { 
  output.println(frameCount + "," + frameRate);  
  }
   
void keyPressed()
  {
  output.flush();       // Writes the remaining data to the file
  output.close();       // Finishes the file
  exit();               // Stops the program
  }

Graph of “Framerate Hiccups” (frameCount vs frameRate):
image

This is the same consistent behavior that I observe on my PC at the start of code.

If you want to get a clearer picture, you might want to consider measuring an inter-frame millis(), not frameRate. I seem to recall (?) that frameRate is a smoothed rolling average.

1 Like

I had some fun visualizing the data available to me.

I displayed frameRate data (which averages) to framerate calculated from time between frames.
https://processing.org/reference/frameRate.html

Code (you will have to select use “true” or “false” to see different modes of output):

PrintWriter output;

Table table;
int time = 16*60+40;       //16s*60f/s = 960f hiccups end after this time
boolean dispMode = true;  // "true" stores to file and plots after "time", "false" plots in real time

long lastTime;
long currTime;
long diffTime;

void setup() 
  {
  size(1000, 600, P3D);
  background(0);
  frameRate(60);
  
  if (!dispMode)
    {
    output = createWriter("frameRate.csv");
    output.println("frameCount" + "," + "frameRate");
    textSize(30);
    textAlign(CENTER, CENTER);
  //  text("Please wait for "+ time + " frameCounts...", width/2, height/2);
    text("Please wait for "+ time/60 + " seconds...", width/2, height/4);
    lastTime = System.nanoTime(); 
    }
  }

void draw() 
  {
  if (dispMode)
    {
    currTime = System.nanoTime();
    diffTime = currTime - lastTime;
  //  println(frameCount, frameRate, 1000000000.0/diffTime);
    lastTime = currTime;
   
    if (frameCount%width == 0)
      background(0);   
    float scale = 5;
    float size = map(abs(1000000000.0/diffTime-60), 0, 100, 2, 30);
    strokeWeight(size);
    stroke(0, 255, 0);
    point(frameCount%width, float(height) - scale*1000000000.0/diffTime);
    strokeWeight(3);
    stroke(255, 255, 0);
    point(frameCount%width, float(height) - scale*(frameRate));
    }
  
  if (!dispMode)
    {
    output.println(frameCount + "," + frameRate);
    if (frameCount == time) closeFile(); 
    if (frameCount > time)
      {  
      table = loadTable("frameRate.csv", "header");  
      for (TableRow row : table.rows()) 
        {
        int x = row.getInt("frameCount");
        float y = row.getFloat("frameRate");
  
  // Add some colour
        if (y < 55)             // Red for framerate < 55
          stroke(255, 0, 0);
        else if (y < 59)        // Yellow for 55<= framerate <59 
          stroke(255, 255, 0);
        else
          stroke(0, 255, 0);    // Green for   59<= framerate  It can be > 60 fps!
  // Plot         
        strokeWeight(2);
        point(x%width, float(height)-5*y);
        }      
      }
    }
  }    

void closeFile()
  {
  output.flush();       // Writes the remaining data to the file
  output.close();       // Finishes the file
//  exit();
  }

Plot of Average Processing “frameRate” (yellow) and Calculated Framerate (green) with enhancements:

Same program where it only collects data and plots after 16s:

Data visualization can be fun!

I was just in the process of posting this… and did one better and used System.nanoTime();

:slight_smile:

I wrote very similar “frame rate independent” code a while back when I discovered the code I run at home (fast PC) did not run quite right on my display case PC (slow PC) at work.

Thanks for sharing!

Plot looks good. Seems like the frameRate line is just not informative about the actual frame duration data and can be dropped – those frameRate slopes are just artifacts of those 3 individual slow frames.

On reruns are you always getting exactly 3, and at almost exactly the same times or frameCounts? Or are you just randomly getting a few (2-7) slow frames over that duration?

We had a long discussion on the old forum about ways to really nail a timing loop down to low millisecond fluctuations. @quark showed how to create your own timer – I created a simple method designed for lower framerates that skips frames and burns slack to get to within 1ms each time – it later morphed into that sync class. The old discussion is here:

2 Likes

Thank you to everyone for sharing and all your insights into this.

I have been working around the startup hiccups (first 15 secs) on my PC for years and will live with these.

“Wait for framerates to settle!” is in some of my code from 2 years ago!

:slight_smile:

I’d be interested what you see with framerate set to 120, or 240 or higher, or maybe even not multiples of 60. JOGL should lock to vsync so should end up with a similar output framerate on a 60Hz display. I think there may be some clock drift going on between the timer and GPU though. I wonder what this would be like on a 75Hz display too?!