Framerate in live video delay project : Ed Tannenbaum, Recollections

Thanks @jeremydouglass and @jb4x!
jb4x I still didn’t implement your idea, but I definitly will give it a try!
jeremydouglass your idea is working well, but I don’t know why this part make my sketch loose more than 30fps :

void colorGlobalPg() {
    pgBW.loadPixels(); 
    //println("new"); 
    for (int i = 0; i < pgBW.width; i++) {
      for (int j = 0; j < pgBW.height; j++) {
        color c = pgBW.pixels[j*pgBW.width+i];
        float b = brightness(c); 
        if (b!=0) {
          //pgColorTotal.pixels[j*pgBW.width+i]=color(255); // This run at 60 fps
          // pgColorTotal.pixels[j*pgBW.width+i]=lerpColors(constrain(b/255, 0, 1), colors); //when I use your custom lerpColors function from an other post fps go down at 20 fps
          pgColorTotal.pixels[j*pgBW.width+i]= colors[int(b%colors.length)]; //Same (fps at around 20 fps) when I grab color directly in a global color array
        }
      }
    }
  }

A.

Perhaps I should open a new topic, but I have a strange fps drop in some case :

  void colorAllPg() {
    pgBW.loadPixels(); 
    for (int y = 0; y < pgBW.height; y++) {
      for (int x = 0; x < pgBW.width; x++) {
        color c = pgBW.pixels[x+y*pgBW.width];
        float b = brightness(c); 
        if (b!=0) {
          int colors2[] = { #D8075B, #0AEDFF, #FAEE0A, #BCBDAC, #CFBE27, #F27435, #F02475, #3B2D38};
          //color cc=colors2[int(b%colors.length)];  //drop until 20fps
          // color cc=colors2[int(random(colors2.length))]; //drop until 30fps
          color cc=colors2[0]; //stay around 60fps
          pgColorTotal.pixels[x+y*pgBW.width]=cc;
        }
      }
    }
  }

Do you have any idea why there are such big difference just by accessing a local array? It’s looks like it’s even worst if it’s global. Any suggestion to fix it?

Thanks very much,

A.

Just my optimization attempt. Not tested! No idea if it’s any faster than yours… :woozy_face:

static final color[] PALETTE = { 
  #D8075B, #0AEDFF, #FAEE0A, #BCBDAC, 
  #CFBE27, #F27435, #F02475, #3B2D38
};

void colorAll(final PImage src, final PImage dst, final color[] palette) {
  src.loadPixels();
  
  final color[] p1 = src.pixels, p2 = dst.pixels;
  final int colors = palette.length;
  int idx = 0;
  
  for (final color c : p1) {
    final color b = (color) brightness(c);
    if (b > 0)  p2[idx] = palette[b % colors];
    ++idx;
  }
  
  dst.updatePixels();
}

Can it be the modulus operator or the random() function in addition with the int() function that causes the FPS drop and not accessing the array? :thinking:

You are calling them quite a lot in one frame.

Thanks @GoToLoop! it’s definitly an improvement (around 40fps), I have to investigate a bit more this static, final and all the structure of your algorythm :wink:

@jb4x I try to make a local variable with something like this :

 int colCounter=int(b%colors.length); 
 pgColorTotal.pixels[x+y*pgBW.width]= colors[colCounter];

But, no suprisly, it doesn’t change a lot…do you have something else in mind?
I guess I will keep @GoToLoop version, I’m around 50fps, that’s actually quite great :wink:

A.

You are still computing the int() operation and the modulus one for every pixel of the image.

Again, I really don’t know if this can be it, I’m not qualified enough to be sure about the performance of those functions. Someone more capable can surely tell us if it can be that.

Edit:

You can try something like this.
You create an array with 256 values let’s call it brightToCol.
Since you have 8 different colors you would fill it in like this:

brightToCol[0] = 0;
brightToCol[1] = 1;
...
brightToCol[6] = 6;
brightToCol[7] = 0;
brightToCol[8] = 1;
...

Then you can get your color with this line:

color cc=colors2[brightToCol[b]];

Is this better? Specially combined with @GoToLoop optimization.

Hi, thanks jb4x, I’ll perhaps need to try this, but actually my project went in a different direction, I’ll open a new topic :wink: But thanks for all your help with this!
A.

Hi All,

I just stumbled onto this thread. I’m the “Ed Tannenbaum” that designed the installations that you are trying to emulate with Processing. I’m not sure you are still attempting this, but I’ll give you a few hints…

I designed the first one in 1981. I built a 4 bit framebuffer with some digitizing logic that thresholded a video image and put it into the framebuffer only where it saw the silhouette. It was assigned a number based on time from 1-15. Zero was the background. The output of the framebuffer went to a hardware color lookup table where the numbers were assigned a color. The lookup table changed each frame during the vertical interval. It was controlled by an Apple II computer that was programmed in Forth.

Note that I used a single framebuffer. Memory was really expensive back then. With 4 bits I could display 16 colors at a time. It was the “colormap rotation” that created the animation.

Since then I’ve used the same method to create new versions in software. I’m not quite ready to release the code yet. I wouldn’t attempt it in Processing, or even openFrameworks as (from what I’ve been able to gleen from the docs) the rudimentrary interface to the necessary part of the graphics display isn’t there. It is built into the graphics cards, but isn’t available through OpenGL it seems. Perhaps I’m wrong. Please let me know.

I’d love to be able to write a shader that would create the effect.

ET

3 Likes

Hi Ed!

It’s a surprise, a pleasure and a honor to be able to speak to you! :wink:
First of all, my congratulations for all the different Recollections, they are truly inspiring (and inspired), innovative projects, thank you very much for taking the time to share these various advices.
I have actually achieved quite good results with the help anc the techniques proposed by @jb4x, @GoToLoop and @jeremydouglass (thanks again to them! ) but I finally didn’t continue in that direction : I still haven’t taken the time to upload the video of the final project, but here are some screenshots :

It still uses a Kinect, but this time users are immersed in a fluctuating world, with which they can interact and merge…Quite mystic :wink:

It’s an audiovisual work, and for the visuals I actually also wrote shaders, inspired by some I found on shadertoy.

I hope that one day I could experiment with your works in live
Thanks again for your work and advices,

All the best,

Adrien

4 Likes

Thank you so much for sharing this description, Ed – and for the inspirational art.

I’m not sure–setting shaders aside for the moment, by “rudimentary interface”, do you mean the pixels array? It is an int array of colors that you can write directly to with bit shifting, for example.