Avoid 'clunky' sketch/framerate

Processing Coding Questions
1

Hi everybody,

I am currently working on a project where I have an image wall 5 x 5 grid.

The project selects and display random images from data folder after each iteration. I have hacked together a working demo over the last couple of days.

At the moment I am using 733 images, however, it is hoped that the project may draw from a selection of over 100,000 jpegs.

As I said, it works at the moment but can be quite ‘clunky’. When I change the framerate from 2 to 16, there is no difference.

I am assuming this is down to my ‘bad’ coding and am wondering if there is a better way for me to approach this project/code to make it more robust?

I am using a macbook pro at the moment, but I would love to have this project ultimately on rasp pi…possible wishful thinking on my part…anyway here is the code if anyone has any suggestions:

void setup() {
  size(500, 500);
  background(0);
  frameRate(16);
};
void draw() {
  int i = 0;  
  int j = 0;
  int k = 0;
  int l = 0;
  int m = 0;
  
  while (i < 5) {
    float a = random(0,733);
    int ranNum = int(a);
    PImage img;
    img = loadImage("image" + ranNum + ".jpg");
    img.resize(100,100);
    image(img, (i * 100), 0); 
    i = i + 1;
  }
  
  while (j < 5) {
    float a = random(0,733);
    int ranNum = int(a);
    PImage img;
    img = loadImage("image" + ranNum + ".jpg");
    img.resize(100,100);
    image(img, (j * 100), 100); 
    j = j + 1;
  }
  
   
  while (k < 5) {
    float a = random(0,733);
    int ranNum = int(a);
    PImage img;
    img = loadImage("image" + ranNum + ".jpg");
    img.resize(100,100);
    image(img, (k * 100), 200); 
    k = k + 1;
  }
  
  
  while (l < 5) {
    float a = random(0,733);
    int ranNum = int(a);
    PImage img;
    img = loadImage("image" + ranNum + ".jpg");
    img.resize(100,100);
    image(img, (l * 100), 300); 
    l = l + 1;
  
  }
  while (m < 5) {
    float a = random(0,733);
    int ranNum = int(a);
    PImage img;
    img = loadImage("image" + ranNum + ".jpg");
    img.resize(100,100);
    image(img, (m * 100), 400); 
    m = m + 1;
    println(ranNum + " " );
  }

  }
1 Like
2

It’s because you’re calling loadImage() inside your draw function so it gets called multiple times which is affecting your performance. Try loading the images into a map or list first in setup()

2 Likes
4

Thanks for the reply mcd4u.

Can i ask you to elaborate on ‘map’ or ‘list’…or link to an example. Also by doing this in setup() would I not be stuck with the same random selection of images for all iterations?

Thanks again,

Opt Out

1 Like
5

You should take a look at the documentation on maps. In this case, I would suggest having the number in your filename as the key, and the actual PImage object as the value. So in draw(), you can simply get the image from the map (using .get(ranNum)) instead of calling loadImage() multiple times.

You won’t be stuck with the same random selection as long as you keep the random number generator code in your draw() function. You’re really only changing the loadImage() part.

As a side note, since you mentioned that your program will handle up to 100,000 images, loading all of those PImages will cost a lot of memory. One way to solve this is to combine all of those images into 1 big image atlas beforehand so that you end up only loading 1 image. Then you have to figure out a way to get the correct portion of that atlas using the random number.

MCD4U

1 Like
6

Much appreciated mcd4u!

1 Like
7

Wouldn’t it be more performance friendly to have a vector of 25 images (the ones to display) and load new ones when needed using a separate thread; and simply replace the image that needs to be changed in the vector of images with the new one ?

This way no need for a big image of over a 100,00 jpegs or an array of 100,000 images.

3 Likes
8

Hey jb4x,

Thanks for the reply and suggestion.

This sounds like a good approach if I understand you correctly. Do you mean create a vector of 25 images within processing, within setup(), load it within draw and the repeat this process?

And if so, could you please point me in the right direction as to how to create such a vector, please and thanks!

9

I haven’t tested it so I’m not sure it is completely functional but you at least have the basic idea:

PImage[] images = new PImage[25];
long timerStart = 0;
PImage tempImage;
boolean changePic = false;
boolean readyForNewPic;

void setup() {
  size(1000, 1000);
  
  for (int i = 0; i < 25; i++) {
    images[i] = loadImage("image_" + (int)random(10000) + ".jpeg");
  }
}

void draw() {
  if (changePic) {
    int idx = (int)random(25);
    images[idx] = tempImage.copy();
    
    changePic = false;
    readyForNewPic = true;
  }
  
  if (millis() - timerStart > 10000) { // Change pic every 10 seconds
    if (readyForNewPic) {
      readyForNewPic = false;
      changePic = false;
      thread("getNewPic");
      timerStart = millis();
    }
  }
  
  for (int i = 0; i < 5; i++) {
    for (int j = 0; j < 5; j++) {
      image(images[i * 5 + j], i * 200, j * 200, 200, 200);
    }
  }
}

void getNewPic() {
  tempImage = loadImage("image_" + (int)random(10000) + ".jpeg");
  changePic = true;
}
2 Likes
10

Hey jb4x,

Thanks you for your efforts and patience.

Yes, that loads 5 x 5 grid. One thing it is not doing is looping/changing so it just stays the same. I tried to call getNewPic() in draw() but that didn’t work.

Also, is there a way of automatically resizing selected images so they keep their ratio?

Thanks again,

Opt Out

11

Should the images be repeating or non-repeating? Do you want to shuffle them like a deck and then deal 25 at a time, or do you want to pull 25 from a bag and then put them back in the bag and draw again?

Top things you can do for performance:

  1. don’t resize on the fly – that is extremely slow. Make a copy of your image set, then pre-resize your image cache once before running your sketch. You could write a separate sketch to do this if you wish, or use OS X Automator.
  2. rather than loading your new images in the frame when you want to change them, load them in a separate thread in the frame after you updated using requestImage https://processing.org/reference/requestImage_.html
  3. much more minor, consider loading a list of all image file names, then shuffle that list. https://processing.org/reference/StringList.html. Go through them in batches with an offset index, reshuffle when you run out.

So not this:

  1. loadImage for images for this frame
  2. display images
  3. wait 10 seconds of draw loop, repeat at frameCount%600==0

But this:

  1. in setup, preload first set of images
  2. display images
  3. requestImage for images for 10 seconds from now
  4. wait 10 seconds of draw loop, repeat at frameCount%600==0

https://processing.org/reference/requestImage_.html

12

Hi Jeremy,

Thanks for your reply.

The images can repeat. If there is over 100,000 of them and only 25 each iteration, I guess there will still be plenty of variety!

  1. Yeah, I figured this. but it’s a shame. The batch of images are all different sizes, also as the project progress I might like to do variations (10x10 grids etc), so having to resize externally is not ideal.

  2. I think this is what @jb4x suggested, except I haven’t used requestimage, perhaps this is a better approach?

  3. Oh, this I might try and do!

Thanks,

Opt Out

13

As long as you are only resizing at load time (and not every frame!) then doing it all in the sketch on-demand – eg 10 images every 10 seconds – probably shouldn’t be a big deal. However, as soon as you want to resize 100 images on the fly in 10 seconds for a 10x10 layout…

Even when they are tuned right, setups with randomly collected images as input are unreliable under some conditions, however – for example if you have a mix of very high resolution images in your source folder then when randomly selected they could produce unpredictable lag or late loading tiles.

Will you be changing layouts during the run, or just between runs?

1 Like
14

I tested the code and there was a problem with the initial states of the boolean variables.

That version works:

PImage[] images = new PImage[9];
long timerStart = 0;
PImage tempImage;
boolean changePic = false;
boolean readyForNewPic = true;

void setup() {
  size(600, 600);
  
  for (int i = 0; i < 9; i++) {
    images[i] = loadImage("Image_" + (int)random(13) + ".jpg");
  }
}

void draw() {
  if (changePic) {
    int idx = (int)random(9);
    images[idx] = tempImage.copy();
    
    changePic = false;
    readyForNewPic = true;
  }
  
  if (millis() - timerStart > 5000) { // Change pic every 10 seconds
    if (readyForNewPic) {
      readyForNewPic = false;
      changePic = false;
      thread("getNewPic");
      timerStart = millis();
    }
  }
  
  for (int i = 0; i < 3; i++) {
    for (int j = 0; j < 3; j++) {
      image(images[i * 3 + j], i * 200, j * 200, 200, 200);
    }
  }
}

void getNewPic() {
  tempImage = loadImage("Image_" + (int)random(13) + ".jpg");
  changePic = true;
}

Also, don’t try to do that:

If you do you go back to what you were doing before (almost). Using the thread() function, the image is loaded on another thread and so it oes not affect the performance of the initial sketch.

15

Or better yet, use requestImage(), which is already threaded: :bulb:

3 Likes
16

Thanks for the advice jeremy.

As you’ve probably noticed by now, I’m still really thinking my way through this! I’m beginning to think it might be best for me to resize all images external to the sketch (either through a python script - or if anyone knows a way of batch resizing via processing?)

If I want to change the spec of the project (eg from 5 x 5 to 10 x 10 etc) I’ll just have to rezise them all accordingly.

I guess it makes sense not to try too much in one sketch, so the priority of the grid sketch is just that, projecting a grid of images all the same size.

17

Thanks again @jb4x.

OK, I may be going slightly crazy here. This sketch works, but it is only changing one image/tile every 10 seconds.

I want the sketch to change all images/tiles every 10 seconds. So every 10 seconds a new grid is produced containing a new selection of random images.

I have played around with your code and compared it to your first sketch…the only difference I can find is as you stated in your post

I have changed

void draw() {
 if (changePic) {
 int idx = (int)random(25);
 images[idx] = tempImage.copy();

to

void draw() {
 if (changePic) {
if (changePic) {
     for (int i = 0; i < 25; i++) {
    images[i] = loadImage("image" + (int)random(733) + ".jpg");
     }

which works, but obviously this brings me back to the problem of having loadImage() in draw() again.

Sorry about this and thanks for your help/patience.

18

Thanks @GoToLoop . I’m going to try and get (and understand!) @jb4x above sketch working…I’ll then start having a look at requestImage().

19

Hi,

For batch resizing why not simply use Irfan view for exemple? No need to reinvent the wheel every time =)

Now to change all the pics at the same. Time, the idea is the same but instead of having 1 temp image you need to have an array of temp images and you will still load them in the threaded method.

20

:man_facepalming::man_facepalming: how/why did I not think of this :roll_eyes::grin::grin:

I’m not sure what you mean (a reflection on my skills not your explanation), but I’ll start trying :smile:

21

Hey all,

At the risk of driving everyone nuts on this thread…

using @jb4x sketch above I am able to change one random tile in 7 x 5 grid every x amount of seconds

 int idx = (int)random(35);
    images[idx] = tempImage.copy();

Just asking if there is a way to change this so that I can change two tiles simultaneously?

I have (naively) tried

int idx = (int)random(35) +  (int)random(35);

but that’s just adding two random numbs between 0-35 together :man_facepalming: