Optimizing for memory with images

Processing Coding Questions
1

Hi all,

I’m working on a sketch that will require hundreds of thousands of large (3200 x 5000px) images to be drawn. However, I am just beginning to build the code and I am referencing only 15 images (not thousands), and I’m already running out of memory.

Does anyone have any suggestions for how to make this program more efficient. I can tell you that it doesn’t need to run quickly. It could run at 0.1 frames per second if needed.

UPDATE: I tried increasing the available memory to 10,000, just to see if that would help, and it did seem to fix the problem. (Previously I had thought that the maximum memory you could dedicate was 2,024. It appears that has changed?)

None-the-less, I’m wondering if I should be going about this in a different way? Is there a more efficient way to deal with a database of hundreds of thousands of high-resolution images?

Thanks!

Here’s the code:

PImage[] SpecimensFrame = new PImage[5]; 

int frame = 0; 
int numFrames = 5; 

void setup() {
  size(1280, 720);
  background(175);
  for (int i = 0; i < 5; i++) {  
    String fileName = i + ".jpg"; 
    SpecimensFrame[i] = loadImage(fileName);
  }
}

void draw() {
  image(SpecimensFrame[frame], 0, 0);

  frame++; 
  
  if (frame == numFrames) { 
    frame = 0; 
  }

  saveFrame("line-######.png");
}

If I load the images in the setup as below, when the draw runs it should theoretically run much more quickly but even with 50 images the sketch runs out of memory.

void setup() {
  size(1280, 720);
  background(175);
  for (int i = 0; i < 50; i++) {  
    String fileName = i + ".jpg"; 
    SpecimensFrame[i] = loadImage(fileName);
  }
}
2

Hi,

Wow it seems to be pretty expansive anyway to load and process.

You are right, if you go on this image size calculator website, the approximate file size of a 3200 x 5000px image is ~61 Mb so it makes :

61.03515625 * 5
=> 305.17578125 Mb > default 256Mb of memory

So increasing the available memory might help.

One way you can optimize this is to only load a fraction of the total number of images (say 1 out of 5 in your code) and then replace the loaded images with the new one so they can be garbage collected.

This is explained also in this thread : Memory issues, increase memory. - Processing Forum

Also be careful because in your draw() function, you are saving a .png image at each draw loop which is around 60 frames per second (except here where saving an image takes time) so you can quickly get thousands of images on your drive… If you only need to process some images once, use the noLoop() function.

Hope it helps!

1 Like
3

Oh, this is incredibly helpful!! Thanks so much :slight_smile:

A couple of points:

  • The idea of only loading a fraction of the total number of images is a great one. In fact, at any given time, I only need one loaded. I’ve looked at the link you provided and I would like to do as you suggest. Is the code below what you’re suggesting doing?

  • I’ve not come across the noLoop function before–I’m excited to have been introduced to it–thank you! I went to the reference page for noLoop() but I’m not sure how I would implement it in my sketch. Could you hint and how I could take advantage of it?

  • My impression with the code below is that a new .png file is saved at the end of each loop. If it takes Processing one second to load and draw the image, then am I correct in thinking that it will also only be saving one .png per second?

PImage[] SpecimensFrame = new PImage[100]; 

int frame = 0; 
int numFrames = 100; 

void setup() {
  size(1280, 720);
  background(175);
}

void draw() {
  SpecimensFrame[frame] = loadImage(frame + ".jpg");
  image(SpecimensFrame[frame], 0, 0);

  frame++; 
  
  if (frame == numFrames) { 
    frame = 0; 
  }

  saveFrame("line-######.png");  
}
4

With noLoop(), the draw() method will only called be called once (rather than indefinitely, at 60fps). To call draw() again, it must be done manually, with redraw().

The code would boil down to something like this:

void setup() {

noLoop();

}

void draw() {

PImage img;

  for (int i = 0; i < 10000; i++) {  
    String fileName = i + ".jpg"; 
    img = loadImage(fileName); // replace img object with current image
    image(img, x, y); // immediately rasterise image in the sketch
  }

saveFrame("line-######.png");  

}
5

Why don’t you load the images individually in draw(), execute your functions and release them with
SpecimensFrame[i] = null;

6

Nice to know that it helps!

Let me answer your questions :

  • For the loading part, your code is not exactly what I was suggesting. In fact at each draw() call, you save an image loaded with loadImage() inside your array of hundred of images. It’s doing the following :

    in setup:
SpecimensFrame = []

frame 0: 
SpecimensFrame = [ PImage("0.jpg") ]

frame 1: 
SpecimensFrame = [ PImage("0.jpg"), PImage("1.jpg") ]

...

    So you are just storing images inside an array (in memory then) but they are not going to be deleted from memory by the garbage collector.

    You need to create a single global variable at the top of your program and then override it when you load your image :

    PImage img;

void setup() {

}

void draw() {
  img = loadImage(frame + ".jpg");

  // Do your operations on the image
}

  • I don’t know what is the final goal of your program, but I usually use noLoop() at the end of the draw() function to stop the loop when I don’t need to make an animation. If you just process images without user interaction, you can just use the setup function. After all, you can write Java code in Processing.

1 Like
7

Hi Josephh,

Thanks once again for your response–I can feel my brain starting to make fundamental shifts around how I can structure the code.

Let me explain a little about what I’m trying to do:
I want to ultimately create a video animation. The frames that are being saved in the saveFrame() function are going to be fed into Premiere (or Processing’s Movie Maker) to ultimately be animated into an .mov. The Processing sketch is a means to that end, and so it is not interactive.

I think that I still need to use the draw() function because I need to take advantage of the frames (in which I can draw an image, take a snap shot with saveFrame(), and then repeat that process for each new image). Am I right in this thinking?

Is the following an accurate summary of what you’ve identified about my memory problem:
I am creating an array with:

PImage[] SpecimensFrame = new PImage[5];

When I assign values to it in draw, each new value gets placed in a new element in the array, but it is never actually erased. Therefore, after every frame, I add more memory strain. However, I don’t actually need the program to remember every element and am therefore adding unnecissary memory strain. Instead, I can create a single PImage variable (and not a PImage array):

PImage img;

When I do that, I can load a new image to that variable in each frame thereby replacing/overwriting the previous image. Ultimately this means that the memory isn’t holding onto every image from every frame, but rather just one image at any given time.

Is that correct?

Also, is:

  for (int i = 0; i < 10000; i++) {  
    String fileName = i + ".jpg"; 
    img = loadImage(fileName);
    image(img, x, y);
  }

in someway better than:

  for (int i = 0; i < 10000; i++) {  
    img = loadImage(i + ".jpg");
    image(img, x, y);
  }

Why the extra line of code? Is it simply a clarity thing for coders reading it later on?

Thanks again for your help on this!

8

Hello,

I have made such “movies” and saving to a png was very slow in my case.

There is a statement here about speed vs disk space:

image
image532×504 15.2 KB

I use the tif format because I had issues with tga:

This may now be fixed.

Have fun!

:)

1 Like
9

Hi,

I can clearly see that it’s starting to make sense!

Indeed it’s exactly what I (and others, @micycle @noel ) were suggesting. :wink:

Note that the garbage collector is not instantly releasing a previous overwritten PImage. It’s up to the JVM (Java virtual machine, running the bytecode produced by the Java compiler) to decide when to free the previous image. Anyway it’s going to greatly reduce memory usage!

I found those useful StackOverflow links about garbage collector and how to deal with it :

For your last question, this is just a question of clarity and style because this is equivalent in terms of result. For future code readers, you can add comments but at this point of the project is probably not necessary.

If you were worried about memory for creating an intermediate variable, you are creating it in a for loop, therefore it’s in a local scope meaning that as soon as you go out of the block (at the next iteration), the object can be freed from memory by the garbage collector. And also thousands of String is not that much.

10

Maybe this code from KevinWorkman can assist.

void setup() {
  size(200, 200);
  println(totalMem());
}
 
void draw() {
  PImage img = loadImage("myImage.bmp");
  image(img, 0, 0, width, height);
 
 
  if(frameCount % 10 == 0){
    int percent = (int)(100*(double)usedMem()/totalMem());
    println(percent + "%");
  }
 
}
 
public long totalMem() {
  return Runtime.getRuntime().totalMemory();
}
 
public long usedMem() {
  return Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
}
2 Likes
11

Thanks for this Noel–it was super helpful. I can see using this in other applications in the future as well. :slight_smile:

12

Thanks again for all of your help everyone. I’ve learned a lot in this thread!