Memory leak in PGraphics

Processing Coding Questions
1

Hi, I am working on a scientific visualization project for my degree and it requires us to draw many circles to the screen representing trees. This we did via the following code which runs as a thread and updates its PGraphic called graphics each time we filter or change zoom. Currently, if I try to re-render by clicking on the screen and moving around it works perfectly but if I do it too quickly and the threads get interrupted (which we allow) ram usage spikes up and we never get it back.

I have tried all sorts of fixes including using removeCache and image.dispose() as well as system.gc().
The only thing that fixes it is called g.removeCache on the specific graphics and System.gc() every frame (done in draw). This works perfectly but is obviously super inefficient and we drop frames madly.
Other weird behaviour is if I call only g.removeCache but do it every frame the memory spikes up and down to and from the same places (eg 700mb to 1.2gb in a few seconds and then back down to 700mb). Any help would be appreciated

while (true) {
  try {
    if (current) {
      sleep(10);
      //yield();
      continue;
    }

    int currentBuildCount = buildCount.get();
    // drawing loop
    graphics.beginDraw();
    //graphics.scale(1.0 / 2);
    graphics.clear();
    graphics.noStroke();
    graphics.blendMode(ADD);
    graphics.ellipseMode(CENTER);
    for (PlantTree.Node node : plants.filteredPlantsInRect(newPos, zoom.layerPlantRadius(layer).mult(1.1))) {
      color speciesColor = species.colors.get(node.plant.speciesId);
      graphics.fill(speciesColor, 200);
      if (buildCount.get() != currentBuildCount) {
        //System.out.println("Thread restarted!");
        g.removeCache(graphics);
        break;
      }
      // draw current plant
      PVector scaledPlantPos = zoom.plantSpaceToLayerSpace(node.plant.pos, newPos, layer);
      graphics.circle(scaledPlantPos.x, scaledPlantPos.y, max(node.plant.canopyRadius * 2 * zoom.layerScale[layer], 1));
    }
    graphics.endDraw();
    if (buildCount.get() != currentBuildCount) {
      //System.out.println("Thread restarted!");
      g.removeCache(graphics);
      continue;
    }

    // set pos after image is finished
    transferPixels(graphics);
    g.removeCache(graphics);
    graphics.clear();
    this.pos = this.newPos.copy();
    current = true;
  
  }
}

}

2

It depends how many new PImage calls or create graphics call your making. @GoToLoop will be more knowledgeable on this, but images are only cleared by the java garbage collector, setting something to null does not remove it from memory.

3

We create 6 PGraphics (one per thread) and we only create them once. We then copy the pixels to a PImage and use that to display it. Once a new layer must be rendered we stop making the thread wait and it calls beginDraw() of the same PGraphic. Doing this, and by clicking again before the thread re-renders we can go from 1gb to 3gb easily. @GoToLoop would appreciate any help on this, have checked many of the other threads you’ve replied to on similar issues but nothing has worked thus far.

4

Most I can do is tell you what I know about how a PGraphics caches a PImage.

Processing got 3 methods to display a PImage on a PGraphics:

  1. background()
  2. set()
  3. image()

Both background() & set() merely transfer a PImage::pixels[] to a PGraphics disregarding any transparency & tint between them; but having the advantage of not being cached.

Method image() is more complete, but it caches its PImage argument if it’s not been yet.

Each cache primarily consists of a whole clone of a PImage::pixels[], thus more than doubling memory requirements!

As a general rule prefer reusing a PImage object rather than discarding it, especially if we end up passing it as an image() argument.

Be careful about using method PImage::get() to make a new clone of it.

Even though the cloned PImage got the same content as its original, method image() will consider it a different object, therefore it ends up caching the clone as well!

2 Likes
5

I’ve done some tests and I could be wrong but I believe that the issue is from drawing a circle to the graphic. Each circle is a plant and our largest file has 760,000 plants and so we are drawing many of these and I think they are just being cached so quickly that they can’t be GC’ed. Any ideas?

6

Merely drawing circles to a PGraphics doesn’t cache anything, but rather change the contents of its pixels[].

I’m lost to whatever is causing your sketch to spike its RAM usage.

Aren’t you somehow creating too many temporary objects before drawing those circles?

7

image
image1516Ă—736 105 KB

This is from the memory profiler I have been using. The RenderThread is the thread class we create to create a graphic for each zoom level (only 6 threads for 6 zoom levels). We only create the graphic once and then just redraw onto the graphic. From what I can see it isn’t other temporary objects and maybe just something in the PGraphic we don’t know about. On top of that, if we only draw to the Pgraphic by updating its pixels[] array we don’t experience any memory issues, but drawing like that is difficult since we would have to manually set the correct pixels into a circle.

8

Ugh! All of that just to render a single circle() over a PGraphicsJava2D? :scream:

I’m afraid there’s no much we can do. :disappointed:

Maybe call noSmooth() & blendMode(REPLACE) over the PGraphics to turn off most of its effects?

2 Likes
9

Well I guess the issue really comes from the fact that we need to draw around 760,000 circles onto a single PGraphic at the highest zoom layer, and then we thread the other layers so they are drawing the zoomed in layers at the same time… just a whole bunch of bloat :frowning:
Thank you for the answers though!

10

We need transparency so we are already using blendMode(ADD), but making the change to all layers having noSmooth() seemed to actually make a massive difference, as well as I made it so that we only render the current layer instead of all layers. I’m sure that theres still some issues but thank you for the suggestion it really helped!

1 Like
11

hello, maybe my input is a bit off your request, but I cannot help to wonder…

why do you need a thread in the first place ?
if you zoom/move around, the setup of your trees will not be changed.
if you change filter options, obviously you will have to create a new display/arrangment of your trees. that - of course - takes time. and you can only provide an updated image after it has been created…

what would you need to happen in that time ? should the user switch filters another time while you re-caclulate ? it doesn’t make too much sense to me… after selecting a certain configuration, I would supposedly want to see the result.
in fact, someone clicking buttons 5 times in a row should be prevented from doing so, no ?

when you scroll/ zoom, the typical advantage may be that you do not have to calculate all the trees (which might take long). but in my experience calculations don’t eat up as much performance as actually displaying or loading stuff does.

so when you move the image, is it really necessary to recalculate the whole bunch of trees ?
or… how long does it take to calculate all trees offscreen ?
maybe you can keep an image of all the trees that you use for moving, scaling. or use “buckets” of offscreen parts that you update when the user does nothing.

however, if there is a long procedure that simply takes time, the user should be aware to expect that it is not being updated in realtime…

in any case: my own experiences with the phenomenon you describe is actually when I call some procedure a second time BEFORE it has been finished.so imagine that (not knowing your method) that by moving around, you call your thread again and again.
afaik processing uses a method that you call as a thread, yet every time you start this thread (that is many threads in parallel) they literally access the same method having to wait for each other to use it.

so that all sounds strongly like a number of threads stepping on each others feet, because the second call to a thread starts before the first one has finished.

secondly you might be interested in our last work. you may find it quiet… familiar :wink:
https://www.black-moon.at

it has only 10k trees at most, although each tree is holding more than a single circle.
we actually used shapes to create the form of each tree at startup, while filtering controls the positioning of trees but not their form.

still displaying all trees takes a couple of seconds, but imo its more of an display capacity issue.

yet, the only thread we use is while loading the 10k shapes into memory, that’s it.

3 Likes