Processing-java on Ubuntu Linux - Out of Memory issue

Processing
1

Hey there,

I have some data-vis sketches running via processing-java on an Ubuntu AWS instance that are throwing Out Of Memory errors, or more often, simply hanging and maxing-out the server CPU when they run.

Here’s an example of the error:
There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 50667520 bytes for committing reserved memory.

Sketches are running headless with XVFB (command line called from a php script), then pulling in some data, rendering and saving several images.

All sketches run as intended on my mac from the same Processing release (3.5.3)
And the simpler ones run reliably on the server. But more complex images seem to kill it.

I’m guessing I need to increase the memory available to processing-java, but I’m unclear on how to do that, or if there’s something else.

Any wisdom much appreciated.

1 Like
2

Try to increase the MEM resources allocated to that AWS machine by x2 or x4 fold and see if that solves the issue. Can you use any AWS monitoring to confirm it is the memory max out?

Kf

3

Thanks @kfrajer . Increasing the RAM is how I got the simpler ones to run reliably (I started on a T2 micro @ 1Gb and switched to T2 large @8Gb) But anything that isn’t dead simple still breaks it, so I’m guessing that processing-java isn’t getting to use the additional memory.

4

Some post that are relevant but nothing conclusive:

I am curious, are you running on BeamStalk or directly on a EC2?

From what I understand, the sketch runs fine with simple images. Can you describe what is a complex case? I am referring to:

Here are some other ideas:

  1. You need to confirm the issue is memory by using AWS monitoring. I believe you can use CloudWatch.
  2. Try the image optimization on some of those links I shared above.
  3. You are using processing-java --run. Have you consider exporting your application and calling the exported version?

I am not sure if you can pass memory flags to the jvm using processing-java. If option 3 above does not work, you can use the source code generated during the export application and build + run your application manually via java making sure you setup proper resource’s paths. I think I did this a while ago. If you get it to work locally, then you should be able to run the same command in your instance.

Kf

1 Like
5

Thank you @kfrajer, there’s some great info there.

It turns out I’m an idiot and the problem was an incorrect loadShape() path. It made processing-java hang, and after about 3 or 4 minutes, chew up 100% of the server CPU.

In the process I switched over to using exported linux executables, as you suggested, instead of sketches running via processing-java. These still failed but didn’t bring the server down with them.

All solved now, and executables seem more stable.

Cheers
Dan

1 Like