Processing on Raspberry pi 4 very slow?

Hi there. I’m having lots of performance issues with processing in my raspberry pi 4. On some pretty basic sketches i’m not getting more than 5 or 6 frames per second, which to me seems odd for a brand new raspberry pi 4. For instance, I’m using this example of the IPCapture class to display a live stream of a webcam. At native resolution (640x480) I’m getting a solid 15-20 fps but in fullscreen (1920x1080) the framerate drops to about 4 fps. Processing is also practically unusable in fullscreen, I’m getting almost 6 seconds of lag trying to scroll down and writing something.

I’ve looked at memory usage and it doesn’t go past 500MB (out of 4GB). Maximum CPU usage was 60%. I’ve tried overclocking the CPU and GPU but nothing really changed.
Then I thought maybe it could be the JDK I was using, so I tried installing the Oracle JDK and using that instead of an openjdk, but I was still having the same problems.

Any ideas? Is anyone else having serious performance issues?

Thank you!!

2 Likes

yes, possible, but compare something with that code / lib / link to a non existing IP cam
is difficult.

but i have a nice stress test for you

float rset=120, r;
int x = 0;
int w = 400;
int h = 200;
int grid=25, many = grid*grid;
boolean stressenable=true;

void settings() {
  size(w, h); // default use JAVA2D 
  // not on RPI  size(w, h,FX2D);
  //  size(w, h,P2D);
  //  size(w, h,P3D);
}

void setup() {
  frameRate(rset);
  println("target "+rset+" FPS");
  background(200, 200, 0);
  println("use key [p] for sysinfo, [n] numinfo, \n[s] stresstest ( "+grid+"*"+grid+" = "+many+" ) points, [+][-] grid +-5" );
}

void draw() {
  surface.setTitle("SYS INFO "+ nf(r, 0, 1)+" FPS" );
  if ( stressenable ) stress();
  r = frameRate;
  if ( r > 30 )      stroke(0, 200, 0); 
  else               stroke(200, 0, 0);
  line(x, height-2, x, height-2-r);            // FPS graph
  x++;
  if ( x > width ) { 
    x = 0; 
    background(200, 200, 0);
  }
}

void stress() {
  push();
  for ( int i=0; i<grid; i++) {
    for ( int j=0; j<grid; j++) {
      stroke(0, 255-random(255), random(255) );
      point(5+i, 5+j);
    }
  }
  pop();
}

void sysinfo() {
  println( "__SYS INFO :");
  println( "System     : " + System.getProperty("os.name") + "  " + System.getProperty("os.version") + "  " + System.getProperty("os.arch") );
  println( "JAVA       : " + System.getProperty("java.home")  + " rev: " +javaVersionName);
  //println( System.getProperty("java.class.path") );
  //println( "\n" + isGL() + "\n" );
  println( "OPENGL     : VENDOR " + PGraphicsOpenGL.OPENGL_VENDOR+" RENDERER " + PGraphicsOpenGL.OPENGL_RENDERER+" VERSION " + PGraphicsOpenGL.OPENGL_VERSION+" GLSL_VERSION: " + PGraphicsOpenGL.GLSL_VERSION);
  println( "user.home  : " + System.getProperty("user.home") );
  println( "user.dir   : " + System.getProperty("user.dir") );
  println( "user.name  : " + System.getProperty("user.name") );
  println( "sketchPath : " + sketchPath() );
  println( "dataPath   : " + dataPath("") );
  println( "dataFile   : " + dataFile("") );
  println( "frameRate  : target "+nf(rset, 0, 1)+" actual "+nf(r, 0, 1));
  println( "canvas     : width "+width+" height "+height+" pix "+(width*height));
}

void numinfo() {
  println( "__NUM INFO :");
  println( "byte    min: "+Byte.MIN_VALUE+   "\t\t\t max: "+Byte.MAX_VALUE);
  println( "short   min: "+Short.MIN_VALUE+  "\t\t max: "+Short.MAX_VALUE);
  println( "int     min: "+Integer.MIN_VALUE+"\t\t max: "  +Integer.MAX_VALUE);
  println( "long    min: "+Long.MIN_VALUE+   "\t max: "    +Long.MAX_VALUE);
  println( "float   min: "+Float.MIN_VALUE+  "\t\t max: "+Float.MAX_VALUE);
  println( "double  min: "+Double.MIN_VALUE+ "\t\t max: "+Double.MAX_VALUE);
}

void keyPressed() {
  if ( key == 'p' ) sysinfo();
  if ( key == 'n' ) numinfo();
  if ( key == 's' ) stressenable = ! stressenable;
  if ( key == '+' ) {
    grid +=5;
    many = grid*grid;
    println("stresstest ( "+grid+"*"+grid+" = "+many+" ) points" );
  }
  if ( key == '-' ) {
    grid -=5;
    many = grid*grid;
    println("stresstest ( "+grid+"*"+grid+" = "+many+" ) points" );
  }
}

i see 40 FPS on a RPI 3B+
++ without stress : key [s] 118 FPS

the RPI4B-4GB (60euro) should be faster as my 3B+,
BUT NOT 10 times slower as a 600euro desktop
but yes, big canvas/fullscreen is to avoid for dynamic applications.

3 Likes

Thanks! Very straightforward test. I’m getting 70 fps under stress and about 100 fps without. If I enable fullscreen though, the framerate drops to 7 fps. It seems to be working fine with a small window but drops performance drastically with a larger window.

Why is this? I figure i should be able to view a stream on fullscreen at at least 15 fps…

2 Likes

i try newest setup:
2019-09-26-raspbian-buster-full.zip
incl latest release of video lib:
https://github.com/processing/processing-video/releases/tag/r6-v2.0-beta4 !

and see 25FPS for 640*480 from my RPI cam v1 (rev 1.3 )

2 Likes

That was my issue, I got good fps for a low resolution screen but it got unusable when in fullscreen. Right now I am trying the GL Video library instead of IP Capture to see whether it handles video better. I haven’t been able to get it running though, I tried the example sketch “PixelArrayCapture” of the library and got this error message:

(java:10870): GStreamer-WARNING **: 10:34:05.771: Failed to load plugin '/home/pi/sketchbook/libraries/glvideo/library/linux-armv6hf/gstreamer-1.0/libgstopengl.so': libpng12.so.0: cannot open shared object file: No such file or directory
GLVideo requires the P2D or P3D renderer.

(java:10870): GLib-CRITICAL **: 10:34:06.001: g_error_free: assertion 'error != NULL' failed
RuntimeException: Could not load GStreamer

It looks like some sort of dependency issue? I have checked and the file does exist in that directory.

Any help would be appreciated!

2 Likes

if you have a new setup the

  • video

AND

  • GLvideo

( both from contribution manager ) not work anymore.

pls. load the new video lib as i link above.

if you have a older system ( as the raspbian zip i used above )
GLvideo works for the capture but for the
video need a repair, i did by a own build might read here

but best is just do a new setup / new processing install /
and video lib from
https://github.com/processing/processing-video/releases ,


all only tested on RPI3B+ hardware

3 Likes

I am using the latest Raspbian Buster release. Sorry I am kind of a noob at this, how do I proceed to install the new video library?

Thanks!

1 Like

did you download the file
https://github.com/processing/processing-video/releases/download/r6-v2.0-beta4/video-2.0-beta4.zip

and unzip it to /home/pi/sketchbook/libraries/video


i did it on a clean install, so no idea what you have to delete first
( it is very big and most of it you not need / ( /mac, /winxx, /src ) can delete later after testing )

1 Like

I unzipped the file in the libraries directory. I tried the “Reverse” example and got this error:

0:00:01.097548975 e[334m24339e[00m 0x93071b50 e[31;01mERROR  e[00m e[00m                 omx gstomx.c:1154:gst_omx_component_set_parameter:<omxh264dec-omxh264dec0>e[00m Set video_decode parameter at index 0x02000001: Bad parameter (0x80001005)
0:00:01.098285066 e[334m24339e[00m 0x93071b50 e[31;01mERROR  e[00m e[00m                 omx gstomx.c:1333:gst_omx_port_update_port_definition:<omxh264dec-omxh264dec0>e[00m Updated video_decode port 130 definition: Bad parameter (0x80001005)

And trying the GL Video example again, I’m geting the same error message as before.

1 Like

old video and GLvideo ( from contribution manager )
are not compiled for that new system, delete them.

new unzip
video

RPI cam and video work,
but the reverse ? is stuck at beginning?
try speed example…

but as i say, i not have your hardware

1 Like

I can’t try the capture examples because I am getting the stream over the network. So i just delete the video and gl video libraries? Then unzip the new video library into the same directory?

1 Like

so give me the code ( with public link ) i could try it?


yes, same as you had it from contribution manager
/libraries/video/…

if i show you a tree picture
terminal: tree -d
i would only confuse you as i changed my sketchbook path (preferences)
to same style as it is on windows:
Documents/Processing/libraries/video

2 Likes

hello,
i’ve got a raspberry pi4 too (4 giga) and testing some basic sketches and i noticed too a really really bad performance for processing. Not involving video capture but even simple moving ellipse around.
This is really sad because the main reason i’ve got the rasp is for processing installations.

1 Like

Have you tried enabling fake kms? However one thing I found doesn’t work for me on PI4 is video capture.

2 Likes

yes, without any improvements. I even set graphic memory to 256, but maybe is not enough.

1 Like

As a straight 1-1 comparison, the install on the pi4 I just got was much more unstable and slower then on the pi3. The other issue is that it appears i2c is broken, as my program crashes whenever I call anything i2c related.

I feel like I remember reading somewhere that the person who ported processing to the Pi stopped working on the project, which is a shame. I hope someone else can take up the mantle, as I love processing and I would really love the extra performance of the Pi 4 for my applications.

1 Like

There really is no reason why processing should be slower on PI4, in fact in my experience with PiCrate it is substantially faster. However I have not tried to install vanilla processing on my PI4. In my most recent release I use the jdk11 version installed by Buster, I use Fake KMS. Anyone wishing to work on vanilla processing for the PI should probably clone processing4, but try and work with the Buster installed jdk11 (32 bit hard float etc), rather than bundle the jdk in the vanilla processing way. That way they can take advantage of improvements/developments from PI side, it seems PI4 still has a few rough edges, but it is much improved since I got mine last year.

1 Like

Recently Video Capture is working for me (presumably after some Buster updates) with PiCrate using latest beta release of processing video library (but there are still some rough edges with video library).

1 Like

@ITman496 Regarding I2C is this interesting?

That is interesting! I’ll admit I’m a bit of a noob when it comes to all this stuff, so do you have a guide on how to set up processing on the pi 4 in a way that runs nicely and uses the correct version of java? I was relying on the preconfigured image for the pi3 to use it for my project, which relies on the hardware io libraries. I’m hoping like heck that an equivalent all nice wrapped up setup gets made for the pi 4!