Processing on Raspberry pi 4 very slow?

Hi there. I’m having lots of performance issues with processing in my raspberry pi 4. On some pretty basic sketches i’m not getting more than 5 or 6 frames per second, which to me seems odd for a brand new raspberry pi 4. For instance, I’m using this example of the IPCapture class to display a live stream of a webcam. At native resolution (640x480) I’m getting a solid 15-20 fps but in fullscreen (1920x1080) the framerate drops to about 4 fps. Processing is also practically unusable in fullscreen, I’m getting almost 6 seconds of lag trying to scroll down and writing something.

I’ve looked at memory usage and it doesn’t go past 500MB (out of 4GB). Maximum CPU usage was 60%. I’ve tried overclocking the CPU and GPU but nothing really changed.
Then I thought maybe it could be the JDK I was using, so I tried installing the Oracle JDK and using that instead of an openjdk, but I was still having the same problems.

Any ideas? Is anyone else having serious performance issues?

Thank you!!

1 Like

yes, possible, but compare something with that code / lib / link to a non existing IP cam
is difficult.

but i have a nice stress test for you

float rset=120, r;
int x = 0;
int w = 400;
int h = 200;
int grid=25, many = grid*grid;
boolean stressenable=true;

void settings() {
  size(w, h); // default use JAVA2D 
  // not on RPI  size(w, h,FX2D);
  //  size(w, h,P2D);
  //  size(w, h,P3D);
}

void setup() {
  frameRate(rset);
  println("target "+rset+" FPS");
  background(200, 200, 0);
  println("use key [p] for sysinfo, [n] numinfo, \n[s] stresstest ( "+grid+"*"+grid+" = "+many+" ) points, [+][-] grid +-5" );
}

void draw() {
  surface.setTitle("SYS INFO "+ nf(r, 0, 1)+" FPS" );
  if ( stressenable ) stress();
  r = frameRate;
  if ( r > 30 )      stroke(0, 200, 0); 
  else               stroke(200, 0, 0);
  line(x, height-2, x, height-2-r);            // FPS graph
  x++;
  if ( x > width ) { 
    x = 0; 
    background(200, 200, 0);
  }
}

void stress() {
  push();
  for ( int i=0; i<grid; i++) {
    for ( int j=0; j<grid; j++) {
      stroke(0, 255-random(255), random(255) );
      point(5+i, 5+j);
    }
  }
  pop();
}

void sysinfo() {
  println( "__SYS INFO :");
  println( "System     : " + System.getProperty("os.name") + "  " + System.getProperty("os.version") + "  " + System.getProperty("os.arch") );
  println( "JAVA       : " + System.getProperty("java.home")  + " rev: " +javaVersionName);
  //println( System.getProperty("java.class.path") );
  //println( "\n" + isGL() + "\n" );
  println( "OPENGL     : VENDOR " + PGraphicsOpenGL.OPENGL_VENDOR+" RENDERER " + PGraphicsOpenGL.OPENGL_RENDERER+" VERSION " + PGraphicsOpenGL.OPENGL_VERSION+" GLSL_VERSION: " + PGraphicsOpenGL.GLSL_VERSION);
  println( "user.home  : " + System.getProperty("user.home") );
  println( "user.dir   : " + System.getProperty("user.dir") );
  println( "user.name  : " + System.getProperty("user.name") );
  println( "sketchPath : " + sketchPath() );
  println( "dataPath   : " + dataPath("") );
  println( "dataFile   : " + dataFile("") );
  println( "frameRate  : target "+nf(rset, 0, 1)+" actual "+nf(r, 0, 1));
  println( "canvas     : width "+width+" height "+height+" pix "+(width*height));
}

void numinfo() {
  println( "__NUM INFO :");
  println( "byte    min: "+Byte.MIN_VALUE+   "\t\t\t max: "+Byte.MAX_VALUE);
  println( "short   min: "+Short.MIN_VALUE+  "\t\t max: "+Short.MAX_VALUE);
  println( "int     min: "+Integer.MIN_VALUE+"\t\t max: "  +Integer.MAX_VALUE);
  println( "long    min: "+Long.MIN_VALUE+   "\t max: "    +Long.MAX_VALUE);
  println( "float   min: "+Float.MIN_VALUE+  "\t\t max: "+Float.MAX_VALUE);
  println( "double  min: "+Double.MIN_VALUE+ "\t\t max: "+Double.MAX_VALUE);
}

void keyPressed() {
  if ( key == 'p' ) sysinfo();
  if ( key == 'n' ) numinfo();
  if ( key == 's' ) stressenable = ! stressenable;
  if ( key == '+' ) {
    grid +=5;
    many = grid*grid;
    println("stresstest ( "+grid+"*"+grid+" = "+many+" ) points" );
  }
  if ( key == '-' ) {
    grid -=5;
    many = grid*grid;
    println("stresstest ( "+grid+"*"+grid+" = "+many+" ) points" );
  }
}

i see 40 FPS on a RPI 3B+
++ without stress : key [s] 118 FPS

the RPI4B-4GB (60euro) should be faster as my 3B+,
BUT NOT 10 times slower as a 600euro desktop
but yes, big canvas/fullscreen is to avoid for dynamic applications.

2 Likes

Thanks! Very straightforward test. I’m getting 70 fps under stress and about 100 fps without. If I enable fullscreen though, the framerate drops to 7 fps. It seems to be working fine with a small window but drops performance drastically with a larger window.

Why is this? I figure i should be able to view a stream on fullscreen at at least 15 fps…

1 Like

i try newest setup:
2019-09-26-raspbian-buster-full.zip
incl latest release of video lib:
https://github.com/processing/processing-video/releases/tag/r6-v2.0-beta4 !

and see 25FPS for 640*480 from my RPI cam v1 (rev 1.3 )

1 Like

That was my issue, I got good fps for a low resolution screen but it got unusable when in fullscreen. Right now I am trying the GL Video library instead of IP Capture to see whether it handles video better. I haven’t been able to get it running though, I tried the example sketch “PixelArrayCapture” of the library and got this error message:

(java:10870): GStreamer-WARNING **: 10:34:05.771: Failed to load plugin '/home/pi/sketchbook/libraries/glvideo/library/linux-armv6hf/gstreamer-1.0/libgstopengl.so': libpng12.so.0: cannot open shared object file: No such file or directory
GLVideo requires the P2D or P3D renderer.

(java:10870): GLib-CRITICAL **: 10:34:06.001: g_error_free: assertion 'error != NULL' failed
RuntimeException: Could not load GStreamer

It looks like some sort of dependency issue? I have checked and the file does exist in that directory.

Any help would be appreciated!

1 Like

if you have a new setup the

  • video

AND

  • GLvideo

( both from contribution manager ) not work anymore.

pls. load the new video lib as i link above.

if you have a older system ( as the raspbian zip i used above )
GLvideo works for the capture but for the
video need a repair, i did by a own build might read here

but best is just do a new setup / new processing install /
and video lib from
https://github.com/processing/processing-video/releases ,


all only tested on RPI3B+ hardware

2 Likes

I am using the latest Raspbian Buster release. Sorry I am kind of a noob at this, how do I proceed to install the new video library?

Thanks!

did you download the file
https://github.com/processing/processing-video/releases/download/r6-v2.0-beta4/video-2.0-beta4.zip

and unzip it to /home/pi/sketchbook/libraries/video


i did it on a clean install, so no idea what you have to delete first
( it is very big and most of it you not need / ( /mac, /winxx, /src ) can delete later after testing )

I unzipped the file in the libraries directory. I tried the “Reverse” example and got this error:

0:00:01.097548975 e[334m24339e[00m 0x93071b50 e[31;01mERROR  e[00m e[00m                 omx gstomx.c:1154:gst_omx_component_set_parameter:<omxh264dec-omxh264dec0>e[00m Set video_decode parameter at index 0x02000001: Bad parameter (0x80001005)
0:00:01.098285066 e[334m24339e[00m 0x93071b50 e[31;01mERROR  e[00m e[00m                 omx gstomx.c:1333:gst_omx_port_update_port_definition:<omxh264dec-omxh264dec0>e[00m Updated video_decode port 130 definition: Bad parameter (0x80001005)

And trying the GL Video example again, I’m geting the same error message as before.

old video and GLvideo ( from contribution manager )
are not compiled for that new system, delete them.

new unzip
video

RPI cam and video work,
but the reverse ? is stuck at beginning?
try speed example…

but as i say, i not have your hardware

I can’t try the capture examples because I am getting the stream over the network. So i just delete the video and gl video libraries? Then unzip the new video library into the same directory?

so give me the code ( with public link ) i could try it?


yes, same as you had it from contribution manager
/libraries/video/…

if i show you a tree picture
terminal: tree -d
i would only confuse you as i changed my sketchbook path (preferences)
to same style as it is on windows:
Documents/Processing/libraries/video

1 Like

hello,
i’ve got a raspberry pi4 too (4 giga) and testing some basic sketches and i noticed too a really really bad performance for processing. Not involving video capture but even simple moving ellipse around.
This is really sad because the main reason i’ve got the rasp is for processing installations.

Have you tried enabling fake kms? However one thing I found doesn’t work for me on PI4 is video capture.

1 Like

yes, without any improvements. I even set graphic memory to 256, but maybe is not enough.