Processing on Raspberry pi 4 very slow?

Hi there. I’m having lots of performance issues with processing in my raspberry pi 4. On some pretty basic sketches i’m not getting more than 5 or 6 frames per second, which to me seems odd for a brand new raspberry pi 4. For instance, I’m using this example of the IPCapture class to display a live stream of a webcam. At native resolution (640x480) I’m getting a solid 15-20 fps but in fullscreen (1920x1080) the framerate drops to about 4 fps. Processing is also practically unusable in fullscreen, I’m getting almost 6 seconds of lag trying to scroll down and writing something.

I’ve looked at memory usage and it doesn’t go past 500MB (out of 4GB). Maximum CPU usage was 60%. I’ve tried overclocking the CPU and GPU but nothing really changed.
Then I thought maybe it could be the JDK I was using, so I tried installing the Oracle JDK and using that instead of an openjdk, but I was still having the same problems.

Any ideas? Is anyone else having serious performance issues?

Thank you!!

1 Like

yes, possible, but compare something with that code / lib / link to a non existing IP cam
is difficult.

but i have a nice stress test for you

float rset=120, r;
int x = 0;
int w = 400;
int h = 200;
int grid=25, many = grid*grid;
boolean stressenable=true;

void settings() {
  size(w, h); // default use JAVA2D 
  // not on RPI  size(w, h,FX2D);
  //  size(w, h,P2D);
  //  size(w, h,P3D);
}

void setup() {
  frameRate(rset);
  println("target "+rset+" FPS");
  background(200, 200, 0);
  println("use key [p] for sysinfo, [n] numinfo, \n[s] stresstest ( "+grid+"*"+grid+" = "+many+" ) points, [+][-] grid +-5" );
}

void draw() {
  surface.setTitle("SYS INFO "+ nf(r, 0, 1)+" FPS" );
  if ( stressenable ) stress();
  r = frameRate;
  if ( r > 30 )      stroke(0, 200, 0); 
  else               stroke(200, 0, 0);
  line(x, height-2, x, height-2-r);            // FPS graph
  x++;
  if ( x > width ) { 
    x = 0; 
    background(200, 200, 0);
  }
}

void stress() {
  push();
  for ( int i=0; i<grid; i++) {
    for ( int j=0; j<grid; j++) {
      stroke(0, 255-random(255), random(255) );
      point(5+i, 5+j);
    }
  }
  pop();
}

void sysinfo() {
  println( "__SYS INFO :");
  println( "System     : " + System.getProperty("os.name") + "  " + System.getProperty("os.version") + "  " + System.getProperty("os.arch") );
  println( "JAVA       : " + System.getProperty("java.home")  + " rev: " +javaVersionName);
  //println( System.getProperty("java.class.path") );
  //println( "\n" + isGL() + "\n" );
  println( "OPENGL     : VENDOR " + PGraphicsOpenGL.OPENGL_VENDOR+" RENDERER " + PGraphicsOpenGL.OPENGL_RENDERER+" VERSION " + PGraphicsOpenGL.OPENGL_VERSION+" GLSL_VERSION: " + PGraphicsOpenGL.GLSL_VERSION);
  println( "user.home  : " + System.getProperty("user.home") );
  println( "user.dir   : " + System.getProperty("user.dir") );
  println( "user.name  : " + System.getProperty("user.name") );
  println( "sketchPath : " + sketchPath() );
  println( "dataPath   : " + dataPath("") );
  println( "dataFile   : " + dataFile("") );
  println( "frameRate  : target "+nf(rset, 0, 1)+" actual "+nf(r, 0, 1));
  println( "canvas     : width "+width+" height "+height+" pix "+(width*height));
}

void numinfo() {
  println( "__NUM INFO :");
  println( "byte    min: "+Byte.MIN_VALUE+   "\t\t\t max: "+Byte.MAX_VALUE);
  println( "short   min: "+Short.MIN_VALUE+  "\t\t max: "+Short.MAX_VALUE);
  println( "int     min: "+Integer.MIN_VALUE+"\t\t max: "  +Integer.MAX_VALUE);
  println( "long    min: "+Long.MIN_VALUE+   "\t max: "    +Long.MAX_VALUE);
  println( "float   min: "+Float.MIN_VALUE+  "\t\t max: "+Float.MAX_VALUE);
  println( "double  min: "+Double.MIN_VALUE+ "\t\t max: "+Double.MAX_VALUE);
}

void keyPressed() {
  if ( key == 'p' ) sysinfo();
  if ( key == 'n' ) numinfo();
  if ( key == 's' ) stressenable = ! stressenable;
  if ( key == '+' ) {
    grid +=5;
    many = grid*grid;
    println("stresstest ( "+grid+"*"+grid+" = "+many+" ) points" );
  }
  if ( key == '-' ) {
    grid -=5;
    many = grid*grid;
    println("stresstest ( "+grid+"*"+grid+" = "+many+" ) points" );
  }
}

i see 40 FPS on a RPI 3B+
++ without stress : key [s] 118 FPS

the RPI4B-4GB (60euro) should be faster as my 3B+,
BUT NOT 10 times slower as a 600euro desktop
but yes, big canvas/fullscreen is to avoid for dynamic applications.

2 Likes

Thanks! Very straightforward test. I’m getting 70 fps under stress and about 100 fps without. If I enable fullscreen though, the framerate drops to 7 fps. It seems to be working fine with a small window but drops performance drastically with a larger window.

Why is this? I figure i should be able to view a stream on fullscreen at at least 15 fps…

1 Like

i try newest setup:
2019-09-26-raspbian-buster-full.zip
incl latest release of video lib:
https://github.com/processing/processing-video/releases/tag/r6-v2.0-beta4 !

and see 25FPS for 640*480 from my RPI cam v1 (rev 1.3 )

1 Like

That was my issue, I got good fps for a low resolution screen but it got unusable when in fullscreen. Right now I am trying the GL Video library instead of IP Capture to see whether it handles video better. I haven’t been able to get it running though, I tried the example sketch “PixelArrayCapture” of the library and got this error message:

(java:10870): GStreamer-WARNING **: 10:34:05.771: Failed to load plugin '/home/pi/sketchbook/libraries/glvideo/library/linux-armv6hf/gstreamer-1.0/libgstopengl.so': libpng12.so.0: cannot open shared object file: No such file or directory
GLVideo requires the P2D or P3D renderer.

(java:10870): GLib-CRITICAL **: 10:34:06.001: g_error_free: assertion 'error != NULL' failed
RuntimeException: Could not load GStreamer

It looks like some sort of dependency issue? I have checked and the file does exist in that directory.

Any help would be appreciated!

1 Like

if you have a new setup the

  • video

AND

  • GLvideo

( both from contribution manager ) not work anymore.

pls. load the new video lib as i link above.

if you have a older system ( as the raspbian zip i used above )
GLvideo works for the capture but for the
video need a repair, i did by a own build might read here

but best is just do a new setup / new processing install /
and video lib from
https://github.com/processing/processing-video/releases ,


all only tested on RPI3B+ hardware

2 Likes

I am using the latest Raspbian Buster release. Sorry I am kind of a noob at this, how do I proceed to install the new video library?

Thanks!

did you download the file
https://github.com/processing/processing-video/releases/download/r6-v2.0-beta4/video-2.0-beta4.zip

and unzip it to /home/pi/sketchbook/libraries/video


i did it on a clean install, so no idea what you have to delete first
( it is very big and most of it you not need / ( /mac, /winxx, /src ) can delete later after testing )

I unzipped the file in the libraries directory. I tried the “Reverse” example and got this error:

0:00:01.097548975 e[334m24339e[00m 0x93071b50 e[31;01mERROR  e[00m e[00m                 omx gstomx.c:1154:gst_omx_component_set_parameter:<omxh264dec-omxh264dec0>e[00m Set video_decode parameter at index 0x02000001: Bad parameter (0x80001005)
0:00:01.098285066 e[334m24339e[00m 0x93071b50 e[31;01mERROR  e[00m e[00m                 omx gstomx.c:1333:gst_omx_port_update_port_definition:<omxh264dec-omxh264dec0>e[00m Updated video_decode port 130 definition: Bad parameter (0x80001005)

And trying the GL Video example again, I’m geting the same error message as before.

old video and GLvideo ( from contribution manager )
are not compiled for that new system, delete them.

new unzip
video

RPI cam and video work,
but the reverse ? is stuck at beginning?
try speed example…

but as i say, i not have your hardware

I can’t try the capture examples because I am getting the stream over the network. So i just delete the video and gl video libraries? Then unzip the new video library into the same directory?

so give me the code ( with public link ) i could try it?


yes, same as you had it from contribution manager
/libraries/video/…

if i show you a tree picture
terminal: tree -d
i would only confuse you as i changed my sketchbook path (preferences)
to same style as it is on windows:
Documents/Processing/libraries/video

1 Like