How can I do headless / virtual screen buffer pixel reading?

Processing for Pi
1

Hi

I currently have a setup with Processing running on Raspberry Pi, sending artnet messages across multiple universes to some Teensy’s to do LED lighting animations using @cansik 's artnet library. I’m running into serious framerate issues - with drawing animations to screen, then reading from screen into a pixel buffer and then sending messages over artnet, it runs at approximately 7fps.

I know that I can run the whole thing headless, or by not drawing to screen, my fps bumps up to 45-50fps, which is fine, but it limits my ability to make content through creating video or animations and filling my pixel buffer from there. Does anyone know of ways to do this maybe via a virtual screen buffer? The workflow would be create content and read pixels to a pixel buffer from screen during development, and then switch to a virtual buffer when running live.

Code below. Setting the writeToScreen boolean to false, switches the code to writing directly to the pixel buffer.

import ch.bildspur.artnet.*;
import processing.serial.*;

//___________________________
// setup pattern
boolean readFromScreen = false;
boolean writeToScreen = true;
Pattern patterns[] = {
  new TraceDown(), new TraceDown(), new TraceDown(), new TraceDown(), new FadeTrace(), new TraceDown(), 
  new TraceDown(), new TraceDown(), new TraceDown(), new TraceDown(), new FadeTrace(), new TraceDown(), 
  new TraceDown(), new TraceDown(), new TraceDown()
};

//___________________________
// setup artnet 
ArtNetClient artnet;
int numUniverse = 15;
int numChannels = 510;
byte[] dmxData = new byte[numChannels];
ArtnetDMX Artnetclass = new ArtnetDMX();

//___________________________
// stetup serial
Serial port;  // Create object from Serial class
String data = "0 0";     // Data received from the serial port
int[] nums;
byte[] inBuffer = new byte[4];

int windSpeed;
int windDir;
float windSpeedCal;

//___________________________ 
// setup leds
int numLeds = 88;
color led[][] = new color[numChannels/3][numUniverse];
int size = 2;
color[][] pixelBuffer = new color[numChannels/3][numUniverse];

//___________________________
// setup timer
long[] ellapseTimeMs = new long[numUniverse];
long[] ellapseTimeMsStartTime = new long[numUniverse];
float durationMs = 3000;
boolean direction = true; 


//_________________________________________________________
void setup()
{

  size(300, 80);
  colorMode(HSB, 360, 100, 100);
  textAlign(CENTER, CENTER);
  textSize(20);

  // create artnet client without buffer (no receving needed)
  artnet = new ArtNetClient(null);
  artnet.start();

  // create port
  //String portName = Serial.list()[0];
  //port = new Serial(this, portName, 2000000);
}

//_________________________________________________________
void draw()
{
  // create color
  int c = color(frameCount % 360, 80, 100);

  background(0);
  stroke(0);

  //change direction
  if (ellapseTimeMs[0]> durationMs) direction = !direction;
  // choose pattern to run on LED strip
  // int pattern = 0;  
  for (int i = 0; i <numChannels/3; i++) {
    for (int j = 0; j < numUniverse; j++) {
      if (ellapseTimeMs[j]> durationMs) {
        ellapseTimeMsStartTime[j] = 0;
      } else if (direction==true) {
        float position = i/(float)(numChannels/3);
        float remaining = 1.0 - ellapseTimeMs[j]/durationMs;
        if (readFromScreen == false){
          pixelBuffer[i][j] = patterns[j].paintLed(position, remaining, led[i][j]);
        } else {
          led[i][j] = patterns[j].paintLed(position, remaining, led[i][j]);
        }
       
      } else {
        float position = 1.0 - (i/(float)(numChannels/3));
        float remaining = ellapseTimeMs[j]/durationMs;
        if (readFromScreen == false){
          pixelBuffer[i][j] = patterns[j].paintLed(position, remaining, led[i][j]);
        } else {
          led[i][j] = patterns[j].paintLed(position, remaining, led[i][j]);
        }
      }
    }
  }

  if (writeToScreen == true) {
    showPattern();
  }
  
  if (readFromScreen == true){
    updatePixelBuffer();
  } 
  
  Artnetclass.updateArtnet(artnet, dmxData, pixelBuffer);
  //oldUpdateArtnet();

  updateEllapseTime();
  println(frameRate);

  // show values
  //text("R: " + (int)red(c) + " Green: " + (int)green(c) + " Blue: " + (int)blue(c), width-200, height-50);
}


// clock function
void updateEllapseTime() {
  for (int j = 0; j < numUniverse; j++) {
    if (ellapseTimeMsStartTime[j] == 0) {
      ellapseTimeMsStartTime[j] = millis();
      ellapseTimeMs[j] = 0;
    } else {
      ellapseTimeMs[j] = millis() - ellapseTimeMsStartTime[j];
    }
  }
}

// storing pixels from screen
void updatePixelBuffer() {
  for (int i = 0; i < numChannels/3; i++) {
    for (int j = 0; j < numUniverse; j++) {
      // read screen pixels and assign to pixel buffer
      //pixelBuffer[i][j] = get(i*size/2+(size/4), j*size+size/2);
      pixelBuffer[i][j] = get(i*size +size/2, j*size+size/2);
      fill(pixelBuffer[i][j]);
      stroke(pixelBuffer[i][j]);
      rect(50+i, 50+j, 1, 1);
    }
  }
}

// draw pattern on screen
void showPattern() {
  for (int i = 0; i < numChannels/3; i++) {
    for (int j = 0; j < numUniverse; j++) {
      // show only pixel buffer if not reading from screen
      if (readFromScreen == false){
          fill(pixelBuffer[i][j]);
        } else {
          fill(led[i][j]);
        }
      rect(i*size+size, j*size+size, size, size);
    }
  }
}
2

@adellelin Perhaps this is useful: https://github.com/processing/processing/wiki/Running-without-a-Display

Unclear what the performance would be when using Xvfb, but it’s headless…

1 Like