Applying Frame differencing tool to video file and exporting data as a csv file?

Processing Libraries
1

Hi, very new to coding, but for a MA project I am trying to develop a script that will read movement in a series of videos I have created and enable me to export the data as a CSV file?

This existing tool is almost perfect, but rather than capturing through a webcam, what changes would I need to make to get the script to read existing video files? And could I get the script to produce a csv file of the numerical data shown in the console?

Frame differencing tool code:

import processing.video.*;

int numPixels;
int[] previousFrame;
Capture video;

void setup() {
  size(640, 480);
  
  // This the default video input, see the GettingStartedCapture 
  // example if it creates an error
  video = new Capture(this, width, height);
  
  // Start capturing the images from the camera
  video.start(); 
  
  numPixels = video.width * video.height;
  // Create an array to store the previously captured frame
  previousFrame = new int[numPixels];
  loadPixels();
}

void draw() {
  if (video.available()) {
    // When using video to manipulate the screen, use video.available() and
    // video.read() inside the draw() method so that it's safe to draw to the screen
    video.read(); // Read the new frame from the camera
    video.loadPixels(); // Make its pixels[] array available
    
    int movementSum = 0; // Amount of movement in the frame
    for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
      color currColor = video.pixels[i];
      color prevColor = previousFrame[i];
      // Extract the red, green, and blue components from current pixel
      int currR = (currColor >> 16) & 0xFF; // Like red(), but faster
      int currG = (currColor >> 8) & 0xFF;
      int currB = currColor & 0xFF;
      // Extract red, green, and blue components from previous pixel
      int prevR = (prevColor >> 16) & 0xFF;
      int prevG = (prevColor >> 8) & 0xFF;
      int prevB = prevColor & 0xFF;
      // Compute the difference of the red, green, and blue values
      int diffR = abs(currR - prevR);
      int diffG = abs(currG - prevG);
      int diffB = abs(currB - prevB);
      // Add these differences to the running tally
      movementSum += diffR + diffG + diffB;
      // Render the difference image to the screen
      pixels[i] = color(diffR, diffG, diffB);
      // The following line is much faster, but more confusing to read
      //pixels[i] = 0xff000000 | (diffR << 16) | (diffG << 8) | diffB;
      // Save the current color into the 'previous' buffer
      previousFrame[i] = currColor;
    }
    // To prevent flicker from frames that are all black (no movement),
    // only update the screen if the image has changed.
    if (movementSum > 0) {
      updatePixels();
      println(movementSum); // Print the total amount of movement to the console
    }
  }
}

Any help welcome!

1 Like
2

exporting data as a csv file

create a table,

make columns / for a CSV header line

on mouse click create record

on key [s] save CSV file

Table table;
String outfile = "data/test.csv"; 

void setup() {
  size(300, 300);
  makeTable();
  strokeWeight(5);
  println("use: mouseClick and key [s] for save to CSV");
}

void draw() {
}

void mousePressed() {
  makeLine();
  point(mouseX, mouseY);
}

void makeTable() {
  table = new Table();
  table.addColumn("time");
  table.addColumn("x");
  table.addColumn("y");
}

void makeLine() {
  TableRow newRow = table.addRow();
  newRow.setString("time", ""+year()+"_"+nf(month(), 2)+"_"+nf(day(), 2)+"_"+nf(hour(), 2)+"_"+nf(minute(), 2)+"_"+nf(second(), 2));
  newRow.setInt("x", mouseX);
  newRow.setInt("y", mouseY);
}

void keyPressed() {
  if ( key == 's' ) {
    saveTable(table, outfile);
    println("save to "+outfile);
  }
}
2 Likes
3

Begin by looking at the examples for video playback in the video library Examples menu.

import processing.video.*;
Movie movie;

void setup() {
  size(640, 360);
  movie2 = new Movie(this, "amovie.mp4");
}

void movieEvent(Movie m) {
  m.read();
}

void draw() {
  if (movie.available() {
    // compute frame differencing on `movie` and store data
  }
}

I’m assuming that you are computing inter-frame differences on a single movie – not comparing simultaneous frames on two different movies…?