Playing videos frame by frame

I found a few posts in older forums but I can’t find a definitive answer.

I have multiple videos in a collage. I want to save the frame of the whole composition to render it in a sequence of PNGs. The thing is: I can’t play frame by frame all the videos and using the code bellow they stop when reaching the end (despite the use of loop and other operation). Any ideas?

Despite I even put a delay after the saveFrame and that I’m using the speed(0) method I still have videos that run too fast in the rendered images (as they jump or lost frames). This is driving me crazy.

For the code bellow you just need a ‘data’ folder with a bunch of mp4 videos. It should work.
Thanks so much for any help!

import processing.video.*;
 
//File path; 
String[] files;
StringList f;
int numfiles=0;
int timer = -5000;
int tcontrol= 2000;
int numVideos=8;
int idx=0;
String scrtxt;
int recording=0;

VideoM[] v = new VideoM[numVideos];
Movie[] vv;


void setup(){
  size(1400, 1400, P2D);
  frameRate(25);
  files = loadfiles(); 
  
  // dirty array randomization
  f  = new StringList();
  for(int i = 0;i < files.length;i++){     
    f.append(files[i]);   
  }

  f.shuffle();

  for(int i = 0;i < f.size()-1;i++){
     files[i]=f.get(i);
  }
  //-------------------------
  
  numfiles = files.length;
  if (numfiles>9) { numfiles=9; }
  loadvideos();
    
  for(int i = 0;i < numVideos;i++){
     v[i] = new VideoM();
  }
}


void draw() {
  
  background(0);
  for(int i = 0; i <numVideos; i++){
     v[i].display();
  }
 
  if ((millis() - timer) > tcontrol)  {
    timer = millis();
    tcontrol= int(random(4,6))*500; 
  }

  saveFrame("render/render-######.png");
  delay(5);
   
}



void loadvideos(){ 
 String video;
 vv = new Movie[numfiles];
 
 for (int i=0; i<numfiles; i++){
   //video= files[int(random(numfiles))];
   video= f.get(i);
   vv[i] = new Movie(this, video);
   vv[i].loop();
   vv[i].pause();
   println ("Loading ", video);
 } 
}

class VideoM {
 
 Movie m;
 String video;

 int x;
 int y;
 int w;
 int h;
 int alpha;
 int alphai;
 int fadeout=0;
 
 float increase = 0;

int movieFPS = 25;

float movieFrameTime = 1.0/movieFPS;

float movieTime;

 VideoM() {
  genera();
  println(numfiles);
  m = vv[idx];idx++; if (idx>=numfiles) idx=0;
  m.loop();
  m.speed(0);
  // m.volume(random(0.4,0.6));
  // m.speed(random(0.6,1.0));
  m.jump(random(m.duration()));  
  //m.play(); 
 }
 
 void genera(){
    x=int(random(50, width/2+width/4)); 
    y=int(random(50, height/2+height/4)); 
     w=int(random(300,720)); //horiz
    //w=int(random(200,620));
    h=int(w/1.88);
    alpha=int(random(100,255)); 
    alphai=0; 
 }
 
 void display(){
   tint(255, alphai);
   if (fadeout==0) {
     alphai++;  if (alphai>alpha) { alphai=alpha; } 
                   // else {this.w++; this.h=int(this.w/1.88);}
   } else {  alphai--; 
      // this.w++; this.h=int(this.w/1.88); 
       if (alphai<0) {alphai=0;fadeout=0;this.newvid();}
     }
   
       movieTime = (increase * movieFrameTime) % m.duration();
       if (movieTime>m.duration()) movieTime=0;
       m.jump(movieTime);
       m.read();

       if(m != null) image(m, x, y, w, h); 
       
        increase++;
 } 
 
 void cambiavid(){
   fadeout=1;
 }
 

void newvid() {  
  // m.pause();
   m=null;
   int j=int(random(numfiles));
   println("cambio: ", j);
   m = vv[j];
   m.loop();
   genera();
   m.jump(random(m.duration())); 
   // println(Thread.currentThread());
 }
}  

 
void newset(){
  for(int i = 0;i < numVideos;i++){
    println(i);
    v[i].newvid();
  }
}


void newvideo(){
  int i = int(random(numVideos));
  v[i].cambiavid();
}

  
void movieEvent(Movie m) {
  m.read();
}
  
boolean bStop,bColor=true;
boolean bSave=false,bVidO=false;
void keyPressed()
{
 // carga nuevo video
  
  if (key == 'v' || key == 'V') {
    background(0); loadvideos(); 
    }
    
  if (key == 'n' || key == 'N') {
    newset();
    }
  
}  

///////
String[] loadfiles(){

// The data path of the folder to look in (write your own)
java.io.File folder = new java.io.File(dataPath("")
);
 
// let's set a filter (which returns true if file's extension is .mp4)
java.io.FilenameFilter pngFilter = new java.io.FilenameFilter() {
  public boolean accept(File dir, String name) {
    return name.toLowerCase().endsWith(".mp4");
  }
};
 
// list the files in the data folder, passing the filter as parameter
String[] filenames = folder.list(pngFilter);
return(filenames);
}

Have you tried converting to MJPEG, or another codec where every frame is effectively a keyframe? MP4 is probably a bad option in the way you’re doing this due to how jump() works, particularly with the seek flags used by the library, which emphasises speed over accuracy.

You could try a different approach using a low value of speed instead of zero too. Even mixing that with triggering redraws from the video callback only when a new frame is available.

1 Like

Hi please excuse the rant here :slight_smile:
I have worked long and hard trying to get frame by frame, accurate, video handing in Processing and have posted on lots of threads trying to get it working, including example code and discussions about the video library and gstreamer.
I just logged in to see how things were going with the new video libraries and:

the forums have changed.
My login does not work
all my posts about video are gone.

This seems like a tremendous waste of resources - there seems to be no way to find all the old discussions we had about this issue, where there was useful stuff!

Rant over!

Hi chrios1001,

You can still access the old forum: https://forum.processing.org/two/

2 Likes

Hi @chrios1001!
Any luck locating those posts? That would be really helpful, I’m still stuck with this.
Thanks a lot for any help!