Monitoring process of ffmpeg in processing

Processing Coding Questions
1

Hi all,
I’m using ffmpeg to output audio tracks from videos using the example by Hamoid found here but slightly modified (code below).

What I would like to do is to pipe the progress from ffmpeg back into processing so I can visualise it.
Is there any way to do this?

Any help much appreciated.
Thanks.

//ExtractAudioWithFFmpeg by Hamoid:
//https://github.com/hamoid/Fun-Programming/tree/master/processing/ideas/2017/05/extractAudioWithFfmpeg

import java.io.InputStreamReader;
import java.io.FilenameFilter;
import java.util.*;
import java.util.Arrays;

String path, fileName, ffmpegPath;
boolean saved = false;
void setup() { 
  // Create data folder if it doesn't exist
  (new File(dataPath(""))).mkdirs();
  String temp[] = loadStrings("ffmpegPath/ffmpegPath.txt");
  ffmpegPath = temp[1];
}

void saveAudio() { 
  saved = true;
  try {
    String[] commands = {
      ffmpegPath, //use location from videoExport settings.json
      "-i", 
      path, //use location from folderSelected
      dataPath(fileName + "_audio.mp3") //use fileName from fileSelected dialog
    };
    Process proc = exec(commands);
  } 
  catch(Exception e) {
    println(e);
  }
}

void fileSelected(File selection) { 
  if (selection == null) {
    println("Window was closed or the user hit cancel.");
  } else {
    path = selection.getAbsolutePath();
    println(path);
    fileName = path;
    fileName = cleanFileName(fileName);
    println(fileName);
  }
}

String cleanFileName(String unclean) { 
  //https://forum.processing.org/two/discussion/16801/is-there-a-way-to-check-what-system-a-processing-sketch-is-running-on
  String[] list;
  char delim = '\\';
  list = split(unclean, delim);
  int arrayPos = list.length-1;
  return list[arrayPos];
}

void draw() { 
  if (path != null && fileName != null && !saved) saveAudio();
  if (saved && frameCount % 60 == 0) exit();
}

void mousePressed() {
  println("mouse button pressed: " + mouseButton);
  selectInput("select movie to extract audio from:", "fileSelected");
}
2

Would I need to so something with stdout or stdin?
This seems to suggest that stderr might be the way forward but the method they use to get the error doesn’t exist in processing.
Has anyone done this or have any knowledge they can share?

any and all advice appreciated,
thanks,

Toast

1 Like
3

So upon second look at Hamoids’ example I saw that he was getting the stdInput and stdError using the code below.
I’ve tried it and i get nothing at all once it prints the “# command output”, not even empty lines.
I’m pretty sure this is what i need, it just doesn’t work.

Any ideas?
@hamoid, are you kicking about?

Thanks,
Toast

BufferedReader stdInput = new BufferedReader(new 
      InputStreamReader(proc.getInputStream()));

    BufferedReader stdError = new BufferedReader(new 
      InputStreamReader(proc.getErrorStream()));

    println("# Command output");
    String s = null;
    while ((s = stdInput.readLine()) != null) {
      println(s);
    }

    println("# Command errors");
    while ((s = stdError.readLine()) != null) {
      println(s);
    }
  }
1 Like
4

One pattern is logging. Have your process (ffmpeg, whatever) write and flush to a log file. Have your Processing sketch periodically check the file and react accordingly.

What does the ffmpeg output look like, and how are you using it? Is this a progress bar, or are you parsing output messages…?

1 Like
5

Hi! To show ffmpeg progress you can do this:

  • When you run ffmpeg, add -report as an argument. This will create a .log file in the same folder, the name in program-YYYYMMDD-HHMMSS.log format.
  • Your program could scan for the most recent .log file and read that file every second.
  • In the beginning of that file you could parse the Duration of the file
Input #0, avi, from 'a_movie.avi':
  Duration: 00:48:17.00, start: 0.000000, bitrate: 2028 kb/s
    Stream #0:0, 41, 1/25: Video: mpeg4, 1 reference frame (DX50 / 0x30355844), yuv420p(left), 688x384 [SAR 1:1 DAR 43:24], 1828 kb/s, 25 fps, 25 tbr, 25 tbn, 30k tbc
    Stream #0:1, 49, 1/24000: Audio: ac3 ([0] [0][0] / 0x2000), 48000 Hz, stereo, fltp, 192 kb/s
  • Below in the log file you will find lines like this
size=    2048kB time=00:02:17.68 bitrate= 121.8kbits/s speed=39.3x

you can compare that Time to the Duration to calculate the percentage. Then show that percentage in Processing.

To have a predictable log filename (instead of the auto generated with date and time) you can use environment variables or redirecting the output: https://stackoverflow.com/questions/11241878/ffmpeg-set-report-log-filename

See the ffmpeg docs (search for -report): https://ffmpeg.org/ffmpeg.html

Maybe find ideas at: https://stackoverflow.com/questions/747982/can-ffmpeg-show-a-progress-bar

2 Likes
6

Hi Hamoid,
Thanks for the detailed response, really useful.

I tested the log output in powershell (i’m using win 10, should have stated that earlier) and it works perfectly.
However, putting those commands into processing, it only outputs a zero byte mp3 and no log file.

I’m not sure why this doesn’t work as it is the same order as I tested in powershell.

This is how the saveAudio() method now looks:

void saveAudio() { 
  saved = true;

  String[] commands = {
    ffmpegPath, 
    "-i", 
    path, 
    dataPath(fileName + "_audio.mp3"),  
    "2> ", 
    fileName + "_Log.txt" 
  };
  Process proc = exec(commands);
}
2 Likes
7

You’re welcome :slight_smile:

Could you maybe print commands to verify that everything looks correct? Could it be that the paths need to be absolute because the current working directory is different when running Processing?

You could also use ProcessBuilder instead of exec which is what I use in the VideoExportLibrary. There I use redirectOutput with ffmpeg and it logs stuff to a file. See https://github.com/hamoid/video_export_processing/blob/master/src/com/hamoid/VideoExport.java#L632

2 Likes
8

Hamoid you champion!
All it took was using processBuilder.

Thanks so much for your help,

Toast

For completeness sake, here’s the saveAudio() method in it’s newly minted state:

void saveAudio() {
  saved = true;

  String[] commands = {
    ffmpegPath,
    "-i", 
    path,
    dataPath(fileName + "_audio.mp3")
  };

  ProcessBuilder processBuilder = new ProcessBuilder(commands);
  Process process;
  File ffmpegOutputLog;
  processBuilder.redirectErrorStream(true);
  ffmpegOutputLog = new File(dataPath(fileName+"_ffmpegLog.txt"));
  processBuilder.redirectOutput(ffmpegOutputLog);
  processBuilder.redirectInput(ProcessBuilder.Redirect.PIPE);
  try {
    process = processBuilder.start();
  } 
  catch (Exception e) {
    e.printStackTrace();
    println(ffmpegOutputLog);
  }
}
3 Likes