Reducing latency when capturing and displaying an RTSP stream

Processing Libraries
1

I am trying to display the video being streamed by an IP camera through the RTSP protocol.
After finding many outdated examples (which did rely on the old codeanticode.GSvideo library, which is not available for Processing 3), I did find a sample of code which is working with my IP camera:

import processing.video.*;
Movie movie;

void setup() {
  size(2048, 1024, P2D);
  Movie.supportedProtocols[0] = "rtsp";
  println(Movie.supportedProtocols[0]);
  try {
    movie = new Movie(this, "rtsp://192.168.1.168:8557/PSIA/Streaming/channels/2?videoCodecType=H.264");
    movie.play();
  } 
  catch (Exception e) {
    println(e);
  }
}

void movieEvent(Movie m) {
  m.read();
}

void draw() {
  background(0);
  image(movie, 0, 0, width, height);
}

Only problem is the terrible 3-second latency. Which comes definitely from Processing, not from the camera, as using other tools for capturing the RTSP stream (for example PraxisLive) I get a much smaller latency, something as 100-200 ms, not 3 seconds!.
So Processing is doing some large buffering of the incoming RTSP stream.
Does anyone know how to reduce this buffering, or avoiding it entirely?
As I did in PraxisLive with the following setting for the device:

rtspsrc latency=0 buffer-mode=0 udp-buffer-size=400000 location="rtsp://192.168.1.168:8557/PSIA/Streaming/channels/2?videoCodecType=H.264" ! rtph264depay ! decodebin

Thanks in advance for any answer…
Sincerely yours
Angelo Farina
University of Parma, ITALY

2

I was searching so long for an example on how to view a h.264 webcam stream in processing and finally found your example. Thank you very much Angelo!
This also works with AXIS cameras if you set it up correctly (no user and password needed for live stream).

movie = new Movie(this, "rtsp://192.168.178.143/axis-media/media.amp?videocodec=h264&resolution=1920x1080");

I have encountered the same latency problem as you have discribed and tried to search through the video library for any functions to change the buffersize, but without any luck or results.
Honestly im not good in understanding these libraries.
It seems that youve got lower latencies by reducing the ethernet buffer and now im wondering if this is also possible in the processing library?

Side Note: i know this thread is half a year old but i didn´t want to open another one.

3

No it isn’t! Only just seen this post. The buffering comes from GStreamer defaults. For some reason I can’t get the Capture pipeline: url to work with an rtsp pipeline at first attempt - I think there’s a fix missing in the released Video library at the moment that would allow this.

However, it’s possible to set the latency on the rtspsrc inside the Movie’s PlayBin. This would be a little easier if Video had a recent version of our bindings, but this should work -

import processing.video.*;
import org.freedesktop.gstreamer.*;

Movie movie;

void setup() {
  size(2048, 1024, P2D);
  Movie.supportedProtocols[0] = "rtsp";
  println(Movie.supportedProtocols[0]);
  try {
    movie = new Movie(this, "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov");
    movie.playbin.connect(new ElementAdded());
    movie.play();
  } 
  catch (Exception e) {
    println(e);
  }
}

void movieEvent(Movie m) {
  m.read();
}

void draw() {
  background(0);
  image(movie, 0, 0, width, height);
}

class ElementAdded implements Bin.ELEMENT_ADDED {
   public void elementAdded(Bin bin, Element element) {
     println("Element added : " + element);
     ElementFactory factory = element.getFactory();
     if (factory == null) {
        return;
     }
     if (element instanceof Bin) {
       ((Bin) element).connect(this); 
     }
     if ("rtspsrc".equals(factory.getName())) {
        println("Setting latency on " + element);
        element.set("latency", 100); 
     }
  }
}

Obviously, keep using PraxisLIVE instead though - the integration with GStreamer and Processing is better! :smiley:

2 Likes
4

This works very well, thank you so much!
Unfortunately i cant set your answer to solution.

I had searched through the GStreamer info site aswell and also came to conclusion that the latency has to be set somewhere, but i just could not find out how.

1 Like
5

Great! Glad it’s working. There is a mailing list for the Java GStreamer bindings too - you can always ask questions there as well - https://groups.google.com/g/gstreamer-java It’ll still probably be me answering them! :smiley:

1 Like
6

@neilcsmith

hello. I read your post and leave it like this.
I added the ‘jar’ file to the ‘code’ folder, but the error is not resolved.
I don’t know why.

import processing.video.*;
import org.freedesktop.gstreamer.*;

Movie movie;

String url1 = "rtsp://xxxx.smp";

void setup() {
  size(2048, 1024, P2D);
  Movie.supportedProtocols[0] = "rtsp";
  println(Movie.supportedProtocols[0]);
  try {
    movie = new Movie(this, url1);
    movie.playbin.connect(new ElementAdded());
    movie.play();
  } 
  catch (Exception e) {
    println(e);
  }
}

void movieEvent(Movie m) {
  m.read();
}

void draw() {
  background(0);
  image(movie, 0, 0, width, height);
}

class ElementAdded implements Bin.ELEMENT_ADDED {
   public void elementAdded(Bin bin, Element element) {
     println("Element added : " + element);
     ElementFactory factory = element.getFactory();
     if (factory == null) {
        return;
     }
     if (element instanceof Bin) {
       ((Bin) element).connect(this); 
     }
     if ("rtspsrc".equals(factory.getName())) {
        println("Setting latency on " + element);
        element.set("latency", 100); 
     }
  }
}

Error message.

The method connect(PlayBin2.ABOUT_TO_FINISH) in the type PlayBin2 is not applicable for the arguments (movie_test.ElementAdded)

7

sorry.
It was a library version problem of movie.
have a good day.

1 Like