I decide to create this post to capture any future effort and to encourage future volunteers in case they are looking for a project to work on. I do not have enough experience with GStreamer and the requirements to bring the Video library up to speed to the latest GStreamer. @neilcsmith Do you mind commenting what are the skills required to work in this project? I understand from your previous post:
There is version 1.0.1 of the video library, released in 2015, which one would currently get when downloading the library through the Contribution Manger; this uses GStreamer version 0.10. Let’s call this the “current” version of the library. It works (unless it doesn’t).
Then, there are also more recent changes that were done to the main processing-video repository: these switch the library over to GStreamer version 1.x (using Neil’s bindings). Andres released a test version of the library containing these changes. Let’s call this the “next” version.
Now the issue with the “next” version, and that’s also why it was released as a “test” release only, was that it still has some features missing or not correctly working, compared to the “current” release. (The most complete list of these things is here.)
@kfrajer the direct link to the GStreamer 1.x bindings is https://github.com/gstreamer-java/gst1-java-core They were updated for PraxisLIVE in 2015, but not based on it. They are a fork of the 0.10 bindings already used in the Video library, of which I was one of the maintainers. There were a number of failed attempts to write a new binding from scratch for GStreamer 1.x, including one that was a Processing GSoC project. I needed GStreamer 1.x support for a paid commission, hence the initial fork, but they are now used in a variety of free and commercial projects, and a lot of other people have now contributed. I’m still the only maintainer though - something I’m quite happy to share!
@gohai did much of the work required to update the Video library to work with the gst1-java-core bindings. Although, as he says, there’s a few things missing. One other thing missing seems to be direct texture upload when using the OpenGL renderers? This is in the original video library. It’s quite a performance hit if it’s not there.
In my opinion, now is the time to rethink the Video library rather than just update it. There are a fair few issues with the API as far as I’m concerned - threading issues which really hamper performance, the device listing and lack of ability to really use the power of the underlying libraries by passing in custom pipelines, lack of direct GPU support. I’m not saying they should all be fixed immediately, but they should be considered in breaking the API now and not trying to make it backwards compatible. I also find the idea that a video is a PImage rather than has a sequence of PImages (frames) odd conceptually and potentially makes things more difficult than they need to be.
Despite using and supporting Processing, PraxisLIVE has always had its own video library. I’ve recently updated the PraxisLIVE video components in order to allow them to be (live) recoded by the user. In doing this I initially tried to replicate the current Video library API, and gave up - there was no easy way to do that without making it perform far worse - therefore IMO the current API is part of the problem.
