How to import processing4-video into a java project?

Hi,

I’m trying to get processing working within the context of a java project. So far I’ve succesfully managed to pull in processing-core using maven, but I’m struggling with processing-video. I am a total noob when it comes to the whole java ecosystem, so I might just be fumbling some basics here…

My pom.xml looks like this:

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.roosnaflak</groupId>
    <artifactId>ProcessingTest</artifactId>
    <version>1.0-SNAPSHOT</version>

    <name>ProcessingTest</name>
    <!-- FIXME change it to the project's website -->
    <url>http://www.example.com</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        </properties>

    <repositories>
        <repository>
            <id>jitpack.io</id>
            <url>https://jitpack.io</url>
            </repository>
        </repositories>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
            </dependency>
        <dependency>
            <groupId>com.github.micycle1</groupId>
            <artifactId>processing-core-4</artifactId>
            <version>4.3</version>
            </dependency>
        </dependencies>

    <build>
        <pluginManagement><!-- lock down plugins versions to avoid using Maven defaults (may be moved to parent pom) -->
            <plugins>
                <!-- clean lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#clean_Lifecycle -->
                <plugin>
                    <artifactId>maven-clean-plugin</artifactId>
                    <version>3.1.0</version>
                    </plugin>
                <!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->
                <plugin>
                    <artifactId>maven-resources-plugin</artifactId>
                    <version>3.0.2</version>
                    </plugin>
                <plugin>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.8.0</version>
                    </plugin>
                <plugin>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.22.1</version>
                    </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-shade-plugin</artifactId>
                    <version>3.5.2</version>
                    <executions>
                        <execution>
                            <phase>package</phase>
                            <goals>
                                <goal>shade</goal>
                                </goals>
                            <configuration>
                                <transformers>
                                    <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                        <mainClass>org.sonatype.haven.HavenCli</mainClass>
                                        </transformer>
                                    </transformers>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-install-plugin</artifactId>
                    <version>3.1.1</version>
                    <configuration>
                        <groupId>org.processing</groupId>
                        <artifactId>video</artifactId>
                        <version>2.2.2</version>
                        <packaging>jar</packaging>
                        <file>dependencies/video.jar</file>
                        <generatePom>true</generatePom>
                        </configuration>
                    <executions>
                        <execution>
                            <id>install-jar-lib</id>
                            <goals>
                                <goal>install-file</goal>
                                </goals>
                            <phase>validate</phase>
                            </execution>
                        </executions>
                    </plugin>
                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>exec-maven-plugin</artifactId>
                    <version>3.2.0</version>
                    <executions>
                        <execution>
                            <goals>
                                <goal>exec</goal>
                                </goals>
                            </execution>
                        </executions>
                    <configuration>
                        <executable>maven</executable>
                        <mainClass>com.roosnaflak.App</mainClass>
                        </configuration>
                    </plugin>
                <plugin>
                    <artifactId>maven-jar-plugin</artifactId>
                    <version>3.0.2</version>
                    <configuration>
                        <archive>
                            <manifest>
                                <addClasspath>true</addClasspath>
                                <!-- here we specify that we want to use the main method within the App class -->
                                <mainClass>com.roosnaflak.App</mainClass>
                                </manifest>
                            </archive>
                        </configuration>
                    </plugin>
                <plugin>
                    <artifactId>maven-deploy-plugin</artifactId>
                    <version>2.8.2</version>
                    </plugin>
                <!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->
                <plugin>
                    <artifactId>maven-site-plugin</artifactId>
                    <version>3.7.1</version>
                    </plugin>
                <plugin>
                    <artifactId>maven-project-info-reports-plugin</artifactId>
                    <version>3.0.0</version>
                    </plugin>
                </plugins>
            </pluginManagement>
        </build>
    </project>

I’m symlinking the video jar from Processing’s sketchbook/library/video/library/video.jar into the dependencies directory in my project. I suspect that this is one of the places where things go wrong…

The main file I’m trying to compile looks like this:

package com.roosnaflak;

import processing.core.PApplet;
import processing.video.*;

public class App extends PApplet
{
    Movie movie;

    public void settings() {
        size(640, 480);
        background(0);
        movie = new Movie(this, "/home/kf/Videos/100-sketches-May/hands.mp4");
        movie.loop();
    }

    public void movieEvent(Movie m) {
        m.read();
    }

    public void draw(){
        // background(0);
        // ellipse(mouseX, mouseY, 20, 20);
        image(movie, 0, 0, width, height);
    }

    public static void main(String[] args){
        String[] processingArgs = {"MySketch"};
        App mySketch = new App();
        PApplet.runSketch(processingArgs, mySketch);
    }
}

which gives me this error:

~/processing-java/ProcessingTest is 📦 v1.0-SNAPSHOT via ☕ v21.0.2 
❯ mvn compile          
[MVNVM] Using maven: 3.9.6
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------< com.roosnaflak:ProcessingTest >--------------------
[INFO] Building ProcessingTest 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- install:3.1.1:install-file (install-jar-lib) @ ProcessingTest ---
[INFO] pom.xml not found in video.jar
[INFO] Installing /home/kf/processing-java/ProcessingTest/dependencies/video.jar to /home/kf/.m2/repository/org/processing/video/2.2.2/video-2.2.2
.jar
[INFO] Installing /tmp/mvninstall7078025395252386902.pom to /home/kf/.m2/repository/org/processing/video/2.2.2/video-2.2.2.pom
[INFO] 
[INFO] --- resources:3.0.2:resources (default-resources) @ ProcessingTest ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/kf/processing-java/ProcessingTest/src/main/resources
[INFO] 
[INFO] --- compiler:3.8.0:compile (default-compile) @ ProcessingTest ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /home/kf/processing-java/ProcessingTest/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/kf/processing-java/ProcessingTest/src/main/java/com/roosnaflak/App.java:[8,5] cannot find symbol
  symbol:   class Movie
  location: class com.roosnaflak.App
[ERROR] /home/kf/processing-java/ProcessingTest/src/main/java/com/roosnaflak/App.java:[17,28] cannot find symbol
  symbol:   class Movie
  location: class com.roosnaflak.App
[ERROR] /home/kf/processing-java/ProcessingTest/src/main/java/com/roosnaflak/App.java:[4,1] package processing.video does not exist
[ERROR] /home/kf/processing-java/ProcessingTest/src/main/java/com/roosnaflak/App.java:[13,21] cannot find symbol
  symbol:   class Movie
  location: class com.roosnaflak.App
[INFO] 4 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.892 s
[INFO] Finished at: 2024-02-28T07:55:48+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project ProcessingTest: Compilati
on failure: Compilation failure: 
[ERROR] /home/kf/processing-java/ProcessingTest/src/main/java/com/roosnaflak/App.java:[8,5] cannot find symbol
[ERROR]   symbol:   class Movie
[ERROR]   location: class com.roosnaflak.App
[ERROR] /home/kf/processing-java/ProcessingTest/src/main/java/com/roosnaflak/App.java:[17,28] cannot find symbol
[ERROR]   symbol:   class Movie
[ERROR]   location: class com.roosnaflak.App
[ERROR] /home/kf/processing-java/ProcessingTest/src/main/java/com/roosnaflak/App.java:[4,1] package processing.video does not exist
[ERROR] /home/kf/processing-java/ProcessingTest/src/main/java/com/roosnaflak/App.java:[13,21] cannot find symbol
[ERROR]   symbol:   class Movie
[ERROR]   location: class com.roosnaflak.App
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Running this on Arch Linux, java-21-openjdk.

Would greatly appreciate any help!

OK, getting a bit further here… Edited the pom.xml to include video as a dependency:

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
            </dependency>
        <dependency>
            <groupId>com.github.micycle1</groupId>
            <artifactId>processing-core-4</artifactId>
            <version>4.3</version>
            </dependency>
        <dependency>
            <groupId>org.processing</groupId>
        <artifactId>video</artifactId>
            <version>2.2.2</version>
            </dependency>
        </dependencies>

Now, however, I get this when building:

❯ mvn exec:java        
[MVNVM] Using maven: 3.9.6
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------< com.roosnaflak:ProcessingTest >--------------------
[INFO] Building ProcessingTest 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- exec:3.2.0:java (default-cli) @ ProcessingTest ---
[WARNING] 
java.lang.NullPointerException: Cannot invoke "processing.core.PGraphics.background(int)" because "this.g" is null
    at processing.core.PApplet.background (PApplet.java:14445)
    at com.roosnaflak.App.settings (App.java:12)
    at processing.core.PApplet.handleSettings (PApplet.java:853)
    at processing.core.PApplet.runSketch (PApplet.java:10113)
    at com.roosnaflak.App.main (App.java:30)
    at org.codehaus.mojo.exec.ExecJavaMojo.doMain (ExecJavaMojo.java:385)
    at org.codehaus.mojo.exec.ExecJavaMojo.doExec (ExecJavaMojo.java:374)
    at org.codehaus.mojo.exec.ExecJavaMojo.lambda$execute$0 (ExecJavaMojo.java:296)
    at java.lang.Thread.run (Thread.java:1583)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.528 s
[INFO] Finished at: 2024-02-28T09:25:33+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.2.0:java (default-cli) on project ProcessingTest: An exception occurred while
 executing the Java class. Cannot invoke "processing.core.PGraphics.background(int)" because "this.g" is null -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

I tried removing the background call, and got this:

❮ mvn exec:java
[MVNVM] Using maven: 3.9.6
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------< com.roosnaflak:ProcessingTest >--------------------
[INFO] Building ProcessingTest 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- exec:3.2.0:java (default-cli) @ ProcessingTest ---
[WARNING] 
java.lang.NoClassDefFoundError: org/freedesktop/gstreamer/Element
    at com.roosnaflak.App.settings (App.java:13)
    at processing.core.PApplet.handleSettings (PApplet.java:853)
    at processing.core.PApplet.runSketch (PApplet.java:10113)
    at com.roosnaflak.App.main (App.java:30)
    at org.codehaus.mojo.exec.ExecJavaMojo.doMain (ExecJavaMojo.java:385)
    at org.codehaus.mojo.exec.ExecJavaMojo.doExec (ExecJavaMojo.java:374)
    at org.codehaus.mojo.exec.ExecJavaMojo.lambda$execute$0 (ExecJavaMojo.java:296)
    at java.lang.Thread.run (Thread.java:1583)
Caused by: java.lang.ClassNotFoundException: org.freedesktop.gstreamer.Element
    at org.codehaus.mojo.exec.URLClassLoaderBuilder$ExecJavaClassLoader.loadClass (URLClassLoaderBuilder.java:181)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:526)
    at com.roosnaflak.App.settings (App.java:13)
    at processing.core.PApplet.handleSettings (PApplet.java:853)
    at processing.core.PApplet.runSketch (PApplet.java:10113)
    at com.roosnaflak.App.main (App.java:30)
    at org.codehaus.mojo.exec.ExecJavaMojo.doMain (ExecJavaMojo.java:385)
    at org.codehaus.mojo.exec.ExecJavaMojo.doExec (ExecJavaMojo.java:374)
    at org.codehaus.mojo.exec.ExecJavaMojo.lambda$execute$0 (ExecJavaMojo.java:296)
    at java.lang.Thread.run (Thread.java:1583)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.573 s
[INFO] Finished at: 2024-02-28T09:28:56+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.2.0:java (default-cli) on project ProcessingTest: An exception occurred while
 executing the Java class. org/freedesktop/gstreamer/Element: org.freedesktop.gstreamer.Element -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

When removing all references to Movie the project compiles and runs fine.

One more attempt: noticing that processing-video has gst1-java-core as a dependecy I added it to the project.

Getting this now:

[INFO] --- exec:3.2.0:java (default-cli) @ ProcessingTest ---
can't load library gstrtspserver-1.0 (gstrtspserver-1.0|libgstrtspserver-1.0|libgstrtspserver-1.0-0) with -Djna.library.path=/usr/lib64. Last erro
r:java.lang.UnsatisfiedLinkError: Unable to load library 'gstrtspserver-1.0':
libgstrtspserver-1.0.so: cannot open shared object file: No such file or directory
libgstrtspserver-1.0.so: cannot open shared object file: No such file or directory

<--- snip--->
can't load library gstvalidate-default-overrides-1.0 (gstvalidate-default-overrides-1.0|libgstvalidate-default-overrides-1.0|libgstvalidate-defaul
t-overrides-1.0-0) with -Djna.library.path=/usr/lib64. Last error:java.lang.UnsatisfiedLinkError: Unable to load library 'gstvalidate-default-over
rides-1.0':
libgstvalidate-default-overrides-1.0.so: cannot open shared object file: No such file or directory
libgstvalidate-default-overrides-1.0.so: cannot open shared object file: No such file or directory
Native library (linux-x86-64/libgstvalidate-default-overrides-1.0.so) not found in resource path ([file:/home/kf/processing-java/ProcessingTest/ta

<--- snip--->
can't load library avresample (avresample|libavresample|libavresample-0) with -Djna.library.path=/usr/lib64. Last error:java.lang.UnsatisfiedLinkE
rror: Unable to load library 'avresample':
libavresample.so: cannot open shared object file: No such file or directory
libavresample.so: cannot open shared object file: No such file or directory
Native library (linux-x86-64/libavresample.so) not found in resource path ([file:/home/kf/processing-java/ProcessingTest/target/classes/, file:/ho

<---snip-->

(process:280349): libsoup-ERROR **: 09:37:52.901: libsoup3 symbols detected. Using libsoup2 and libsoup3 in the same process is not supported.

You’ll have to work around what ridiculous incantation of settings is necessary to get Processing Video to just load GStreamer from the normal path (I’m assuming via jitpack it doesn’t package the native libs, and you don’t want that anyway). See processing-video/src/processing/video/Video.java at main · processing/processing-video · GitHub

This whole library loading thing in Processing Video is a mess! And I say that as the lead developer and maintainer of gst1-java-core.

Phew… not sure I have the capacity to do that… Thanks for pointing it out, though! (oh, and the libsoup2/3 incompatibility issue also shows up in the Processing IDE on Linux these days, making it impossible to run video this platform until the issue has been resolved.)

Yes, it’s annoying as hell! GStreamer and gst1-java-core work well together on Linux without this nonsense. I don’t see an easy way of just getting it to trust the system path, and the hardcoded locations look wrong.

If you want to combine video and Processing, you could also look at PraxisLIVE. That’s the project that gst1-java-core was first built for. Unfortunately the Processing GStreamer integration in there is not available as a standalone library at the moment.

PraxisLive looks pretty cool! Still seems like there’s just a bit too many obstacles in the way of getting a good java/processing/linux/neovim workflow going for the moment. Once the processing-video bug gets sorted out I might give it another try. In the meantime I’ll slink back to my familiar openFrameworks setup (which is not without its share of gstreamer-related hickups, unfortunately…)

1 Like