Video Capture with py5

Gallery
1

On linux this is a simple as putting the video jars (video.jar, jna.jar & gst1-java-core.jar) in a jar folder and in this example the glsl filter in a data folder (experience might be different Windows or Mac). Here I use my usb webcam for capture and apply my droste filter (to see unfiltered press your mouse).

import py5
Capture = py5.JClass('processing.video.Capture')

def settings():
    py5.size(1280, 960, py5.P2D)


def setup():
    global cam, my_filter
    # sketch_title 'Droste'
    my_filter = py5.load_shader('data/droste.glsl')
    my_filter.set('resolution', py5.width, py5.height)
    cam = Capture(py5.get_current_sketch(), "UVC Camera (046d:0825)")
    cam.start()


def draw():
    cam.read()
    py5.background(0)
    py5.image(cam, 0, 0, py5.width, py5.height)
    my_filter.set('frameCount', py5.frame_count)
    if (py5.is_mouse_pressed):
        return
    py5.apply_filter(my_filter)

py5.run_sketch()

Here’s the glsl filter created after a shadertoy filter.

// This implementation is based on GLSL code by ArKano22:
// http://www.gamedev.net/topic/590070-glsl-droste/
uniform int frameCount;
uniform sampler2D texture; // iChannel0 in Shadertoy
uniform vec2 resolution; // iResolution in Shadertoy
uniform int mode;

const float TWO_PI = 3.141592*2.0;
//ADJUSTABLE PARAMETERS:
const float Branches = 1.0;
const float scale = 0.4;
const float off = 0.6;
//Complex Math:
vec2 complexExp(in vec2 z){
	return vec2(exp(z.x)*cos(z.y),exp(z.x)*sin(z.y));
}
vec2 complexLog(in vec2 z){
	return vec2(log(length(z)), atan(z.y, z.x));
}
vec2 complexMult(in vec2 a,in vec2 b){
	return vec2(a.x*b.x - a.y*b.y, a.x*b.y + a.y*b.x);
}
float complexMag(in vec2 z){
	return length(z);
}
vec2 complexReciprocal(in vec2 a){ return vec2(a.x, -a.y) / dot(a, a); }
vec2 complexDiv(in vec2 a,in vec2 b){
	return complexMult(a, complexReciprocal(b));
}
vec2 complexPower(in vec2 a, in vec2 b){
	return complexExp( complexMult(b,complexLog(a))  );
}
//Misc Functions:
float nearestPower(in float a, in float base){
	return pow(base,  ceil(  log(abs(a))/log(base)  )-1.0 );
}
float map(float value, float istart, float istop, float ostart, float ostop) {
	   return ostart + (ostop - ostart) * ((value - istart) / (istop - istart));
}

void main( void ){

	//SHIFT AND SCALE COORDINATES
	vec2 uv=gl_FragCoord.xy/resolution.xy - off;

	//ESCHER GRID TRANSFORM:
	float factor = pow(1.0/scale,Branches);
	uv= complexPower(uv, complexDiv(vec2( log(factor) ,TWO_PI), vec2(0.0,TWO_PI) ) );

	//RECTANGULAR DROSTE EFFECT:
	float FT = float(frameCount % 100 / 100.0);
	FT = log(FT+1.)/log(2.);
	uv *= 1.0+FT*(scale-1.0);

  float npower = max(nearestPower(uv.x,scale),nearestPower(uv.y,scale));
	uv.x = map(uv.x,-npower,npower,-1.0,1.0);
	uv.y = map(uv.y,-npower,npower,-1.0,1.0);

	//UNDO SHIFT AND SCALE:
	gl_FragColor =  texture(texture,uv*off+vec2(off));
}

Performance is somewhat similar to JRubyArt @hx2a @villares @tabreturn might be interested.

4 Likes
2

Very useful! Thanks @monkstone

1 Like
3

Excellent work!

Are you using the latest version of py5? If so, you can import the Capture class with from processing.video import Capture. The latest version activates some jpype magic to make importing Java classes just like importing from Python.

1 Like
4

@hx2A In my hands the following works from atom ide:-

from processing.core import PVector

But that’s not too useful as it does support +, - operators as would be anticipated for python (I had some idea to translate recent sketch by @villares which I may yet do by adding support for operators). However

from processing.video import Capture

Only works when sketch is run from the sketch folder.

python droste.py

I sometimes encounter similar problems with my ruby-processing projects and java, usually solved using absolute paths if only under the hood.

5

I agree, it isn’t useful to use the Java PVector class like that. I’d want that operator support also.

Hmmm, where is the video.jar file located? That class is separate from the core.jar stuff. If you have it in a jars directory, py5 will only find it automatically if that directory’s parent is the current working directory (os.getcwd()) when the sketch is run. What I’m trying to say there is that when you run a sketch, py5 gets the current working directory with os.getcwd(), looks for a jars subdirectory, and adds any jar found in there to the classpath. Therefore, if you are running some code and you aren’t running it from the same folder each time, you might get different results because it will look in different locations for the jars directory.

To address this, you can use py5_tools.add_jars() or py5_tools.add_classpath() to add jars explicitly. Or, explicitly set the current working directory to the parent with os.chdir().

6

For the moment the jars folder solution has appeal over approach used by vanilla processing (a dedicated libraries folder) especially when there are sometimes conflicts between libraries using the same jars. In ruby-processing I have created a library_loader that can either load jars from a local folder, or a default location. It is complicated and involves dynamic CLASS_PATH loading but the user does not see this. Eventually since there a lot of useful libraries out there py5 might do well to create a specific folder for at least some key libraries like video. PS: its only the

from processing.video import Capture

that caused the issue running from atom, was working fine until tried this more pythonesque convenience method, which would be completely negated if I then had to use py5_tools explicitly.

1 Like
7

Interesting idea. Maybe py5 could look for an environment variable called PY5_JARS and load jar files from there, in addition to the jars subdirectory? Implementing this is straightforward and clearly useful.

1 Like