I would like to layer multiple videos that I’ve made in processing sketches on top of each other. They all have black backgrounds, which is necessary to create the initial sketch but when i play them back on top of each other I would like the black background to be removed so the multiple videos can mixed with out dimming and having there backgrounds interfere with content.
I have researched and tried playing the movies with alpha channels but haven’t had much luck on either end.
In my current sketch I’m using loadpixels to turn the black pixels transparent. This works but since its transparent the previous frames aren’t cleared or drawn over. I tried calling background in draw to clear the frame but it doesn’t work at all. Any idea on why that is? Something to do with being a movie?
If you have any better ideas on how i could approach this let me know as i feel there are probably better ways to go about this.
thanks
import processing.video.*;
Movie myMovie;
Movie myMovie2;
void setup() {
size(500, 500);
frameRate(10);
background(255,0,0);
//myMovie = new Movie(this, "particleCircle.mov");
// myMovie.loop();
myMovie2 = new Movie(this, "conexported.mov");
myMovie2.loop();
}
void draw() {
background(255,60,200);
image(myMovie2, 0, 0);
loadPixels();
myMovie2.loadPixels();
for (int x = 0; x < 500; x++){
for (int y = 0; y < 500; y++){
int loc = y * 500 + x;
float brightness =brightness(pixels[loc]);
if (brightness == 0) { //if pixel is black
pixels[loc] = color(0,0); //make it transparent
}
}
updatePixels();
}
}
void movieEvent(Movie m) {
m.read();
}
It’s rare that a pixel is completely dark at one time (brightness == 0) due to video compression. You can try adding a threshold to check if brightness is below a certain value:
final float threshold = 30;
if (brightness < threshold) { // if pixel is *almost* black
...
Play around the threshold value and see which value gives you the best result.
You could also want to look into filters that make the video cleaner to reduce noise.
The videos in this case are processing sketches I’ve created and exported as videos, so they do have black backgrounds.
With my current sketch I was thinking I could clear the canvas by drawing a background between every frame of the video but that doesn’t appear to be possible.
I exported the videos with an alpha channel and can overlay them perfectly in premiere. Is there anyway to do the same in processing without masking?
I might suggest uploading three example frames of your content, and one example frame of what successful compositing should look like (e.g. from premiere).
Keep in mind that the sketch canvas itself does not support alpha – only PImage and PGraphics support alpha. For order of operations compositing onto the canvas, have you tried using either blendMode() or blend()?
Alternately, you can mask-on-the-fly by copying non-black pixels, as @FSXAC suggest. Since your blacks are pure, the easiest and fastest way to do this is to check for equality before copying. However, keep in mind that video frame compression algorithms can surprise you – you think you wrote pure black in, but that isn’t what you get back out.
Here is a very simple example sketch that loops the “transit” video on top of a demo video from the internet. Both videos loop at different lengths, so the mask interaction changes. Press space bar to switch between two modes – exact, and brightnesss threshold.
/**
* VideoMask
* 2019-03 Jeremy Douglass - Processing 3.4
* Press spacebar to switch between exact and threshold modes.
* - 640x360 short video file from www.sample-videos.com/index.php#sample-mp4-video
* - 640x360 short video file from "Loop" Processing example sketch
* discourse.processing.org/t/playing-multiple-videos-with-transparent-backgrounds/8990/3
**/
import processing.video.*;
Movie[] movs;
color maskColor = color(0);
float maskBrightness = 48;
boolean testMode;
void setup() {
size(640, 360);
movs = new Movie[]{
new Movie(this, "SampleVideo_640x360_1mb.mp4"),
new Movie(this, "transit.mov"),
};
for (Movie mov : movs) {
mov.play();
mov.loop();
}
}
void draw() {
background(0);
int pixel;
loadPixels();
for (int m = 0; m < movs.length; m++) {
movs[m].loadPixels();
for (int i=0; i < pixels.length; i++) { // assumes movie frame size == canvas size
pixel = movs[m].pixels[i];
if (testPixel(pixel)) {
pixels[i] = pixel;
}
}
}
updatePixels();
}
void keyReleased(){
testMode = !testMode;
}
void movieEvent(Movie m) {
m.read();
}
boolean testPixel(int pixel){
if(testMode){ // exact match only
return pixel!=maskColor;
}
return brightness(pixel) > maskBrightness; // loose match
}
You might also be interested in an older discussion of cross-fading video content:
