Hi there,
I’m struggling with something that is supposed to be quite simple, any form of help is welcome!
A horizon line draws a mountain range shape itself depending on the amplitude of the ambiant sound. There’s also a video looping in the back, the whole thing is in 4K (3840x2160).
import processing.sound.*;
import processing.video.*;
Movie myMovie;
float t = 0;
float noise_y = 0;
AudioIn input;
Amplitude analyzer;
float prevVol = 0;
PGraphics p;
void setup() {
size(3840,2160,P2D);
myMovie = new Movie(this, "skieurOptimal.mp4");
myMovie.loop();
p = createGraphics(width, height / 2);
input = new AudioIn(this, 0);
input.start();
analyzer = new Amplitude(this);
analyzer.input(input);
}
void draw() {
image(myMovie, 0, 0);
float vol = analyzer.analyze();
vol = lerp(prevVol, vol, 0.045);
prevVol = vol;
p.beginDraw();
p.stroke(255);
p.strokeWeight(1.85);
for (float x = 0; x < width; x++) {
float volume = map(vol, 0, 0.2, 0, height / 2);
float noise = noise(x / 300, noise_y) ;
float y = map(noise, 0, 1.5, height / 2, 0);
p.point(x, y - volume);
}
p.endDraw();
image(p, 0, 0);
noise_y = noise_y - 0.0085;
fadePG();
println(frameRate + " FPS");
}
void movieEvent(Movie m) {
m.read();
}
void fadePG() {
for (int x = 0; x < p.width; x++) {
for (int y = 0; y < p.height; y++) {
int index = x + y * p.width;
float alpha = alpha(p.pixels[index]);
if (alpha > 0) {
p.pixels[index] = color(red(p.pixels[index]), green(p.pixels[index]), blue(p.pixels[index]), alpha - 5);
}
}
}
p.updatePixels();
}
The whole thing works, but not great… It’s stuck at 10fps while it should run around 25 fps (same as the video background). I realized that what kills the framerate is the void fadePG() ; which allows the “horizon line” to keep fading away over and over after a few seconds and works fine with the fact that this has to play as a continuous loop.
If anyone knows how to display the same kind of effect while keeping a framerate around 25 fps that’d be just amazing !
Don’t call color(). You don’t need to extract red, green and blue to merge them together again to just change the alpha value. You can use bit shifting to extract the alpha, change it, and put it back into the color. See below.
Don’t do two loops (x, y). You can just do one loop to iterate over the pixel indices.
Finally, the really fast way is to write a shader to do this, because that can operate in parallel in thousands of pixels (instead of doing it serially one after the other).
This is how you can reduce 5 units the alpha of one pixel on a PGraphics:
int argb = pg.pixels[i];
int a = argb >> 24 & 0xFF; // get the alpha as an integer
a = max(0, a - 5); // decrease alpha by 5 while avoiding negative values
pg.pixels[i] = (a << 24) | (argb & 0x00FFFFFF);
// AAAAAAAA RRRRRRRR GGGGGGGG BBBBBBBB <- a color as 4 bytes
The way it works: an int color is actually 4 bytes (8 bits each, so 32 bits in total). The first byte is alpha, then red, green and blue. So it’s 8 bits alpha, 8 bits red, 8 bits green and 8 bits blue. If we shift those 32 bits 24 bits to the right, we get rid of R G and B and what remains is alpha. We need to do & 0xFF to convert the signed byte into an unsigned byte.
Then we decrease the alpha, and finally we reassemble the number. We use | to combine two parts: the new alpha with the old rgb color. I’ll start with the second: we extract the RGB part from the original argb by doing an & operation with 0x00FFFFFF, which discards the original alpha. Notice the 00. So far we have a color without alpha. For the alpha, we take the newly calculated alpha (a) and shift it 24 bits to the left, so it’s back on the first byte. Then we combine that alpha and the rgb to have a full color.
It’s less intuitive, but should be much faster because red(), green(), blue(), color() do a lot of calculations internally, which is fine for a few pixels, but for 8 million pixels it takes a toll.
With a shader and a dedicated GPU it should definitely run at 60fps or more at 4K.
Yes, never mix P2D with use of pixels if you want good framerate. Even downloading the pixels from the GPU is slow.
However, you shouldn’t need a shader for this. You can do it just with blendMode and a full screen rectangle. Try something like this where you begin drawing on p.
Not if you use the MULTIPLY blend mode as I outlined IIRC. But try it. I’m saying this from memory right now I’ve done this various ways for different purposes so can suggest / test something different later if need be.
That’s really kind of you
So I tried it, and unfortunately it adds a black rectangle in front of my video, and doesn’t show any “mountain line”. I’m I doing this wrong ?
Well, you’ve removed the line where you set the stroke colour for your points. But it still shouldn’t cause a black rectangle! I’ll have a look ASAP. Blending is a little screwy in Processing.
Hi again hamoid, and thanks for sharing this with me.
I have to admit I’m struggling to understand even though you explained it, and even more to make my code work using your method. Could you please take more of your time to share it again with me, once integrated in the totality of the code ?
Cannot for the life of me get the multiplying alpha channel to work correctly in Processing - need to look into that further. The alternative of using additive blending on the output works for what you want I think -
So at some point Processing has been “fixed” to add rather than multiply the alpha channel. The previous behaviour didn’t work anything like multiply blending should, but was useful for masking like this. Now it fulfils neither function correctly!
Assuming you change to create a P2D offscreen, this also works -
Please don’t be sorry, I’m the one asking for advices ! Aaaand it’s working perfectly, 60 FPS !! Thank you so much for taking the time to help me, I really appreciate it !
void fadePG() {
p.beginDraw();
p.loadPixels();
for (int i = 0; i < p.pixels.length; i++) {
int argb = p.pixels[i];
int a = argb >> 24 & 0xFF;
a = max(0, a - 5);
p.pixels[i] = (a << 24) | (argb & 0x00FFFFFF);
}
p.updatePixels();
p.endDraw();
}
Should be faster than the original approach (because it removes one for loop, red(), green(), blue() and color()), but not faster than drawing a rect covering the screen, which does blending on the GPU.
I’ve done this various ways for different purposes so can suggest / test something different later if need be.
