Shader from shadertoy : buffer issue and lots of strange differences

Adrien · October 9, 2018, 1:44pm

Hi,

I’m relatively new with shaders.
I’m trying to port a shader from shadertoy (https://www.shadertoy.com/view/4dK3Ww) on processing 3.
It’s days I’m trying, still didn’t find the/a solution, because at the moment I just get a black screen…

Here is my code (simplified) :

Processing :

PShader firstShader;
PShader secondShader;  

PGraphics pg1;
PGraphics pg2;

void setup() {
  size(displayWidth, displayHeight, P2D);  

  pg1=createGraphics(width, height, P2D); 
  firstShader = loadShader("firstShader.glsl");
  firstShader.set("resolution", float(width), float(height));

  secondShader = loadShader("secondShader.glsl");
  secondShader.set("resolution", float(width), float(height));
}

void draw() {
  firstShader.set("time", millis()/1000.0);
  firstShader.set("iFrame", frameCount);
  firstShader.set("iChannel0", pg1); // I tried pg1.get() also, also result with just a black screen
  pg1.beginDraw();
  // pg1.background(0);
  pg1.shader(firstShader);
  pg1.rect(0, 0, width, height); 
  pg1.endDraw();  


  secondShader.set("iChannel0", pg1); //tried pg1.get() also
  shader(secondShader);
  rect(0, 0, width, height);
}

firstShader.glsl :

#ifdef GL_ES
precision highp float;
#endif

#define PROCESSING_COLOR_SHADER

uniform float time;
uniform int iFrame;
uniform vec2 resolution;
uniform sampler2D iChannel0;

void main( void ) {
    vec3 e = vec3(vec2(1.)/resolution.xy,0.);
    vec2 q = gl_FragCoord.xy/resolution.xy;
    vec4 c;
    c = texture(iChannel0, q);
    float p11 = c.y;
    float p10 = texture(iChannel0, q-e.zy).x;
    float p01 = texture(iChannel0, q-e.xz).x;
    float p21 = texture(iChannel0, q+e.xz).x;
    float p12 = texture(iChannel0, q+e.zy).x;
    float d = 0.;
    
    // Simulate rain drops
    float t = time*2.;
    vec2 pos = fract(floor(t)*vec2(0.456665,0.708618))*resolution.xy;
    float amp = 1.-step(.05,fract(t));
    d = -amp*smoothstep(2.5,.5,length(pos - gl_FragCoord.xy));

    // The actual propagation:
    d += -(p11-.5)*2. + (p10 + p01 + p21 + p12 - 2.);
    d *= .99; // dampening
    d *= float(iFrame>=2); // clear the buffer at iFrame < 2
    d = d*.5 + .5;
    
    // Put previous state as "y":
    //fragColor = vec4(d, c.x, 0, 0);
    gl_FragColor= vec4(d, c.x, 0, 0);
}

secondShader.glsl

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

// Type of shader expected by Processing
#define PROCESSING_COLOR_SHADER

uniform sampler2D iChannel0;
uniform vec2 resolution;
varying vec4 vertColor;
varying vec4 vertTexCoord;

void mainImage( out vec4 fragColor, in vec2 fragCoord );

void main() {
    mainImage(gl_FragColor,gl_FragCoord.xy);
}
void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
    vec2 q = fragCoord.xy/resolution.xy;
    float h = texture(iChannel0, q).x;
    float sh = 1.35 - h*2.;
    vec3 c =
    vec3(exp(pow(sh-.75,2.)*-10.),
         exp(pow(sh-.50,2.)*-20.),
         exp(pow(sh-.25,2.)*-10.));
    fragColor = vec4(c,1.);
}

With this code, I juste have a black screen, but I should see drops automatically appear (I delete some stuff - the iChannel1 image for the secondShader, the mouse interaction, but I check directly on shadertoy, it’s working, so it’s my processing port that have a pb). I tried copy pgGraphics on PImage, use ppixels in firstshader, etc., but still no results…

I checked other possibilities :
https://forum.processing.org/two/discussion/22683/ripples-shader-demo
It’s also a ripple shader, but it’s not the same Hugo Elias algorithm, and the ripples are not as thick, it’s not as “luiquid” as I want it to be.

I try the .get()/flag for initial texture as discuss here : https://forum.processing.org/two/discussion/23359/ppixels-feedback-and-colors-determining-the-color-at-a-location-of-texture but with no sucess

Perhaps is it a pb of PGraphics/shadertoy buffer resolution comptability (as discuss here : https://forum.processing.org/two/discussion/17629/how-to-get-round-using-16-bit-image-buffers-shadertoy-question). If so, do you have an idea how to make it works?

Any help is welcome!

Thanks,

A.

hamoid · October 9, 2018, 7:22pm

Hi! Try drawing something into pg1 right after creating it. You will notice it goes through the shaders, but it does not evolve. At least you will get past the black-background phase

Adrien · October 9, 2018, 8:41pm

Hi, thanks hamoid!

First phase pass

So it definitly go through the 1st shader, pass to the 2nd one. But it’s not the result I expect : I have the same result if I draw a PImage inside pg1 and after I pass pg1 to the second shader as if I directly assign the PImage to the iChannel0 of the second shader :

secondShader.set("iChannel0", pg1);
same “result” as
secondShader.set("iChannel0", myPImage);

So I guess (as I already expected, because I already made this kind of test), that is the code/feedback of the 1st shader that doesn’t work as expected.

But thanks anyway for your help!

A.

hamoid · October 10, 2018, 7:14am

What about trying with filter, like in https://github.com/processing/processing-docs/tree/master/content/examples/Topics/Shaders/BlurFilter ?

Or this one https://github.com/processing/processing-docs/tree/master/content/examples/Topics/Shaders/SepBlur actually makes two passes. It uses #define PROCESSING_TEXTURE_SHADER instead of #define PROCESSING_COLOR_SHADER.

I wonder if that makes any difference… I’m very curious about what’s the issue…

neilcsmith · October 10, 2018, 8:52am

You can’t do -

  firstShader.set("iChannel0", pg1);
  pg1.beginDraw();
  pg1.shader(firstShader);

You can’t use a texture as input to its own output - this is undefined. One option may be to not set iChannel0 at all, but rename it in the shader as ppixels. Processing supports providing the previous image as a texture, but only in the ppixels uniform.

You also need to either use pg1.background(0) or (preferably) pg1.blendMode(REPLACE) as the shader sets an alpha value of zero.

I’ve got the first shader running in a sketch inside PraxisLIVE now, but it has an odd drift towards the bottom left. I’m wondering if this is related to use of ppixels or something else.

neilcsmith · October 10, 2018, 9:51am

You need to use noSmooth() on the offscreen PGraphics or the coordinates in the shader are slightly off!

Adrien · October 10, 2018, 12:29pm

Hey guys!

Thanks to your help I was able to make it work!

@neilcsmith, it’s strange because i tried ppixels before but it was not working…Perhaps it’s the combination of background(0), blendMode(REPLACE) for the firstShader and initiation channel1 for the secondShader…I’ll look a bit deeper at this.

Just have to change
vec4 c = texture(iChannel1, fragCoord.xy*2./iChannelResolution[1].xy + grad.xy*.35);
with
vec4 c = texture(iChannel1, fragCoord.xy/(iResolution.xy) + grad.xy*.35);
in the secondShader.

The result is not exactly the same as in shadertoy (it’s looks a bit more “granular” than on shaderToy, less “thick”), but I can start work with that.

Here is the complete code for others in need

Processing

PShader firstShader;
PShader secondShader;  
PGraphics buffer1; 
PImage img; 

void setup() {
  size(600, 600, P2D);
  noSmooth();
  img=loadImage("rocks.jpg"); 

  buffer1 = createGraphics(width, height, P2D);
  buffer1.noSmooth(); 
  buffer1.beginDraw();
  buffer1.background(0); 
  buffer1.blendMode(REPLACE); 
  buffer1.endDraw();

  firstShader = loadShader("firstShader.glsl");
  firstShader.set("iResolution", float(width), float(height));

  secondShader = loadShader("secondShader.glsl");
  secondShader.set("iResolution", float(width), float(height));
  secondShader.set("iChannel1", img);
}  


void draw() {
  firstShader.set("iFrame", frameCount); 
  firstShader.set("iTime", millis()/1000.);
  if (mousePressed)  firstShader.set("mouse", (float)mouseX, (float)height-mouseY); // trick to get mouse click status 
  else firstShader.set("mouse", -1, (float)height-mouseY);

  buffer1.beginDraw();
  buffer1.background(0); 
  buffer1.shader(firstShader);
  buffer1.rect(0, 0, width, height);
  buffer1.endDraw();

  secondShader.set("iChannel0", buffer1); 
  shader(secondShader);
  rect(0, 0, width, height);
}

firstShader.glsl :


#ifdef GL_ES
precision highp float;
#endif

#define PROCESSING_COLOR_SHADER

uniform float iTime;
uniform int iFrame;
uniform vec2 iResolution;

uniform vec2 mouse;
vec4 iMouse = vec4(mouse,0.0,0.0); // zw would normally be the click status
uniform sampler2D ppixels;

void mainImage( out vec4 fragColor, in vec2 fragCoord );

void main() {
    mainImage(gl_FragColor,gl_FragCoord.xy);
}

void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
    vec3 e = vec3(vec2(1.)/iResolution.xy,0.);
    vec2 q = fragCoord.xy/iResolution.xy;
    
    vec4 c = texture(ppixels, q);
    
    float p11 = c.y;
    
    float p10 = texture(ppixels, q-e.zy).x;
    float p01 = texture(ppixels, q-e.xz).x;
    float p21 = texture(ppixels, q+e.xz).x;
    float p12 = texture(ppixels, q+e.zy).x;
    
    float d = 0.;
    
    if (iMouse.x > 0.)
    {
        // Mouse interaction:
        d = smoothstep(4.5,.5,length(iMouse.xy - fragCoord.xy));
    }
    else
    {
        // Simulate rain drops
        float t = iTime*2.;
        vec2 pos = fract(floor(t)*vec2(0.456665,0.708618))*iResolution.xy;
        float amp = 1.-step(.05,fract(t));
        d = -amp*smoothstep(2.5,.5,length(pos - fragCoord.xy));
    }
    
    // The actual propagation:
    d += -(p11-.5)*2. + (p10 + p01 + p21 + p12 - 2.);
    d *= .99; // dampening
    d *= float(iFrame>=2); // clear the buffer at iFrame < 2
    d = d*.5 + .5;
    
    // Put previous state as "y":
    fragColor = vec4(d, c.x, 0, 0);
}

secondShader.glsl

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

// Type of shader expected by Processing
#define PROCESSING_COLOR_SHADER

uniform sampler2D iChannel0;
uniform sampler2D iChannel1;
uniform vec2 iResolution;
uniform vec2 iResolutionChannel1;
varying vec4 vertColor;
varying vec4 vertTexCoord;


void mainImage( out vec4 fragColor, in vec2 fragCoord );

void main() {
    mainImage(gl_FragColor,gl_FragCoord.xy);
}


#define TEXTURED 1

void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
    vec2 q = fragCoord.xy/iResolution.xy;
    
#if TEXTURED == 1
    
    vec3 e = vec3(vec2(1.)/iResolution.xy,0.);
    float p10 = texture(iChannel0, q-e.zy).x;
    float p01 = texture(iChannel0, q-e.xz).x;
    float p21 = texture(iChannel0, q+e.xz).x;
    float p12 = texture(iChannel0, q+e.zy).x;
    
    // Totally fake displacement and shading:
    vec3 grad = normalize(vec3(p21 - p01, p12 - p10, 1.));
    vec4 c = texture(iChannel1, fragCoord.xy/(iResolution.xy) + grad.xy*.35);
    vec3 light = normalize(vec3(.2,-.5,.7));
    float diffuse = dot(grad,light);
    float spec = pow(max(0.,-reflect(light,grad).z),32.);
    fragColor = mix(c,vec4(.7,.8,1.,1.),.25)*max(diffuse,0.) + spec;
    
#else
    
    float h = texture(iChannel0, q).x;
    float sh = 1.35 - h*2.;
    vec3 c =
    vec3(exp(pow(sh-.75,2.)*-10.),
         exp(pow(sh-.50,2.)*-20.),
         exp(pow(sh-.25,2.)*-10.));
    fragColor = vec4(c,1.);
    
#endif
}

Thanks again for your help!

A.

neilcsmith · October 10, 2018, 1:42pm

Glad you got it working. I’ve been meaning to work on this for ages anyway so a good excuse! I have a few old (non-Processing) projects that used the software equivalent (eg. https://youtu.be/YNe03b5BxLM ) and have been wanting to update some with shaders.

You don’t need background(0) if you use blendMode(REPLACE), and it should be a little bit more efficient without.

hamoid · October 11, 2018, 9:27am

Could you please explain what’s the difference between using and not using REPLACE?

neilcsmith · October 11, 2018, 9:58am

blendMode(REPLACE); effectively switches off alpha blending. The shader writes zero to the alpha channel, which means that if you have blending on it doesn’t work correctly. You need the output pixel value to be exactly what the shader outputs.

You don’t need background(0) then. In fact, thinking about it I’m not sure that would work in Processing itself anyway (it probably only works in the PraxisLIVE Processing support because I extended P2D/P3D to support premultiplied alpha).

Adrien · October 11, 2018, 11:46am

Hi,

So, it’s working, but after a lots of testing, there are still a big number of problems :

First, If I do :

secondShader.set("iChannel1", img);

where img is a the bg PImage, this image is reverse and strech.
The reverse, I guess it’s something to do with P2D PGraphics (I already get this in other case, where the PGraphics is reversed).
I tried to display the img on an other PGraphics, it solved the reverse pb.
But the strech is still here. I guess it’s somehting to do with this line from the original Image shader (here)

  vec4 c = texture(iChannel1, fragCoord.xy*2./iChannelResolution[1].xy + grad.xy*.35);

That I change for this one in my secondShader :

vec4 c = texture(iChannel1, fragCoord.xy/(iResolution.xy) + grad.xy*.35);

I tried some divid, mult, etc, I can’t get my bgImg at the correct resolution…If somebody has clues about this…

Edit : It’s strange, I test it on a PC, and I don’t have the croping pb : the image completly appear…

Second :

The color are more “white” : if I use a full black as background in the second shader, that’s the color I actually have (it’s the same with a PImage, colors are less vivid)

Third :

As you can see, the ripples are less “thick” than the ones on the original shader, there are a bit more granular, you feel less than if you really have a liquid in front of you.

Edit : Making this capture, I realise something : when the program is running (fullscreen), the bg img is “cut” just behind the leave you can see on the left, on the middle. But when I make the capture (with saveFrame()), I got the full image. On new stuff to investigate…

Fourth, perhaps the strangest :

When you first create drop, after they start to propage, they are not round anymore, they transform in a kind of square, with edge moving in straight line (sure it’s more obvisous with movement, but you can have an idea with the next capture) :

What’s very strange, is after the first ripples propage in all the screen, if you make new ones, they doesn’t transform in straight edge square, because they disapear quicker…

I gonna change the title of the post, cause it’s not anymore a buffer issue (by the way, i tried to implement the 2-buffer version of the original shader, I have the same pbs that the ones described here).

Does someone have any clue about what’s going on?

Thanks,

A.

hamoid · October 11, 2018, 2:32pm

So if we had (1.0, 0.0, 0.0, 1.0) in a pixel (the existing value), and the fragment shader produces (0.0, 1.0, 0.0, 0.5) for that same pixel, by default we would get (0.5, 0.5, 0.0, 1.0) as a result, but with REPLACE we would actually get (0.0, 1.0, 0.0, 0.5) instead?

neilcsmith · October 11, 2018, 3:46pm

Exactly! It’s something to be wary of if the shader alpha values aren’t relevant, as in this case, or where you just want the output colour (eg. cutting out masks with fill(0,0); )

psamead · September 22, 2023, 7:11am

.background(0) will set the alpha channel in shader to 1.0 (most of the cases). .blendMode(REPLACE) sets the alpha channel to 0.0

Topic		Replies	Views
Some problem with shadertoy to processing！ Coding Questions	17	591	August 19, 2021
How to converting ShaderToy shader color channel to Processing correctly Coding Questions	2	338	March 9, 2024
Starting out with Shaders Processing	3	710	December 18, 2021
Need some help with glsl shaders Coding Questions	17	293	January 26, 2023
Trying to convert shadertoy 'ping pong feedback' buffer(?) to Processing Coding Questions	10	498	February 20, 2024

Shader from shadertoy : buffer issue and lots of strange differences

Related Topics