Trying to convert shadertoy 'ping pong feedback' buffer(?) to Processing

Hi my friends,
I am trying to convert this shadertoy effect to the porcessing with shaders. In the shadertoy solution, it appears the Buffer A tap loads
a channel with a buffer b and the Buffer C tap loads channel with a buffer a, which they seems to interlinked to each other. I did a bit research with my limited knowledge this seems to be the topics on ping pong feedback, frame buffer object and texture swapping technique etc., which I am not very sure how to implement in my case. Sorry, if I am putting the wrong term here.
My conversion method borrows some steps from RipplesP5, but the result seem quite different.
my conversion on processing:


the shadertoy:

The processing one seems the pixel is not able to recover to its previous state.

Does anyone have some clue on solve this?

Here is my code:
Processing:

PShader BufferAShader;
PShader BufferBShader;
PShader BindShader;
PGraphics buffer1, buffer2, buffer3;
int looping = 0;

void setup()
{
  size(800, 600, P3D);
  
  buffer1 = createGraphics(width, height, P2D);
  buffer1.noSmooth();
  buffer1.beginDraw();
  buffer1.blendMode(REPLACE);
  buffer1.endDraw();
  
  buffer2 = createGraphics(width, height, P2D);
  buffer2.noSmooth();
  buffer2.beginDraw();
  buffer2.blendMode(REPLACE);
  buffer2.endDraw();
  
  buffer3 = createGraphics(width, height, P2D);
  buffer3.noSmooth();
  buffer3.beginDraw();
  buffer3.blendMode(REPLACE);
  buffer3.endDraw();
  
  
  BufferAShader = loadShader("FiniteWater_BufferA.glsl");
  BufferAShader.set("iResolution", float(width), float(height));

  BufferBShader = loadShader("FiniteWater_BufferB.glsl");
  BufferBShader.set("iResolution", float(width), float(height));
  
  BindShader = loadShader("FiniteWater_Binding.glsl");
  BindShader.set("iResolution", float(width), float(height));
}


void draw()
{  
  PGraphics buffTemp = null;
  PGraphics buffLast = null;
  PGraphics buffLast2 = null;
  
  int loopSize = 3;
  
  PGraphics[] buffs = {buffer1, buffer2, buffer3};
  
  buffTemp = buffs[looping%loopSize];
  buffLast = buffs[(looping+1)%loopSize];
  buffLast2 = buffs[(looping+2)%loopSize];
  
  looping++;

  BufferAShader.set("iTime", millis()/1000.);

  if(mousePressed)
  {
    BufferAShader.set("mouseInput", (float)mouseX, (float)height-mouseY);
  }
  else 
  {
    BufferAShader.set("mouseInput", -1, (float)height-mouseY);
  }


  BufferAShader.set("iChannel0", buffLast); 

  buffLast.beginDraw();
  buffLast.shader(BufferAShader); 
  buffLast.rect(0, 0, width, height);
  buffLast.endDraw();
  
  
  BufferBShader.set("iChannel0", buffLast);
  buffLast2.beginDraw();
  buffLast2.shader(BufferBShader);
  buffLast2.rect(0, 0, width, height);
  buffLast2.endDraw();


  BindShader.set("iChannel0", buffLast2);
  //BindShader.set("iChannel0", buffLast);
  buffTemp.beginDraw();
  buffTemp.shader(BindShader);
  buffTemp.rect(0, 0, width, height);
  buffTemp.endDraw();
  
  image(buffTemp, 0, 0);

FiniteWater_BufferA.glsl:

#ifdef GL_ES
precision highp float;
#endif

#define PROCESSING_COLOR_SHADER

uniform float iTime;
uniform vec2 iResolution;

uniform vec2 mouseInput;
//vec4 iMouse = vec4(mouseInput, 0.0, 0.0);
vec4 iMouse = vec4(mouseInput, mouseInput);
uniform sampler2D iChannel0;


void main()
{
  // Mouse excitation
  vec2 r = gl_FragCoord.xy - iMouse.xy;
  float d = 0.001 * dot(r, r);
  if(iMouse.z > 0.0 && d < 0.05)
  {
    gl_FragColor = vec4(0.0, 0.2, 0.0, 0.0);
    return;
  }
  
  // Periodic excitation
  r = gl_FragCoord.xy - vec2(50, 70);
  d = 0.001 * dot(r, r);
  if(mod(iTime, 0.5) < 0.1 && d < 0.05)
  {
    gl_FragColor = vec4(0.0, 0.1, 0.0, 0.0);
    return;  
  }
  
  // just copy
  vec2 uv = gl_FragCoord.xy / iResolution.xy;
  gl_FragColor = texture2D(iChannel0, uv);

}

FiniteWater_BufferB.glsl:

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

// Type of shader expected by Processing
#define PROCESSING_COLOR_SHADER

uniform vec2 iResolution;
uniform sampler2D iChannel0;

void main()
{
  float dx = 1.0 / iResolution.x;
  float dy = 1.0 / iResolution.y;
  vec2 uv = gl_FragCoord.xy / iResolution.xy;
  
  vec2 udu = texture2D(iChannel0, uv).xy;
  // old elevation
  float u = udu.x;
  // old velocity
  float du = udu.y;
  
  // Finite differences
  float ux  = texture2D(iChannel0, vec2(uv.x + dx, uv.y)).x;
  float umx = texture2D(iChannel0, vec2(uv.x - dx, uv.y)).x;
  float uy  = texture2D(iChannel0, vec2(uv.x, uv.y + dy)).x;
  float umy = texture2D(iChannel0, vec2(uv.x, uv.y - dy)).x;
  
  // new elevation
  float nu = u + du + 0.5*(umx+ux+umy+uy-4.0*u);
  nu = 0.99*nu;
  
  // store elevation and velocity
  gl_FragColor = vec4(nu, nu-u, 0.0, 0.0);
}

FiniteWater_Binding.glsl:

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

#define PROCESSING_COLOR_SHADER

uniform vec2 iResolution;
uniform sampler2D iChannel0;

void main()
{
  vec2 uv = gl_FragCoord.xy / iResolution.xy;
  gl_FragColor = vec4(2.0*texture2D(iChannel0, uv).x + 0.5);
}

It’s pretty easy to double-buffer even using a single PGraphics. The problem, however, is with precision.

That shadertoy stores a value in the red channel and a velocity in the green. The velocity has to be able to go negative if you want the value to both rise and fall. Based on this, I assume that shadertoy uses full 32-bit floats for the buffer/channels. Processing, however, only supports 8-bit per channel images which can only hold values from 0 to 1, i.e., not negative.

Furthermore, Processing always multiplies the rgb by the alpha channel, so to get data out of a shader, you have to always set alpha to 1. This implies that you can get at most 24 bits of data per pixel out of a shader.

Here’s my version of that shadertoy. The velocity is centered around 0.5, so values below are negative and above are positive. Likewise, the “elevation” converges on 0.5 rather than on 0.0. If you scribble enthusiastically with the mouse, you can sometimes trigger a degenerate state with a checkerboard of full white / full black pixels that I’m not sure how to clean up.

Also, because Processing uses an upside-down coordinate system, you have to flip the y with 1.-y.

PShader updateShader;
PShader displayShader;
PGraphics buffer;

void setup() {
  size( 1200, 900, P2D );
  noSmooth();
  buffer = createGraphics(width, height, P2D);
  buffer.noSmooth();
  buffer.beginDraw();
  buffer.background(127);
  buffer.endDraw();
  
  updateShader = new PShader( this, vertSrc, updateFragSrc );
  displayShader = new PShader( this, vertSrc, displayFragSrc );
}

void draw() {
  buffer.beginDraw();
  updateShader.set( "pstate", buffer.get() );
  buffer.filter( updateShader );
  //buffer.rect( 0, 0, width, height );
  buffer.endDraw();

  if( mousePressed ) {
    buffer.beginDraw();
    buffer.resetShader();
    buffer.fill( 0, 200, 0 );
    buffer.circle( mouseX, mouseY, 10 );
    buffer.endDraw();
  }
  displayShader.set( "data", buffer.get() );
  shader( displayShader );
  rect( 0, 0, width, height );
  resetShader();
}


String[] vertSrc = { """
#version 330 core
uniform mat4 transformMatrix;
in vec4 position;
void main() {
  gl_Position = transformMatrix * position;
}
""" };


String[] updateFragSrc = {"""
#version 330 core
uniform sampler2D pstate;
uniform vec2 resolution;       // provided by Processing

out vec4 fragColor;

void main() {
  float dx = 1.0/resolution.x;
  float dy = 1.0/resolution.y;
  vec2 uv = gl_FragCoord.xy / resolution;
  uv.y = 1.0 - uv.y;

  vec3 udu = texture(pstate, uv).rgb;
  // old elevation
  float u = udu.r;
  // old velociy
  float du = udu.g;
  
  // Finite differences
  float ux = texture(pstate, vec2(uv.x+dx, uv.y)).x;
  float umx = texture(pstate, vec2(uv.x-dx, uv.y)).x;
  float uy = texture(pstate, vec2(uv.x, uv.y+dy)).x;
  float umy = texture(pstate, vec2(uv.x, uv.y-dy)).x;

  // new elevation
  float nu = u + (du-0.5) + 0.5*(umx+ux+umy+uy-4.0*u);
  nu = 0.97*(nu-0.5)+0.5;
  
  // store elevation and velocity
  fragColor = vec4(clamp(nu, 0, 1 ),clamp(nu-u+0.5, 0, 1),0.0,1.0);
}
"""};


String[] displayFragSrc = {"""
#version 330 core
uniform vec2 resolution;    // provided by Processing
uniform sampler2D data;

out vec4 fragColor;

void main() {
  vec2 p = gl_FragCoord.xy / resolution;
  p.y = 1.0 - p.y;
  fragColor = vec4( vec3(texture(data, p).x), 1. );
}
"""};
2 Likes

Hi scudly,
Thanks for your reply and giving a such detail explanation, much appreciated. I am trying to run your code in the Processing, but it keeps giving error message on missing blanket “}”.


I don’t know how to use this type of string array: String displayFragSrc = {“”"

“”"};
Could you give some more explanation?

Those are multi-line strings that require the Java version used in Processing 4.0 or higher.

1 Like

I see. Sorry, I should mention this before, I am still using Processing 3.5 . Is there a similar syntax in 3.5 version?

No, in 3.5, use your previous method of loading the shaders from external files.

1 Like

We can concatenate each string line too, as in this code:

  text("διχθαδίας κῆρας φερέμεν θανάτοιο τέλος δέ.\n" +
    "Бу́ря мгло́ю не́бо кро́ет, Ви́хри сне́жные крутя́;\n" + 
    "富士の風や扇にのせて江戸土産\n" + "Ako sa voláš", width>>1, height>>1);

https://Studio.ProcessingTogether.com/sp/pad/export/ro.9hlrt8LyDd9uA

2 Likes

Alternatively we can place each line inside a String[] array, and then join() them together:


“Unicode_Letters_Array.pde”:

/** 
 * Unicode Letters Array (v1.0.2)
 * GoToLoop (2024/Feb/11)
 *
 * https://Discourse.Processing.org/t/
 * trying-to-convert-shadertoy-ping-pong-feedback-buffer-to-processing/43863/10
 */

static final String[] MULTI_LINES = {
  "διχθαδίας κῆρας φερέμεν θανάτοιο τέλος δέ.", 
  "Бу́ря мгло́ю не́бо кро́ет, Ви́хри сне́жные крутя́;", 
  "富士の風や扇にのせて江戸土産", 
  "Ako sa voláš"
};

static final String FULL_TEXT = join(MULTI_LINES, '\n');

void setup() {
  size(600, 200);
  noLoop();

  fill(#FFFF00);
  textAlign(CENTER, CENTER);

  if (1/2 != 1/2.) textFont(createFont("", 24, true));
  else textSize(24);

  println(FULL_TEXT);
}

void draw() {
  background(#0000FF);
  text(FULL_TEXT, width >> 1, height >> 1);
}

“index.html”:

<!DOCTYPE html>
<meta charset=utf-8>

<script defer src=https://Unpkg.com/processing-js></script>
<canvas data-processing-sources=Unicode_Letters_Array.pde></canvas>

“start.bat”:

set /a port = %random% * 40000 / 32768 + 10000
start servor . index.html %port%
start http://localhost:%port%

2 Likes

Hi @GoToLoop,
Thanks for your helpful information. I will definitely try it. :slight_smile:

1 Like

Hi @scudly ,

Thanks for your helpful reply. I really learn a lot. You mention about the precision difference of the Shadertoy and Processing, which, I think, is critical to the conversion of the Shadertoy shader to Processing. I have experienced similar case and posed in here.

I assume that shadertoy uses full 32-bit floats for the buffer/channels.

Does this mean the temporary texture map to swap and be written between different passes is a 32-bit-float map, which it is something like this in some game engine such as the Unity (just to get an analogy), the Processing only supports 8-bit-float map as the temporary map to let processing’s shader to write ? but the processing temporary texture map has alpha channel, so this is a different topic?

I am not quite getting this:

That shadertoy stores a value in the red channel and a velocity in the green.

Processing always multiplies the rgb by the alpha channel, so to get data out of a shader, you have to always set alpha to 1. This implies that you can get at most 24 bits of data per pixel out of a shader.

I am a bit confused, would you give a bit more information on this or some readings? :slight_smile:

In addition, I got the Processing4 version and run the code. It seems to get a very low frame rate on my poor 5 years old surface laptop. :joy:Is this causing by the new string array feature?

This is rendering on1200x900.

This is rendering on 800x600:

This is rendering on a decent desktop, but the frame rate seems to get limiting on 30 fps:

And the spreading ripples will turn into rectangle shape from its initial circle shape , so is it the processing’s problem on executing the shader?


Inspired by your code, I am trying to run my code on the processing 3.5 with a little bit modification ( just trying not to modify much on the original shadertoy code) but it seems to get some channel issues. Is there a way to fix? :slight_smile:



The frame rate seems to be very good with the shader file loading method.

Here is my code:
the Processing code:

PShader BufferAShader;
PShader BufferBShader;
PShader BindShader;
PGraphics buffer1, buffer2, buffer3;
int looping = 0;

void setup()
{
  size(800, 600, P3D);
  
  buffer1 = createGraphics(width, height, P2D);
  buffer1.noSmooth();
  buffer1.beginDraw();
  buffer1.blendMode(REPLACE);
  buffer1.endDraw();
  
  buffer2 = createGraphics(width, height, P2D);
  buffer2.noSmooth();
  buffer2.beginDraw();
  buffer2.blendMode(REPLACE);
  buffer2.endDraw();
  
  buffer3 = createGraphics(width, height, P2D);
  buffer3.noSmooth();
  buffer3.beginDraw();
  buffer3.blendMode(REPLACE);
  buffer3.endDraw();
  
  
  BufferAShader = loadShader("FiniteWater_BufferA.glsl");
  BufferAShader.set("iResolution", float(width), float(height));

  BufferBShader = loadShader("FiniteWater_BufferB.glsl");
  BufferBShader.set("iResolution", float(width), float(height));
  
  BindShader = loadShader("FiniteWater_Binding.glsl");
  BindShader.set("iResolution", float(width), float(height));
}


void draw()
{  
  PGraphics buffTemp = null;
  PGraphics buffLast = null;
  PGraphics buffLast2 = null;
  
  int loopSize = 3;
  
  PGraphics[] buffs = {buffer1, buffer2, buffer3};
  
  buffTemp = buffs[looping%loopSize];
  buffLast = buffs[(looping+1)%loopSize];
  buffLast2 = buffs[(looping+2)%loopSize];
  
  looping++;
  
  //println((looping+1)%loopSize);

  BufferAShader.set("iTime", millis()/1000.);

  if(mousePressed)
  {
    BufferAShader.set("mouseInput", (float)mouseX, (float)height-mouseY);
  }
  else 
  {
    BufferAShader.set("mouseInput", -1, (float)height-mouseY);
  }
  
  BufferAShader.set("iChannel0", buffLast); 
  buffLast.beginDraw();
  buffLast.shader(BufferAShader); 
  buffLast.rect(0, 0, width, height);
  buffLast.endDraw();
  
  
  BufferBShader.set("iChannel0", buffLast);
  buffLast2.beginDraw();
  buffLast2.shader(BufferBShader);
  buffLast2.rect(0, 0, width, height);
  buffLast2.endDraw();


  BindShader.set("iChannel0", buffLast2);
  //BindShader.set("iChannel0", buffLast);
  buffTemp.beginDraw();
  buffTemp.shader(BindShader);
  buffTemp.rect(0, 0, width, height);
  buffTemp.endDraw();
  
  image(buffTemp, 0, 0);

}

FiniteWater_Binding.glsl:

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

#define PROCESSING_COLOR_SHADER

uniform vec2 iResolution;
uniform sampler2D iChannel0;

void main()
{
  vec2 uv = gl_FragCoord.xy / iResolution.xy;
  //gl_FragColor = vec4(2.0*texture2D(iChannel0, uv).x + 0.5);
  gl_FragColor = vec4(clamp(texture2D(iChannel0, uv).x, 0.0, .998));
}

FiniteWater_BufferA.glsl:

#ifdef GL_ES
precision highp float;
#endif

#define PROCESSING_COLOR_SHADER

uniform float iTime;
uniform vec2 iResolution;

uniform vec2 mouseInput;
//vec4 iMouse = vec4(mouseInput, 0.0, 0.0);
vec4 iMouse = vec4(mouseInput, mouseInput);
uniform sampler2D iChannel0;


void main()
{
  // Mouse excitation
  vec2 r = gl_FragCoord.xy - iMouse.xy;
  float d = 0.001 * dot(r, r);
  if(iMouse.z > 0.0 && d < 0.05)
  {
    gl_FragColor = vec4(0.0, 0.2, 0.0, 0.0);
    return;
  }
  
  // Periodic excitation
  r = gl_FragCoord.xy - vec2(50, 70);
  d = 0.001 * dot(r, r);
  if(mod(iTime, 0.5) < 0.1 && d < 0.05)
  {
    gl_FragColor = vec4(0.0, 0.1, 0.0, 0.0);
    return;  
  }
  
  // just copy
  vec2 uv = gl_FragCoord.xy / iResolution.xy;
  //gl_FragColor = texture2D(iChannel0, uv);
  gl_FragColor = clamp(texture2D(iChannel0, uv), 0.0, 0.998);

}

FiniteWater_BufferB.glsl:

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

// Type of shader expected by Processing
#define PROCESSING_COLOR_SHADER

uniform vec2 iResolution;
uniform sampler2D iChannel0;

void main()
{
  float dx = 1.0 / iResolution.x;
  float dy = 1.0 / iResolution.y;
  vec2 uv = gl_FragCoord.xy / iResolution.xy;
  
  vec2 udu = texture2D(iChannel0, uv).xy;
  // old elevation
  float u = udu.x;
  // old velocity
  float du = udu.y;
  
  // Finite differences
  float ux  = texture2D(iChannel0, vec2(uv.x + dx, uv.y)).x;
  float umx = texture2D(iChannel0, vec2(uv.x - dx, uv.y)).x;
  float uy  = texture2D(iChannel0, vec2(uv.x, uv.y + dy)).x;
  float umy = texture2D(iChannel0, vec2(uv.x, uv.y - dy)).x;
  
  // new elevation
  float nu = u + du + 0.5*(umx+ux+umy+uy-4.0*u);
  nu = 0.99*nu;

  
  nu = clamp(nu, 0.0, 1.0);
  
  // store elevation and velocity
  gl_FragColor = vec4((1.0-nu)*0.998, (1.0-(nu-u))*0.998, 0.0, 1.0);
  //gl_FragColor = vec4((1.0-nu)*0.998, (1.0-(nu-u))*0.998, 0.0, 0.0);
  //gl_FragColor = vec4(nu, nu-u, 0.0, 0.0);



}

When you run a shader, the output is sent to a framebuffer. OpenGL (Processing) and OpenGL ES (web browsers / shadertoy) support many different pixel formats for framebuffers including both float and integers at different bit depths from 8 to 32 bits per color channel.

Shadertoy uses RGBA 8-bit color for its main buffer, since that creates the image that appears in the browser window, but, as far as I can tell, uses 32-bit floats per channel for each of its other buffers.

Processing, however, only supports RGBA 8-bit color framebuffers. From the shader, you write them as floats from 0.0 to 1.0, but in Processing, you access them from the pixels[] array where each is stored as a single integer as ARGB. If you use the default display, the RGB values will be multiplied by the alpha and alpha will be replaced by 255. In a PGraphics, though, it looks like you can get all four values out, so I was wrong about only being able to read 24 bits from the RGB channels. So, if you want higher precision, you could store the wave “value” in AR and the “velocity” in GB wtih 16 bits of precision for each.

There is no reason I’m aware of that Processing 4 should be slower than 3. Is it possible that it’s not using your GPU for some reason? On laptops, sometimes Windows will only use the GPU for apps if you explicitly tell it to. Perhaps in the past, you told it to use the GPU for 3.5 and need to again for 4. I’m running on a 9 year old linux desktop with an nVidia GPU and didn’t have any slowdowns when switching to version 4 last year.

3 Likes