[JAVA] Remove background and record screen through chroma key

Processing Coding Questions
1

  1. Is it possible to remove the picture background(ex. green color) through the chroma key technique?
    (Is there a library or an easy workaround?) / (ex. Chromakey)
    fig

  2. I want to record the screen after removing the background. Is there any way?

2

Hi GWAK,
i don’t know library for this, i will go for glsl filter…
but, more easily, you can process each pixels, get the color, if it is “green” replace by a color or same pixel from another image,
like here for movie frames

i found too this in the forum: key-out-color-from-video
i hope it help…

1 Like
3

@th75

Thanks for the quick reply and kind explanation.

I have one question.
Link: Key out color from video

You can see the code on the site above.

PShader chroma;
PImage guy;
PImage beach;
void setup(){
 size(1080,720,P2D);
 chroma = loadShader("chroma.glsl");
 chroma.set("u_res",(float)width,(float)height);
 beach=loadImage("beach.jpg");
 guy=loadImage("fig.jpg"); // guy.jpg
 chroma.set("u_tex",guy);
}

void draw(){
  resetShader();
  image(beach,0,0);
  shader(chroma);
  
  float HUE, SAT, BRI,
        HUE_range,SAT_range,BRI_range,
        yikes;
  // hsb values
  // HUE=0.39; SAT=0.80; BRI=0.90;
  HUE=0.427; SAT=0.098; BRI=0.256;
  
  //hsb tolerances
  HUE_range=0.10; SAT_range=0.60; BRI_range=1.80;
  //experiment
  yikes=0.47;
  
  chroma.set("u_low",HUE-HUE_range,SAT-SAT_range,BRI-BRI_range);
  chroma.set("u_high",HUE+HUE_range,SAT+SAT_range,BRI+BRI_range);
  chroma.set("u_foo",yikes);
  rect(0,0,width,height);
}

HUE=0.39; SAT=0.80; BRI=0.90;

How do I get the above values? It looks like a green background color, but I can’t figure out how it came out like that.

4

hi,
colors can be coded in many different ways, (processing implement 2 methods RGB and HSB)
here its hsb for hue, saturation, brightness
in processing menu “tools”, “color selector”, you can choose a color and get the value in both RGB and HSB
i guess HSB is a good choice for chroma key, one value H will be stable for green and B will change if
there is shadows
from the picture you sent, h=108, s=23, b=73

(i’m not totally sure your question was about this, tell me…)

1 Like
5

@th75

  1. Information on the background value for the figure is
  • RGB : r=153, g=189, b=145
  • HSB: h=108, s=23, b=73
    You can get it through pictures.

  1. (Curious) Is there a way to convert RGB values to HSB values in Processing?

    • ex) input : rgb value, output : hsb value

  2. These are the results of actual testing.
    I divided 400.0. I don’t know why. But it seems to have come out somewhat.

PShader chroma;
PImage guy;
PImage beach;
void setup(){
 size(1080,720,P2D);
 chroma = loadShader("chroma.glsl");
 chroma.set("u_res",(float)width,(float)height);
 beach=loadImage("beach.jpg");
 guy=loadImage("fig.jpg");  // guy.jpg
 chroma.set("u_tex",guy);
}

void draw(){
  resetShader();
  image(beach,0,0);
  shader(chroma);
  
  float HUE,SAT,BRI,
        HUE_range,SAT_range,BRI_range,
        yikes;
  // hsb values
  // HUE=0.39;  SAT=0.80;  BRI=0.90;
  // HUE=0.6;  SAT=0.741;  BRI=0.568;
  
  float ref_f = 400.0;  
  HUE=108/ref_f; SAT=23/ref_f; BRI=73/ref_f;
  
  
  //hsb tolerances
  HUE_range=0.10;  SAT_range=0.60;  BRI_range=1.80;
  //experiment
  yikes=0.47;
  
  chroma.set("u_low",HUE-HUE_range,SAT-SAT_range,BRI-BRI_range);
  chroma.set("u_high",HUE+HUE_range,SAT+SAT_range,BRI+BRI_range);
  chroma.set("u_foo",yikes);
  rect(0,0,width,height);
}

fig1

fig2
fig21082Ă—747 261 KB

6

great,
yes from processing you can get HSB value:

color c=color(153,189,145);  // set a color in rgb mode
colorMode(HSB, 360, 100, 100);
println(hue(c),saturation(c),brightness(c)); //print HSB values

note here that we set range for each parameters: hue is 0-360 and others 0-100, and its arbitrary
on your code this is a glsl shader performing the color evaluation, so i guess there are all from 0.0 to 1.0 (it explain why you had to divide by 400)
so i will try with:

color c=color(153,189,145);
colorMode(HSB, 1, 1, 1);
println(hue(c),saturation(c),brightness(c));

0.3030 0.2328 0.7412

for the small artefact, i guess it s due to unnoticeable colors shift due to the image compression
for same reason (color change on a green background due to shadows, reflections…)
the code send to the shader both the reference color and a range, i will try to keep range for H low (stay green) and a bigger amount for brightness range to include this shades of green

last, it looks your pictures have 2 colors background, (this black layer around) for this you can do another pass with black as reference, black is not a color, so it will be any hue (hue=0 and hue range 1.0) any saturation but brightness 0 and brightness range 0.05 or 0.1

replacing black pixels will certainly unleash the dog :wink:

1 Like
7

@th75

Dear th75

I liked your explanation, so I could understand.
It’s understandable enough.
thank you.