Transparent/translucent window (drawing graphics on desktop). How to capture mouse and keyboard?

Processing Coding Questions
1

Hello forum :slight_smile:

I found a fairly simple way to use processing to draw graphics directly to your screen, to make it seem like your window is transparent or translucent, or that your graphics background has full transparency.

The problem though is user input no longer works :frowning:

keyPressed() and mousePressed() don’t get called, and mouseX and Y are always 0.

You can use import java.awt.MouseInfo; and MouseInfo.getPointerInfo().getLocation().x to somewhat solve the problem, but I was hoping to find a way that fairly seamlessly integrated with processing.

I wonder if anyone has a clever way to solve this? I think plenty of people would be interested in way to draw graphics without an opaque background.

Here’s my code, hope it helps if you we’re looking for way to do what is does:

//Import Libraries
import processing.awt.PSurfaceAWT;
import javax.swing.JPanel;
import javax.swing.JFrame;
import java.awt.Color;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;

//Declare Gobal Variables
PGraphics pg;
JFrame frame;
JPanel panel;

void setup()
{

size(200, 200); 

frame = (JFrame)((PSurfaceAWT.SmoothCanvas) getSurface().getNative()).getFrame();
frame.removeNotify();
frame.setUndecorated(true);
frame.setLayout(null);
frame.addNotify();

pg = createGraphics(width, height);

JPanel panel = new JPanel() {
 @Override
   protected void paintComponent(Graphics graphics) {
   if (graphics instanceof Graphics2D) {
     Graphics2D g2d = (Graphics2D) graphics;
     g2d.drawImage(pg.image, 0, 0, null);
   }
 }
};

frame.setContentPane(panel);
}

void draw()
{
if(mouseX!=0) {
 println(mouseX);
}
pg.beginDraw();
pg.background(0, 0);
pg.fill(0, 153, 204, frameCount%255);  
pg.ellipse(frameCount%width, frameCount%height, 60, 60);
pg.endDraw();
frame.setBackground(new Color(0, 0, 0, 0));
}

void mousePressed() {
println(mouseX);  
}

void keyPressed() {
println(key); 

}
2 Likes
2

Adding a listener is the way to go. This below hopefully get you started. Idea from this. Now, you need to see if you can hook this up to P3 keyboard/mouse events. Check the mouseEvent header in the Library Basics as it could provide some guidance.

Kf

  panel.addMouseListener(new MouseAdapter() {
    public void mouseClicked(MouseEvent e) {
      System.out.println("Clicked!");
    }
  }
  );

  frame.setContentPane(panel);  //THIS line is in your demo above
3

Thanks for the help @kfrajer! Yes, your code example was just what I needed. The chunk of code I ended up adding looked like this:

  MouseAdapter mA = 
    new MouseAdapter() {
    public void mousePressed(MouseEvent me) {
      mousePressed = true;
      applet.mousePressed();
    }
    public void mouseReleased(MouseEvent me) {
      mousePressed = false;
      applet.mouseReleased();
    }
  };

  panel.addMouseListener(mA);

which required some importing:
import java.awt.event.MouseAdapter; import java.awt.event.MouseMotionListener; import java.awt.event.MouseEvent; import java.awt.MouseInfo;

and a global applet object

PApplet applet = this;

I updated the demo to include mouse motion and keyboard input as well as the mouse clicking. More or less ready for someone to play around with now I think!

//Import Libraries
import processing.awt.PSurfaceAWT;
import javax.swing.JPanel;
import javax.swing.JFrame;
import java.awt.Color;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseMotionListener;
import java.awt.event.MouseEvent;
import java.awt.MouseInfo;
import java.awt.event.KeyEvent;
import java.awt.event.KeyListener;

//Declare Gobal Variables
PGraphics pg;
JFrame frame;
JPanel panel;
PApplet applet = this;

void setup()
{

  size(200, 200); 

  frame = (JFrame)((PSurfaceAWT.SmoothCanvas) getSurface().getNative()).getFrame();
  frame.removeNotify();
  frame.setUndecorated(true);
  frame.setLayout(null);
  frame.addNotify();

  pg = createGraphics(width, height);

  JPanel panel = new JPanel() {
    @Override
      protected void paintComponent(Graphics graphics) {
      if (graphics instanceof Graphics2D) {
        Graphics2D g2d = (Graphics2D) graphics;
        g2d.drawImage(pg.image, 0, 0, null);
      }
    }
  };

  frame.setContentPane(panel);
  panel.setFocusable(true);
  panel.setFocusTraversalKeysEnabled(false);
  panel.requestFocus();
  panel.requestFocusInWindow();

  MouseAdapter mA = 
    new MouseAdapter() {
    public void mousePressed(MouseEvent me) {
      mousePressed = true;
      applet.mousePressed();
    }
    public void mouseReleased(MouseEvent me) {
      mousePressed = false;
      applet.mouseReleased();
    }
  };

  panel.addMouseListener(mA);

  mA = new MouseAdapter() {

    public void mouseDragged(MouseEvent me) {
      mouseX = MouseInfo.getPointerInfo().getLocation().x-frame.getLocation().x;
      mouseY = MouseInfo.getPointerInfo().getLocation().y-frame.getLocation().y;
      applet.mouseDragged();
    }
    public void mouseMoved(MouseEvent me) {
      mouseX = MouseInfo.getPointerInfo().getLocation().x-frame.getLocation().x;
      mouseY = MouseInfo.getPointerInfo().getLocation().y-frame.getLocation().y;
      applet.mouseMoved();
    }
  };

  panel.addMouseMotionListener(mA);

  KeyListener kL = 
    new KeyListener() {

    public void keyTyped(KeyEvent e) {
      key = e.getKeyChar();
      keyCode = e.getKeyCode();
      applet.keyTyped();
    }

    public void keyReleased(KeyEvent e) {
      key = e.getKeyChar();
      keyCode = e.getKeyCode();
      applet.keyReleased();
      keyPressed = false;
    }

    public void keyPressed(KeyEvent e) {
      key = e.getKeyChar();
      keyCode = e.getKeyCode();
      applet.keyPressed();
      keyPressed = true;
    }
  };

  panel.addKeyListener(kL);
}

void draw()
{
  pg.beginDraw();
  pg.background(0, 0);
  if (mousePressed) {
    pg.fill(0, 153, 204, 126);
  } else {
    pg.fill(0, 204, 153, 126);
  }
  pg.ellipse(mouseX, mouseY, 60, 60);
  pg.fill(0, 255);
  pg.stroke(0, 255);
  pg.text(key, mouseX-3, mouseY-10);
  pg.endDraw();
  frame.setBackground(new Color(0, 0, 0, 0));
}
3 Likes
4

Thxs for sharing. Really good example. I can see what you need that extra Applet object. Here I want to add a previous post from the older forum although yours takes it to a different level.

Kf

2 Likes