How can I mask graphics to a resizable rectangle?

Processing Coding Questions
1

Hey, I’m in the process of writing a windowed environment in processing. An issue that I’ve struggled with for a while now is being able to mask the contents of a window to the window. Say if I wanted to scroll some text in the window, I would always have to either show or hide the text completely to avoid it overlapping the edge. This looks acceptable with text but with things like large images, the pop-in and out is really noticeable.

So far, I’ve tried a few things.

  • Large PGraphics elements arent performant in the default renderer, but resizing them is instant
  • In the P2D renderer large PGraphics elements run MUCH faster, but resizing causes the fps to tank

To mitigate this i’ve tried:

  • creating a full screen PGraphics element for each window, and a mask PGraphics element that is drawn to by each window and then used to mask the window’s PGraphics element. This solves the resizing issue but also causes perfomance issues due to multiple large canvases and mask() being slow
  • only resizing the PGraphics element every 15 frames instead of every frame. To make up for the slow refresh rate custom drawing methods are used to maintain the scale of everything when the image is drawn at a stretched scale. This is okay, but it will not work for text and looks bad when going from very small to large window sizes.

Example method for how I scale rectangles when the window is resized:

void relativeRect(int tx, int ty, int tw, int th,PGraphics p){
    float rw = w;
    float rh = h-titlebarh;
    if(resizing){
      rw=preresizew;
      rh=preresizeh;
    }
    float newx = (map(tx,0,w,0,rw));
    float newy = (map(ty,0,h-titlebarh,0,rh));
    float neww = (map(tw,0,w,0,rw));
    float newh = (map(th,0,h-titlebarh,0,rh));
    p.rect(newx,newy,neww,newh);
  }

If anyone could suggest anything else that would be awesome. I feel like I’ve tried everything at this point but maybe some other peoples minds can help me see something i’m not seeing.
Thank you :smiley:

2

I think user mgood7123 was asking basically the same question a while ago – you can check his posts.

What exactly isn’t performant with PGraphics in the default renderer?

Are you resizing PGraphics’ contents (scaling) using resize(), or just increasing the dimensions of the PGraphics but keeping the contents the same size (a larger resolution)?

3

I’m not using resize(), just increasing the dimensions of the PGraphics to match the window size.

Here’s some of the important bits from the code:
Window class:

class Window {
  void mp(){   //ran when mouse pressed
    if(resizeHovered){
      resizing=true;
      resizex=x+w-mouseX;
      resizey=y+h-mouseY;
      canvas = createGraphics(w,h-titlebarh);
      preresizew=w;
      preresizeh=h-titlebarh;
    }
  }
  
  void mr(){    // ran when mouse released
    if(resizing){
      canvas = createGraphics(w,h-titlebarh);
    }
    dragging=false;
    resizing=false;
  }

  void update() { // this is run once a frame
    if(resizing){
      w=mouseX-x+resizex;
      h=mouseY-y+resizey;
      h=constrain(h,titlebarh+24,height);
      w=constrain(w,100,width);
      canvas = createGraphics(w,h-titlebarh); // this is the line that causes the issues, I can limit the amount
                                             // of times this is called but the scaling looks a bit blurry
    }

    h=constrain(h,titlebarh+24,height);
    w=constrain(w,100,width);
    preresizeh=constrain(preresizeh,titlebarh+24,height);
    preresizew=constrain(preresizew,100,width);
    drawFrame(x,y,w,h);
  }
  void drawFrame(int fx, int fy, int fw, int fh){
    
      fill(20);
      rect(fx,fy,fw,fh);
      try{
        canvas.beginDraw();
        canvas.background(20);
        canvas.noFill();
        canvas.noStroke();
        canvas.ellipse(mouseRelativeX, mouseRelativeY, 60, 60);
        for(int i = 0;i<20;i++){
          for(int j = 0;j<20;j++){
            if(i%2==0^j%2==0){
              canvas.fill(255);
              relativeRect((i*20),int(j%20)*20,20,20,canvas);
            }
          }
        }
        canvas.endDraw();
      }catch(Exception e){
        
      }
    try{
        image(canvas,x,y+titlebarh,w,h-titlebarh);
      }catch(Exception e){
        bprint("Could not draw canvas"+e.getMessage());
      }
  }
}

drawing large PImage graphics (e.g for a full screen window) also makes it run pretty slow.
Thanks for replying! I will check out those posts.

4

Hello,
maybe I misunderstand. But if you are saying you have a lot of content (bigger than the display window) that you want move around. So you have a long text, and only a part should be shown in the display (?)
or in a smaller area.

Why don’t you use translate(x,y) ?
You can draw outside the PGraphics afaik. Then move it around. like scrolling it up.

PGraphics p;
float positionY = 0;
//-------------------------------------------
void setup() {
  size(600, 400, P2D);
  background(0);
  frameRate(30);
  p = createGraphics(width, height, P2D);
  p.fill(255);
  p.textSize(48);
}
//-------------------------------------------

void draw() {
  p.beginDraw();
  p.pushMatrix();
  p.translate(0,positionY);
  p.background(0);
  p.textSize(48);
  p.text("Testtext", width/2, height/2);
  p.rect(20,-100,300,50);
  p.endDraw();
  p.popMatrix();
  image(p,0,0); 
  positionY += 1;
}

if I understand… no strange, huge stuff :smiley:

ps.: I think one advantage would be that you don’t need to provide a huge PGraphics buffer. Not being an expert, I feel that everything outside the buffer will not even be drawn, while objects intersecting its borders will be calculated (masked).

5

Thanks for the reply! Here’s a diagram to better explain what I’m trying to do:

This is what stuff does currently (unless I dont draw the entire text)

whatitdoes
whatitdoes693×431 177 KB

And this is what I’m trying to achieve

whatiwant
whatiwant693×431 156 KB

Here’s another example with images:

problem1
problem1705×337 79.5 KB

problem2
problem2705×337 75.7 KB

6

oh, so you are not using multiple windows but make your own custom windows in a single processing patch ?

still, if each window is its own PGraphics, content you draw into it should not be visible outside if the PGraphics has the actual size of the window. unless you draw into the display and not the PGraphics. OR the PGraphics is always full screen size.
So… maybe you create the PGraphics anew, only when a resize has been detected…

7

I have a feeling you might want to look at the nine argument version of image() for this - see Alternative to using get() in a big PGraphics

2 Likes
8

Ok, usually one should not post too much solutions :slight_smile: but I got carried away.

maybe this helps…
you need to resize the window to find the image. +/- lets you move it around, the window can be moved or resized.

(maybe there is a little overweight, because I used parts from a button class I made)

Window w;
//--------------------------------------------------------------
void setup() {
  size(1920, 1080, P2D);
  w = new Window();
}
//--------------------------------------------------------------
void draw () {
  background (255);
  w.updateWindow();
  w.renderWindow();
  
  if(keyPressed && key == '+')w.offsetY += 10;
  if(keyPressed && key == '-')w.offsetY -= 10;
}
//--------------------------------------------------------------
void mouseDragged() {
  if (pmouseY > mouseY) w.offsetY += 1;
  if (pmouseY < mouseY) w.offsetY -= 1;
}
//--------------------------------------------------------------

and:

//--------------------------------------------------------------
class Window {
  //-------------------------------------------------------------- variables
  PGraphics myCanvas;
  PImage img;
  int wWidth = 200;
  int wHeight = 100;
  float wX = 0;
  float wY = 0;
  int lowerRightCornerRange = 10;
//---------------
  float startX, startY; // starting coordinates when a drag begins
  float endX, endY; //  coordinates when a drag ends
//---------------
  boolean isInside = false;
  boolean isInsideCorner = false;
  boolean isPressed; // is the mouse pressed
  boolean hasBeenPressed; // has the mouse been pressed before
  boolean isReleased;
  boolean isClicked;
  int pressedTime; // record time when the button is pressed to detect mouseReleased
  int mouseReleasedLimit = 250; // define the time how long after a mousePress a mouseCLick will be accepted
//---------------
  boolean pressedInsideWindow = false;
  boolean pressedInsideCorner = false;
//---------------
  float offsetY = 0; // move the image around -> or use translate and leave the x,y values be
  //-------------------------------------------------------------- constructor
  Window () {
    initializeWindow(wWidth, wHeight);
    img = loadImage("https://occ-0-1068-92.1.nflxso.net/dnm/api/v6/E8vDc_W8CLv7-yMQu8KMEC7Rrr8/AAAABceL_FxRrEg1Jm2LyiYugCJwBkJ2v4GmCBWQ_JNLBXCu1tpO1CMoOxGk9R74PCzrCR0FLIrjdgZlyIHnZEuiHArY6C9G.jpg?r=a82");
  }
  //-------------------------------------------------------------- methods
  void initializeWindow (int w, int h) {
    myCanvas = createGraphics(w, h, P2D);
    myCanvas.beginDraw();
    myCanvas.clear();
    myCanvas.fill(0);
    myCanvas.noStroke();
    myCanvas.endDraw();
  }
  //--------------------------------------------------------------
  void updateWindow() {
    checkCollision();
  }
  //--------------------------------------------------------------
  void renderWindow() {
    myCanvas.beginDraw();
    myCanvas.clear();
    myCanvas.fill(0);
    myCanvas.rect(0, 0, wWidth, wHeight);
    myCanvas.image(img, 200, 200+offsetY);
    if (isInsideCorner) {
      myCanvas.fill(255, 0, 0);
      myCanvas.rect(wX+wWidth-lowerRightCornerRange, wY+wHeight-lowerRightCornerRange, lowerRightCornerRange, lowerRightCornerRange);
    }
    myCanvas.endDraw();
    image(myCanvas, wX, wY);
  }
  //--------------------------------------------------------------
  void checkCollision() {
    if (mouseX > wX && mouseX < wX + wWidth && mouseY > wY && mouseY < wY + wHeight) { // is the mouse inside
      isInside = true;       
      if (mouseX > wX+wWidth-lowerRightCornerRange && mouseX < wX+wWidth && mouseY > wY+wHeight-lowerRightCornerRange && mouseY < wY+wHeight) { // is the mouse inside the corner area
        isInsideCorner = true;
      } else {
        isInsideCorner = false;
      }
    } else {
      isInside = false; 
      isInsideCorner = false;
    }
    //----------------------------------
    if (mousePressed && isInside && !isInsideCorner) {     
      isPressed = true;      
      if (!hasBeenPressed) {
        pressedTime = millis();
        hasBeenPressed = true;
        pressedInsideWindow = true;
        pressedInsideCorner = false;
        println ("Pressed Inside Window");
        startX = mouseX-wX;
        startY = mouseY-wY;
        println ("Start at: "+ startX + " " + startY);
      }
    } else {
      isPressed = false;
    }
    //----------------------------------
    if (mousePressed && isInsideCorner) {
      isPressed = true;      
      if (!hasBeenPressed) {
        pressedTime = millis();
        hasBeenPressed = true; 
        pressedInsideWindow = false;
        pressedInsideCorner = true;
        println ("Pressed Inside Corner");
        startX = mouseX-wWidth;
        startY = mouseY-wHeight;
        println ("Start at: "+ startX + " " + startY);
      }
    } else {
      isPressed = false;
    }
    //----------------------------------
    if (hasBeenPressed) {
      endX = mouseX;
      endY = mouseY;
      if (pressedInsideWindow) {
        wX = endX-startX;
        wY = endY-startY;
      }
      if (pressedInsideCorner) {
        wWidth = int(endX-startX);
        wHeight = int(endY-startY);
        fill(255, 0, 0);
        rect(wX, wY, wWidth, wHeight);
      }
    }
    if (!mousePressed && hasBeenPressed) {
      println ("End at: "+ endX + " " + endY);
      hasBeenPressed = false;
      initializeWindow(wWidth, wHeight);
    }
    //----------------------------------
  }
  //--------------------------------------------------------------
}

cheers

1 Like
9

I just tried out your solution and while it doesn’t seem to solve the problem I’m having it did teach me that you can set the renderer to P2D in createGraphics(), so thanks! I’ve already been able to do this masking using a PGraphics object, the main issue is just that resizing it is very slow. There’s lots of window animations in BrastinOS so I need to be able to resize and display multiple resizing windows in real time. I listed the optimisations that I’ve already tried in my original post. Making sure the PGraphics is using the P2D renderer does help bump the performance up when displaying windows but it still causes big issues when resizing.
Basically I’m just trying to find an alternative to running canvas = createGraphics() every frame while the window is being resized. (or some sort of optimisation)

10

I have a feeling you might want to look at the nine argument version of image() for this - see Alternative to using get() in a big PGraphics
This looks promising. Thanks!

11

yes… that is why I decided to use createGrpahics only at the end of a drag and not every frame.
instead I draw a black rectangle and re-create the buffer only once.

that’s probably a little less pretty than seeing the “correct” window all the time…
i imagine it would be viable to update the window to a new size every n frames or so, using % modulo.
could be no visible difference but good for performance :slight_smile:

or put the resize method into a thread (which I am not an expert with…)

in any case… a user cannot resize more than one window at once. and it might simply be enough to use an overlay rect until the mouse is released, then create the resized PGraphics one single time.

1 Like
12

Yeah, I had set it up with a % to run every 30 frames which bumps the framerate back up to 60, but to make it a little smoother I also made custom drawing functions that draw everything relative to the window size, so the transitions are less jagged. I’ll look into multithreading and also that other image() suggestion too

13

i do not think that I can contribute much more to your problem. I just want to mention that there seems something to be off, to me. if the user of your work resizes the window only once in a while, and one single resize event does not let the patch freeze, where should such a delay come from…

obviously the content of the window (the pGraphics) does not refresh at that point (it stays he same while resizing). obviously the user cannot resize one than more window in that moment… maybe the reason for your rate dropping so strongly lies somewhere else…

1 Like
14

The reason why it slows down is because I want it to update at every instance the window is resized, which is not always due to the user resizing the window. Here’s a demo of everything working - just without the window masking functionality.

But anyway, thank you so much for taking the time to respond and help.

1 Like