Extending the mouse range past the screen size

Processing Coding Questions
1

You see this in a lot of games, this kind of infinite mouse movement, where you can move your mouse around as much as you want, without ever getting stopped by the edges of your screen.

I’ve made something like this, and also made it so that the mouse movement, although getting reset each time the mouse is moved, can actually be extracted.

import java.awt.*;
Robot robot;
PVector mouse = new PVector();
boolean autoMove = false;

void setup() {
  fullScreen(P3D);
  mouse.set(width/2, height/2);
  strokeWeight(5);
  noCursor();
  try {
    robot = new Robot();
  } 
  catch (Exception e) {
  }
}

void draw() {
  background(255);
  if (!autoMove && (mouseX-pmouseX != 0 || mouseY-pmouseY != 0)) {
    mouse.add(mouseX-width/2, mouseY-height/2);
    robot.mouseMove(width/2, height/2);
    autoMove = true;
  } else {
    autoMove = false;
  }
  point(mouse.x, mouse.y);
  println(mouse.x, mouse.y);
}

As you can see, I’ve done it through the Robot class, which automatically resets the mouse whenever it is moved. However, I am disappointed by a couple of things:

(1) I was unable to make this work in a windowed mode. For that, I would need to somehow get the parameters for the window’s position on the screen in the code, but nothing helped, all the guides online were either for an older version of Processing, or too complicated for me to understand.

(2) I couldn’t use the mouseMoved() function, because the resetting of the mouse also triggers it, and that would instantly undo any saved mouse movement away from the point on screen to which the robot resets the mouse. Instead, I’ve had to create my own function, or rather, a conditional statement, which activates only if it didn’t activate the last time, and vice versa.

(3) The movement is quite laggy.

Is there any way I could improve this sketch? Mainly so that I could resolve these three points above?

1 Like
2

If you’re only doing this with the P2D or P3D renderer you’d be better getting access to the JOGL Window and using confinePointer() and/or warpPointer()

https://jogamp.org/deployment/jogamp-next/javadoc/jogl/javadoc/com/jogamp/newt/Window.html

1 Like
3

Try this sketch! It should do everything perfectly regardless of windowed or fullscreen. I guess you could try integrating its code into your code?

import com.jogamp.newt.opengl.GLWindow;
GLWindow r;
float posX=100;
float posY=100;
boolean Lock=false;
int offsetX=0;
int offsetY=0;           //If we lock the pointer, it will be teleported 
                         //to the center of the screen only as soon as 
                         //the draw() function ends, and until then the
                         //position value will be on a random spot
                         //where the user clicked.
                         //If we take into account that spot for that
                         //one frame, we can correct the position.

float Sensitivity=0.5;

void setup(){
  r=(GLWindow)surface.getNative(); //Ooh, complex OpenGL things I don't know much about myself.
  size(640,480,P2D);
  surface.setResizable(true); //I like resizable surfaces.
}


void draw(){
  
  
  if(Lock){posX+=(mouseX-offsetX-width/2)*Sensitivity; posY+=(mouseY-offsetY-height/2)*Sensitivity;} //If lock, then adjust position...
  //Any random code that depends on those positions...
  
  background(0);
  
  fill(192);
  ellipse(posX,posY,20,20);
  
  fill(255);
  text(Lock?"Locked":"Unlocked",2,15);
  text("posX: "+posX+"\nposY: "+posY,2,35);
  
  
  
  
  if(Lock){
    offsetX=offsetY=0;
  r.setPointerVisible(false); //When locked and trying to move, the pointer jerks all over the place, so best to hide it.
  r.warpPointer(width/2,height/2); //Move it to the exact center of the sketch window.
  r.confinePointer(true); //Locks pointer inside of the sketch's window so it doesn't escape.
                          //The pointer will still hit the window's edges, limiting moves fast enough.
                          //But as soon as the pointer is outside of the window it's visible until it's
                          //teleported back inside the window, which is less preferable.
                          
  }else{r.confinePointer(false); r.setPointerVisible(true);} //Else undo that stuff.
  
  
}



void mousePressed(){
  Lock=!Lock;
  if(Lock){
    offsetX=mouseX-width/2; offsetY=mouseY-height/2;
  }else{
    r.warpPointer(round(posX),round(posY)); //Teleport it to an expected position. Handier that way!
  }
  
}

void keyPressed(){
  if(key=='r'){posX=posY=50;}
}

Also, this might be a bad idea that would confuse everyone including you, but I’ve made some random messy code that allows you to navigate around 3D space similar to what’s in games (like you’re flying around in “noclip” mode, if you know what I mean), and here it is: https://pastebin.com/J4wnp6e2

And yes, I did break my head a couple of times there and made some stuff overly complicated, so please don’t question it! ._.

(also that project of mine ended up having a bunch of unrelated functions in it that I pasted in from my other projects, so, uh, have that for free too, I guess? lol)

2 Likes
4

I have simplified that code into the following:

import com.jogamp.newt.opengl.GLWindow;
GLWindow r;

PVector mousePos = new PVector();

void setup() {
  size(640, 480, P3D);
  r = (GLWindow)surface.getNative();
  r.setPointerVisible(false);
  mousePos.set(width/2, height/2).mult(3);
}

void draw() {
  r.warpPointer(width/2, height/2);
  mousePos.add(mouseX - width/2, mouseY - height/2);
  background(0);
  ellipse(mousePos.x, mousePos.y, 20, 20);
  println(mousePos.x, mousePos.y, mouseX, mouseY);
}

This did everything I needed. However, I don’t understand two things.
Firstly, that .mult(3) factor.
According to the console data, after draw is run once, mouseX and mouseY are yet not initialized at the center of the sketch, which they should have, due to the line r.warpPointer(width/2, height/2); already being run once. But it’s even weirder that they’re not initialized even the second time over.

All this results in the ellipse initially appearing at negative coordinates, unless I scale the initial position by a factor of 3. Weird.

Second, how come r.warpPointer() can stand before mousePos.add()?
Shouldn’t the resetting of the mouse’s position before saving its offset from the center of the screen into the mousePos vector cause the .add() method to output a vector of (0, 0), ergo, not causing the ellipse to move at all?

1 Like
5

mouseX and mouseY are Processing variables only set before draw() is called so the call to warpPointer() happens asynchronously - calling it doesn’t result in the mouse variables being updated until a later run through draw. IIRC the warpPointer() call will be passed on to the JOGL event thread, and then the mouse updated back on the animation thread, which might explain more than one frame delay.

I’m also confused just looking at it exactly what the mult(3) is required for, although the first few mouse positions will be quite off.

1 Like
6

Yes, that’s why I’ve added it in, due to mouseX and mouseY variables being 0, ergo .add() outputting negative values for the first few frames. Though, r.confinePointer(true); seems to change that. Adding that in into setup(), I only need a scaling of 2, not three. And moving the position calculation and mouse reset into mouseMoved() further messes with the initial position.

7

Yes, but that doesn’t seem very reliable. It may not behave identically on different systems. You could ignore 0,0 and maybe also clamp the maximum distance from centre in your calculations?

8

Currently can’t test, but what if you replace the .add() string with this?
if(frameCount>2) mousePos.add(mouseX - width/2, mouseY - height/2);

It would be less responsive for those 2 first frames, but that would effectively ignore any messy positions the pointer might have during them.

9

My current solution is to start modifying the mousePos vector only if frameCount is higher than 2, as @Architector_4 said. It seems to be the safest bet so far.

10

Only if you assume that a couple of frames is always enough for this to take effect on every system you want to run it on!

11

Well, it’s still the best way out(at least that we are aware of).
Another way that could work is to keep ignoring the pointer position until it’s equal to the window center position, but it would rely on the user to not flail their mouse around on the startup, else it would keep on ignoring until the user stops. lol

12

Or within a certain distance, which is what I was suggesting above. You could even slowly raise that allowed distance over time. The idea being to dampen out invalid and noisy values.

13

I was also thinking of something like: if (mouseX + mouseY != 0) {...}, which would ignore the beginning values, but also ignore a jerky move that places the mouse at (0, 0) before resetting it again. Though, the probability of that happening is low.