Reversing Projection like `gluUnproject()`

Processing
1

Edit, made a day later: This Stackoverflow Post seems to be of some help.

ORIGINAL POST:

Hello there! As the title states, I want to “reverse” the effect that applying projection makes to a vector.

So, in Processing, we have the perspective() and ortho() functions to apply Projection.

What I did, was I made a system to take a vector from my “game world” (such as one that represents the position of a vertex to be drawn), normalize it / cap it / convert it to a unit vector, and multiply it by the screen’s dimensions (the width and height variables in Processing), then use one of the projection functions to achieve proper scaling of graphics to screens of all sizes.

Does Processing have any built-in ways to achieve this? Is my question not sensible and am I stupid? (:rofl:) Should I look further into raycasting?

Also, I found out during my own research that you can get the Projection Matrix that Processing builds up with this code:

// I couldn't think of a better name, sorry!:
PMatrix3D f = ((PGraphicsOpenGL)g).projection.get();

Please note that the values in this Matrix are in a ‘normalized range’, i.e., between -1.0f and 1.0f.

2 Likes
2

Originally, I was normalizing the Screen co-ordinate and then multiplying it to convert it to a World co-ordinate just the same way I did the world-to-screen conversion.

This,

PMatrix3D f = new PMatrix3D(((PGraphicsOpenGL)g).projection);
// Or `f = ((PGraphicsOpenGL)g).projection.get();
f.invert();
PVector worldVec = f.mult(screenVec, null);
// (Or `f.mult(screeVec, worldVec)`)

…did not do what I expected.

My final solution, which surprised me, was actually just a simple map() for each component of the vector.

Great! I have a solution now :smiley:

(Found the solution a day and almost 4 hours before writing this post.)

Immediate edit: wondering what I could do for the z-axis now :0

3

It has been months! I have had this solution for very long, but I absolutely never posted it.

And yes, I have realized, that apparently, it is better to use screenX(), screenY() and screenZ() for hit/intersection/collision tests rather than un-projecting.
This, is still good to have, however.

Please inform me if this breaks. Sometimes, however, you WILL need to adjust the
mouse.sub() line, in which case, I cannot help! Ask any questions you like, too!
(Also, the cx and cy variables are width / 2 and height / 2, respectively.)

You might need to not use the line with cameraInv as well, if using perspective().

Please note that you need to declare this somewhere in your code, outside any functions or classes:
PGraphicsOpenGL glGraphics;

Then inside setup(), write this!:
glGraphics = (PGraphicsOpenGL)g;

(I feel that it should be glGraphics = (PGraphicsOpenGL)getGraphics(); instead, but the code given above works, so… I didn’t worry as much…)

…alright, here you go! ":D:

PVector mouse = new PVector();

void unprojectMouse() {
    mouse.set(mouseX, mouseY);
    mouse = glGraphics.modelviewInv.mult(mouse, null);
    mouse = glGraphics.cameraInv.mult(mouse, null);

    // This part of the code changes depending on what projection you are using.
    // If it is orthographic (`ortho()`),  this may work:
    
    //mouse.sub(width + cx, height + cy);
    //mouse.x = -mouse.x;
    //mouse.z = 0;


    // Use this otherwise:
    //mouse.sub(width, height);
}

// I generally like to call this inside a method called "`pre()`".
// To use it, however, we need to do this:

void setup() {
  registerMethod("pre", this);
}

// ..the reason being, that this is a method that Processing has made for libraries to use.
// It is called before `draw()`, and helps me write cleaner code.

void pre() {
  unprojectMouse();
}

// .. you can choose to not do all these weird things 
// and just call `unprojectMouse()` inside `draw()` :)

}

I also used this on Android (Only the version that uses perspective() was tested, although both should work given proper modifications).

void unprojectTouches() {
  for (int i = 0; 
    i < nTouches; i++) {
    projectedTouches[i]
      .set(touches[i].x, 
      touches[i].y);
    projectedTouches[i] 
      = glGraphics.modelviewInv
      .mult(projectedTouches[i], 
      null);
    projectedTouches[i] 
      = glGraphics.cameraInv
      .mult(projectedTouches[i], 
      null);
    projectedTouches[i].sub(
      width, height);
  }
}

(I know it looks messy, but those who choose to paste this into APDE will love it, :D!

…here’s a cleaner version :joy:

void unprojectTouches() {
  for (int i = 0; i < nTouches; i++) {
    projectedTouches[i].set(touches[i].x, touches[i].y);

    projectedTouches[i]  = glGraphics
      .modelviewInv.mult(projectedTouches[i], null);

    projectedTouches[i] = glGraphics
     .cameraInv.mult(projectedTouches[i], null);

    projectedTouches[i].sub(width, height);
  }
}

:sweat_smile:)

…but, you also need to include this part:

int nTouches;
PVector[] projectedTouches; // Should be called 
// `unprojectedTouches` - I know, I know...
PGraphicsOpenGL glGraphics; // This was mentioned at the top of this post.

void setup() {
  fullScreen(P3D); // Make sure you're using `P3D`.

  registerMethod("pre", this); // As stated earlier, this is optional.

  glGraphics = (PGraphicsOpenGL)g; // Necessary.
  // ..or,
  //glGraphics = (PGraphicsOpenGL)getGraphics();

  // Free boilerplate code!!!:
  for (int i = 0; i < 10; i++)
    projectedTouches[i] = new PVector();
}

// ..and this is optional too! 
// Call `unprojectTouches()` in `draw()` all you like, ":D!
void pre() {
  unprojectTouches();
}

void touchStarted() {
  nTouches++;
}

void touchEnded() {
  nTouches--;
}
1 Like
4

(Edit: Web Archive version, just in case… gluUnproject() for P3D and OPENGL Sketches)

Selection_in_P3D_OPENGL_A3D  unprojector
  = new Selection_in_P3D_OPENGL_A3D();
PVector mouse = new PVector();

void setup() {
}

void draw() {
  // After `perspective()` and `camera()` have been called,
  unprojector.captureViewMatrix((PGraphics3D)g);
  unprojector.gluUnProject(mouseX, mouseY, 0, mouse);

  // Use `mouse` now!
}