Following my series of graphical bezier and shape Editors I am writing a 3D bezier editor.
On mouse click I want to add a new Point to the bezier curve.
When I place the mouse and use its x and y for the 3D point those are very wrong (even if I jist assume Z is 0).
In a way I am looking for the opposite of screenX and screenY here.
My idea is that since I add the new point to a list of existing points I take the Z Position from the last or nearest points in the list and use it as new Z for the new point also.
But how can I calculate the x and y Position correctly?
Since the existing points are drawn in 3D, it’s an projection. When I place the mouse next to an existing point A, the position looks correct. When I click the mouse, and use mouseX and mouseY for the new point B, those 2 values are way off.
So I need a function of an inverted screenX and screenY.
Bear with me here
The idea is: of the point A I know the x,y,z position and the screenX and screenY.
They are in a relationship (formula F that leads from xyz to screenX and screenY).
Now I have the mouseX and mouseY of point B which is its screenX and screenY.
When I apply the opposite of F to the 2 values and use the z value of A for z of B I would derive at x,y for B.
So, please what’s the formula for opposite of F?
Just to make sure I’m understanding – are you trying to find the equivalent of drawing a rectangular plane around your first point and then ray-casting the mouse onto that plane? Would the plane always be camera-aligned, or sometimes not?
Some previous related discussions in the old forum:
Hope this is helpful – here is a very, very simple example of one method using screenX() / screenY().
rotate the reference frame into place in the model.
sample screen coordinates and model coordinates from a grid of points in the frame
compare the mouse location to screen coordinates to look up model coordinates
/**
* find nearby points on a 3D plane
* 2018-12-20 Jeremy Douglass - Processing 3.4
* https://discourse.processing.org/t/pro-question-place-a-point-in-3d-with-mouse/6664/4
*/
PVector cam = new PVector();
boolean camMove = true;
PVector[][] modelPoints;
PVector[][] screenPoints;
PVector mouse = new PVector();
PVector mouseCheck = new PVector();
int planeSize = 400;
int gridStep = 10;
void setup() {
size(600, 600, P3D);
noFill();
modelPoints = new PVector[gridStep+1][gridStep+1];
screenPoints = new PVector[gridStep+1][gridStep+1];
}
void draw() {
background(0);
text("Space toggles camera control; mouse over points", 20, 20);
lights();
// center drawing
translate(width/2.0, height/2.0, 0);
// rotate camera based on mouse
cam();
pushMatrix();
// draw fixed "selected" point
stroke(200);
sphere(20);
// draw reference plane for interactions with that point
box(planeSize, planeSize, planeSize);
rect(-planeSize/2.0, -planeSize/2.0, planeSize, planeSize);
stroke(0, 200, 0);
for (int y=0; y<=gridStep; y++) {
int ystep = (y*planeSize/gridStep)-planeSize/2;
for (int x=0; x<=gridStep; x++) {
int xstep = (x*planeSize/gridStep)-planeSize/2;
// update current screen locations of sampled grid coordinates on that plane
if (camMove) {
modelPoints[y][x] = new PVector(xstep, ystep, 0);
screenPoints[y][x] = new PVector(screenX(xstep, ystep, 0), screenY(xstep, ystep, 0));
}
pushMatrix();
translate(xstep, ystep, 0);
sphere(10);
popMatrix();
}
}
popMatrix();
// find spacePoint(s) near mouse and display up corresponding modelPoint(s)
// this "find all" could be changed to find-first, or closest distance
mouse.x = mouseX;
mouse.y = mouseY;
stroke(200, 0, 0);
for (int y=0; y<=gridStep; y++) {
for (int x=0; x<=gridStep; x++) {
mouseCheck = new PVector(screenPoints[y][x].x, screenPoints[y][x].y, 0);
if (mouseCheck.dist(mouse)<9) {
PVector hit = modelPoints[y][x];
pushMatrix();
translate(hit.x, hit.y, 0);
sphere(15);
popMatrix();
}
}
}
}
void keyReleased() {
camMove = !camMove;
}
void cam() {
if (camMove) {
cam.x = map(mouseY, 0, height, -PI, PI);
cam.y = map(mouseX, 0, width, -PI, PI);
}
rotateX(cam.x);
rotateY(cam.y);
}
This is a very simple example (and really inefficient, especially the call to dist()), but you could get the first matching point, or measure the nearest point.
Or you could ditch the point grid and use a geometric approach based only on the four corners of the grid and the mouse point. The goal is to find the mouse point inside the quadrilateral once you normalize it to a rectangle – this will tell you where your mouse is in model space. For example, try a raycast, or a PMatrix transform, or affine transform.
or use a third party library like OpenCV which supports homography transformations for a point
…or find the point on the quadrilateral on a corresponding rectangle more directly using either barycentric interpolation on a triangle or bilinear interpolation:
The point being, there are many ways to move from an approximation based on point sampling to a full geometric solution, and some are probably more efficient, but they are math-intensive rather than just logical.
It does make sense! I think that is what I did – I listed several ways to map a screenX back to a modelX – given a defined reference plane ABCD. It isn’t an “opposite formula” in the sense that you can’t invert an operation to turn a 2D coordinate into a 3D coordinate – information has been lost, so you need to introduce that missing information using either sampled points or a square/rect/quad and a mapping. That mapping could be any of the things I mentioned above.
Of course, I haven’t tried these solutions personally beyond the sample point demo, and there might be others, but I think things like affine transforms and homography are very widely used in lots of applications – from satellite cartography to shaders. So it seems likely that if many people are already finding points on a 3D plane in that way, then that is (one) good way of doing it.
