Pro-Question: place a point in 3D with mouse

Processing Project Guidance
6

Hope this is helpful – here is a very, very simple example of one method using screenX() / screenY().

  1. rotate the reference frame into place in the model.
  2. sample screen coordinates and model coordinates from a grid of points in the frame
  3. compare the mouse location to screen coordinates to look up model coordinates

Nearest3DPlanePoint--screenshot--small

/**
 * find nearby points on a 3D plane
 * 2018-12-20 Jeremy Douglass - Processing 3.4
 * https://discourse.processing.org/t/pro-question-place-a-point-in-3d-with-mouse/6664/4
 */

PVector cam = new PVector();
boolean camMove = true;
PVector[][] modelPoints;
PVector[][] screenPoints;
PVector mouse = new PVector();
PVector mouseCheck = new PVector();

int planeSize = 400;
int gridStep = 10;

void setup() {
  size(600, 600, P3D);
  noFill();
  modelPoints = new PVector[gridStep+1][gridStep+1];
  screenPoints = new PVector[gridStep+1][gridStep+1];
}

void draw() {
  background(0);
  text("Space toggles camera control; mouse over points", 20, 20);
  lights();
  // center drawing
  translate(width/2.0, height/2.0, 0);
  // rotate camera based on mouse
  cam();

  pushMatrix();
  // draw fixed "selected" point
  stroke(200);
  sphere(20);
  // draw reference plane for interactions with that point
  box(planeSize, planeSize, planeSize);
  rect(-planeSize/2.0, -planeSize/2.0, planeSize, planeSize);

  stroke(0, 200, 0);
  for (int y=0; y<=gridStep; y++) {
    int ystep = (y*planeSize/gridStep)-planeSize/2;
    for (int x=0; x<=gridStep; x++) {
      int xstep = (x*planeSize/gridStep)-planeSize/2;
      // update current screen locations of sampled grid coordinates on that plane
      if (camMove) {
        modelPoints[y][x] = new PVector(xstep, ystep, 0);
        screenPoints[y][x] = new PVector(screenX(xstep, ystep, 0), screenY(xstep, ystep, 0));
      }
      pushMatrix();
      translate(xstep, ystep, 0);
      sphere(10);
      popMatrix();
    }
  }
  popMatrix();

  // find spacePoint(s) near mouse and display up corresponding modelPoint(s)
  // this "find all" could be changed to find-first, or closest distance
  mouse.x = mouseX;
  mouse.y = mouseY;
  stroke(200, 0, 0);
  for (int y=0; y<=gridStep; y++) {
    for (int x=0; x<=gridStep; x++) {
      mouseCheck = new PVector(screenPoints[y][x].x, screenPoints[y][x].y, 0);
      if (mouseCheck.dist(mouse)<9) {
        PVector hit = modelPoints[y][x];
        pushMatrix();
        translate(hit.x, hit.y, 0);
        sphere(15);
        popMatrix();
      }
    }
  }
}

void keyReleased() {
  camMove = !camMove;
}

void cam() {
  if (camMove) {
    cam.x = map(mouseY, 0, height, -PI, PI);
    cam.y = map(mouseX, 0, width, -PI, PI);
  }
  rotateX(cam.x);
  rotateY(cam.y);
}

This is a very simple example (and really inefficient, especially the call to dist()), but you could get the first matching point, or measure the nearest point.

Or you could ditch the point grid and use a geometric approach based only on the four corners of the grid and the mouse point. The goal is to find the mouse point inside the quadrilateral once you normalize it to a rectangle – this will tell you where your mouse is in model space. For example, try a raycast, or a PMatrix transform, or affine transform.

or use a third party library like OpenCV which supports homography transformations for a point

…or find the point on the quadrilateral on a corresponding rectangle more directly using either barycentric interpolation on a triangle or bilinear interpolation:


The point being, there are many ways to move from an approximation based on point sampling to a full geometric solution, and some are probably more efficient, but they are math-intensive rather than just logical.

2 Likes