Picking library failing on many spheres

Processing Libraries
1

Hello, I’m relatively new to Processing. I’m trying to use the Picking library (installed directly through the Processing app) to have a bunch of spheres be clickable. However, Picking seems to not work well with many spheres and I can’t figure out why. Here’s a snippet demonstrating my issue:

import picking.*;

int size = 40;
int n = 6;
Picker picker;

void setup() {
  size(800,800,P3D);
  picker = new Picker(this);
}

void draw() {
  background(127);
  lights();
  translate(size*2,size*2,0);
  noStroke();
  
  // make an n by n grid of spheres each with a unique picker id and colour
  for (int i = 0; i < n; i++) {
    pushMatrix();
    for (int j = 0; j < n; j++) {
      picker.start(n*i+j);
      fill(i*255/n,j*255/n,0);
      sphere(size);
      translate(size*3,0,0);
    }
    popMatrix();
    translate(0,size*3,0);
  }
}

void mousePressed() {
  println(picker.get(mouseX,mouseY));
}

In this example, only the bottom right sphere clicks as expected (printing 35). When clicking anywhere else, it prints something related to the actual colour at the mouse position.

The reason I mention spheres in particular is that if I change them to boxes, it works as expected. This example also works when there are only 25 spheres instead of 36, so at this point I’m very confused.

I’ve thought about making my own implementation using screenX() and screenY() as suggested in some forum posts I found, but I would prefer if I could get Picking to work.

Thanks in advance for the help!

2 
for (int j = 0; j < n; j++) {
  picker = new Picker(this);
  picker.start(n*i+j);
  fill(i*255/n,j*255/n,0);
  sphere(size);
  translate(size*3,0,0);
}

and if you instantiate the picker i your loop?

1 Like
3

Simply moving the picker instantiation into the loop caused errors with popMatrix() so I rewrote it to not use pushMatrix() and popMatrix():

...
  for (int i = 0; i < n; i++) {
    for (int j = 0; j < n; j++) {
      picker = new Picker(this);
      picker.start(n*i+j);
      fill(i*255/n,j*255/n,0);
      sphere(size);
      translate(size*3,0,0);
    }
    translate(-size*3*n,0,0);
    translate(0,size*3,0);
  }
...

This causes the error: “RuntimeException: Too many pushFrameBuffer calls”

4

For whatever it’s worth I get this error message on a Mac. Possibly has something to do with your issue:
error

5

Hello @lavalaph,

This seemed to work here for homogenous colors:

import picking.*;

Picker picker;

void setup() 
  {
  size(600, 600, P3D);
  picker = new Picker(this);
  background(127);  
  }

void draw()
  {
  background(127);
  ortho();          // Look up the reference! Try without it.
  noStroke();
  
  // First work with a homogenous color
  for (int i = 0; i < 5; i++) 
    {
    picker.start(i);
    int x = i*100+100;
    int y = i*100+100;
    fill(i*255/5, i*255/5, 0);
    circle(x, y, 120);
    picker.stop();    
    } 

  // Overlay spheres with lights AFTER the picking.
  lights();
  for (int i = 0; i < 5; i++) 
    {
    int x = i*100+100;
    int y = i*100+100;
   pushMatrix();
    fill(i*255/5, i*255/5, 0);
    translate(x, y);
    sphere(60);
   popMatrix();     
    }
  }

void mouseClicked()
  {
  int id = picker.get(mouseX, mouseY);
  println(id);
  }

You can adapt this for your grid; I did this for a grid and it worked for that as well.

The OpenGL Error: 1282 error appears here even with the example code but it still seems to work.

This code did not generate an error:

This did not generate an error! < Click to see code! 
// OpenGL error 1282 at top endDraw(): invalid operation
// on first mouseClick

import picking.*;

Picker picker;

void setup() 
  {
  size(600, 600, P3D);
  picker = new Picker(this);
  background(127);  
  }

void draw()
  {
  background(127);
  ortho();          // Look up the reference! Try without it.
  noStroke();
  
  // First work with a homogenous color
  for (int i = 0; i < 5; i++) 
    {
    picker.start(i);
    int x = i*100+100;
    int y = i*100+100;
    fill(i*255/5, i*255/5, 0);
    circle(x, y, 120);
    //picker.stop();    
    }
    
  // This did not generate:
  // "OpenGL error 1282 at top endDraw(): invalid operation"
  if(b)
   {
   println(picker.get(mx, my));
   b = false;
   }     

  // Overlay spheres with lights.
  lights();
  for (int i = 0; i < 5; i++) 
    {
    int x = i*100+100;
    int y = i*100+100;
   pushMatrix();
    fill(i*255/5, i*255/5, 0);
    translate(x, y);
    sphere(60);
   popMatrix();     
    }
  }

int mx, my;
boolean b;

void mouseClicked()
  {
  mx =mouseX; 
  my = mouseY;
  b = true;
  }

Scrutinize the code to try and understand what was done.

:)

6

@glv I tried adapting your first example to a grid. It seemed to work fine with a 5x5 grid, but not with a 6x6 grid, even when overlaying lights after the picker is finished. I also tried extending your example to a line of 36 spheres, which has the same issue where only the last sphere works as expected:

import picking.*;

Picker picker;

void setup() {
  size(800, 800, P3D);
  picker = new Picker(this);
  background(127);  
}

void draw() {
  background(127);
  ortho();          // Look up the reference! Try without it.
  noStroke();
  
  // First work with a homogenous color
  for (int i = 0; i < 36; i++) {
    picker.start(i);
    int x = i*20 + 20;
    int y = i*20 + 20;
    fill(i*255/35, i*255/35, 0);
    circle(x, y, 24);
    picker.stop();
  } 

  // Overlay spheres with lights AFTER the picking.
  lights();
  for (int i = 0; i < 36; i++) {
    int x = i*20 + 20;
    int y = i*20 + 20;
   pushMatrix();
    fill(i*255/35, i*255/35, 0);
    translate(x, y);
    sphere(12);
   popMatrix();
  }
}

void mouseClicked() {
  int id = picker.get(mouseX, mouseY);
  println(id);
}

(As an aside, I like the smaller indent on the pushMatrix() and popMatrix() lines. I’ll definitely be doing that too from now on.)

7

Experiment. That is what I did. I am not the expert here on this library but can often find a solution.
I will certainly revisit the Picking library and examples to understand it better and iterate towards a solution.
Try looking at examples and swap out the box with a sphere… that looks promising.

This (my fix for error and the extra background()) corrected the longer diagonal row:

image
image859×742 42.7 KB

image
image802×839 23.3 KB

You may want to consider other approaches:

:)

8

Here is an alternative technique for you to test. It uses an ArrayList of PVectors and ‘dist’ to tell when the mouse is clicked on a sphere.

ArrayList<PVector> vectors = new ArrayList<PVector>();

PVector v;

void sphereGrid(int left, int top, int w, int vg, int hg) {
  for (int k = 0; k < 7; k++) {
    for (int j = 0; j < 6; j++) {
      int x = left + j*(w+vg);
      int y = top + k*(w+hg);
      pushMatrix();
      noStroke();
      lights();
      translate(x, y, 0);
      vectors.add(new PVector(x, y));
      sphere(w);
      popMatrix();
    }
  }
}

void setup() {
  size(800, 800, P3D);
  background(0, 0, 245);
  sphereGrid(160, 100, 40, 50, 50);
}

void draw() {
}

void mousePressed() {
  println(mouseX, mouseY);
  for (int i = 0; i < vectors.size(); i++) {
    v = vectors.get(i);
    if(dist(mouseX,mouseY,v.x,v.y) < 40) {
      println("id =", i);
    }
  }
}
9

Not sure if this is the cause, but there used to be an issue with rendering spheres using a size of anything other than 1 - the sphere would simply not close properly at the top. That you have this issue specifically with sphere and not boxes is telling. Instead of drawing a sphere(size);, I would pushMatrix(); scale(size); sphere(1); popMatrix(); instead. Try it and see if that helps.

Cheers,

  • TfGuy44 (I wrote the first version of the code to do picking that inspired the Picking library many, many years ago!)
2 Likes
10

As you have discovered the picking library is very easy to use especially for a newbie to Processing. There are other libraries and this animation shows picking spheres in a 3D rotating space and was created using the Shapes3D library and the sketch code below.

As you can see the code is more complex and instead of using Processing’s sphere() method the library supports a wide range of shapes including Ellipsoid.

Maybe you are not ready to try this library yet but it might be something to keep in mind as you get more used to Processing. Shapes3D has its own website for detailed information on it’s API and use.

/**
 * ========================================================
 * Shape picking with a mouse click or a selection marquee
 * ========================================================
 *
 * This sketch demonstrates how you can use a
 * marquee selection and find all shapes/shape
 * parts inside the selection rectangle.
 *
 * Simple click and drag a selection area, when you
 * release the mouse button all ball faces inside the
 * selection area will be given a new random colour.
 *
 * For a single ball a simple click on it works.
 
 * Notice there is a slight pause on first use,
 * this is caused by Shapes3D creating the pick
 * graphics buffer and will not happen again unless
 * you resize the display.
 *
 *    created by Peter Lager 2024
 */

import shapes3d.*;
import shapes3d.contour.*;
import shapes3d.org.apache.commons.math.*;
import shapes3d.org.apache.commons.math.geometry.*;
import shapes3d.path.*;
import shapes3d.utils.*;

int nbrBalls = 20;
Ellipsoid[] ball = new Ellipsoid[20];
int[] bcolor = new int[ball.length];


float bsize, a, d = 50, c = 120;

// Marquee variables
int mx0, my0, mx1, my1;
boolean mvisible = false;

void setup() {
  size(400, 400, P3D);
  cursor(CROSS);
  textSize(20);
  textAlign(CENTER, CENTER);
  for (int i = 0; i < ball.length; i++) {
    bcolor[i] = randomColor();
    bsize = 5 + (int)random(12);
    ball[i] = new Ellipsoid(bsize, bsize, bsize, 20, 12);
    ball[i].moveTo(random(-d, d), random(-d, d), random(-d, d));
    ball[i].fill(bcolor[i]);
    ball[i].drawMode(S3D.SOLID);
    ball[i].tag = "#:" + i;
  }
}

void draw() {
  background(64);
  push();
  lights();
  a += 0.006;
  camera(c * sin(a), 10, c * cos(a), 0, 0, 0, 0, 1, 0);
  for (int i = 0; i < ball.length; i++) {
    ball[i].draw(getGraphics());
  }
  pop();
  push();
  // Draw 2D stuff above 3D scene
  hint(DISABLE_DEPTH_TEST);
  // Draw selection area
  if (mvisible) {
    stroke(255);
    strokeWeight(2);
    fill(255, 160);
    rect(mx0, my0, mx1 - mx0, my1-my0);
  }
  noStroke();
  fill(0);
  rect(0, height - 30, width, 30);
  fill(160, 255, 160);
  text("[R] to restore original colors", 0, height - 34, width, 30);
  hint(ENABLE_DEPTH_TEST);
  pop();
}

// Select single ball with mouse click
void mouseClicked() {
  Picked p = Shape3D.pick(this, getGraphics(), mouseX, mouseY);
  if (p != null)
    println("Balls selected  " + p.shape.tag);
}

void mousePressed() {
  mx0 = mx1 = mouseX;
  my0 = my1 = mouseY;
  mvisible = true;
}

void mouseDragged() {
  mx1 = mouseX;
  my1 = mouseY;
}

// Find all balls that intersect with a selection area
void mouseReleased() {
  Picked[] picked = Shape3D.pick(this, getGraphics(), mx0, my0, mx1, my1);
  if (picked.length > 0) {
    int c = randomColor();
    println("Balls intersecting with marquee selection  ");
    for (Picked p : picked) {
      p.shape.fill(c);
      println(p.shape.tag);
    }
    println("---------------------------------");
  }
  mvisible = false;
}

void keyReleased() {
  if (key == 'r') {
    for (int i = 0; i < ball.length; i++) {
      ball[i].fill(bcolor[i]);
    }
  }
}

int randomColor() {
  return color(random(100, 220), random(100, 220), random(100, 220));
}
1 Like
11

Thanks for all the help, everyone.

Just to give an update - I decided that the easiest and cleanest way for me would be to write my own picker using screenX and screenY. It turned out similar to svan’s method. It works well enough for what I need it for, so I don’t need anything more complex. If I did, I think I’d try taking a look at Shapes3D.

I also tried out the scale trick that TfGuy44 mentioned, but that didn’t seem to work on its own.

1 Like
12

I’ve tried having the start point at the centre, then having the end point [radius] distance away in all different directions.

It basically tries to put all balls in various orders into the pipe and measures which combination is the shortest.

13

another picking:

Everything we draw on screen we also draw on a PGraphics that we don’t display. We use PGraphics only to check it against the mouse. The items / boxes we draw on the PGraphics we draw with an unique color.
Then we can use pg.get(mouseX,mouseY) to check against the pixel.
We need some kind of list to know which item / box has thi color we found under the mouse.

It is a picking mechanism: The color underneath the mouse is checked against a unique color that every box has. This unique color is invisible on a PGraphics that gets not displayed. In this PGraphics the check of the mouse is performed.

This is useful because we don’t have to do any calculations with mouse position and box position but can just use the colors. We have to use the PGraphics because in the visible graphic, the colors are distorted because we use lights(). The PGraphics has the same size as the visible window and also uses the same camera data of course. But noLights() command

see Looking for more resources about ArrayLists - #45 by Chrisir

1 Like