Particle-interaction based on images' pixel brightness

Processing Coding Questions
1

Hello,

I hope someone is able and willing to aid me in figuring out how to approach a concept I want to try. The idea is rather simple: I want particles to interact with a loaded image.

I have a Processing sketch with a Particle class. This particle is essentially a moving point calculated with a radius and an angle from the center of the sketch. With an array I can make many particles that rotate around the center of the sketch with movement based on a noise-feed. I have managed this.

I want the particles to interact based on the brightness-values of pixels of a loaded black and white .png-image. For example changing the color or increasing/decreasing the scale of the particle:
If the pixel of the image is white -> color = red;
If the pixel of the image is black -> color = blue;
This has proven to be more difficult in practice than I had expected. Right now I have a sketch where I load the image, I read all its pixels, and store the x and y location of the pixels with a brightness value higher than 2 in an ArrayList of PVectors. I’m trying to let the Particle class interact with it but it doesn’t work the way I want it to work.

int amount = 200;
float maxUpperIncr = 0.001;

PImage letter_img;

public ArrayList<PVector> spots;
Balls[] b = new Balls[amount];

void setup() {
  size(1000, 1000);
  background(0);
  
  spots = new ArrayList<PVector>();

  letter_img = loadImage("aphex_l.png");
  
  for (int y = 0; y < letter_img.height; y++) {
    for (int x = 0; x < letter_img.width; x++) {
      int index = x + y * width;
      color c = letter_img.pixels[index];
      float b = brightness(c);

      if (b>2) {
        spots.add(new PVector(x, y));
      }
    }
  }

  //Load class Balls
  for (int i = 0; i<amount; i++) {
    b[i] = new Balls(0, random(10), random(-maxUpperIncr, maxUpperIncr), random(1000), random(1));
  }

  println(spots.size());
}

void draw() {
  background(0);

  for (int i = 0; i!=amount; i++) {
    b[i].show();
    b[i].calc();
    //println(b[i].upperIncr);
  }
}


class Balls {
  float num = 0;

  float a;
  float r = 0;

  PVector loc;
  float x;
  float y;

  float incr;
  float Rincr = 1;

  float scl = 10;

  float upper;
  float upperIncr;
  float n;

  float rad = 20;

  PVector place;

  Balls(float _a, float _num, float _upperIncr, float _upper, float _incr) {
    a = _a;

    upperIncr = _upperIncr;
    upper = _upper;

    num = _num;
    incr = _incr;

    int randomSpot = int(random(0, spots.size()));
    place = spots.get(randomSpot);
  }

  void show() {
    imageMode(CENTER);

    pushMatrix();
    translate(width/2, height/2);

    noStroke();
    fill(255, 255, 255, 255);
    rectMode(CENTER);
    rect(x, y, scl, scl);
    popMatrix();
  }

  void calc() {
    //ANGLE
    num += incr;
    a = radians(num);

    //RADIUS
    upper += upperIncr;
    n = map((noise(upper)), 0, 1, -500, 500);

    x = n * cos(a);
    y = n * sin(a);

    for (int i = 0; i < spots.size(); i++) {
      if (int(x) == spots.get(i).x) {
        //println("RIGHT NOW - 30");
      } else {
        //print("NOT NOW -2");
      }
    }    

  }

I have an image with 1000x1000 pixels. The size of the ArrayList after loading all the bright pixels is 176.252, which is obviously huge. What I tried initially (as you can somewhat see above in the “calc” function of the Particle class) was compare the x- and y-location of one Particle to all of these 176.252 spots to see if they match and therefor if it’s ‘covering’ a white pixel. This didn’t go very well as you might imagine, as it has to do millions of comparisons if I want to start working with more than 1 particle.
What I am now trying to understand is how I should go about doing this. Should I work with coloured rectangles in stead of pixels? Is there a more clever way to achieve what I’m trying to do?

I hope my idea and code are clear. Any help is much appreciated. Thanks in advance.

2

Yea, so there are two ways better ways to go about doing this that i can think of, the good way, and the great way. Although as with all things the good way is the easier to implement option. So basically instead of loading every bright spot into an array and running through that, just save all the image pixels into a two dimensional array then if your particle is at location 200,500 for instance just plug the values into the 2D array and see what color is at that location on the image then adjust the particle color based on that.
The great way is by creating your own custom vertex and fragment shader using GLSL to query the texture coordinates of your image and scale and color the shape using those. The advantage of the GLSL way is that all the particles are computed concurrently at the same time meaning the program will run much faster but again that is much more complicated.

2 Likes
3

Thanks 8BitRift,
After some more digging I indeed found that this could potentially move more into shader territory. Since I have no experience yet with GLSL I’m going to give the 2D-array strategy a try. Thanks for giving me a lead on where to go with this idea.
If you happen to have any good leads on where/how to start with GLSL/shaders and Processing I’m curious and all ears.

Thanks again!

4

Your PImage.pixels is already a 1D array with brightness values, and it is accessible via a 1D index or via 2D (direct math or using get).

Here is an example with 2000 bouncing ball particles moving around on a screen. The take their size and color from an underlying image – including becoming transparent.

PImage img;
ArrayList<PixelBall> balls = new ArrayList<PixelBall>();
int BALLCOUNT = 2000;

void setup() {
  size(512, 512);
  // create background
  img = loadImage("processing3-logo.png"); // https://processing.org/img/processing3-logo.png

  // create balls moving in random directions
  PVector direction;
  for (int i=0; i<BALLCOUNT; i++) {
    direction = PVector.random2D();
    balls.add(new PixelBall(random(0, width), random(0, height), 
      direction.x, direction.y, img));
  }
  noStroke();
}

void draw() {
  background(128);
  // draw bouncing balls
  for (PixelBall ball : balls) {
    ball.move();
    ball.trap();
    ball.draw();
  }
}

/**
 * a bouncing ball whose fill and size are controlled
 * by the corresponding pixel on an associated image.
 */
class PixelBall {
  float[] box;
  PVector pos;
  PVector speed;
  PImage img;
  /**
   * constructor: create ball with position x,y, speed x,y,
   * and an image img to control its fill based on position.
   */
  PixelBall(float x, float y, float speedx, float speedy, PImage img) {
    this.pos = new PVector(x, y);
    this.speed = new PVector(speedx, speedy);
    this.box = new float[]{0, 0, width-1, height-1};
    this.img = img;
  }
  
  /** bounce, update position based on speed */
  void move() {
    if (pos.x >= box[2] || pos.x <= box[0]) {
      speed.x *= -1;
    }
    if (pos.y >= box[3] || pos.y <= box[1]) {
      speed.y *= -1;
    }
    pos.add(speed);
  }
  
  /** force ball inside box */
  void trap() {
    pos.x = constrain(pos.x, box[0], box[2]);
    pos.y = constrain(pos.y, box[1], box[3]);
  }
  
  /** fill ball from corresponding image pixel */
  void draw() {
    try {
      int idx = (int)pos.y * img.width + int(pos.x);
      color c = img.pixels[idx];
      fill(img.pixels[idx]); // faster than get(x, y);
      int bsize = (int)map(brightness(c), 0, 255, 8, 24);
        ellipse(pos.x, pos.y, bsize, bsize);
    } 
    catch (ArrayIndexOutOfBoundsException e) {
      // println("ball outside pixels:", int(pos.y), int(pos.x));
    }
  }
}

The image is never displayed, but I think you can tell from the balls that it is the logo of a certain piece of software…

PixelBalls_screenshot
PixelBalls_screenshot512×512 92.9 KB

3 Likes
5

Thanks so much Jeremy, this is essentially exactly what I was looking for. As per usual my head was in the right place, but I was making the solution too complicated.
I spent the last evening trying to understand what exactly is going on and I’ve learned a lot from your example. For example: I didn’t know you could use the pixel-array of an image to set a color value, based on where a point is and what the pixels color of that point is. It makes sense, but it wouldn’t have passed my mind.

I understand how it works for the majority now, but I was hoping you were willing to elaborate on a few questions I had regarding your coded example:

  • I wasn’t familiar with the try and catch commands. As I understand them now they are used to keep the code running whenever an exception would happen that would normally cause the code to stop (for example an IndexOutOfBoundsException), correct?
  • What exactly happens when the catch exception is met? Does the object that meets the exception just get terminated, hence you’re using an ArrayList and not an array?
  • Why is there a draw function in your Pixelball class, and what are its implementations there? How does this work? I have not seen this before.
  • Other than syntax-wise, is there a difference between (int)x and int(x) for converting a value to an integer? I came across both in your example and wasn’t familiar with the former.

I hope you can come back to this, as I’m mostly a self-thaught coder answers like these significantly help me in learning how to make/understand more complicated sketches. Thanks a bunch!

1 Like
6

Great questions!

  1. try: here is the try reference page: https://processing.org/reference/try.html That code I added doesn’t do anything – so long as all balls are spawned within the area where there are pixels. But what if you create a ball at x=600, y=600? If you do, it will try to access pixels[360000] in the image–and that pixel doesn’t exist in the pixels array, because the image is only 512x512. So the sketch will crash, printing ArrayIndexOutOfBoundsException. The try/catch says "okay, now I want to TRY to draw an ellipse based on a color in this array – but if that fails, catch the error, and then just do nothing–ignore the error and keep going. If you put code in the catch block, it could also do something else instead – for example, moving the illegal-location ball to the center of the screen. This might seem a bit silly given that balls are all bouncing within the legal area, but many forms of bounce code actually have balls that regularly move outside the legal area by a few pixels, change direction, then move back in. In that case, bouncing balls won’t crash the whole program while they are bouncing.

  2. Sorry Pixelball.draw() was confusing. That isn’t a special Processing keyword–it could be named anything. I often name that method (the one that draws the object) “render()” to avoid exactly that confusion. So if you write a method Pixelball.foo(), create a Pixelball myball, then call it with myball.foo(), it runs whatever code is in that method. In this case, the object draws itself – but we could call it “render” or “display” or “show” to make it clear that this isn’t a special Processing keyword. If the Processing syntax highlighter recognizes draw and highlights it then this adds to the confusion.

  3. There is no practical difference between (int)x and int(x). Processing uses int(x) to making everything more consistent for new learners – it is a function which is doing something, and returns a result. However, in Processing you can also use the built in casting syntax of the underlying Java language, in which (int)x means “x as an int”. (float)x etc. works the same way. https://syntaxdb.com/ref/java/type-cast Again, my fault for writing it not in the Processing way to no particular purpose.

Let me know if you have any other questions,

best,
Jeremy

2 Likes
7

Although both styles truncate the value by removing its fractional part, b/c (int) is a Java operator, it is faster than invoking int(), which is transpiled as a PApplet method. :nerd_face:

2 Likes
8

You made everything ultra-clear to me. Like I said before, it’s instances and elaboration like this that really help me learn and understand more complex sketches better. I don’t have any other questions about this anymore simply because you explained it very clearly. Thanks a lot Jeremy.
Right now I’m trying to implement elements from your example to a sketch that uses polar coordinates instead of cartesian coordinates as a next challenge.

Maybe the only question I have is: What methods would you recommend to people like me to improve their coding skills? As I stated earlier I’m mostly a self-thaught coding enthusiast, but I would like to take a more serious approach to it considering its relevance to my field of work. Possibly moving into the world of shaders later too. Most of my basics-knowledge is from partaking in Daniel Shiffman’s YouTube videos. Have you got any advice?

Thanks again. You helped a lot. :slight_smile:

1 Like
9

Your wealth of knowledge never ceases to amaze me GoToLoop. I’ve seen your replies on lots of forum posts I’ve come across earlier, searching for answers to my coding-questions. Have also implemented some of your solutions to stuff of my own. Thanks for elaborating! :grinning:

1 Like