Pixel sorting in processing 3.4

Hi

I have a somewhat simple question (I think). Hope someone is able to help. I’m doing a project where I need to sort the pixels of an image by different parameters. As I’m very new to processing and self-learning I’m aiming for some very simple coding. I also posted a question yesterday, but realised that I wasn’t to clear :slight_smile:

I have this code so far - sorting the pixels in my image by brightness. I don’t like the very static output and would like to be able to play around with how the output image can get more interesting. (Ideally I would like something more organic and interesting - I have looked at works by Jeff Thompson and Kim Asendorf - very nice). I need to be able to explain the code though, so it has to be more simple than that, as I’m so new to processing).

I’ve attached and image that explains how (top drawing) the sorting is now. The lowest drawing explains another way the pixels could be sorted… and here is my question; Does anyone know how I could change my code so it’s sorting like that?

And just an open question: Does anyone have any great ideas for other simple codes I could try to make?

PImage img;
PImage sorted;

void setup() {
  size(600, 480);
  
  img = loadImage("kbhnv600x480.jpg");
  sorted = img.get();
  sorted.loadPixels();

  
  for (int i = 0; i < sorted.pixels.length; i++) {
    float record = -1;
    int selectedPixel = i;
    for (int j = i; j < sorted.pixels.length; j++) {
      color pix = sorted.pixels[j];
      
      float b = brightness(pix);
      if (b > record) {
        selectedPixel = j;
        record = b;
      }
    }

    // I guess this is where I need to change my code??
    color temp = sorted.pixels[i];
    sorted.pixels[i] = sorted.pixels[selectedPixel];
    sorted.pixels[selectedPixel] = temp;
  }

  sorted.updatePixels();
}

void draw() {
  background(0);
  image(sorted, 0, 0);
  save("kbhnv600x480_brightness.jpg");
}
1 Like

One way you can go about creating the effect you explain without changing your sorting code is by creating an array where the numbers match the bottom grid. Then, for every position (x, y) in your grid, look at the number and get the nth brightest pixel. For example, at (x=4, y=0) is 10 so you would get the 10th brightest pixel. Then, put the 10th brightest pixel in a new image at (x=4, y=0).

Creating that grid might not be as easy as it looks though. There is probably a simple formula for each index, but it’s not immediately obvious to me. It would be faster for me to create it by copying what you do it by hand (e.g. start at 0-0, move down and add 1, go diagonally up, etc.).

Another thing to note when sorting colors is that the built in brightness function doesn’t always do the best job because it doesn’t account for us being more sensitive to green. I think I have seen brightness functions that handle this called luminance.


but if you want to play around with different sorting functions, here is a version of luminance you can use in place of brightness():

float luma(color c){
  return (.299f*red(c)) + (.587f*green(c)) + (.114f*blue(c));
}

Another thing you could do is bit shift the colors, it usually looks cool. This effect is just pixel << 1.
colors
If you don’t know where to start, your setup() function might look like this:

void setup(){
  // size();
  // sorting code { ... }
  // sorted.updatePixels();
  
  loadPixels();
  // the map from sorted to diagonal sorted
  int[] index = indexBuilder(width, height); 
  for(int i=0; i<sorted.pixels.length; i++){
    pixels[index[i]] = sorted.pixels[i] << 1;
  }
  updatePixels();
}
Where indexBuilder is the grid and would hold the values [0, 11, 1, 22, 12, 2, 33, 23, ...] from your example above (the second index is "11" because the second brightest pixel should be placed at that index. If you look at your grid, 2 and 12 are in the same place, not 2 and 11. But I am off by one for using zero based indexing). Well, hopefully some of that made sense.
3 Likes

Thank you so much!

That’s very helpful – so nice of you to explain it a bit more for me.

Your picture looks like what I was aiming for!

It all make sense - I still have some questions though. As I’m so new it is very difficult to translate my logic into the logic of the coding…

Here is my code again.


//
PImage img;
PImage sorted;

void setup() {
  size(600, 480);
  
  img = loadImage("img.jpg");
  sorted = img.get();
  sorted.loadPixels();

  // Selection sort!
  for (int i = 0; i < sorted.pixels.length; i++) {
    float record = -1;
    int selectedPixel = i;
    for (int j = i; j < sorted.pixels.length; j++) {
      color pix = sorted.pixels[j];
      
      float b = brightness(pix);
      if (b > record) {
        selectedPixel = j;
        record = b;
      }
    }

    // Swap selectedPixel with i
    color temp = sorted.pixels[i];
    sorted.pixels[i] = sorted.pixels[selectedPixel];
    sorted.pixels[selectedPixel] = temp;
  }

  sorted.updatePixels();
  
  sorted.loadPixels();
  
// As I understand here I need to create a new array/map called indexBuilder?
//According to the picture I have attached it would hold the values {0, 600, 1, 700, 601, 2, 800, 701, 602, 3, etc.} .. this array withholds 288.000 objects though.. so how do I write that? 



  int [] index = indexBuilder(width, height);
  for(int i =0; i<sorted.pixels.length; i++) {
    pixels[index[i]] = sorted.pixels[i] <<1;
    
  }
  updatePixels();
}

void draw() {
  background(0);
  image(sorted, 0, 0);
  save("img_output.jpg");
}

2 Likes

It’s a little more complicated that it seems, it’s probably better to walk through what to do than try to explain.

One thing I hope is clear is that 1D arrays and 2D arrays are basically the same thing, so int[3][3] is similar to int[9]. So even though I said we needed a “grid”, it’s fine to create a 1D grid, and we just have to know the width in order to convert from 1D to 2D. For example, if you wanted to set the pixel (3,3) to red without using the built in set() function, you can convert (3,3) to a 1D index with pixels[y*width + x] and fill the y, width, and x values:

pixels[3*480 + 3] = color(255, 0, 0);

So yeah, making the indexBuilder is a bit of a pickle. The easiest way would just be to write the whole list out:

int[] index = {0, 600, 1, 700, ...};

But as you saw, it’s ridiculously long. So instead of just putting them in, I just waved my hands and said

int[] index = indexBuilder(...);

that way, I didn’t have to go to the trouble of actually making it, I can just pretend I have already made it. When processing gets to that line, it will look for a function called “indexBuilder()”, and run that code before moving to the next line. The function call will get replaced by whatever the function returns, like {0, 1, 2, …}. What we want now is to make the function that builds the index. We can place our function anywhere in the file, I put mine beneath draw(){...}, and it will look something like this:

type name(){ }

where “type” is what it will give us back, and “name” is whatever we want to call it. I called mine indexBuilder:

type indexBuilder(){ }

We know we want it to return us an array, because on the line:

int[] index = indexBuilder(width, height);

there is an array to the left of the equals sign. So we should make the function match that:

int[] indexBuilder(){ }

and then since we said it will return a type of integer array, it will complain at us until we actually do it:

int[] indexBuilder(){
	return new int[288000];
}

When you make a new array, it is just full of zeros. If we wanted to make the top grid, we could just use a for loop:

int[] indexBuilder(){
	int[] array = new int[288000];
	for(int i=0; i<array.length; i++){
		array[i] = i;
	}
	return array; // returns {0, 1, 2, ...}
}

You can think of the line array[i] = i; as being like the math function “y = x”, the input matches the output.
f(0) = 0,
f(1) = 1,
f(2) = 2, etc.
So first, I used pen and paper to see if I could find the formula for the bottom grid, where we could just straight replace i. The left column pattern is clearly “y = sum from 1 to x”, and the top row is the same sum, just -1. So I tried summing the x and y rows together like so:

int[] indexBuilder(){
	int[] array = new int[288000];
  	for (int x=0; x<5; x++) {
    	for (int y=0; y<5; y++) {
      		int y_sum = sum(1, y+1) - 1;
      		int x_sum = sum(y, y+x-1) + x; 
      		array[i] = x_sum + y_sum;
    	}
  	}
	return array; // returns {0, 2, 5, 9, ...}
}

It looks correct, but it only works for the top-left half of the array, and things get inverted when you cross the diagonal. I’m sure there is a nice function to generate the numbers we need, but at this point, it might take a while to find it. And it is probably easier to just create and fill in a table by following what you would do by hand. So I wrote this instead:

int[] indexBuilder(int wide, int high) { 
  int len = wide * high;
  int[] index = new int[len];
  int index_position = 0; // current index being calculated
  int w=wide, h=high;

  for (int i=0; i<h; i++) { // go down left side from top to bottom
    int x = 0, y = i;
    while (y >= 0 && x < w) { // stop if on top row or right column
      index[index_position++] = (y * w) + x; // convert position from 1d to 2d
      y--; // move up
      x++; // move right
    }
  }
  for (int i=1; i<w; i++) { // go across bottom row from left to right
    int x = i, y = h-1;
    while (y >= 0 && x < w) { // stop if on top row or right column
      index[index_position++] = (y * w) + x; // convert position from 1d to 2d
      y--; // move up
      x++; // move right
    }
  }
  return index;
}

If you find this a little hard to read, that makes two of us. I realise I just pulled a “how to draw an owl” on you, but as you can see, it’s a little intricate. I don’t know how to explain that to somebody. You might find a simpler version that will be easier to read by searching for “traverse array diagonal”. Hopefully you get the gist of what is going on by seeing a solution though.

how_owl

One more thing is that although sorting only takes a minute or two for me, it is possible to sort all the numbers in milliseconds by using a different sorting algorithm (ideally it would take n*log(n) steps on average). I just mention this in case you want to do bigger images. I would add a print statement to your for loop though, so you can track your progress:

for (int i ...) {
  for (int j ...) {...}
  // swap pixels, etc.

  println(i*100f/sorted.pixels.length + " %");
}

Oh yeah, and if the output looks fuzzy, you can swap from saving a jpg to a png.

3 Likes

Thank you again.

It’s unbelievably nice of you. I have more questions, but feel free to say that you don’t want to spent more time on this anytime :slight_smile:

It is very cool that you explain it so much, because I’ve been able to find new tutorials and stuff to help me.

Anyway - here is my code now - I haven’t been able to get further as I get an error at
int[] indexBuilder() {

it says: "expecting DOT, found ‘indexBuilder’

You wrote that you put your code in draw() - I tried to do that too, but same error… so that is why I tried to move it to setup, just to see… But it’s the same.

Is it an obvious mistake a beginners eye doesn’t see or am I on the wrong track, because I misunderstood something maybe?

Thanks.

PImage img;
PImage sorted;


void setup() {
  size(600, 480);
  
  img = loadImage("island.jpg");
  sorted = img.get();
  sorted.loadPixels();

  // Selection sort!
  for (int i = 0; i < sorted.pixels.length; i++) {
    float record = -1;
    int selectedPixel = i;
    for (int j = i; j < sorted.pixels.length; j++) {
      color pix = sorted.pixels[j];
      
      float b = brightness(pix);
      if (b > record) {
        selectedPixel = j;
        record = b;
      }
    }

    // Swap selectedPixel with i
    color temp = sorted.pixels[i];
    sorted.pixels[i] = sorted.pixels[selectedPixel];
    sorted.pixels[selectedPixel] = temp;
  }

  sorted.updatePixels();
  loadPixels();
 
 // the map from sorted to diagonal sorted
 
  int[] indexBuilder() {
  return new int[288000];
}
  int[] indexBuilder(){
  int[] array = new int[288000];
    for (int x=0; x<5; x++) {
      for (int y=0; y<5; y++) {
          int y_sum = sum(1, y+1) - 1;
          int x_sum = sum(y, y+x-1) + x; 
          array[i] = x_sum + y_sum;
      }
    }
  return array; 
}

int[] indexBuilder(int wide, int high) { 
  int len = wide * high;
  int[] index = new int[len];
  int index_position = 0; 
  int w=wide, h=high;

  for (int i=0; i<h; i++) { 
    int x = 0, y = i;
    while (y >= 0 && x < w) { 
      index[index_position++] = (y * w) + x; 
      y--; 
      x++; 
    }
  }
  for (int i=1; i<w; i++) { 
    int x = i, y = h-1;
    while (y >= 0 && x < w) { 
      index[index_position++] = (y * w) + x; 
      y--; 
      x++; 
    }
  }
  return index;
}
  
  
  //
  
  int[] index = indexBuilder(width, height); 
  for(int i=0; i<sorted.pixels.length; i++){
    pixels[index[i]] = sorted.pixels[i] << 1;
  }

  updatePixels();
}

void draw() {
  background(0);
  image(sorted, 0, 0);
 save("island_sorted.png");
}
1 Like

No problem, happy to help

Oh yes that was unclear, it goes underneath draw(), not inside, and you only need the last version. Also, you don’t want the stuff in draw() to run because it will overwrite the image we wrote in setup(). So it would look like this:

PImage img;
PImage sorted;

void setup() {
  size(600, 480);

  img = loadImage("island.jpg");
  sorted = img.get();
  sorted.loadPixels();
  image(sorted, 0, 0);

  // Selection sort!
  for (int i = 0; i < sorted.pixels.length; i++) {
    float record = -1;
    int selectedPixel = i;
    for (int j = i; j < sorted.pixels.length; j++) {
      color pix = sorted.pixels[j];

      float b = brightness(pix);
      if (b > record) {
        selectedPixel = j;
        record = b;
      }
    }

    // Swap selectedPixel with i
    color temp = sorted.pixels[i];
    sorted.pixels[i] = sorted.pixels[selectedPixel];
    sorted.pixels[selectedPixel] = temp;
    println(i*100f/sorted.pixels.length + " %");
  }
  sorted.updatePixels();
  //image(sorted, 0, 0);

  loadPixels();
  // the map from sorted to diagonal sorted
  int[] index = indexBuilder(width, height); 
  for (int i=0; i<sorted.pixels.length; i++) {
    pixels[index[i]] = sorted.pixels[i];
  }
  updatePixels();
}

void draw() {
  //background(0);
  //image(sorted, 0, 0);
  //save("island_sorted.png");
}

int[] indexBuilder(int wide, int high) { 
  int len = wide * high;
  int[] index = new int[len];
  int index_position = 0; 
  int w=wide, h=high;

  for (int i=0; i<h; i++) { 
    int x = 0, y = i;
    while (y >= 0 && x < w) { 
      index[index_position++] = (y * w) + x; 
      y--; 
      x++;
    }
  }
  for (int i=1; i<w; i++) { 
    int x = i, y = h-1;
    while (y >= 0 && x < w) { 
      index[index_position++] = (y * w) + x; 
      y--; 
      x++;
    }
  }
  return index;
}
1 Like

Dear HD161693,

first of all thanks. For these wodernful pieces of processing code. I am working on a similar issue for a while now.

Question how can the pixels be sorted vertically and horizontally? Which part of the code do I need to play with to manage this? I say “play with” since I am not a programmer:)

Let’s says I have an image with only 8 colors, and I want my end result to be a vertical / horizontal color palette.

Thanks