Cutting cards in an image

Hi there,

Currently I am selling Yu-Gi-Oh! cards on Ebay and each time, I need to scan them (8 at a time).
My problem is the following : usually I trim them by hand so it’s tedious. :anguished:
Then I said to me : why not using processing to do this automatically?

This is what a scan looks like :

I tried to solve the problem with this solution :
As the background is white, I can check if a pixel is part of the background or a card.
Then I look at every pixel and I shift it to the left until there’s no more white pixels.

So I have an image with all the cards next to each other without white spaces :

Then I can divide them since I know their width and height :

Well the problem is that this is obviously not the best way to do it (there’s flickering lines).
If you have some solutions or advices, it would be great :slight_smile:

Thanks

My code :

Summary
import java.util.Arrays;

PImage img, copy;
int global_treshold = 15;
int cardWidth = 176, cardHeight = 261;

void setup() {
  size(876, 619);
  img = loadImage("yugioh_test.jpg");
  copy = createImage(img.width, img.height, RGB);
  Arrays.fill(copy.pixels, color(255));

  scaleX();
}


void draw() {
  image(copy, 0, 0);
  save("trimmed.jpg");
  int compteur = 0;
  for (int y = 0; y<2; y++) {
    for (int x = 0; x<4; x++) {
      PImage card = get(x*cardWidth, y*cardHeight, cardWidth, cardHeight);
      card.save(compteur + ".jpg");
      compteur ++;
    }
  }
  noLoop();
}

Boolean closeToWhite(color c, int treshold) {
  if (255 - (red(c)+green(c)+blue(c))/3 < treshold ) return true;
  return false;
}


void deleteBackground(int treshold) {
  img.loadPixels();
  for (int i=0; i<img.pixels.length; i++) {
    if (closeToWhite(img.pixels[i], treshold)) {
      img.pixels[i] = color(255, 0, 0);
    }
  }
  img.updatePixels();
}

void scaleX() {
  img.loadPixels();
  copy.loadPixels();
  for (int y=0; y<img.height; y++) {
    int n = 0;
    for (int x = 0; x<img.width; x++) {
      if (! closeToWhite(img.pixels[x+y*img.width], global_treshold)) {
        int xpos = x;
        while (xpos-1 >= 0 && closeToWhite(img.pixels[xpos-1+y*img.width], global_treshold) && xpos-1 >= n) xpos--;
        copy.pixels[xpos+y*img.width] = img.pixels[x+y*img.width];
        img.pixels[x+y*width] = color(255, 255, 255);
        n++;
      }
    }
  }
  copy.updatePixels();
  img.updatePixels();
}
1 Like

Hi,

You can use a flood algorithm to get (and delete) all the background.

Then use another flood algorithm for each cards. Get height and width for each one of them a use a PImage to print them out

Thanks for your answer. I just don’t understand this line :

If I use a flood algorithm to delete the background, I have only the pixels of the cards then I want to detect their positions in the image and their width and height.

Exactly.

So just after you got rid of the background you look for a non deleted pixel. It will be part of a card. So you expand the selection of that pixel to the full card (with flood algorithm). You print that card and delete it. Then you find another non deleted pixel and you start over 6 more times to have all your cards.

1 Like

I could be wrong, but you could try to use get() (https://processing.org/reference/get_.html) to cut it up into equal segments if you know that the size of each card is going to be the same.

The problem is that the cards are not always exactly at the same place.

Thanks @jb4x I’ll try to implement that in Processing and I’ll post my code. :grin:

This is what I did with my previous code.

@jb4x
The problem with the flood fill algorithm is that I have rapidly a StackOverflow error because of the recursion. As my image contains a lot of pixels (even better quality than I posted before), there’s a problem.

Hi @josephh,

I don’t have experience in the field so cannot say with certainty but it seems that your problem is a computer vision problem. And as in most computer vision problems, it starts with edge detection.

You can either implement the algorithm yourself or use a library like OpenCV. Maybe check the WarpPerspective example.

3 Likes

@solub
You are absolutly right, I didn’t think about OpenCV. I’ll do it :+1:

Hi joseph,

I made this piece of code for you. It should work.
I also tried to be as clear as I could with lots of comments but if you find things that you don’t get, please ask :slight_smile:

PImage originalPic;              // The pic with your 8 pictures
int w, h, wh;                    // Width and height of originalPic and product of the 2 variables
boolean[] pixelsUsed;            // Store the pixels that we have already used for the background or a cart
ArrayList<Integer> queuedPixels; // Store the pixels that will need to be analyzed
boolean[] pixelsAlreadySelected; // Store the pixels that have been analyzed and the one that are waiting to be analyzed in the queudPixels array
PImage[] cards;                  // Store the cards
int imgToShow;


void setup() {
  fullScreen();
  background(20);

  // Init
  originalPic = loadImage("ImgTest.jpeg");
  w = originalPic.width;
  h = originalPic.height;
  wh = w * h;
  cards = new PImage[8];
  imgToShow = 0;
  queuedPixels = new ArrayList<Integer>();  
  pixelsUsed = new boolean[wh];
  pixelsAlreadySelected = new boolean[wh];


  // Initializing the pixelAnalyzed array with false values
  for (int i = 0; i < pixelsUsed.length; i++) {
    pixelsUsed[i] = false;
  }


  // Initializing the pixelsAlreadySelected array with false values
  for (int i = 0; i < pixelsAlreadySelected.length; i++) {
    pixelsAlreadySelected[i] = false;
  }


  // Do the magic
  removeWhiteBackground();

  for (int i = 0; i < 8; i++) {
    getCard(i);
    cards[i].save("card_" + i + ".jpg");
  }

  image(cards[imgToShow], 0, 0);
}




void removeWhiteBackground() {
  // Getting rid of the white background (assuming the top right pixels is white)
  
  // Init
  queuedPixels.add(0);
  pixelsAlreadySelected[0] = true;
  
  
  while (queuedPixels.size() > 0) { // While there is some pixels to analyzed
    int idx = queuedPixels.get(0); // Get the index of the next pixel to analyzed

    if (colorDistance2(originalPic.pixels[idx], color(255, 255, 255)) <= 25) { // If that pixel is white
      pixelsUsed[idx] = true; // We now have used that pixel for the background

      if (idx - w >= 0 && pixelsAlreadySelected[idx - w] == false) { // Spread to the top
        pixelsAlreadySelected[idx - w] = true;
        queuedPixels.add(idx - w);
      }

      if ((idx % w) != (w - 1) && idx + 1 < wh && pixelsAlreadySelected[idx + 1] == false) { // Spread to the right
        pixelsAlreadySelected[idx + 1] = true;
        queuedPixels.add(idx + 1);
      }

      if (idx + w < wh && pixelsAlreadySelected[idx + w] == false) { // Spread to the bottom
        pixelsAlreadySelected[idx + w] = true;
        queuedPixels.add(idx + w);
      }

      if ((idx % w) != 0 && idx - 1 >= 0 && pixelsAlreadySelected[idx - 1] == false) { // Spread to the left
        pixelsAlreadySelected[idx - 1] = true;
        queuedPixels.add(idx -1);
      }
    }

    queuedPixels.remove(0); // Don't forget to remove the pixel from the queue
  }

  SmoothWhiteBackground();
}




// Used to avoid some issues with edge pixels that are some times black
// Consider a pixel as a background if it has pixels above and under or to the left and to the right (the dist variable set how many pixels to the top rigth bottom left we are checking)
void SmoothWhiteBackground() {
  boolean[] pixelsUsedCopy; 
  int dist = 2;
  boolean up, right, bottom, left;

  pixelsUsedCopy = new boolean[wh];
  arrayCopy(pixelsUsed, pixelsUsedCopy);

  for (int i = 0; i < wh; i++) {
    if (pixelsUsed[i] == false) {
      int x = i % w; 
      int y = i / w;

      up = false;
      right= false;
      bottom = false;
      left = false;

      // Up
      if (y < dist) {
        up = true;
      } else {
        for (int j = 0; j < dist; j++) {
          if (pixelsUsedCopy[i - (j + 1) * w] == true) {
            up = true;
          }
        }
      }

      // Right
      if (x > w - dist - 1) {
        right = true;
      } else {
        for (int j = 0; j < dist; j++) {
          if (pixelsUsedCopy[i + j + 1] == true) {
            right = true;
          }
        }
      }

      // Bottom
      if (y > h - dist - 1) {
        bottom = true;
      } else {
        for (int j = 0; j < dist; j++) {
          if (pixelsUsedCopy[i + (j + 1) * w] == true) {
            bottom = true;
          }
        }
      }

      // Up
      if (x < dist) {
        left = true;
      } else {
        for (int j = 0; j < dist; j++) {
          if (pixelsUsedCopy[i - (j + 1)] == true) {
            left = true;
          }
        }
      }

      // Update the array
      if ((up && bottom) || (left && right)) {
        pixelsUsed[i] = true;
      }
    }
  }
}




void getCard(int imgNb) {
  int xMax, xMin, yMax, yMin; // The boundary of the card

  // Init
  xMax = 0;
  xMin = w;
  yMax = 0;
  yMin = h;

  // Find the first pixel that is not part of the background or that wasn't used for another card
  int i = 0;
  while (pixelsUsed[i] == true) {
    i++;
  }

  // Reset the queue
  queuedPixels.clear();
  queuedPixels.add(i);

  // Reset pixelsAlreadySelected 
  for (int j = 0; j < pixelsAlreadySelected.length; j++) {
    pixelsAlreadySelected[j] = false;
  }
  pixelsAlreadySelected[i] = true;

  // Get the card
  while (queuedPixels.size() > 0) {
    int idx = queuedPixels.get(0);

    if (pixelsUsed[idx] == false) { // It is part of the card
      pixelsUsed[idx] = true;

      // transform idx in x, y coordinate
      int x = idx % w; 
      int y = idx / w;

      // Getting the lower and upper value
      if (x > xMax) {
        xMax = x;
      }

      if (x < xMin) {
        xMin = x;
      }

      if (y > yMax) {
        yMax = y;
      }

      if (y < yMin) {
        yMin = y;
      }

      //Spreading
      if (idx - w >= 0 && pixelsAlreadySelected[idx - w] == false) { // Spread to the top
        pixelsAlreadySelected[idx - w] = true;
        queuedPixels.add(idx - w);
      }

      if ((idx % w) != (w - 1) && idx + 1 < wh && pixelsAlreadySelected[idx + 1] == false) { // Spread to the right
        pixelsAlreadySelected[idx + 1] = true;
        queuedPixels.add(idx + 1);
      }

      if (idx + w < wh && pixelsAlreadySelected[idx + w] == false) { // Spread to the bottom
        pixelsAlreadySelected[idx + w] = true;
        queuedPixels.add(idx + w);
      }

      if ((idx % w) != 0 && idx - 1 >= 0 && pixelsAlreadySelected[idx - 1] == false) { // Spread to the left
        pixelsAlreadySelected[idx - 1] = true;
        queuedPixels.add(idx -1);
      }
    }

    queuedPixels.remove(0);
  }

  cards[imgNb] = originalPic.get(xMin, yMin, xMax - xMin, yMax - yMin);
}




int colorDistance(color col1, color col2) {
  int deltaR = (col1 >> 16 & 0xFF) - (col2 >> 16 & 0xFF);
  int deltaG = (col1 >> 8 & 0xFF)  - (col2 >> 8 & 0xFF);
  int deltaB = (col1 & 0xFF)       - (col2 & 0xFF);

  return deltaR * deltaR + deltaG * deltaG + deltaB * deltaB;
}





int colorDistance2(color col1, color col2) {
  int deltaR = (col1 >> 16 & 0xFF) - (col2 >> 16 & 0xFF);
  int deltaG = (col1 >> 8 & 0xFF)  - (col2 >> 8 & 0xFF);
  int deltaB = (col1 & 0xFF)       - (col2 & 0xFF);

  return max(abs(deltaR), abs(deltaG), abs(deltaB));
}




void draw() {
}




void mouseClicked() {
  imgToShow++;
  if (imgToShow > 7) {
    imgToShow = 0;
  }

  background(20);
  image(cards[imgToShow], 0, 0);
}

At the end, you can click the mouse to loop through all the cards.

3 Likes

And to complement @solub answer,

If you are afraid your cards will not be really straights, you can also use the previous algorithm to easily found the corners of each cards and then use open CV to make your cards completely straight.

It seems that openCV has trouble to recognize the countours of the cards (even when I played with the threshold and the blur, here it’s threshold = 130 and blur = 1) :

Thank you for giving so much time and effort for me :+1:, your code works very well and I saw that you used imperative declaration of the flood algorithm rather than using the recursive definition (stackOverflow error).

I got everything so it’s perfect :slight_smile:

Happy to hear that :slight_smile:

Unfortunately there’s an error with this image (much bigger) :

It tries to use get() with negative size in parameters :

java.lang.IllegalArgumentException: Width (-1189) and height (-1682) cannot be <= 0
	at java.awt.image.DirectColorModel.createCompatibleWritableRaster(DirectColorModel.java:1016)
	at java.awt.image.BufferedImage.<init>(BufferedImage.java:324)
	at processing.core.PImage.saveImageIO(PImage.java:3228)
	at processing.core.PImage.save(PImage.java:3406)
	at card_cut.setup(card_cut.java:59)
	at processing.core.PApplet.handleDraw(PApplet.java:2404)
	at processing.awt.PSurfaceAWT$12.callDraw(PSurfaceAWT.java:1557)
	at processing.core.PSurfaceNone$AnimationThread.run(PSurfaceNone.java:313)
Error while saving image.
java.io.IOException: image save failed.
	at processing.core.PImage.saveImageIO(PImage.java:3275)
	at processing.core.PImage.save(PImage.java:3406)
	at card_cut.setup(card_cut.java:59)
	at processing.core.PApplet.handleDraw(PApplet.java:2404)
	at processing.awt.PSurfaceAWT$12.callDraw(PSurfaceAWT.java:1557)
	at processing.core.PSurfaceNone$AnimationThread.run(PSurfaceNone.java:313)
1189 1682 -1189 -1682

Before that I changed this line of code (line 187):

while (i < pixelsUsed.length-1 && pixelsUsed[i] == true) {
    i++;
}

Because there was a IndexOutofBounds error.

I think the error comes from the fact that there’s a black line due to scanning on the top of the image so it interprets it as part of a card.

I did some debugging and found the problem.

The thing is that your white is not completely white and even with the threshold sometimes spots are left aside.

That’s I added the smoothWhiteBackground() that was getting rid of small gaps by checking if it was surrounded by close pixels. The biggest problem was with the black edge on the top between the 3 upper cards on the right.

To solve your problem I changed the distance to 5 pixels (instead of 2 pixels) and then called it 2 times at the end of the removeWhiteBackground() function instead of 1 time. It works but that’s not enough to ensure that everything will go correctly.

It will crash if you have more than 8 zones identified. What you should do is, after you remove the white background (just the main part actually), use another flood algorithm to get the size of each “air pocket” that were created. Then if the size is less than x pixels, you get rid of them, they are part of the background.

Anyway, there is the “improved” version:

PImage originalPic;              // The pic with your 8 pictures
int w, h, wh;                    // Width and height of originalPic and product of the 2 variables
boolean[] pixelsUsed;            // Store the pixels that we have already used for the background or a cart
ArrayList<Integer> queuedPixels; // Store the pixels that will need to be analyzed
boolean[] pixelsAlreadySelected; // Store the pixels that have been analyzed and the one that are waiting to be analyzed in the queudPixels array
PImage[] cards;                  // Store the cards
int imgToShow;
int xPos, yPos;


void setup() {
  //fullScreen();
  background(20);

  // Init
  originalPic = loadImage("ImgTest2.jpg");
  w = originalPic.width;
  h = originalPic.height;
  wh = w * h;
  cards = new PImage[8];
  imgToShow = 0;
  queuedPixels = new ArrayList<Integer>();  
  pixelsUsed = new boolean[wh];
  pixelsAlreadySelected = new boolean[wh];
  xPos = 0;
  yPos = 0;


  // Initializing the pixelAnalyzed array with false values
  for (int i = 0; i < pixelsUsed.length; i++) {
    pixelsUsed[i] = false;
  }


  // Initializing the pixelsAlreadySelected array with false values
  for (int i = 0; i < pixelsAlreadySelected.length; i++) {
    pixelsAlreadySelected[i] = false;
  }


  // Do the magic
  removeWhiteBackground();

  //originalPic.loadPixels();
  //for (int i = 0; i < wh; i++) {
  //  if (pixelsUsed[i]) {
  //    originalPic.pixels[i] = color(255, 0, 0);
  //  } else {
  //    originalPic.pixels[i] = color(255, 255, 255);
  //  }
  //}
  //originalPic.updatePixels();

  for (int i = 0; i < 8; i++) {
    getCard(i);
    cards[i].save("card_" + i + ".jpg");
  }

  //image(cards[imgToShow], 0, 0);
  
  println("Done");
}




void removeWhiteBackground() {
  // Getting rid of the white background (assuming the top right pixels is white)

  // Init
  queuedPixels.add(0);
  pixelsAlreadySelected[0] = true;


  while (queuedPixels.size() > 0) { // While there is some pixels to analyzed
    int idx = queuedPixels.get(0); // Get the index of the next pixel to analyzed

    if (colorDistance2(originalPic.pixels[idx], color(255, 255, 255)) <= 25) { // If that pixel is white
      pixelsUsed[idx] = true; // We now have used that pixel for the background
      //if (idx == 3547934 || idx == 3551439 || idx == 3554942 || idx == 3551437) {
      //    println(idx);
      //  }

      if (idx - w >= 0 && pixelsAlreadySelected[idx - w] == false) { // Spread to the top
        pixelsAlreadySelected[idx - w] = true;
        queuedPixels.add(idx - w);
      }

      if ((idx % w) != (w - 1) && idx + 1 < wh && pixelsAlreadySelected[idx + 1] == false) { // Spread to the right
        pixelsAlreadySelected[idx + 1] = true;
        queuedPixels.add(idx + 1);
      }

      if (idx + w < wh && pixelsAlreadySelected[idx + w] == false) { // Spread to the bottom
        pixelsAlreadySelected[idx + w] = true;
        queuedPixels.add(idx + w);
      }

      if ((idx % w) != 0 && idx - 1 >= 0 && pixelsAlreadySelected[idx - 1] == false) { // Spread to the left
        pixelsAlreadySelected[idx - 1] = true;
        queuedPixels.add(idx -1);
      }
    }

    queuedPixels.remove(0); // Don't forget to remove the pixel from the queue
  }

  SmoothWhiteBackground();
  SmoothWhiteBackground();
}




// Used to avoid some issues with edge pixels that are some times black
// Consider a pixel as a background if it has pixels above and under or to the left and to the right (the dist variable set how many pixels to the top rigth bottom left we are checking)
void SmoothWhiteBackground() {
  boolean[] pixelsUsedCopy; 
  int dist = 5;
  boolean up, right, bottom, left;

  pixelsUsedCopy = new boolean[wh];
  arrayCopy(pixelsUsed, pixelsUsedCopy);

  for (int i = 0; i < wh; i++) {
    if (pixelsUsed[i] == false) {
      int x = i % w; 
      int y = i / w;

      up = false;
      right= false;
      bottom = false;
      left = false;

      // Up
      if (y < dist) {
        up = true;
      } else {
        for (int j = 0; j < dist; j++) {
          if (pixelsUsedCopy[i - (j + 1) * w] == true) {
            up = true;
          }
        }
      }

      // Right
      if (x > w - dist - 1) {
        right = true;
      } else {
        for (int j = 0; j < dist; j++) {
          if (pixelsUsedCopy[i + j + 1] == true) {
            right = true;
          }
        }
      }

      // Bottom
      if (y > h - dist - 1) {
        bottom = true;
      } else {
        for (int j = 0; j < dist; j++) {
          if (pixelsUsedCopy[i + (j + 1) * w] == true) {
            bottom = true;
          }
        }
      }

      // Up
      if (x < dist) {
        left = true;
      } else {
        for (int j = 0; j < dist; j++) {
          if (pixelsUsedCopy[i - (j + 1)] == true) {
            left = true;
          }
        }
      }

      // Update the array
      if ((up && bottom) || (left && right)) {
        pixelsUsed[i] = true;
        //if (i == 3547934 || i == 3551439 || i == 3554942 || i == 3551437) {
        //  println(i);
        //}
      }
    }
  }
}




void getCard(int imgNb) {
  //originalPic.loadPixels();
  //color col = color(random(255), random(255), random(255));



  int xMax, xMin, yMax, yMin; // The boundary of the card

  // Init
  xMax = 0;
  xMin = w;
  yMax = 0;
  yMin = h;

  // Find the first pixel that is not part of the background or that wasn't used for another card
  int i = 0;
  while (pixelsUsed[i] == true) {
    i++;
  }

  // Reset the queue
  queuedPixels.clear();
  queuedPixels.add(i);

  // Reset pixelsAlreadySelected 
  for (int j = 0; j < pixelsAlreadySelected.length; j++) {
    pixelsAlreadySelected[j] = false;
  }
  pixelsAlreadySelected[i] = true;

  // Get the card
  while (queuedPixels.size() > 0) {
    int idx = queuedPixels.get(0);

    if (pixelsUsed[idx] == false) { // It is part of the card
      pixelsUsed[idx] = true;

      //originalPic.pixels[idx] = col;


      // transform idx in x, y coordinate
      int x = idx % w; 
      int y = idx / w;

      // Getting the lower and upper value
      if (x > xMax) {
        xMax = x;
      }

      if (x < xMin) {
        xMin = x;
      }

      if (y > yMax) {
        yMax = y;
      }

      if (y < yMin) {
        yMin = y;
      }

      //Spreading
      if (idx - w >= 0 && pixelsAlreadySelected[idx - w] == false) { // Spread to the top
        pixelsAlreadySelected[idx - w] = true;
        queuedPixels.add(idx - w);
      }

      if ((idx % w) != (w - 1) && idx + 1 < wh && pixelsAlreadySelected[idx + 1] == false) { // Spread to the right
        pixelsAlreadySelected[idx + 1] = true;
        queuedPixels.add(idx + 1);
      }

      if (idx + w < wh && pixelsAlreadySelected[idx + w] == false) { // Spread to the bottom
        pixelsAlreadySelected[idx + w] = true;
        queuedPixels.add(idx + w);
      }

      if ((idx % w) != 0 && idx - 1 >= 0 && pixelsAlreadySelected[idx - 1] == false) { // Spread to the left
        pixelsAlreadySelected[idx - 1] = true;
        queuedPixels.add(idx -1);
      }
    }

    queuedPixels.remove(0);
  }

  cards[imgNb] = originalPic.get(xMin, yMin, xMax - xMin, yMax - yMin);



  //originalPic.updatePixels();
}




int colorDistance(color col1, color col2) {
  int deltaR = (col1 >> 16 & 0xFF) - (col2 >> 16 & 0xFF);
  int deltaG = (col1 >> 8 & 0xFF)  - (col2 >> 8 & 0xFF);
  int deltaB = (col1 & 0xFF)       - (col2 & 0xFF);

  return deltaR * deltaR + deltaG * deltaG + deltaB * deltaB;
}





int colorDistance2(color col1, color col2) {
  int deltaR = (col1 >> 16 & 0xFF) - (col2 >> 16 & 0xFF);
  int deltaG = (col1 >> 8 & 0xFF)  - (col2 >> 8 & 0xFF);
  int deltaB = (col1 & 0xFF)       - (col2 & 0xFF);

  return max(abs(deltaR), abs(deltaG), abs(deltaB));
}




void draw() {
  //background(20);
  //image(originalPic, xPos, yPos, 7008, 4954);
  
  
  
  //fill(0);
  //noStroke();
  
  //int x = 3551438 % w; 
  //int y = 3551438 / w;
  //ellipse(2 * x + xPos, 2 * y + yPos, 5, 5);
}




void mouseClicked() {
  //imgToShow++;
  //if (imgToShow > 7) {
  //  imgToShow = 0;
  //}

  //background(20);
  //image(cards[imgToShow], 0, 0);
}



void keyPressed() {
  if (keyCode == 38) {
    yPos += 100;
  }
  if (keyCode == 39) {
    xPos -= 100;
  }
  if (keyCode == 40) {
    yPos -= 100;
  }
  if (keyCode == 37) {
    xPos += 100;
  }
}
2 Likes