Image tracking processing

Processing Electronics (Arduino, etc.)
1

hi forum. i just combining processing code brightest tracking to send x val to arduino but the servo wont move. this is the code

import processing.video.*;
import processing.serial.*;  //import the serial library
Capture video;
int new_angle, this_angle, temp_angle;
Serial port;   //define a port of theSerial type

void setup() {
  size(640, 480);
    video = new Capture(this, width, height);
  video.start();  
  strokeWeight(15);  //for the graphics objects
  port = new Serial(this, "COM14", 9600);   //connect to the Arduino serial port
  noStroke();
  smooth();
}

void draw(){

   if (video.available()) {
    video.read();
    image(video, 0, 0, width, height); // Draw the webcam video onto the screen
    int brightestX = 0; // X-coordinate of the brightest video pixel
    int brightestY = 0; // Y-coordinate of the brightest video pixel
    float brightestValue = 0; // Brightness of the brightest video pixel
    // Search for the brightest pixel: For each row of pixels in the video image and
    // for each pixel in the yth row, compute each pixel's index in the video
    video.loadPixels();
    int index = 0;
    for ( int y = 0; y < video.height; y++) {
      for ( int x = 0; x < video.width; x++) {
        // Get the color stored in the pixel
        int pixelValue = video.pixels[index];
        // Determine the brightness of the pixel
        float pixelBrightness = brightness(pixelValue);
        // If that value is brighter than any previous, then store the
        // brightness of that pixel, as well as its (x,y) location
        if (pixelBrightness > brightestValue) {
          brightestValue = pixelBrightness;        
          brightestY = y;
          brightestX = x;
               float X = map(brightestX, 0, width, 10, 170);
         int Xx = round(X);
         runServo ( Xx); 
        }
        index++;
      }
    }
    // Draw a large, yellow circle at the brightest pixel
    fill(255, 204, 0, 128);
    ellipse(brightestX, brightestY, 200, 200);
                  
  }

}

int runServo (int XX){
  int angle = XX;
  this_angle = angle;
   new_angle = 90;
//background(angle_color, 0, 255);  //clear the canvas
  if (this_angle < new_angle) {
    //println("1) ", new_angle, angle);
    this_angle = this_angle + 5;
  }
  if (this_angle > new_angle) {
    //println("2) ", new_angle, angle);
    this_angle = this_angle - 5;
  }
if (temp_angle != this_angle) {
    port.write(this_angle);  //write the servo angle to the serial port
    temp_angle = this_angle;
    println(this_angle);
  }
  return angle;
}

thank you for any help.

2

Hi @doni , In you’re question there are several steps that you should think of and test separately. Write value to Arduino, Ard receive the value, Ard writes to servo, Servo moves (or not).

Suggest you add print statements to see what Processing is sending to Ard. Seeing what is going on inside the Ard is more difficult. What Ard is it? do you have any LCD or LED displays? (Simple LEDs?), TTL to serial adaptor? Test the servo with the examples sketch?

In your code starting “if (this_angle < new_angle)” you do + 5. This changes the situation for the next test " if (this_angle > new_angle)". You might want “else if” at that point. (and this_angle += 5 would make the code more compact).

3

when i execute this code the servo moves occording to this_angle var

import processing.serial.*;  //import the serial library


int new_angle, this_angle, temp_angle;
color angle_color = 200;

Serial port;   //define a port of theSerial type

void setup() {
  size(180, 180);
  port = new Serial(this, "COM14", 9600);   //connect to the Arduino serial port
}

void draw(){
    this_angle = mouseX;
   new_angle = 90;
background(angle_color, 0, 255);  //clear the canvas
  if (this_angle < new_angle) {
    //println("1) ", new_angle, angle);
    this_angle = this_angle + 5;
  }
  if (this_angle > new_angle) {
    //println("2) ", new_angle, angle);
    this_angle = this_angle - 5;
  }
 if (temp_angle != this_angle) {
    port.write(this_angle);  //write the servo angle to the serial port
    temp_angle = this_angle;
    println(this_angle);
  }
}

but when i execute following code the servo is idle


  import processing.video.*;
  import processing.serial.*;  //import the serial library
  Capture video;
  int new_angle, this_angle, temp_angle;
  Serial port;   //define a port of theSerial type
  
  void setup() {
  size(640, 480);
  video = new Capture(this, width, height);
  video.start();  
  strokeWeight(15);  //for the graphics objects
  port = new Serial(this, "COM14", 9600);   //connect to the Arduino serial port
  noStroke();
  smooth();
  }
  
  void draw(){
  
  if (video.available()) {
  video.read();
  image(video, 0, 0, width, height); // Draw the webcam video onto the screen
  int brightestX = 0; // X-coordinate of the brightest video pixel
  int brightestY = 0; // Y-coordinate of the brightest video pixel
  float brightestValue = 0; // Brightness of the brightest video pixel
  // Search for the brightest pixel: For each row of pixels in the video image and
  // for each pixel in the yth row, compute each pixel's index in the video
  video.loadPixels();
  int index = 0;
  for ( int y = 0; y < video.height; y++) {
  for ( int x = 0; x < video.width; x++) {
  // Get the color stored in the pixel
  int pixelValue = video.pixels[index];
  // Determine the brightness of the pixel
  float pixelBrightness = brightness(pixelValue);
  // If that value is brighter than any previous, then store the
  // brightness of that pixel, as well as its (x,y) location
  if (pixelBrightness > brightestValue) {
  brightestValue = pixelBrightness;        
  brightestY = y;
  brightestX = x;
      float X = brightestX;
        float H = map(X, 0, width, 10, 170);
  int Xx = round(H);
  //int angle = Xx;
  this_angle = Xx;
  new_angle = 90;
  if (this_angle < new_angle) {
  this_angle = this_angle + 5;
  }
  if (this_angle > new_angle) {
  this_angle = this_angle - 5;
  }
  if (temp_angle != this_angle) {
  port.write(this_angle);  //write the servo angle to the serial port
  temp_angle = this_angle;
  println(this_angle);
  }
  }
  index++;
  }
  }
  fill(255, 204, 0, 128);
  ellipse(brightestX, brightestY, 200, 200);  
  }
  }
4

Alright, the first code works and the server follows mouseX. In the Ard, are you using standard AnalogOut and the range is 0 to 255? So as you send mouseX values 0 to 255 moves the servo? In the second code your value is calculated from the video. What numbers do you get? Is it simply that they are > 255?

new_angle = 90; - that’s just a temporary test? The code with new_angle and this_angle - not sure why this is there, hope you’ve tested it and it does what you want for all values. e.g. if this_angle is 89 and new_angle is 90, it will +5 and then -5. (Try the syntax x += 5, makes the code more compact, easier to read.)

5

this arduino code

#include <Servo.h> 
#include <Wire.h>
#include <LiquidCrystal_PCF8574.h>

LiquidCrystal_PCF8574 lcd(0x27);  // set the LCD address to 0x27 for a 16 chars and 2 line display
Servo myservo;  
int pos = 0;    // variable to store the servo position
int ServoPin = 23; //identify which pin the servo is connected to
int ServoAngle;  //create a variable

void setup()
{
  Serial.begin(9600);
  Wire.begin();
  Wire.beginTransmission(0x27);
  lcd.begin(16, 2); // 
  myservo.attach(ServoPin);
} // setup()

void loop()
{  if (Serial.available() > 0) {
    // read the incoming byte:
    ServoAngle = Serial.read(); //read the angle that was sent from the Processing sketch
    myservo.write(ServoAngle); // move the servo to the new angle
  }
     lcd.setBacklight(255);
    lcd.home(); lcd.clear();
    lcd.print(ServoAngle);
}

and this the code of processing try to send data over serial

import processing.video.*;
import processing.serial.*;  //import the serial library
Capture video;
int new_angle, this_angle, temp_angle;
Serial port;   //define a port of theSerial type

void setup() {
  size(640, 480);
  video = new Capture(this, width, height);
  video.start();  
  strokeWeight(15);  //for the graphics objects
  port = new Serial(this, "COM14", 9600);   //connect to the Arduino serial port
  noStroke();
  smooth();
}

void draw(){
    int brightestX = 0; // X-coordinate of the brightest video pixel
    int brightestY = 0; // Y-coordinate of the brightest video pixel
  if (video.available()) {
    video.read();
    image(video, 0, 0, width, height); // Draw the webcam video onto the screen
    float brightestValue = 0; // Brightness of the brightest video pixel
    // Search for the brightest pixel: For each row of pixels in the video image and
    // for each pixel in the yth row, compute each pixel's index in the video
    video.loadPixels();
    int index = 0;
    for ( int y = 0; y < video.height; y++) {
        for ( int x = 0; x < video.width; x++) {
          int pixelValue = video.pixels[index];
          float pixelBrightness = brightness(pixelValue);
          if (pixelBrightness > brightestValue) {
              brightestValue = pixelBrightness;        
              brightestY = y;
              brightestX = x;
 
          }
          index++;
        }
    }
    fill(255, 204, 0, 128);
    ellipse(brightestX, brightestY, 200, 200);  
    }
 
    float H = map(brightestX, 0, width, 10, 170);
    int Xx = round(H);
    this_angle = Xx;
    new_angle = 90;
  
    if (this_angle < new_angle) {
      this_angle +=5;
    }else{
      this_angle -=5;
    }
    
    if (temp_angle != this_angle) {
      port.write(this_angle);  //write the servo angle to the serial port
      temp_angle = this_angle;
      println(this_angle);
    }
}

i already attaced LCD 16X2 to see what the ard receive. and the result is nothing.
but when i use this code

import processing.serial.*;  //import the serial library


int new_angle, this_angle, temp_angle;
color angle_color = 200;

Serial port;   //define a port of theSerial type

void setup() {
  size(180, 180);
  port = new Serial(this, "COM14", 9600);   //connect to the Arduino serial port
}

void draw(){
    this_angle = mouseX;
   new_angle = 90;
//background(angle_color, 0, 255);  //clear the canvas
  if (this_angle < new_angle) {
    this_angle +=5;
  }else{
  this_angle -=5;
  }
 if (temp_angle != this_angle) {
    port.write(this_angle);  //write the servo angle to the serial port
    temp_angle = this_angle;
    println(this_angle);
  }
}

the servo is moving and the lcd show the coresponding value from processing
6

I see you’ve used my suggestion on +=. Maybe you need to allow for this_angle == new_angle. Anyway, good progress with the LCD added. A good thing is that you have something that works, so the problem must be in your Processing code with the image. When you run it, what numbers does it print out? As it prints each number, does the RX light on the Ard flash?

7

in processing console it show exactly what brightestX value, but in ard LCD its shows nothing

8

yes the brightestX values shows in processing console but nothing in ard LCD

9

Good so far. Exactly what values are printing? and on Ard, is the RX light flashing?

10

in processing console it print brightestX number

11

the lcd shows nothing

12

yes the RX light flashing?

13

Yes, but what numbers are printing? are they too big?

14

no
because i do mapping

    float H = map(brightestX, 0, width, 10, 170);
    int Xx = round(H);
    this_angle = brightestX;
    new_angle = 90;
  
    if (this_angle < new_angle) {
      this_angle +=5;
    }else{
      this_angle -=5;
    }
    
    if (temp_angle != this_angle) {
      port.write(this_angle);  //write the servo angle to the serial port
      temp_angle = this_angle;
      println(this_angle);
    }
15

I’m looking at the reference for Processing serial write, and Ard serial read. The write says that it sends the variable that you give it, i.e. an int, which is 4 bytes. The read gives you the first byte, or maybe you get all 4 in turn. Hmm… but why didn’t that problem happen with the mouseX sketch? This might not be what you want finally, but try changing the value you are sending from int to byte. And make sure the values are in the range 0 to 255.

16

would you like to help me how to do it? thank you

17 
          if (temp_angle != this_angle) 
          {
            //write the servo angle to the serial port
            byte b1;
            b1 = (byte) this_angle;
            port.write(b1);            
            temp_angle = this_angle;
            println(this_angle);
          }

It’s called ‘casting’. You are telling the compiler you want to use just one byte of the int, you know what you are doing, and if the int is > 255 you want it to ignore that part of the int. Keep the values 0…255. (The other route you might want to think about is making the Ard read 4 bytes and assemble a long. Look up int and long in both references. Processing int and Ard int are not the same.)

18

so how about code in the ard side do i need to change what serial.read to byte also?

19

No. The compiler allows you to read a single byte into int (2 bytes) because you are not losing anything. For the moment, I’m going for getting a single byte of 0…255 across. Both sides working with the same. If you need more resolution, go for the 2nd scheme later. See both values printed out:

20

i’ll try and will see you when there is a progress. thank you