# Image tracking processing

**URL:** <https://discourse.processing.org/t/image-tracking-processing/34864>\
**Category:** Electronics (Arduino, etc.)\
**Created:** [January 27, 2022, 6:10pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864 "2022-01-27T18:10:03Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 27, 2022, 6:10pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/1 "2022-01-27T18:10:03Z")

</div>

hi forum. i just combining processing code brightest tracking to send x val to arduino but the servo wont move. this is the code

```auto
import processing.video.*;
import processing.serial.*; //import the serial library
Capture video;
int new_angle, this_angle, temp_angle;
Serial port; //define a port of theSerial type

void setup() {
  size(640, 480);
    video = new Capture(this, width, height);
  video.start();  
  strokeWeight(15); //for the graphics objects
  port = new Serial(this, "COM14", 9600); //connect to the Arduino serial port
  noStroke();
  smooth();
}

void draw(){

   if (video.available()) {
    video.read();
    image(video, 0, 0, width, height); // Draw the webcam video onto the screen
    int brightestX = 0; // X-coordinate of the brightest video pixel
    int brightestY = 0; // Y-coordinate of the brightest video pixel
    float brightestValue = 0; // Brightness of the brightest video pixel
    // Search for the brightest pixel: For each row of pixels in the video image and
    // for each pixel in the yth row, compute each pixel's index in the video
    video.loadPixels();
    int index = 0;
    for ( int y = 0; y < video.height; y++) {
      for ( int x = 0; x < video.width; x++) {
        // Get the color stored in the pixel
        int pixelValue = video.pixels[index];
        // Determine the brightness of the pixel
        float pixelBrightness = brightness(pixelValue);
        // If that value is brighter than any previous, then store the
        // brightness of that pixel, as well as its (x,y) location
        if (pixelBrightness > brightestValue) {
          brightestValue = pixelBrightness;        
          brightestY = y;
          brightestX = x;
               float X = map(brightestX, 0, width, 10, 170);
         int Xx = round(X);
         runServo ( Xx); 
        }
        index++;
      }
    }
    // Draw a large, yellow circle at the brightest pixel
    fill(255, 204, 0, 128);
    ellipse(brightestX, brightestY, 200, 200);
                  
  }

}

int runServo (int XX){
  int angle = XX;
  this_angle = angle;
   new_angle = 90;
//background(angle_color, 0, 255); //clear the canvas
  if (this_angle < new_angle) {
    //println("1) ", new_angle, angle);
    this_angle = this_angle + 5;
  }
  if (this_angle > new_angle) {
    //println("2) ", new_angle, angle);
    this_angle = this_angle - 5;
  }
if (temp_angle != this_angle) {
    port.write(this_angle); //write the servo angle to the serial port
    temp_angle = this_angle;
    println(this_angle);
  }
  return angle;
}

```

thank you for any help.

---

<div class="post-metadata">

**Author:** ![RichardDL](https://avatars.discourse-cdn.com/v4/letter/r/f475e1/32.png) [@RichardDL](https://discourse.processing.org/u/RichardDL)\
**Post date:** [January 28, 2022, 5:53pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/2 "2022-01-28T17:53:15Z")

</div>

Hi @doni , In you’re question there are several steps that you should think of and test separately. Write value to Arduino, Ard receive the value, Ard writes to servo, Servo moves (or not).

Suggest you add print statements to see what Processing is sending to Ard. Seeing what is going on inside the Ard is more difficult. What Ard is it? do you have any LCD or LED displays? (Simple LEDs?), TTL to serial adaptor? Test the servo with the examples sketch?

In your code starting “if (this\_angle \< new\_angle)” you do + 5. This changes the situation for the next test " if (this\_angle \> new\_angle)". You might want “else if” at that point. (and this\_angle += 5 would make the code more compact).

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 4:05am UTC](https://discourse.processing.org/t/image-tracking-processing/34864/3 "2022-01-29T04:05:16Z")

</div>

> [@RichardDL](#):
>
> Hi @doni , In you’re question there are several steps that you should think of and test separately. Write value to Arduino, Ard receive the value, Ard writes to servo, Servo moves (or not).

when i execute this code the servo moves occording to this\_angle var

```auto
import processing.serial.*; //import the serial library

int new_angle, this_angle, temp_angle;
color angle_color = 200;

Serial port; //define a port of theSerial type

void setup() {
  size(180, 180);
  port = new Serial(this, "COM14", 9600); //connect to the Arduino serial port
}

void draw(){
    this_angle = mouseX;
   new_angle = 90;
background(angle_color, 0, 255); //clear the canvas
  if (this_angle < new_angle) {
    //println("1) ", new_angle, angle);
    this_angle = this_angle + 5;
  }
  if (this_angle > new_angle) {
    //println("2) ", new_angle, angle);
    this_angle = this_angle - 5;
  }
 if (temp_angle != this_angle) {
    port.write(this_angle); //write the servo angle to the serial port
    temp_angle = this_angle;
    println(this_angle);
  }
}

but when i execute following code the servo is idle

```

```
  import processing.video.*;
  import processing.serial.*; //import the serial library
  Capture video;
  int new_angle, this_angle, temp_angle;
  Serial port; //define a port of theSerial type
  
  void setup() {
  size(640, 480);
  video = new Capture(this, width, height);
  video.start();  
  strokeWeight(15); //for the graphics objects
  port = new Serial(this, "COM14", 9600); //connect to the Arduino serial port
  noStroke();
  smooth();
  }
  
  void draw(){
  
  if (video.available()) {
  video.read();
  image(video, 0, 0, width, height); // Draw the webcam video onto the screen
  int brightestX = 0; // X-coordinate of the brightest video pixel
  int brightestY = 0; // Y-coordinate of the brightest video pixel
  float brightestValue = 0; // Brightness of the brightest video pixel
  // Search for the brightest pixel: For each row of pixels in the video image and
  // for each pixel in the yth row, compute each pixel's index in the video
  video.loadPixels();
  int index = 0;
  for ( int y = 0; y < video.height; y++) {
  for ( int x = 0; x < video.width; x++) {
  // Get the color stored in the pixel
  int pixelValue = video.pixels[index];
  // Determine the brightness of the pixel
  float pixelBrightness = brightness(pixelValue);
  // If that value is brighter than any previous, then store the
  // brightness of that pixel, as well as its (x,y) location
  if (pixelBrightness > brightestValue) {
  brightestValue = pixelBrightness;        
  brightestY = y;
  brightestX = x;
      float X = brightestX;
        float H = map(X, 0, width, 10, 170);
  int Xx = round(H);
  //int angle = Xx;
  this_angle = Xx;
  new_angle = 90;
  if (this_angle < new_angle) {
  this_angle = this_angle + 5;
  }
  if (this_angle > new_angle) {
  this_angle = this_angle - 5;
  }
  if (temp_angle != this_angle) {
  port.write(this_angle); //write the servo angle to the serial port
  temp_angle = this_angle;
  println(this_angle);
  }
  }
  index++;
  }
  }
  fill(255, 204, 0, 128);
  ellipse(brightestX, brightestY, 200, 200);  
  }
  }

```

```auto

```

---

<div class="post-metadata">

**Author:** ![RichardDL](https://avatars.discourse-cdn.com/v4/letter/r/f475e1/32.png) [@RichardDL](https://discourse.processing.org/u/RichardDL)\
**Post date:** [January 29, 2022, 12:19pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/4 "2022-01-29T12:19:05Z")

</div>

Alright, the first code works and the server follows mouseX. In the Ard, are you using standard AnalogOut and the range is 0 to 255? So as you send mouseX values 0 to 255 moves the servo? In the second code your value is calculated from the video. What numbers do you get? Is it simply that they are \> 255?

new\_angle = 90; - that’s just a temporary test? The code with new\_angle and this\_angle - not sure why this is there, hope you’ve tested it and it does what you want for all values. e.g. if this\_angle is 89 and new\_angle is 90, it will +5 and then -5. (Try the syntax x += 5, makes the code more compact, easier to read.)

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 1:53pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/5 "2022-01-29T13:53:36Z")

</div>

this arduino code

```auto
#include <Servo.h> 
#include <Wire.h>
#include <LiquidCrystal_PCF8574.h>

LiquidCrystal_PCF8574 lcd(0x27); // set the LCD address to 0x27 for a 16 chars and 2 line display
Servo myservo;  
int pos = 0; // variable to store the servo position
int ServoPin = 23; //identify which pin the servo is connected to
int ServoAngle; //create a variable

void setup()
{
  Serial.begin(9600);
  Wire.begin();
  Wire.beginTransmission(0x27);
  lcd.begin(16, 2); // 
  myservo.attach(ServoPin);
} // setup()

void loop()
{ if (Serial.available() > 0) {
    // read the incoming byte:
    ServoAngle = Serial.read(); //read the angle that was sent from the Processing sketch
    myservo.write(ServoAngle); // move the servo to the new angle
  }
     lcd.setBacklight(255);
    lcd.home(); lcd.clear();
    lcd.print(ServoAngle);
} 

```

and this the code of processing try to send data over serial

```auto
import processing.video.*;
import processing.serial.*; //import the serial library
Capture video;
int new_angle, this_angle, temp_angle;
Serial port; //define a port of theSerial type

void setup() {
  size(640, 480);
  video = new Capture(this, width, height);
  video.start();  
  strokeWeight(15); //for the graphics objects
  port = new Serial(this, "COM14", 9600); //connect to the Arduino serial port
  noStroke();
  smooth();
}

void draw(){
    int brightestX = 0; // X-coordinate of the brightest video pixel
    int brightestY = 0; // Y-coordinate of the brightest video pixel
  if (video.available()) {
    video.read();
    image(video, 0, 0, width, height); // Draw the webcam video onto the screen
    float brightestValue = 0; // Brightness of the brightest video pixel
    // Search for the brightest pixel: For each row of pixels in the video image and
    // for each pixel in the yth row, compute each pixel's index in the video
    video.loadPixels();
    int index = 0;
    for ( int y = 0; y < video.height; y++) {
        for ( int x = 0; x < video.width; x++) {
          int pixelValue = video.pixels[index];
          float pixelBrightness = brightness(pixelValue);
          if (pixelBrightness > brightestValue) {
              brightestValue = pixelBrightness;        
              brightestY = y;
              brightestX = x;
 
          }
          index++;
        }
    }
    fill(255, 204, 0, 128);
    ellipse(brightestX, brightestY, 200, 200);  
    }
 
    float H = map(brightestX, 0, width, 10, 170);
    int Xx = round(H);
    this_angle = Xx;
    new_angle = 90;
  
    if (this_angle < new_angle) {
      this_angle +=5;
    }else{
      this_angle -=5;
    }
    
    if (temp_angle != this_angle) {
      port.write(this_angle); //write the servo angle to the serial port
      temp_angle = this_angle;
      println(this_angle);
    }
}

```

i already attaced LCD 16X2 to see what the ard receive. and the result is nothing.  
but when i use this code

```auto
import processing.serial.*; //import the serial library

int new_angle, this_angle, temp_angle;
color angle_color = 200;

Serial port; //define a port of theSerial type

void setup() {
  size(180, 180);
  port = new Serial(this, "COM14", 9600); //connect to the Arduino serial port
}

void draw(){
    this_angle = mouseX;
   new_angle = 90;
//background(angle_color, 0, 255); //clear the canvas
  if (this_angle < new_angle) {
    this_angle +=5;
  }else{
  this_angle -=5;
  }
 if (temp_angle != this_angle) {
    port.write(this_angle); //write the servo angle to the serial port
    temp_angle = this_angle;
    println(this_angle);
  }
}

the servo is moving and the lcd show the coresponding value from processing

```

---

<div class="post-metadata">

**Author:** ![RichardDL](https://avatars.discourse-cdn.com/v4/letter/r/f475e1/32.png) [@RichardDL](https://discourse.processing.org/u/RichardDL)\
**Post date:** [January 29, 2022, 2:06pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/6 "2022-01-29T14:06:40Z")

</div>

I see you’ve used my suggestion on +=. Maybe you need to allow for this\_angle == new\_angle. Anyway, good progress with the LCD added. A good thing is that you have something that works, so the problem must be in your Processing code with the image. When you run it, what numbers does it print out? As it prints each number, does the RX light on the Ard flash?

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 2:09pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/7 "2022-01-29T14:09:08Z")

</div>

in processing console it show exactly what brightestX value, but in ard LCD its shows nothing

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 2:11pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/8 "2022-01-29T14:11:52Z")

</div>

> [@RichardDL](#):
>
> As it prints each number, does the RX light on the Ard flash?

yes the brightestX values shows in processing console but nothing in ard LCD

---

<div class="post-metadata">

**Author:** ![RichardDL](https://avatars.discourse-cdn.com/v4/letter/r/f475e1/32.png) [@RichardDL](https://discourse.processing.org/u/RichardDL)\
**Post date:** [January 29, 2022, 2:14pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/9 "2022-01-29T14:14:15Z")

</div>

Good so far. Exactly what values are printing? and on Ard, is the RX light flashing?

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 2:20pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/10 "2022-01-29T14:20:33Z")

</div>

> [@RichardDL](#):
>
> Exactly what values are printing?

in processing console it print brightestX number

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 2:21pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/11 "2022-01-29T14:21:09Z")

</div>

> [@RichardDL](#):
>
> and on Ard

the lcd shows nothing

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 2:22pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/12 "2022-01-29T14:22:09Z")

</div>

> [@RichardDL](#):
>
> is the RX light flashing?

yes the RX light flashing?

---

<div class="post-metadata">

**Author:** ![RichardDL](https://avatars.discourse-cdn.com/v4/letter/r/f475e1/32.png) [@RichardDL](https://discourse.processing.org/u/RichardDL)\
**Post date:** [January 29, 2022, 2:33pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/13 "2022-01-29T14:33:25Z")

</div>

Yes, but what numbers are printing? are they too big?

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 2:35pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/14 "2022-01-29T14:35:22Z")

</div>

no  
because i do mapping

```auto
    float H = map(brightestX, 0, width, 10, 170);
    int Xx = round(H);
    this_angle = brightestX;
    new_angle = 90;
  
    if (this_angle < new_angle) {
      this_angle +=5;
    }else{
      this_angle -=5;
    }
    
    if (temp_angle != this_angle) {
      port.write(this_angle); //write the servo angle to the serial port
      temp_angle = this_angle;
      println(this_angle);
    }

```

---

<div class="post-metadata">

**Author:** ![RichardDL](https://avatars.discourse-cdn.com/v4/letter/r/f475e1/32.png) [@RichardDL](https://discourse.processing.org/u/RichardDL)\
**Post date:** [January 29, 2022, 2:47pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/15 "2022-01-29T14:47:39Z")

</div>

I’m looking at the reference for Processing serial write, and Ard serial read. The write says that it sends the variable that you give it, i.e. an int, which is 4 bytes. The read gives you the first byte, or maybe you get all 4 in turn. Hmm… but why didn’t that problem happen with the mouseX sketch? This might not be what you want finally, but try changing the value you are sending from int to byte. And make sure the values are in the range 0 to 255.

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 2:57pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/16 "2022-01-29T14:57:08Z")

</div>

would you like to help me how to do it? thank you

---

<div class="post-metadata">

**Author:** ![RichardDL](https://avatars.discourse-cdn.com/v4/letter/r/f475e1/32.png) [@RichardDL](https://discourse.processing.org/u/RichardDL)\
**Post date:** [January 29, 2022, 3:04pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/17 "2022-01-29T15:04:51Z")

</div>

```auto
          if (temp_angle != this_angle) 
          {
            //write the servo angle to the serial port
            byte b1;
            b1 = (byte) this_angle;
            port.write(b1);            
            temp_angle = this_angle;
            println(this_angle);
          }

```

It’s called ‘casting’. You are telling the compiler you want to use just one byte of the int, you know what you are doing, and if the int is \> 255 you want it to ignore that part of the int. Keep the values 0…255. (The other route you might want to think about is making the Ard read 4 bytes and assemble a long. Look up int and long in both references. Processing int and Ard int are not the same.)

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 3:06pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/18 "2022-01-29T15:06:17Z")

</div>

so how about code in the ard side do i need to change what serial.read to byte also?

---

<div class="post-metadata">

**Author:** ![RichardDL](https://avatars.discourse-cdn.com/v4/letter/r/f475e1/32.png) [@RichardDL](https://discourse.processing.org/u/RichardDL)\
**Post date:** [January 29, 2022, 3:10pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/19 "2022-01-29T15:10:09Z")

</div>

No. The compiler allows you to read a single byte into int (2 bytes) because you are not losing anything. For the moment, I’m going for getting a single byte of 0…255 across. Both sides working with the same. If you need more resolution, go for the 2nd scheme later. See both values printed out:

> [@RichardDL](#):
>
> ` println(String.format("%6d %4d", this_angle, b1));`

---

<div class="post-metadata">

**Author:** ![doni](https://avatars.discourse-cdn.com/v4/letter/d/a698b9/32.png) [@doni](https://discourse.processing.org/u/doni)\
**Post date:** [January 29, 2022, 3:16pm UTC](https://discourse.processing.org/t/image-tracking-processing/34864/20 "2022-01-29T15:16:15Z")

</div>

> [@RichardDL](#):
>
> (The other route you might want to think about is making the Ard read 4 bytes and assemble a long. Look up int and long in both references. Processing int and Ard int are not the same.)

i’ll try and will see you when there is a progress. thank you

[Next page](https://discourse.processing.org/t/image-tracking-processing/34864.md?page=2)