Intermittent ArrayIndexOutOfBoundsException reading thermopile over serial

Hello all. I’m trying to integrate the heatcam example sketches(Github) related to the Grid Eye Qwiik breakout from Sparkfun, into a bigger project, but am not able to it running reliably on it’s own. When it works, it works just fine and I am seeing the results from both the Arduino and Processing sides are operating as expected. Here are the example sketches:

Arduino
/*
  Visualizing the Panasonic Grid-EYE Sensor Data using Processing
  By: Nick Poole
  SparkFun Electronics
  Date: January 12th, 2018
  
  MIT License: Permission is hereby granted, free of charge, to any person obtaining a copy of this 
  software and associated documentation files (the "Software"), to deal in the Software without 
  restriction, including without limitation the rights to use, copy, modify, merge, publish, 
  distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the 
  Software is furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in all copies or 
  substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING 
  BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 
  NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, 
  DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
  
  Feel like supporting our work? Buy a board from SparkFun!
  https://www.sparkfun.com/products/14568
  
  This example is intended as a companion sketch to the Processing sketch found in the same folder.
  Once this code is running on your hardware, open the accompanying Processing sketch and run it 
  as well. The Processing sketch will receive the comma separated values generated by this code and
  use them to generate a thermal image. If you don't have Processing, you can download it here:
  https://processing.org/
  
  Hardware Connections:
  Attach the Qwiic Shield to your Arduino/Photon/ESP32 or other
  Plug the sensor onto the shield
*/

#include <Wire.h>
#include <SparkFun_GridEYE_Arduino_Library.h>

GridEYE grideye;

void setup() {

  // Start your preferred I2C object 
  Wire.begin();
  // Library assumes "Wire" for I2C but you can pass something else with begin() if you like
  grideye.begin();
  // Pour a bowl of serial
  Serial.begin(115200);

}

void loop() {

  // Print the temperature value of each pixel in floating point degrees Celsius
  // separated by commas 
  for(unsigned char i = 0; i < 64; i++){
    Serial.print(grideye.getPixelTemperature(i));
    Serial.print(",");
  } 

  // End each frame with a linefeed
  Serial.println();

  // Give Processing time to chew
  delay(100);

}
Processing
/*
  Visualizing the Panasonic Grid-EYE Sensor Data using Processing
  By: Nick Poole
  SparkFun Electronics
  Date: January 12th, 2018
  
  MIT License: Permission is hereby granted, free of charge, to any person obtaining a copy of this 
  software and associated documentation files (the "Software"), to deal in the Software without 
  restriction, including without limitation the rights to use, copy, modify, merge, publish, 
  distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the 
  Software is furnished to do so, subject to the following conditions:

  The above copyright notice and this permission notice shall be included in all copies or 
  substantial portions of the Software.

  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING 
  BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 
  NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, 
  DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
  
  Feel like supporting our work? Buy a board from SparkFun!
  https://www.sparkfun.com/products/14568
  
  This example is intended as a companion sketch to the Arduino sketch found in the same folder.
  Once the accompanying code is running on your hardware, run this Processing sketch. 
  This Processing sketch will receive the comma separated values generated by the Arduino code and
  use them to generate a thermal image.
  
  Hardware Connections:
  Attach the Qwiic Shield to your Arduino/Photon/ESP32 or other
  Plug the sensor onto the shield
*/

import processing.serial.*;

String myString = null;
Serial myPort;  // The serial port

float[] temps =  {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0};

// The statements in the setup() function 
// execute once when the program begins
void setup() {
  size(400, 400);  // Size must be the first statement
  noStroke();
  frameRate(30);
  
  // Print a list of connected serial devices in the console
  printArray(Serial.list());
  // Depending on where your GridEYE falls on this list, you
  // may need to change Serial.list()[0] to a different number
  myPort = new Serial(this, Serial.list()[2], 115200);
  myPort.clear();
  // Throw out the first chunk in case we caught it in the 
  // middle of a frame
  myString = myPort.readStringUntil(13);
  myString = null;
  // change to HSB color mode, this will make it easier to color
  // code the temperature data
  colorMode(HSB, 360, 100, 100);
}

// The statements in draw() are executed until the 
// program is stopped. Each statement is executed in 
// sequence and after the last line is read, the first 
// line is executed again.
void draw() { 
  
  // When there is a sizeable amount of data on the serial port
  // read everything up to the first linefeed
  if(myPort.available() > 64){
    //delay(100);
  myString = myPort.readStringUntil(13);
  
  // generate an array of strings that contains each of the comma
  // separated values
  if (myString !=null) // had to enclose the following in this if statement to solve null pointer exception errors
  {
  String splitString[] = splitTokens(myString, ",");
  printArray(splitString);
  // for each of the 64 values, map the temperatures between 20C and 40C
  // to the blue through red portion of the color space
  for(int q = 0; q < 64; q++){
   
    temps[q] = map(float(splitString[q]), 35, 39, 100, 360);
    
  }
  }
  }
  
  
  // Prepare variables needed to draw our heatmap
  int x = 0;
  int y = 0;
  int i = 0;
  background(0);   // Clear the screen with a black background
  
  
  // each GridEYE pixel will be represented by a 50px square: 
  // because 50 x 8 = 400, we draw squares until our y location
  // is 400
  while(y < 400){
  
    
  // for each increment in the y direction, draw 8 boxes in the 
  // x direction, creating a 64 pixel matrix
  while(x < 400){
  // before drawing each pixel, set our paintcan color to the 
  // appropriate mapped color value
  fill(temps[i], 100, 100);
  rect(x,y,50,50);
  x = x + 50;
  i++;
  }
  
  y = y + 50;
  x = 0;
  }
  
  // Add a gaussian blur to the canvas in order to create a rough
  // visual interpolation between pixels.
  filter(BLUR,10);
}

However, I have to start the Processing sketch many times before it does behave- I have probably a 30% success rate every time. When it does not work, I get the following error (Not always 23):

map(NaN, 35, 39, 100, 360) called, which returns NaN (not a number)
ArrayIndexOutOfBoundsException: 23

I put a printArray right after the String splitString[] = splitTokens(myString, ",");
And this is what is in the console when it throws the error:

[0] "COM1"
[1] "COM3"
[2] "COM4"
[0] "16.25"
[1] "16.75"
[2] "16.25"
[3] "16.00"
[4] ".00"
[5] "17.50"
[6] "17.50"
[7] "17.25"
[8] "17.50"
[9] "18.00"
[10] "18.50"
[11] "18.50"
[12] "18.75"
[13] "19.50"
[14] "18.00"
[15] "18.00"
[16] "17.25"
[17] "18.00"
[18] "18.00"
[19] "18.00"
[20] "19.00"
[21] "19.00"
[22] "
"
map(NaN, 35, 39, 100, 360) called, which returns NaN (not a number)
ArrayIndexOutOfBoundsException: 23

I’ve tried solutions from the issues section of the github as well as some from other posts both here and in the Arduino forum but I just can’t figure out what is wrong.

1 Like

One possibility is to change the loop on the Processing side:

for(int q = 0; q < (splitString.length-1); q++){
    temps[q] = map(float(splitString[q]), 35, 39, 100, 360);
}

A more robust solution would be to check against Float.NaN after a float conversion.
Also, I use length-1 because the end-line character is included in the array (index 22 in your example).
While it will convert to a valid number, it will not be an accurate temperature.

1 Like

Hi

Maybe this helps

Hello @The_Wolf_of_Walmart,

Take a look here:
https://discourse.processing.org/t/heatcam-examples-please-help/29390/4

I had the same reaction (below) to this as I did in my post above!

Have fun!

:)

I will give that a try this afternoon and see how it goes- fingers crossed.

Could you elaborate a little more on what that might look like?

@jafal & @glv : thank you for your replies. I spent a good bit of time implementing the suggestions from that post but didn’t have any success. Perhaps I can give it another try and post the code in case I didn’t do it correctly.

@glv Actually, it looks like I still have that code open on my laptop. Here is what I had on Processing side when I tried your suggestion:

Processing

import processing.serial.*;

String myString = null;
Serial myPort;  // The serial port

//See if you can isntantiate the temp array with [64] instead of 64x 0's
float[] temps =  {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
int comPort = 1;    //Change active comport here

// The statements in the setup() function 
// execute once when the program begins
void setup() {

  size(400, 400);  // Size must be the first statement
  noStroke();
  frameRate(30);

  // Print a list of connected serial devices in the console
  printArray(Serial.list());

  // Depending on where your GridEYE falls on this list, you
  // may need to change Serial.list()[0] to a different number
  myPort = new Serial(this, Serial.list()[comPort], 115200);
  myPort.clear();
  // Throw out the first chunk in case we caught it in the 
  // middle of a frame
  myString = myPort.readStringUntil(13);
  myString = null;

  // change to HSB color mode, this will make it easier to color
  // code the temperature data
  colorMode(HSB, 360, 100, 100);
}

// The statements in draw() are executed until the 
// program is stopped. Each statement is executed in 
// sequence and after the last line is read, the first 
// line is executed again.
void draw() { 

  // When there is a sizeable amount of data on the serial port
  // read everything up to the first linefeed
  if (myPort.available() > 64) {
    myString = myPort.readStringUntil(13);    //13 = carriage return  or \r

    if (myString !=null) // had to enclose the following in this if statement to solve null pointer exception errors
    {
      float frameMin; //lowest temp in frame
      float frameMax; //highest temp in frame
      float frameAvg; //avg temp of frame
      float avgDelta; //average of change of each pixel from it's previous reading
      float sumDelta; //total amount of pixel change since last frame
      int hottestPixel; //XY of hottest pixel in frame

      // generate an array of strings that contains each of the comma
      // separated values


      trim(myString);
      String splitString[] = splitTokens(myString, ",");
      printArray(splitString);
      if (splitString.length == 65)
      {

        // for each of the 64 values, map the temperatures between 20C and 40C
        // to the blue through red portion of the color space
        for (int q = 0; q < 64; q++) {

          // orig temps[q] = map(float(splitString[q]), 20, 40, 240, 360);

          // This controls the contrast sesitivity/temp resolution, arguments 2 & 3 should probably be average min/max
          // of that observed during given stage of sleep
          temps[q] = map(float(splitString[q]), 35, 39, 100, 360);    //temps variable = temp scaled to color range
        }
      }
    }


    // Prepare variables needed to draw our heatmap
    int x = 0;
    int y = 0;
    int i = 0;
    background(0);   // Clear the screen with a black background


    // each GridEYE pixel will be represented by a 50px square: 
    // because 50 x 8 = 400, we draw squares until our y location
    // is 400
    while (y < 400) {


      // for each increment in the y direction, draw 8 boxes in the 
      // x direction, creating a 64 pixel matrix
      while (x < 400) {
        // before drawing each pixel, set our paintcan color to the 
        // appropriate mapped color value
        fill(temps[i], 100, 100);
        rect(x, y, 50, 50);
        x = x + 50;
        i++;
      }

      y = y + 50;
      x = 0;
    }

    // Add a gaussian blur to the canvas in order to create a rough
    // visual interpolation between pixels.
    filter(BLUR, 10);
  } 
}

Hello @The_Wolf_of_Walmart ,

Seems to work with this Arduino test code:

Arduino Code
int count;

void setup() 
  {
  Serial.begin(115200);
  delay(1000);
  }

void loop() 
  {
  Serial.print(count++); // First element is a counter! Useful when debugging code.
  Serial.print(",");
  for(unsigned char i = 0; i < 63; i++){
    Serial.print((i+1)*1.01);
    Serial.print(",");
  } 
  Serial.println();
  delay(500);
  }

I got this from the numbers I was sending:

image

My use of trim in topic provided was incorrect!
I spotted that right away! Live and learn…
It did not matter in this case since we did not use last element of array.
Will go back and fix it in original post.

This is correct:

myString = trim(myString);
String splitString[] = splitTokens(myString, ",");
printArray(splitString);
if (splitString.length == 64)

Another topic that may be of interest:

You may have been experiencing some “garbage” in the buffers if you were just stopping and running Processing.
There are better ways to do things like handshaking between devices… when you are ready.

:)

1 Like

You can do something like:

float value = float(splitString[q]);
if (Float.NaN != value)
{
  temps[q] = map(value, 35, 39, 100, 360);
}
2 Likes

Hi @The_Wolf_of_Walmart, Re getting things to work reliably. On a couple of projects I’ve found that the Mega does not like receiving serial when it’s not ready. See this topic.

1 Like

I made your suggested changes to the code again and it’s working great! :slight_smile: Thank you very much. Thank you for the link as well- my next step is to integrate accelerometer and RTC, so I’m sure I will be referring to it.