Processing + Arduino Force Sensor Visualizing

[frzsl]

Hi everyone! I’m working with a FSR Matrix Array from Sensitronics. I’m trying to make a basic visualizer that displays a black 16x16 grid. When force is detected on the MatrixArray, we’ll shade the corresponding grid cell green, with intensity proportional applied force. I’m obtaining a 16x16 array from Arduino and I want to send this values to Processing via serial in order to visualize the force that is being applied to the sensor. I attach the array that I’m obtaining in Arduino, however I’m having some trouble to display these values in Processing in real time. Can someone help me please? Thanks! In here is an example of the visualizing made by Sensitronics, however their processing code only works for its optimized arduino code and right now I’m working with their not optimized code: https://www.sensitronics.com/tutorials/fsr-matrix-array/page8.php

// 2D Array of objects
import processing.serial.*;
Cell[][] grid;
Serial sPort;


// Number of columns and rows in the grid
int cols = 16;
int rows = 16;
int CELL_SIZE = 40;
char got_byte;
char force;

void setup() {
   int port_count = Serial.list().length;

  sPort = new Serial(this, Serial.list()[port_count - 1], 115200);  
  
  size(640,640);
  grid = new Cell[cols][rows];
  for (int i = 0; i < cols; i++) {
    for (int j = 0; j < rows; j++) {
      // Initialize each object
      grid[i][j] = new Cell(i*40,j*40,40,40,i+j);
    }
  }
}

void draw() {
  
   while(sPort.available() > 0) {
  
  background(0);
  // The counter variables i and j are also the column and row numbers and 
  // are used as arguments to the constructor for each object in the grid.  
  for (int i = 0; i < cols; i++) {
    got_byte = (char) sPort.read(); 
      print(got_byte);
    for (int j = 0; j < rows; j++) {
      
      // Oscillate and display each object
      grid[i][j].force();
      grid[i][j].display();
      
    }
  }
   }
}


// A Cell object
class Cell {
  // A cell object knows about its location in the grid 
  // as well as its size with the variables x,y,w,h
  float x,y;   // x,y location
  float w,h;   // width and height
  float angle; // angle for oscillating brightness

  // Cell Constructor
  Cell(float tempX, float tempY, float tempW, float tempH, float tempAngle) {
    x = tempX;
    y = tempY;
    w = tempW;
    h = tempH;
    angle = tempAngle;
  } 
  
  void force() {
    got_byte = force;
  }

  void display() {
    stroke(255);
    // Color calculated using sine wave
    fill(0,0,force);
    rect(x,y,w,h); 
  }
}

[RichardDL]

Hi @frzsl, This touch sensing matrix looks interesting. It’s a little tricky to help you without the same hardware and your Arduino program, but let’s try…

Your printed output from Arduino, looks like single digit numbers, with 2 spaces. Is it always like that, or does it change as the numbers get bigger? (You could grab text off the serial monitor, more useful for others to copy.)

In Processing where you read that, you’re reading individual bytes, so I think you will get sequentially the chars for 1 space space 1 etc. Suggest you make a test Processing sketch that uses readStringUntil looking for the linefeed at the end of each line. Get the test sketch to print each line of text, when that looks right think about converting it to an array of integers. I’ve done similar with text from csv file.

    String[] sNums = split(text_line, ',');
    sNums = trim(sNums);
    int[] iNums = int(sNums);

Until you get this right, suggest you modify the Ard sketch to send 1 row/Sec so you can see what’s going on. In the Ard sketch don’t use println() as it will give you a \r\n. Just use print(’\n’) where you want the line ending. Only 1 space between values. This will mess it up visually on serial monitor, but it’s easier to decode in Processing.

In Processing print your integers and check they are as expected.

Your code assumes that Ard’s transmissions are received correctly and everything stays in sync. Leave that for now until it’s seen to be a problem.

background(0) - you can’t want to clear the whole display each time you receive a character - suggest remove that for now. Each cell should redisplay when it gets a new value.

Not sure of your intention with got_byte and force. I think force should be declared inside Cell and set by force(int iforce) or cell.force = got_value.

Try some of that and say how it is getting better

[glv]

Hello @frzsl,

I don’t have the sensor so I simulated the data that it would send from the example in the code in the links you provided.

Reference for Arduino code:
https://www.sensitronics.com/tutorials/fsr-matrix-array/page3.php

I would change the Arduino code directly to CSV data terminated with a line feed.
Later you can optimize it if required.

Data sent with leading zeroes:

Simulated Data Code < Click here to open!

// Simulate Formatted Data from Sensor
// v1.0.0
// GLV 2021-06-03

//https://www.sensitronics.com/tutorials/fsr-matrix-array/page3.php
//https://www.sensitronics.com/tutorials/fsr-matrix-array/page4.php

String [] data;
int size = 16;
int dia = 50;

public void settings() 
  {  
  size(16*dia, 16*dia, P3D); 
  }

public void setup() 
  {
  data = new String [size];
  
  //Generate random data to simulate what is sent from Arduino
  println("Data from Arduino (String):");
  for (int row = 0; row<size ; row++)
    {
    String str = "";
    for (int col = 0; col<size ; col++)
      {
      int num = int(random(256)); // Random from 0 to 255
      String num1 = str.format("%" +  4 + "s", str(num)); // Pad with leading spaces
      //println(num1);
      
      str += num1;
      //println(str);
      }
     str += '\n';
     data[row] = str;
     print(str);
    }
    
  println();  
    
  // Removes spaces and replaces with a ','
  println("CSV Data from Arduino (String):");
  for (int row = 0; row<size ; row++)
    {
    String temp = data[row];
    temp = temp.replaceAll("   ",",");
    temp = temp.replaceAll("  ",",");
    temp = temp.replaceAll(" ",",");
  
  // Removes a ',' if it is first character  
    if (temp.charAt(0) == ',')
      {
      temp = temp.substring(1);
      }
    print(temp);
    data[row] = temp.trim(); // Trim the '/n' from the stirng
    }
  }

public void draw() 
  {
  background(0);
  
  // Draws a grid and adds color based of "force" value
  for(int row=0; row<size; row++)
    {
    for(int col=0; col<size; col++)
      {
      int [] data2 = int(split(data[row], ","));
      ellipseMode(CORNER);
      fill(data2[col]);
      circle(col*dia, row*dia, dia);
      }
    }
}

This is what I generated:

This is a 3D map (code not provided):

Once you get the data to Processing you can visualize any way you can imagine.
A 16x16 grid best represents this and I used color and height for the force.

Processing Serial readStringUntil() reference:
Serial::readStringUntil() \ Language (API) \ Processing 3+

:)