Serial plotter over bluetooth

Hi,
I’m trying to create a portable ECG with ESP32, AD8232 and battery 18650. I modified a sketch for arduino and processing pde to use bluetooth communication instead of serial over usb.

My Arduino sketch work perfectly with Arduini Serial monitor. I’m able to plt using both usb serial communication and bluetooth one and in both cases the resolution is fine (baud rate is 115200).

When I use Processing sketch, the issue pops-up. Using serial over usb the plot is quite acceptable while with bluetooth the resolution is poor. Here below the *.pde:

Is there any known issue on serial bluetooth in Processing?

import processing.serial.*;

Serial myPort;        // The serial port

int xPos = 5;         // horizontal position of the graph
float height_old = 0;
float height_new = 0;
float inByte = 0;


void setup () {
  // set the window size:
  size(1600, 900);        

  // List all the available serial ports
  printArray(Serial.list());
  // Open whatever port is the one you're using.
  myPort = new Serial(this, Serial.list()[0], 115200);
  // don't generate a serialEvent() unless you get a newline character:
  myPort.bufferUntil('\n');
  // set inital background:
  background(0xff);
}


void draw () {
  //Map and draw the line for new data point
  //    inByte = map(inByte, 0, 5000, 0, height);

  height_new = height - inByte; 
  line(xPos - 5, height_old, xPos, height_new);
  height_old = height_new;

  // at the edge of the screen, go back to the beginning:
  if (xPos >= width) {
    xPos = 0;
    background(0xff);
  } else {
    // increment the horizontal position:
    //xPos++;
    xPos = xPos + 5;
  }
}

void serialEvent (Serial myPort) {
  // get the ASCII string:
  String inString = myPort.readStringUntil('\n');

  if (inString != null) {
    // trim off any whitespace:
    inString = trim(inString);

    // If leads off detection is true notify with blue line
    if (inString.equals("!")) { 
      stroke(0, 0, 0xff); //Set stroke to blue ( R, G, B)
      inByte = 2048;  // middle of the ADC range (Flat Line)
    }
    // If the data is good let it through
    else {
      stroke(0xff, 0, 0); //Set stroke to red ( R, G, B)
      inByte = float(inString);
    }
    //    inByte = map(inByte, 0, 4200, 0, height);
    inByte = inByte * 0.2;
  }
}

Arduino sketch is below:

 
#include "BluetoothSerial.h"
BluetoothSerial SerialBT;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  SerialBT.begin("ECG"); //Bluetooth device name
  pinMode(23, INPUT); // Setup for leads off detection LO +
  pinMode(33, INPUT); // Setup for leads off detection LO -

}


void loop() {
  
  if((digitalRead(23) == 1)||(digitalRead(33) == 1)){
    SerialBT.println('!');
    Serial.println('!');
  }
  else{
    SerialBT.println(analogRead(4));
    Serial.println(analogRead(4));
  }
  delay(1);
}

The pde attached above uses baudrate 9600. As mentioned, I tried with 115200 with same results

Hello,

Please format your code for readability and so we can copy it easily:
https://discourse.processing.org/faq#format-your-code

Try to cut and paste your code into Processing and you will understand.

When I hover over this code I can use the copy icon that pops up in upper right.

Example:

void setup() 
	{
  size(700, 150);
  background(0);
  fill(0, 255, 0);
  textAlign(CENTER, CENTER);
  textSize(48);
  text("Please format your code. :)", width/2, height/2-10);
	}

I have never had any issues with Arduino and Processing using serial over BT with an HC-05; it if works with USB serial it works with BT serial.

I may look at code once it is formatted.

Please include Arduino code; this can be minimal code with just print statements.

Visual example only:
Arduino to Processing

:)

Thanks for your tips, I’m a noob :slight_smile:

Bluetooth connects very fast and in Arduino serial plotter the resolution over bluetooth is ok…issue is in processing.
I tried to print a csv in processing with serial data, using both serial over usb and bluetooth. In the first case I got about 300 datapoints while in the second about 80…same time range.

Hi @alexthesmeus

I don’t think Processing knows what type of Serial device you have. It just connects to a port and gets what arrives.

The baud rate is between the port and the BT device. (You’re using a Windows PC? and its built-in BT device?) Bluetooth does it’s own thing with protocol and speed. The baud rates at either end don’t have to be the same. I have 2 Ards talking to their HC06s at 9600. Then an Ard using 38400 baud to an HC05 talks BT to them (in turn).

Speculative guesses: Using USB serial the data arrives at an even rate, but with BT it’s arriving in at varying rate, and your Processing sketch sometimes doesn’t keep up. And/or maybe the serial monitor handles missing values in a way that doesn’t show.

Your Arduino sketch has delay(1), so you’re asking for 1000 measurements/Sec. Do you need that much? Say max pulse rate is 180, 3 Hz, would 100 values give a good enough trace? Change the delay to 3 might allow everything to keep up.

Most of us use serial in a simple obvious way, but if the rate is high and the media is unreliable, we should be using some or all of: checksums, message numbers, acknowledgements, re-sends, error counters…

This code:

    SerialBT.println(analogRead(4));
    Serial.println(analogRead(4));

Would be better as:

int iValu;

iValu = analogRead(4);
SerialBT.println(iValu);
Serial.println(iValu);
  • don’t make the hardware do more than necessary.
  • be sure the same value is being sent to both outputs.

.

Hello,

I wrote some code to illustrate what is happening.

Processing:
Draw updates 60 fps:
.
frequency f = 60Hz
Period T= 16.67 ms

If you set the Arduino delay below 18 ms (try this incrementally) it will start skipping data points to draw on the Processing side.

Modify and run my example.
I substituted another serial port for this on my side instead of BlueTooth.

Arduino Monitor:
Updates in real time as data is received.

Arduino sketch:
int counter;

void setup() 
  {
  Serial.begin(9600);
  Serial3.begin(9600);
  }
  
void loop() 
  {      
  Serial.println(counter);
  Serial3.println(counter);
  counter++;
  if (counter > 255) counter = 0;
  delay(18);
  }
Processing Sketch
import processing.serial.*;
Serial myPort;        

int xPos = 5;         
int inByte = 0;
int inLast = 0;

boolean flag = false;

void setup () 
  {
  size(600, 300);        

  // List all the available serial ports
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[0], 9600);
  myPort.bufferUntil('\n');
  background(0);
  strokeWeight(1);
  stroke(255, 255, 0);
  }


void draw () 
  {
  if (flag)
    {
    println(inByte);
    if (inByte -inLast > 1) println("Missing data!");
    inLast = inByte;
    if (xPos > width) 
      {
      xPos = 0;
      background(0);
      } 
    else 
      {
      xPos = xPos + 1;  
      point(xPos, height - inByte);
      }
    flag = false;
    } 
  }

void serialEvent (Serial myPort) 
  {
  String inString = myPort.readString();
  //println(inString);

  if (inString != null) 
    {
    inString = trim(inString);
    inByte = int(inString);
    //println(inByte);
    flag = true;
    }
  }

If you want to plot more data per frame you will have to buffer data into an array and draw this each frame.

:)

Thanks!

While I was trying your suggestions, I made last test with a small modification on arduino sketch and pde:

  • I modified delay in arduino sketch from 1 to 5 milliseconds (I already tried it in the past days without any improvement in resolution)
  • I moved in *.pde the code inside “void serialevent” into “void draw”, as per below:
import processing.serial.*;

Serial myPort;        // The serial port

int xPos = 1;         // horizontal position of the graph
float height_old = 0;
float height_new = 0;
float inByte = 0;


void setup () {
  // set the window size:
  size(1600, 900);        

  // List all the available serial ports
  printArray(Serial.list());
  // Open whatever port is the one you're using.
  myPort = new Serial(this, Serial.list()[0], 115200);
  // don't generate a serialEvent() unless you get a newline character:
  //myPort.bufferUntil('\n');
  myPort.bufferUntil(10);
  // set inital background:
  background(0xff);
}


void draw () {

  String inString = myPort.readStringUntil('\n');

  if (inString != null) {
    // trim off any whitespace:
    inString = trim(inString);
    if (inString.equals("!")) { 
      stroke(0, 0, 0xff); //Set stroke to blue ( R, G, B)
      inByte = 2048;  // middle of the ADC range (Flat Line)
    }
    // If the data is good let it through
    else {
      stroke(0xff, 0, 0); //Set stroke to red ( R, G, B)
      inByte = float(inString);
    }
  }
  inByte = inByte * 0.2;


  height_new = height - inByte; 
  line(xPos - 1, height_old, xPos, height_new);
  height_old = height_new;

  // at the edge of the screen, go back to the beginning:
  if (xPos >= width) {
    xPos = 0;
    background(0xff);
  } else {
    // increment the horizontal position:
    //xPos++;
    xPos = xPos + 1;
  }
}

Now the resolution is perfect, better than arduino serial plotter that was already satisfactory! But a new problem came-up…It seems processing plotter requires a lot of time to draw the datapoints and in the meanwhile stores a lot of data in memory so ECG isn’t in real time anymore.

I will try increasing delay in arduino sketch but 5 milliseconds means 200 Hz that was my target at the beginning of the job. In case of issue heart rate can reach 200 bpm…it means that in 1 seconds there are 3,3 beats and each beat has a waveform with 3 peaks…I think 60 datapoints for each beat are necessary to re-built it

I got that the delay is due to processing refresh rate that should be 30 fps by default. Increasing it at 200 fps with command frameRate(200) causes to have many point at zero…as said by gls I should introduce data buffering into an array (I need to study :stuck_out_tongue:).
Since framerate(100) seems working fine, a compromise is to reduce sample rate at 100Hz

Default is 60 fps:
https://processing.org/reference/draw_.html
https://processing.org/reference/frameRate_.html

In my example I used serialEvent():
https://processing.org/reference/libraries/serial/serialEvent_.html

You can “buffer” received data into an array in serialEvent():

  • use a counter\pointer to element of array
  • set a flag to indicate when data is ready
  • be sure to reset counter\pointer to element in array

In draw();

  • when flag is set you can use that data in draw()
  • be sure to reset flag

It will take some effort to work through and understand this; work through it slowly and build on that.

There are many ways to handle serial data; you can also send a buffer all at once from Arduino:

:)

If I buffer data with an array in Arduino before sending to Processing, do I have to store relevant milliseconds too? Because if I 'd like to calc heartrate I will not be able to do it due to the buffer

You can do that! Nice to see you putting a lot of thought into this.

You will have to frame the data string you are sending correctly and terminate with a ‘/n’ (in my example.

Example of received string only:

// Trim and split an data string Rx from Arduino.
// v1.0.0
// GLV 2020-MM-DD
                  
void setup()
  {
  background(0);
  size(400, 500);
  // Data packet received from Arduino: 
  //                 t1,  d1, t2,  d2, ... t=time, d=data 
  String inString = "1000,250,1100,300,1200,375,1300,380,1400,375,1500,300,1600,250,1700,200\n"; // '\n' is a linefeed character 
  print(inString);
  
  inString = inString.trim(); //Works!
  inString = trim(inString); //Works!
  print(inString);
  println("end"); //confirms that '/n' was removed!
  
  println();
  String[] list = split(inString, ',');
  printArray(list);
  
  int [] data = int( split(inString, ',') );
  printArray(data);
  
  stroke(255, 255, 0);
  strokeWeight(5);
  
  for(int i = 0; i< data.length; i+=2)
    {
    point((data[i]-1000)/5, data[i+1]);   
    }
  }  

Next step I leave with you. :slight_smile:

:)

Thanks!
I started working on the revised files with buffer, I hope to show results :crossed_fingers: asap

New code is ready, I have just to comment before attach it here!

I’ve just last question: currently when the ECG line reaches the end of the plot area, there is a command to clear everything and the line restarts from left to right.
It’s enough but I’d like to have the old line not cleared all at once time but disappearing once the new line is approching it. How could I manage it?

1 Like

In the meanwhile, this is my last result

2 Likes

Hi @alexthesmeus

I would say that depends on how you are storing data and drawing it.
I would suggest that if you detect that x-axis value is >= width of the screen or the plot, then remove that point from the array where you store your data.

Here is a code that I have developed for plotting my own data. It is an example but the main idea is there for your problem.

Plotter plot;

ArrayList[] arrayOfArrayList;
float range = 400.0;
void setup() {
  size(400, 400);
  
  plot = new Plotter(width / 2 - 100, height / 2, 100, 200, 100.0f);
}


void draw() {
  background(0);
  // draw axis
  plot.chartAxis();
  //genreate random Y axis value;
  generateRandom();
  
  //  plot data
  plot.plotData();
}


void generateRandom() {
  float t = random(-10.0, 10.0);
  // add points to plot
  plot.addPoints(t);
}

The plotter class is as follows. The part of interest for you is the plotData function()

class Plotter {
  ArrayList MagnitudePoints;
  int xpos, ypos;
  int axisHeight, axisWidth;
  float range;

 // Position of plot, dimensions and range
  Plotter(int xpos, int ypos,  int axisHeight, int axisWidth, float range) {
    MagnitudePoints = new ArrayList<Points>();
    this.xpos = xpos;
    this.ypos = ypos;
    this.axisHeight = axisHeight;
    this.axisWidth = axisWidth;
    this.range = range;
  }
 
  //plot axis and text range and origin
  void chartAxis() {
    stroke(255);
    fill(255);
    line(xpos, ypos - axisHeight, xpos, ypos + axisHeight);
    line(xpos, ypos, xpos + axisWidth, ypos);
    //textFont
    textAlign(RIGHT, CENTER);
    text("0", xpos, ypos);
    text(String.format("%.1f", range), xpos, ypos + axisHeight);
    text(String.format("%.1f", range), xpos, ypos - axisHeight);
    noStroke();
  }

  //Plot the real data!
  void plotData() {
    pushMatrix();
    beginShape();
    noFill();
    for (int i = 0; i < MagnitudePoints.size(); i++) {
     
      Points P = (Points)MagnitudePoints.get(i);
      stroke(187, 0, 0);
      if (P!=null) {
        // Plot curve
        curveVertex(P.x, P.y);
        // Did I get to the end of the plot??
        if (P.x>xpos + axisWidth)MagnitudePoints.remove(i); //If so, remove the point
        P.x++;

      }
    }
    endShape();
    popMatrix();
  }
  
 //Cartesian points
  class Points {
    float x, y;
    Points(float x, float y) {
      this.x = x;
      this.y = ypos + (axisHeight/range) * y;
   
    }
  }
  
  void addPoints(float point) {
    MagnitudePoints.add(new Points(xpos, point));
  }
}

If you have any doubt I will try to clarify it! Hope it helps! :slight_smile:
By the way, nice work!

Best regards

Thanks Miguel.
My problem is that I haven’t any array (or I don’t know where it is) where data points are stored.
I send points from arduino with a buffer of 5 points per time and they are used to plot lines. When I reach the end of the plot area (1600 points), everything is cleared and lines drawing restart from the left.
I don’t know where these 1600 points are stored in the meanwhile.

Can you post your processing code?

Let’s take it part by part:
1.1) - I would say that you are receiving the data points
1.2) - You are storing data into an array/buffer (Let’s call it Buf)
1.3) - You pass this buffer to a function that plot’s your data.

So all you have to do is go backwards:
2.1) - What function are you using to draw the points? It should be receiving your data correct?
2.2) - What is the name of this buffer or array storage passed as input in 2.1)?
2.3) - Where is it being called for you to place data received?

From here you should be able to work around your problem :slight_smile:

Examples of fading:
https://processing.org/examples/movingoncurves.html
https://processing.org/examples/simpleparticlesystem.html
There may be others…

Slimmed down (to just a line) version of fading I use:

int i;
int alpha;

public void setup() 
  {
  size(512, 200);   
  background(0);
  }

public void draw() 
  {   
  background(0);
  
  i = 300; 
   
  for(int j = 0; j<i; j++)
    {
    alpha = 0;
    if (j > i - 255)
    alpha = 255 - (i - j);  // tail fades away 
      
    stroke(0, 255, 0, alpha);
    strokeWeight(10);
    point(j, height/2); 
    }  
  }

I used the above for my scrolling ECG plot:

:)

Hello @alexthesmeus

Based on the code you have provided so far, you are not using an array to store your data. You are using a variable to keep track of the previous point and then you are tracing a line from the previous point to the “recently received” new point as you can see next:

For your current case requirement you will need to use an array (or any other data container, for example FloatList) to store all the data that you receive and then draw will continuously plot this data in this data container. Please check the examples provided in the link above and share your code if you have any questions.

Kf

Thanks!
I have to try it. And I think I could improve BPM calculation that relies currently on a simple threshold to detect the peak.
In the meanwhile I share the result of your support, it works fine!

Arduino

// ECG Bluetooth (Arduino)
// v1.0.0
// Alexthesmeus 2020-07-08


#include "BluetoothSerial.h"
BluetoothSerial SerialBT;
int samplerate = 250;   // samplerate 250 Hz
int FPS = 50;           // processing refresh set at 50 fps
String buf;             

int del = 1000 / samplerate;   //time between 2 samples
int b = samplerate/FPS;        // buffer size to meet samplerate and refresh time in processing

void setup() {
  Serial.begin(115200);
  SerialBT.begin("ECG"); //Bluetooth device name
  pinMode(23, INPUT); // Setup for leads off detection LO +
  pinMode(33, INPUT); // Setup for leads off detection LO -
}

void loop() {
  for (int i = 0; i < b; i++) {
    buf += millis();      //buffer contains milliseconds for heart rate calculation
    buf += ',';

    if ((digitalRead(23) == 1) || (digitalRead(33) == 1)) {   
      buf += -999;       //value sent in case at least one lead is disconnected, to be filtered
    }
    else {
      buf += analogRead(4);
    }

    delay(del);
    
    if (i == (b-1)){              //buffer sent when reach is lenght
      SerialBT.println(buf);
      Serial.println(buf);
      buf="";
    }
    else {
      buf += ',';
    }

  }
}

Processing

// ECG Bluetooth (Processing)
// v1.0.0
// Alexthesmeus 2020-07-08

import processing.serial.*;

Serial myPort;        

int xPos = 1;        
float height_old = 0;
float height_new = 0;
float inByte = 0;
int[] data;
int step=125;
int step2=25;

int BPM = 0;
int beat_old = 0;
float[] beats = new float[5];  
int beatIndex;
float threshold = 2600;  
boolean belowThreshold = true;
PFont font;



void setup () {
  frameRate(50);
  size(1625, 625);        

  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[0], 115200);
  myPort.bufferUntil(10);
  background(0xff);
  font = createFont("Arial", 40, true);

  //initialize grid
  stroke(255, 140, 140);
  strokeWeight(1);
  for (int k = 0; k <= width/step2; k++ ) {
    line(k*step2, 0, k*step2, height);
    line(0, k*step2, width, k*step2);
  }
  strokeWeight(2);
  for (int k = 0; k <= width/step; k++ ) {
    line(k*step, 0, k*step, height);
    line(0, k*step, width, k*step);
  }
}


void draw () {
  //Draw BPM
  stroke(255, 140, 140);
  fill(255);
  rect(0, 0, 250, 50);
  fill(0);
  textFont(font);
  text("BPM: " + BPM, 15, 40);

  //Read buffer
  String inString = myPort.readStringUntil('\n');
  println(inString);
  if (inString != null) 
  {
    inString = trim(inString);
    data = int(split(inString, ','));
  }

  for (int i = 1; i< data.length; i+=2)
  {
    if (data[i]==-999) { 
      stroke(0, 0, 0xff); 
      inByte = 2048;     //if at least one lead is disconnected, draw flat blue line
    }
    else {
      stroke(0, 0, 0); 
      inByte=data[i];
    }
    if (inByte > threshold && belowThreshold == true)    //BPM calculation
    {
      int beat_new = data[i-1];    
      int diff = beat_new - beat_old;    
      float currentBPM = 60000 / diff;    
      beats[beatIndex] = currentBPM;  
      float total = 0.0;
      for (int j = 0; j < 5; j++) {
        total += beats[j];
      }
      BPM = int(total / 5);
      beat_old = beat_new;
      beatIndex = (beatIndex + 1) % 5;  

      belowThreshold = false;
    } else if (inByte < threshold)
    {
      belowThreshold = true;
    }

    inByte = map(inByte, 0, 4096, 0, height);
    height_new = height - inByte; 
    line(xPos - 1, height_old, xPos, height_new);
    height_old = height_new;
    if (xPos >= width) {
      xPos = 0;          //clear graph and restart when end is reached
      background(0xff);

      stroke(255, 140, 140);
      strokeWeight(1);
      for (int k = 0; k <= width/step2; k++ ) {
        line(k*step2, 0, k*step2, height);
        line(0, k*step2, width, k*step2);
      }
      strokeWeight(2);
      for (int k = 0; k <= width/step; k++ ) {
        line(k*step, 0, k*step, height);
        line(0, k*step, width, k*step);
      }
    } else {
      xPos++;
    }
  }
}

void keyPressed()    //print ECG 
{
  saveFrame(nf(hour(), 2) + nf(minute(), 2) + nf(second(), 2)+ "_" + nf(day(), 2)+  nf(month(), 2) + nf(year(), 4) + ".tif");
}