Collect data from a local web server

I found a project (https://maker.pro/arduino/tutorial/how-to-interface-arduino-and-the-mpu-6050-sensor) in which the processing code could get the data from the serial port.
I want to make a different version of this project, where an Arduino mega gets the data, sends it to a NodeMCU 1.0 (ESP8266) board, which in turn creates a local web server and uploads the value to it. But I need to collect the data from the server in the processing code. Can someone help me with doing that?

1 Like

PDE / File / Examples / Libraries / Network / HTTPClient

would allow you to get a ?semi?static text from webpage.
you would need to repeat that to get updated info.

i just tested

c = new Client(this, "192.168.1.204", 3000); 

to my RPI node server


but make sure there is no real HTML page you are reading from,
why not put just a CSV line there ( with or without header line )


but well all that is very far from the good USB serial steam you have now.

should look into different ways?
OSC or MQTT

2 Likes

Sorry, but being a beginner I couldn’t understand what you are talking about. Most of the programming I did was just getting the code from the internet putting different programs together and editing to get the required result.
What do you mean by semi static text?
Also what is a CSV line?
And what are OSC and MQTT?

CSV “comma-separated values” would mean pure text like

1023,1023,1023

as data from 3 analog readings,
without any HTML code it would also just look like that in the browser


ok, question back, the arduino data ( from sensors ) you show on a
ESP web server site, how does that look?
in code AND from a browser?

1 Like

Just the data.

client.println("HTTP/1.1 200 OK"); client.println("Content-type:text/html"); client.println("Connection: close"); client.println(); client.write(teapotPacket, 14); client.stop();
These are the only lines in the NodeMCU code that output to the client.
teapotPacket is the array containing the data and it prints the data in ASCII encoding to the client.
So the output is actually a set of 14 characters with ‘\r’ and ‘\n’ being the last of them.
If I understood your reply correctly this is CSV.

so run the above mentioned example
with using the ESP servers IP and port
and see what you get into processing console

1 Like

I have already seen that example and tried it. But the client disconnects without receiving data.

so please ignore the processing question for a moment,
call the ESP from your browser and show me what you see?

2 Likes

You mean I should open that server in my browser and tell what I get?


This is what I get.

1 Like

besides that is not ASCII and that is not CSV comma separated values,
it is something,
should also see something from processing?

ESP:

  client.println("HTTP/1.1 200 OK");
  client.println("Content-Type: text/html");
  client.println(""); //  do not forget this one

  client.println("1023,1023,1023");
  client.println(""); 

and test to not use the

//client.stop();

Processing:

c = new Client(this, "192.168.1.4", 80); 
2 Likes

Is there any problem in changing the line c.write(“GET / HTTP/1.0\r\n”);
to c.write(“GET / HTTP/1.1\r\n”); ?
Because I thought it was just an older version that has to be updated.
Doing what you said seemed to work. I got 1023, 1023, 1023 printed in the console of the processing code. But extending it to the project seems to have some problem.

1 Like

The codes:
Mega code:

#include "I2Cdev.h"

#include "MPU6050_6Axis_MotionApps20.h"
//#include "MPU6050.h"
#if I2CDEV_IMPLEMENTATION == I2CDEV_ARDUINO_WIRE
    #include "Wire.h"
#endif

MPU6050 mpu;
#define OUTPUT_TEAPOT



#define INTERRUPT_PIN 2  
#define LED_PIN 13 
bool blinkState = false;

bool dmpReady = false;  
uint8_t mpuIntStatus;   
uint8_t devStatus;      
uint16_t packetSize;    
uint16_t fifoCount;     
uint8_t fifoBuffer[64]; 
Quaternion q;           // [w, x, y, z]         quaternion container
VectorInt16 aa;         // [x, y, z]            accel sensor measurements
VectorInt16 aaReal;     // [x, y, z]            gravity-free accel sensor measurements
VectorInt16 aaWorld;    // [x, y, z]            world-frame accel sensor measurements
VectorFloat gravity;    // [x, y, z]            gravity vector
float euler[3];         // [psi, theta, phi]    Euler angle container
float ypr[3];           // [yaw, pitch, roll]   yaw/pitch/roll container and gravity vector

// packet structure for InvenSense teapot demo
uint8_t teapotPacket[14] = { '$', 0x02, 0,0, 0,0, 0,0, 0,0, 0x00, 0x00, '\r', '\n' };

volatile bool mpuInterrupt = false;     
void dmpDataReady() {
    mpuInterrupt = true;
}

void setup() {
    #if I2CDEV_IMPLEMENTATION == I2CDEV_ARDUINO_WIRE
        Wire.begin();
        Wire.setClock(400000); 
    #elif I2CDEV_IMPLEMENTATION == I2CDEV_BUILTIN_FASTWIRE
        Fastwire::setup(400, true);
    #endif
    Serial.begin(115200);
    Serial1.begin(115200);
    while (!Serial); 
    Serial.println(F("Initializing I2C devices..."));
    mpu.initialize();
    pinMode(INTERRUPT_PIN, INPUT);

    Serial.println(F("Testing device connections..."));
    Serial.println(mpu.testConnection() ? F("MPU6050 connection successful") : F("MPU6050 connection failed"));
    Serial.println(F("Initializing DMP..."));
    devStatus = mpu.dmpInitialize();
    mpu.setXGyroOffset(220);
    mpu.setYGyroOffset(76);
    mpu.setZGyroOffset(-85);
    mpu.setZAccelOffset(1788); 
    if (devStatus == 0) {
        Serial.println(F("Enabling DMP..."));
        mpu.setDMPEnabled(true);
        Serial.println(F("Enabling interrupt detection (Arduino external interrupt 0)..."));
        attachInterrupt(digitalPinToInterrupt(INTERRUPT_PIN), dmpDataReady, RISING);
        mpuIntStatus = mpu.getIntStatus();
        Serial.println(F("DMP ready! Waiting for first interrupt..."));
        dmpReady = true;
        packetSize = mpu.dmpGetFIFOPacketSize();
    } else {
        Serial.print(F("DMP Initialization failed (code "));
        Serial.print(devStatus);
        Serial.println(F(")"));
    }

    pinMode(LED_PIN, OUTPUT);
}


void loop() {
    if (!dmpReady) return;

    while (!mpuInterrupt && fifoCount < packetSize) {
    }

    mpuInterrupt = false;
    mpuIntStatus = mpu.getIntStatus();

    fifoCount = mpu.getFIFOCount();

    if ((mpuIntStatus & 0x10) || fifoCount == 1024) {
        mpu.resetFIFO();
        Serial.println(F("FIFO overflow!"));

    } else if (mpuIntStatus & 0x02) {
        while (fifoCount < packetSize) fifoCount = mpu.getFIFOCount();

        mpu.getFIFOBytes(fifoBuffer, packetSize);
        
        fifoCount -= packetSize;
        teapotPacket[2] = fifoBuffer[0];
        teapotPacket[3] = fifoBuffer[1];
        teapotPacket[4] = fifoBuffer[4];
        teapotPacket[5] = fifoBuffer[5];
        teapotPacket[6] = fifoBuffer[8];
        teapotPacket[7] = fifoBuffer[9];
        teapotPacket[8] = fifoBuffer[12];
        teapotPacket[9] = fifoBuffer[13];
        String str = "";
        for(int i = 0; i <= 13; i++)
          str += char(teapotPacket[i]);
        if(Serial1.available()) {
          if((Serial1.readString()).indexOf("Ready") > 0)
            Serial1.print(str);
        }
        teapotPacket[11]++; // packetCount, loops at 0xFF on purpose
        blinkState = !blinkState;
        digitalWrite(LED_PIN, blinkState);
    }
}

NodeMCU code:

#include <ESP8266WiFi.h>
const char* ssid     = "YOUR_SSID";
const char* password = "PASSWORD";
WiFiServer server(80);
String header;
unsigned long currentTime = millis();
unsigned long previousTime = 0; 
String IPadd;
const long timeoutTime = 2000;
void setup()
{
  Serial.begin(115200);
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED)
  {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected.");
  Serial.println("IP address: ");
  IPadd = IpAddress2String(WiFi.localIP());
  Serial.println(IPadd);
  server.begin();
}
void loop()
{
  WiFiClient client = server.available();  
  if (client)
  {                             
    Serial.println("New Client.");          
    String currentLine = "";                
    currentTime = millis();
    previousTime = currentTime;
    while (client.connected() && currentTime - previousTime <= timeoutTime)
    { 
      currentTime = millis();         
      if (client.available())
      {             
        char c = client.read();             
        header += c;
        if (c == '\n') 
        { 
          if (currentLine.length() == 0) 
          {
            client.println("HTTP/1.1 200 OK");
            client.println("Content-type:text/html");
            client.println("Connection: close");
            client.println("");
            uint8_t teapotPacket[14];
            String str;
            Serial.println("Ready");
            if(Serial.available())
              str = Serial.readStringUntil('\n');
            str += '\n';
            Serial.print(str);
            for(int i = 0; i <= 13; i++)
              teapotPacket[i] = int(str.charAt(i));
            client.write(teapotPacket, 14);
            break;
          } 
          else 
          { 
            currentLine = "";
          }
        } 
        else if (c != '\r') 
        {  
          currentLine += c;
        }
      }
    }
    header = "";
//    client.stop();
    Serial.println("Client disconnected.");
    Serial.println("");
  }
}
String IpAddress2String(const IPAddress& ipAddress)
{
  return String(ipAddress[0]) + String(".") +\
  String(ipAddress[1]) + String(".") +\
  String(ipAddress[2]) + String(".") +\
  String(ipAddress[3])  ; 
}

Processing code

import processing.serial.*;
import processing.opengl.*;
import toxi.geom.*;
import toxi.processing.*;
import processing.net.*;

Client c;


ToxiclibsSupport gfx;

Serial port;                         
char[] teapotPacket = new char[14];  
int serialCount = 0;                 
int synced = 0;
int interval = 0;

float[] q = new float[4];
Quaternion quat = new Quaternion(1, 0, 0, 0);

float[] gravity = new float[3];
float[] euler = new float[3];
float[] ypr = new float[3];

void setup() {
    size(300, 300, OPENGL);
    gfx = new ToxiclibsSupport(this);

    // setup lights and antialiasing
    lights();
    smooth();
    c = new Client(this, "192.168.1.4", 80);
    c.write("GET / HTTP/1.0\r\n");
    c.write("\r\n");
}

void draw() {
  if (millis() - interval > 1000) {
        interval = millis();
    }
    clientEvent(c);
    // black background
    background(0);
    
    // translate everything to the middle of the viewport
    pushMatrix();
    translate(width / 2, height / 2);


    
    // 3-step rotation from yaw/pitch/roll angles (gimbal lock!)
    // ...and other weirdness I haven't figured out yet
    //rotateY(-ypr[0]);
    //rotateZ(-ypr[1]);
    //rotateX(-ypr[2]);

    // toxiclibs direct angle/axis rotation from quaternion (NO gimbal lock!)
    // (axis order [1, 3, 2] and inversion [-1, +1, +1] is a consequence of
    // different coordinate system orientation assumptions between Processing
    // and InvenSense DMP)
    float[] axis = quat.toAxisAngle();
    rotate(axis[0], -axis[1], axis[3], axis[2]);

    // draw main body in red
    fill(255, 0, 0, 200);
    box(10, 10, 200);
    
    // draw front-facing tip in blue
    fill(0, 0, 255, 200);
    pushMatrix();
    translate(0, 0, -120);
    rotateX(PI/2);
    drawCylinder(0, 20, 20, 8);
    popMatrix();
    
    // draw wings and tail fin in green
    fill(0, 255, 0, 200);
    beginShape(TRIANGLES);
    vertex(-100,  2, 30); vertex(0,  2, -80); vertex(100,  2, 30);  // wing top layer
    vertex(-100, -2, 30); vertex(0, -2, -80); vertex(100, -2, 30);  // wing bottom layer
    vertex(-2, 0, 98); vertex(-2, -30, 98); vertex(-2, 0, 70);  // tail left layer
    vertex( 2, 0, 98); vertex( 2, -30, 98); vertex( 2, 0, 70);  // tail right layer
    endShape();
    beginShape(QUADS);
    vertex(-100, 2, 30); vertex(-100, -2, 30); vertex(  0, -2, -80); vertex(  0, 2, -80);
    vertex( 100, 2, 30); vertex( 100, -2, 30); vertex(  0, -2, -80); vertex(  0, 2, -80);
    vertex(-100, 2, 30); vertex(-100, -2, 30); vertex(100, -2,  30); vertex(100, 2,  30);
    vertex(-2,   0, 98); vertex(2,   0, 98); vertex(2, -30, 98); vertex(-2, -30, 98);
    vertex(-2,   0, 98); vertex(2,   0, 98); vertex(2,   0, 70); vertex(-2,   0, 70);
    vertex(-2, -30, 98); vertex(2, -30, 98); vertex(2,   0, 70); vertex(-2,   0, 70);
    endShape();
    
    popMatrix();
}

void clientEvent(Client c) {
    interval = millis();
    while(c.available() > 0) {
                  
        int ch = c.read();

        if (synced == 0 && ch != '$') return;   // initial synchronization - also used to resync/realign if needed
        synced = 1;
        print ((char)ch);

        if ((serialCount == 1 && ch != 2)
            || (serialCount == 12 && ch != '\r')
            || (serialCount == 13 && ch != '\n'))  {
            serialCount = 0;
            synced = 0;
            return;
        }

        if (serialCount > 0 || ch == '$') {
            teapotPacket[serialCount++] = (char)ch;
            if (serialCount == 14) {
                serialCount = 0; // restart packet byte position
                // get quaternion from data packet
                q[0] = ((teapotPacket[2] << 8) | teapotPacket[3]) / 16384.0f;
                q[1] = ((teapotPacket[4] << 8) | teapotPacket[5]) / 16384.0f;
                q[2] = ((teapotPacket[6] << 8) | teapotPacket[7]) / 16384.0f;
                q[3] = ((teapotPacket[8] << 8) | teapotPacket[9]) / 16384.0f;
                for (int i = 0; i < 4; i++) if (q[i] >= 2) q[i] = -4 + q[i];
                
                // set our toxilibs quaternion to new data
                quat.set(q[0], q[1], q[2], q[3]);

            }
        }
    }    
}

void drawCylinder(float topRadius, float bottomRadius, float tall, int sides) {
    float angle = 0;
    float angleIncrement = TWO_PI / sides;
    beginShape(QUAD_STRIP);
    for (int i = 0; i < sides + 1; ++i) {
        vertex(topRadius*cos(angle), 0, topRadius*sin(angle));
        vertex(bottomRadius*cos(angle), tall, bottomRadius*sin(angle));
        angle += angleIncrement;
    }
    endShape();
    
    // If it is not a cone, draw the circular top cap
    if (topRadius != 0) {
        angle = 0;
        beginShape(TRIANGLE_FAN);
        
        // Center point
        vertex(0, 0, 0);
        for (int i = 0; i < sides + 1; i++) {
            vertex(topRadius * cos(angle), 0, topRadius * sin(angle));
            angle += angleIncrement;
        }
        endShape();
    }
  
    // If it is not a cone, draw the circular bottom cap
    if (bottomRadius != 0) {
        angle = 0;
        beginShape(TRIANGLE_FAN);
    
        // Center point
        vertex(0, tall, 0);
        for (int i = 0; i < sides + 1; i++) {
            vertex(bottomRadius * cos(angle), tall, bottomRadius * sin(angle));
            angle += angleIncrement;
        }
        endShape();
    }
}
2 Likes

For reasons I don’t understand, using client.print(str); instead of

for(int i = 0; I <= 13; i++) 
  teapotPacket[i] = str.charAt(i);
client.write(teapotPacket, 14);

seemed to work.
But how do I loop it so that I can keep getting the data?
Should I move

c.write("GET / HTTP/1.0\r\n");
c.write("\r\n");

to draw()?

I tried this on the Arduino side with the serial print\write as I expected the behavior to be the same as the client print\write.

This is what I got:

Arduino code:

void setup() 
  {
  Serial.begin(9600); // Start serial communication at 9600 bps
  }
 
 void loop() 
  {
  // packet structure for InvenSense teapot demo
  uint8_t teapotPacket1[14] = { '$', 0x02, 0, 0, 0, 0, 0, 0, 0, 0, 0x00, 0x00, '\r', '\n' };
  String str = "";
  for(int i = 0; i <= 13; i++)
    str += char(teapotPacket1[i]);
  Serial.print(str);

  uint8_t teapotPacket2[14] = { '$', 0x02, 0, 0, 0, 0, 0, 0, 0, 0, 0x00, 0x00, '\r', '\n' };
  for(int i = 0; i <= 13; i++) 
    teapotPacket2[i] = str.charAt(i);
  Serial.write(teapotPacket2, 14);

  uint8_t teapotPacket3[14] = { '$', 0x02, 0, 0, 0, 0, 0, 0, 0, 0, 0x00, 0x00, '\r', '\n' };
  for(int i = 0; i <= 13; i++) 
    teapotPacket3[i] = int(str.charAt(i));
  Serial.write(teapotPacket3, 14);

  uint8_t teapotPacket4[14] = { '$', 0x02, 0, 0, 0, 0, 0, 0, 0, 0, 0x00, 0x00, '\r', '\n' };
  for(int i = 0; i <= 13; i++) 
    Serial.write(teapotPacket4[i]);

  delay(5000);
  }

I used Brays Terminal to view output in hex:

I receive this data on a terminal:
image

:slight_smile:

2 Likes

Thanks for bringing this topic to us.
It is my plan to do some work related to this and this inspired me!

:slight_smile: