Convert UDP packet to floats?

Processing Libraries
1

Hello everyone, I’m working with UDP for the first time, using a program called opentrack which sends out packets of bytes over UDP. I can receive the packets in processing, but I’m having trouble figuring out to convert them into floats so that I can do what I want with them.

Here is the code thus far:

import hypermedia.net.*;
int PORT_RX= 6000; //port : set in opentrack
String HOST_IP="192.168.86.28"; // must be computer IP address, use ipconfig to find
UDP udp;
String receivedFromUDP = "";

void setup() {
  size(1000,100);
  udp= new UDP(this,PORT_RX,HOST_IP);
  udp.log(true);
  udp.listen(true);
  super.start();
}



void draw() {
  background(0);
  text(receivedFromUDP, 50, 50);
  delay(500);
}

void receive(byte[] data, String HOST_IP, int PORT_RX) {
  receivedFromUDP ="";
  for (int i = 0; i < data.length; i++) {
    receivedFromUDP += str(data[i]) + " ";
  }
}

And here is a an example of the output:

102 10 -27 -86 92 17 40 64 -120 9 -9 104 -19 55 -48 -65 48 -76 -8 -105 66 -105 47 -64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Any help would be greatly appreciated, thank you!

2

This seems more complicated than I expected: https://github.com/opentrack/opentrack/issues/747

“Convert bytes to long in a little-endian format (i.e. reverse the left shift order).”

3

Here is the data output from opentrack when I set it to save output to a CSV:

dt,rawTX,rawTY,rawTZ,rawYaw,rawPitch,rawRoll,correctedTX,correctedTY,correctedTZ,correctedYaw,correctedPitch,correctedRoll,filteredTX,filteredTY,filteredTZ,filteredYaw,filteredPitch,filteredRoll,mappedTX,mappedTY,mappedTZ,mappedYaw,mappedPitch,mappedRoll
1e-06,32.8114,-0.0625441,-85.999,-9.20088,-1.08892,98.013,-0,-0,-0,-0,0,-0,-0,-0,-0,-0,0,-0,0,0,0,0,0,0
0.007644,32.8114,-0.0625441,-85.999,-9.20088,-1.08892,98.013,-0,-0,-0,-0,0,-0,0,0,0,0,0,0,0,0,0,0,0,0
0.003583,32.8114,-0.0625441,-85.999,-9.20088,-1.08892,98.013,-0,-0,-0,-0,0,-0,0,0,0,0,0,0,0,0,0,0,0,0
0.003823,32.8114,-0.0625441,-85.999,-9.20088,-1.08892,98.013,-0,-0,-0,-0,0,-0,0,0,0,0,0,0,0,0,0,0,0,0
0.00381,32.8114,-0.0625441,-85.999,-9.20088,-1.08892,98.013,-0,-0,-0,-0,0,-0,0,0,0,0,0,0,0,0,0,0,0,0
0.003812,32.8114,-0.0625441,-85.999,-9.20088,-1.08892,98.013,-0,-0,-0,-0,0,-0,0,0,0,0,0,0,0,0,0,0,0,0
0.003772,32.8114,-0.0625441,-85.999,-9.20088,-1.08892,98.013,-0,-0,-0,-0,0,-0,0,0,0,0,0,0,0,0,0,0,0,0
0.003801,32.8114,-0.0625441,-85.999,-9.20088,-1.08892,98.013,-0,-0,-0,-0,0,-0,0,0,0,0,0,0,0,0,0,0,0,0
0.00377,32.8114,-0.0625441,-85.999,-9.20088,-1.08892,98.013,-0,-0,-0,-0,0,-0,0,0,0,0,0,0,0,0,0,0,0,0
0.00381,32.8114,-0.0625441,-85.999,-9.20088,-1.08892,98.013,-0,-0,-0,-0,0,-0,0,0,0,0,0,0,0,0,0,0,0,0

4

The data I want to extract for now is just X,Y and Z. Once converted properly those numbers should be like:

X: 7
Y: -1
Z: -12

5

Hi @jetjaguar

I normally use the following function to convert byte arrays to float

float byteArrayToFloat(byte[] bytes) {
    int intBits = 
      bytes[3] << 24 | (bytes[2] & 0xFF) << 16 | (bytes[1] & 0xFF) << 8 | (bytes[0] & 0xFF);
    return Float.intBitsToFloat(intBits);  
}

So then I define a byte array of length 4 to copy data and convert to floating point value.

Hope it helps! :slight_smile:
Best regards

6

Thank you Miguel, I’m sure it does, but I am struggling on how exactly to implement it.

7

I believe I’m starting to get close with it.


import hypermedia.net.*;
int PORT_RX= 6000; //port : set in opentrack
String HOST_IP="192.168.86.28"; // must be computer IP address, use ipconfig to find
UDP udp;
String receivedFromUDP = "";


void setup() {
  size(1000,100);
  udp= new UDP(this,PORT_RX,HOST_IP);
  udp.log(true);
  udp.listen(true);
  super.start();
}



void draw() {
  background(0);
  text(receivedFromUDP, 50, 50);
  delay(500);
}

void receive(byte[] data, String HOST_IP, int PORT_RX) {
  receivedFromUDP ="";
  for (int i = 0; i < data.length; i++) {
    receivedFromUDP = str(byteArrayToFloat(data));
  }
}

float byteArrayToFloat(byte[] data) {
    int intBits = 
      data[3] << 24 | (data[2] & 0xFF) << 16 | (data[1] & 0xFF) << 8 | (data[0] & 0xFF);
    return Float.intBitsToFloat(intBits);  
}
8

Hi @jetjaguar ,

So I don’t know how you are sending data, or how many floats you are sending, but this code should work if your packets are all floats and each packet has a fixed number of floating values


import hypermedia.net.*;
int PORT_RX= 6000; //port : set in opentrack
String HOST_IP="192.168.86.28"; // must be computer IP address, use ipconfig to find
UDP udp;
String receivedFromUDP = "";

void setup() {
  size(1000, 100);
  udp= new UDP(this, PORT_RX, HOST_IP);
  udp.log(true);
  udp.listen(true);
  super.start();
}



void draw() {
  background(0);
  text(receivedFromUDP, 50, 50);
  delay(500);
}

void receive(byte[] data, String HOST_IP, int PORT_RX) {
  receivedFromUDP ="";
  //number of bytes in packet 
  int bytes_packet = data.length;
  //each float has 4 bytes 
  int FLOATS = bytes_packet/4;
  //array to hold all floats from conversion
  float[] floats = new float[FLOATS];

  //Loop for each float
  for (int countFloats = 0; countFloats < FLOATS; countFloats++) {
    byte[] incomingByte = new byte[4];
    //group bytes in 4 to convert to float
    for (byte idx = 0; idx < 4; idx++) {
      incomingByte[idx] = data[countFloats * 4 + idx];
    }
    floats[countFloats] = byteArrayToFloat(incomingByte);
    println(floats[countFloats]);
  }
}

float byteArrayToFloat(byte[] data) {
  int intBits = 
    data[3] << 24 | (data[2] & 0xFF) << 16 | (data[1] & 0xFF) << 8 | (data[0] & 0xFF);
  return Float.intBitsToFloat(intBits);
}

Hope it helps! :slight_smile:

9

Thank you so much again Miguel!

I’m a bit confused about the output coming from the program I’m using (have had trouble finding detailed documentation), but based upon the data I posted above I believe there are 25 floats, and they are not all the same size.

Right now I’m just trying different combinations because I’m still not 100% clear on how this all works.

Your last post seems awfully close, it’s separating out the data, but maybe not in the correct places because it does not match the values I’m expecting (the program, “opentrack”, shows me the data in real-time, so I can compare that with the processing sketch in real-time as well).

output from your code:

[21-02-17 17:58:06.840 -0500] receive packet <- from /192.168.86.28, port:57304, length: 48
0.013514109
2.6377223
160.74216
1.7041404
2.0692345E16
-2.0070057
0.0
0.0
0.0
0.0
0.0
0.0
10

Hi @jetjaguar ,

the length of the packet is 48bytes, so if they are all floats, it means that you are only receiving 12 of them.
Are you sending any chars non digit? like commas?

My advice would be to try with a simple example so you know what to expect on the output as well :slight_smile: Then build on top of that

Best regards

11

I’ll try that and report back. I’m confused about exactly what is being sent, I think it’s just floats and spaces, but I’m still researching opentrack (it’s UDP output method doesn’t seem to be commonly used).

Thank you Miguel.

12

I believe I have found the relevant output data details here: opentrack/fttypes.h at 058942f40e17e091b91df5436d771d61203ccc73 · opentrack/opentrack · GitHub

/* only 6 headpose floats and the data id are filled -sh */
typedef struct FTData__ {
    uint32_t DataID;
    int32_t CamWidth;
    int32_t CamHeight;
    /* virtual pose */
    float  Yaw;   /* positive yaw to the left */
    float  Pitch; /* positive pitch up */
    float  Roll;  /* positive roll to the left */
    float  X;
    float  Y;
    float  Z;
    /* raw pose with no smoothing, sensitivity, response curve etc. */
    float  RawYaw;
    float  RawPitch;
    float  RawRoll;
    float  RawX;
    float  RawY;
    float  RawZ;
    /* raw points, sorted by Y, origin top left corner */
    float  X1;
    float  Y1;
    float  X2;
    float  Y2;
    float  X3;
    float  Y3;
    float  X4;
    float  Y4;
} volatile FTData;
13

This is what I need to figure out, converting the incoming byte array to doubles, then to float.

https://discourse.processing.org/t/convert-double-to-bytes/12154/3

14

You could try using a ByteBuffer. You should also be able to modify the byte order (big endian vs little endian) of the ByteBuffer if necessary as well.

Assuming the first 24 bytes in your data are three doubles:

import java.nio.ByteBuffer;

void receive(byte[] data, String HOST_IP, int PORT_RX) {
  ByteBuffer bb = ByteBuffer.wrap(data);
  double xx = bb.getDouble();
  double yy = bb.getDouble();
  double zz = bb.getDouble();
  ...

https://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html

1 Like
15

Thank you rbrauer.

I’m stumbling all over this, the deeper I dig the more complicated it seems.

If it helps at all, this is what the output looks like from the following bit of code which reads incoming byte array data. This seems to be correct, it’s converting it to floats (or ints are fine) that match opentracks live data readout that still evades me.

output:

[-107, 3, 53, 87, 126, 46, 49, -64, -34, 113, 81, 2, -1, -14, 37, 64, 0, 0, 0, 0, 0, -64, 82, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

code:

import java.net.*;
import java.io.*;
import java.util.Arrays;

DatagramSocket socket;
DatagramPacket packet;

byte[] buf = new byte[48]; //Set your buffer size as desired

void setup() {
  try {
    socket = new DatagramSocket(7000); // Set your port here
  }
  catch (Exception e) {
    e.printStackTrace(); 
    println(e.getMessage());
  }
}

void draw() {
  try {
    DatagramPacket packet = new DatagramPacket(buf, buf.length);
    socket.receive(packet);
    InetAddress address = packet.getAddress();
    int port = packet.getPort();
    packet = new DatagramPacket(buf, buf.length, address, port);

    //Received as bytes:
    println(Arrays.toString(buf));
    
  }
  catch (IOException e) {
    e.printStackTrace(); 
    println(e.getMessage());
  }
}
16

I think you are correct about the size, rbrauer, but when I try that I get exceedingly small values, like “-1.1191590107135656E220” .

17

It looks like the data is actually little endian so I think it works if you specify that. Does this look more like what you would exepct?

-17.181616
10.974602
75.0

import java.nio.ByteBuffer;
import java.nio.ByteOrder;

byte[] test = new byte[] {
  -107, 3, 53, 87, 126, 46, 49, -64, 
  -34, 113, 81, 2, -1, -14, 37, 64, 
  0, 0, 0, 0, 0, -64, 82, 64, 
  0, 0, 0, 0, 0, 0, 0, 0, 
  0, 0, 0, 0, 0, 0, 0, 0, 
  0, 0, 0, 0, 0, 0, 0, 0
};

void setup() {
  ByteBuffer bb = ByteBuffer.allocate(test.length);
  bb.order(ByteOrder.LITTLE_ENDIAN);
  bb.put(test);
  bb.rewind();
  float xx = (float) bb.getDouble();
  float yy = (float) bb.getDouble();
  float zz = (float) bb.getDouble();
  println(xx);
  println(yy);
  println(zz);
  exit();
}
1 Like
18

I did see references to little endian, and those are the exact numbers I was expecting for those bytes, thank you! I think I’m starting to understand it now that I see all of the parts working together.

Here is the final code, but I will also start a new thread with this code in case anyone wants to do something similar (here).

import hypermedia.net.*;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;

UDP udp;
int PORT_RX= 7000; //port : set in opentrack
String HOST_IP="192.168.86.28"; // must be computer IP address, use ipconfig to find
String receivedFromUDP = "";
int x,y,z;

void setup() {
  size(200, 100);
  udp= new UDP(this, PORT_RX, HOST_IP);
  //udp.log(true);
  udp.listen(true);
  super.start();
}

void draw() {
  background(0);
  textSize(20);
  text(x, 20, 30);
  text(y, 20, 50);
  text(z, 20, 70);
}

void receive(byte[] data) {    
  ByteBuffer bb = ByteBuffer.allocate(data.length);
  bb.order(ByteOrder.LITTLE_ENDIAN);
  bb.put(data);
  bb.rewind();
  x = (int) bb.getDouble();
  y = (int) bb.getDouble();
  z = (int) bb.getDouble();
  //println("x,y,z: " + xx + ", " + yy + ", " + zz);
}

float byteArrayToFloat(byte[] data) {
  int intBits = 
    data[3] << 24 | (data[2] & 0xFF) << 16 | (data[1] & 0xFF) << 8 | (data[0] & 0xFF) |
    data[4] << 32 | (data[5] & 0xFF) << 40 | (data[6] & 0xFF) << 48 | (data[7] & 0xFF) << 56;
  return Float.intBitsToFloat(intBits);
}