Request for guidance:- Making 3D objects appear static on a webcam feed using arduino & an IMU:

I hope some of this helps…

Arduino:

  1. Sends quaternion data (X, Y, Z, W) to serial port as a string and terminates with a ‘/n’.
    I used the Adafruit library for this.

Processing:

  1. Reads quaternion data and stores it in an array
    readQuatData(); // Stored in q[]

  2. Convert quaternions to Euler angles
    quatToEuler_1(q, Euler);

  3. Rotate
    rotateEuler();

  4. Draw to canvas

I also displayed data to screen and axis of the BNO055, Euclidean formulas and Processing display to help me visualize all of this. The screen follows the orientation of my device!

Note:
I also had code to convert to rotate matrix from quaternions.
rotateMatrix();

This site was very useful:

//**************************************************************************************************
// Example #7 EuclideanSpace - Mathematics and Computing
//
// Can be tough to navigate!
//
// https://www.euclideanspace.com/maths/geometry/rotations/conversions/index.htm
// https://www.euclideanspace.com/maths/geometry/rotations/conversions/quaternionToEuler/index.htm
// http://www.euclideanspace.com/maths/standards/index.htm
// https://www.euclideanspace.com/maths/geometry/rotations/index.htm

Once I defined my co-ordinate systems and mapped them I just had to piece it all together.
I had to make lots of notes and wrap my head around the different co-ordinate systems and mapping\converting them from one to another.
I wrote all of my own code with the exception of the working formulas from the website.

Some notes:

//**************************************************************************************************
// GLV Notes
// Things to consider:
// 1) Axis of device; in this case BNO055
//
// 2) Conversion of Quaternion to Euler
//
// 3) Convert from one coordinate system to another
//
// 4) Coordinate system of code used; this varies widely!  
//
// 4) Convert from device > world > display
//    BNO055 > Euclidian > Processing
// 
// Attempt to map BNO055 to Euclidean to Processing:

// B        E         P
// +X P/A   -Z R/B    +Z           
// +Y R/B   -X P/A    -X 
// +Z Y/H   +Y Y/H    -Y         
//
//************************************************************************************************** 
// Coordinate systems - GLV hand stitched
// 
// // 1) BNO055 (RHS) x, y, z: 
//                        
//                      +z         P       
//                       *        +x
//                       *       * 
//                       *     *
//                       *   *
//           R           * *
//          +y * * * * * *              
//     
// 2) Euclidean (RHS) x', y', z':   
//                         
//                       +y'(+z)    P
//                        *        -z'(+x)
//                        *       *      
//                        *     *
//                        *   *
//       R                * *
//      -x'(+y) * * * * * * 

// Processing (LHS) x", y", z":             using scale (1, 1, 1) default
//                             
//                       -y"        P  
//                        *        -z"  
//                        *       *
//                        *     *
//                        *   *
//           R            * *
//          -x" * * * * * *
//
// Processing (RHS):                         using scale(1, 1, -1)
//                             
//                       -y"        P  
//                        *        z"  
//                        *       *
//                        *     *
//                        *   *
//           R            * *
//          -x" * * * * * *
//
//
//**************************************************************************************************
// Eucidean mapping:
//
// BNO055     QE        Processing
// X  att     Z att       Z att
// Y  roll    X roll      Y head
// Z  head    Y head      X roll

// Euclidean:
// Heading = rotation about y axis
// Attitude = rotation about z axis
// Bank = rotation about x axis
//
// Heading applied first
// Attitude applied second
// Bank applied last 

//****************************************************

After all was said and done I can go back and remap things differently one of these days.

2 Likes

Thanks @glv . I am going to give this a shot and share what I end up with!

So, just to make sure I am understanding this correcty, what this video demonstrates is the sphere rotating in sync with the rotation of the sensor; am I correct?

And to achieve this:
a) Pull in quaternion data from Arduino to Processing via serial (which I’ve done)
b) Store data in an array & convert to Euler (in Processing?)
c) Rotate the object based on the Euler data
d) Display / draw the object

Am I getting this right?

Thank you!

The screen images move in sync with device.

a) Great! Send the data received to the Processing canvas or console to visualize; this helps debugging in this step and after converting.
b) In Processing I used an array with 4 elements; you could have used 4 variables such as qx, qy, qz and qw unless you want to store data. I was using data received in real time.
The data I was receiving was in a string and I had to convert to floats.
I put these in another array euler[] to store yaw, pitch, roll.
c) Rotate in the correct order if using Euler angles. I used matrices later for this and was able to choose.
d) Draw.

I make it sound easier than it look! This took me a couple of weekends to really understand it.

Don’t worry about final orientation until last; this took some effort to get from co-ordinate system of device to co-ordinate system of the screen. I flipped the Z scale(1, 1, -1) but also had to change camera which complicated things; I will rework this for future.

The Euclidean site has some canned formulas and he does define his co-ordinate systems.

I also used functions and\or the keyboard to generate data for debugging and I can switch between incoming serial data and “local” data from keyboard or functions.

:slight_smile:

1 Like

On a humorous note…
The green spheres I used were generated in another project.
They used one of my other videos in this article:


as “an example of some of the content that might be in the second message” to aliens.
I took the data from the YouTube analytics for the video and mapped it on a sphere.
That was fun!

2 Likes

So the green spheres / markers on the sphere are IP address distributions? What did you map? :slight_smile:

I will provide a “bare bones” example of how to use some of the formulas from the Euclidean website with keyboard control for the community. :slight_smile:

More to follow later today…

:slight_smile:

2 Likes

I provided a separate topic to provide examples to community:

Enjoy the journey!

:slight_smile:

1 Like

The YouTube video had an Analytics > Geography section and has data for Geography (country) and “Views”, “Watch time” and other data. I got the geographic coordinates (lat and long) elsewhere and was able to plot these on a sphere.

:slight_smile: