# Request for guidance:- Making 3D objects appear static on a webcam feed using arduino & an IMU:

**URL:** <https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902>\
**Category:** Electronics (Arduino, etc.)\
**Created:** [May 4, 2019, 9:34am UTC](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902 "2019-05-04T09:34:48Z")\
**Posts on this page:** 9\
**Page:** 2

<div class="post-metadata">

**Author:** ![glv](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/glv/32/18785_2.png) [@glv](https://discourse.processing.org/u/glv)\
**Post date:** [May 12, 2019, 4:53pm UTC](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902/21 "2019-05-12T16:53:54Z")

</div>

I hope some of this helps…

Arduino:

1. Sends quaternion data (X, Y, Z, W) to serial port as a string and terminates with a ‘/n’.  
I used the Adafruit library for this.

Processing:

1. Reads quaternion data and stores it in an array  
readQuatData(); // Stored in q[]

2. Convert quaternions to Euler angles  
quatToEuler\_1(q, Euler);

3. Rotate  
rotateEuler();

4. Draw to canvas

I also displayed data to screen and axis of the BNO055, Euclidean formulas and Processing display to help me visualize all of this. The screen follows the orientation of my device!

Note:  
I also had code to convert to rotate matrix from quaternions.  
rotateMatrix();

This site was very useful:

//\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
// Example #7 EuclideanSpace - Mathematics and Computing  
//  
// Can be tough to navigate!  
//  
// [https://www.euclideanspace.com/maths/geometry/rotations/conversions/index.htm](https://www.euclideanspace.com/maths/geometry/rotations/conversions/index.htm)  
// [https://www.euclideanspace.com/maths/geometry/rotations/conversions/quaternionToEuler/index.htm](https://www.euclideanspace.com/maths/geometry/rotations/conversions/quaternionToEuler/index.htm)  
// [http://www.euclideanspace.com/maths/standards/index.htm](http://www.euclideanspace.com/maths/standards/index.htm)  
// [https://www.euclideanspace.com/maths/geometry/rotations/index.htm](https://www.euclideanspace.com/maths/geometry/rotations/index.htm)

Once I defined my co-ordinate systems and mapped them I just had to piece it all together.  
I had to make **lots of notes** and wrap my head around the different co-ordinate systems and mapping\converting them from one to another.  
I wrote all of my own code with the exception of the working formulas from the website.

Some notes:

```auto
// **************************************************************************************************
// GLV Notes
// Things to consider:
// 1) Axis of device; in this case BNO055
//
// 2) Conversion of Quaternion to Euler
//
// 3) Convert from one coordinate system to another
//
// 4) Coordinate system of code used; this varies widely!  
//
// 4) Convert from device > world > display
// BNO055 > Euclidian > Processing
// 
// Attempt to map BNO055 to Euclidean to Processing:

// B E P
// +X P/A -Z R/B +Z           
// +Y R/B -X P/A -X 
// +Z Y/H +Y Y/H -Y         
//
// ************************************************************************************************** 
// Coordinate systems - GLV hand stitched
// 
// // 1) BNO055 (RHS) x, y, z: 
//                        
// +z P       
// * +x
// * * 
// * *
// * *
// R * *
// +y * * * * * *              
//     
// 2) Euclidean (RHS) x', y', z':   
//                         
// +y'(+z) P
// * -z'(+x)
// * *      
// * *
// * *
// R * *
// -x'(+y) * * * * * * 

// Processing (LHS) x", y", z": using scale (1, 1, 1) default
//                             
// -y" P  
// * -z"  
// * *
// * *
// * *
// R * *
// -x" * * * * * *
//
// Processing (RHS): using scale(1, 1, -1)
//                             
// -y" P  
// * z"  
// * *
// * *
// * *
// R * *
// -x" * * * * * *
//
//
// **************************************************************************************************
// Eucidean mapping:
//
// BNO055 QE Processing
// X att Z att Z att
// Y roll X roll Y head
// Z head Y head X roll

// Euclidean:
// Heading = rotation about y axis
// Attitude = rotation about z axis
// Bank = rotation about x axis
//
// Heading applied first
// Attitude applied second
// Bank applied last 

// ****************************************************

```

After all was said and done I can go back and remap things differently one of these days.

---

<div class="post-metadata">

**Author:** ![moh\_maya](https://avatars.discourse-cdn.com/v4/letter/m/5f9b8f/32.png) [@moh\_maya](https://discourse.processing.org/u/moh_maya)\
**Post date:** [May 12, 2019, 6:29pm UTC](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902/22 "2019-05-12T18:29:11Z")

</div>

Thanks @glv . I am going to give this a shot and share what I end up with!

---

<div class="post-metadata">

**Author:** ![moh\_maya](https://avatars.discourse-cdn.com/v4/letter/m/5f9b8f/32.png) [@moh\_maya](https://discourse.processing.org/u/moh_maya)\
**Post date:** [May 13, 2019, 6:41am UTC](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902/23 "2019-05-13T06:41:03Z")

</div>

So, just to make sure I am understanding this correcty, what this video demonstrates is the sphere rotating in sync with the rotation of the sensor; am I correct?

And to achieve this:  
a) Pull in quaternion data from Arduino to Processing via serial (which I’ve done)  
b) Store data in an array & convert to Euler (in Processing?)  
c) Rotate the object based on the Euler data  
d) Display / draw the object

Am I getting this right?

Thank you!

---

<div class="post-metadata">

**Author:** ![glv](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/glv/32/18785_2.png) [@glv](https://discourse.processing.org/u/glv)\
**Post date:** [May 13, 2019, 11:23am UTC](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902/24 "2019-05-13T11:23:55Z")

</div>

The screen images move in sync with device.

a) Great! Send the data received to the Processing canvas or console to visualize; this helps debugging in this step and after converting.  
b) In Processing I used an array with 4 elements; you could have used 4 variables such as qx, qy, qz and qw unless you want to store data. I was using data received in real time.  
The data I was receiving was in a string and I had to convert to floats.  
I put these in another array euler[] to store yaw, pitch, roll.  
c) Rotate in the correct order if using Euler angles. I used matrices later for this and was able to choose.  
d) Draw.

I make it sound easier than it look! This took me a couple of weekends to really understand it.

Don’t worry about final orientation until last; this took some effort to get from co-ordinate system of device to co-ordinate system of the screen. I flipped the Z scale(1, 1, -1) but also had to change camera which complicated things; I will rework this for future.

The Euclidean site has some canned formulas and he does define his co-ordinate systems.

I also used functions and\or the keyboard to generate data for debugging and I can switch between incoming serial data and “local” data from keyboard or functions.

🙂

---

<div class="post-metadata">

**Author:** ![glv](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/glv/32/18785_2.png) [@glv](https://discourse.processing.org/u/glv)\
**Post date:** [May 13, 2019, 11:36am UTC](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902/25 "2019-05-13T11:36:33Z")

</div>

On a humorous note…  
The green spheres I used were generated in another project.  
They used one of my other videos in this article:

> **[Scientists Just Sent A Message To Aliens And Hope To Get A Reply In 25 Years](https://www.iflscience.com/space/scientists-just-sent-a-message-to-aliens-and-hope-to-get-a-reply-in-25-years/)**
>
> A group of scientists has revealed they have sent a detailed message to a nearby planet in the hope of contacting alien life – and hope to send another soo

  
as “an example of some of the content that might be in the second message” to aliens.  
I took the data from the YouTube analytics for the video and mapped it on a sphere.  
That was fun!

---

<div class="post-metadata">

**Author:** ![moh\_maya](https://avatars.discourse-cdn.com/v4/letter/m/5f9b8f/32.png) [@moh\_maya](https://discourse.processing.org/u/moh_maya)\
**Post date:** [May 13, 2019, 3:03pm UTC](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902/26 "2019-05-13T15:03:06Z")

</div>

> [@glv](#):
>
> I took the data from the YouTube analytics for the video and mapped it on a sphere.  
> That was fun!

So the green spheres / markers on the sphere are IP address distributions? What did you map? 🙂

---

<div class="post-metadata">

**Author:** ![glv](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/glv/32/18785_2.png) [@glv](https://discourse.processing.org/u/glv)\
**Post date:** [May 14, 2019, 6:16pm UTC](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902/27 "2019-05-14T18:16:05Z")

</div>

I will provide a “bare bones” example of how to use some of the formulas from the Euclidean website with keyboard control for the community. 🙂

More to follow later today…

> **[glvjr/BNO055](https://github.com/glvjr/BNO055)**
>
> Contribute to glvjr/BNO055 development by creating an account on GitHub.

🙂

---

<div class="post-metadata">

**Author:** ![glv](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/glv/32/18785_2.png) [@glv](https://discourse.processing.org/u/glv)\
**Post date:** [May 15, 2019, 11:14am UTC](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902/28 "2019-05-15T11:14:14Z")

</div>

I provided a separate topic to provide examples to community:

> [@Quaternions and Euler Rotations Exploration](https://discourse.processing.org/t/quaternions-and-euler-rotations-exploration/11262):
>
> Hello, I did an exploration of rotations with quaternions and Euler angles and sharing the “bare bones” code I wrote. The original project (a year ago) got out of hand and I really need to clean up the code and references and this was a start. This code was written to test formulas from: [http://www.euclideanspace.com/maths/geometry/rotations/conversions/index.htm](http://www.euclideanspace.com/maths/geometry/rotations/conversions/index.htm) This program: Generates Euler angles with keyboard control Euler angles are converted to quaternions quaternions are converted …

Enjoy the journey!

🙂

---

<div class="post-metadata">

**Author:** ![glv](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/glv/32/18785_2.png) [@glv](https://discourse.processing.org/u/glv)\
**Post date:** [May 15, 2019, 11:05pm UTC](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902/29 "2019-05-15T23:05:01Z")

</div>

The YouTube video had an Analytics \> Geography section and has data for Geography (country) and “Views”, “Watch time” and other data. I got the geographic coordinates (lat and long) elsewhere and was able to plot these on a sphere.

🙂

[Previous page](https://discourse.processing.org/t/request-for-guidance-making-3d-objects-appear-static-on-a-webcam-feed-using-arduino-an-imu/10902.md?page=1)