I suspect that it may be your use of the “yaw” angle that is the issue; this was the case for me with my 3-axis accelerometer.
Here is a discussion about the “yaw” angle:
https://robotics.stackexchange.com/questions/4677/how-to-estimate-yaw-angle-from-tri-axis-accelerometer-and-gyroscope
I am starting to program a quaternion class and this device is now of interest to me.
According to Adafruit it will “spit out data you can use in quaternions, Euler angles or vectors.”
https://learn.adafruit.com/adafruit-bno055-absolute-orientation-sensor/overview
Have fun!