How to use data from accelerometer to control camera and perspective in Processing

Processing Electronics (Arduino, etc.)
1

I’m new to the forum, I wonder if anyone can help me to use data from accelerometer to control camera and perspective in processing? I’m making a 3D model of a galaxy and wish to use accelerometer to control the perspective (zoom in or out, etc.) Here is my codes right now. The 3D model and serial part is done but I don’t know how to use the acceleration value
import processing.serial.*;
Serial myPort;
String inString ;
int[] data = new int [3];
int xA = int(data[0]);
int yA = int(data[1]);
int zA = int(data[2]);

Planet sun;
//PImage starfield;
PImage sunTexture;
PImage[] textures = new PImage[10];

float fov = PI/3.0;
float cameraZ = (height/2.0) / tan(fov/2.0);

void setup() {
size(600, 375, P3D);
//fullScreen(P3D);

myPort = new Serial(this, Serial.list()[1], 9600);
println(Serial.list()[1]);
myPort.bufferUntil(’\n’);

//perspective(fov, float(width)/float(height), cameraZ/10.0, cameraZ*10.0);
//translate(width/2, height/2, 0);
//rotateX(-PI/6);
//rotateY(PI/3);

sunTexture = loadImage(“sun.jpg”);
textures[0] = loadImage(“dirt.jpg”);
textures[1] = loadImage(“eris.jpg”);
textures[2] = loadImage(“golden.jpg”);
textures[3] = loadImage(“green.jpg”);
textures[4] = loadImage(“ice.jpg”);
textures[5] = loadImage(“lava.jpg”);
textures[6] = loadImage(“north.jpg”);
textures[7] = loadImage(“ocean.jpg”);
textures[8] = loadImage(“pink.jpg”);
textures[9] = loadImage(“purple.jpg”);

//starfield = loadImage(“starfield.jpeg”);

sun = new Planet(50, 0, 0, sunTexture);
sun.spawnMoons(10, 1);
}

void draw() {
//background(starfield);
//translate(width/2, height/2, -width);
background(0);
sun.show();
sun.orbit();
perspective(fov, float(width)/float(height), cameraZ/10.0, cameraZ*10.0);
translate(width/2, height/2,-width);
//translate(xA,yA,zA);
//rotateX(-PI/6);
//rotateY(PI/3);

camera(width/2.0, height/2.0, (height/2.0) / tan(PI30.0 / 180.0), (width/2.0)-300, (height/2.0)-187, 0, 0, 1, 0);
//camera(xA/100, yA/100, zA/100, (width/2.0)-800, (height/2.0)-500, 0, 0, 1, 0);
//camera(0, 0, (height/2) / tan(PI30.0 / 180.0), width/2.0, height/2.0, cameraZ, 0, 1, 0);
//translate(width/2,height/2,mouseX);
//lights();
//pointLight(255,255,255,0,0,0);
}

void serialEvent(Serial myPort) {
String inString = myPort.readStringUntil(’\n’);
if (inString != null) {

inString = trim(inString);
println(inString);
data = int (split(inString, ','));
//println(data[0]);

}
}

2

Okay, great idea!

Is the accelerometer on an Arduino?

Can you please tell us the product name of the accelerometer? It should come with a documentation what data are sent.

3

Yes, the accelerometer is an Arduino sensor. The product name is: Gravity-I2C LIS331HH 3-axis Accelerometer, made by DFRobot. The link to its illustration is: Gravity: I2C LIS331HH Triple Axis Accelerometer Sensor Wiki - DFRobot
Thanks for noticing me!

1 Like
4

What is the result of this please?

When it’s something like [x=…, y=…, z= …]
you need to get the numbers without the other letters/ signs.

Store them in 3 variables.

Since this is not the position in 3D space but acceleration, we need to define a camera position that gets changed by the values (x,y,z) maybe with a factor like times 0.1 or times 10 and maybe with easing (that’s for later).

so

camX += x*10;
camY += y*10;
camZ += z*10;

And use the 3 in camera command as “eye”, see reference

(not tested)

5

The result is like: num, num, num
In the Arduino side I wrote:

void loop(void){

    //Get the acceleration in the three directions of xyz
    //The mearsurement range can be ±6g, ±12g or ±24g, set by the setRange() function
    long ax,ay,az;
    ax = acce.readAccX();//Get the acceleration in the x direction
    ay = acce.readAccY();//Get the acceleration in the y direction
    az = acce.readAccZ();//Get the acceleration in the z direction
    //Serial.print("x: "); //print acceleration
    Serial.print(ax);
    Serial.print(",");
    Serial.print(ay);
    Serial.print(",");
    Serial.println(az);
    //Serial.println("\n");
    //Serial.println(" mg");
    delay(100);
}

So Processing side’s output is like:
130896,131001,130193
310,130987,129900
345,213,130251
427,468,130348
468,380,130330
424,418,130357
(just a few examples, the numbers went on and on)
The problem is, I’m not a master in physics and not know how this data is measured…

Now I’m also thinking maybe I should consider the physic laws about acceleration, velocity, and displacement, such as

import processing.serial.*;
Serial myPort;
String inString ;
int[] data = new int [3];
int xA = int(data[0]);
int yA = int(data[1]);
int zA = int(data[2]);

float xV = xA*0.1;
float yV = yA*0.1;
float zV = zA*0.1;

float xS = 1/2*xA*0.01+xV*0.1;
float yS = 1/2*yA*0.01+yV*0.1;
float zS = 1/2*zA*0.01+zV*0.1;

But I still do not know what to fill as the parameters in camera() and perspective()
now I’m writing

perspective(fov, float(width)/float(height), cameraZ/10.0, cameraZ*10.0);
translate(width/2, height/2,-width);
camera(xS, yS, zS, (width/2.0)-300, (height/2.0)-187, 0, 0, 1, 0);
1 Like
6

xS, yS, zS is the eye (= camera position).

camera(eyeX, eyeY, eyeZ, 
centerX, centerY, centerZ, 
upX, upY, upZ);

So, it’s good that you define xS, yS, zS etc.

Does it work? Also try my idea.

And use float throughout when possible.

7

The xS, yS, zS doesn’t work for now. If I run it, the whole screen went black and my planets can’t be shown.
Now I figured out how to paste codes here. You can take a look of what I have right now. I didn’t include the Planet class initially

import processing.serial.*;
Serial myPort;
String inString ;
int[] data = new int [3];
int xA = int(data[0]);
int yA = int(data[1]);
int zA = int(data[2]);

float xV = xA*0.1;
float yV = yA*0.1;
float zV = zA*0.1;

float xS = 1/2*xA*0.01+xV*0.1;
float yS = 1/2*yA*0.01+yV*0.1;
float zS = 1/2*zA*0.01+zV*0.1;


Planet sun;
PImage sunTexture;
PImage[] textures = new PImage[10];


float fov = PI/3.0;
float cameraZ = (height/2.0) / tan(fov/2.0);

void setup() {
  size(600, 375, P3D);

  myPort = new Serial(this, Serial.list()[1], 9600);
  println(Serial.list()[1]);
  myPort.bufferUntil('\n');


  sunTexture = loadImage("sun.jpg");
  textures[0] = loadImage("dirt.jpg");
  textures[1] = loadImage("eris.jpg");
  textures[2] = loadImage("golden.jpg");
  textures[3] = loadImage("green.jpg");
  textures[4] = loadImage("ice.jpg");
  textures[5] = loadImage("lava.jpg");
  textures[6] = loadImage("north.jpg");
  textures[7] = loadImage("ocean.jpg");
  textures[8] = loadImage("pink.jpg");
  textures[9] = loadImage("purple.jpg");


  sun = new Planet(50, 0, 0, sunTexture);
  sun.spawnMoons(10, 1);
}

void draw() {

  background(0);
  sun.show();
  sun.orbit();
  perspective(fov, float(width)/float(height), cameraZ/10.0, cameraZ*10.0);
  translate(width/2, height/2,-width);
  
  camera(xS, yS, zS, (width/2.0)-300, (height/2.0)-187, 0, 0, 1, 0);
  
}


void serialEvent(Serial myPort) {
  String inString = myPort.readStringUntil('\n');
  if (inString != null) {

    inString = trim(inString);
    println(inString);
    data = int (split(inString, ','));

  }
}
class Planet {
  float radius;
  float angle;
  float distance;
  Planet[] planets;
  float orbitspeed;
  PVector v;
  
  PShape globe;
  

  Planet(float r, float d, float o, PImage img) {
    v = PVector.random3D();
    radius = r;
    distance = d;
    v.mult(distance);
    angle = random(TWO_PI);
    orbitspeed = o;
    
    noStroke();
    noFill();
    globe = createShape(SPHERE,radius);
    globe.setTexture(img);
  }
  void orbit() {
    angle = angle + orbitspeed;
    if (planets != null) {
      for (int i=0; i<planets.length; i++) {
        planets[i].orbit();
      }
    }
  }
  void spawnMoons(int total, int level) {
    planets = new Planet[total];
    for (int i=0; i<planets.length; i++) {
      float r = radius/(level*4);
      float d = i*6 + random((radius + r), (radius+r)*6);
    
      float o = random(-0.01, 0.01);
      
      planets[i] = new Planet(r, d, o, textures[i]);
      if (level < 2) {
        int num = int(random(0,2));
        planets[i].spawnMoons(num, level+1);
      }
    }
  }


  void show() {
    pushMatrix();
    noStroke();
    fill(255);
    //rotate(angle);
    PVector v2 = new PVector(1,0,1);
    PVector p = v.cross(v2);
    rotate(angle,p.x,p.y,p.z);

    
    translate(v.x,v.y,v.z);
    
    
    shape(globe);
    if(distance ==0){
      pointLight(255,255,255,300,187,0);
    }
    

    if (planets != null) {
      for (int i=0; i<planets.length; i++) {
        planets[i].show();
      }
    }
    popMatrix();
  }
}

It’s a bit messy but you can take it as reference. I’m trying your idea right now.

1 Like
8

This won’t work when you only write it before setup().
In this way, it’s not executed throughout. That’s very important
to notice. The “=” sign is a command that gets only executed when the line is run and not throughout.

Repeat it in the function serialEvent without the data type (otherwise especially xS,yS,zS would not be global and therefore not known in draw())

xA = int(data[0]); // float ? 
yA = int(data[1]);
zA = int(data[2]);

xV = xA*0.1;
yV = yA*0.1;
zV = zA*0.1;

xS = 1/2*xA*0.01+xV*0.1;
yS = 1/2*yA*0.01+yV*0.1;
zS = 1/2*zA*0.01+zV*0.1;
9

I guess this should happen at the start of draw(), not at its end.

Also translate() must be before you draw the solar system, not after it.

I would also comment out the perspective command for now:

    //  perspective(fov, float(width)/float(height), cameraZ/10.0, cameraZ*10.0);
10

I revised in this way and I got the message of: “Error, disabling serialEvent() for (my USBport)” and null as output…

11

okay, we have to check this!

Meanwhile I made a version without serial,
when you press a key, camera changes.

(The key press simulates a
accelerometer / Acceleration sensor change.)



String inString ;

float xS =width/2;
float yS =height/2;
float zS =333;

Planet sun;
PImage sunTexture;
PImage[] textures = new PImage[10];

float fov = PI/3.0;
// float cameraZ = (height/2.0) / tan(fov/2.0);

// ---------------------------------------------------------------------------

void setup() {
  size(1600, 1375, P3D);

  //myPort = new Serial(this, Serial.list()[1], 9600);
  // println(Serial.list()[1]);
  // myPort.bufferUntil('\n');

  /*
  sunTexture = loadImage("sun.jpg");
   textures[0] = loadImage("dirt.jpg");
   textures[1] = loadImage("eris.jpg");
   textures[2] = loadImage("golden.jpg");
   textures[3] = loadImage("green.jpg");
   textures[4] = loadImage("ice.jpg");
   textures[5] = loadImage("lava.jpg");
   textures[6] = loadImage("north.jpg");
   textures[7] = loadImage("ocean.jpg");
   textures[8] = loadImage("pink.jpg");
   textures[9] = loadImage("purple.jpg");
   */

  sun = new Planet(50, 0, 0, null);
  sun.spawnMoons(10, 1);
}

void draw() {
  background(0);
  lights(); 

  camera(xS, yS, zS, 
    (width/2.0)-300, (height/2.0)-187, 0, 
    0, 1, 0);

  // perspective(fov, float(width)/float(height), cameraZ/10.0, cameraZ*10.0);
  translate(width/2, height/2, -width); 

  sun.show();
  sun.orbit();
}

// void serialEvent(Serial myPort) {
void keyPressed() {

  int[] data = new int [3];
  int xA = int(data[0]);
  int yA = int(data[1]);
  int zA = int(data[2]);

  float xV = xA*0.1;
  float yV = yA*0.1;
  float zV = zA*0.1;

  // String inString = myPort.readStringUntil('\n');
  String inString=" 345,213,130251"; 
  if (inString != null) {
    inString = trim(inString);
    println(inString);
    data = int (split(inString, ','));

    xA = int(data[0]);
    yA = int(data[1]);
    zA = int(data[2]);

    xV = xA*0.1;
    yV = yA*0.1;
    zV = zA*0.1;

    xS = 1/2*xA*0.01+xV*0.1;
    yS = 1/2*yA*0.01+yV*0.1;
    zS = 1/2*zA*0.01+zV*0.1;
  }
}

// ========================================================================================

class Planet {
  float radius;
  float angle;
  float distance;
  Planet[] planets;
  float orbitspeed;
  PVector v;

  PShape globe;


  Planet(float r, float d, float o, PImage img) {
    v = PVector.random3D();
    radius = r;
    distance = d;
    v.mult(distance);
    angle = random(TWO_PI);
    orbitspeed = o;

    noStroke();
    noFill();
    globe = createShape(SPHERE, radius);
    //  globe.setTexture(img);
  }

  void orbit() {
    angle = angle + orbitspeed;
    if (planets != null) {
      for (int i=0; i<planets.length; i++) {
        planets[i].orbit();
      }
    }
  }
  void spawnMoons(int total, int level) {
    planets = new Planet[total];
    for (int i=0; i<planets.length; i++) {
      float r = radius/(level*4);
      float d = i*6 + random((radius + r), (radius+r)*6);

      float o = random(-0.01, 0.01);

      planets[i] = new Planet(r, d, o, textures[i]);
      if (level < 2) {
        int num = int(random(0, 2));
        planets[i].spawnMoons(num, level+1);
      }
    }
  }


  void show() {
    pushMatrix();
    noStroke();
    fill(255);
    //rotate(angle);
    PVector v2 = new PVector(1, 0, 1);
    PVector p = v.cross(v2);
    rotate(angle, p.x, p.y, p.z);


    translate(v.x, v.y, v.z);


    // shape(globe);
    sphere (112);

    if (distance ==0) {
      pointLight(255, 255, 255, 300, 187, 0);
    }


    if (planets != null) {
      for (int i=0; i<planets.length; i++) {
        planets[i].show();
      }
    }
    popMatrix();
  }
}
12

for your error:

myPort = new Serial(this, Serial.list()[1], 9600);

I sometimes see this

   String portName = Serial.list()[0];

   //    myPort = new Serial(this, "COM16", 9600);
   myPort = new Serial(this, portName, 9600);

OR

myPort = new Serial(this, Serial.list()[0], 9600);

with 0 maybe?

13

This looks interesting! But when I run it on my computer, it pops up “Syntax Error: You may be mixing active and static modes.”
I did something similar before, I used Peasy Camera library to realize controlling the camera with mouse:)

14

You are totally right. I changed to 0 and not it’s receiving the data from the USB port. However, the screen is still in absolute dark:(((so sad

15

runs here when I copy it from the forum.

maybe you missed a closing } at the end?

16

we are making progress!!

so, please before setup()

replace the xS, yS,zS lines with this:

float xS =width/2;
float yS =height/2;
float zS =333;
17

I wonder why zS = 333?
The solar system appeared!!! But it’s not at the center of the screen. It’s slanted to the lower right, I assume it might relate to the screen size.

1 Like
18

Also, it’s not receiving Serial data…

1 Like
19

well I added another }
hmmm… :thinking:

20

Excuse me, I gotta go now. I am sorry.

Chrisir