Mapping Sound to 3D Terrain generation


#1

Hey everyone,
I’m just a Beginner in Processing, so I have a question:
is it possible to map a sound to a 3D Terrain Generation?

With the help of this tutorial: https://www.youtube.com/watch?v=IKB1hWWedMk I created a 3D Terrain. So this is the basic code:

int rows;
int columns;
int scale = 20;
int w = 1200;
int h = 900;
float [] [] terrain;
float flying = 0;

void setup() {
size(600,600, P3D);
columns = w / scale;
rows = h / scale; 
terrain = new float [columns] [rows];
}

void draw() {  
flying -= 0.05;
float yoff = flying;
for (int y = 0; y < rows; y++) {
   float xoff = 0;
   for (int x = 0; x < columns; x++) {
   terrain [x] [y] = map(noise (xoff,yoff), 0, 1, -160, 120);
xoff += 0.1;
 }
yoff += 0.1;
 }

background(0, 0, 100);
stroke (150, 0, 0);
noFill();
translate (width/2, height/2);
rotateX(PI/3);

translate(-w/2, -h/2);
for (int y = 0; y < rows-1; y++) {
   beginShape(TRIANGLE_STRIP);
 for (int x = 0; x < columns; x++) {
   vertex(x*scale, y*scale, terrain [x] [y]);
   vertex(x*scale, (y+1)*scale, terrain [x] [y+1]);
   //rect(x*scale, y*scale, scale, scale);
   
}
   endShape();
}
}

The “mountains” of the terrain are moving because randomly because I used “noise”. But now I want them to move to a sound/beat. So I thought the next steps are creating a player and loading a sound. I already know, that I have to analyze the sound with FFT, but I don’t get exactly what to do.
I thought that I could use this “terrain [x] [y] = map(noise (xoff,yoff), 0, 1, -160, 120);” to map the sound instead of the noise.

I added a bit, so now I have this code:

 import ddf.minim.analysis.*;
 import ddf.minim.*;

Minim minim;
AudioPlayer player;
FFT         fft;

int rows;
int columns;
int scale = 20;

int w = 1200;
int h = 900;
float [] [] terrain;

float flying = 0;

void setup() {

minim = new Minim(this);
player = minim.loadFile("song.mp3");
player.loop();

fft = new FFT( player.bufferSize(), player.sampleRate() );

size(600,600, P3D);
columns = w / scale;
rows = h / scale; 
terrain = new float [columns] [rows];

 }

 void draw() {  
//fft.forward (player.mix );

  flying -= 0.05;
    float yoff = flying;
    for (int y = 0; y < rows; y++) {
    float xoff = 0;
    for (int x = 0; x < columns; x++) {
     terrain [x] [y] = map(noise (xoff,yoff), 0, 1, -160, 120);
     xoff += 0.1;
   
   }
   yoff += 0.1;
   }
  background(0, 0, 100);
   stroke (150, 0, 0);
    noFill();
   
  translate (width/2, height/2);    
  rotateX(PI/3); 

  translate(-w/2, -h/2);
  for (int y = 0; y < rows-1; y++) {
   beginShape(TRIANGLE_STRIP);
  for (int x = 0; x < columns; x++) {
   vertex(x*scale, y*scale, terrain [x] [y]);
   vertex(x*scale, (y+1)*scale, terrain [x] [y+1]);
   //rect(x*scale, y*scale, scale, scale);
   
  }
   endShape();
  }
  }

But I really got stucked. Because I’m a Beginner I actually don’t know what’s possible and I couldn’t find anything to map a sound to a 3D terrain.

I found this example:

import ddf.minim.analysis.*;
import ddf.minim.*;

Minim       minim;
AudioPlayer jingle;
FFT         fft;

void setup()
{
size(512, 200, P3D);

 minim = new Minim(this);

 // specify that we want the audio buffers of the AudioPlayer
 // to be 1024 samples long because our FFT needs to have 
 // a power-of-two buffer size and this is a good size.
jingle = minim.loadFile("jingle.mp3", 1024);

 // loop the file indefinitely
jingle.loop();

 // create an FFT object that has a time-domain buffer 
 // the same size as jingle's sample buffer
// note that this needs to be a power of two 
// and that it means the size of the spectrum will be half as large.
fft = new FFT( jingle.bufferSize(), jingle.sampleRate() );

}

void draw()
{
background(0);
stroke(255);

// perform a forward FFT on the samples in jingle's mix buffer,
// which contains the mix of both the left and right channels of the file
fft.forward( jingle.mix );

for(int i = 0; i < fft.specSize(); i++)
{
 // draw the line for frequency band i, scaling it up a bit so we can see it
line( i, height, i, height - fft.getBand(i)*8 );
}
}

But I don’t know how to combine

flying -= 0.05;
float yoff = flying;
for (int y = 0; y < rows; y++) {
   float xoff = 0;
   for (int x = 0; x < columns; x++) {
     terrain [x] [y] = map(noise (xoff,yoff), 0, 1, -160, 120);
     xoff += 0.1;

from the first code with

for(int i = 0; i < fft.specSize(); i++)

from the last code.

I hope that someone could help me!
Celine


#2

so i understand you want mix

with
processing IDE:
/File/Examples/Contributed Libraries/Minim/Analysis/FFT/SoundSpectrum
? but as a start try
/File/Examples/Contributed Libraries/Minim/Analysis/FFT/SoundEnergyBeatDetection
looks more easy???

ok i found the one you use:
/File/Examples/Contributed Libraries/Minim/Basics/AnalyseSound
and i print:
fft.specSize() is 513
so you could use
fft.getBand(x)
in

terrain [x] [y] = map(noise (xoff,yoff), 0, 1, -160, 120);

like

terrain [x] [y] = map(fft.getBand(x), 0, 100, -160, 120);  // ?? 0 .. 100 ?? 

but only if columns < 513 or you will get a error


#3

@celine1 – were you able to resolve this issue?

You may find this discussion from the old forum helpful: