Draw line and play corresponding sound

Processing Libraries
1

Hi, sorry for the title, didn’t know what to put.
My question is mostly theoretical rather than actual code.
I made a simple canvas where a user can draw a line (as image below).
Now, assuming that vertical axis corresponds to frequencies in range (0-n) and horizontal axis is time (0-nseconds) I’d like to implement a player that plays that line of frequencies (sine oscillator is more than enough for now) in the correct time.

My issues:

  • I can add points in the canvas (and each point has a set frequency), but how do I calculate the exact frequency to play for each point the line is made of?
  • Should I first calculate all the frequencies (on the line) and store them in a buffer to be played back later, or just calculate everything in real time?
  • I understand the Lerp function generates values between 2 points at specific interval. Could it be useful in this situation? I think it could work since the lines are straight but I will have many points. Also I need to make sure the values generated by the Lerp are the same as those the line is made of (so to make sure the line is actually representing the sound that is played). In addition, the line might not be straight in the future (I might add possibility to bend it). In this case, the Lerp might not work?
  • If I calculate all these values and store them in a buffer (array I guess), how do I play them “smoothly”?
  • How can I add a vertical player line that moves left to right and should, obviously, synced with the sound position.

In general I’m asking about how to approach this, sorry for the broad question.
Thank you!

p1
p1.png1032×432 4.15 KB

1 Like
2

Hi, @plux

Interesting question :smirk_cat: … I tried to write it before, but it was more complicated than I thought :sweat_smile:, and finally I ended up doing this :point_down::

float n = 1000,ax=10,ay=10,bx = 100,by = 120, playery = 0,
      playerx =0, slope, currentFrecuency;

void setup(){
  size(120,140);
  frameRate(5);
  playery = ay;
  slope = (by-ay)/(bx-ax);  
}

void draw(){
  background(255);
  line(ax,ay,bx,by);
  line(playerx,0,playerx,height);
  playerx++;
  if(playerx >= ax && playerx <= bx){
    playery += slope;
    currentFrecuency = map(playery, 0,height,0,n);
 //Probably using the Lerp function can you avoid to use
 //the "playery += slope;" line; but I never used the Lerp
 //function yet, and forgot to take a look to it 
 //( had in mind the map function) 
  }
  
  if(playerx > width){
    playerx = 0;
    playery = ay;
  }
println("frecuency: " + currentFrecuency + "playery: " + playery);
}

Take a look, I think is very intuitive, don’t know if is what you want and for sure, to combine with your code, you have to touch some things, but maybe can give you and idea about how to do it :grimacing:.

Hope it helps, regards :hugs:

1 Like
3

Hi, thanks, that’s a good starting point.
I’ve added sound to test and it sounds good. There’s a bit of delay between sound and graphics (no idea if that’s fixable) but for now I don’t mind.

From here, the first thing I’d need to do is remove the dependency on frameRate.
Let’s say I decide I want to play the whole thing in 5 seconds. Visually it would take 5 seconds for the player line to cover the whole width (start at 0).

import processing.sound.*;

float n = 1000,ax=10,ay=10,bx = 100,by = 120, playery = 0,
      playerx =0, slope, currentFrecuency;

Oscillator s =new SinOsc(this);

void setup(){
  size(120,140);
  //frameRate(5);
  playery = ay;
  slope = (by-ay)/(bx-ax);  
}

void draw(){
  background(255);
  line(ax,ay,bx,by);
  line(playerx,0,playerx,height);
  playerx++;
  if(playerx >= ax && playerx <= bx){
    playery += slope;
    currentFrecuency = map(playery, 0,height,0,n);
 //Probably using the Lerp function can you avoid to use
 //the "playery += slope;" line; but I never used the Lerp
 //function yet, and forgot to take a look to it 
 //( had in mind the map function) 
  }
  
  if(playerx > bx){
    playerx = 0;
    playery = ay;
  }
  
  s.freq(currentFrecuency);
  s.amp(0.5);
  s.play();
println("frecuency: " + currentFrecuency + "playery: " + playery);
}
1 Like
4

Here it is: :partying_face: (a little bit later that I wanted)

import processing.sound.*;

float n = 1000,ax=10,ay=10,bx = 100,by = 120, playery = 0,
      playerx =0, slope, currentFrecuency;
      
double xPerSecond,seconds,totalTime = 5;

Oscillator s =new SinOsc(this);

void setup(){
  size(120,140);
  playery = ay;
  slope = (by-ay)/(bx-ax);  
}

void draw(){
  if(seconds >= totalTime)
    seconds = 0;
  seconds += 1/frameRate;
  xPerSecond = width / totalTime;
  background(255);
  line(ax,ay,bx,by);
  line(playerx,0,playerx,height);
  playerx = (float)(seconds*xPerSecond);
  if(playerx >= ax && playerx <= bx){
    playery += slope;
    currentFrecuency = map(playery, 0,height,0,n);
  }else
    currentFrecuency = 0;
  
  if(playerx >= width){
    playerx = 0;
    playery = ay;
    println(" frecuency: " + currentFrecuency + "playery: " + playery + " Seconds " + seconds);
  }
  
  s.freq(currentFrecuency);
  s.amp(0.5);
  s.play();
//println(" frecuency: " + currentFrecuency + "playery: " + playery + " Seconds " + seconds);
}

seconds += 1/frameRate to make every second the increment to be constant.
xPerSecond = width/totalTime to assign the X increment every second.
And playerx = (float)(seconds*xPerSecond) to travel the total width.

What’s next? :smiley:

3 Likes