Polysynth notes stop

[b_and_z]

Hello all
I am creating a tool to translate image to sound, specifically get brightness of certain pixels and use these as a trigger for a polyphonic synth. As long as a certain pixel has a certain brightness, the synth should be playing. Here is a simplified version:
https://editor.p5js.org/RBvonRB/sketches/_4yp__kwd

My problem is that now seemingly randomly, when the circle is moved, certain notes that should be playing stop, or the new notes play for a second and then stop, no decay. Try playing around and hopefully you see what i mean.

Anyone have any idea? The max voices is set to 12, so that shouldn’t be the problem. Also, same problem when the sketch runs locally instead of the editor.

[tony]

First off, your synths sound great.

I was able to recreate the bug by waving the cursor around a ton, and I think it comes down to these:

• loadPixels()
• this.playing

I’m not familiar with the sound library, but I think I can say, with some confidence, that calling loadPixels() is not necessarily a cheap operation.

I did some profiling (thanks Chrome) and it looks like the function eating up the most CPU is readPixels(), called by get() AND loadPixels():

But what does any of this have to do with my awesome poly synths not playing?
My guess is that there’s some sort of default behavior in the case that loadPixels() can’t finish in time to spit out the next frame. Perhaps the pixels array defaults to black? Or perhaps brightness() is receiving bad values and defaults to 0.

I don’t know where, but the one fact you do know is that somewhere, between loadPixels, get(), and brightness(), somethings going wrong, and the high cost of calling loadPixels() every frame isn’t helping.

But grabbing pixel values is my whole deal
Don’t worry! Im sure there’s a solution. But to start tackling this problem, it might help to look into faster ways of accessing the data you want. Dan Schiffman gives a good introduction.

TL;DR, behavior might be wonky bc loadPixels() is expensive.

[b_and_z]

Thank you tony for this thorough answer! I was hoping to avoid the pixel array, but as it is more efficient, I will give it a try and report back.