Video not syncing to audio correctly when playing in Firefox

p5.js Coding Questions
1

Hi!

I’m trying to create a video player using p5 that a user can hit “start video” on and then can pan audio using a slider. I used the video tag in html to source the vids in multiple formats from my directory and then used document.getElementById in my js file to call them and play/stop them. However, I’m finding that in Firefox, the video and audio aren’t entirely syncing. Is there a way to specify video formats without using createVideo() to instantiate a video player?

2

Hi! I switched the category from processing (java) to p5.js (javascript).

do you want to elaborate it a bit more or ideally share the code? I don’t get whether you are using <video> tag and selecting them with getElementById or you are using createVideo to load the videos.

3

Hi!

Thanks! I used <video> and getElementById to load videos. I’m linking the Google drive folder to my directory here since some of the files are too big to upload to the p5 editor or to github.

4

I haven’t tested the code but how you do it may not sync video and audio. I would set the playback time based on one reference - say take currentTime from the video element and set audio elements’ currentTime to the same time. But this will still not perfectly sync the playback because of the timing difference of play() methods being triggered.

5

Got it. So a follow up question. If I just use the original video with all audio attached to it, how can I split the audio channels or streams so that I can pan one voice in the video left and the other right and adjust their volumes with the slider? Or do you suggest any better methods to pan the separate voices that are connected to the audio of the video?

6

you can create a web audio node from <audio> or <video> and then connect it to a panner node

But this is a bit complicated. I don’t know, alternatively, p5.Sound can take a <video> element as a source.

I suggest, if you are using static (prerecorded) video, simply separate the video using ffmpeg or similar tools in advance, which makes it so much easier.

By the way you seem to use old version of p5.js (0.7.2). The sound library in this version may lack some features. But beware, upgrading to the current version of p5.js may break something.