Hey so I am kinda new to this,
But I am making an API using Fastify & NodeJS and want to add an overlay to a video, I have looked at multiple things and found that p5js would be the best for me, but I am failing to set it up. Could someone tell me how to set it up since there is no browser for me to do things on since its just an API, you request data and get data in return.
Thank you in advance!
FYI: I tried https://www.npmjs.com/package/node-p5 too and its just super outdated…
Really interesting, I never heard of p5 on the server side, as it was originally meant for client side graphics.
Judging by the NPM page, it sounds like you could, say, generate custom pngs in response to a quart but I don’t know how applicable that might be to a video overlay, in that the video itself would be streamed from the server.
The way most people make an overlay with p5 is to pipe it in as a stream, then, client side, draw over it, within the canvas
What exactly are you trying to do?
Thank you for your reply!
Maybe I should’ve explained it a bit better xD
But yea I want p5 on server side, I am creating an API that changes video to gif and more, but I would like the users to be able to add an “overlay” to their video, some pre-made and some custom. It will also include like animations if you have multiple vids etc. Thats why I found p5js to be the best for this case.
For the video stuff itself I found:
And for exporting it I would just basically get a picture of every frame.
So in the end I would need to get pictures of each frame of the video with animations/overlay.
I hope this explains it a bit better! 
Ah okay interesting. That old library seems like the place to start. Im not super familiar with Nodejs, especially what goes on under the hood, so I can’t give you great advice.
My initial thought was, create it client side, get the exporting working, then port over to node and cross your fingers haha.
So if I made my own node-p5 now, but with saving the canvas I get this error:
TypeError: get_URL(...).createObjectURL is not a function
at new FileSaver (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:19390:44)
at Object.saveAs (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:19406:26)
at Object._main.default.downloadFile (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:70088:34)
at https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:65134:39
at D:\CodeEnSHiz\project\API\node_modules\jsdom\lib\jsdom\living\nodes\HTMLCanvasElement-impl.js:80:9
Is this a p5 issue? Or is this an issue I can fix?
createObjecrURL() is a client-side method, part of the web-api.
In the past, I’ve used it to immediately display a users new profile pic. Instead of waiting for the upload, processing, and download, I used createObjectURL() to create a local ref that an href could understand.
That seems to be what your code is trying to do. Maybe see if you can tweak a line somewhere to instead use the node-servers local filesystem?
If you can contact me on discord (Bas950#0950) I can show you what I am doing,
right now all I have is setting up JSDOM
const dom = new JSDOM(
`
<head>
<script src="https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js"></script>
<script>${readFileSync("sketch.js")}</script>
</head>
<body>
<div>foo</div>
</body>`,
{
runScripts: "dangerously",
resources: "usable",
pretendToBeVisual: true
}
);
and then sketch.js:
console.log("abc");
function setup() {
let c = createCanvas(100, 100);
console.log("canvas created");
console.log(c);
background(255, 0, 0);
console.log("background set");
saveCanvas(c, "myCanvas", "png");
console.log("saved");
}
everything is working correctly only saving is just not working…
Hm, one last thought; maybe you can try writing your own save() function?
If the canvas holds a canvas object, perhaps you can save it directly, like in this tutorial:
Well I would need to import fs into the sketch file then…
but yea that doesnt work I keep getting errors on that and get linked to:
FYI: Got it working 
Nice! What was the fix?
Loading in JSDOM:
const virtualConsole = new VirtualConsole(),
dom = new JSDOM(
`
<head>
<script src="https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js"></script>
<script>${readFileSync("util/sketch.js")};</script>
</head>
<body>
<div>foo</div>
</body>`,
{
virtualConsole,
runScripts: "dangerously",
resources: "usable",
pretendToBeVisual: true
}
);
virtualConsole.sendTo(console);
Then just do the p5js stuff in the sketch file
And to send the data back I send back each frame as png
and sending it back via JSDOM virtual console.
console.info(c.canvas.toDataURL("image/png"))
@tony the only thing is it wont let me load videos…
So I think I will need to get each frame and then edit each frame…
But it takes like 2 seconds to load each frame so do you maybe know a fix for this:
Error: Not implemented: HTMLMediaElement.prototype.load
at module.exports (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\browser\not-implemented.js:9:17)
at HTMLVideoElementImpl.load (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\nodes\HTMLMediaElement-impl.js:110:5)
at HTMLVideoElement.load (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\generated\HTMLMediaElement.js:111:34)
at _main.default.MediaElement.play (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:60539:24)
at setup (about:blank:13:11)
at _setup (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:48784:27)
at p5._runIfPreloadsAreDone (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:48722:27)
at p5._start (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:48707:25)
at new p5 (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:49057:22)
at _globalInit (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:48197:17)
at processTicksAndRejections (internal/process/task_queues.js:93:5) undefined
Error: Not implemented: HTMLMediaElement.prototype.play
at module.exports (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\browser\not-implemented.js:9:17)
at HTMLVideoElementImpl.play (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\nodes\HTMLMediaElement-impl.js:116:5)
at HTMLVideoElement.play (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\generated\HTMLMediaElement.js:145:60)
at _main.default.MediaElement.play (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:60540:34)
at setup (about:blank:13:11)
at _setup (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:48784:27)
at p5._runIfPreloadsAreDone (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:48722:27)
at p5._start (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:48707:25)
at new p5 (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:49057:22)
at _globalInit (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:48197:17)
at processTicksAndRejections (internal/process/task_queues.js:93:5) undefined
Error: Uncaught [TypeError: Cannot read property 'functionName' of undefined]
at reportException (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\helpers\runtime-script-errors.js:62:24)
at innerInvokeEventListeners (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\events\EventTarget-impl.js:333:9)
at invokeEventListeners (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\events\EventTarget-impl.js:274:3)
at EventTargetImpl._dispatch (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\events\EventTarget-impl.js:221:9)
at reportAnError (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\helpers\runtime-script-errors.js:31:12)
at reportException (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\helpers\runtime-script-errors.js:58:19)
at runAnimationFrameCallbacks (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\browser\Window.js:462:13)
at Timeout.<anonymous> (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\browser\Window.js:438:11)
at listOnTimeout (internal/timers.js:551:17)
at processTimers (internal/timers.js:494:7) TypeError: Cannot read property 'functionName' of undefined
at processStack (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:46279:72)
at fesErrorMonitor (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:46395:35)
at callTheUserObjectsOperation (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\generated\EventListener.js:26:30)
at innerInvokeEventListeners (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\events\EventTarget-impl.js:318:25)
at invokeEventListeners (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\events\EventTarget-impl.js:274:3)
at EventTargetImpl._dispatch (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\events\EventTarget-impl.js:221:9)
at reportAnError (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\helpers\runtime-script-errors.js:31:12)
at reportException (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\helpers\runtime-script-errors.js:58:19)
at runAnimationFrameCallbacks (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\browser\Window.js:462:13)
at Timeout.<anonymous> (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\browser\Window.js:438:11)
Error: Uncaught [TypeError: The first argument must be an object]
at reportException (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\helpers\runtime-script-errors.js:62:24)
at runAnimationFrameCallbacks (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\browser\Window.js:462:13)
at Timeout.<anonymous> (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\browser\Window.js:438:11)
at listOnTimeout (internal/timers.js:551:17)
at processTimers (internal/timers.js:494:7) TypeError: The first argument must be an object
at CanvasRenderingContext2D.ctx.<computed> [as drawImage] (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\living\nodes\HTMLCanvasElement-impl.js:124:17)
at _main.default.Renderer2D.image (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:51051:35)
at p5._main.default.image (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:66018:28)
at draw (about:blank:23:5)
at p5._main.default.redraw (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:56238:19)
at _draw (https://cdn.jsdelivr.net/npm/p5@1.1.9/lib/p5.js:48842:25)
at runAnimationFrameCallbacks (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\browser\Window.js:460:13)
at Timeout.<anonymous> (D:\CodeEnSHiz\HiddenProjectName\API\node_modules\jsdom\lib\jsdom\browser\Window.js:438:11)
at listOnTimeout (internal/timers.js:551:17)
at processTimers (internal/timers.js:494:7)
Code used:
let c
let outputFrames = [];
let video;
function preload() {
video = createVideo(bla);
video.hide();
}
function setup() {
c = createCanvas(576, 1024);
frameRate(30);
video.play();
video.volume(0);
video.onended(() => {
noLoop();
console.log("done");
sendData();
});
}
function draw() {
background(220);
image(video, 0, 0);
textSize(32);
text(video.time(), 10, 30);
outputFrames.push(c.canvas.toDataURL("image/png"));
}
NOTE: the inputted video is base64 data.
using JSDOM basically is mocking up the whole HTML/CSS rendering stack. In fact all we really need is the canvas for p5js to work (unless you’re using a webGL renderer of course). Have you looked at node-canvas? that would be much more lightweight I believe.
But I’m not sure how things like the render loop are used eg draw() getting called in the JS runtime after P5 loads it. It’s not just a canvas API it’s a bunch of engine calls. Similarly how P5 is often polluting the global JS namespace with it’s commands. Perhaps that’s what the node-p5 package is handling?
I’m just looking for a simple example to render a P5 canvas to a file… 
I could post my new solution to NPM or something like that if you want,
its basically just a class and you can .exec p5 commands etc.