Code is not executed in Processing, but in p5.js Web Editor

p5.js Beginners
1

Hello everyone, as the title suggests I am having a problem running my javascript code in Processing. However, if I copy this code into the p5.js web editor (https://editor.p5js.org/) the code works without problems. I use the p5.js extension. Maybe someone knows the problem and has a solution how I can fix this and run this code in processing. Thanks a lot!!!

var deviceList = [];

function preload() {
  navigator.mediaDevices.enumerateDevices().then(getDevices);
}

function setup() {
  var constraints = {
  video: {
      deviceId: {
        exact: deviceList[1].id
      },
    }
  };

canvas = createCanvas(width, height);
background(255);
video = createCapture(constraints);
//console.log(deviceList);
}

function getDevices(devices) {

  //arrayCopy(devices, deviceList);
  for (let i = 0; i < devices.length; ++i) {
    let deviceInfo = devices[i];
    
      //Only get videodevices and push them into deviceList
      if (deviceInfo.kind == 'videoinput') {
        deviceList.push({
        label: deviceInfo.label,
        id: deviceInfo.deviceId
      });
//      console.log("Device name :", devices[i].label);
//      console.log("DeviceID :", devices[i].deviceId);
    }
  }
}

Edit: When I check the developer tools, i get the following error:

Received_Error
Received_Error834×194 7.83 KB

2

There’s no guarantee that the array deviceList[] will be fully filled when the code reaches this line exact: deviceList[1].id within setup()!

3

The array is not filled in getDevices(); therefore we can’t expect it to be filled in setup(); Furthermore, the following console.log is with two cameras connected. navigator.mediaDevices.enumerateDevices() found only one camera and the DeviceInfo fields are not filled in. This is with Processing IDE in mode p5.js and a Safari browser on a Mac. It does not work correctly in the p5.js web editor with a Chrome browser either with an initial setting of deviceList[1]. As soon as I switch it to deviceList[0] it works. I can then switch to deviceList[1] and get the second camera (after starting with zero). The deviceList array is filled correctly in getDevices() in Chrome.

getDevices
getDevices1392×106 27 KB

Try running the following code in Processing IDE (I used Safari browser) then try the same code in p5.js web editor preferably with Chrome. Check the console output of each and note the differences. This code came from: https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/enumerateDevices

function setup() {
if (!navigator.mediaDevices || !navigator.mediaDevices.enumerateDevices) {
  console.log("enumerateDevices() not supported.");
  return;
}

// List cameras and microphones.
navigator.mediaDevices.enumerateDevices()
.then(function(devices) {
  devices.forEach(function(device) {
    console.log(device.kind + ": " + device.label + " id = " + device.deviceId);
  });
})
.catch(function(err) {
  console.log(err.name + ": " + err.message);
});
}

function draw() {
}
4

Thank you for the answer. Yes I thought so too, that is why I packed the function into the preload(). Unfortunately that didn’t change anything either. Is there a way to execute the exact: deviceList[1].id only once it exists?

5

Thanks for the reply. Yes, I should have written that. I have 2 webcams on my PC and would like to read the ID of both, or then use them in the connection, therefore deviceList[1]…

I also tried the code with my Chrome browser and get the following output:

response_Chrome

However, if I paste the code back into the p5.js online web editor I get the following as a response:

videoinput: HD Pro Webcam C920 (046d:082d) id = 3ff1dc25d725025fc915c7064c6eaee3a4dfb92658f8b0064770f6c9204872c6
videoinput: HD Webcam (04f2:b71a) id = c19baf8272627d653c907eda9908783b32d4f5b58c90f3887098fdfeab477551

So it is working there.

I don’t think it’s because the deviceList[0] is not loading fast enough. If I remove the comments in the last two lines and display the current device name and ID, it works…

(I mean the following lines of the original code)

//      console.log("Device name :", devices[i].label);
//      console.log("DeviceID :", devices[i].deviceId);
6

Callback preload() only works for p5js asset loading functions such as loadImage(), loadJSON(), loadFont(), etc.

Also for addon libraries which internally register their own loading functions into the p5js such as loadSound() from the p5.sound library.

Anything else preload() won’t await for them to get fully finished!

Yes, many ways! But IMO the easiest approach would be to postpone the automatically initialization of the p5js library until your callback getDevices() has finished its job.

When p5js library is loaded it searches for callbacks setup() & draw() in the browser’s global context.

So the trick to cancel that is to hide both setup() & draw() until all your unregistered loadings have been finished.

In order to hide both setup() & draw() we’ll have to slightly modify their function declaration so they won’t automatically go into the global context:

Instead of function setup() {} go w/ const setup = function () {};.

Same for function draw() {}. Just replace it w/ const draw = function () {};.

Let’s now create a function which will manually start the p5js library:

function startP5() {
  globalThis.setup = setup; // place callback setup() into global context
  globalThis.draw  = draw;  // place callback draw()  into global context

  new p5; // manually instantiate the p5js library
}

Invoke startP5() at the end of getDevices() callback after its devices[] loop:

And of course move your navigator loading stuff out of callback preload(), b/c it won’t be automatically called back until p5js is manually started. :wink:

P.S.: You won’t be able to access anything from the p5js API until after you invoke startP5(). :warning:

7

It works! Thanks a lot! That was definitely a huge help to me. I thought I post here again the completed code. Maybe it is useful for someone else :slight_smile:

var deviceList = [];

navigator.mediaDevices.enumerateDevices().then(getDevices);

const setup = function () {
    
    var constraints = {
        video: {
            deviceId: {
                exact: deviceList[0].id
      },
    }
  };
    var constraints1 = {
        video: {
            deviceId: {
                exact: deviceList[1].id
      },
    }
  };
canvas = createCanvas(width, height);
background(255);
video = createCapture(constraints);
    video2 = createCapture(constraints1);
console.log(deviceList);
};

const draw = function () {
    
};


function startP5() {
    globalThis.setup = setup; // place callback setup() into global context
    globalThis.draw  = draw;
    new p5;
}


function getDevices(devices) {

  //arrayCopy(devices, deviceList);
  for (let i = 0; i < devices.length; ++i) {
    let deviceInfo = devices[i];
    
      //Only get videodevices and push them into deviceList
      if (deviceInfo.kind == 'videoinput') {
        deviceList.push({
        label: deviceInfo.label,
        id: deviceInfo.deviceId
      });
//      console.log("Device name :", devices[i].label);
//      console.log("DeviceID :", devices[i].deviceId);
    }
  }
    startP5();
}
8

Would you mind posting code that will run in Processing IDE? I tried the code that you posted in SO, but it will not run in Safari on my system. I see the error msg below. Also there was a suggestion that new p5 have parentheses, new p5(). Thanks.

err

9

For me the code works only in Firefox. In Chrome I get the same error. If you have only one camera you have to change deviceList[1].id to deviceList[0].id as well.

comment: But I have already been able to solve the problem using GoToLoops procedure. The answer I have also already posted, but must be released by a forum admin for the thread.

10

I just tried the SO code in p5.js web editor with Chrome and it runs without error. I have two cameras connected to my Mac and it flicked both of them on at startup. Right now my conclusion is that it won’t run in Safari. Other possibility is Processing IDE, but supposedly it turns everything over to the browser when running p5.js.

11

If you need to pick the last device found you can do this instead:
exact: deviceList[deviceList.length - 1].id

If you’ve got a more updated browser you can shorten it using method at():
exact: deviceList.at(-1).id

Another option is picking a random device:
exact: random(deviceList).id

12

Please don’t take this personally, but I respectfully disagree with your assessment that preload() should not be used for this problem. My assessment is that it is more a function of which browser plays nice with navigator.mediaDevices.enumerateDevices() and which ones do not. As long as the poster is happy that’s really all that counts for this thread. But for the sake of finding the truth, I think there is more to it than has been stated here. Please consider that the following code also runs without error in a Chrome browser running p5.js web editor. Both cameras on my system (Mac) are turned on simultaneously. Note that the demo uses preload() and does not try to hide setup() and draw().

var deviceList = [];

function preload(){  
  navigator.mediaDevices.enumerateDevices().then(getDevices);
}

function setup() {
 var constraints = {
  video: {
  deviceId: {
    exact: deviceList[0].id
    },
   }
 };
 var constraints1 = {
  video: {
  deviceId: {
   exact: deviceList[1].id
  },
  }
 };
 canvas = createCanvas(width, height);
 background(255);
 video = createCapture(constraints);
 video2 = createCapture(constraints1);
 console.log(deviceList);
}

function draw () {
}

function getDevices(devices) {
  for (let i = 0; i < devices.length; ++i) {
    let deviceInfo = devices[i];
    //Only get videodevices and push them into deviceList
    if (deviceInfo.kind == 'videoinput') {
      deviceList.push( {
      label: deviceInfo.label,
        id: deviceInfo.deviceId
      }
      );
    }
  }
}

Things aren’t what they seem.

I have repeatedly tested the above code by opening and closing the browser and p5.js web editor multiple times in succession. I concede that sometimes preload() fails. Sometimes it works ok, sometimes it fails to load the array even though it gets to the end of the getDevices() function before going to setup(). I have not tested the other technique as rigorously which I intend to do.

13

You don’t have to believe me! Just check w/ your own eyes what are the functions registered to work w/ preload() callback by default straight from p5js’ source code: :eyes:

There are only 9 p5js loading functions which cause preload() to await their full completion by default.

Anything else gotta precall registerPreloadMethod() so preload() knows how to await for them as well.

Just placing some unregistered stuff inside callback preload() won’t magically make it work!

The p5.js web editor got lotsa other stuff running behind-the-scene.

It’s more or less expected for the p5js library and its callbacks to run much later on such environments.

In order for the sketch to work the callback getDevices() must run before p5js setup() is called back.

Otherwise the array deviceList[] will still be empty when the code reaches setup().

Leaving the sketch on a state which depends on how fast a callback is invoked before the other is unacceptable IMO. :no_good_man:

Such code is clearly buggy! :bug:

14

Well, since you put it that way, you win!

If it were only a matter of waiting long enough for the deviceList array to fill, then what’s your explanation for why the code using your method won’t run in Processing IDE (4.02b) with a Safari browser (Mac)? Error message below is for two connected cameras:

Safari error
Safari error1212×142 16 KB

You may not have the hardware to test this yourself, but this is why I think there’s more to it than meets the eye. If you are unable to test it, I would appreciate someone else testing it for confirmation. Thanks.

preload() should be renamed preloadForCertainThings() IMO.

Processing IDE with Safari Browser Code to Test (Mac) - Two Cameras

var deviceList = [];

navigator.mediaDevices.enumerateDevices().then(getDevices);

const setup = function () {
 var constraints = {
  video: {
  deviceId: {
    exact: deviceList[0].id
    },
   }
 };
 var constraints1 = {
  video: {
  deviceId: {
   exact: deviceList[1].id
  },
  }
 };
 canvas = createCanvas(width, height);
 background(255);
 video = createCapture(constraints);
 video2 = createCapture(constraints1);
 console.log(deviceList);
};

const draw = function () {
};

function startP5() {
  globalThis.setup = setup; // place callback setup() into global context
  globalThis.draw  = draw;
  new p5();
}

function getDevices(devices) {
  for (let i = 0; i < devices.length; ++i) {
    let deviceInfo = devices[i];
    //Only get videodevices and push them into deviceList
    if (deviceInfo.kind == 'videoinput') {
      deviceList.push( {
      label: deviceInfo.label,
        id: deviceInfo.deviceId
      }
      );
         console.log("Device name :", devices[i].label);
         console.log("DeviceID :", devices[i].deviceId);
    }
  }
  startP5();
}
15

Indeed I don’t. Neither P4 (using P3) or a Mac or Safari.

Actually I don’t use Processing’s IDE p5js mode at all.

I prefer to make my own “index.html” & “sketch.js” files.

BtW, I’ve modified your code and I’m posting it below.

However I can’t test it here. My desktop Win 8.1 PC w/ SlimJet browser only have audio devices listed.

index.html:

<script async src=https://cdn.JsDelivr.net/npm/p5></script>
<script defer src=sketch.js></script>

sketch.js:

/**
 * createCapture() for Found Devices (v1.0.1)
 * modded by GoToLoop (2022-Jan-28)
 *
 * https://Discourse.Processing.org/t/
 * code-is-not-executed-in-processing-but-in-p5-js-web-editor/34867/15
 */

'use strict';

const videoDevicesFound = [], captures = [];

addEventListener('load', getAvailableVideoDevices); // delay device search

function getAvailableVideoDevices() {
  navigator.mediaDevices
    .enumerateDevices()
    .then(collectAvailableVideoDevices)
    .catch(console.error);
}

function collectAvailableVideoDevices(devices, type = 'videoinput') {
  console.table(devices);

  for (const device of devices)  if (device.kind == type)
    videoDevicesFound.push(device); // collect video devices only

  startP5();
}

function createCaptures() {
  for (const { deviceId, groupId } of videoDevicesFound) {
    const constraints = {
      video: {}
    };

    if (deviceId)  constraints.video.deviceId = {
      exact: deviceId
    };

    if (groupId)  constraints.video.groupId = {
      exact: groupId
    };

    captures.push(createCapture(constraints));
  }
}

function startP5() {
  globalThis.setup = setup; // place callback setup() into global context
  globalThis.draw  = draw;  // place callback draw()  into global context

  new p5; // manually instantiate the p5js library
}

const setup = function () { // setup() callback is hidden from p5js for now
  createCanvas(1000, 700);
  createCaptures();
};

const draw = function () { // draw() callback is hidden from p5js for now
  frameCount % 60 || background('#' + hex(~~random(0x1000), 3));
};
16

Interesting demo. The good news is that it does run in Processing IDE with Safari browser on a Mac, which is pretty good since you had no way to test it. The bad news is that you used another method of finding the devices and only opened one of my two cameras, unlike the original demo. The code also placed a multi-color background, flashing canvas on top of the video feed. You could set the canvas to size 0,0 and forget the background changes and it would improve the demo in my opinion. Since you used a new technique of finding the devices and the code ran in Processing IDE I will use this as further evidence that there is a problem using navigator.mediaDevices.enumerateDevices() in Safari until someone proves otherwise.

There were many warnings and one error which you might be interested in knowing.

Use the function form of “use strict”.
‘console’ is not defined.
Expected ‘{’ and instead saw ‘if’.
Expected ‘{’ and instead saw ‘videoDevicesFound’.
Expected ‘{’ and instead saw ‘constraints’.
‘createCapture’ is not defined.
‘globalThis’ is not defined.
‘p5’ is not defined.
‘createCanvas’ is not defined.

error
error952×64 8.52 KB

In spite of all these warnings it still ran.

17

B/c draw() had nothing on it I’ve added background() just so I knew the sketch was running in my PC.

Of course you can delete it, comment that out or replace it w/ something else.

You should also try out other browsers and/or environments such as My Sketch - OpenProcessing.

“p5.js mode” isn’t always up-to-date w/ latest p5js library version (currently at v1.4.0).

It’s pretty much the same approach as yours.

I’ve only added property groupId in addition to deviceId when creating the constraints{} object for createCapture() within createCaptures().

18

I really don’t get why your original demo would get your both cameras and mine couldn’t. :confused:

The most notable difference is that b/c you have enumerateDevices() inside the preload() callback, the whole thing starts much later in comparison to my demo, which invokes it ASAP. :man_running:

Maybe if I change my demo to also call enumerateDevices() much later, after the whole page and its resources have been fully loaded, I wonder if I could get the same success result as yours on the Safari browser? :thinking:

So instead of straight invoking my getAvailableVideoDevices(), I’d register it as a load event callback:
addEventListener('load', getAvailableVideoDevices);

19

I really don’t get why your original demo would get your both cameras and mine couldn’t.

I think your method is only getting one video input and the field strings are empty. Just like navigator.mediaDevices.enumerateDevices() except your version goes ahead and opens the camera, while the original technique just turned on the camera’s little green light but did not capture the image. Here’s the array from function collectAvailableVideoDevices(): I used console.log() instead of console.table()

v2_err
v2_err1382×122 27.7 KB

20

W/ both properties deviceId & groupId having an empty string there’s no way to pick a specific camera using that approach.

My code will end up getting the default camera instead.

Besides empty strings you get a “Failed to load resource” error, perhaps outta enumerateDevices().

Seems like your Safari browser is refusing to reveal your available devices’ detailed info.