Edit, Update:
I can now reproduce the error 100% of the time across all browsers in any context. By much trial and error, I discovered that by rapidly clicking the right arrow key on the first instruction screen (where the player’s character is assigned to them) following the surveys and welcome screen, I can make the bug appear 100% of the time. My guess is that the rapid key presses cause the event listener to be fired multiple times before it should be, resulting in some function arguments being undefined for some reason. Not sure how to proceed. Would putting a delay in using sleep() after the arrow key press help? Does this have something to do with how P5 records key presses?
TL; DR
Wrote a video game in p5.js and am hosting it as a Google Firebase web app. I use a custom event listener to loop through the video game levels. About 30% - 40% of users report a bug where the game cycles rapidly through all levels without user input. I can only reproduce the error about 10% of the time, and only in specific contexts. Usually no error messages appear in the console or via an error reporting service when the bug occurs.
Longer Explanation
I am running a psychological study online using a service for recruiting subjects called Prolific. The study includes several surveys followed by a video game task. I wrote the video game using p5.js. There are several different task conditions for the study, and they are programmed as different p5 sketches. To loop through the proper sequence of trials, I use a custom written event listener that fires when the appropriate conditions for ending the current trial have been satisfied, removing the current sketch and starting the next one. I am hosting the surveys and video game as a web app via Google Firebase. The way the study works, users on Prolific who want to participate are directed to the web app URL (I can provide this URL if you message me privately).
About 30% - 40% of my study subjects are reporting an unusual bug that I have difficulty reproducing. The game will run smoothly for a few trials, and then suddenly it will crash, rapidly cycling through all of the remaining trials without any user input until it reaches the end of the study. I have attempted to reproduce the error on all major browsers (Firefox, Chrome, Safari) on all major operating systems (Windows, Mac OS, Linux) on several different machines. Across all cases, I am only able to reproduce the error about 10% of the time, and even then only in specific circumstances (see below).
The vast majority of the time (on both my end and the user’s end), no unhandled exceptions (error messages) show up in the console or get reported by my error monitoring service during the crash (Sentry.io). Whenever errors do show up (less than 10% of the runs on which the app crashes), they always take the form of TypeErrors, with two errors being the most common:
-
NetworkError when attempting to fetch resource / Failed to fetch resource. I am not sure why this error appears, and it appears during both successful and crashed runs of the app, which leads me to believe that this is not the culprit. The only resources being loaded are images in thepublicdirectory of the web app. However, sometimes on crashed runs, at the time of the crash (I am unable to determine whether it is before, during, or after the crash point), sometimes an image that was supposed to load does not show up (but no errors reported by P5). -
As shown in the example code below, a function argument
ais used to identify which sketch to draw to the canvas for the current trial. I sometimes get the errora is undefined. This one does not make sense to me since according to the Javascript scope chain, the variableparam_seq[i]that I pass as argumentato the functiondefineSketch(a)should have a global scope, and is definitely defined. My guess is that this is the culprit, but the only thing I can think of is that, occasionally, some weird asynchronous evaluation of the code or some other timing issue happens that causes the argument todefineSketchto be undefined, leading to the crash. No idea how to verify or debug this though.
Potentially Relevant Details
I can only reproduce the bug in the following situations:
- Chromium 93 browser (no other versions tried, crashes on less than 50% of runs)
- Chrome 92 browser (no other versions tried, crashes on less than 50% of runs)
- Safari 13.1.3 browser when launching the app from Prolific (no other versions tried, bug does not show up when visiting the app URL directly, crashes on less than 50% of runs)
- Firefox 92 when launching the app from Prolific (older versions of Firefox do not reproduce the bug, bug does not show up when visiting the app URL directly, crashes on less than 50% of runs)
It is potentially significant that in Safari and Firefox, the bug does not show up unless the app is launched from Prolific. Prolific attaches a query string to the end of the URL that enables subject identification so that they may be paid for their time. Not sure why this would be an issue, but I could see a weird scenario where the query string somehow messes with communication with my database.
Additionally, I use $ as the name of a property. One of the Javascript libraries I use to gather survey data within the web app employs jQuery, and I recently realized that $ is a shortcut for jQuery, so this could potentially cause an issue.
Minimal Working Example
The video game is complicated and involves a lot of code, so it is difficult to write up a minimal working example. The example code below contains the logic of the custom event listener used to loop through the different trials as well as the functions that generates the p5 sketches, each with a different form of user interaction or condition for the end of the trial. Though there are no surveys in the MWE, I include the Javascript libraries needed to program them in case that matters.
<!DOCTYPE html>
<html>
<head>
<!-- Load in survey format scripts -->
<script src="https://unpkg.com/jquery"></script>
<script src="https://unpkg.com/survey-jquery@1.8.34/survey.jquery.min.js"></script>
<link href="https://unpkg.com/survey-knockout@1.8.34/modern.css" type="text/css" rel="stylesheet"/>
<!-- Load in p5.js -->
<script src="https://cdn.jsdelivr.net/npm/p5@1.2.0/lib/p5.js"></script>
</head>
<body></body>
<script>
// listened variable
// we set a listner which will be fired each time the value of bool is changed.
let sketchIsRunning = {
$: false,
listener: function(val) {},
set bool(val) {
this.$ = val;
this.listener(val);
},
get bool() {
return this.$;
},
registerListener: function(listener) {
this.listener = listener;
}
};
/* Function to define sketch with parameter 'a' */
function defineSketch(a) {
switch(a) {
case 0:
return function(p) {
let x = 100;
let y = 100;
p.setup = function() {
p.createCanvas(700, 410);
};
p.draw = function() {
p.background(0);
p.fill(255);
p.rect(x, y, 50, 50);
};
/* Remove sketch on mouse press */
p.mousePressed = function() {
p.remove();
sketchIsRunning.bool = !sketchIsRunning.bool
console.log('sketch is running ?', sketchIsRunning.bool)
};
};
break;
case 1:
return function(p) {
let x = 200;
let y = 200;
p.setup = function() {
p.createCanvas(700, 410);
// Length of time to show sketch
p.sketch_length = p.random(1e3, 5e3);
// Start time of sketch
p.sketch_start = window.performance.now();
};
p.draw = function() {
p.background(0);
p.fill(255);
p.ellipse(x, y, 50, 50);
p.rect(y, x, 50, 50);
/* Remove sketch after random time */
if (window.performance.now() - p.sketch_start >= p.sketch_length) {
p.remove();
sketchIsRunning.bool = !sketchIsRunning.bool
console.log('sketch is running ?', sketchIsRunning.bool)
}
};
};
break;
case 2:
return function(p) {
let x = 0;
let y = 0;
p.setup = function() {
p.createCanvas(700, 410);
};
p.draw = function() {
p.background(0);
p.fill(255);
p.ellipse(x, y, 50, 50);
};
/* Remove sketch on key press */
p.keyPressed = function() {
p.remove();
sketchIsRunning.bool = !sketchIsRunning.bool
console.log('sketch is running ?', sketchIsRunning.bool)
};
};
break;
default:
console.log("No match")
break;
}
}
/* Initialize sketch variable */
let trialSketch;
/* Array of parameters */
let param_seq = [0, 1, 2];
// we nest a call to the function itself to loop through the param.seq array
const instanceP5sketches = (i = 0) => {
sketchIsRunning.$ = !sketchIsRunning.$;
trialSketch = defineSketch(param_seq[i]);
new p5(trialSketch);
sketchIsRunning.registerListener(function(val) {
if (param_seq.length - 1 >= i){
instanceP5sketches(i+1);
} else {
console.log('No more sketches.')
}
});
}
// Begin looping through the sketches
instanceP5sketches();
</script>
</html>
Any help with this is greatly appreciated, I’m at my wits end trying to work through it.