Thank you very much for your help!
Yes, admittedly I need to pin point the problem better than this. However, I really don’t know what is the problem! 
Regarding the tensorflow.js, Daniel did this in this example and I followed his guidelines. I think my game mechanics is causing the problem.