I was checking how to use Processing with Python support. However, I read that in order to use Python libraries I would have to place then in the folder of my Processing project. Moreover, I also read that I would not be able to use libraries with C extensions such as numpy or pytorch.
That’s exactly why I would like to use processing. I would like to define some environments/simulations and then after training, visualize then using Processing.
Is there any way to do both together? Use Pytorch and Processing at the same time?
p5 (or p5py) is a native Python port of the Processing API. It’s a Python library that doesn’t utilize Processing at all. You can import and run any Python library with p5py.
Each solution has its own set of advantages and disadvantages.
I see that PyTorch introduced Java bindings, so there’s probably some way you can run it using Processing.py (importing it as a Java library that Jython can leverage).
I like Python, it’s my favourite language at the moment. I do also like Processing IDE (PDE) and it’s unfortunate that python mode in PDE is tied to Jython and that prevents using important data science libraries. I find though Jupyter Notebook to be better environment to test and play with data science stuff , including deep learning.
I haven’t yet tried if p5py is usable for visualising data science or deep learning models or is it better to use libraries made for Python or web. I believe D3.js could be a strong option bit it would be a new tool to learn unlike processing framework
visualize the output of a model after your trained it (still image) ?
visualize your model as you train it (animation) ?
build and visualize your model within the same environment ?
As you rightly pointed out, you cannot import numpy or any library with C extensions in Processing.py. So in the first 2 cases you would have to run your model in a separate environment (Jupyter Notebooks suggested above by tabreturn is great – if you are not familiar with it already) and then import your trained data in Processing.py as a JSON or CSV file. For animations, you could save the result of each epoch or iteration and then play the learning process back in a Processing sktech.
Regarding the last case, I guess you could use p5js along with ml5js (high-level interface to TensorFlow.js) and build + visualize your model within the same sketch but then no Python nor Pytorch.
Now imagine that I have a model (Deep Reinf. Learning or simple Q-learning) it does not matter, all that matters is that at every draw() the Processing must interact with the model (which is in Python).
If we are talking about vision-based strategy, it would have to feed the pixels of every frame into the model, which would forward and give an output (in this case, given the frame of the game if it must be pressed down or up) which then would be used in the draw to update the state of the game.
Based on the answers that you give me, I can not use Processing to model my environment/game if I am using Python machine learning libraries.
Not exactly. @tabreturn summed it up well in his answer above, you can either:
try to import a Java binding of PyTorch in your Processing.py sketch (manipulating the binding with Python 2.7 flavour)
try to import your original PyTorch installation (and all your machine learning libraries) but in a different environment called ‘p5py’ (non-official WIP native Python 3 port of Processing)
That said, have you considered using other programs like PyGame, Pyglet or even Blender for your project ?
@willtl, you can use p5py for visualising your reinforcement learning model. You can create the processing sketch using few global variables. And, after every training step, update the global variables. The p5py sketch will automatically get updated accordingly.