# Deep Reinforcement Learning with Pytorch and Processing
**URL:** https://discourse.processing.org/t/deep-reinforcement-learning-with-pytorch-and-processing/24453
**Category:** p5py
**Created:** [October 8, 2020, 9:36pm UTC](https://discourse.processing.org/t/deep-reinforcement-learning-with-pytorch-and-processing/24453 "2020-10-08T21:36:13Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@willtl](https://discourse.processing.org/u/willtl)
#### Post date: [October 8, 2020, 9:36pm UTC](https://discourse.processing.org/t/deep-reinforcement-learning-with-pytorch-and-processing/24453/1 "2020-10-08T21:36:13Z")
I was checking how to use Processing with Python support. However, I read that in order to use Python libraries I would have to place then in the folder of my Processing project. Moreover, I also read that I would not be able to use libraries with C extensions such as _numpy_ or _pytorch_.
That’s exactly why I would like to use processing. I would like to define some environments/simulations and then after training, visualize then using Processing.
Is there any way to do both together? Use Pytorch and Processing at the same time?
---
### Author:  [@tabreturn](https://discourse.processing.org/u/tabreturn)
#### Post date: [October 8, 2020, 10:01pm UTC](https://discourse.processing.org/t/deep-reinforcement-learning-with-pytorch-and-processing/24453/2 "2020-10-08T22:01:35Z")
Welcome @willtl
[p5 (or p5py)](https://p5.readthedocs.io/en/latest/) is a native Python port of the Processing API. It’s a Python library that doesn’t utilize Processing at all. You can import and run any Python library with p5py.
[Processing.py (or Python Mode for Processing)](https://py.processing.org/) uses Jython to run Python code in Processing – this is largely incompatible with Python libraries using C extensions.
Each solution has its own set of advantages and disadvantages.
I see that PyTorch [introduced Java bindings](https://www.infoq.com/news/2020/02/pytorch-releases-java-bindings/), so there’s probably some way you can run it using Processing.py (importing it as a Java library that Jython can leverage).
---
### Author:  [@SomeOne](https://discourse.processing.org/u/SomeOne)
#### Post date: [October 9, 2020, 6:54am UTC](https://discourse.processing.org/t/deep-reinforcement-learning-with-pytorch-and-processing/24453/3 "2020-10-09T06:54:05Z")
I like Python, it’s my favourite language at the moment. I do also like Processing IDE (PDE) and it’s unfortunate that python mode in PDE is tied to Jython and that prevents using important data science libraries. I find though Jupyter Notebook to be better environment to test and play with data science stuff , including deep learning.
I haven’t yet tried if p5py is usable for visualising data science or deep learning models or is it better to use libraries made for Python or web. I believe [D3.js](https://d3js.org/) could be a strong option bit it would be a new tool to learn unlike processing framework
---
### Author:  [@solub](https://discourse.processing.org/u/solub)
#### Post date: [October 9, 2020, 12:22pm UTC](https://discourse.processing.org/t/deep-reinforcement-learning-with-pytorch-and-processing/24453/4 "2020-10-09T12:22:44Z")
Hi willtl,
I am not sure to understand your needs.
> [@willtl](#):
>
> I would like to define some environments/simulations **and then after training** , visualize then using Processing.
> [@willtl](#):
>
> Is there any way to **do both together**?
Are you looking for a way to:
- visualize the output of a model _after_ your trained it (still image) ?
- visualize your model _as you train_ it (animation) ?
- build and visualize your model within the same environment ?
As you rightly pointed out, you cannot import numpy or any library with C extensions in Processing.py. So in the first 2 cases you would have to run your model in a separate environment (Jupyter Notebooks suggested above by tabreturn is great – if you are not familiar with it already) and then import your trained data in Processing.py as a JSON or CSV file. For animations, you could save the result of each epoch or iteration and then _play_ the learning process back in a Processing sktech.
Regarding the last case, I guess you could use [p5js](https://p5js.org/) along with [ml5js](https://ml5js.org/) (high-level interface to TensorFlow.js) and build + visualize your model within the same sketch but then no Python nor Pytorch.
---
### Author:  [@willtl](https://discourse.processing.org/u/willtl)
#### Post date: [October 9, 2020, 2:18pm UTC](https://discourse.processing.org/t/deep-reinforcement-learning-with-pytorch-and-processing/24453/5 "2020-10-09T14:18:35Z")
Let me introduce an example so you all can understand my needs.
Imagine I have an environment/game such as this one [https://www.youtube.com/watch?v=IIrC5Qcb2G4](https://www.youtube.com/watch?v=IIrC5Qcb2G4).
Now imagine that I have a model (Deep Reinf. Learning or simple Q-learning) it does not matter, all that matters is that at every draw() the Processing must interact with the model (which is in Python).
If we are talking about vision-based strategy, it would have to feed the pixels of every frame into the model, which would forward and give an output (in this case, given the frame of the game if it must be pressed down or up) which then would be used in the draw to update the state of the game.
Based on the answers that you give me, I can not use Processing to model my environment/game if I am using Python machine learning libraries.
---
### Author:  [@solub](https://discourse.processing.org/u/solub)
#### Post date: [October 9, 2020, 8:58pm UTC](https://discourse.processing.org/t/deep-reinforcement-learning-with-pytorch-and-processing/24453/6 "2020-10-09T20:58:41Z")
I see, thank you for the clarification.
> [@willtl](#):
>
> Based on the answers that you give me, I can not use Processing to model my environment/game if I am using Python machine learning libraries.
Not exactly. @tabreturn summed it up well in his answer above, you can either:
- try to import a Java binding of PyTorch in your Processing.py sketch (manipulating the binding with Python 2.7 flavour)
- try to import your original PyTorch installation (and all your machine learning libraries) but in a different environment called ‘p5py’ (non-official WIP native Python 3 port of Processing)
That said, have you considered using other programs like [PyGame](https://youtu.be/ra2o2bPZlwk), Pyglet or even Blender for your project ?
---
### Author:  [@parsoyaarihant](https://discourse.processing.org/u/parsoyaarihant)
#### Post date: [October 10, 2020, 1:48am UTC](https://discourse.processing.org/t/deep-reinforcement-learning-with-pytorch-and-processing/24453/7 "2020-10-10T01:48:26Z")
@willtl, you can use p5py for visualising your reinforcement learning model. You can create the processing sketch using few global variables. And, after every training step, update the global variables. The p5py sketch will automatically get updated accordingly.
If you see this example: [https://p5.readthedocs.io/en/latest/examples/transform/translate.html](https://p5.readthedocs.io/en/latest/examples/transform/translate.html) we have declared a few global variables on the top. If you update these variables from your training functions, the sketch will automatically get updated.