I’m happy to present you a project I’ve been working on for over a year now. With OpenCV for Processing we have a good base library for traditional image processing. However, these algorithms are more and more pushed into the background by machine learning.
Therefore I started a year ago to develop a library that (mainly) uses the Deep Neural Network (DNN) module of OpenCV to bring machine learning, especially CNN into the Processing world.
The library is not about training these networks, but about inferencing (executing a prediction) them.
The API of the library can and will still change, because it is just a pre-release.
Example: SSD MobileNet & Lightweight Open Pose
The library can be downloaded as usual from the Processing contribution manager.
At the moment, more than 25 different networks are implemented, from YOLO to Lightweight OpenPose to MIDAS. All of them have pre-trained weights. These networks are divided into different categories:
- Object Detection
- Object Segmentation
- Object Recognition
- Keypoint Detection
- Depth Estimation
- Image Processing
To keep the library small I developed a repository system that automatically downloads the files you need for the chosen network.
The implementation was optimized for the CPU and many networks run with 20-30 FPS on a good CPU. But it is also possible to use the CUDA backend of OpenCV. There you have to download a special version of the library, which only works on Linux x86/x64 and Windows and is quite big (2-5 GB).
At the moment I’m working on improving the speed on the CPU by optimizing the pre and post process. I am also experimenting with ONNX directly to have a second inference engine.
Additionally I’m still working on the documentation and more examples and I would be very happy if someone wants to contribute to it.
Thank you very much for testing & feedback.