Deep Vision - Machine Learning Computer Vision for Processing (Library)

Hello Together

I’m happy to present you a project I’ve been working on for over a year now. With OpenCV for Processing we have a good base library for traditional image processing. However, these algorithms are more and more pushed into the background by machine learning.

Therefore I started a year ago to develop a library that (mainly) uses the Deep Neural Network (DNN) module of OpenCV to bring machine learning, especially CNN into the Processing world.

The library is not about training these networks, but about inferencing (executing a prediction) them.

:zap: The API of the library can and will still change, because it is just a pre-release.


Example: SSD MobileNet & Lightweight Open Pose

Installation

The library can be downloaded as usual from the Processing contribution manager.

Networks

At the moment, more than 25 different networks are implemented, from YOLO to Lightweight OpenPose to MIDAS. All of them have pre-trained weights. These networks are divided into different categories:

  • Object Detection :sparkles:
  • Object Segmentation
  • Object Recognition :blue_car:
  • Keypoint Detection :woman_playing_handball:t2:
  • Classification :cat2:
  • Depth Estimation :dark_sunglasses:
  • Image Processing

To keep the library small I developed a repository system that automatically downloads the files you need for the chosen network.

Performance

The implementation was optimized for the CPU and many networks run with 20-30 FPS on a good CPU. But it is also possible to use the CUDA backend of OpenCV. There you have to download a special version of the library, which only works on Linux x86/x64 and Windows and is quite big (2-5 GB).

Next Steps

At the moment I’m working on improving the speed on the CPU by optimizing the pre and post process. I am also experimenting with ONNX directly to have a second inference engine.

Additionally I’m still working on the documentation and more examples and I would be very happy if someone wants to contribute to it.

Thank you very much for testing & feedback.

14 Likes

This is amazing, really. I’ve been looking for something like this! :tada:

1 Like

This is amazing! Thank you so much. I will be using this to teach my students. Fantastic work

1 Like