# How to do shape recognition in video feed?
**URL:** https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408
**Category:** Libraries
**Created:** [March 10, 2021, 7:10pm UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408 "2021-03-10T19:10:37Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@samuset](https://discourse.processing.org/u/samuset)
#### Post date: [March 10, 2021, 7:10pm UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/1 "2021-03-10T19:10:37Z")
Hi, I’m looking to have a number of cubes with different irregular polygons on top of them on a table, with a camera looking down onto them. I’d like to get the position and ID of each cube, based on the shapes on them. Is this possible with some sort of shape tracking, computer vision library? Thank you for the help in advance.
---
### Author:  [@micuat](https://discourse.processing.org/u/micuat)
#### Post date: [March 10, 2021, 8:50pm UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/2 "2021-03-10T20:50:15Z")
Hi! Do you have anything already in mind e.g. a photo/image of those shapes?
RGB camera based tracking can be hard. I’m an enthusiastic supporter of open source software, but if you are new to computer vision, first looking into tools like [vuforia](https://developer.vuforia.com/) or [arkit](https://developer.apple.com/documentation/arkit/content_anchors/detecting_images_in_an_ar_experience) may help knowing what’s possible and what isn’t (right, they are not for Processing - but sometimes it helps using other tools and combining with e.g. OSC communication can be a faster way to achieve it).
You can of course develop something with OpenCV like [feature matching](https://docs.opencv.org/4.5.1/d7/dff/tutorial_feature_homography.html) but if you are not familiar with computer vision (or even if you are familiar with it) you can easily spend hours and hours just tuning parameters and end up not getting good results (which used to happen to me)
---
### Author:  [@samuset](https://discourse.processing.org/u/samuset)
#### Post date: [March 11, 2021, 9:52am UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/3 "2021-03-11T09:52:52Z")
Hi, thanks for the advice. I have something like this feature in BoofCV in mind, as described [here](http://boofcv.org/index.php?title=Example_Detect_Black_Polygons):
If only it could not only detect the black polygons, but also differentiate between them (ie. return their type) and their 2D screen position…
---
### Author:  [@micuat](https://discourse.processing.org/u/micuat)
#### Post date: [March 11, 2021, 11:02am UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/4 "2021-03-11T11:02:51Z")
Cool, I never heard of this. Is there a reason why you don’t want to use this library?
[http://boofcv.org/index.php?title=Tutorial\_Processing](http://boofcv.org/index.php?title=Tutorial_Processing)
---
### Author:  [@samuset](https://discourse.processing.org/u/samuset)
#### Post date: [March 11, 2021, 11:14am UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/5 "2021-03-11T11:14:41Z")
I would happily if it could also detect the position and type of the black polygons. I’ve used BoofCV for other projects for different use cases and it’s pretty solid.
---
### Author:  [@micuat](https://discourse.processing.org/u/micuat)
#### Post date: [March 11, 2021, 11:18am UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/6 "2021-03-11T11:18:09Z")
I haven’t looked at the code but I’m surprised if it doesn’t give you the positions if they claim sub pixel precision
---
### Author:  [@micuat](https://discourse.processing.org/u/micuat)
#### Post date: [March 11, 2021, 7:41pm UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/7 "2021-03-11T19:41:11Z")
looking at this example, doesn’t it give you complete polygon info on the screen space…?
>
---
### Author:  [@samuset](https://discourse.processing.org/u/samuset)
#### Post date: [March 15, 2021, 9:32am UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/8 "2021-03-15T09:32:51Z")
Oh I was looking at another piece of code, where I didn’t see this. Thank you. Will look into it this week and report back.
---
### Author:  [@samuset](https://discourse.processing.org/u/samuset)
#### Post date: [March 17, 2021, 4:49pm UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/9 "2021-03-17T16:49:33Z")
Right, so yes, thank you, this does indeed work. It gives you the vertices of the recognized polygon and from those, one can calculate the centroid of the polygon in one way or another.
One way the type of each polygon, as if they were unique markers, can be deduced is by getting the # of vertices each polygon has. This depends on the precision of the readings, of course.
---
### Author:  [@micuat](https://discourse.processing.org/u/micuat)
#### Post date: [March 17, 2021, 8:27pm UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/10 "2021-03-17T20:27:38Z")
another tip is to give ids to the recognized shapes, check the centroids from the previous frame and inherit the id from the closest shape. But it all depends on the situation (if there’s occlusion etc). An example is ofxCv from openFrameworks
>
---
### Author:  [@samuset](https://discourse.processing.org/u/samuset)
#### Post date: [March 18, 2021, 1:02pm UTC](https://discourse.processing.org/t/how-to-do-shape-recognition-in-video-feed/28408/11 "2021-03-18T13:02:48Z")
Yes, that would work for tracking them by id. What I was saying, though, is assigning the same id to quads, same other id to triangles, same third id to hexagons each time they appear. I.e. a kind of marker recognition.