Update, I adapted some code from https://www.apress.com/gp/book/9781484227749
Chapter 6_07:
https://github.com/Apress/pro-processing-images-and-computer-vision-opencv/tree/master/Chapter%206/Chapter06_07
Basically it creates blobs on motion detected change between two frames.
I mapped those x,y blob position coordinates to a rectangle that has an arrive function.
But i cannot make it to work well, it moves the rectangle but it doesnt get fixed to that position, just wanders through all the screen.
I just want to associate detected blobs with a moving object, that in case if the blob dissapears it would stay in the last blob position, and just if it is near that blob area.
My idea is to use 2 objects one for each hand, and move one or another only if it is nearby the detected movement area.
here is the code: