# Doubts regarding pose estimation on 2D video using posenet?

**URL:** <https://discourse.processing.org/t/doubts-regarding-pose-estimation-on-2d-video-using-posenet/9675>\
**Category:** Libraries\
**Created:** [March 27, 2019, 9:28am UTC](https://discourse.processing.org/t/doubts-regarding-pose-estimation-on-2d-video-using-posenet/9675 "2019-03-27T09:28:50Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![basicvisual](https://avatars.discourse-cdn.com/v4/letter/b/9dc877/32.png) [@basicvisual](https://discourse.processing.org/u/basicvisual)\
**Post date:** [March 27, 2019, 9:28am UTC](https://discourse.processing.org/t/doubts-regarding-pose-estimation-on-2d-video-using-posenet/9675/1 "2019-03-27T09:28:51Z")

</div>

Hello I have been playing with the pre trained model for posenet . I had a general question, I tried to do pose estimation on a random video . I seem to get results totally where the skeleton is drawn totally off the human. I was wondering if posenet is not good for detecting poses from a video sequence or it is good for pose estimation with webcams , where the person / persons are facing the webcam . In the video sequence the camera is changing ( we have different angles ). It could be also wrong in something I am doing in the code . Help and explanation would be appreciated .

![failure](https://canada1.discourse-cdn.com/flex036/uploads/processingfoundation1/original/2X/9/92332551e7682e98a7418e8a00c2be0237deb697.jpeg)

```auto
var video;
// file from:'http://clips.vorwaerts-gmbh.de/big_buck_bunny.ogv';
// var videofile = 'data/big_buck_bunny.ogv';
var videofile = 'test.mp4';
let poseNet;
let poses = [];
let skeletons = [];
var imageScaleFactor= 0.3;
var outputStride=16 ;
var minConfidence = 0.5;
var maxPoseDetections = 1;
var scoreThreshold = 0.5 ;
var multiplier = 0.75;

function setup() {
  createCanvas(400, 400);
  video = createVideo(videofile,onLoad);
  //Posenet
  poseNet = ml5.poseNet(video,imageScaleFactor,outputStride,minConfidence, maxPoseDetections,scoreThreshold,multiplier);
  console.log(ml5);
  //poseNet.on('pose', gotPoses);
  poseNet.on('pose', function (results) {
    poses = results;
    });
    video.hide();

}

function onLoad() { // This function is called when the video loads
// print("start auto play after load");
// video.play();
  print("mouse click to start");
}

function modelReady() {
  console.log('model ready');
}

//Pose Net 
function gotPoses(poses) {
   console.log(poses);
}

function draw() {
  image(video, 0, 0, width, height);

  // We can call both functions to draw all keypoints and the skeletons
 drawKeypoints();
  drawSkeleton();
}
//=======Draw skeleton and keypoints ===/

// A function to draw ellipses over the detected keypoints
function drawKeypoints() {
  // Loop through all the poses detected
  for (let i = 0; i < poses.length; i++) {
    // For each pose detected, loop through all the keypoints
    for (let j = 0; j < poses[i].pose.keypoints.length; j++) {
      // A keypoint is an object describing a body part (like rightArm or leftShoulder)
      let keypoint = poses[i].pose.keypoints[j];
      // Only draw an ellipse is the pose probability is bigger than 0.2
      if (keypoint.score > 0.1) {
        fill(255, 0, 0);
        noStroke();
        ellipse(keypoint.position.x, keypoint.position.y, 10, 10);
      }
    }
  }
}

// A function to draw the skeletons
function drawSkeleton() {
  // Loop through all the skeletons detected
  for (let i = 0; i < poses.length; i++) {
    // For every skeleton, loop through all body connections
    for (let j = 0; j < poses[i].skeleton.length; j++) {
      let partA = poses[i].skeleton[j][0];
      let partB = poses[i].skeleton[j][1];
      stroke(255, 0, 0);
      line(partA.position.x, partA.position.y, partB.position.x, partB.position.y);
    }
  }
}

//======End skeleton ============/
function mousePressed() {
  video.loop(); // set the video to loop mode ( and start )
  print("set loop mode");
}

```

---

<div class="post-metadata">

**Author:** ![jeremydouglass](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/jeremydouglass/32/20_2.png) [@jeremydouglass](https://discourse.processing.org/u/jeremydouglass)\
**Post date:** [March 28, 2019, 9:01pm UTC](https://discourse.processing.org/t/doubts-regarding-pose-estimation-on-2d-video-using-posenet/9675/2 "2019-03-28T21:01:07Z")

</div>

I don’t _think_ cuts in the video content should matter – I _think_ (don’t know for certain) that each video frame gets an independent estimate. That said, I do think it is optimized for centered subjects facing the camera – and/or those would give best results as they contain more information.

I’m not sure, but I think that getting some erroneous arm data on certain frames is pretty normal – I can see that happening in this writeup, for example, with the center person’s arms often not reading correctly:

[https://cdn-images-1.medium.com/max/1200/1\*Gsx7MLBj2LKiaDsab0iNjg.gif(image larger than 4096KB)](https://cdn-images-1.medium.com/max/1200/1*Gsx7MLBj2LKiaDsab0iNjg.gif)

> **[Real-time Human Pose Estimation in the Browser with TensorFlow.js](https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5)**
>
> Posted by: Dan Oved, freelance creative technologist at Google Creative Lab, graduate student at ITP, NYU. 
> Editing and illustrations…

---

<div class="post-metadata">

**Author:** ![basicvisual](https://avatars.discourse-cdn.com/v4/letter/b/9dc877/32.png) [@basicvisual](https://discourse.processing.org/u/basicvisual)\
**Post date:** [March 29, 2019, 8:23am UTC](https://discourse.processing.org/t/doubts-regarding-pose-estimation-on-2d-video-using-posenet/9675/3 "2019-03-29T08:23:32Z")

</div>

Hi  
Thank you ! I was also under the impression that there was a pose estimation on each frame. From the examples, that one sees , as you mentioned the subject must be camera facing . I am not sure who to ask . Raised this as a issue in the github repository. Not sure if thats the right place. ’