Combining PoseNet with Daniel Shiffman's Fractal Trees Example to create leaves

p5.js Libraries
1

Hi! I’m trying to create a project with posenet and using Daniel Shiffman’s Coding Challenge #17: Fractal Trees - Space Colonization. Using his example, instead of the leaves generating randomly, I want them to only generate at the position of my right wrist. In daniel schiffman’s leaf.js example code, I want the leaves to generate from the right wrist, hence doing

function Leaf() {

if (pose) {
RHand = pose.rightWrist;
createVector(RHand.x, RHand.y);
this.reached = false;

this.show = function() {
  fill(255,100);
  noStroke();
  ellipse(this.pos.x, this.pos.y, 4, 4);
};

}
}

However, I think it has a problem with copying vectors? It wont allow me to set the new position of the leaves to my wrist position, with the error saying:

p5.min.js:3 Uncaught (in promise) TypeError: Cannot read property 'copy' of undefined
    at l.default.Vector.dist (p5.min.js:3)
    at Function.l.default.Vector.dist (p5.min.js:3)
    at new Tree (tree.js:28)
    at setup (sketch.js:62)
    at _setup (p5.min.js:3)
    at _start (p5.min.js:3)
    at new _ (p5.min.js:3)
    at p5.min.js:3

How can I tackle this?

I tried doing the same thing in tree.js,

if (pose) {

for (var i = 0; i < 1500; i++) {
  this.leaves.push(new Leaf(random(width), random(height - 100)));
}

}
But it also creates an error whenever I put it in the pose loop. I’m definately doing it wrongly, but how can I give the leaves the posenet keypoints?

Thank you so much!

1 Like
2

Hi @yuinjann,

Hard to tell without access to your complete source code. What stands out to me is the line where a vector is created but not assigned to anything. Did you intend to do something like this, maybe?

this.pos = createVector(RHand.x, RHand.y);
3

Hi @Sven ! Yes sorry I copied the code over wrong sorry! I did

function Leaf() {

if (pose) {
RHand = pose.rightWrist;
this.pos =createVector(RHand.x, RHand.y);
this.reached = false;

this.show = function() {
  fill(255,100);
  noStroke();
  ellipse(this.pos.x, this.pos.y, 4, 4);
};

}
}

I was trying to assign the position of the leaves to the position of my right wrist, and the code is the same as Daniel Schiffman’s source code,, except me adding a camera, posenet, and changing the leaf position to that of the right wrist. Not sure if its because of the vectors? It works if I assign the leaf value to other values like (width/2, height/2), just not the poseNet ones

function setup() {
createCanvas(640,480);
video = createCapture(VIDEO);
video.hide();
poseNet = ml5.poseNet(video, modelLoaded);
poseNet.on(‘pose’, gotPoses);
tree = new Tree();

pixelDensity(1);
pg = createGraphics(width,height);
}
function modelLoaded(){
console.log(‘poseNet ready’);
}
function gotPoses(results) {
poses = results;
if (results.length > 0) {
pose = results[0].pose;
skeleton = results[0].skeleton;
}

thank you so much for replying by the way!

1 Like
4

The posenet model works as well, just not when combined

5

thank you so much for replying by the way!

You’re most welcome! :smiling_face: If you share your complete project on https://editor.p5js.org or GitHub or similar, I will happily troubleshoot further.

If you, for some reason or another, don’t want to share your project: create a Minimal, Reproducible Example.

2 Likes
6

Thank you so much @Sven that is so kind!
https://editor.p5js.org/yuinjann/sketches/124deMgkj

I was also trying to initiate the root of the tree from a hand point, but couldn’t get it to work in setup, hence the uncommented areas.

If you go into leaf.js and uncomment the first function, the original code will work. Thank you so so much again!

1 Like
7

Okay, let us take a closer look at Leaf and ponder about the code for a while.

function Leaf() {

  this.hands = [];

  if (pose) {
    RHand = pose.rightWrist;
    //error here
    this.pos = createVector(RHand.x, RHand.y);
  }

  // this.pos = createVector(random(width), random(height - 100));
  this.reached = false;

  this.show = function() {
    fill(255, 100);
    noStroke();
    ellipse(this.pos.x, this.pos.y, 4, 4);
  };
}

Early on, there’s an if statement checking if pose is truthy. It’s undefined, however, so the code inside the if statement never runs. Is that what you expect?

1 Like
8

hi @Sven ! I intended to put this.reached and this.show in the pose loop too, however that causes the same error. If I draw an ellipse at RHand.x, RHand.y in the pose loop, it works, hence I thought that writing this.pos in the pose loop will assign the leaf position to (RHand.x, RHand.y). I thought that the only way to access the poses is by putting it in the pose loop so I put it in there. The pose is from the gotPoses in index.js :slight_smile:

9

Yeah, the error won’t go away until the pose variable stops being undefined. It’s probably caused by a race condition. The tree (and thus the leaves) are instantiated in the setup function when your sketch starts. The gotPoses function (that sets the pose variable) runs whenever there are poses available. On my machine, that’s several seconds after the sketch is started.

So, when the tree (and leaves) are drawn, there are no poses (and right wrists) available to look at. You have to defer drawing the tree until after pose is set for the first time.

1 Like
10

@Sven thank you so so much! That really helped me! :smiley: I managed to get the eye to generate a leaf and the tree to grow at my right wrist! Could I ask you one more piece of advice though… I wanted the tree branches to connect to the leaves created by eye position, but the tree only connects to one of the eye position each time.
snippet from tree.js, but its in the sketch link below:

function Tree(x,y) {
this.leaves = ;
this.branches = ;
if (pose) {
RHand = pose.rightWrist;
LEye = pose.leftEye;
ellipse (LEye.x,LEye.y,100)
this.leaves.push(new Leaf(LEye.x, LEye.y));

Whereas before, I could generate many leaves with this iteration loop

for (var i = 0; i < 1500; i++) {
this.leaves.push(new Leaf(random(width), random(height - 100)));    }

What can I do so that the branch connects to each new eye position point? I made an array, but that just creates many trees connected to the one point.

Thanks again for everything!!!

1 Like
11

Hey I figured it out, I just push it to the tree :woman_facepalming: :woman_facepalming: :woman_facepalming: thank you so much again! :grin: :grin: :grin:

2 Likes