# Using PoseNet/Runway on Processing

**URL:** <https://discourse.processing.org/t/using-posenet-runway-on-processing/14979>\
**Category:** Libraries\
**Created:** [October 28, 2019, 6:50am UTC](https://discourse.processing.org/t/using-posenet-runway-on-processing/14979 "2019-10-28T06:50:43Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![natttyk](https://avatars.discourse-cdn.com/v4/letter/n/e274bd/32.png) [@natttyk](https://discourse.processing.org/u/natttyk)\
**Post date:** [October 28, 2019, 6:50am UTC](https://discourse.processing.org/t/using-posenet-runway-on-processing/14979/1 "2019-10-28T06:50:43Z")

</div>

Hello! Newbie here.

I’m using the PoseNet Library for Runway on Processing ([https://github.com/runwayml/processing/blob/master/posenet/posenet.pde](https://github.com/runwayml/processing/blob/master/posenet/posenet.pde)). The current examples only map out the joints in ellipses, but I want to be able to visualize the skeletal structure as well. I’m guessing it’s something I need to add here, but can’t figure it out.

// A function to draw humans body parts as circles  
void drawParts() {  
// Only if there are any humans detected  
if (data != null) {  
humans = data.getJSONArray(“poses”);  
for(int h = 0; h \< humans.size(); h++) {  
JSONArray keypoints = humans.getJSONArray(h);  
// Now that we have one human, let’s draw its body parts  
for (int k = 0; k \< keypoints.size(); k++) {  
// Body parts are relative to width and weight of the input  
JSONArray point = keypoints.getJSONArray(k);  
float x = point.getFloat(0);  
float y = point.getFloat(1);  
ellipse(x \* width, y \* height, 5, 5);  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![kll](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.processing.org/kll/32/964_2.png) [@kll](https://discourse.processing.org/u/kll)\
**Post date:** [October 28, 2019, 9:15am UTC](https://discourse.processing.org/t/using-posenet-runway-on-processing/14979/2 "2019-10-28T09:15:03Z")

</div>

hi,

-a- please post your code  
using the

```auto
</> button from the editor menu

```

looks like

> ```  
> type or paste code here  
> ```

* * *

-b- after we know exactly what library you have installed? this one?  
[https://github.com/runwayml/processing-library/releases/download/latest/RunwayML.zip](https://github.com/runwayml/processing-library/releases/download/latest/RunwayML.zip)  
and when?, as there is a very recent update ( about the JSON data structure )

-c- can you post the whole code that we can try it  
but best is to say also from what example you started?

* * *

but possibly all your question is about?  
connect all the (xi,yi) of that FOR loop with a line?

or as  
`line(startX,startY,endX,endY);`  
like in the PoseNetWebcam example?

---

<div class="post-metadata">

**Author:** ![natttyk](https://avatars.discourse-cdn.com/v4/letter/n/e274bd/32.png) [@natttyk](https://discourse.processing.org/u/natttyk)\
**Post date:** [October 28, 2019, 9:27pm UTC](https://discourse.processing.org/t/using-posenet-runway-on-processing/14979/3 "2019-10-28T21:27:00Z")

</div>

Hi there,

This is the code that I’m using

```auto
// Copyright (C) 2019 RunwayML Examples
// 
// This file is part of RunwayML Examples.
// 
// Runway-Examples is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// 
// Runway-Examples is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// 
// You should have received a copy of the GNU General Public License
// along with RunwayML. If not, see <http://www.gnu.org/licenses/>.
// 
// ===========================================================================

// RUNWAYML
// www.runwayml.com

// PoseNet Demo:
// Receive OSC messages from Runway
// Running PoseNet model

// Import OSC
import oscP5.*;
import netP5.*;

// Runway Host
String runwayHost = "127.0.0.1";

// Runway Port
int runwayPort = 57100;

int width = 400;
int height = 350;

OscP5 oscP5;
NetAddress myBroadcastLocation;

// This array will hold all the humans detected
JSONObject data;
JSONArray humans;

// This are the pair of body connections we want to form. 
// Try creating new ones!
String[][] connections = {
  {"nose", "leftEye"},
  {"leftEye", "leftEar"},
  {"nose", "rightEye"},
  {"rightEye", "rightEar"},
  {"rightShoulder", "rightElbow"},
  {"rightElbow", "rightWrist"},
  {"leftShoulder", "leftElbow"},
  {"leftElbow", "leftWrist"}, 
  {"rightHip", "rightKnee"},
  {"rightKnee", "rightAnkle"},
  {"leftHip", "leftKnee"},
  {"leftKnee", "leftAnkle"}
};

void setup() {
  size(400, 350);
  frameRate(25);

  OscProperties properties = new OscProperties();
  properties.setRemoteAddress("127.0.0.1", 57200);
  properties.setListeningPort(57200);
  properties.setDatagramSize(99999999);
  properties.setSRSP(OscProperties.ON);
  oscP5 = new OscP5(this, properties);

  // Use the localhost and the port 57100 that we define in Runway
  myBroadcastLocation = new NetAddress(runwayHost, runwayPort);

  connect();

  fill(255);
  stroke(255);
}

void draw() {
  background(0);
  // Choose between drawing just an ellipse
  // over the body parts or drawing connections
  // between them. or both!
  drawParts();
  drawSkeleton();
}

// A function to draw humans body parts as circles
void drawParts() {
  // Only if there are any humans detected
  if (data != null) {
    humans = data.getJSONArray("poses");
    for(int h = 0; h < humans.size(); h++) {
      JSONArray keypoints = humans.getJSONArray(h);
      // Now that we have one human, let's draw its body parts
      for (int k = 0; k < keypoints.size(); k++) {
        // Body parts are relative to width and weight of the input
        JSONArray point = keypoints.getJSONArray(k);
        float x = point.getFloat(0);
        float y = point.getFloat(1);
        ellipse(x * width, y * height, 5, 5);
      }
    }
  }
}

void drawSkeleton() {
  // Only if there are any humans detected
  if (data != null) {
    humans = data.getJSONArray("poses");
    for(int h = 0; h < humans.length(); h++) {
      JSONArray keypoints = humans.getJSONArray(h);
      // Now that we have one human, let's draw its body parts
      for (int k = 0; k < keypoints.length(); k++) {
        // Body parts are relative to width and weight of the input
        JSONArray point = keypoints.getJSONArray(k);
        float x = point.getFloat(0);
        float y = point.getFloat(1);   
        line(x * width * 0.25, y * height * 0.25, x, y);
      }
    }
  }
}

void connect() {
  OscMessage m = new OscMessage("/server/connect");
  oscP5.send(m, myBroadcastLocation);
}

void disconnect() {
  OscMessage m = new OscMessage("/server/disconnect");
  oscP5.send(m, myBroadcastLocation);
}

void keyPressed() {
  switch(key) {
    case('c'):
      /* connect to Runway */
      connect();
      break;
    case('d'):
      /* disconnect from Runway */
      disconnect();
      break;

  }
}

// OSC Event: listens to data coming from Runway
void oscEvent(OscMessage theOscMessage) {
  if (!theOscMessage.addrPattern().equals("/data")) return;
  // The data is in a JSON string, so first we get the string value
  String dataString = theOscMessage.get(0).stringValue();

  // We then parse it as a JSONObject
  data = parseJSONObject(dataString);
}

```

from here: [https://github.com/runwayml/processing/blob/master/posenet/posenet.pde](https://github.com/runwayml/processing/blob/master/posenet/posenet.pde)  
I installed it a couple days ago.

Yes I think I need to connect the (xi,yi) as mentioned, but I can’t seem to get the right points.  
Super newbie so still struggling with this simple thing.

Thank you for all the help!