Need help to send video with oscP5

Processing Libraries
1

Hello guys, I need some help with a couple of sketches where I’m using the oscP5 and the video libraries. In the first sketch I capture the video from my camera and then I extrapolate the RGB float variables and send them with an osc message to the second pc running the other sketch. The second pc receive the floats but i’m probably doing something wrong because it doesn’t organize the pixels in the right way (the result is a glitch video); Can you please help me understand what I’m doing wrong?
Here’s the firts code

//Sender
import oscP5.*;
import netP5.*;
import processing.video.*;

OscP5 oscP5;
NetAddress myRemoteLocation;

Capture cam;

float r;
float g;
float b;

void setup() {
  size(640,480);
  frameRate(25);

  String[] cameras = Capture.list();
  
  cam = new Capture(this,width,height);

  cam = new Capture(this, cameras[0]);
  
  cam.start();
  
  oscP5 = new OscP5(this,12000);
  
// In this post I use the same IP address to make you run the sketches //on the same pc
  myRemoteLocation = new NetAddress("127.0.0.1",13000);
}

void draw() {
  background(0); 
      
  if (cam.available() == true) {
    cam.read();
  }
  
  cam.loadPixels();
  
  for (int y = 0; y < height; y++) {
    for (int x = 0; x < width; x++) {
     
      int loc = x + y * width;
      
      r = red(cam.pixels[loc]);
      g = green(cam.pixels[loc]);
      b = blue(cam.pixels[loc]); 
            
      oscP5.send("/b", new Object[] {new Float(r), new Float(g), new Float(b)}, myRemoteLocation); 
    }        
  } 
  
}

Here’s the second code

import oscP5.*;
import netP5.*;

PImage vid;

OscP5 oscP5;
NetAddress myRemoteLocation;

float r;
float g;
float b;

void setup() {
  size(640,480);
  frameRate(60);

  oscP5 = new OscP5(this,13000);
  
  vid = createImage(width,height,RGB);

}

void draw() {

 background(0); 
 
 vid.loadPixels();
 
 for (int y = 0; y < height; y++) {
   for (int x = 0; x < width; x++) {
     
     int loc = x + y * width;
     
     float red = r;
     float green = g;
     float blue = b;
     
     vid.pixels[loc] = color(red,green,blue);
   }    
 }
 
 vid.updatePixels();
 
 image(vid,0,0,width,height);
 
 
}

void oscEvent(OscMessage theOscMessage) {
 
  if(theOscMessage.checkAddrPattern("/b")==true) {

    if(theOscMessage.checkTypetag("fff")) {
 
      r = theOscMessage.get(0).floatValue();  // get the first osc argument
      g = theOscMessage.get(1).floatValue(); // get the second osc argument
      b = theOscMessage.get(2).floatValue(); // get the third osc argument
      print("### received an osc message /b with typetag fff.");
      println(" values: "+r+", "+g+", "+b);
      return;
    } 
  }
  println("### received an osc message. with address pattern "+
          theOscMessage.addrPattern()+" typetag "+ theOscMessage.typetag());
}
2

Hi! I don’t have a webcam so I cannot test it now, but at a glance it seems not a good idea to send all the pixels in separate messages. Basically, it’s not realistic to be able to send 640x480 messages at 25 or 30 fps. You should start with downsampling (change x++ and y++ to something like x+=32).

The message only contains rgb, so you need to address which pixel it’s about by adding x and y, or pack all the pixels in one array… but again, array of 640x480 is too big and won’t be able to send in one message (there is a parameter to make this value bigger, but in this case it’s simply not possible to send video over OSC). Also red, green, blue are inefficient as written in the reference. You can actually just send cam.pixels[loc] as a single int value I suppose.

1 Like
3

got it, I’ll try this way you suggested me. Do you know any other library that can make me send the video capture from a pc to another in the same network?

4

it really depends on the setup and objective - I would use NDI to capture the video and send it to the other machine. On that machine, you can use “webcam input” to use the NDI stream as a virtual webcam, but I’m not sure if Processing supports that.

1 Like
5

mmm i hope so, the purpose of this project is the making of an interactive projection, the idea was to take the video with a dls camera on one pc and extrapolate the rgb data, then send those to the other machine to resynthesize the image end project that with obs. I wanted to split the work on two computers to not strain only one machine

6

In theory if you added in the NDI plugin for GStreamer, and forced the Video library to use a system installed version, you could use it as the capture source.

I would look into another streaming source though. You can pass a pipeline configuration into the Video capture device, so any stream type supported natively by GStreamer should work as input.

1 Like
7

@neilcsmith Just a question (please forgive me but I’m not properly a professional programmer :cry: I use processing for some of my works and I’m still learning this lenguage):

  • This plugin is only for linux or is there a version for windows? Because the link you shared takes me to the github page for the linux version;
8

No idea! Sorry, I know it exists from conversations I’ve had in the past, but never made use of it.

One way of reading my “in theory” is look for something other than NDI to stream from the other computer as they’ll be easier to hook into Processing via GStreamer (as used in the Video library) on the receiving end. Unless there’s an NDI receiver software that exposes as a virtual webcam? Or Spout?

1 Like
9

I’ll do some researches about that, thanks anyway guys, now I know what i have to look for clearlier :+1: I will post what I find if I succeed

10

yes it’s bundled in NDI tools (but their naming convention is so confusing and I forgot what it’s called) - but another potential problem is that Processing is not able to read that source (I never tried reading it with Processing)

1 Like
11

Guys your suggestions were really usefull! I think I’ve found the solution to my problem, i’ll use Spout as a receiver for the sketch plus the NDI/Spout plugin, then i send the video to NDI Test Monitor. I can send the NDI video source on OBS thanks to the NDI/OBS plugin wich allows me to receive any NDI source on this software :raised_hands: :raised_hands: :raised_hands:

1 Like