Deserializing objects sent from a client program to a server program

Processing Libraries
1

Hi!

I’m trying to make 2 diffrerent programs using the net library included with processign and I’m trying to send data between a server and a client using serializable objects

The problem is that the client and the server are called diferently: “NCliente” and “NServidor” and their classes are “NCliente$ImagenSerializable” and “NServidor$ImagenSerializable”

When I try to deserialize the data sent from the client, it trows me a “java.lang.ClassNotFoundException: NCliente$ImagenSerializable” because the class used to serialize is “NCliente$ImagenSerializable”

I have found two workarrounds:

  1. Calling both files the same so their classes are the same
  2. Changing the bytes at one end to trick the program into thinking that it has been serialized whith it’s own class

I don’t like neither idea and I would like to know if there is a better way of avoiding the error using a more elegant solution

Thanks!

1 Like
2

Are these server and client programs defined using .pde sketches? Are you defining the ImagenSerializable class inside each sketch?

I’m just guessing, as I don’t have experience with this kind of serialization (and haven’t seen example code froom you) but I think you need to separate your ImagenSerializable class from your processing sketch – e.g. in a java file – and import it. Otherwise your class will be compiled by Processing as an inner class of your PApplet, which I believe (not sure) will result in you serializing the whole enclosing PApplet object unless the inner class is static. Then you would have two objects whose types don’t match – and which are unnecessarily large.

3

Thank you for your help and sorry for the long time of this response

Yes, both the server and the client are different .pde files

Yes, both ImagenSerializable are defined as different classes inside each scketch in their own .pde file (both files are equal but not the same)

When I do that and try to run the scketch it says that static is an illegal modifier for the class
The Serializable interface requires the class implementing int to be static

This is the code in case it helps:

// NCliente/NCliente.pde

import processing.net.*;
import java.io.*;

Client cliente = new Client(this, "62.151.23.106", 5204);
PImage rana;

void setup() {
  size(858, 536);

  rana = loadImage("data/frog.jpg");
  enviarImagen(rana);
}

void draw() {
  background(0);
  image(rana, 0, 0);
}

void enviarImagen(PImage imagen) {
  try {
    ImagenSerializable imagenSerializable = new ImagenSerializable(imagen);
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    ObjectOutputStream oos = new ObjectOutputStream(baos);

    oos.writeObject(imagenSerializable);
    byte[] datos = baos.toByteArray();

    baos.close();
    oos.close();

    String tamanoDatos = String.format("%8d", datos.length).replace(" ", "0");
    
    cliente.write(tamanoDatos);
    cliente.write(datos);
  } catch (Exception e) {
    e.printStackTrace();
  }
}

// NServidor/NServidor.pde

import processing.net.*;
import java.io.*;

Server servidor = new Server(this, 5204);
PImage imagenRecibida;

int tamanoDatos = -1;
byte[] datosIncompletos = new byte[0];

void setup() {
  size(858, 536);
}

void draw() {
  background(0);
  if (imagenRecibida != null) image(imagenRecibida, 0, 0);
}

void clientEvent(Client cliente) {
  byte[] datosRecibidos = splice(cliente.readBytes(), datosIncompletos, 0);
  datosIncompletos = new byte[0];

  while (datosRecibidos.length > 0) {
    // Determinar el tamaño de los datos recibidos
    if (datosRecibidos.length >= 8) {
      if (tamanoDatos == -1) {
        tamanoDatos = byteArrayAInteger(subset(datosRecibidos, 0, 8));
        datosRecibidos = subset(datosRecibidos, 8);
      }
    } else {
      datosIncompletos = datosRecibidos;
      return;
    }

    // Procesar los datos si hay suficientes
    if (datosRecibidos.length >= tamanoDatos) {
      try {
        ByteArrayInputStream bais = new ByteArrayInputStream(datosRecibidos);
        ObjectInputStream ois = new ObjectInputStream(bais);

        ImagenSerializable imagenSerializable = (ImagenSerializable)ois.readObject();
        imagenRecibida = imagenSerializable.aImagen(this);

        bais.close();
        ois.close();

        datosRecibidos = subset(datosRecibidos, tamanoDatos);
        tamanoDatos = -1;
      } catch (Exception e) {
        e.printStackTrace();
      }
    } else {
      datosIncompletos = datosRecibidos;
      return;
    }
  }
}


int byteArrayAInteger(byte[] bytes) {
  int numero = 0;
  for (int i = 0; i < bytes.length; i++) numero += Character.getNumericValue(bytes[i]) * pow(10, 7 - i);
  return numero;
}

// NCliente/ImagenSerializable.pde
// NServidor/ImagenSerializable.pde

static class ImagenSerializable implements Serializable {
  private static final long serialVersionUID = 1L;

  int formato;
  int ancho;
  int alto;
  int[] pixeles;

  ImagenSerializable(PImage imagen) {
    this.formato = imagen.format;
    this.ancho = imagen.width;
    this.alto = imagen.height;
    this.pixeles = imagen.pixels;
  }

  PImage aImagen(PApplet pApplet) {
    PImage imagen = pApplet.createImage(ancho, alto, formato);
    imagen.pixels = pixeles;
    return imagen;
  }
}

Thank you

4

I believe that this will not work. Processing concatonates all .pde files into a single text file and then transpiles that code it into a single Java class (PApplet) – so your ImagenSerializable will end up becomming an inner class. You cannot put it in a .pde tab. You need to separate your ImagenSerializable class from your processing sketch – not .pde file, but written in a separate .java file.

5

I’ve tried that and now it gives me this error:
Illegal modifier for the class ImagenSerializable; only public, abstract & final are permitted

I can’t make a top-level static class, but to implement the Serializable interface I need the class to be static I don’t know how to either make it non-static or non-top-level

Thank you

6 
static public class ImagenSerializable implements java.io.Serializable {
  static protected final long serialVersionUID = 1;

  int ancho, alto, formato, pixeles[];

  public ImagenSerializable(final PImage img) {
    ancho = img.width;
    alto = img.height;
    formato = img.format;
    img.loadPixels();
    pixeles = img.pixels.clone();
  }

  PImage crearImagen(final PApplet p) {
    final PImage img = p.createImage(ancho, alto, formato);
    img.loadPixels();
    arrayCopy(pixeles, img.pixels);
    img.updatePixels();
    return img;
  }
}
7

It still says Illegal modifier for the class ImagenSerializable; only public, abstract & final are permitted

8 
/**
 * Serializable PImage (v1.0.1)
 * GoToLoop (2019/Dec/26)
 *
 * https://Discourse.Processing.org/t/
 * deserializing-objects-sent-from-a-client-program-to-a-server-program/16124/8
 */

import java.io.Serializable;

import java.io.FileOutputStream;
import java.io.ObjectOutputStream;
import java.io.ObjectOutput;

import java.io.FileInputStream;
import java.io.ObjectInputStream;
import java.io.ObjectInput;

static final String FILENAME = "imagen.serial", URL =
  "https://" + "Forum.Processing.org/" + "processing-org.jpg";

PImage img;

void settings() {
  img = loadImage(URL);
  size(img.width, img.height);
  noLoop();
}

void setup() {
  saveSerial(FILENAME, new SerializableImage(img));
  img = ((SerializableImage) loadSerial(FILENAME)).createImage(this);
}

void draw() {
  background(img);
}

void saveSerial(final String filename, final Object o) {
  final File f = dataFile(filename);
  createPath(f);

  try {
    final ObjectOutput out = new ObjectOutputStream(new FileOutputStream(f));
    out.writeObject(o);
    out.close();
  }

  catch (final IOException e) {
    e.printStackTrace();
  }
}

Object loadSerial(final String filename) {
  final File f = dataFile(filename);

  try {
    final ObjectInput in = new ObjectInputStream(new FileInputStream(f));
    final Object o = in.readObject();
    in.close();
    return o;
  }

  catch (final IOException e) {
    e.printStackTrace();
  }

  catch (final ClassNotFoundException e) {
    e.printStackTrace();
  }

  return null;
}

static public class SerializableImage implements Serializable {
  static protected final long serialVersionUID = 123;

  public int w, h, type, scale, pix[];

  public SerializableImage(final PImage img) {
    w = img.width;
    h = img.height;
    type = img.format;
    scale = img.pixelDensity;
    img.loadPixels();
    pix = img.pixels.clone();
  }

  PImage createImage(final PApplet p) {
    final PImage img = new PImage(w, h, type, scale);
    img.parent = p;
    img.loadPixels();
    arrayCopy(pix, img.pixels);
    img.updatePixels();
    return img;
  }
}
9

Yes, your sketch works but because it is serializing and deserializing whithin the same scketch and thus class
But what I’m having problems with is having 2 programs talking to each other (deserializing the data recived), the rest is good
But when I try to deserialize java complains because the classes are different even tough they share the same source code

I tried using .java files to make processing make the class without the .pde so it would give me ImagenSerializable and not NCliente$ImagenSerializable but I can’t get it to work

Sorry if I didn’t explain my problem properly
Thank you for your help

1 Like
10

Based on // https://shiffman.net/processing.org/udp/2010/11/13/streaming-video-with-udp-in-processing/ I just implemented what you are looking for.

This is the sender. Don’t forget to place a picture in the data directory of the sketch.
Save this as sender.pde

// Send Images over UDP 
import java.net.*;
import javax.imageio.*;
import java.awt.image.*; 
import java.io.*;

// This is the port we are sending to
int clientPort = 11111; 
// This is our object that sends UDP out
DatagramSocket ds; 
// This is the image we are sending
PImage photo;

void setup() {
  size(320,240);
  photo = loadImage("data//space_shuttle.jpg");
  
   // Setting up the DatagramSocket, requires try/catch
  try {
    ds = new DatagramSocket();
  } 
  catch (SocketException e) {
    e.printStackTrace();
  }
  
  noLoop();
}

void draw() {
  image(photo, 0, 0);
  broadcast(photo);  
}


// Many Thanks to Daniel Shiffmann for the initial idea 
// See https://shiffman.net/processing.org/udp/2010/11/13/streaming-video-with-udp-in-processing/

// Function to broadcast a PImage over UDP
// Special thanks to: http://ubaa.net/shared/processing/udp/
// (This example doesn't use the library, but you can!)

void broadcast(PImage img) {

  // We need a buffered image to do the JPG encoding
  BufferedImage bimg = new BufferedImage( img.width,img.height, BufferedImage.TYPE_INT_RGB );

  // Transfer pixels from localFrame to the BufferedImage
  img.loadPixels();
  bimg.setRGB( 0, 0, img.width, img.height, img.pixels, 0, img.width);
  // Need these output streams to get image as bytes for UDP communication
  ByteArrayOutputStream baStream  = new ByteArrayOutputStream();
  BufferedOutputStream bos    = new BufferedOutputStream(baStream);

  // Turn the BufferedImage into a JPG and put it in the BufferedOutputStream
  // Requires try/catch  
  try {
    ImageIO.write(bimg, "jpg", bos);   
  } 
  catch (IOException e) {
    e.printStackTrace();
  }
 // Get the byte array, which we will send out via UDP!
  byte[] packet = baStream.toByteArray();
  // Send JPEG data as a datagram
  // println("Sending datagram with " + packet.length + " bytes");
  try {
    ds.send(new DatagramPacket(packet,packet.length, InetAddress.getByName("localhost"),clientPort)); 
} 
  catch (Exception e) {
    e.printStackTrace();
  }
}type or paste code here

And the receiver below.
Save as receiver.pde. Run both sketeches and it will work. More work is required if you place sender and receiver in different system.


// https://shiffman.net/processing.org/udp/2010/11/13/streaming-video-with-udp-in-processing/

import java.awt.image.*; 
import javax.imageio.*;
import java.net.*;
import java.io.*;

// Port we are receiving.
int port = 11111; 
DatagramSocket ds; 
// A byte array to read into (max size of 65536, could be smaller)
byte[] buffer = new byte[65536]; 

PImage video;

void setup() {
  size(400,300);
  try {
    ds = new DatagramSocket(port);
  } catch (SocketException e) {
    e.printStackTrace();
  } 
  video = createImage(320,240,RGB);
}

 void draw() {
  // checkForImage() is blocking, stay tuned for threaded example!
  checkForImage();

  // Draw the image
  background(0);
  imageMode(CENTER);
  image(video,width/2,height/2);
}

void checkForImage() {
  DatagramPacket p = new DatagramPacket(buffer, buffer.length); 
  try {
    ds.receive(p);
  } catch (IOException e) {
    e.printStackTrace();
  } 
  byte[] data = p.getData();

  println("Received datagram with " + data.length + " bytes." );

  // Read incoming data into a ByteArrayInputStream
  ByteArrayInputStream bais = new ByteArrayInputStream( data );

  // We need to unpack JPG and put it in the PImage video
  video.loadPixels();
  try {
    // Make a BufferedImage out of the incoming bytes
    BufferedImage img = ImageIO.read(bais);
    // Put the pixels into the video PImage
    img.getRGB(0, 0, video.width, video.height, video.pixels, 0, video.width);
  } catch (Exception e) {
    e.printStackTrace();
  }
  // Update the PImage pixels
  video.updatePixels();
}
1 Like
11

Find an update from my code on my github account -> see https://github.com/f41ardu/imagestreaming
Max size of datagram using udp is restricted to 65536. So I’ve split larger images to a chunk size of 1460 bytes or lower and append them on the receive side. So now we can send larger images and videos.
So that a major step towards my ongoing investigation to decode Tello h264 videostream using Processing ;->

3 Likes