IPC shared memory

Processing Libraries
1

Was looking for IPC shared memory examples for processing but didnt find any. Anyone used shared memory in processing or have a guide/ example for it?

Thanks

2

You can look up Java Memory Mapped Files. Even very high level Java programmers misunderstand those. I think because they don’t understand how operating systems actually handle paged memory.

I did some experiments and they are as fast as array memory. The assumption from people not understanding the OS too well is file acess=very slow. And also the clunky method access looks like it would be slow. However the hotspot compiler just optimizes away the method call so you are just left with a standard DRAM memory access.

3

Maybe something like this, not saying I’m an expert.

import java.io.File;
import java.io.RandomAccessFile;
import java.nio.*;
import java.nio.channels.FileChannel;
import java.nio.file.Files;
void setup() {
  int elements=25*1024*1024; // 100Mb of floats (4 bytes per float)
  try {
    File temp = Files.createTempFile("float-mmap", ".bin").toFile();
    temp.deleteOnExit();
    RandomAccessFile raf = new RandomAccessFile(temp, "rw");
    FileChannel fc = raf.getChannel();
    raf.setLength(elements*4); // ensure file size (4 bytes for each float)
    MappedByteBuffer mbb = fc.map(FileChannel.MapMode.READ_WRITE, 0, elements*4);
    // make sure native order
    mbb.order(ByteOrder.nativeOrder());// Otherwise data in network order = slow
    FloatBuffer mapped = mbb.asFloatBuffer();

    // Fill mapped buffer data
    double sum = 0; // Use double to reduce round off error
    double sumMapped = 0; // Use double to reduce round off error
    for (int i = 0; i < elements; i++){
      sum+=i*1f;
      mapped.put(i, i * 1.0f);
    }
    for (int i = 0; i < mapped.capacity(); i++){
      sumMapped += mapped.get(i);
    }
    println(sum+" Mapped:"+sumMapped);
    raf.close();
    temp.delete();
  }catch(IOException e) {
    println(e);
  }
}

You can probably do better with the exception handling than I did.

Anyway you would have both processes using the same file name, and then they get to share paged memory together, with the OS using the file system for back-up if it doesn’t have any free memory pages. You can also use it to save data, I think you use the force method to make sure the OS has written every page to its backing file position.

4

At one stage I though I would need to handle massive memory for neural networks. That’s why I looked into memory mapped file.

In fact the neural networks I made have very small memory requirements.