Toxiclibs + GPU (openGL) processing 3


#1

Hi! Does anyone know how to create surfaces with toxiclibs using low level OpenGL functions? i’ve seen documentation but only for processing 2 or below, but can’t figure out how to make it work.

with big surfaces the frameRate is low so i want to gain speed performance through the GPU.


#2

Hi :slight_smile: there are two parts in what you are trying to do I think. One is generating meshes. Two is to rendering meshes. (And step three might be moving or deforming the mesh)

The thing that slows down frame rate is sending lots of data to the GPU on every single frame.

By keeping the generated mesh data in the GPU the rendering is much faster than when you send a new mesh to the GPU on every frame. If you are generating a heavy new mesh on every frame it will not be fast, even if you use low level OpenGL. But you can still deform big meshes in the GPU with shaders, which can be fast because you don’t need to upload all the vertices again and again, only a few values to deform what you have.

What are you trying to do? Do you have an image to share?


#3

Hi again! And I don’t want to deform the surface, only rendering it, and manipulate it (like rotation, scaling)

but since they are from a big dataset, they’re made from a lot of triangular faces wich slows down the interactivity.


#4

For faster rendering it’s important to keep the shape in the GPU. I don’t remember right now if ToxicLibs was updated to do that. When I used it it was not the case, so I wrote some simple code to convert the mesh from ToxicLibs to PShape. PShape shapes are kept in the GPU.

Do you know how many vertices your shapes have? And what kind of GPU you use?

Here some incomplete Java code that may be useful as a starting point:

public static void toxicToPShape(final TriangleMesh m, final PShape shp) {
                List<Face> triangles = m.getFaces();

                shp.beginShape(PConstants.TRIANGLES);
                shp.textureMode(PConstants.NORMAL);

                for (final Face t : triangles) {
                        addFaceToPShape(shp, t.a, t.b, t.c);
                }
                shp.endShape();
                shp.setSpecular(0xFFFFFFFF);
                shp.setShininess(50); // * (float) Math.random()
        }
        public static void addFaceToPShape(PShape shp, Vec3D a, Vec3D b, Vec3D c,
                        Vec2D uva, Vec2D uvb, Vec2D uvc) {
                shp.vertex(a.x, a.y, a.z, uva.x, uva.y);
                shp.vertex(b.x, b.y, b.z, uvb.x, uvb.y);
                shp.vertex(c.x, c.y, c.z, uvc.x, uvc.y);
        }                          
        
        public static void addFaceToPShape(PShape shp, Vec3D a, Vec3D b, Vec3D c) {
                shp.vertex(a.x, a.y, a.z);
                shp.vertex(b.x, b.y, b.z);
                shp.vertex(c.x, c.y, c.z);
        }

        public static void setFaceInPShape(PShape shp, int index, Vec3D a, Vec3D b,
                        Vec3D c) {
                shp.setVertex(index, a.x, a.y, a.z);
                shp.setVertex(index + 1, b.x, b.y, b.z);
                shp.setVertex(index + 2, c.x, c.y, c.z);
        }

Maybe simpler would be to export your shapes from ToxicLibs and load them as PShape, if generating them on the fly is not important.


#5

Yeah :slight_smile: this is what I want to do, I don’t need to genereted it on the fly, so I’ll do the last thing that you mentioned.

I’ll see your tutorials at funprogramming.org I think there’s what I’m looking for.

And about the specs of my laptop i have an intel i5 processor and an MD Radeon™ R7 M360, the number of vertices of the three volumes are around the million.


#6

So after a little research on your tutorials, i have a little question, can i export the volumes from toxiclibs to an obj. file and then, pass that obj. file into a PShape? or this will not have the desire effect?

Thanks for all your help!


#7

If you look at the rendering code in ToxicLibs what you see is that each time you render the mesh it sends all the vertices and normals one by one to the GPU, which is slow.

So you could try the approach you suggest. Here’s an example of exporting a ToxicLibs mesh to obj.
Then you could load the shapes with https://processing.org/reference/loadShape_.html and render them.

If you want even better frameRate, add one extra middle step: simplify the meshes with http://www.meshlab.net/ (or even Blender if you know how to use it). You might be able to reduce the vertex counts greatly (to 10% ~ 50%) without much noticeable effects and make it easier to achieve higher frame rates.


#8

A function that could be used to convert ToxicLibs meshes to PShape:

PShape toxicToPShapeSmooth(final Mesh3D m) {
    m.computeVertexNormals();
    Collection<Face> triangles = m.getFaces();

    shp = createShape();
    shp.setStroke(false);
    shp.beginShape(PConstants.TRIANGLES);
    shp.textureMode(PConstants.NORMAL);

    for (final Face t : triangles) {
        shp.normal(t.a.normal.x, t.a.normal.y, t.a.normal.z);
        shp.vertex(t.a.x, t.a.y, t.a.z, t.uvA.x, t.uvA.y);

        shp.normal(t.b.normal.x, t.b.normal.y, t.b.normal.z);
        shp.vertex(t.b.x, t.b.y, t.b.z, t.uvB.x, t.uvB.y);

        shp.normal(t.c.normal.x, t.c.normal.y, t.c.normal.z);
        shp.vertex(t.c.x, t.c.y, t.c.z, t.uvC.x, t.uvC.y);
    }
    shp.endShape();
    shp.setSpecular(0xFFFFFFFF);
    shp.setShininess(50);
}

#9

Hey! many thanks! i tried the option to export it into an obj file an then to a pshape, but its to slow and gets errors like “java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead limit exceeded” So i think the best option is to use your function toxicToPShapeSmooth but, as your functions that you implemented in java, i don’t know how to use the List or Collection in processing, i was trying to change it for something like arrayList but without success. i’ll be trying that :slight_smile:


#10

Hi Abe, many thanks for your help and guidance. Now it works and its impressive how using the PShape class helps to improve the frameRate. Processing didn’t recognized the Collection variable, even the import java.util.Collections;but it works with the import java.util.List; so using the codes that mentioned before, i used the List, and in order to use this, i have to modify Mesh3D with TriangleMesh, and convert the input data by casting.

Also, i was getting NullPointerException error in the shp.vertex(t.a.x, t.a.y, t.a.z, t.uvA.x, t.uvA.y); line, so i deleted the arguments t.uvA.x, t.uvA.y because i don’t know their function (i’m still curious about it).


PShape toxicToPShapeSmooth(final TriangleMesh m) {
    m.computeVertexNormals();
    
   // Collection<Face> triangles = m.getFaces();
    List<Face> triangles = m.getFaces();
    PShape shp;
    shp = createShape();
    shp.setStroke(false);
    shp.beginShape(PConstants.TRIANGLES);
    shp.textureMode(PConstants.NORMAL);

    for (final Face t : triangles) {
        shp.normal(t.a.normal.x, t.a.normal.y, t.a.normal.z);
        shp.vertex(t.a.x, t.a.y, t.a.z);

        shp.normal(t.b.normal.x, t.b.normal.y, t.b.normal.z);
        shp.vertex(t.b.x, t.b.y, t.b.z);

        shp.normal(t.c.normal.x, t.c.normal.y, t.c.normal.z);
        shp.vertex(t.c.x, t.c.y, t.c.z);
    }
    shp.endShape();
    shp.setSpecular(0xFFFFFFFF);
    shp.setShininess(50);
    return shp;
}

Thanks again!


#11

It’s not a variable but an interface. Classes & interfaces act as datatypes too: :face_with_monocle:
Docs.Oracle.com/en/java/javase/11/docs/api/java.base/java/util/Collection.html

The class Collections isn’t the same as the interface Collection: :expressionless:
Docs.Oracle.com/en/java/javase/11/docs/api/java.base/java/util/Collections.html

For the interface Collection, it’s this import statement instead: :innocent:
import java.util.Collection;

It’d be better if @hamoid’s code had included all the needed import statements as well. :stuck_out_tongue_winking_eye:


#12

I’m glad it helped and that you were able to figure out the rest :slight_smile:

Sorry about not including the imports. I use the IntelliJ Idea IDE where I just press Alt+Enter and it adds the imports automatically. It even suggests improvements to the code :slight_smile:

The vertices may contain between 1 and 3 parts:

  • The actual vertex position. A triangle has 3 vertices (t.a, t.b, t.c) and each vertex has .x, .y and .z.
  • The normal. That’s a vector pointing outwards from the vertex, used for light calculations.
  • The texture coordinate. The triangle may have 3 texture coordinates (t.uvA, t.uvB, t.uvC, one for each vertex), which are 2D points representing a position on an image, also known as texture.

Probably your model has no uv mapping (textures) so it was giving you an NPE.

If you like “low poly” look you can replace the first line in the function by m.computeFaceNormals(); (or just comment out the existing line), which will make the three normals in a face equal, which basically disables smoothing.

How much did the frame rate increase?


#13

Very good, I’ll test it later, and see that IDE. And it increased it from 0.5 fps to 25/30 fps, my gpu is cheap so presumably it’ll go higher in another one.
many many thanks for your help


#14

Using IDE’s like Eclipse or IntelliJ Idea helps a lot with development of projects containing multiple files and libraries. Those IDEs make it very easy to move code around, rename classes and variables, re-use code across projects, include amazing tooltips, autocomplete and suggestions to make your code better.

BUT the code needs to be adapted. For prototyping simple ideas I still use the Processing IDE (one-file experiments). But for anything that has 2 or more classes, I switch to a different IDE. The main things that need to be adapted are: float numbers need an f behind them (3.0f), colors can not be specified using the # character (use 0xFFFFFFFF, not #FFFFFF), and the scope in your classes is not the PApplet, which means you can not just call line() in your class, you need to pass the scope into the class, and then do something like p5.line(...).

These changes feel like an overkill for small projects and for people with less experience. But for medium to large projects I think the extra effort is totally worth it.


#15

We can still choose to place them all as inner classes inside the PApplet subclass. :crazy_face:

However, everything would be inside the same “.java” file. :coffee:

Which of course, defeats the purpose of using a pro IDE. :flushed: