3D Lidar point cloud to Blender

please format code with </> button * homework policy * asking questions

Hi Everyone,

I want to scan in 3D and mill these with CAM.
I have access to some nice tools to do so:

  • Livox Mid-40 LiDAR
  • Fusion 360 and its CAM tools
  • production CNC lathe with milling tools

My idea is to do following steps:

  • get .csv from the lidar with its point cloud
  • calculate a height map from these points
  • use blender with the displacement on a plane to create a 3d model (soft modelled)
  • import the soft model into Fusion 360 and generate CAM toolpaths
  • mill it on the CNC

What i achieved so far:

Code for processing:

ArrayList<scanPoint>scanPoints = new ArrayList<scanPoint>();

float minX = 0;
float maxX = 0;
float minY = 0;
float maxY = 0;
float minZ = 0;
float maxZ = 0;

float overrideMaxZValue = 1.8;
boolean overrideMaxZ = false;

int counter = 0;
int calculatedPixels = 0;
int calculationStarted = 0;

PImage heightmap;

boolean calculating = false;

void setup(){
  size(500,500,P2D);
  heightmap = new PImage(width,height);
  parseFile();
}

void draw(){
  if(!calculating){
    image(heightmap,0,0);
  }
  else{
    background(0);
    text("Calculated pixels: " + calculatedPixels + " of: "+ (width*height),10,10);
    float percentage = (float(calculatedPixels)/( float(width) *float(height)))*100;
    text(nf(percentage,0,3) + "%",10,30);
    float seconds = (millis()-calculationStarted)/1000;
    float minutes = seconds/60;
    float hours = minutes/60;
    text("Time since calculation start: "+int(hours) + ":"+int(minutes%60)+ ":"+int(seconds)%60,10,50);
    float eSeconds = (100/percentage)*(millis()-calculationStarted)/1000;
    float eMinutes = eSeconds/60;
    float eHours = eMinutes/60;
    text("Estimation: "+int(eHours) + ":"+int(eMinutes%60)+ ":"+int(eSeconds)%60,10,70);
  }
}

void mouseClicked(){
  if(mouseX > 0 && mouseX < width && mouseY > 0 && mouseY < height && mouseButton == LEFT){
    calculating = true;
    calculationStarted = millis();
    thread("drawHeightmapFilled");
  }
  else if(mouseButton == RIGHT){
    heightmap.save("C:/Users/***/Desktop/Privat/heightmaps/heightmap from livox scan.png");
    println("saved heightmap");
  }
}

void drawHeightmapSimple(){
  println("start drawing heightmap");
  for(int i = 0; i<scanPoints.size();i++){
    scanPoint sp = scanPoints.get(i);
    float pX = map(sp.pX,minX,maxX,width,0);
    float pY = map(sp.pY,minY,maxY,height,0);
    int whiteValue = int(map(sp.pZ,minZ,maxZ,255,0));
    color pZ = color(whiteValue,whiteValue,whiteValue);
    heightmap.set(int(pX),int(pY),pZ);
    calculatedPixels ++;
  }
  println("done drawing heightmap");
  calculating = false;
}

void drawHeightmapFilled(){
  println("start drawing heightmap");
  for(int X = 0; X<width;X++){
    for(int Y = 0; Y<height;Y++){
      float distance = 9000;
      color pZ = color(0,0,0);
      for(int i = 0; i<scanPoints.size();i++){
        scanPoint sp = scanPoints.get(i);
        float pX = map(sp.pX,minX,maxX,width,0);
        float pY = map(sp.pY,minY,maxY,height,0);
        float bufferDistance = dist(X,Y,pX,pY);
        if(bufferDistance<distance){
          distance = bufferDistance;
          if(sp.pZ < minZ || sp.pZ > maxZ){
            pZ = color(255,0,0);
          }
          else{
            int whiteValue = int(map(sp.pZ,minZ,maxZ,255,0));
            pZ = color(whiteValue,whiteValue,whiteValue);
          }
        }
      }
      heightmap.set(X,Y,pZ);
      calculatedPixels ++;
    }
  }
  println("done drawing heightmap");
  calculating = false;
}

void parseFile() {
  BufferedReader reader = createReader("C:/Users/***/Desktop/Privat/heightmaps/kopfscan.csv");
  String line = null;
  try {
    while ((line = reader.readLine()) != null) {
      String[] coord = split(line, ",");
      float Xa = stringToFloat(coord[14]);
      float Ya = stringToFloat(coord[15]);
      float Za = stringToFloat(coord[13]);
      float Z = dist(0,0,0,Xa,Ya,Za);
      float X = tan(Xa/Za);
      float Y = tan(Ya/Za);
      
      if(X>maxX)maxX = X; else if(X<minX) minX = X; 
      if(Y>maxY)maxY = Y; else if(Y<minY) minY = Y; 
      if(Z>maxZ)maxZ = Z; else if(Z<minZ) minZ = Z; 
      if(counter == 1){
        minX = X;
        maxX = X;
        minY = Y;
        maxY = Y;
        minZ = Z;
        maxZ = Z;
      }
      if(!Float.isNaN(X) && !Float.isNaN(Y) && !Float.isNaN(Z)){
        scanPoints.add(new scanPoint(X,Y,Z));
      }
      counter++;
    }
    reader.close();
  } catch (IOException e) {
    e.printStackTrace();
  }
  if(overrideMaxZ) maxZ = overrideMaxZValue;
  println("maxX: "+maxX+" ||minX: "+minX);
  println("maxY: "+maxY+" ||minY: "+minY);
  println("maxZ: "+maxZ+" ||minZ: "+minZ);
  println("number of points: "+counter);
}

class scanPoint{
  float pX = 0;
  float pY = 0;
  float pZ = 0;
  
  scanPoint(float x, float y, float z){
    pX = x;
    pY = y;
    pZ = z;
  }
}

Float stringToFloat(String str){
  float buffer = 0;
  buffer = float(str);
  if(Float.isNaN(buffer)){
    if(str.equals("true")){
      buffer = 1;
    }
    else{
      buffer = 0;
    }
  }
  return buffer;
}

I have got two problems with my code:

  1. I do have a curvature error and i think it comes from the perspective view which i calculated in the parseFile() function. A Orthographic view would make more sense for the displacement in blender but then i do have point shadows and overlapping geometry.
  2. It takes a long time to calculate the heightmap. A scan has ~300k points. I calculate for each pixel the distance to the nearest point and save its Z value. An image of 500x500 takes about 15 minutes. I seperated it into a second thread to show its progress in the draw function. Do i have to seperate the image into even more threads or would CUDA make sense?

I would appreciate any suggestion or comment.

Kind regards,
StainlessHolz

Hi

This is project close to your project

His project is the step before my project. He scans objects and creates a point cloud (which in itself is impressive). My project is to use this point cloud to generate geometry and use it to fabricate something.

Yes I have noticed but I mean it’s simple scanner than yours

My scanner is not the problem. It is very easy to use and has great results.
The problem is how to get a mesh from the point cloud.

Just general idea

I have made this scanner

And i used meshlab software to convert point cloud to STL file

Then I fabricated it with my my cnc


1 Like

meshlab seems to be what i need, i will have a look into it.
Thanks, i will update if i have more info.

Ive made some progress.
Meshlab can make 3D models out of point clouds, but either the mesh is really big (face count) or i loose details with point redistribution, also it takes a long time with each try.
With the way my LiDar works i do have a high point density in the middle axis of the scan and it decreases the further it moves out of the middle.

I decided to go on with the heightmap/displacement method.
Ive added a way faster method of parsing a file:

  1. Adding all Points to their corresponding Pixels and averaging them
  2. For all empty pixel search their neighbor Pixels for heights and averaging them

Thats way faster and better than comparing every pixel with every point since i dont only get one point per pixel and dont have to do 75Billion for a 500x500 heighmap with 300k points.

this is the 3d model i get out of this method:


This is the heightmap for this 3D model:

For anyone interested in the code (too long for a comment):

regards,
StainlessHolz

edit: forgot to mention that the curvature error was 40cm before the monitor :grinning_face_with_smiling_eyes: . I have just to use the original XYZ values and ignore overlapping geometry.

1 Like