Augmented Reality made easy (?)

Processing for Android
1

Hi, I’m trying to get fiduciary marker detection working in Processing on Android and I’m having library compatibility issues…

From what I’ve found, the available toolkits (PapARt and nyARToolkit) require Processing’s core video library which is not supported in Android mode (or APDE). The openCV library has the same problem.
(I’ve got marker detection working with nyARToolkit and openCv on windows/mac).

I found a processing-video-android library, but even the example sketches just crash the app. I assume it’s not up to date.

The Ketai library get’s the camera in and can track faces by wrapping Android’ face detector. But can it track markers?

Have you had success where I haven’t yet? Is there another library I’m missing?

2

@Jay ===

I have successfully used nyartoolkit with android but it was with AS; yet i see here that somebody seems to have used it with P5 and android mode+ketai, though i am not sure that some of the imports are working (nfc): https://www.mns.kyutech.ac.jp/~hanazawa/education/AR/AR1/

3

Hi, thanks for your reply. I have read about implementations with AS or Eclipse, but I don’t have familiarity with those.

The Hanazawa ketai/nyARToolkit sketch you linked is similar to one I’ve been working on, adapting a nyARToolkit example sketch to use KetaiCamera. I’ve just managed to get it working on Android 8!

There were filepath problems with instantiating the MultiMarker object. I got around it by copying the camera_para.dat file from the library data folder to the sketch folder, along with the patt.hiro and patt.kanji marker files. (I guess this is what you meant by imports not working).
I’m still getting OpenGL errors from the P3D renderer, but they don’t stop the app from building:

OpenGL error 1280 at bot beginDraw(): invalid enum
OpenGL error 1280 at bot endDraw(): invalid enum

There are still some issues to work out. (It’s not running for me on Android 6 yet). But here’s the full working code:


/**
 * NyARToolkit for proce55ing/3.0.5
 * (c)2008-2017 nyatla
 * airmail(at)ebony.plala.or.jp
 * 
 * This sketch handles 2 coordinate system in same time.(left and right).
 * Each markers show different coordinate system.
 * The marker is "patt.hiro" and "patt.kanji"
 * Any pattern and configuration files are found in libraries/nyar4psg/data inside your sketchbook folder. 
 */
 
/**
 * Modified by J. Kelly - MMT TCD (2019)
 * Reworked the example "Rotation.pde" to use the Ketai library for Android.
 * The following files should be copied to the sketch's data folder:
 *                                                                   - camera_para.dat
 *                                                                   - patt.hiro
 *                                                                   - patt.kanji
 */
 
import ketai.camera.*;
import jp.nyatla.nyar4psg.*;

KetaiCamera cam;
MultiMarker nya_r;
MultiMarker nya_l;
PFont font;

void setup() {
  size(960,720, P3D);

  orientation(LANDSCAPE);
  font=createFont("FFScala", 32);
  colorMode(RGB, 100);
  //println(MultiMarker.VERSION);
  
  cam = new KetaiCamera(this,960,720, 24);
  nya_l=new MultiMarker(this,width,height,"camera_para.dat",new NyAR4PsgConfig(NyAR4PsgConfig.CS_LEFT_HAND,NyAR4PsgConfig.TM_NYARTK));
  nya_l.addARMarker("patt.hiro",80);
  
  nya_r=new MultiMarker(this,width,height,"camera_para.dat",new NyAR4PsgConfig(NyAR4PsgConfig.CS_RIGHT_HAND,NyAR4PsgConfig.TM_NYARTK));
  nya_r.addARMarker("patt.kanji",80);
  cam.start();
}

int c=0;
void drawgrid()
{
  pushMatrix();
  stroke(0);
  strokeWeight(2);
  line(0,0,0,100,0,0);
  textFont(font,20.0); text("X",100,0,0);
  line(0,0,0,0,100,0);
  textFont(font,20.0); text("Y",0,100,0);
  line(0,0,0,0,0,100);
  textFont(font,20.0); text("Z",0,0,100);
  popMatrix();
}
void draw()
{
  c++;
  cam.read();
  nya_r.detect(cam);
  nya_l.detect(cam);
  background(0);
  nya_r.drawBackground(cam);

  //right
  if((nya_r.isExist(0))){
    nya_r.beginTransform(0);
    fill(0,0,255);
    drawgrid();
    translate(0,0,20);
    rotate((float)c/100);
    box(40);
    nya_r.endTransform();
  }
  //left
  if((nya_l.isExist(0))){
    nya_l.beginTransform(0);
    fill(0,255,0);
    drawgrid();
    translate(0,0,20);
    rotate((float)c/100);
    box(40);
    nya_l.endTransform();
  }
}