Unicode and Smart quotes not being drawn properly

Processing Coding Questions
1

I am trying to display text entries by drawing them in my sketch. All of the text I will be using has quotation marks, and the source I am getting them from has them formatted as “curly” or “smart” quotes.
However, when I loadStrings() from a text file formatted in unicode, and attempt to draw it to the canvas, I get missing characters between each letter:

Capture
Capture1830×944 6 KB

Even when I save the .txt file with ANSI encoding, the smart quotes don’t get drawn properly:

Capture1
Capture1744×206 2.95 KB

I have tried with multiple fonts, ones that I know support curly quotes. Is there a way to force Processing to treat Unicode text as ascii or ansi? Is there another fix for this? thanks :slight_smile:

String[] s;

void setup(){
  size(800, 800);
  fill(0);
  textSize(35);
  textAlign(CENTER, CENTER);
  
  s = loadStrings("testFile.txt");
}

void draw(){
  background(255);
  text(s[0], width / 2, height / 2);
  noLoop();
}

the testFile.txt file contains the following text:

The use of the verb “to fall” (tipol)

I have encoded both as Unicode and ANSI in different trials of my program.

2

Docs.Oracle.com/en/java/javase/11/docs/api/java.base/java/lang/String.html#replace(char,char)

3

Just tried this using both possible character notations:

s[0] = s[0].replace('\u201D', '\"');
 s[0] = s[0].replace('“', '\"');

Still the exact same results :confused: Am I using the method incorrectly?

4

Update: When I don’t read the strings from a file, and instead just initialize the String within the program, it works fine (even with using smart quotes)

String s;

void setup(){
  size(800, 800);
  fill(0);
  textSize(35);
  textAlign(CENTER, CENTER);
  
  s = "The use of the verb “to fall” (tipol)";
  //s = s.replace('\u201D', '\"');
  //s = s.replace('“', '\"');
  println(s);
}

void draw(){
  background(255);
  text(s, width / 2, height / 2);
  noLoop();
}

Capture
Any idea why it’s different when reading from file? And suggestion to how I’d deal with this?

5

Starting with Processing release 0134, all files loaded and saved by the Processing API use UTF-8 encoding. In previous releases, the default encoding for your platform was used, which causes problems when files are moved to other platforms.

LoadStrings_ANSI___UNICODE

/**
 * LoadStrings ANSI & UNICODE (v1.0.1)
 * GoToLoop (2019/Aug/06)
 *
 * https://Discourse.Processing.org/t/
 * unicode-and-smart-quotes-not-being-drawn-properly/13213/5
 */

static final String UTF16 = "The use of the verb “to fall” (tipol)";

static final String FILE_ANSI = "testFile.ansi.txt";
static final String FILE_UTF8 = "testFile.utf8.txt";

static final int FONT_SIZE = 030;

String ansi, utf8;

void setup() {
  size(600, 200);
  noLoop();

  colorMode(RGB);
  fill(#FFFF00);

  textMode(MODEL);
  textAlign(CENTER, CENTER);
  textSize(FONT_SIZE);

  ansi = loadStrings(FILE_ANSI)[0];
  utf8 = loadStrings(FILE_UTF8)[0];

  println(UTF16);
  println(int(UTF16.toCharArray())); // [20]: 8220, [28]: 8221

  println(ENTER + ansi);
  println(int(ansi.toCharArray()));  // [20]: 65533, [28]: 65533

  println(ENTER + utf8);
  println(int(utf8.toCharArray()));  // [20]: 8220, [28]: 8221
}

void draw() {
  background(#0000FF);

  final int cx = width >> 1, cy = height >> 1, qy = height >> 2;
  text(UTF16, cx, qy);  // top
  text(ansi, cx, cy);   // middle
  text(utf8, cx, 3*qy); // bottom
}