Trying to make my own version of the "quick draw" game to build a library and teach an AI as a BEGINNER

[anntie]

Hello! I am working on a design project about emotive writing. I would like to teach an AI to detect “emotions” from simple hand-drawn lines, and maybe use a GAN to generate lines according to specified emotional categories.
To do that I first need to collect a lot of Images of lines that represent certain emotions, and I would like to do that via processing. My idea is to create a emotive version of the google quick draw game:
The app randomly selects an emotional category, and the user has one try to draw a line, a letter, a sign, whatever. The drawing has to be stored as a file, with information about which category it belongs to, before the next category is set.

edit: 06.06
Thank you for your help so far, I’m super happy to see that the code is coming together and that it’s working!! I thought it might be helpful to have a draft of how the interface should look like. I noticed that there needs to be a little break between the drawing sessions so the line you’re about to draw doesn’t accidentally end up in the next category. So maybe it’s possible to lock the screen for 1-2 seconds, with a little announcement that the next category will start when you’re ready. This also prevents that empty frames are saved to the library - the timer starts only when you start to draw, and the frame is automatically saved when time’s up or the enter key is pressed. I need a lot of images, so it would be nice to have the program just loop over and over again, but to give the person who is drawing a sense of progress i’d like to show a little counter in the corner, that says how many of the 28 categories are still open. The categories repeat randomly after completion, and everytime the program starts. That way, even if the session ends before all categories have been drawn, there should be an even amount of images for all categories. Here’s a little walkthrough.

Do you think I could run this on my website and store the images in a drive folder or something? If I limit the total amount of images?
The code is still a work in progress, if you’re interested have a look in the comments how it’s going, always glad about feedback and help

[anntie]

  float prevMouseX = 0;
  float prevMouseY = 0;
  
            //emotional Lexicon
            String a = "amused";
            String b = "joyful, cheerful";
            String c = "energized, pumped up";
            String d = "annoyed";
            String e = "excited";
            String f = "anxious, tense";
            String g = "dreamy";
            String h = "creeped out, scared";
            String i = "disgusted, revolted";    
            String j = "angry";
            String k = "amazed, fascinated"; 
            String l = "calm, realxed";  
            String m = "bored";  
            String n = "sad";  
            String o = "flirty";  
            String p = "proud, strong"; 
            String q = "romantic, loving"; 
            String r = "confrontational, argumentative";
            String s = "erotic, desirous";
            String t = "tender, longing";
            String u = "triumphant, heroic";
            String v = "bittersweet, melancholic";
            String w = "euphoric, ecstatic";
            String x = "disappointed";
            String y = "confused";
            String z = "remorseful, sorry";
            String A = "silly, goofy";
            String B = "jealous";
            String C = "sympathetic, pitiful";

void setup(){
  //run one time as soon as the program starts 
  size(640, 360);
  background(250);
  
  F = loadFont("GTFAdieuTRIAL-Regular-48.vlw");
  
  surface.setTitle("Emotive Drawing");
  surface.setResizable(true);
  surface.setLocation(100, 100);
}

void draw() {
  stroke (50);
  strokeWeight (10);
  //line(pmouseX,pmouseY,mouseX,mouseY);
}

void mousePressed(){
  prevMouseX = mouseX;
  prevMouseY = mouseY;
}

void mouseDragged(){
  fill(0);  
  stroke(0);
  float distanceX;
  float distanceY;
  
  distanceX = abs(mouseX - prevMouseX);
  distanceY = abs(mouseY - prevMouseY);
  
  float avDistance;
  avDistance = (distanceX + distanceY)/2;
  //set range for strokeWeight: squash 0-150 (input range) into 1-50 (output range) 
  avDistance = map(avDistance,0,150,1,50);
  
  //ellipse(mouseX, mouseY, 10, 10);
  //ellipse(mouseX, mouseY, avDistance, avDistance);
  
  prevMouseX = mouseX;
  prevMouseY = mouseY;
  
  //line(pmouseX,pmouseY,mouseX,mouseY);
  strokeWeight(avDistance);
  line(pmouseX,pmouseY, prevMouseX, prevMouseY);
  
}

void keyPressed(){
  saveFrame("drawings/emotion###.png");
  background(250);
  //text (string name, x-position, y-position)
  textSize (64);
  textFont(F);
  fill(0);
  stroke(0);
  text (a, 50, 50);
}

[anntie]

This is my code so far. Right now I have so many questions
Problem 1 How do I tell processing to select and show only one emotional category? I put them as separate strings just to have them listed somehow, but idk what i’m doing tbh
Problem 2 The Image needs to be saved with the name of the randomly selected category - how?
Problem 3 I need a way to show the category, without having it appear in the saved image. Maybe it would be good to allow the user to only draw in a certain area? or should I make the text disappear before saving?
If you have a bit more experience with processing (or any at all, I had a class in school and never really got around to learn more and now I’m sorry for having advanced plans with a total lack of skill ) maybe you have some ideas how to do this, or where i should look for reference and help. Thank you for reading, and thank you for your advice or feedback

[Chrisir]

@anntie

The drawing has to be stored as a file, with information about which category it belongs

Okay, you can use save() to store the image as a file. The file name is the category.

Check out arrays PImage listImages = new PImage [20];

Use this instead of single images a b c

[debxyz]

@anntie

Problem 1 How do I tell processing to select and show only one emotional category?

Perhaps using a switch statement for this part?

[anntie]

helloo, thanks for the tip! I found an example of a switch statement here
and tried to apply it to my code like this

int category = int(random(7)); switch (category) { case 1: System.out.println("dreamy"); break; case 2: System.out.println("amused"); break; case 3: System.out.println("joyful, cheerful"); break; case 4: System.out.println("energized, pumped up"); break; case 5: System.out.println("annoyed"); break; case 6: System.out.println("excited"); break; case 7: System.out.println("anxious, tense"); break; }

[anntie]

https://www.ocf.berkeley.edu/~acowen/music.html#
https://s3-us-west-1.amazonaws.com/vocs/map.html#

I got a lot of inspiration and most of the “emotional lexicon” from these projects btw, really interesting and really fun to play around with

[anntie]

Soo… I solved the random naming with a simple array, I think it works for what I want to do?

  float prevMouseX = 0;
  float prevMouseY = 0;
  
  PFont F;

  String[] words = { "amused", "joyful, cheerful", "energized, pumped up", "annoyed", "excited","anxious, tense", "dreamy", "creeped out, scared", "disgusted, revolted", "angry", "amazed, fascinated", "calm, realxed", "bored", "sad", "flirty", "proud, couragous", "romantic, loving", "confrontational, argumentative", "tender, longing", "triumphant, heroic", "bittersweet, melancholic", "euphoric, ecstatic", "disappointed", "confused", "remorseful, sorry", "silly, goofy", "jealous", "sympathetic, pitiful"};
  int index = int(random(words.length));

void setup(){
  size(640, 640);
  background(250);
  
  F = loadFont("GTFAdieuTRIAL-Regular-48.vlw");
  
  surface.setTitle("Emotive Drawing");
  surface.setResizable(true);
  surface.setLocation(100, 100);
  
  textSize (40);
  textFont(F);
  fill(0);
  stroke(0);
  //text (string name, x-position, y-position)
  int index = int(random(words.length));
  text (words[index], 20, 50);
  println(words[index]);
}

void draw() {
  stroke (50);
  strokeWeight (10);
  //line(pmouseX,pmouseY,mouseX,mouseY);
}

void mousePressed(){
  prevMouseX = mouseX;
  prevMouseY = mouseY;
}

void mouseDragged(){
  fill(0);  
  stroke(0);
  float distanceX;
  float distanceY;
  
  distanceX = abs(mouseX - prevMouseX);
  distanceY = abs(mouseY - prevMouseY);
  
  float avDistance;
  avDistance = (distanceX + distanceY)/2;
  //set range for strokeWeight: squash 0-200 (input range) into 2-10 (output range) 
  avDistance = map(avDistance,0,150,1,50);
  
  //ellipse(mouseX, mouseY, 10, 10);
  //ellipse(mouseX, mouseY, avDistance, avDistance);
  
  prevMouseX = mouseX;
  prevMouseY = mouseY;
  
  //line(pmouseX,pmouseY,mouseX,mouseY);
  //strokeWeight(avDistance);
  line(pmouseX,pmouseY, prevMouseX, prevMouseY);
  
}

void keyPressed(){
  saveFrame("drawings/emotion###.png");
  background(250);
  
  textSize (40);
  textFont(F);
  fill(0);
  stroke(0);
  //text (string name, x-position, y-position)
  int index = int(random(words.length));
  text (words[index], 20, 50);
  println(words[index]);
}

Now some new questions are
1 can I use a timer instead of keyPressed (ln.66),
so that the program automatically starts again after a while?
2. can I make it so that the frame is only saved if something was actually drawn?
3 still with the naming of the files: can I go with something like

  if (index = "amused") {  
      saveFrame("drawings/amused'###.png");
}

for each category? (obvs not like this because it tells me “cannot convert from int to boolean” )

[Chrisir]

@anntie

saveFrame("drawings/amused'###.png");

saveFrame(“drawings/”+words[index]+“###.png”);

[anntie]

thank you! Tried this without the “+” and thought it’s not possible, now it kind of works… It just doesn’t change the name after keyPressed, so the name remains the first category that was called in setup

[Chrisir]

as I mentioned just use save and not saveFrame

also say index++ after saving

[anntie]

  float prevMouseX = 0;
  float prevMouseY = 0;
  
  PFont F;
 
void setup(){
  
  size(640, 640);
  background(250);
  
  F = loadFont("GTFAdieuTRIAL-Regular-48.vlw");
  
  surface.setTitle("Emotive Drawing");
  surface.setResizable(true);
  surface.setLocation(100, 100);
  
  textSize (40);
  textFont(F);
  fill(0);
  stroke(0);
  //text (string name, x-position, y-position)
  
  String[] words = { "amused", "joyful, cheerful", "energized, pumped up", "annoyed", "excited","anxious, tense", "dreamy", "creeped out, scared", "disgusted, revolted", "angry", "amazed, fascinated", "calm, realxed", "bored", "sad", "flirty", "proud, couragous", "romantic, loving", "confrontational, argumentative", "tender, longing", "triumphant, heroic", "bittersweet, melancholic", "euphoric, ecstatic", "disappointed", "confused", "remorseful, sorry", "silly, goofy", "jealous", "sympathetic, pitiful"};
  int index = int(random(words.length));
  text (words[index], 20, 50);
  println(words[index]);
}

void draw() {
  stroke (50);
  strokeWeight (10);
  //line(pmouseX,pmouseY,mouseX,mouseY);
}

void mousePressed(){
  prevMouseX = mouseX;
  prevMouseY = mouseY;
}

void mouseDragged(){
  fill(0);  
  stroke(0);
  float distanceX;
  float distanceY;
  
  distanceX = abs(mouseX - prevMouseX);
  distanceY = abs(mouseY - prevMouseY);
  
  float avDistance;
  avDistance = (distanceX + distanceY)/2;
  //set range for strokeWeight: squash 0-200 (input range) into 2-10 (output range) 
  avDistance = map(avDistance,0,150,1,50);
  
  //ellipse(mouseX, mouseY, 10, 10);
  //ellipse(mouseX, mouseY, avDistance, avDistance);
  
  prevMouseX = mouseX;
  prevMouseY = mouseY;
  
  //line(pmouseX,pmouseY,mouseX,mouseY);
  //strokeWeight(avDistance);
  line(pmouseX,pmouseY, prevMouseX, prevMouseY);
  
}

void keyPressed(){
  saveFrame("drawings/"+words[index]+"###.png");
  setup();
}

tried to move the word-string and random choice to setup and recall it after keyPressed, but now it tells me “words cannot be resolved to a variable” when it comes to the saving…
sry for all these little questions and replies, i’m trying to solve things as i go along and most of it is just rookie mistakes, but every little piece of input really helps and is greatly appreciated

[Chrisir]

Why int index = int(random(words.length));

should we not start with 0 ?

AND

@anntie

can I use a timer instead of keyPressed

why?

It’s better when the user can press a key to go on to the next image…

[Chrisir]

here is a working version

float prevMouseX = 0;
float prevMouseY = 0;

PFont F;

String[] words = {
  "amused", "joyful, cheerful", "energized, pumped up", "annoyed", "excited", "anxious, tense", "dreamy", 
  "creeped out, scared", "disgusted, revolted", "angry", "amazed, fascinated", "calm, realxed", "bored", "sad", 
  "flirty", "proud, couragous", "romantic, loving", "confrontational, argumentative", "tender, longing", 
  "triumphant, heroic", "bittersweet, melancholic", "euphoric, ecstatic", "disappointed", 
  "confused", "remorseful, sorry", "silly, goofy", "jealous", "sympathetic, pitiful"
};
int index = 0; //  = int(random(words.length));

void setup() {
  size(640, 640);
  background(250);

  F = createFont("GTFAdieuTRIAL-Regular-48.vlw", 40);

  surface.setTitle("Emotive Drawing");
  surface.setResizable(true);
  surface.setLocation(100, 100);

  textSize (40);
  textFont(F);
}

void draw() {
  stroke (50);
  strokeWeight (10);
  //line(pmouseX,pmouseY,mouseX,mouseY);

  fill(0);
  stroke(0);
  //text (string name, x-position, y-position)
  //   int index = int(random(words.length));

  text (words[index], 20, 50);
  //println(words[index]);
}

// ------------------------------------------------------------------------------------

void mousePressed() {
  prevMouseX = mouseX;
  prevMouseY = mouseY;
}

void mouseDragged() {
  fill(0);  
  stroke(0);
  float distanceX;
  float distanceY;

  distanceX = abs(mouseX - prevMouseX);
  distanceY = abs(mouseY - prevMouseY);

  float avDistance;
  avDistance = (distanceX + distanceY)/2;
  //set range for strokeWeight: squash 0-200 (input range) into 2-10 (output range) 
  avDistance = map(avDistance, 0, 150, 1, 50);

  //ellipse(mouseX, mouseY, 10, 10);
  //ellipse(mouseX, mouseY, avDistance, avDistance);

  prevMouseX = mouseX;
  prevMouseY = mouseY;

  //line(pmouseX,pmouseY,mouseX,mouseY);
  //strokeWeight(avDistance);
  line(pmouseX, pmouseY, prevMouseX, prevMouseY);
}

void keyPressed() {

  // remove text !!!!!!!
  fill(250);
  noStroke(); 
  rect( 0, 0, width, 60); 

  // save now !!!!!
  save("drawings/"
    +words[index]
    + ".png");


  // prepare next word !!!!  
  index ++;

  background(250);

  textSize (40);
  textFont(F);
  fill(0);
  stroke(0);

  //text (string name, x-position, y-position)
  //  int index = int(random(words.length));
  text (words[index], 20, 50);
  println(words[index]);
}
//

[anntie]

@Chrisir

Why int index = int(random(words.length));

should we not start with 0 ?

hm I’m not sure what you mean… starting with 0 when naming the files would be nice? I took this snippet of code from the third example of the random() reference.

@Chrisir

@anntie

can I use a timer instead of keyPressed

why?

It’s better when the user can press a key to go on to the next image…

to prevent that the signs get too elaborate or that somebody just paints the whole thing black.
Idk if a timer is the best option, you’re right that it can be annoying. I had something in mind like quick, draw! uses

Quick, Draw!

thank you!!

[Chrisir]

in this new version: 19 seconds timer

float prevMouseX = 0;
float prevMouseY = 0;

PFont F;

final String[] words = {
  "amused", 
  "joyful, cheerful", 
  "energized, pumped up", 
  "annoyed", 
  "excited", 
  "anxious, tense", "dreamy", 
  "creeped out, scared", "disgusted, revolted", "angry", "amazed, fascinated", "calm, realxed", "bored", "sad", 
  "flirty", "proud, couragous", "romantic, loving", "confrontational, argumentative", "tender, longing", 
  "triumphant, heroic", "bittersweet, melancholic", "euphoric, ecstatic", "disappointed", 
  "confused", "remorseful, sorry", "silly, goofy", "jealous", "sympathetic, pitiful"
};

int index = 0; //  = int(random(words.length));

int timer; 

void setup() {
  size(640, 640);
  background(250);

  surface.setTitle("Emotive Drawing");
  surface.setResizable(true);
  surface.setLocation(100, 100);

  F = createFont("GTFAdieuTRIAL-Regular-48.vlw", 40);
  textSize (40);
  textFont(F);

  timer = millis();
}

void draw() {
  // When we are done? 
  if (index>=words.length) {
    background(255);

    fill(0);
    stroke(0);
    text("we are done", 230, 200);
    return; // LEAVE HERE
  }

  // -----

  stroke (50);
  strokeWeight (10);

  // remove text !!!!!
  fill(250);
  noStroke(); 
  rect( 0, 0, width, 60); 

  // show text 
  fill(0);
  stroke(0);
  text (words[index], 20, 50);

  if (millis()-timer > 19 * 1000) {
    // prepare next word !!!!  
    index++;
    background(250);
    textSize (40);
    textFont(F);
    fill(0);
    stroke(0);
    timer=millis();
  }
}

// ------------------------------------------------------------------------------------

void mousePressed() {
  prevMouseX = mouseX;
  prevMouseY = mouseY;
}

void mouseDragged() {
  fill(0);  
  stroke(0);
  float distanceX;
  float distanceY;

  distanceX = abs(mouseX - prevMouseX);
  distanceY = abs(mouseY - prevMouseY);

  float avDistance;
  avDistance = (distanceX + distanceY)/2;
  //set range for strokeWeight: squash 0-200 (input range) into 2-10 (output range) 
  avDistance = map(avDistance, 0, 150, 1, 50);

  //ellipse(mouseX, mouseY, 10, 10);
  //ellipse(mouseX, mouseY, avDistance, avDistance);

  prevMouseX = mouseX;
  prevMouseY = mouseY;

  //line(pmouseX,pmouseY,mouseX,mouseY);
  //strokeWeight(avDistance);
  line(pmouseX, pmouseY, prevMouseX, prevMouseY);
}

void keyPressed() {

  if (index>=words.length) {
    return; // LEAVE
  }

  // remove text !!!!!!!
  fill(250);
  noStroke(); 
  rect( 0, 0, width, 60); 

  // save now !!!!!
  save("drawings/"
    +words[index]
    + ".png");

  // prepare next word !!!!  
  index++;
  background(250);
  textSize (40);
  textFont(F);
  fill(0);
  stroke(0);
  timer=millis();
}
//

[Chrisir]

this new version can now store different images (enumerated) for the same emotion (when you restart your Sketch)

it also stops when we are done; (OR set index = 0; )

Now you need some image recognition like with openCV - I can’t help you with that.

gotta go!

float prevMouseX = 0;
float prevMouseY = 0;

PFont F;

final String[] words = {
  "amused", 
  "joyful, cheerful", 
  "energized, pumped up", 
  "annoyed", 
  "excited", 
  "anxious, tense", "dreamy", 
  "creeped out, scared", "disgusted, revolted", "angry", "amazed, fascinated", "calm, realxed", "bored", "sad", 
  "flirty", "proud, couragous", "romantic, loving", "confrontational, argumentative", "tender, longing", 
  "triumphant, heroic", "bittersweet, melancholic", "euphoric, ecstatic", "disappointed", 
  "confused", "remorseful, sorry", "silly, goofy", "jealous", "sympathetic, pitiful"
};

int index = 0; //  = int(random(words.length));

int timer; 

void setup() {
  size(640, 640);
  background(250);

  surface.setTitle("Emotive Drawing");
  surface.setResizable(true);
  surface.setLocation(100, 100);

  F = createFont("GTFAdieuTRIAL-Regular-48.vlw", 40);
  textSize (40);
  textFont(F);

  timer = millis();
}

void draw() {
  // When we are done? 
  if (index>=words.length) {
    background(255);

    fill(0);
    stroke(0);
    text("we are done", 230, 200);
    return; // LEAVE HERE
  }

  // -----
  // NOT done 

  stroke (50);
  strokeWeight (10);

  // remove text !!!!!
  fill(250);
  noStroke(); 
  rect( 0, 0, width, 60); 

  // show text 
  fill(0);
  stroke(0);
  text (words[index], 20, 50);

  if (millis()-timer > 19 * 1000) {
    // prepare next word !!!!  
    SaveAndPrepareNextWord();
  }
}

// ------------------------------------------------------------------------------------

void mousePressed() {
  prevMouseX = mouseX;
  prevMouseY = mouseY;
}

void mouseDragged() {
  fill(0);  
  stroke(0);
  float distanceX;
  float distanceY;

  distanceX = abs(mouseX - prevMouseX);
  distanceY = abs(mouseY - prevMouseY);

  float avDistance;
  avDistance = (distanceX + distanceY)/2;
  //set range for strokeWeight: squash 0-200 (input range) into 2-10 (output range) 
  avDistance = map(avDistance, 0, 150, 1, 50);

  //ellipse(mouseX, mouseY, 10, 10);
  //ellipse(mouseX, mouseY, avDistance, avDistance);

  prevMouseX = mouseX;
  prevMouseY = mouseY;

  //line(pmouseX,pmouseY,mouseX,mouseY);
  //strokeWeight(avDistance);
  line(pmouseX, pmouseY, prevMouseX, prevMouseY);
}

void keyPressed() {

  if (index>=words.length) {
    return; // LEAVE
  }

  SaveAndPrepareNextWord();
}

// ------------------------------------------------------------------------------------

void SaveAndPrepareNextWord() {

  // remove text !!!!!!!
  fill(250);
  noStroke(); 
  rect( 0, 0, width, 60); 

  // save now !!!!!
  String newName="drawings\\"
    +words[index]
    + ".png";
  if (fileExists(sketchPath("")+""+newName)) {
    //!!problem
    int i=0;

    while (fileExists(sketchPath("")+""+newName)) {

      newName="drawings\\"
        +words[index] 
        +trim(str(i))
        + ".png";
      i++;
    }
    save(newName);
  } else {
    save(newName);
  }

  // prepare next word !!!!  
  index++;
  background(250);
  textSize (40);
  textFont(F);
  fill(0);
  stroke(0);
  timer=millis();
}


boolean fileExists(String fileName) {
  File file=new File(fileName);
  boolean exists = file.exists();
  if (!exists) {
    println(fileName);
    return false;
  } else {

    return true;
  }
} 

//

[Chrisir]

@anntie

hm I’m not sure what you mean… starting with 0 when naming the files would be nice?

You are right again!

using a random is a very good idea !

[anntie]

awesome awesoomee!! thank you so much, that really looks a lot better now!! I’m still going through it and figuring out whats happening. The sketchPath is new to me, and something there is not quite right: It used to save the frames to a folder called “drawings”, now they are stored in the sketch library.

  // remove text !!!!!!!
  fill(250);
  noStroke(); 
  rect( 0, 0, width, 60); 

  // save now !!!!!
  String newName=
    "categories[index]"
    + ".png";
  if (fileExists(sketchPath("")+""+newName)) {
    //!!problem
    int i=0;

    while (fileExists(sketchPath("")+""+newName)) {

      newName=
        "categories[index]" 
        +trim(str(i))
        + ".png";
      i++;
    }
    save("drawings/newName");
  } else {
    save("drawings/newName");
  }

currently attempting to sort it out. But I had the idea, maybe its more useful to store each category in a seperate folder and name them with a time stamp variable + number or something?

@Chrisir

Now you need some image recognition like with openCV - I can’t hel you with that.

or is it easier to have everything stored in 1? super excited about open CV, that would be the next level!! Glad to have you here in this thread, thanks!

[Chrisir]

@anntie

The sketchPath is new to me, and something there is not quite right: It used to save the frames to a folder called “drawings”, now they are stored in the sketch library.

When I run it, the folder “drawings” is used and the folder “drawings”, is still in the Sketch’s folder.

I just use the sketchPath () function because the function fileExists seems to require a full path (absolute path) and not only a relative path.