Interaction Lab

A. Good Parents or Not? – Yuebei Xu (Monica yx2121) – Professor Andy Garcia

B.CONCEPTION AND DESIGN

Project Concept: Our project is a story development game in the context of family relationships between parents and children. Our previous preparatory research inspired us to think about the reasons behind the increasing depression rate among children and youth. With attention to kids’ mental health, we contemplated the relationship between parents and kids. We believe it is meaningful to develop a story between parents and kids based on real-life experiences, making players role as parents and engaging in kids’ growing process. We expect two major kinds of user interaction. First, the story development game itself is a direct interaction because the story ending is determined based on choices the player makes for their virtual kids. Second, we include a scale on which players are required to put cubes that correspond to the stress of their choices. Put simply, we aim to use the scale to visualize the pressure that parents put on their kids. It is possible that we provide a hint for players to be careful with each choice they make by visualizing that. 

In the user testing, we got several valuable feedback. First, we only had 6 questions, and several testers suggested we add a few more questions so the story is more complete. So we eventually have 10 questions. Second, our button device looked terrible, so we made good decorations afterward. Third, we found some people are a bit confused with the instructions, so we modified the instruction on the introduction page as well. Those changes are necessary and effective. 

C. FABRICATION AND PRODUCTION 

This project is heavily on the digital part that was developed based on Processing code, with 4 buttons and scale on the Arduino part. The most important part of our project, I believe, is the code for the smooth transition from one scenario to another. This part is somewhat difficult because we had to use “previous value” to record different states so that the scenario only changes once when players press the button. Otherwise, since Arduino analog keeps reading values, the states can iterate very fast while the players even only press the button once. We didn’t have many options besides this, we discussed with the professor over this problem and concluded this would be an ideal solution to avoid such a tech glitch.

D. CONCLUSIONS

The goal of our project is to educate and remind parents of the potential stress they can put on kids when they make different decisions in their kid’s development process. I believe we achieved the stated goal because we have 4 different endings for kids based on kids’ final mental health grades and academic performance grades. With the feedback, players will be able to see their expected outcome and their actual outcome. This contrast serves as direct feedback for players. The project is aligned with the definition of interaction because the project communication is two-way. The story is constructed by players throughout the experience and it returns feedback to users. 

As for improvement, for one thing, we would be happy to include more storylines and narrow down the grade range further in order to enrich the story plots and variations. In addition, we would also like to record our voices and add filters to make it a good story narrative, which ideally will be applied to each scenario. We also identified the confusion of instruction on the “scale” part, so further clarification is necessary as well. Given the project is heavily digital, the setbacks primarily came from coding. And I think the key obstacle is how to perfectly manage digital communication between Arduino and Processing, specifically, the timing we need to make use of the recorded value and the timing we wish to stop the communication. While this project is very time-consuming due to the coding part, it rewards me a great sense of accomplishment during the final presentation when I saw people showing curiosity and interest in our project. And it is also very interesting to see people responding by the end with “Aha, I’m a good parent!”, “I didn’t know I put so much stress on my kids”, “My kid will be happy but only mediocre in academic performance?”. The feedback perfectly indicates the goal of the project is realized. 

E. APPENDIX

All technical documentation that was not included in previous sections should be attached here. Include your code from both Processing and Arduino, wiring diagrams (e.g. made with TinkerCad Circuits), pictures, videos and all related documentation that would help to understand the technical development process. If you used code or any assets that were not your own, properly cite them.

Video for Complete Project

Pictures

    Laser Cutting Draft for button box

Add ABCD option has the 

In the process of Button Box installation

Adding emoji decorations to Button Box.

Cut different weight cubes (credits to Professor Andy)

  Button, Scale, Screen Display, Cubes (final presentation)

Code from Arduino

import processing.serial.*;
import processing.sound.*;
Serial serialPort;
SoundFile sound;

int NUM_OF_VALUES_FROM_ARDUINO = 5;  /* CHANGE THIS ACCORDING TO YOUR PROJECT */
/* This array stores values from Arduino */
int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO];


PImage kid1;
PImage kid2;
PImage kid3;
PImage a1;
PImage a2;
PImage a3;
PImage a4;
PImage a5;
PImage a6;
PImage a7;
PImage a8;
PImage a9;

PImage ed1;
PImage ed2;
PImage ed3;
PImage ed4;


// use state to record if the button is pressed
int state = 0;
// value we need for game: ap and mp
int AP = 100;
int MP = 100;
Float weight;
int grade;

int preVal;
int preMP;

//String[] instruction = new String[7];
//instruction[0] = "You and your lover have a kid, and you really love him/her";
//instruction[1] = "You acoompanied with your kid through a carefree childhood";
//instruction[2] = "As time goes by, your kid needs to attend primary school";
//instruction[3] = "Suddenly, you need to make so many decicisions for your kid";
//instruction[4] = "No matter you like it or not, it is your duty...";
//instruction[5] = "Now, it is your turn to make choices";
//instruction[6] = "Press Any Button To Continue...";


void setup() {
  size(1200, 750);
  kid1 = loadImage("kid1.jpg");
  kid2 = loadImage("kid2.jpg");
  kid3 = loadImage("kid3.jpg");
  a1 = loadImage("a1.jpg");
  a2 = loadImage("a2.jpg");
  a3 = loadImage("a3.jpg");
  a4 = loadImage("a4.jpg");
  a5 = loadImage("a5.jpg");
  a6 = loadImage("a6.jpg");
  a7 = loadImage("a7.jpg");
  a8 = loadImage("a8.png");
  a9 = loadImage("a9.jpg");
  ed1 = loadImage("ed1.jpg");
  ed2 = loadImage("ed2.jpg");
  ed3 = loadImage("ed3.png");
  ed4 = loadImage("ed4.jpg");


  //start page
  image(kid1, 0, 0);

  // BGM
  frameRate(30);
  println("Loading mp3...");
  sound = new SoundFile(this, "Under the Heavens.mp3");
  sound.loop();


  printArray(Serial.list());
  // put the name of the serial port your Arduino is connected
  // to in the line below - this should be the same as you're
  // using in the "Port" menu in the Arduino IDE
  serialPort = new Serial(this, "/dev/cu.usbmodem142301", 9600);
}



void draw() {
  //println(state);
  //println(MP);
  //receive the values from Arduino
  getSerialData();
  // use the values like this:
  int a = arduino_values[0];
  int b = arduino_values[1];
  int c = arduino_values[2];
  int d = arduino_values[3];
  
  float e = arduino_values[4];
  println (e);
  weight = map (e, 76070, 118125, 0, 100);
  grade = int (100 - weight);
  
  println(weight);
  println(grade);
  // 76000 nothing; largest box: 80800

  if (state == 100) {
    funeral();
  }
  if (state == 200) {
    if (MP>=60 && AP >=80) {
      ed1();
    } else if (MP<60 && AP >=80) {
      ed2();
    } else if (MP>=60 && AP <=80) {
      ed3();
    } else {
      ed4();
    }
  }


  if (state == 0) {
    noStroke();
    fill(255, 255, 255, 100);
    triangle(800, 200, 900, 200, 750, 250);
    ellipse(950, 150, 350, 250);
    fill(255, 255, 255, 1);
    rect(750, 575, 350, 100);
    String s = "What Do You Want Me";
    String t = "To Be Like?";
    fill(#778283);
    strokeWeight (10);
    textSize(30);
    text (s, 810, 140);
    text (t, 880, 190);
    String m = "Press Any Button To";
    String n = "Start The Story Game!";
    fill(255, 255, 255, 10);
    text (m, 795, 615);
    text (n, 780, 650);
  }


  // page 2 - Introduction Page -- no need to make choice, just press button to proceed
  if (state == 1) {
    image(kid2, 0, 0);
    fill(255);
    textSize(30);
    text ("You and your lover have a kid, and you really love him/her", 150, 120);
    text ("You acoompanied with your kid through a carefree childhood", 150, 170);
    text ("As time goes by, your kid needs to attend school", 150, 220);
    text ("Suddenly, you face so many decicisions", 150, 270);
    text ("No matter you like it or not, it is your duty...", 150, 320);
    text ("Now, it is your turn to make choices", 150, 370);
    textSize(23);
    text ("For each choice, add weight if you feel your chocie are somewhat harsh", 150, 650);
    text ("take off weight if you feel your chocie relief kid's pressure", 150, 685);
    textSize (22);
    text ("Press Any Button To Continue...", 900, 710);

    //for (int i = 0; i < instruction.length; i++) {
    //  fill(255);
    //  textSize(30);
    //  text(instruction[i], 100, 50 * i + 50);
    //  delay (200);
    //}
  }

  if (state == 2) {
    a1();
    // Choose A
    if (a == 1) {
      if (a != preVal) {
        MP+=5;
        AP+=5;
        preVal = 0;
      }
    }
    // Choose B
    if (b == 1) {
      if (b != preVal) {
        //MP unchange
        MP+=5;
        AP-=5;
        preVal = 0;
      }
    }
    // Choose C
    if (c == 1) {
      if (c!= preVal) {
        MP-=10;
        AP-=5;
        preVal = 0;
      }
    }
    // Choose D
    if (d == 1) {
      if (d != preVal) {
        MP-=20;
        AP-=20;
        preVal = 0;
      }
    }
    if (MP <=0) {
      state = 100;
    }
  }

  if (state == 3) {
    a2();
    // Choose A
    if (a == 1) {
      if (a != preVal) {
        MP+=5;
        AP+=5;
        preVal = 0;
      }
    }
    // Choose B
    if (b == 1) {
      if (b != preVal) {
        //MP unchange
        MP-=10;
        AP+=5;
        preVal = 0;
      }
    }
    // Choose C
    if (c == 1) {
      if (c!= preVal) {
        MP-=10;
        AP-=10;
        preVal = 0;
      }
    }
    // Choose D
    if (d == 1) {
      if (d != preVal) {
        MP-=20;
        AP-=20;
        preVal = 0;
      }
    }
    if (MP <=0) {
      state = 100;
    }
  }

  if (state == 4) {
    a3();
    if (a == 1) {
      if (a != preVal) {
        MP+=5;
        AP+=5;
        preVal = 0;
      }
    }
    // Choose B
    if (b == 1) {
      if (b != preVal) {
        //MP unchange
        MP+=5;
        AP-=10;
        preVal = 0;
      }
    }
    // Choose C
    if (c == 1) {
      if (c!= preVal) {
        MP-=5;
        AP+=10;
        preVal = 0;
      }
    }
    // Choose D
    if (d == 1) {
      if (d != preVal) {
        MP-=10;
        AP-=10;
        preVal = 0;
      }
    }
    if (MP <=0) {
      state = 100;
    }
  }

  if (state == 5) {
    a4();
    if (a == 1) {
      if (a != preVal) {
        MP-=20;
        AP-=10;
        preVal = 0;
      }
    }
    // Choose B
    if (b == 1) {
      if (b != preVal) {
        //MP unchange
        MP-=10;
        AP-=5;
        preVal = 0;
      }
    }
    // Choose C
    if (c == 1) {
      if (c!= preVal) {
        MP-=5;
        AP+=10;
        preVal = 0;
      }
    }
    // Choose D
    if (d == 1) {
      if (d != preVal) {
        MP+=5;
        AP-=20;
        preVal = 0;
      }
    }
    if (MP <=0) {
      state = 100;
    }
  }

  if (state == 6) {
    a5();
    if (a == 1) {
      if (a != preVal) {
        MP-=20;
        AP-=5;
        preVal = 0;
      }
    }
    // Choose B
    if (b == 1) {
      if (b != preVal) {
        //MP unchange
        MP-=10;
        AP+=10;
        preVal = 0;
      }
    }
    // Choose C
    if (c == 1) {
      if (c!= preVal) {
        MP+=5;
        AP+=10;
        preVal = 0;
      }
    }
    // Choose D
    if (d == 1) {
      if (d != preVal) {
        MP+=10;
        AP+=10;
        preVal = 0;
      }
    }
    if (MP <=0) {
      state = 100;
    }
  }

  if (state == 7) {
    a6();
    if (a == 1) {
      if (a != preVal) {
        MP-=10;
        AP+=5;
        preVal = 0;
      }
    }
    // Choose B
    if (b == 1) {
      if (b != preVal) {
        //MP unchange
        MP-=20;
        AP+=10;
        preVal = 0;
      }
    }
    // Choose C
    if (c == 1) {
      if (c!= preVal) {
        MP+=10;
        AP-=10;
        preVal = 0;
      }
    }
    // Choose D
    if (d == 1) {
      if (d != preVal) {
        MP-=20;
        AP-=20;
        preVal = 0;
      }
    }
    if (MP <=0) {
      state = 100;
    }
  }

  if (state == 8) {
    a7();
    if (a == 1) {
      if (a != preVal) {
        MP-=30;
        AP-=5;
        preVal = 0;
      }
    }
    // Choose B
    if (b == 1) {
      if (b != preVal) {
        //MP unchange
        MP-=10;
        AP-=10;
        preVal = 0;
      }
    }
    // Choose C
    if (c == 1) {
      if (c!= preVal) {
        MP+=10;
        AP-=10;
        preVal = 0;
      }
    }
    // Choose D
    if (d == 1) {
      if (d != preVal) {
        MP+=20;
        AP+=10;
        preVal = 0;
      }
    }
    if (MP <=0) {
      state = 100;
    }
  }

  if (state == 9) {
    a8();
    if (a == 1) {
      if (a != preVal) {
        MP-=30;
        AP+=-10;
        preVal = 0;
      }
    }
    // Choose B
    if (b == 1) {
      if (b != preVal) {
        //MP unchange
        MP-=5;
        AP-=10;
        preVal = 0;
      }
    }
    // Choose C
    if (c == 1) {
      if (c!= preVal) {
        MP+=10;
        AP-=10;
        preVal = 0;
      }
    }
    // Choose D
    if (d == 1) {
      if (d != preVal) {
        MP-=5;
        AP-=20;
        preVal = 0;
      }
    }
    if (MP <=0) {
      state = 100;
    }
  }


  if (state == 10) {
    a9();
    if (a == 1) {
      if (a != preVal) {
        MP-=20;
        AP-=10;
        preVal = 0;
      }
    }
    // Choose B
    if (b == 1) {
      if (b != preVal) {
        //MP unchange
        MP+=10;
        AP-=10;
        preVal = 0;
      }
    }
    // Choose C
    if (c == 1) {
      if (c!= preVal) {
        MP-=10;
        AP-=20;
        preVal = 0;
      }
    }
    // Choose D
    if (d == 1) {
      if (d != preVal) {
        MP+=5;
        AP+=20;
        preVal = 0;
      }
    }
    if (MP <=0) {
      state = 100;
    }
  }

  if (state == 11) {
    if (MP <=0) {
      state = 100;
    }
    if (MP>0) {
      state = 200;
    }
  }



  // press a button, and move on to next image
  if (a == 1 || b == 1 || c == 1 || d == 1 ) {
    preVal = 1;
  }
  if (a!=preVal && b!=preVal && c!=preVal && d!=preVal) {
    state +=1;
    preVal = 0;
  }
  //draw() end
}





//the helper function below receives the values from Arduino
// in the "arduino_values" array from a connected Arduino
// running the "serial_AtoP_arduino" sketch
// (You won't need to change this code.)

void getSerialData() {
  while (serialPort.available() > 0) {
    String in = serialPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
    if (in != null) {
      //print("From Arduino: " + in);
      String[] serialInArray = split(trim(in), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          arduino_values[i] = int(serialInArray[i]);
        }
      }
    }
  }
}


// here we define function to be used
void a1() {
  noStroke();
  image(a1, 0, 0);
  fill(#B7B7B7, 180);
  rect(130, 110, 500, 77, 25);
  fill(255);
  textSize(40);
  text ("I got a low grade in school…", 150, 160);

  fill(#B7B7B7, 180);
  rect(130, 390, 915, 280, 25);
  fill(255);
  textSize(35);
  text("A. It's Ok, You Will Make Progress", 150, 430);                               // MP+5, AP+5
  text("B. You don't think grade is that important in primary school", 150, 500);      // MP+5, AP-5
  text("C. Hold Temper, but Actually Anxious about Peer Competition", 150, 570);      // MP-10, AP-5
  text("D. Angrily: You Didn't Work Hard Enough", 150, 640);                          // MP-20, AP-20
};

void a2() {
  image(a2, 0, 0);
  fill(#B7B7B7, 180);
  rect(130, 110, 900, 77, 25);
  fill(255);
  textSize(35);
  text ("You find Your Kid Got a Very Low Grade Without Telling You...", 150, 160);

  fill(#B7B7B7, 180);
  rect(130, 390, 860, 280, 25);
  fill(255);
  textSize(30);
  text("A. 'Why did you hide the Grade?'", 150, 430);  // MP+5, AP+5
  text("B. 'Why did you get this Grade?'", 150, 500);         // MP-10, AP+5
  text("C. You talk to teacher first", 150, 570);                                         // MP-10, AP-10
  text("D. Shocked and Angry,'You Didn't Work Hard Enough and You Lie?'", 150, 640);  // MP-20, AP-20
};


void a3() {
  image(a3, 0, 0);
  fill(#B7B7B7, 180);
  rect(130, 110, 690, 77, 25);
  fill(255);
  textSize(35);
  text ("Can I hang out with my friends on Weekend?", 150, 160);

  fill(#B7B7B7, 180);
  rect(130, 350, 900, 290, 25);
  fill(255);
  textSize(30);
  text("A. Yes but Back Home Before Dinner, You have Exams tomorrow", 150, 400);                   // MP+5, AP+5
  text("B. Sure, go have some fun in Weekend", 150, 470);                                         // MP+5, AP-10
  text("C. Well...Did You Review for BIG Exams on Monday?", 150, 540);                            // MP-5, AP+10
  text("D. No Unless You finished Exam Practice", 150, 610);                                     // MP-10,AP-10
};


void a4() {
  image(a4, 0, 0);
  fill(#B7B7B7, 180);
  rect(130, 95, 900, 100, 25);
  rect(130, 390, 885, 300, 25);
  fill(255);
  textSize(35);
  text ("I don't want to attend class today, I want some sleep.", 150, 160);
  textSize(30);
  text("A. You need to get up right now!", 150, 430);                  // MP-20,AP-10
  text("B. You must go unless you are not feeling well", 150, 500);             // MP-10,AP-5
  text("C. OK but you need to talk with teachers over the reasons by yourself", 150, 570);    // MP-5,AP+10
  text("D. OK. I text teacher", 150, 640);                           // MP+5, AP-20
};

void a5() {
  image(a5, 0, 0);
  fill(#B7B7B7, 180);
  rect(130, 95, 860, 150, 25);
  rect(130, 390, 885, 290, 25);
  fill(255);
  textSize(35);
  text ("You heard from other parents about good extra courses", 150, 160);
  text ("that help kids improve exam scores", 150, 210);
  textSize(30);
  text("A. Nervous about peer competition, sign kid up for many courses", 150, 430);            // MP-20, AP-5
  text("B. Sign kid up for a one/two courses", 150, 500);                                       // MP-10, AP+10
  text("C. You Believe courses in school are enough, will not send kids there", 150, 570);      // MP+5, AP+10
  text("D. Prioritize Kid's Willingness, Respect Kid's Decisions", 150, 640);                   // MP+10, AP+10
};


void a6() {
  image(a6, 0, 0);
  fill(#B7B7B7, 180);
  rect(130, 100, 730, 160, 25);
  rect(130, 430, 890, 270, 25);
  fill(255);
  textSize(35);
  text ("I really don't like taking that class after school", 150, 160);
  text ("Can I just don't Go Please...", 150, 220);
  textSize(30);
  text("A. Insist it is good for kid's future development", 150, 480);  // MP-10, AP+5
  text("B. Angry: You should work harder", 150, 540);                                // MP-20, AP+10
  text("C. Yes, sorry to make you feel you are forced to take that class", 150, 600);       // MP+10, AP-10
  text("D. Yes but can you promise your exam performance?", 150, 660);                      // MP-20, AP-20
};



void a7() {
  image(a7, 0, 0);
  fill(#B7B7B7, 180);
  rect(130, 100, 775, 110, 25);
  rect(130, 430, 840, 280, 25);
  fill(255);
  textSize(35);
  text ("You are told Kid has girlfriend/boyfriend at school...", 150, 160);
  textSize(30);
  text("A. Angry, Stop the Relationship", 150, 480);                                          // MP-30, AP-5
  text("B. Want to Stop, but Ask kid how is the other person first", 150, 540);               // MP-10, AP-10
  text("C. Don't think it is a big deal", 150, 600);                                          // MP+10, AP-10
  text("D. Feel Truly Happy for Kid", 150, 660);                                              // MP+20, AP+5
};


void a8() {
  image(a8, 0, 0);
  fill(#B7B7B7, 180);
  rect(130, 100, 500, 110, 25);
  rect(130, 430, 890, 280, 25);
  fill(255);
  textSize(35);
  text ("'What if I give up college?'", 150, 160);
  textSize(30);
  text("A. Impossible, Why can't You make it when others are all out there?", 150, 480);      // MP-30, AP-10
  text("B. Why", 150, 540);                                                                   // MP-5, AP-10
  text("C. You are an adult, you make decisions and be responsible for that", 150, 600);                                  // MP+10, AP-10
  text("D. Anxious but no idea what to do", 150, 660);                                        // MP-5, AP-20
};



void a9() {
  image(a9, 0, 0);
  fill(#B7B7B7, 180);
  rect(130, 100, 730, 110, 25);
  rect(130, 430, 800, 280, 25);
  fill(255);
  textSize(35);
  text ("I'm so stressed at graded mock exams recently", 150, 160);
  textSize(30);
  text("A. Stress pushes you forward", 150, 480);                                      // MP-20, AP-10
  text("B. Do you want to take a rest tommorrow?", 150, 540);                          // MP+10, AP-10
  text("C. It's just mock exams", 150, 600);                                           // MP-10, AP-20
  text("D. Try to set lower expextations for grade", 150, 660);                        // MP+5,  AP+20
};



void funeral() {
  image (kid3, 0, 0);
  fill (255);
  textSize(37);
  text ("Sad Story: YOU LOST YOUR KID", 520, 200);
  text ("Kid's Mental Health Points is", 520, 280);
  text("You expect kid's Mental Health to be:", 520, 360);
  text(grade, 520, 440);
  text (MP, 980, 280);
}


void ed1() {
  noStroke();
  image (ed1, 0, 0);
  fill(#B7B7B7, 180);
  rect(140, 77, 900, 327, 25);
  fill (255);
  textSize(35);
  text("You expect kid's Mental Health to be:", 180, 120);
  text(grade, 740, 120);
  text("Kid's Mental Heath Grade:", 180, 180);
  text(MP, 570, 180);
  text("Kid's Academic Performance Grade:", 180, 240);
  text(AP, 720, 240);
  text("Your kid gets into a great college and has a", 180, 300);
  text("great time in college. After graduation, your", 180, 350);
  text("kid lives their own lives smoothly and happily", 180, 400);
}

void ed2() {
  noStroke();
  image (ed2, 0, 0);
  fill(#B7B7B7, 180);
  rect(140, 77, 900, 327, 25);
  fill (255);
  textSize(35);
  text("You expect kid's Mental Health to be:", 180, 120);
  text(grade, 740, 120);
  text("Kid's Mental Heath Grade:", 180, 180);
  text(MP, 570, 180);
  text("Kid's Academic Performance Grade:", 180, 240);
  text(AP, 720, 240);
  text("Your kid goes to a good college, but when he becomes", 180, 300);
  text("self-indulgent when he/she has full control over life.", 180, 350);
  text("He/she fails college and lives an unhappy and normal life", 180, 400);
}

void ed3() {
  noStroke();
  image (ed3, 0, 0);
  fill(#B7B7B7, 180);
  rect(140, 77, 900, 327, 25);
  fill (255);
  textSize(35);
  text("You expect kid's Mental Health to be:", 180, 120);
  text(grade, 740, 120);
  text("Kid's Mental Heath Grade:", 180, 180);
  text(MP, 570, 180);
  text("Kid's Academic Performance Grade:", 180, 240);
  text(AP, 720, 240);
  text("Your child goes to a mediocre college and leads a", 180, 300);
  text("very normal but fulfilling life.", 180, 360);
}

void ed4() {
  noStroke();
  image (ed4, 0, 0);
  fill(#B7B7B7, 180);
  rect(140, 77, 900, 327, 25);
  fill (255);
  textSize(35);
  text("You expect kid's Mental Health to be:", 180, 120);
  text(grade, 740, 120);
  text("Kid's Mental Heath Grade:", 180, 180);
  text(MP, 570, 180);
  text("Kid's Academic Performance Grade:", 180, 240);
  text(AP, 720, 240);
  text("Your child fails to enter a college, and escapes from", 180, 300);
  text("home to find his/her freedom.", 180, 360);
}

A. Electronic Music Master – Monica Xu – Professor Andy Garcia

B. CONTEXT AND SIGNIFICANCE

In the previous group project, our group created a human-machine interactive device that helps solve the pain points people encountered during the pandemic, for example, inconvenient groceries shopping. Therefore, it inspired me to contemplate else pain points that trouble people in different scenarios. My initial idea was to design an “alert” with a distance sensor that helps people stay within a range around laptops so as to keep them focusing on study and work. My group mate, Jae, has a completely different idea – Music Box – which involves more entertaining elements in the design draft. After discussing with professor Andy, we believed the “electronic music instrument” was a more feasible artifact we could make for the midterm project given what we have learned (code) in class. Our creation is unique because it involves two-hand cooperation at the same time; we have also seen two people playing simultaneously during the final presentation to make different sounds (with different pitches), making the performance sounds like a band playing multiple instruments. Speaking of existing projects, professor Andy indeed shared two examples that inspired us. One is called simple Arduino Synthesizer Build (we therefore related distance sensor to pitches changing) (https://www.youtube.com/watch?v=x07FF-A67Nc&t=155s), another is called One Man Band designed by student Christophe Lu that inspired us to include multiple music sounds in our design. Apparently, those works involve a huge amount of human interaction as the music sounds out only when players press, touch, or use multiple body parts to control the artifacts. I wouldn’t say our project was way better than any of them, but we derived ideas from both so our design has certain combined features that make the user experience quite interesting. Meanwhile, from my perspective, we do not have a targeted audience group (perhaps people who are into electronic music if I have to think of one) because our project is more for entertainment instead of having a particular purpose, people from different age groups and engaged in different industries are welcome to play with our project. Music and art in general have no boundaries, it brings limitless joy to everyone. Our project may remind people of certain interesting childhood memories such as the background music of Mario, electronic music games, etc).

C. CONCEPTION AND DESIGN

At the very beginning, my group mate and I didn’t realize instructions would be necessary until the user testing stage. We simply considered that players will interact by automatically pressing the button and moving their hands up and down above the distance sensor, so we have the button lined up in a row. The feedback we obtained from user testing was that 1) some people did not recognize the distance sensor, instead, they might perceive it as a loudspeaker 2) players could lift their hands after pressing the button, which is the common habit we acquired based on real-life button user instructions; but our project required players to keep pressing the button even when they playing round other functions 3) we didn’t glue the cup (which serves as a loudspeaker for grove speaker) to our main cardboard box, which made player have to hold the cup when playing such that they could not easily control different parts (difficult to handle both cup, button, and distance sensor at the same time). Therefore, we glued the cup and grove sensor to the box and put up instructions in the final presentation. But as one of the players suggested, it would be better if we wrote instructions right on the box instead of printing it out on a separate paper.

Two straightforward criteria we used for material selection are “accessibility” and “feasibility – meaning we are able to write code for it in order to realize certain functions. The materials we used in this project include: a sliding rheostat, a grove sensor, a cup, a stepper motor, a distance sensor, five colorful buttons, cardboard, wires, jumper cables, a white plastic bucket, etc. More explanation for criteria: 1) primarily, we filtered some motors and sensors based on functions we intended to realize 2) we consulted with professors and learning assistants about what kinds of materials to use. Instead of saying they were best suited, I’d rather say we chose them because we have learned coding similar functions in previous classes and recitations, therefore we can start more easily. We first thought about using a servo motor, but we eventually choose the stepper motor because it has a circular top on which we could connect a rotation bar to realize the drum-beating function & the maximum rotation range for a servo motor is 180 degrees which are not enough as we wanted it to be able to make a full 360-degree circle.

D. FABRICATION AND PRODUCTION

Steps in the building process: 

Electronic Music Part:

1. We started with coding to make each button have one piece of sound played out by a buzzer. Initially, we planned to assign each button with one notes[], which we began with making an Array consisting of several notes, for example, NOTE_A3. To make the buzzer sound last longer, we set the lasting duration at 500 for each press (each press corresponded to one sound). We also use the servo motor in the initial design, we extracted the code from the example code of the servo motor and adjusted its rotating range.
2. Error 1: When we ran the code, we realized that it caused certain errors when we run the program, such as overlapping of different sounds, suddenly silent, and unexpected sound-lasting duration. A professor kindly pointed out that it might result from the servo motor code part because it was included in the project’s big loop and the entire program had to wait for servo motors to turn 90 degrees to start a new Loop. With the professor’s help, we changed the code so that the servo motor only turned 1 degree in each Loop. See the code in the picture attached here. (Excuse me for using the picture as the file for this early version code is no longer accessible).                   
3. Error 2: our entire project suddenly went wrong during user testing. The professor kindly suggests that might be the problem from function delay() in between different pressings. Therefore, we tried to reduce the delay duration until deleted it completely. 
4. As the self-experienced audiences, our group believed there are better “sound out” tools better than buzzers, so we tried different speakers. However, some speakers better fit high-volume devices and some require SD cards, which Arduino is unable to achieve. After careful selection, we settled on the grove speaker. To solve the low volume issue, we attached the grove speaker to the bottom of a cup to make a loudspeaker/sound amplifier.
5. We also changed the interval for each pitch several times. We first set the working distance in range (10, 50), but we later found many users started with a very close distance to the distance sensor such that they couldn’t understand how the entire project functioned very easily. So we lowered the lower boundary and the interval for each pitch so that players could better feel the change in the sounds. 

Drum Beats Part: In our initial design, we didn’t have the “drum beats” feature. As professor Andy suggested, it would be better if we have various musical elements (sounds of different instruments), for instance, drum beats and sounds of knocking. With a number of tools being offered, it occurred to us that our previous work focused on the Stepper motor could serve as the punching part on which we put an empty lip cream on its top, so it can hit the bucket when the motor rotated. We also added a red sliding rheostat to control the speed of the stepper motor rotation. 

Roles & Contribution & Teamwork: In general, we went through everything including the brainstorming, coding, and building process together. Specifically, Jae is more familiar with coding given her previous coding experience, so she helped me a lot in terms of introducing different functions and how to efficiently code for each function we aimed to realize. I enjoyed our teamwork, sometimes we worked together in Studio and sometimes we did work in turn due to different class schedules. Communication was fantastic and my groupmate is a creative girl 🙂

As already mentioned above, we obtained good suggestions during user testing, for example, instructions could be necessary and helpful, and gluing the cup and speaker part to the music box could free players’ one hand so that they could handle other parts more easily, etc. We adopted those suggestions as we found them accurate and conducive, and it turned out in good ways in the final presentation.

My code for Mechanism Part:

// int MoPin = 7;    // vibrator Grove connected to digital pin 9
#include 
#define PIN_SLIDE_POT_A A0
int value_slide_pot_a;
int spd;
int DIR_PIN = 2;
int STEP_PIN = 3;
int EN_PIN = 4;
int pt_PIN = 8;
int ding;
int DING;

// Define a stepper and the pins it will use
// AccelStepper::DRIVER means a stepper driver (with step and direction pins)
AccelStepper stepper(AccelStepper::DRIVER, STEP_PIN, DIR_PIN);

void setup() {
  // pinMode(MoPin, OUTPUT);
  pinMode(PIN_SLIDE_POT_A, INPUT);
  pinMode(EN_PIN, OUTPUT); 
  digitalWrite(EN_PIN, LOW);
  Serial.begin(9600);
  // The run() function will accelerate up to
  // the speed set here
  stepper.setMaxSpeed(1000);
}

void loop() {
 spd = map(value_slide_pot_a, 120, 1023, 0, 800);
  stepper.setSpeed(spd);
  value_slide_pot_a = analogRead(PIN_SLIDE_POT_A);
  stepper.runSpeed();
  Serial.println(value_slide_pot_a);

} 

My code for Tone Part: (another tab “pitches.h” is automatically generated)

#include "pitches.h"
const int threshold = 10;  // minimum reading of the sensors that generates a note

// notes to play, corresponding to the 5 sensors:
int notesY[] = {
  NOTE_C4, NOTE_D4, NOTE_E4, NOTE_F4, NOTE_G4
};
int notesW[] = {
  NOTE_D5, NOTE_E5, NOTE_F5, NOTE_G5, NOTE_A6
};
int notesBlack[] = {
  NOTE_F3, NOTE_G3, NOTE_A4, NOTE_B4, NOTE_C4
};
int notesG[] = {
  NOTE_B3, NOTE_C3, NOTE_D3, NOTE_E3, NOTE_F3
};
int notesBlue[] = {
  NOTE_G4, NOTE_A5, NOTE_B5, NOTE_C5, NOTE_D5
};

// defines pins numbers
const int trigPin = 9;
const int echoPin = 10;
// defines variables
long duration;
int distance;


void setup() {
  pinMode(trigPin, OUTPUT);  // Sets the trigPin as an Output
  pinMode(echoPin, INPUT);   // Sets the echoPin as an Input
  Serial.begin(9600);        // Starts the serial communication
}

int servoPos;
int servoDir = 1;

void loop() {
  // Clears the trigPin
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  // Sets the trigPin on HIGH state for 10 micro seconds
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);
  // Reads the echoPin, returns the sound wave travel time in microseconds
  duration = pulseIn(echoPin, HIGH);
  // Calculating the distance
  distance = duration * 0.034 / 2;
  // Prints the distance on the Serial Monitor
  Serial.print("Distance: ");
  Serial.println(distance);


  int button_0 = digitalRead(2);
  int button_1 = digitalRead(3);
  int button_2 = digitalRead(4);
  int button_3 = digitalRead(5);
  int button_4 = digitalRead(6);

  if (button_0 == 1) {
    if (distance >= 5 && distance < 14) { delay(20); tone(8, notesY[0], 1000); } if (distance >= 14 && distance < 23) { delay(20); tone(8, notesY[1], 1000); } if (distance >= 23 && distance < 32) { delay(20); tone(8, notesY[2], 1000); } if (distance >= 32 && distance < 41) { delay(20); tone(8, notesY[3], 1000); } if (distance >= 41 && distance < 50) { delay(20); tone(8, notesY[4], 1000); } } if (button_1 == 1) { if (distance >= 5 && distance < 14) { delay(20); tone(8, notesW[0], 1000); } if (distance >= 14 && distance < 23) { delay(20); tone(8, notesW[1], 1000); } if (distance >= 23 && distance < 32) { delay(20); tone(8, notesW[2], 1000); } if (distance >= 32 && distance < 41) { delay(20); tone(8, notesW[3], 1000); } if (distance >= 41 && distance < 50) { delay(20); tone(8, notesW[4], 1000); } } if (button_2 == 1) { if (distance >= 5 && distance < 14) { delay(20); tone(8, notesBlack[0], 1000); } if (distance >= 14 && distance < 23) { delay(20); tone(8, notesBlack[1], 1000); } if (distance >= 23 && distance < 32) { delay(20); tone(8, notesBlack[2], 1000); } if (distance >= 32 && distance < 41) { delay(20); tone(8, notesBlack[3], 1000); } if (distance >= 41 && distance < 50) { delay(20); tone(8, notesBlack[4], 1000); } } if (button_3 == 1) { if (distance >= 5 && distance < 14) { delay(20); tone(8, notesG[0], 1000); } if (distance >= 14 && distance < 23) { delay(20); tone(8, notesG[1], 1000); } if (distance >= 23 && distance < 32) { delay(20); tone(8, notesG[2], 1000); } if (distance >= 32 && distance < 41) { delay(20); tone(8, notesG[3], 1000); } if (distance >= 41 && distance < 50) { delay(20); tone(8, notesG[4], 1000); } } if (button_4 == 1) { if (distance >= 5 && distance < 14) { delay(20); tone(8, notesBlue[0], 1000); } if (distance >= 14 && distance < 23) { delay(20); tone(8, notesBlue[1], 1000); } if (distance >= 23 && distance < 32) { delay(20); tone(8, notesBlue[2], 1000); } if (distance >= 32 && distance < 41) { delay(20); tone(8, notesBlue[3], 1000); } if (distance >= 41 && distance < 50) {
      delay(20);
      tone(8, notesBlue[4], 1000);
    }
  }
} 

Pictures

(user instruction)  (music box draft work by Jae Yang)

VIDEO

E. CONCLUSIONS

Project Goal: In this project, we wish to make an electronic music box that allows players to create different melodies by pressing different buttons (buttons corresponding to notes, moving their hand above the distance sensor (distance corresponds to different pitches), modifying sliding rheostat value to control drum beats speed. 

Alignment and User interaction feedback: I believed our project is aligned with my definition of interaction. Players can only create a melody when they participate, and they are expected to spend time exploring how to integrate different notes, pitches, and drum beatings to create mellifluously electronic music. So the engagement and interaction are continuous as well as immersive. As stated above, players successfully demonstrated the expected interaction with our magic music box. Looking back, however, I would consider a better design to guide players to explore the function by themselves (perhaps with some hint on the music box) instead of printing detailed instructions on a separate paper. 

Improvement: If time allowed, we initially wished to have two stepper motors as suggested by the professor. We would add an additional potentiometer to another stepper motor so that the two motors could function together but hopefully beat at different paces. If so, we would have one more varying element in our drum beats part and it would be great.

Stebakcs & Accomplishment: The setbacks came from two primary aspects. One is we found it difficult to code perfectly to support some great functions we wished to establish, so we had to make easier versions of it and achieve more basic functions. Second is that we had very limited time to add more decorations and run code again and again before the final presentation. An additional setback was that I found our group worked more efficiently together than when we handled different tasks alone in our personal spare time. Speaking of accomplishment, we refined the code and initial design over and over again with help from the professor, which eventually allowed us to realize the most basic functions we wished to have. Moreover, the teamwork got better and better in the process of the project and we cherish the teammate friendship all way along. 

Step 1 – Reflection on Group Work

Our Group’s idea derivation: in the beginning, each group member shared thoughts on “Step 1: Research”, “Step 2: Read”, and the indispensable elements in an “interaction” performance, for example, two-way feedback among different entities. Inspired by interactive designs “Ghostwriter” and “Phoneix”, we contemplated the relationship between AI/Robot and humans. Based on fictional stories, we settled on the idea to address both pros and cons of technology development. Choosing pandemic as the story context, we would like to present a scenario in which people enjoy the benefit of advanced technology (i.e. using rings and bracelets to command robots to buy groceries, handle difficult tasks, and make remote birthday celebrations possible, etc.) but become partly over-relied on technology (AI/Robot) (they became half-petrified and rigid, this point might be a bit subtle in our performance). In this regard, we related the performance theme with the concerns along with technology development emphasized in our readings. In terms of interaction, the robots received orders from the host, and they gave feedback to the host after completing tasks as well. 

Sketch:

Performance Agenda:

1. The phone call between humans A & B
2. Human A calls the bakery to book a cake
3. Robot A gives Robot B a ride to the bakery
4. Robot B pays at the counter
5. Robot A takes robot B back home
6. Robot B gives the cake to host

Discussion on success & failure: the success of this artifact is that it helped solve pain points (inconvenience in life) that most people have experienced during the pandemic lockdown; I cannot easily think of obvious failures of this design, but if with the second thought, I may consider the artifact is a little bit redundant with multiple pieces of controller/sensors that the host must wear all the time in order to control the robot. Instead, we could simply make one multi-functional sensor.

Group work: I was not the one that made the most contribution in the brainstorming stage, but I  made efforts in story development, including performance structure, plots, and lines. In my eyes, each group member made an equal contribution to the artifact making. Our team coped very well throughout the project, from scheduling regular group meetings to helping each other build different parts of artifacts and to group rehearsal. Two of our team members were lucky to attend the cardboard workshop on Tuesday, so the rest of the group had a clearer idea of how to make things out of recycled materials based on which we also developed the story. 

Step 2 – Thoughts on Other Group’s Performance

I’m impressed by the group that created the helmet with the function of making people remember their dreams. It was interesting to see the rest of the group act out different dreams, and that was also clever to visually help audience understand the idea they wished to convey. In my opinion, they might have related their artifact to the fictional story in which the house is served by AI/Robot butler and the possibility to create an immersive and interactive environment based on people’s imagination. Personally speaking, I like their performance quite much because it was both clear and amusing, so I think their performance met the basic criteria of this assignment. In terms of the design, I didn’t see their helmet has the turn on/off function during their performance, it would be good to have one if that’s not included in their initial design. Their performance reminded me of “lucid dream” and the movie “Inception” where people can control and structure the environment and story in their dreams. I wouldn’t say their performance is unexpectedly creative because such performance design could most effectively help the audience comprehend their ideas, our group thought of similar ways of performing as well. In terms of advice for improvement, I believe they delivered a very good performance in general, but it might be helpful if they widened the distance between each other a little bit more so the stage may look even better 🙂

Step 3: In your blog entry, you should include your script. You can include pictures of your props and costumes etc. If you have a video of the rehearsal or a recording of the actual performance, you may include it as well. Describe clearly how your teamwork was, the different roles, the task allocations, and how was the communication within the team.

Scripts — Characters: Ziyan, Gerry, Ziyi, Monica, Kevin

[Ziyi and Ziyan are each at their own homes, sitting still in their seat]
Ziyi [Dial the phone with tiny finger movements]: Hi Ziyan!
Ziyan: Hi Ziyi!
Ziyi: Ziyan, Happy birthday!
Ziyan: Oh you remember it? Thank you!
Ziyi: Yea… You’re welcome! How are you going to celebrate that?
Ziyan: The pandemic is very severe, so… I don’t want to celebrate it.
Ziyi: Shall we buy a birthday cake?
Ziyan: A birthday cake? But the bakery is pretty far away from here.
Ziyi: Ya… And we have become half statues. When we arrive at the bakery, it’s time to celebrate your million-year-old birthday.
What about our robots? We can ask them to…
Ziyan: Is that realistic?
Ziyi: Well, my robot Monica can fly, who can take your robot, Gerry, to the cake shop.
Ziyan: Oh that’s a great idea! Thank you so much.
Ziyi: You’re welcome.

[Kevin is at his bakery]
Ziyi [Dial the phone]
Kevin: Hello, Kevin’s Bakery. Anything I can help you with?
Ziyi: Hi. I want a cake for my friend.
Kevin: Ok, what size and flavor do you want?
Ziyi: 8 inches, cake flavor.
Kevin: Cake flavor cake, that’s a good choice! You can pick it in 30 minutes.
Ziyi: Thanks, bye.

[At Ziyan’s home: Ziyan called her robot Garry, controlling him through the sensor];
[At Ziyi’s home: Ziyi gave instructions to her robot Monica, controlling Monica to fly to Ziyan’s home]
Ziyi: Monica, monica!
Monica: Hello~
Ziyi: Take my friend’s robot Garry to the bakery.
Monica: Order copied.
Ziyi: Century Avenue, No. 1555.
Monica: Yes, address copied: Century Avenue, No. 1555.
[With Ziyi’s remote control, Monica flies to Ziyan’s house to get Garry. Monica knocks on the door: dang dang dang! ]
Ziyan (to Garry): Open the door, and hold her tightly.
[Garry holds on to Monica’s wings, Monica will bring Garry to the bakery.]
Monica: I won’t stop you if you wanna fall …
Garry: (screaming) AHHH AHHH AHHH AHHH……

[Monica and Garry are at the bakery.]
Kevin: Hi, how can I help you?
Garry: Cake, now!
Kevin: Ok, may I have your telephone number?
Monica: Yes, my host placed the order, with the name Monica and phone number: 12345.
Kevin: Ok, let me see. The cake’s rate is 1 billion in total. Please pay.
Garry: [smash money on the table]
Kevin: Ok, that’s enough.
Monica: [gives Kevin some money] Here are the tips for you.
Kevin: Thank you! [hands the cake to Garry] This is your cake, please enjoy!
[Garry takes the cake and puts it on his head.]
[Monica takes Garry back to Ziyan’s home, but on the way back, Garry drops the cake. Garry picks it up, holding Monica’s wings again, screaming]

[Monica and Garry have arrived at Ziyan’s home.]
Garry: (the robot gives the cake to Ziyan) Cakes for you.
Ziyan: Thank you so much! (To Garry) Go to charge.
Garry: [Kneels down and hides head inside the body.]
Monica: It’s time for me to go back home. [Flies to Ziyi’s home, crouching down and folding her wings.]

[In the bakery, Kevin calls Ziyan, Ziyan receives the call with a tiny finger movement at her home.]
Kevin: Hello, I’m just calling you to make sure you have got the cake.
Ziyan: Oh yeah, I have just received it.
Kevin: Is the cake alright? Because when your robot was at the store, there was something wrong. Could you please check if the quality of the cake is still good?
Ziyan: Yeah, the cake is perfect and the flavor is good.
Kevin: Alright, alright, thank you, thank you! Enjoy it!
Ziyan: Thank you.

Ziyi: [Calls Ziyan] Ding li lingling~
Ziyan:[Answers the phone.]Hello Ziyi.
Ziyi: Have you received the cake?
Ziyan: Yeah, I have received the cake, but it’s a pity that we couldn’t celebrate it together.
Ziyi: Well, what about singing a birthday song together?
Ziyan: Oh, thank you so much!
[Everyone is singing a birthday song together to Ziyan]
Ziyan: Thank you, guys!

How was the teamwork: our teamwork went on well, we share and discuss different ideas, set the agenda together, and each individual attended online and in-person meetings on time. Meanwhile, each person in our group worked hard toward the goal. So in general, I would say it was a very pleasant teamwork experience.

Different roles
Ziyan: half-petrified person, whose birthday is today
Gerry: Ziyan’s robot, can carry lots of stuff
Ziyi: half-petrified person, who would like to help Ziyan to celebrate her birthday
Monica: Ziyi’s robot, can fly
Kevin: bakery owner

Task allocation: in terms of task allocations, our team basically did everything together with equal contribution to the entire project and performance.

How was communication within the team: pretty good! We used both WeChat and slack for communication, and set group notifications for each task and upcoming meeting so that every person made it on time. 

Step 4 – Images and Videos for Documentation

Our “vintage” Cashier at the cake store:

Beautiful wings we made for Robot Monica …

Part of the remote control sensors that the host and robots wear, we have sensors on the host’s each finger (see finger sensor in one wrist controller) as well.