Have A Cuppa Tea!
–Interaction Lab Final Project
Jean Zhang
Instructor: Eric Parren
CONCEPTION AND DESIGN
This project aimed to give the participants an experience of making a cup of British tea in a playing-house game with great visuals in the form of a hands-on interactive game and also to teach the participants the manners of British tea culture. The participants can use the props within the whole 3D-printed tea set and follow the instructions on the screen to complete the game. With different sensors on the props, the movements of the props can be detected and the game will move on to the later steps. We did quite a lot of research on British tea culture and Luna wrote the script for the game and produced short videos for different steps, showing the process moving on in the game. (The full script is in the appendix.)
In the User Testing session, I found that most of the steps moving on controlled by the stages in the codes worked well, but some of the sensors were too sensitive and some not sensitive enough, and some more interesting interactions could be added. For example, the teapot was useless as it did not bring to another step. Therefore, with the advice of the users and learning from Ninj and Audury’s project, I decided to add a tilt sensor to allow the pouring movement of the teapot and the creamer. I replaced some other sensors to make the sensing of the movements more appropriate and accurate. Also, we found the suggestion of adding more sound effects and having the lines on the screen read out loud with a British accent very helpful. I’ve also simplified and removed some of the unnecessary steps in the flowchart in favor of more concise and clear steps after the session. For example, after a failure, the participants can proceed to the next step by pressing a certain button instead of starting over.
FABRICATION AND PRODUCTION
Fabrications
For the props, with the help of Thingiverse.com, I found the pretty tea set model and printed them with the 3D printer. It took more than 40 hours in total to print the whole tea set, but it was a significant result to be seen. Getting rid of the support printed required great patience, and I learned how to use the BambuStudio. Great thanks to Dalin, Freddie, and the 3D printer! And Luna then painted them into a more adorable tea set with colors.
To attach the sensor to the prop, Dalin taught me to use a drill to create small holes in the tea set to get the wires through. (Using drill is really a stress-relieving work :))
Also, laser cutting was used in the project. For the tray of tea sets, Luna did the design part, and I did calculations according to the size of the props and got it cut. Also, I used makercase.com to design the box to hide the electronics. I drew some circles on the board and carved some words on the board. With the woodworking glue and hot glue, I assembled the box and the tray. Also, I cut some holes on the board to give the way for the wires to get out.
Cuttle Design:
Electronics
For the electronics part, I tested different sensors for detecting different movements upon the props.
I used sensors including light sensors, touch sensors, tilt sensors, force sensors, and an ultrasonic distance sensor. I tried these sensors for different functions needed in the game. For example, I originally used a light sensor to detect if the teabag was placed in the teacup. However, it turned out that it was kind of hard to control the data as it was the analog input, and the teabag was not always put in the perfect position. Therefore, I chose the magnet sensor as the digital reading is easier and I can stick some magnets to the teabag (which was also laser-cut). Also, the other sensors were used for detecting different movements. (The diagram of the circuit and the comments in the codes show details.)
Sensor Testing:
Also, I met the problem of the tilt sensor being too sensitive and there was chance that when there is no relevant movement and then other factors intervene and lead to the wrong detection. Gohai helped me with this problem by providing me the circuit used in the recitation in the previous semesters and taught me how to add the capacitors in the circuit to make the tilt sensor work more stable.
To make sure the wires are long enough for being connected to the props, I soldered the sensors to longer wires.
I put a lot of pull-down sensors to make sure the sensing work well. Here is the simplified diagram for the circuits. (Some of the sensors cannot be found in Tinkercad, so I simply drew them by myself.)
Codes
The coding part is the most important thing making the whole project work. For the different steps, I used stage function to show the steps. With the sensors detected and the serial communication, I used if structure to lead the stages into the later steps. With the basic function of playing movies, the certain video would be played in the certain stage. With the int of stage, boolean of stageX and changing the stage in the process of the game, the problem of the errors recur incorrectly or won’t jump correctly from one step to the next. I met quite a lot problems in the process of coding, and I would like to list a few most interesting and major problems I met as follows.
Problem 1: Sounds
As I input the videos in setup, the sounds are automatically played at the beginning and won’t be played again. The visuals also get stuck due to this problem of the videos looping in the very first place. Therefore, I replaced those with putting only one loop before the certain video have to be played.
When the audio goes crazy:
Problem 2: Time count
Time count was used a lot in this design. For the most important part that the participants should remove the teabag from the teacup, the time count would decide which stage the game is going to enter. With Kevin and Andy’s help, I revised my prevVal int and make the time count in the stage statement instead of letting it count itself and use the data afterwards. In this way, the time is only counted once, and the results can lead the game into the other stages, moving on in the steps of interactions.
if (stage == 3) { if (D1Looped == false) { D1.loop(); D1Looped = true; } image(D1, 0, 0, width, height);//if water poured, start time count C1.stop(); val1 = arduino_values[7]; if (stage3 == true) { //if water poured, start timecount if (waterfirst==false) { startTime1 = millis(); waterfirst = true; } } if (prevVal1 ==1 && val1 == 0) { print("Teabag Removed after "); println(millis() - startTime1); timecount1 =millis()-startTime1; //timecount for picking teabag out if (timecount1 > 10000 && timecount1 < 12000) { stage = 7; stage7 = true;//if succeed in timecount, add milk } if (timecount1 > 12000) { stage = 8; stage8 = true;//slow } if (timecount1 < 10000) { stage = 9; stage9 = true; } } prevVal1 = val1; }
Problem 3: Stages
The stages must be the most important part in the game like this with different steps. I initailly put the if functions that are related to the sensors in Arduino out of the stage and that makes the sensing and stage defining all at the very beginning. With the help of Gohai, I decided to put these if functions that decide the stages in the previous stages to make sure they are deciding after one certain step is finished. In this way, the process is moving on step by step and the movements in different stages would not intervene each other.
CONCLUSIONS
For the project itself, I believe it achieved the initial goal of giving the audience a great experience of interactive game. There are multiple steps the audience can do (even more than the amount expected) being detected in the game and the interaction varies from each other. In terms of improvement, I think I can include more interesting steps and make the sensing of the movements more accurate. For myself, I did not expect that I can complete such complicated codes (at least they are complex enough for me). I felt the serial communication is really interesting and I could even play a lot more with the sensors and steps in the game for improvement if time permitted.
DISASSEMBLY
I returned all of the items I borrowed from ER. And the other stuff are all from the kits, therefore I just brought the project home 🙂
FULL VIDEO DOCUMENTATION
(cr. Jean)
Product Trailer (cr. Luna)
APPENDIX
Full Scripts (Early version):
START: Have a cuppa tea ->a1
A1: Boil the water. Kettle or microwave oven? Kettle->b1; Microwave oven->a2
A2: For crying out loud, never microwave your brew. It will taste like anything but tea. (Failed)
B1: Bloody brilliant choice. Warm the cup and teapot by swirling boiling water around them. ->c1
C1: Put water and teabags into the teapot. Loose tea is recommended though. ->d1
D1: Wait for 5 minutes. Watch out for the time…you can still be faffing around though.
In time->e1; Fast->d2; Slow->d3
D2: Oh…Hold your horses! (Failed)
D3: Too slow! (Failed)
E1: Add the milk from the creamer or milk jug, whatever you call it. ->f1
F1: Though it’s horses, of course, add some sugar if you’d like to. ->g1
G1: Stir the tea gently.
Proper->h1; Improper->g2
G2: Oh…Naff noise. It should be in a back-and-forth, up-and-down manner, and remember to keep the spoon away from touching the cup. (Failed)
H1: Hunky-dory! One more step. Place the spoon back to the saucer.
Proper->i1; Improper->h2
H2: Lad…Stop throwing a wobbly and leave it out. Put it behind the cup. (Failed)
I1: Take the cup and sip the tea.
Proper->fin; Improper->i2
I2: Mate, you are throwing a spanner in the works again…No cradling the cup and raising your little finger. Use your thumb and forefinger to hold, never hook them under the handle. (Failed)
FIN: That’s a cracking cuppa, innit? Enjoy it, and Cheers!
References
First-step research (British tea culture):
https://www.oxfordinternationalenglish.com/dictionary-of-british-slang/
https://twinings.co.uk/blogs/news/tea-etiquette
https://www.huffpost.com/entry/british-tea-etiquette-how-to-drink-it-downton-abbey-style_b_8656634
https://www.wikihow.com/Hold-a-Teacup
Narrations: https://ondoku3.com/en/
BGMs: https://pixabay.com/music/
Box Design: https://en.makercase.com/#/basicbox
3D Printing (Tea set): https://www.thingiverse.com/thing:5913112
Props
Additional Visuals
IMA Show🤩
Full Codes
Arduino:
const int trigPin = 9;
const int echoPin = 10;
int val;
int prevVal;
long startTime;
float duration, distance;
void setup() {
pinMode(trigPin, OUTPUT);
pinMode(echoPin, INPUT);
Serial.begin(9600);
}
void loop() {
// to send values to Processing assign the values you want to send
// this is an example:
int sensor0 = digitalRead(13); //button, control if the game starts
int sensor1 = analogRead(A1); //light, if the teabag is placed correctly
int sensor2 = digitalRead(12); //tilt1, teapot movement
int sensor3 = digitalRead(8); //tilt2, creamer movement
int sensor4 = analogRead(A0); //fsr1, if the spoon touch the cup when stir
int sensor5 = digitalRead(2); //touch1, if the spoon is put back to the sauser correctly
int sensor6 = digitalRead(7); //magnet1, putting the teabag to the certain position
int sensor7 = analogRead(A2); //fsr2
int sensor8 = analogRead(A3); //fsr3
int sensor9 = digitalRead(4); //touch2, if the spoon is put back in wrong ways
// int sensor10 = digitalRead(3); //tilt3, teacup movement
digitalWrite(trigPin, LOW);
delayMicroseconds(2);
digitalWrite(trigPin, HIGH);
delayMicroseconds(10);
digitalWrite(trigPin, LOW); //ultrasonic
duration = pulseIn(echoPin, HIGH);
distance = (duration * .0343) / 2;
// send the values keeping this format
Serial.print(sensor0);
Serial.print(","); // put comma between sensor values
Serial.print(sensor1);
Serial.print(","); // put comma between sensor values
Serial.print(sensor2);
Serial.print(","); // put comma between sensor values
Serial.print(sensor3);
Serial.print(","); // put comma between sensor values
Serial.print(distance);
Serial.print(","); // put comma between sensor values
Serial.print(sensor4);
Serial.print(","); // put comma between sensor values
Serial.print(sensor5);
Serial.print(",");
Serial.print(sensor6);
Serial.print(",");
Serial.print(sensor7);
Serial.print(",");
Serial.print(sensor8);
Serial.print(",");
Serial.print(sensor9);
// Serial.print(",");
// Serial.print(sensor10);
Serial.println();
delay(20);
// too fast communication might cause some latency in Processing
// this delay resolves the issue
// end of example sending values
}
Processing:
import processing.serial.*; import processing.video.*; Serial serialPort; int NUM_OF_VALUES_FROM_ARDUINO = 11; /* CHANGE THIS ACCORDING TO YOUR PROJECT */ /* This array stores values from Arduino */ int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO]; Movie start; Movie A1; Movie A2; Movie B1; Movie C1; Movie D1; Movie D2; Movie D3; Movie E1; Movie F1; Movie G1; Movie G2; Movie H1; Movie H2; Movie I1; Movie I2; Movie fin; boolean jumped = false; boolean stage1 = false; boolean stage2 = false; boolean stage3 = false; boolean stage4 = false; boolean stage5 = false; boolean stage6 = false; boolean stage7 = false; boolean stage8 = false; boolean stage9 = false; boolean stage10 = false; boolean stage11 = false; boolean stage12 = false; boolean stage13 = false; int stage = 0; int val1; int prevVal1; int val2; int prevVal2; int val3; int prevVal3; long startTime1; float timecount1; int startTime2; float timecount2; int startTime3; float timecount3; int startTime4; float timecount4; int startTime5; float timecount5; boolean firstVideoPlayed = false; void setup() { fullScreen(); size(1000, 700); printArray(Serial.list()); // put the name of the serial port your Arduino is connected // to in the line below - this should be the same as you're // using in the "Port" menu in the Arduino IDE serialPort = new Serial(this, "/dev/cu.usbmodem1101", 9600); start = new Movie(this, "start.mp4"); //don't play audio //start.loop(); A1 = new Movie(this, "A1.mp4"); //A1.loop(); A2 = new Movie(this, "A2.mp4"); //A2.loop(); B1 = new Movie(this, "B1.mp4"); //B1.loop(); C1 = new Movie(this, "C1.mp4"); //C1.loop(); D1 = new Movie(this, "D1.mp4"); //D1.loop(); D2 = new Movie(this, "D2.mp4"); //D2.loop(); D3 = new Movie(this, "D3.mp4"); //D3.loop(); E1 = new Movie(this, "E1.mp4"); //E1.loop(); F1 = new Movie(this, "F1.mp4"); //F1.loop(); G1 = new Movie(this, "G1.mp4"); //G1.loop(); G2 = new Movie(this, "G2.mp4"); //G2.loop(); H1 = new Movie(this, "H1.mp4"); //H1.loop(); H2 = new Movie(this, "H2.mp4"); //H2.loop(); I1 = new Movie(this, "I1.mp4"); //I1.loop(); I2 = new Movie(this, "I2.mp4"); //I2.loop(); fin = new Movie(this, "fin.mp4"); //fin.loop(); } boolean startLooped = false; boolean A1Looped = false; boolean A2Looped = false; boolean B1Looped = false; boolean C1Looped = false; boolean D1Looped = false; boolean D2Looped = false; boolean D3Looped = false; boolean E1Looped = false; boolean F1Looped = false; boolean G1Looped = false; boolean G2Looped = false; boolean H1Looped = false; boolean H2Looped = false; boolean I1Looped = false; boolean finLooped = false; boolean waterfirst=false; boolean stirfirst=false; boolean pourfirst=false; boolean milkfirst=false; void draw() { println("stage: " + stage); getSerialData(); if (start.available()) { start.read(); } if (A1.available()) { A1.read(); } if (A2.available()) { A2.read(); } if (B1.available()) { B1.read(); } if (C1.available()) { C1.read(); } if (D1.available()) { D1.read(); } if (D2.available()) { D2.read(); } if (D3.available()) { D3.read(); } if (E1.available()) { E1.read(); } if (F1.available()) { F1.read(); } if (G1.available()) { G1.read(); } if (G2.available()) { G2.read(); } if (H1.available()) { H1.read(); } if (H2.available()) { H2.read(); } if (I1.available()) { I1.read(); } if (I2.available()) { I2.read(); } if (fin.available()) { fin.read(); } if (stage == 0) { if (startLooped == false) { start.loop(); startLooped = true; } image(start, 0, 0, width, height);//play start if (arduino_values[0] == 1) { stage = 1; stage1 = true; }//if button pressed, choose k/m } if (stage == 1) { if (A1Looped == false) { A1.loop(); A1Looped = true; } image(A1, 0, 0, width, height);//if button pressed, choose k/m start.stop(); if (key == 'm' || key == 'M') { if (A2Looped == false) { A2.loop(); A2Looped = true; } image(A2, 0, 0, width, height); A1.stop(); }//if choose microwave, fail the game, tell on the screen to press K if (key == 'k' || key == 'K') { if (B1Looped == false) { B1.loop(); B1Looped = true; } image(B1, 0, 0, width, height); A1.stop(); A2.stop(); }//if choose kettle, go on to put teabag if (arduino_values[7] ==1) { println("Teabag detected"); stage = 2; stage2 = true; }//if teabag put to the certain position in the cup, start to pour water } if (stage == 2) { if (C1Looped == false) { C1.loop(); C1Looped = true; } image(C1, 0, 0, width, height); B1.stop(); val2 = arduino_values[2]; if (prevVal2 == 0 && val2 == 1) { if (pourfirst==false) { startTime4 = millis(); pourfirst = true; println("water start pouring"); } } if (prevVal2 == 1 && val2 == 0) { timecount4=millis()-startTime4; print("water pouring stops after"); println(timecount4); } if (timecount4 > 5000) { stage = 3; stage3 = true; }//if water poured and pot put back, start time count prevVal2=val2; } if (stage == 3) { if (D1Looped == false) { D1.loop(); D1Looped = true; } image(D1, 0, 0, width, height);//if water poured, start time count C1.stop(); val1 = arduino_values[7]; if (stage3 == true) { //if water poured, start timecount if (waterfirst==false) { startTime1 = millis(); waterfirst = true; } } if (prevVal1 ==1 && val1 == 0) { print("Teabag Removed after "); println(millis() - startTime1); timecount1 =millis()-startTime1; //timecount for picking teabag out if (timecount1 > 10000 && timecount1 < 12000) { stage = 7; stage7 = true;//if succeed in timecount, add milk } if (timecount1 > 12000) { stage = 8; stage8 = true;//slow } if (timecount1 < 10000) { stage = 9; stage9 = true; } } prevVal1 = val1; } if (stage == 7) { if (E1Looped == false) { E1.loop(); E1Looped = true; } image(E1, 0, 0, width, height);//if succeed in timecount, add milk D1.stop(); D2.stop(); D3.stop(); val3 = arduino_values[3]; if (prevVal3 == 0 && val3 == 1) { if (milkfirst==false) { startTime5 = millis(); milkfirst = true; println("milk start pouring"); } } if (prevVal3 == 1 && val3 == 0) { timecount5=millis()-startTime4; print("milk pouring stops after"); println(timecount5); } if (timecount5 > 5000) { stage = 4; stage4 = true; }//if water poured and pot put back, start time count prevVal3=val3; } if (stage == 8) { if (D3Looped == false) { D3.loop(); D3Looped = true; } image(D3, 0, 0, width, height);//too slow D1.stop(); if (key == ENTER) { stage = 7; stage7 = true; } } if (stage == 9) { if (D2Looped == false) { D2.loop(); D2Looped = true; } image(D2, 0, 0, width, height);//too fast D1.stop(); if (key == ENTER) { stage = 7; stage7 = true; } } if (stage == 4) { if (F1Looped == false) { F1.loop(); F1Looped = true; } image(F1, 0, 0, width, height);//if creamer moves, go on to add sugar E1.stop(); if (arduino_values[4] < 7) { stage = 5; stage5 = true; }//if sugar added, go on to stir } if (stage == 5) { if (G1Looped == false) { G1.loop(); G1Looped = true; startTime2 = millis(); } image(G1, 0, 0, width, height);//if sugar added, go on to stir F1.stop(); if (arduino_values[5] > 4) { stage = 13; stage13 = true; } else if (arduino_values[8] > 4) { stage = 13; stage13 = true; } else if (arduino_values[9] > 4) { stage = 13; stage13 = true; } if (millis() - startTime2 > 9500) { stage = 10; stage10 = true;//stir ends, go on to put down spoon } } // } //backup plan for getting to the next step //if (arduino_values[7] == 1) { // stage = 10; // stage10 = true; //}//if time up (use magnet sensor here), go on to put the spoon // } //else if(arduino_values[5] == 0){ // image(H1, 0, 0, width, height); //} ////if the spoon doesn't touch the cup, continue to put back the spoon if (stage == 13) { if (G2Looped == false) { G2.loop(); G2Looped = true; } image(G2, 0, 0, width, height); G1.stop(); //if the spoon touch the cup(pressure on the sensor-fsr), fail the game if (key == 'g' || key == 'G') { stage = 10; stage10 = true; } } if (stage == 10) { if (H1Looped == false) { H1.loop(); H1Looped = true; } image(H1, 0, 0, width, height);//if stir well, put back the spoon G1.stop(); G2.stop(); if (arduino_values[6]==1) { stage = 6; stage6 = true;//put back spoon correctly } else if (arduino_values[10]==1) { stage = 11; stage11 = true;//put back spoon incorrectly } } if (stage == 11) { if (H2Looped == false) { H2.loop(); H2Looped = true; } image(H2, 0, 0, width, height);//put the spoon wrong, fail G1.stop(); G2.stop(); H1.stop(); if (key == 's' || key == 'S') { stage = 6; stage6 = true; } } if (stage == 6) { if (I1Looped == false) { I1.loop(); I1Looped = true; startTime3 = millis(); } image(I1, 0, 0, width, height);//put the spoon correctly, move on to drink G1.stop(); G2.stop(); H1.stop(); H2.stop(); if (millis() - startTime3 > 8000) { stage = 12; stage12 = true;//teacup movement, drink tea } } if (stage == 12) { if (finLooped == false) { fin.loop(); finLooped = true; } image(fin, 0, 0, width, height);//drink tea, to the end I1.stop(); } //still have to make the whole thing work step by step } void getSerialData() { while (serialPort.available() > 0) { String in = serialPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII if (in != null) { print("From Arduino: " + in); String[] serialInArray = split(trim(in), ","); if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) { for (int i=0; i<serialInArray.length; i++) { arduino_values[i] = int(serialInArray[i]); } } } } }
Credits to
Andy Garcia
Eric Parren
Gottfried Haider
Kevin Xu
Shengli Wen
Dalin Zang
Rachel Li
Amelia Shao
Anya Zhukova
Freddie Yang
Sophia Wan
Yanran Deng
Tina Liu
Ting Yang
Lareina Chen
Matsa Sun
Greatest gratitude also to all the friends and testers who helped or gave comments during the whole process!!!
