Author: mlr565

Documentation of “A Typical Day at NYUSH”

• CONCEPTION AND DESIGN:

When conceptualizing the project, I started with my definition of interaction:  a cycle of communication between two or more actors which involves “listening,” processing, and responding.                                                                                    The inspiration for this definition was based mostly on Crawford’s “What Exactly is Interactivity?” (in that interaction comes on a spectrum) and Igoe and O’Sullivan’s “Introduction to Physical Computing”   (in that the future of interaction should focus on how the project “sees” us). This research leads me to prefer projects which use language and dialogue — that is the way humans naturally communicate, after all.

Knowing that I wanted to work with language, I found myself inclined to work with interactive fiction (thinking the “Choose Your Own Adventure” books or “Black Mirror: Bandersnatch”). I had already been playing around in Processing for a while in order to create a small game in my free time, but after finishing my final project research I decided I wanted to work with the format I was creating to make a project which would emulate interactive fiction while adding another element: a third choice.

The format I decided to adapt for the final game (note the textbox and the strobe text, which both appear in the final project):

I knew that I wanted to add a twist ending (a third option), but I knew that I wanted users to have to explore and discover it on their own. I also knew that I wanted to have a physical button setup — this was how I decided what the final project would look like. My laptop would sit in the middle, with the buttons sitting next to it. I wanted the setup to be deceptively simple, so I decided to keep this part of the game relatively plain. That way, I had more time to focus on what I felt was the most important part of the game: the code. 

• PRODUCTION: THE CODE

This class is the first time I have ever done coding: for that reason I knew I had to start very early and dedicate a lot of time to the project. I discovered a lot along the way: for example, I did not realize that I could run multiple tabs in processing until I was about halfway through the game! 

Starting with the overall structure of the code, I created a series of menus. This menu system works with three variables: the first is the variable “menu”, which is simply a number. The second is the boolean “pass”. “pass” is only true in certain parts of the game, which is what allows the game to control when the user can press a button to change the menu. Finally, the game works with both serial input (in the form of button presses) and with keys. The original code used keyReleased to switch menus, this way I could work on the code whenever I had free time (for example, when I was first playing with the code I discovered how to setup the menu system when I was eating lunch!). To incorporate the buttons, then, I simple equated the buttons to a key. For example:

//if (sensorValues[0] == 0 && pass == true){
// keyReleased();
// key = 'y';
//}

As I created the code, I realized that I would quickly get lost if I did not have some guide to keep myself clear on the order of the menus. For that reason I started using pen and paper to draft out which menus would go where.  I also used this notebook to sketch what I roughly imagined the plot structure to be. These drawings are inserted here:

I knew that I wanted to add sound in the game. Unfortunately, I was unsuccessful in adding it. In terms of background music, I could not add the file without significantly lagging processing. I suspected that this was due to the file format (mp3), and while presenting the project Marcela confirmed that this may be the case. In terms of sound effects, I had a strange issue: rather than playing a sound file once, the program would play the file over and over again very quickly. Given that I only had a short amount of time left, I decided to cut this element of the project. If I had more time to continue working on the project, this would be the next issue I would like to tackle, as I feel that without sound the game feels a bit shallow and incomplete. 

• PRODUCTION: PHYSICAL CONSTRUCTION

I divide this section into two parts: First, the fabrication of the boxes. Second, the operation of the buttons. 

1. The boxes

Since I had a partner in my midterm project, this was my first time fabricating anything on my own. I would like to thank Andy here for all of his help, and to commend him on his patience and effort in teaching me how to fabricate. The size of my boxes was off, but rather than fixing it by himself he went to the effort to explain each part of the process to me. I feel that if I were to fabricate again, I would be confident in doing it myself. 

My final boxes were quite simple. I did have two problems: first, since the boxes lacked a bottom and a back, they were not stable enough to handle being manipulated by an audience. For that reason I had to hot glue the edges. The hole on top was also too large for the button. To fix this problem, I painted some cardboard to insert the button into. This was then hot glued to the inside of the box to hold the button inside of the hole. Below are pictures of the boxes, the buttons placed inside the cardboard, and the inside of the boxes to show the hot glue:

2. The buttons

In terms of the button setup, I learned something very useful. I was having trouble getting my buttons to register using a resistor, so I was taught to use the pullup function built into the arduino. In this way, I was able to reduce what would have been a multiple-piece circuit to only a few wires. The only extra work that was required was remembering to reverse the code (from using 1 to test if the button was pressed to using 0 to test if the button was pressed). 

• USER TESTING

User testing showed me that people were generally happy with the concept, but there were some minor fixes. I received a lot of comments about adding sound or a mechanism to make the dialogue switch. I did try to add both of these elements, but I think the menu setup I created needs to be fixed. In terms of making the dialogue go faster, I think the timer I used could be better. Rather than using a timer, perhaps I could have used a typewriter effect on the text? But I wonder if this would eventually have the same issue. In the end, I decided having the dialogue be a bit too slow was preferable to having the dialogue be a bit too fast. My roommate is a slow reader, so I sat down with her and went through the entire game to make sure she could read it. I was especially certain while creating the game that I wanted every person to be able to read it — there would be no point if the game was so quick that some people missed out. 

The big issue I came across with user testing was that after a while the buttons would switch functions. I actually thought that the problem had resolved itself on the day we presented in class: when I setup and tested it, the buttons worked fine. Yet after letting the project sit for the period of the class, the buttons switched themselves. Now that I know it is the time element that switches the buttons, I would have liked to sit down with the game and figure out when the switch occurs. 

Below are my notes from user testing: although they are messy, many of them are about certain menus having broken functions. At the time I did not know that the issue was not the order of the menus, but the buttons themselves:

• CONCLUSION

The goal of my project was to subvert the expectations users have when playing an interactive fiction game. I was pleased to find that during testing, users were delighted with two elements of the game: the first being the familiar setting (since the game was set around our university) and the second being the “twist” third option. Given a chance, I would have loved to make the game even longer and to make the twist happen more organically. In fact, my original code for the game had a whole separate path with two celebrities besides Johnny Depp visiting the user — I cut this idea when I realized that the game would be too long to reasonably play in user testing. Even at its current length, the game is simply too  long to present in a small setting. Given this fact, I am pleased to have make my users laugh while playing along with the game. That moment of exploration and subverting expectations is what I envisioned while thinking about what interactivity meant to me: the exploration and communication are the most important elements in the game. 

That being said, the game has obvious problems: there is a lack of sound, the text speed is nonadjustable, and the buttons often switched themselves. I needed to conduct more user testing before the actually user testing period — that way I would have had more time to catch some of these issues. Unfortunately, I did not do this and had to present what I had. Although I learned a lot, these issues show me that there is still a lot I can work on. As I move on to my next semester at NYUSH, I believe I will take this lesson with me moving forward: If I want my work here to mean something to others, I have to engage more with the opinions of others early on in my work. 

Below I have included clips of the game. In order to show as many menus as efficiently as possible, I have included a playthrough with all answers being “yes”, all answers being “no”, and snippets of special sections which showcase answers for Nick Cage. All files have been heavily compressed, so I apologize for any loss in quality:

Yes only:

No only:

Using “Nick Cage” to beat the game:

Using “no” and “Nick Cage” at the menu screen, respectively:

CODE: ARDUINO

const int yesButton = 7;
const int cageButton = 8;
const int noButton = 9;

void setup() {
Serial.begin(9600);
pinMode(yesButton, INPUT_PULLUP);
pinMode(cageButton, INPUT_PULLUP);
pinMode(noButton, INPUT_PULLUP);
}

void loop() {
int sensor1 = digitalRead(yesButton);
int sensor2 = digitalRead(cageButton);
int sensor3 = digitalRead(noButton);

Serial.print(sensor1);
Serial.print(","); // put comma between sensor values
Serial.print(sensor2);
Serial.print(",");
Serial.print(sensor3);
Serial.println(); // add linefeed after sending the last sensor value

delay(100);
}

CODE: PROCESSING
*Note: all images were taken from public sources. The code for the serial communication was taken from the interaction lab folder.

The code is divided by tabs, which I will order here:

PROCESSING TAB: global_variables

boolean pass = false;
int menu;
color bgcl = color(80, 85, 92);
int time;
int filetime;

PImage figure;
PImage lost;
PImage hours;
PImage days;
PImage skip;
PImage library;
PImage gates;
PImage jenni;
PImage flipjenni;
PImage ghostjenni;
PImage susjenni;
PImage miranda;
PImage hall;
PImage ghostmsg;
PImage johnny;
PImage cage;

PFont f; //dialogue
PFont i; //italics dialogue
PFont c; //ghost font
PFont n; //note
PFont s;

PROCESSING TAB: Strings

String[]dlg = {"Oh, there you are. Let's head to class now.", //Part 1: intro to menu 9, where you have the option to split up
"Dope, let me look at this.",
"I thought so.",
"I was up all night doing the homework...",
"...you know we had homework, right?",
"You're not saying anything.", //5
"Say...",
"Well, how do you feel about skipping today?",
"...", //8
"Were you drunk when you did this?",
"These are all wrong.", //10
"I'm kidding...but they really are pretty bad.",
"Maybe we could just skip today?",
"Oh. I was really hoping we could hang out.",
"But that's okay.",
"Have fun being a loser or something.", //15
"Awesome!", //16
"Let's order bubble tea. We can use my phone.",
"What kind do you want?",
"Yes? That wasn't a yes or no question, haha...",
"I'll just get you what I get.", //20
"No? That wasn't a yes or no question, haha...",
"I'm going to drop my bag off.",
"Do you mind staying here?", //Bridges off to spooky route
"I'll go find my locker.",
"I'll meet you by the front gate to get our tea!", //25
};

String[]dlg2 = {"Yes? As in you do mind?",
"Well, uh, I guess you could come with. Let's go.",
"Weird, I thought my locker was over here.",
"Woah!",
"I haven't seen this part of the AB before...",
"!", //5
"I think I heard something back there, want to check it out?",
"Honestly,",
"who knew NYU had a place like this?",
"Wait, let me get out my phone!",
"Hey,", //10
"what's the look for?",
"Huh? You want to leave?",
"But it could be cool!",
"We already skipped anyway--",
"What's that?", // 15
"A ghost?",
"I don't see anything...",
"Oh, there's something on the ground here.",
"Explore?",
"What do you think this means?", //20,
"It's a piece of paper.",
"It's got something written on it,",
"Give me a real answer for once!",
"I mean, you've been acting like this all day and it's just...",
"Sorry--", //25
"I'm a little tired, uh...",
"I think I'm just going to go find my locker.",
"I'll catch you later or something.",
};

String[] dlg4 = {
"You there!",
"I'm here to check up on you.",
"Shouldn't take too long...",
"Trust me?",
"Good.",
"Here's a question:", //5
"Have you figured it out yet?",
"I'll tell you something useful, then.",
"Are you really okay with only two buttons?",
"Explore. You may have more options than you think.",
"That's all -- can't give you anything too easy, after all.", //10
"It's up to you to find what you need.",
"Good luck, friend.",
"I suppose I can't blame you...", //13
"Oh? Prove it.",
"You clearly haven't figured anything out.", //15
"Assuming you aren't just playing games...",
"I'll help you out, anyway."
};

String[] dlg5 = {
"Ha!",
"Pretty daring of you to be hanging around like this.",
"You're more shameless than I thought...",
"I know that you stole our bubble tea to have all to yourself.",
"Just admit it, you're caught!",
"Wait -- really?", //5
"I mean, uh,",
"I knew you would confess!",
"Ha ha ha!",
"'No?' That's it?",
"Don't you have anything more to say for yourself?", //10
"This is ridiculous.",
"I'm calling public safety on you.",
"Spit it out, then!",
};

String[] dlgnc = {
"I don't know what you're talking about." ,
"Huh? Uh...",
"Ah, very good.",
"I'll take my leave. Also...",
"Be ready, because here comes trouble.",
//////////////end of game
"W-what did you say?", //5
"How could you have seen through my disguise?",
"Yes. It was I, Nick Cage.",
"I had planned all along to blame you for taking the tea in order to enjoy it all myself.",
"in order to enjoy it all myself.",
"Yes, you've foiled my plan for now...", //10
"But you'll never catch me!!!",
"Hey!",
"What's that weird look for?",
"Haha, making no sense as usual!",
"I grabbed our tea, let's go!", //15
};

String[] names = {
"Miranda", //0
"Johnny", //1
"Leo", //2
"Cage", //3
"???????",
"Jenni?",
};

String[]fwall = {
"Tap the screen to try again.",
"Really?",
"Well, it's up to you.",
"Tap the screen if you change your mind.",
"Jenni",
"bang!", //5
"YOUR FRIEND GOT POSSESSED.",
"What did you expect?",
"Maybe don't hang around the scary hallway.",
"Here's a reward for your patience: lift the button.",
"Seems like you annoyed her.", //10
"At least you ordered tea, remember?",
"Want to go get it?",
"Well, you've made it to a bad end.", //13
"Who would have thought you would", //14
"order bubble tea and just leave it?", //15
"What a crazy kid you are.",
"Tap the screen to skip class.", //17
"Tap the screen to go claim what is yours.",
"The tea should come if you wait here a while...", //19
"Tap the screen to get out of there!",
"You were kidnapped by Miranda Cosgrove.", //21
"Didn't your mother tell you not to trust strangers?",
"Tap the screen to tell her you're not interested.",
"You're alone again...or are you?", //24
"Uh oh...who's that coming now?", //25
"Way to play yourself.",
"You know, in life you can't always just do",
"what other people say.",
"Tap the screen to stick up for yourself!", //29
"Is this the end...?", //30
"Now is the time to",
"explore",
"some new ideas.",
"To",
"pick up", //35
"the slack, if you will.",
"Tap the screen to defend yourself!",
"Weird..."
};

String[] fwall2 = {
"Woah, there.",
"This is just the start screen,",
"let's not get ahead of ourselves!",
"THE END",
"Thanks for playing!",
"Tap the screen to reset.", //5
};

PROCESSING TAB: misc_draw_functions

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||Title screen
void draw_title_screen() {
int red = int(random(255));
int green = int(random(255));
int blue = int(random(255));
color strobe = color(red, green, blue);

fill(87, 6, 140);
stroke(255, 255, 255);
strokeWeight(20);
rect(0, 0, 1800, 800);
textFont(f, 80);
fill(255, 255, 255);
text("A TYPICAL DAY AT NYUSH", width/4, 400);
textFont(f, 35);
text("WANT TO PLAY?", 50, 600);
fill(strobe);
text("YES", 50, 700);

noStroke();
ellipse(60, 650, 7, 7);
ellipse(80, 650, 7, 7);
ellipse(100, 650, 7, 7);
}

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||Jenni textbox
void draw_jenni_textbox() {
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 650, 1800, 800);
rect(0, 600, 100, 650);
textFont(f, 30);
fill(255, 255, 255);
text(fwall[4], 10, 640);
}

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||Blank textbox
void draw_blank_textbox() {
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 650, 1800, 800);
}

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||Miranda nametag
void draw_miranda_nametag() {
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 600, 135, 650);
textFont(f, 30);
fill(255, 255, 255);
text(names[0], 10, 640);
}

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||Johnny Depp nametag
void draw_johnny_nametag() {
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 600, 130, 650);
textFont(f, 30);
fill(255, 255, 255);
text(names[1], 10, 640);
}

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||Jenni? nametag
void draw_susjenni_nametag() {
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 600, 135, 650);
textFont(f, 30);
fill(255, 255, 255);
text(names[5], 23, 640);
}

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||Nick Cage nametag
void draw_cage_nametag() {
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 600, 100, 650);
textFont(f, 30);
fill(255, 255, 255);
text(names[3], 15, 640);
}

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||???? nametag
void draw_anon_nametag() {
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 600, 130, 650);
textFont(f, 30);
fill(255, 255, 255);
text(names[4], 12, 640);
}

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||Triangles prompt
void draw_prompt() {
int red = int(random(255));
int green = int(random(255));
int blue = int(random(255));
color strobe = color(red, green, blue);
fill(strobe);
noStroke();
triangle(20, 670, 20, 690, 40, 680);
triangle(45, 670, 45, 690, 65, 680);
triangle(70, 670, 70, 690, 90, 680);
}

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||"Nothing to say for yourself" bad end (menu 30)
void menu_30_end() {
fill(87, 6, 140);
stroke(255, 255, 255);
strokeWeight(20);
rect(0, 0, 1800, 800);
textFont(f, 80);
fill(255, 255, 255);
text(fwall[31], 100, height/2);
text(fwall[33], 1065, height/2);
text(fwall[34], 100, (height/2)+150);
text(fwall[36], 490, (height/2)+150);
textFont(i, 80);
text(fwall[32], 785, (height/2));
text(fwall[35], 210, (height/2)+150);
}

//||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||Jenni "I don't know what you're talking about." lib
void jenni_confused_library() {
image(library, 0, 0);
image(jenni, 0, 100);
draw_jenni_textbox();
textFont(f, 30);
fill(255, 255, 255);
text(dlgnc[0], 20, 740);
}

//||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||Jenni "I don't know what you're talking about." hall
void jenni_confused_hall() {
image(flipjenni, width/4, 100);
draw_jenni_textbox();
textFont(f, 30);
fill(255, 255, 255);
text(dlgnc[0], 20, 740);
}

//|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||draw jenni with textbox
void jenni_blank() {
image(jenni, 0, 100);
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 650, 1800, 800);
rect(0, 600, 100, 650);
textFont(f, 30);
fill(255, 255, 255);
text(fwall[4], 10, 640);
textFont(f, 30);
fill(255, 255, 255);
}

////|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||blank susjenni
void susjenni_blank(){
image(gates, 0, 0);
image(susjenni, 100, 40);
draw_blank_textbox();
draw_susjenni_nametag();
textFont(s, 30);
fill(255, 255, 255);
}

////|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||blank cage
void cage_blank(){
image(gates, 0, 0);
image(cage, 400, 80);
draw_blank_textbox();
draw_cage_nametag();
textFont(f, 30);
fill(255, 255, 255);
}

PROCESSING TAB: setup

void setup() {
setupSerial();

size (1800, 800);
rectMode(CORNERS);
ellipseMode(RADIUS);

jenni = loadImage("jenni.png");
flipjenni = loadImage("flipjenni.png");
ghostjenni = loadImage("ghostjenni.png");
ghostjenni.filter(POSTERIZE, 2);
susjenni = loadImage("susjenni.png");
susjenni.filter(POSTERIZE, 8);
miranda = loadImage("miranda.png");
miranda.resize(miranda.width*2, miranda.height*2);
johnny = loadImage("johnny.png");
cage = loadImage("flipnick.png");
cage.resize(550,900);

library = loadImage("library.jpg");
library.resize(1800, 800);
gates = loadImage("gates.jpg");
gates.resize(1800, 800);
hours = loadImage("spongebob3hours.jpg");
days = loadImage("spongebob3days.jpg");
skip = loadImage("timeskip.jpg");
skip.resize(1800,800);
lost = loadImage("where.jpg");
hall = loadImage("hallway.jpg");
hall.resize(1800, 800);
figure = loadImage("slenderman.png");
figure.filter(BLUR, 2);
ghostmsg = loadImage("ghostmsg.png");

f = loadFont("Georgia-40.vlw");
i = loadFont("Georgia-Italic-48.vlw");
c = loadFont("ColonnaMT-48.vlw");
n = loadFont("BellMTItalic-48.vlw");
s = loadFont("ComicSansMS-48.vlw");

menu = 1;
}

PROCESSING TAB: arduino

import processing.serial.*;

String myString = null;
Serial myPort;

int NUM_OF_VALUES = 3; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);
// WARNING!
// You will definitely get an error here.
// Change the PORT_INDEX to 0 and try running it again.
// And then, check the list of the ports,
// find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----"
// and replace PORT_INDEX above with the index number of the port.

myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
myString = null;

sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), ",");
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}}}}}

PROCESSING TAB: keyreleased

void keyReleased() {
if (key == 'r') {/////////////////////////////////////////////////////////////reset
noTint();
menu = 1;
}

if ((menu == 1 && key == 'y')) {////////////////////////////////////////////////////////////////////////////////|||||||||yes/no menus
menu = 2;
time = millis();
} else if ((menu == 1 && key == 'n')) {
menu = 6;
time = millis();
} else if (((menu == 2 || menu == 33) && key == 'y') && pass == true) {
menu = 3;
time = millis();
} else if (((menu == 2 || menu == 33) && key == 'n') && pass == true) {
menu = 4;
time = millis();
} else if (((menu == 3 || menu == 34 || menu == 4) && key == 'n') && pass == true) {
menu = 5;
time = millis();
} else if (((menu == 3 || menu == 34 || menu == 4 && key == 'y')) && pass == true) {
menu = 7;
time = millis();
} else if (((menu == 7 || menu == 35) && key == 'y') && pass == true) {
menu = 8;
time = millis();
} else if (((menu == 7 || menu == 35) && key == 'n') && pass == true) {
menu = 9;
time = millis();
} else if (((menu == 8 || menu == 9 || menu == 36) && key == 'y') && pass == true) {
menu = 10;
time = millis();
} else if ((menu == 10 && key == 'y') && pass == true) {
menu = 11;
time = millis();
} else if ((menu == 10 && key == 'n') && pass == true) {
menu = 12;
time = millis();
} else if ((menu == 12 && (key == 'y' || key == 'n' || key == 'c')) && pass == true) {
menu = 13;
time = millis();
} else if ((menu == 13 && key == 'n') && pass == true) {
menu = 14;
time = millis();
} else if ((menu == 13 && key == 'y') && pass == true) {
menu = 21;
time = millis();
} else if (((menu == 8 || menu == 9 || menu == 36) && key == 'n') && pass == true) {
menu = 16;
time = millis();
} else if (((menu == 18 || menu == 40) && key == 'y') && pass == true) {
menu = 19;
time = millis();
} else if (((menu == 18 || menu == 40) && key == 'n') && pass == true) {
menu = 20;
time = millis();
} else if ((menu == 21 && key == 'y') && pass == true) {
menu = 22;
time = millis();
} else if ((menu == 21 && key == 'n') && pass == true) {
menu = 27;
time = millis();
} else if ((menu == 22 && key == 'y') && pass == true) {
menu = 28;
time = millis();
} else if ((menu == 22 && key == 'n') && pass == true){
menu = 23;
time = millis();

} else if ((menu == 24 && key == 'y') && pass == true) {
menu = 25;
time = millis();
} else if ((menu == 24 && key == 'n') && pass == true) {
menu = 26;
time = millis();
} else if (((menu == 28 && (key == 'y' || key == 'n')) && pass == true)) {
menu = 29;
time = millis();
} else if ((menu == 26 && key == 'n') && pass == true) {
menu = 30;
time = millis();
} else if ((menu == 26 && key == 'y') && pass == true){
menu = 31;
time = millis();
} else if ((menu == 31 && (key == 'y' || key == 'n')) && pass == true) {
menu = 30;
time = millis();/////////////////////////////////////////////////////////////////////////////////////////////////////||||||nick cage menus
} else if ((menu == 1 && key == 'c') && pass == true) {
menu = 32;
time = millis();
} else if ((menu == 2 && key == 'c') && pass == true) {
menu = 33;
time = millis();
} else if (((menu == 3 || menu == 4) && key == 'c') && pass == true) {
menu = 34;
time = millis();
} else if ((menu ==7 && key == 'c') && pass == true) {
menu = 35;
time = millis();
} else if (((menu == 8 || menu == 9) && key == 'c') && pass == true) {
menu = 36;
time = millis();
} else if ((menu == 10 && key == 'c') && pass == true) {
menu = 37;
time = millis();
} else if ((menu == 18 && key == 'c') && pass == true) {
menu = 40;
time = millis();
} else if (((menu == 21 || menu == 22 || menu == 28) && key == 'c') && pass == true) {
menu = 41;
time = millis();
} else if (((menu == 24 || menu == 26 || menu == 31) && key == 'c') && pass == true) {
menu = 42;
time = millis();
} else if ((menu == 42 && (key == 'y' || key == 'n' || key == 'c')) && pass == true) {
menu = 43;
time = millis();
}}

PROCESSING TAB: mousepressed *note: this function was only for bad ends

void mousePressed() {
if ((mousePressed == true && menu == 6) && pass == true) { //bad end off of start screen
menu = 1;
time = millis();
} else if ((mousePressed == true && menu == 5) && pass == true) { //need to change the prompt here: tap screen to skip class
menu = 7;
time = millis();
} else if ((mousePressed == true && menu == 14) && pass == true) {
menu = 21;
time = millis();
} else if ((mousePressed == true && menu == 11) && pass == true) {
menu = 12;
time = millis();
} else if ((mousePressed == true && menu == 19) && pass == true) {
menu = 20;
time = millis();
} else if ((mousePressed == true && menu == 25) && pass == true) {
menu = 26;
time = millis();
} else if ((mousePressed == true && menu == 30) && pass == true) {
menu = 31;
time = millis();
} else if ((mousePressed == true && menu == 44) && pass == true) {
menu = 1;
time = millis();
}}

PROCESSING TAB: main_draw *note that a chunk of menus at around 15-18 is missing, this used to be extra story points that I deleted to slim the project down

void draw() {

println("Menu: " + menu);

updateSerial();
printArray(sensorValues);

if (sensorValues[0] == 0 && pass == true){
keyReleased();
key = 'y';
}
if (sensorValues[1] == 0 && pass == true){
keyReleased();
key = 'c';
}
if(sensorValues[2] == 0 && pass == true){
keyReleased();
key = 'n';
}
if (sensorValues[0] == 0 && sensorValues[2] == 0){
keyReleased();
key = 'r';
}

/////////////////////////////////////////////////////||||||MENU 1
/////////////////////////////////////////////////////|Start screen.
/////////////////////////////////////////////////////|Y: Go to 2.
/////////////////////////////////////////////////////|N: Go to 6.
/////////////////////////////////////////////////////|C: Go to 32.
if (menu == 1) {
draw_title_screen();
pass = true;

/////////////////////////////////////////////////////||||||MENU 2
/////////////////////////////////////////////////////|"Oh hey, let's head to class." ==> "Did you know we had hw?"
/////////////////////////////////////////////////////|Y: Go to 3.
/////////////////////////////////////////////////////|N: Go to 4.
/////////////////////////////////////////////////////|NC: Go to 33.
} else if (menu == 2) {
pass = false;
image(library, 0, 0);
image(jenni, 0, 100);
draw_jenni_textbox();

//"Stayed up all night on the hw.
if (millis() < time + 3000) {
text(dlg[0], 20, 740);
} else if (millis() < time + 7000) {
text(dlg[3], 20, 740);
} else {
text(dlg[4], 20, 720);
text(dlg[5], 20, 760);
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 3
/////////////////////////////////////////////////////|"Of course I know we had hw" ==> "Maybe we should skip."
/////////////////////////////////////////////////////|Y: Go to 7.
/////////////////////////////////////////////////////|N: Go to 5.
/////////////////////////////////////////////////////|NC: Go to 34.
} else if (menu == 3) {
pass = false;
draw_jenni_textbox();

//Really? Then let me see your answers.
if (millis() < time + 5000) {
text (dlg[1], 20, 740);
pass = false;
} else if (millis() < time + 6000) {
text (dlg[8], 20, 740);
pass = false;
} else if (millis() < time + 11000) {
text (dlg[9], 20, 720);
text (dlg[10], 20, 760);
pass = false;
} else if (millis() < time + 16000 ) {
text (dlg [11], 20, 740);
pass = false;
} else {
text (dlg [12], 20, 740); //Maybe we should skip.
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////MENU 4
/////////////////////////////////////////////////////"No, I forgot we had hw." ==> "Maybe we should skip."
/////////////////////////////////////////////////////Y: Go to 7.
/////////////////////////////////////////////////////N: Go to 5.
/////////////////////////////////////////////////////NC: Go to 34.
} else if (menu == 4) {
pass = false;
draw_jenni_textbox();

//Of course you forgot.
if (millis() < time + 3500) {
text(dlg[2], 20, 740);
} else {
text(dlg[7], 20, 740);
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 5
/////////////////////////////////////////////////////|"Don't feel like skipping." Bad end from 3 and 4.
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 5) {
pass = false;
draw_jenni_textbox();

if (millis() < time + 5000) {
text(dlg[13], 20, 740);
} else if (millis() < time + 10500) {
text(dlg[14],20,720);
text(dlg[15], 20, 760);
} else {
pass = true;
fill(87, 6, 140);
stroke(255, 255, 255);
strokeWeight(20);
rect(0, 0, 1800, 800);

textFont(f, 120);
fill(255, 255, 255);
text("Guess you're a loser now?", 100, height/2);
textFont(f, 80);
fill(255, 255, 255);
text(fwall[17], 100, (height/2) + 100);
}

/////////////////////////////////////////////////////||||||MENU 6
/////////////////////////////////////////////////////|Bad end from menu 1.
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 6) {
pass = false;
fill(87, 6, 140);
stroke(255, 255, 255);
strokeWeight(20);
rect(0, 0, 1800, 800);

textFont(f, 100);
fill(255, 255, 255);
if (millis() < time + 2500) {
text(fwall[1], 100, height/2);
} else if (millis() < time + 4950) {
text(fwall[2], 100, height/2);
} else {
textFont(f, 50);
text (fwall[3], 50, 650);
pass = true;
}
/////////////////////////////////////////////////////||||||MENU 7
/////////////////////////////////////////////////////|"Yeah, let's skip." ==> "Bubble tea?"
/////////////////////////////////////////////////////|Y: Go to 8.
/////////////////////////////////////////////////////|N: Go to 9.
} else if (menu == 7) {
pass = false;
image(library, 0, 0);
image(jenni, 0, 100);
draw_jenni_textbox();

//Awesome! Want to order bubble tea? We can use my phone.
if (millis() < time + 5000) {
text(dlg[16], 20, 720);
text(dlg[17], 20, 760);
} else {
text(dlg[18], 20, 740);
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 8
/////////////////////////////////////////////////////|"Yes/That wasn't a y/n question." ==> "Do you mind staying here?"
/////////////////////////////////////////////////////|Y: Go to 10
/////////////////////////////////////////////////////|N: Go to 16.
/////////////////////////////////////////////////////|NC: Go to 36.
} else if (menu == 8) {
pass = false;
draw_jenni_textbox();

//Yes? This isn't a yes or no question... 19
//I'll just get you a normal one. 20
if (millis() < time + 5000) {
text(dlg[19], 20, 740);
} else if (millis() < time + 9000) {
text(dlg[20], 20, 740);
} else {
text(dlg[22], 20, 720);
text(dlg[23], 20, 760);
pass = true;
draw_prompt();
}
/////////////////////////////////////////////////////||||||MENU 9
/////////////////////////////////////////////////////|"No/That wasn't a y/n question." ==> "Do you mind staying here?"
/////////////////////////////////////////////////////|Y: Go to 10.
/////////////////////////////////////////////////////|N: Go to 16.
/////////////////////////////////////////////////////|NC: Go to 36.
} else if (menu == 9) {
pass = false;
draw_jenni_textbox();

//No? This isn't a yes or no question...
//...
//I'll just get you a normal one.
if (millis() < time + 5000) {
text(dlg[21], 20, 740);
} else if (millis() < time +9000) {
text(dlg[8], 20, 740);
} else if (millis() < time + 12000) {
text(dlg[20], 20, 740);
} else {
text(dlg[22], 20, 720);
text(dlg[23], 20, 760);
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 10
/////////////////////////////////////////////////////|"Yes I mind staying." ==> "Do you want to check it out?"
/////////////////////////////////////////////////////|Y: Go to 11.
/////////////////////////////////////////////////////|N: Go to 12.
/////////////////////////////////////////////////////|NC: Go to 37.
} else if (menu == 10) {
pass = false;
draw_jenni_textbox();

if (millis() < time + 4000) {
text(dlg2[0], 20, 740);
} else if (millis() < time + 8000) {
text(dlg2[1], 20, 740);
} else if (millis() < time + 12000) {
image(hours, 0, 0);
hours.resize(1800, 800);
} else if (millis() < time + 15000) {
image(lost, 0, 0);
lost.resize(1800, 800);
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 650, 1800, 800);
rect(0, 600, 100, 650);
textFont(f, 30);
fill(255, 255, 255);
text(fwall[4], 10, 640);
text(dlg2[2], 20, 740);
} else if (millis() < time + 20000) {
clear();
image(days, 0, -100);
days.resize(1800, 00);
} else if (millis() < time + 23000) {
image(hall, 0, 0);
image(jenni, 0, 100);
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 650, 1800, 800);
rect(0, 600, 100, 650);
textFont(f, 30);
fill(255, 255, 255);
text(fwall[4], 10, 640);
text(dlg2[3], 20, 720);
text(dlg2[4], 20, 760);
} else if (millis() < time + 25000) {
image(hall, 0, 0);
image(flipjenni, width/4, 100);
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 650, 1800, 800);
rect(0, 600, 100, 650);
textFont(f, 30);
fill(255, 255, 255);
text(fwall[4], 10, 640);
text(dlg[8], 20, 740);
} else if (millis() < time + 26500) { //"Bang!"
//play = true;
//if (play == true) {/////////////////////////////////////////////////trying to get sound to work here too
// bang.play();
// play = false;
//}
image(hall, 0, 0);
image(jenni, width/4, 100);
draw_jenni_textbox();
fill(0, 0, 0);
stroke(255, 255, 153);
strokeWeight(5);
rect(1100, 200, 1220, 250);

text(dlg2[5], 20, 740);
textFont(f, 34);
fill(254, 27, 7);
text(fwall[5], 1120, 235);
} else { //"Did you hear that? Let's check it out"
image(hall, 0, 0);
tint(255, 126);
image(figure, 890, 260, figure.width, figure.height);
tint(255, 255);
image(flipjenni, width/4, 100);

fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 650, 1800, 800);
rect(0, 600, 100, 650);
textFont(f, 30);
fill(255, 255, 255);
text(fwall[4], 10, 640);
text(dlg2[6], 20, 740);
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 11 -- BAD END FROM 10
/////////////////////////////////////////////////////|"Yes, let's check it out." ==> BAD END
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 11) {
pass = false;
image(hall, 0, 0);
if (millis() < time + 5000) {
tint(255, 126);
image(figure, 890, 250, (figure.width*1.1), (figure.height*1.1));
noTint();
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[7], 20, 720);
text(dlg2[8], 20, 760);
} else if (millis() < time + 6500) {
tint(255, 126);
image(figure, 890, 255, (figure.width*1.15), (figure.height*1.15));
noTint();
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[7], 20, 720);
text(dlg2[8], 20, 760);
} else if (millis() < time + 7000) {
tint(255, 200);
image(figure, 890, 260, (figure.width*1.2), (figure.height*1.2));
noTint();
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[7], 20, 720);
text(dlg2[8], 20, 760);
} else if (millis() < time + 8000){
tint(255, 200);
image(figure, 800, 275, (figure.width*1.4), (figure.height*1.4));
noTint();
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[7], 20, 720);
text(dlg2[8], 20, 760);
} else if (millis() < time + 9000) {
tint(255, 225);
image(figure, 700, 275, (figure.width*2), (figure.height*2));
noTint();
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[9], 20, 740);
} else if (millis() < time + 11800) {
tint(POSTERIZE, 2);
tint(255, 0, 0, 255);
image(figure, 200, 130, (figure.width*8), (figure.height*8));
noTint();
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[10], 20, 740);
} else if (millis() < time + 14500) {
tint(POSTERIZE, 2);
tint(255, 0, 0, 255);
image(figure, 200, 150, (figure.width*9), (figure.height*8));
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[11], 20, 740);
} else if (millis() < time + 15700) {
tint(POSTERIZE, 2);
tint(255, 0, 0, 255);
image(figure, 160, 150, (figure.width*10), (figure.height*8));
image(ghostjenni, width/4, 100);
draw_jenni_textbox();
fill(0, 0, 0);
rect(5, 601, 99, 649);
} else if (millis() < time + 17000) {
fill(0, 0, 0);
rect(0, 0, 1800, 800);
} else if (millis() < time + 20000) {
textFont(c, 100);
fill(255, 255, 255);
text(fwall[6], width/6, height/4);
} else {
textFont(c, 100);
fill(255, 255, 255);
text(fwall[6], width/6, height/4);
textFont(f, 60);
text(fwall[7], 20, 700);
textFont(f, 30);
text(fwall[8], 610, 700);
text(fwall[20], 20, 760);
pass = true;
}
/////////////////////////////////////////////////////||||||MENU 12
/////////////////////////////////////////////////////|"Let's not check it out." ==> "What do you think it means?"
/////////////////////////////////////////////////////|Y: Go to 13.
/////////////////////////////////////////////////////|N: Go to 13.
/////////////////////////////////////////////////////|NC: Go to 13.
} else if (menu == 12) {
pass = false;
image(hall, 0, 0);
if (millis() < time + 2000) {
tint(255, 126);
image(figure, 890, 250, (figure.width*1.1), (figure.height*1.1));
noTint();
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[12], 20, 720);
text(dlg2[13], 20, 760);
} else if (millis() < time + 7000) {
tint(255, 200);
image(figure, 890, 260, (figure.width*1.2), (figure.height*1.2));
noTint();
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[12], 20, 720);
text(dlg2[13], 20, 760);
} else if (millis() < time + 9000) {
tint(255, 225);
image(figure, 400, 275, (figure.width*6), (figure.height*6));
noTint();
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[14], 20, 740);
} else if (millis() < time + 11800) {
image(figure, 200, 130, (figure.width*8), (figure.height*8));
noTint();
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[15], 20, 740);
} else if (millis() < time + 14500) {
image(figure, 200, 150, (figure.width*9), (figure.height*8));
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[16], 20, 740);
} else if (millis() < time + 15700) {
image(hall, 0, 0);
image(jenni, width/4, 100);
draw_jenni_textbox();
} else if (millis() < time + 17000) {
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[17], 20, 740);
} else if (millis() < time + 22000) {
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[18], 20, 740);
} else if (millis() < time + 28000) {
image(flipjenni, width/4, 100);
draw_jenni_textbox();
text(dlg2[21], 20, 720);
text(dlg2[22], 20, 760);
} else if (millis() < time + 32000) {
image(flipjenni, width/4, 100);
draw_jenni_textbox();
textFont(n, 70);
text(fwall[9], 100, 740);
} else {
image(flipjenni, width/4, 100);
draw_jenni_textbox();
textFont(f, 30);
text(dlg2[19], 20, 720);
text(dlg2[20], 20, 760);
draw_prompt();
pass = true;
}

/////////////////////////////////////////////////////||||||MENU 13
/////////////////////////////////////////////////////|"Can't you start making sense?" ==> "I'm leaving."
/////////////////////////////////////////////////////|Y: Go to 15.
/////////////////////////////////////////////////////|N: Go to 14. (bad end)
/////////////////////////////////////////////////////|
} else if (menu == 13) {
pass = false;
image(hall, 0, 0);
image(flipjenni, width/4, 100);
draw_jenni_textbox();
if (millis() < time + 5000) {
text(dlg2[23], 20, 740);
} else if (millis() < time + 10000) {
text(dlg2[24], 20, 740);
} else if (millis() < time + 15000) {
text(dlg2[27], 20, 740);
} else if (millis() < time + 20000) {
text(dlg2[28], 20, 740);
} else if (millis() < time + 25000) {
image(hall, 0, 0);
draw_jenni_textbox();
fill(0, 0, 0);
rect(5, 601, 99, 649);
fill(255, 255, 255);
text(fwall[10], 20, 720);
text(fwall[11], 20, 760);
} else {
image(hall, 0, 0);
draw_jenni_textbox();
fill(0, 0, 0);
rect(5, 601, 99, 649);
fill(255, 255, 255);
text(fwall[12], 20, 740);
draw_prompt();
pass = true;
}

/////////////////////////////////////////////////////||||||MENU 14
/////////////////////////////////////////////////////|BAD END off 13 (say no to getting the tea)
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 14) {
pass = false;
stroke(255, 255, 255);
strokeWeight(20);
fill(87, 6, 140);
rect(0, 0, 1800, 800);

if (millis() < time + 5000) {
textFont(f, 100);
fill(255, 255, 255);
text(fwall[13], 100, height/2);
} else if (millis() < time + 12000) {
textFont(f, 100);
fill(255, 255, 255);
text(fwall[14], 100, height/2);
text(fwall[15], 100, (height/2)+100);
} else {
textFont(f, 100);
fill(255, 255, 255);
text(fwall[16], 100, height/2);
textFont(f, 60);
text(fwall[18], 50, 700);
pass = true;
}

/////////////////////////////////////////////////////||||||MENU 16
/////////////////////////////////////////////////////|"Do you mind staying here? No" -- this menu transitions directly to
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 16) {
pass = false;
image(library, 0, 0);
image(jenni, 0, 100);
draw_jenni_textbox();
if (millis() < time + 3400) {
text(dlg[16], 20, 740);
} else if (millis() < time + 9600) {
text(dlg[24], 20, 740);
} else if (millis () < time + 14000) {
text(dlg[25], 20, 740);
} else if (millis () < time + 19000) {
image(hours, 0, 0);
hours.resize(1800, 800);
} else {
menu = 21;
time = millis();
}

/////////////////////////////////////////////////////||||||MENU 21
/////////////////////////////////////////////////////|trans from going to get tea ==> "Trust me?"
/////////////////////////////////////////////////////|Y: Menu 27.
/////////////////////////////////////////////////////|N: Menu 22.
/////////////////////////////////////////////////////|NC: Menu 41.
} else if (menu == 21) {
pass = false;
image(gates, 0, 0);
draw_blank_textbox();
textFont(f, 30);
fill(255, 255, 255);
if (millis() < time + 3000) {
text(dlg[8], 20, 740);
} else if (millis() < time + 8000) {
text(fwall[19], 20, 740);
} else if (millis () < time + 11000) {
draw_anon_nametag();
text(dlg4[0], 20, 740);
} else if (millis () < time + 16000) {
image(johnny, 400, 40);
draw_blank_textbox();
draw_johnny_nametag();
text(dlg4[1], 20, 740);
} else if (millis () < time + 20400) {
image(johnny, 400, 40);
draw_blank_textbox();
draw_johnny_nametag();
text(dlg4[2], 20, 740);
} else {
pass = true;
image(johnny, 400, 40);
draw_blank_textbox();
draw_johnny_nametag();
text(dlg4[3], 20, 740);
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 22
/////////////////////////////////////////////////////|"Yes, I'll talk to Johnny Depp." ==> "Have you figured it out yet?"
/////////////////////////////////////////////////////|Y: Menu 28.
/////////////////////////////////////////////////////|N: Menu 23.
/////////////////////////////////////////////////////|NC: Menu 41.
} else if (menu == 22) {
image(gates, 0, 0);
image(johnny, 400, 40);
draw_blank_textbox();
draw_johnny_nametag();
textFont(f, 30);
fill(255, 255, 255);
if (millis() < time + 2600) {
text(dlg4[4], 20, 740);
} else {
pass = true;
text(dlg4[5], 20, 720);
text(dlg4[6], 20, 760);
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 23
/////////////////////////////////////////////////////|"You haven't figured it out yet?" ==> "That's all I can say. Good luck."
/////////////////////////////////////////////////////|This menu transitions directly into menu 24.
/////////////////////////////////////////////////////|
} else if (menu == 23) {
image(gates, 0, 0);
image(johnny, 400, 40);
draw_blank_textbox();
draw_johnny_nametag();
textFont(f, 30);
fill(255, 255, 255);
if (millis() < time + 5000) {
text(dlg4[7], 20, 740);
} else if (millis() < time + 10000) {
textFont(i, 35);
text(dlg4[8], 20, 740);
} else if (millis() < time + 15000) {
textFont(i, 35);
text(dlg4[9], 20, 740);
} else if (millis() < time + 20000) {
textFont(f, 30);
text(dlg4[10], 20, 740);
} else if (millis() < time + 25000) {
textFont(f, 30);
text(dlg4[11], 20, 740);
} else if (millis() < time + 26200) {
textFont(f, 30);
text(dlg4[12], 20, 740);
} else if (millis() < time + 31200) {
image(gates, 0, 0);
draw_blank_textbox();
fill(255, 255, 255);
text(fwall[25], 20, 740);
} else {
menu = 24;
time = millis();
}

/////////////////////////////////////////////////////||||||MENU 24
/////////////////////////////////////////////////////|"Hey!" ==> "Just admit it already."
/////////////////////////////////////////////////////|Y:Menu 25(bad end).
/////////////////////////////////////////////////////|N:Menu 26.
/////////////////////////////////////////////////////|NC: Go to 42.
} else if (menu == 24) {
pass = false;
image(gates, 0, 0);
image(susjenni, 100, 40);
draw_blank_textbox();
draw_susjenni_nametag();
textFont(f, 30);
fill(255, 255, 255);
if (millis() < time + 2800) {
text(dlg5[0], 20, 740);
} else if (millis() < time + 7800) {
text(dlg5[1], 20, 740);
} else if (millis() < time + 12500) {
text(dlg5[2], 20, 740);
} else if (millis() < time + 17600) {
text(dlg5[3], 20, 740);
} else {
pass = true;
text(dlg5[4], 20, 740);
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 25 (BAD END)
/////////////////////////////////////////////////////|"Yes, I stole the tea" Bad end from menu 23
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 25) {
pass = false;
image(gates, 0, 0);
image(susjenni, 100, 40);
draw_blank_textbox();
draw_susjenni_nametag();
textFont(f, 30);
fill(255, 255, 255);
if (millis() < time + 3000) {
text(dlg5[5], 20, 740);
} else if (millis() < time + 6500) {
text(dlg5[6], 20, 740);
} else if (millis() < time + 9000) {
text(dlg5[7], 20, 740);
} else if (millis() < time + 11500) {
text(dlg5[8], 20, 740);
} else if (millis() < time + 14800) {
fill(87, 6, 140);
stroke(255, 255, 255);
strokeWeight(20);
rect(0, 0, 1800, 800);
textFont(f, 100);
fill(255, 255, 255);
text(fwall[26], 100, height/2);
} else if (millis() < time + 22000) {
fill(87, 6, 140);
stroke(255, 255, 255);
strokeWeight(20);
rect(0, 0, 1800, 800);
textFont(f, 80);
fill(255, 255, 255);
text(fwall[27], 100, height/2);
text(fwall[28], 100, (height/2) + 100);
} else {
pass = true;
fill(87, 6, 140);
stroke(255, 255, 255);
strokeWeight(20);
rect(0, 0, 1800, 800);
textFont(f, 80);
fill(255, 255, 255);
text(fwall[27], 100, height/2);
text(fwall[28], 100, (height/2) + 100);
textFont(f, 50);
text(fwall[29], 100, 700);
}

/////////////////////////////////////////////////////||||||MENU 26
/////////////////////////////////////////////////////|"No, I didn't take the tea." ==> "Don't you have anything to say for yourself?"
/////////////////////////////////////////////////////|Y: Menu 31.
/////////////////////////////////////////////////////|N: Menu 30.
/////////////////////////////////////////////////////|NC: Go to 42.
} else if (menu == 26) {
pass = true;
image(gates, 0, 0);
image(susjenni, 100, 40);
draw_blank_textbox();
draw_susjenni_nametag();
textFont(f, 30);
fill(255, 255, 255);
text(dlg5[9], 20, 720);
text(dlg5[10], 20, 760);
draw_prompt();

/////////////////////////////////////////////////////||||||MENU 27
/////////////////////////////////////////////////////|"menu 22: No, I don't trust Johnny Depp." ==> transitions to menu 24 (meeting susjenni)
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 27) {
pass = false;
image(gates, 0, 0);
fill(255, 255, 255);
textFont(f, 30);
if (millis() < time + 5000) {
image(johnny, 400, 40);
draw_blank_textbox();
draw_johnny_nametag();
fill(255, 255, 255);
text(dlg4[13], 20, 740);
} else if (millis() < time + 8000) {
image(johnny, 400, 40);
draw_blank_textbox();
draw_johnny_nametag();
fill(255, 255, 255);
text(dlg4[12], 20, 740);
} else if (millis() < time + 12800) {
draw_blank_textbox();
fill(255, 255, 255);
text(fwall[25], 20, 740);
} else {
time = millis();
menu = 24;
}

/////////////////////////////////////////////////////||||||MENU 28
/////////////////////////////////////////////////////|"menu 23: No, I haven't figured it out." ==> "Oh? Then prove it."
/////////////////////////////////////////////////////|Y: Menu 29.
/////////////////////////////////////////////////////|N: Menu 29.
/////////////////////////////////////////////////////|NC: Go to 41.
} else if (menu == 28) {
pass = true;
image(gates, 0, 0);
image(johnny, 400, 40);
draw_blank_textbox();
draw_johnny_nametag();
fill(255, 255, 255);
textFont(f, 30);
text(dlg4[14], 20, 740);
draw_prompt();

/////////////////////////////////////////////////////||||||MENU 29
/////////////////////////////////////////////////////|menu 28 "prove it" ==> "Enough games, *hint*" transitions to menu 24
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 29) {
image(gates, 0, 0);
image(johnny, 400, 40);
draw_blank_textbox();
draw_johnny_nametag();
textFont(f, 30);
fill(255, 255, 255);
if (millis() < time + 5000) {
text(dlg4[15], 20, 740);
} else if (millis() < time + 10000) {
text(dlg4[16], 20, 740);
} else if (millis() < time + 14000) {
text(dlg4[17], 20, 740);
} else if (millis() < time + 24000) {
textFont(i, 35);
text(dlg4[8], 20, 740);
} else if (millis() < time + 28000) {
textFont(i, 35);
text(dlg4[9], 20, 740);
} else if (millis() < time + 32900) {
textFont(f, 30);
text(dlg4[10], 20, 740);
} else if (millis() < time + 38000) {
textFont(f, 30);
text(dlg4[11], 20, 740);
} else if (millis() < time + 39200) {
textFont(f, 30);
text(dlg4[12], 20, 740);
} else if (millis() < time + 44000) {
image(gates, 0, 0);
draw_blank_textbox();
fill(255, 255, 255);
text(fwall[25], 20, 740);
} else {
menu = 24;
time = millis();
}

/////////////////////////////////////////////////////||||||MENU 30 (BAD END)
/////////////////////////////////////////////////////|bad end from 26 and 31
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 30) {
image(gates, 0, 0);
image(susjenni, 100, 40);
draw_blank_textbox();
draw_susjenni_nametag();
textFont(f, 30);
fill(255, 255, 255);
if (millis() < time + 4500) {
text(dlg5[11], 20, 740);
} else if (millis() < time + 9500) {
text(dlg5[12], 20, 740);
} else if (millis() < time + 13700) {
fill(87, 6, 140);
stroke(255, 255, 255);
strokeWeight(20);
rect(0, 0, 1800, 800);
textFont(f, 80);
fill(255, 255, 255);
text(fwall[30], 100, height/2);
} else if (millis() < time + 17000) {
menu_30_end();
} else {
pass = true;
menu_30_end();
textFont(f, 50);
text(fwall[37], 100, 700);
}

/////////////////////////////////////////////////////||||||MENU 31
/////////////////////////////////////////////////////|menu 26 "Yes, I have something to say for myself." ==> "Spit it out!"
/////////////////////////////////////////////////////|Y: menu 30
/////////////////////////////////////////////////////|N: menu 30
/////////////////////////////////////////////////////|NC: Go to 42.
} else if (menu == 31) {
pass = true;
image(gates, 0, 0);
image(susjenni, 100, 40);
draw_blank_textbox();
draw_susjenni_nametag();
textFont(f, 30);
fill(255, 255, 255);
text(dlg5[13], 20, 740);
draw_prompt();

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////nc options
/////////////////////////////////////////////////////||||||MENU 32
/////////////////////////////////////////////////////|If user presses n.c. from start screen
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 32) {
fill(87, 6, 140);
stroke(255, 255, 255);
strokeWeight(20);
rect(0, 0, 1800, 800);
textFont(f, 80);
fill(255, 255, 255);
if (millis() < time + 6000) {
text(fwall2[0], 100, (height/2) - 50);
text(fwall2[1], 100, (height/2) + 100);
text(fwall2[2], 100, (height/2 + 200));
} else {
menu = 1;
}
/////////////////////////////////////////////////////||||||MENU 33
/////////////////////////////////////////////////////|NC from menu 2
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 33) {
image(library, 0, 0);
if (millis() < time + 5000) {
jenni_confused_library();
} else {
jenni_blank();
text(dlg[4], 20, 740);
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 34
/////////////////////////////////////////////////////|NC from menu 3 and 4
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 34) {
image(library, 0, 0);
if (millis() < time + 4500) {
jenni_confused_library();
} else {
jenni_blank();
text (dlg [12], 20, 740); //Maybe we should skip.
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 35
/////////////////////////////////////////////////////|NC from menu 7
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 35) {
image(library, 0, 0);
if (millis() < time + 4500) {
jenni_confused_library();
} else {
jenni_blank();
text(dlg[18], 20, 740);
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 36
/////////////////////////////////////////////////////|NC from menu 8 and 9.
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 36) {
image(library, 0, 0);
if (millis() < time + 4500) {
jenni_confused_library();
} else {
jenni_blank();
text(dlg[22], 20, 720);
text(dlg[23], 20, 760);
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 37
/////////////////////////////////////////////////////|NC from menu 10.
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 37) {
image(hall, 0, 0);
tint(255, 126);
image(figure, 890, 260, figure.width, figure.height);
tint(255, 255);
image(flipjenni, width/4, 100);
if (millis() < time + 4500) {
jenni_confused_hall();
} else {
fill(0, 0, 0);
stroke(255, 255, 255);
strokeWeight(10);
rect(0, 650, 1800, 800);
rect(0, 600, 100, 650);
textFont(f, 30);
fill(255, 255, 255);
text(fwall[4], 10, 640);
text(dlg2[6], 20, 740);
pass = true;
draw_prompt();
}

/////////////////////////////////////////////////////||||||MENU 41
/////////////////////////////////////////////////////|NC from menu 21(trust me?) menu 22 (have you figured it out yet?) and menu 28 (prove it)
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 41) {
image(gates, 0, 0);
textFont(f, 30);
fill(255, 255, 255);
image(johnny, 400, 40);
draw_blank_textbox();
draw_johnny_nametag();
if (millis() < time + 3000) {
text(dlgnc[2], 20, 740);
} else if (millis() < time + 9000) {
text(dlgnc[3], 20, 740);
} else if (millis() < time + 13000) {
text(dlgnc[4], 20, 740);
} else {
menu = 24;
time = millis();
}
/////////////////////////////////////////////////////||||||MENU 42
/////////////////////////////////////////////////////|NC from menu 24(admit it) menu 26 (anything to say) and menu 31 (spit it out)
/////////////////////////////////////////////////////|
/////////////////////////////////////////////////////|
} else if (menu == 42) {
susjenni_blank();
if (millis() < time + 4000) {
text(dlgnc[5], 20, 740);
} else if (millis() < time + 9000) {
text (dlgnc[6], 20, 740);
} else if (millis() < time + 14000) {
cage_blank();
text(dlgnc[7], 20, 740);
} else if (millis() < time + 20000) {
cage_blank();
text(dlgnc[8], 20, 740);
} else if (millis() < time + 24000) {
cage_blank();
text(dlgnc[10], 20, 740);
} else if (millis() < time + 29000) {
cage_blank();
text(dlgnc[11], 20, 740);
} else if (millis() < time + 31000) {
image(gates, 0, 0);
draw_blank_textbox();
textFont(f, 30);
fill(255, 255, 255);
text(fwall[38], 20, 740);
} else if (millis() < time + 34100) {
image(gates, 0, 0);
jenni_blank();
text(dlgnc[12], 20, 740);
} else {
image(gates, 0, 0);
jenni_blank();
text(dlgnc[13], 20, 740);
pass = true;
draw_prompt();
}

///////////////////////////////////////////////////MENU 43 and == END SCREEN
} else if (menu == 43) {
image(gates, 0, 0);
jenni_blank();
if (millis() < time + 5000) {
text(dlgnc[14], 20, 740);
} else if (millis() < time + 9000) {
text(dlgnc[15], 20, 740);
} else {
menu = 44;
time = millis();
}
} else if (menu == 44) {
pass = true;
fill(87, 6, 140);
stroke(255, 255, 255);
strokeWeight(20);
rect(0, 0, 1800, 800);
textFont(f, 200);
fill(255, 255, 255);
text(fwall2[3], 300, height/3);
textFont(f, 100);
text(fwall2[4], 300, (height/2)+150);
textFont(f, 50);
text(fwall2[5], 50, 700);
}
} //end of draw

I was inspired by David Rokeby’s Sorting Daemon (2003), a project which involves heavy use of distortion over the images of people. My project involves a simple digital mechanism (a switch). When the switch is not pressed, the processing program simply projects the image of the person in front of my computer’s webcam. When the switch is pressed, however, the image in front of the camera is condensed into pixels. A koala is projected in front of this image, and the image randomly tints. When the switch is released, the tint remains over the webcam image and the webcam image switches orientation. My concept for the exercise was the idea of being transformed — the koala was chosen for a fun, cute effect, but I think this idea could be taken further, with the viewer “transformed” by some third party. 

The most difficult part of the project was making the image switch orientation upon button release. I struggled with the help of the fellows to make a counter which would create a discrepancy every time the button was released (with this discrepancy then used to tell the program to change the orientation of the image).

If I were to continue working on the project, I would hope to make the orientation truly random instead of changing between one of two states every press. 

Final product:

Processing code:

import processing.serial.*;
import processing.video.*;
Capture cam;

Serial myPort;
int buttonval;
int time = 0;
PImage img;
color clr;

int bp = 0;
int bp2;
int counter;

void setup() {
size(500, 500);
background(0);
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);
cam = new Capture(this, 640, 480);
cam.start();
img = loadImage(“koala.png”);
}

void draw() {
// to read the value from the Arduino
if (cam.available()) {
cam.read();
cam.loadPixels();
}

while ( myPort.available() > 0) {
buttonval = myPort.read();
}

if (buttonval == 0) {
pushMatrix();
translate(width, 0);
scale(-1, 1);
image(cam, 0, 0, width, height);
popMatrix();
if (counter % 2 == 0) {
pushMatrix();
translate(0, height);
scale(1, -1);
image(cam, 0, 0, width, height);
popMatrix();
}
}
if (buttonval == 1) {
int circleSize = 5;
int w = width;
int h = height;
for (int y = 0; y < h; y+=circleSize) {
for (int x = 0; x < w; x+=circleSize) {
int i = x + y*w;
fill( cam.pixels[i] );
ellipse(x, y, circleSize, circleSize);
}
}
image(img, width/4, height/4);
tint(random(255), random(255), random(255), 255);
}
bp = buttonval;
if (bp == 0 && bp2 == 1) {
counter++;
}

bp2 = bp;

println(counter);

//println(buttonval);//This prints out the values from Arduino
cam.updatePixels();
}

Exercise 1: Make a Processing Etch A Sketch

This portion of the recitation did not give me much trouble. After setting up the correct port, I knew that I would need to map the values from the potentiometers into the variables which would be the x and y coordiates of my ellipse. After asking where I should put the ellipse and the variables in the sketch, I wrote the code fairly quickly. I asked for some advice from Kate on making sure that the mapping was done correctly, and it worked fine with a test. 

I do wish that I could make the “etch a sketch” smoother, but I felt rushed to get to the next exercise (as I knew it would be more difficult).

Exercise 2: Make a musical instrument with Arduino

This portion of the recitation gave me a lot of trouble. I had no idea where to start — I had to ask several times where I would define the values of the array. After figuring that out, I quickly wrote a code stating that if (keyPressed == true), one of several keys could fill in a value in the array:

if (keyPressed == true){

   if (key == 'a'){ 

               values[1] = 'a';

}      

and so on.  For the arduino part of the project, I used simple if statements along with the tone function.

This didn’t work. 

It took me a long time to figure out why it wasn’t working — I messed with the delay on the arduino side, eventually taking it out altogether. I also tried many combinations of if/else statements, as well as adding another variable to stand for “no tone”. I also moved the processing portion of the code from void draw to void keypressed. I’m not sure if this made any difference, but I ended up leaving it in the final product. 

Finally, I simplified the code in processing as much as I could and used if and if/else statements in arduino. This finally made the project work — by pressing a,s,d or f on the keyboard, the buzzer plays a different tone. 

Code for Processing:

import processing.serial.*;

int NUM_OF_VALUES = 4; 

Serial myPort;
String myString;

// This is the array of values you might want to send to Arduino.
int values[] = new int[NUM_OF_VALUES];

void setup() {
size(500, 500);
background(0);

printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);

myPort.clear();
myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
myString = null;
}
void keyPressed() {
if (key == 'a') {
values[0] = 'a';
}
if (key == 's') {
values[1] = 's';
}
if (key == 'd') {
values[2] = 'd';
}
if (key == 'f') {
values[3] = 'f';
}
}

void draw() {
background(0);

sendSerialData();

echoSerialData(200);
}

void sendSerialData() {
String data = "";
for (int i=0; i<values.length; i++) {
data += values[i];
if (i < values.length-1) {
data += ","; // add splitter character "," between each values element
}
else {
data += "n"; // add the end of data character "n"
}
}
myPort.write(data);
}

void echoSerialData(int frequency) {
if (frameCount % frequency == 0) myPort.write('e');

String incomingBytes = "";
while (myPort.available() > 0) {
//add on all the characters received from the Arduino to the incomingBytes string
incomingBytes += char(myPort.read());
}
print( incomingBytes );
}

Code for Arduino (including just the void loop):

void loop() {
getSerialData();if (values[0] == 'a'){
tone(6, 262);
} else if (values[1] == 's'){
tone(6,196);
} else if (values[2] == 'd'){
tone(6,220);
} else if (values[3] == 'f'){
tone(6,247);
}

Working Title

“A Typical Day at NYU Shanghai”

Context and Purpose

Thus far, my definition of interaction for the class is a cycle of communication between two or more actors which involves “listening,” processing, and responding. My ideas of and standards for interaction have taken inspiration from Crawford’s “What Exactly is Interactivity?”:  interactivity can occur in a spectrum. Some projects feel more interactive. In my preliminary research for this final project, I found that projects which allow for some curiosity and exploration on the part of the user feel much more interactive.

Further, since I believe that language is key to communication on a human level, my goal for the course has been to incorporate language into my interactive projects to create an interactive feeling. My midterm project (“Aunt Margot”) played on the idea of a Magic 8 Ball by giving more complex answers (i.e. complete sentences, snippets of informal speech, etc.). In this way, I wanted to improve the interactive experience on the side of the project, as the project gave more human answers.With this research and experience in mind, I want my final project to develop more on the user side of the interaction.

Proposal

I want to create a simple text adventure with a small twist. This concept is based on the “Choose Your Own Adventure” book series, as well as simple text-based computer games. I usually enjoy this type of game because of the allure of being part of the story, but often find myself bored of the cyclical choice-making system. For this reason, I would like to construct an interactive fiction game with a twist: the user only has two choices for each prompt within the game, “yes” and “no.” However, if the user follows a hint within the game they are told to (for example) check under the “yes” button to find a third button^1. At the end of the game, the user who finds this third option gets the true ending. In this way, I would like to improve on the traditional interactive fiction game with a physical interaction between the user and the game. 

In terms of timeline, since the physical elements of the project should not take long the most important thing is the coding and story element. I have already tested a format for the code which allows for what I want (based on boolean statements and menus, with a button press switching to a different menu). I have also already drafted how many menus I will need and what kind of plot I would like. For now I will plan on scheduling one or two days to take care of the physical element early on, and then devote the rest of my time to coding.

¹ I currently envision this project to be focused on my laptop (which can be rotated to display only the screen). In front, evenly spaced, will be two small fabricated cubes with buttons. These cubes may be labeled YES and NO. The cubes are hollow, allowing another button to be hidden underneath.