Author: Jessica Xing

The three projects I critiqued were Jackson’s Bongo with Bongo Cat, Sarah’s Tap the World Over, and Justin’s Tip Tap Snap. 

Bongo with Bongo Cat: 

The project is a musical media interface in which the user can experiment with the possibilities using the instrumentals of many popular songs such as “We Will Rock You” by Queen, “DJ’s Got Us Falling In Love,” etc. By tapping the five buttons on each cat paw, the user can interact with “BongoCat,” a reference to a popular internet cat meme that also could play music. Put together the user can either complete the song, or play with the song’s features to create something entirely unique. 

Jackson defined interaction as two or more actors affect one another through a cyclical process, and I believe this project fulfills this definition of interaction because the two paws attached to the digital cats not only allows for multiple players, but also allows for continual interaction with the machine. I always thought of interaction as a conversation personal and individual to one user, so factoring in potentially more people is one avenue I could consider. 

Some feedback I and other group members gave him were that he could: 

• add a more obvious objective for the user to follow along, as it might not seem immediately obvious what he wants the user to do by just looking at the setup. 
• put on headphones so it is more personal to the user and so the music can still be heard over all the other projects during user testing.

Tap the World Over: 

This project wants to incentivize users to recycle plastic bottle caps more by making it into a game: Sarah envisions that there would be cardboard globe attached to a recycling bin, and anytime a user successfully “tosses” a bottle cap in any one of the different holes (each with different point values) on the globe. When the user successfully throws the bottle cap, the point value will correspond to the degrees in which the globe on the screen will turn. 

Sarah defined interaction as a process that incorporates users and provides responses promptly to a particular action placed by the user. I believe this project does follow her definition of interaction because the user sees a direct result of their action, and it molds to each different action the user decides to take. It corresponds to my definition of interaction because I too want an immediate response to when the user “talks” to the machine, and I want the response to be tailored to their individual action. 

Some feedback I and other group members gave were: 

• Add a counter to the top of the screen, because like those refillable water stations people feel like they are contributing to something larger, and it shows a more immediate effect of reward
• make the reward more obvious to incentivize users more to participate in the game, because they might just think it is a cooler trash can and not actively try to throw the bottle caps into the bin
• Increase degrees so the reward is more immediate. 

Tip Tap Snap: 

Justin’s project takes inspiration from popular 2010 music app games such as Tap Tap Revolution, but he emphasizes the competition aspect of it more to make interaction feel more like a communal experience. Using pressure sensors the users compete to correctly “tap” to the beat of the music. At the end there is a photo taken to capture the user’s experience while playing the game.

Justin defines interaction as two parties bouncing ideas off of each other while creating 1+1>2 effect with creativity. This means that with two players, their interactions have to be greater than a simple cause-effect relationship, and the interaction has to be more meaningful beyond the simple call and response. I believe this project corresponds to his definition of interaction, because there are two players, and the multiplayer competition aspect of it greatly affects the experience of the game. I agree with this definition of interaction, and try to incorporate it into my project because I want the interaction to be greater and more personally impactful than just a simple cause (the user’s interaction) and effect (how that interaction shows up on the machine). 

Some feedback I and other group members gave: 

• Incorporate the photo aspect more fluidly into the game: maybe have a “winner” and a “loser” filter at the very end with each user’s photo
• Split the screen vertically instead of horizontally so the game is at eye level with both users

Feedback and Critique Reflection for my project: 

Feedback I received for my project: 

• All group members wanted to know how I would decide to make my monsters “ugly.” Going forward, I will base them on what users have been “taught” to fear through depictions of monsters in movies, books, etc. And will make this research clear upon further presentations of my project. 
• Justin suggested I make the twist of the game more obvious by showing the user what the pet will end up looking like, and asking if the user still wants to take care of the pet
• My group members also suggested I reverse the process and shorten the game, so they will grow to appreciate the monsters from the get go. 
• Make it more obvious what you want the user to do: whether it be add meters for whether or not they are hungry or making it clearer what the order of the buttons need to be pressed. 
• How will the design go? My group members suggested I use more realistic depictions because pixel art allows the user to separate themselves from the pet. 

Reflection on the feedback: 

Most successful part of my presentation was the idea of a “pet” that ends up turning monstrous, because it subverts the usual expectations people have of virtual pet games. The least successful part was how I presented the how I will illustrate the “monstrous” design, as stated above my group members believe I need more research/justification for how I will design the monsters to look more “monstrous.” 

For my project, going forward I plan to incorporate more research into how I decide to design the monsters. I will also make the pet’s needs look more obvious, such as adding a “feed me!” reaction, or making each reaction to each of the physical interactions unique so the user knows that each interaction does something different. While I do not think it is necessary to show the user what the monster will end up like, because I feel that would be too obvious, I do like the idea of making the user question in the design whether or not they still want to take care of the monster. 

For this recitation, we implemented what we learned in lecture about Arduino and Processing communication, but for recitation we experimented with multiple values to broaden our range of Arduino and Processing interaction. This recitation taught me how software values translate to hardware values and vice versa. 

Exercise 1: Etch a Sketch 

This exercise showed me how Arduino values translate to Processing. For this exercise I had the hardest part understanding what values to plug into the ellipse() function. I understood that I had to use the map() for the x and y values, but after putting in the sensor readings, I didn’t know the other numbers needed in the range. After talking with a fellow, I understood then that the range can be found out by opening the Arduino Serial Monitor and seeing the range the Potentiometer goes to. Secondly, I accidentally deleted Serial.println() which I learned was really important to how the processing understands the ASCII line values Arduino sends to it. After I fixed both the code ended up working. 

Here is the code below: 

Arduino: 

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
// int sensor2 = analogRead(A1);
int sensor2 = analogRead(A2);

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
// Serial.print(“,”);
// Serial.print(sensor3);
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Processing: 

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
* Based on the readStringUntil() example by Tom Igoe
* https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
*/

import processing.serial.*;

String myString = null;
Serial myPort;

int NUM_OF_VALUES = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/

void setup() {
size(1000, 1000);
background(0);
setupSerial();
}

void draw() {
updateSerial();
printArray(sensorValues);

// use the values like this!
// sensorValues[0]

// add your code
background(255);
fill(0);
ellipse(map(sensorValues[0],0,1023,0,width),map(sensorValues[1],0,1023,0,height), 100,100);
printArray(sensorValues);

//
}

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);
// WARNING!
// You will definitely get an error here.
// Change the PORT_INDEX to 0 and try running it again.
// And then, check the list of the ports,
// find the port “/dev/cu.usbmodem—-” or “/dev/tty.usbmodem—-”
// and replace PORT_INDEX above with the index number of the port.

myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
myString = null;

sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}

Here is the video attached below: 

Here is the circuit: 

Exercise 2: Musical Instrument 

This one was relatively easier than the first exercise, because since I used ToneMelody a lot the code and the function that makes it work made a lot of sense to me. The problem was deciding whether or not it was the code that wasn’t working or the buzzer — I changed the values[0] and values [1] to reflect the mouse’s position, and I switched the buzzer out. Once I did that the code worked just fine. Here is the code: 

Arduino: 

// IMA NYU Shanghai
// Interaction Lab

/**
This example is to send multiple values from Processing to Arduino.
You can find the Processing example file in the same folder which works with this Arduino file.
Please note that the echo case (when char c is ‘e’ in the getSerialData function below)
checks if Arduino is receiving the correct bytes from the Processing sketch
by sending the values array back to the Processing sketch.
**/

#define NUM_OF_VALUES 2 /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/

/** DO NOT REMOVE THESE **/
int tempValue = 0;
int valueIndex = 0;

/* This is the array of values storing the data from Processing. */
int values[NUM_OF_VALUES];

void setup() {
Serial.begin(9600);
pinMode(13, OUTPUT);
//pinMode(9, OUTPUT);
}

void loop() {
getSerialData();
tone(13,values[0],values[1]);
}
// add your code here
// use elements in the values array
// values[0] // values[1] // if (values[0] == 1) {
// digitalWrite(13, HIGH);
// } else {
// digitalWrite(13, LOW);
// }
//
// if (values[1] == 1) {
// tone(9, values[2]);
// } else {
// noTone(9);
// }
//
//}

//recieve serial data from Processing
void getSerialData() {
if (Serial.available()) {
char c = Serial.read();
//switch – case checks the value of the variable in the switch function
//in this case, the char c, then runs one of the cases that fit the value of the variable
//for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase
switch (c) {
//if the char c from Processing is a number between 0 and 9
case ‘0’…’9′:
//save the value of char c to tempValue
//but simultaneously rearrange the existing values saved in tempValue
//for the digits received through char c to remain coherent
//if this does not make sense and would like to know more, send an email to me!
tempValue = tempValue * 10 + c – ‘0’;
break;
//if the char c from Processing is a comma
//indicating that the following values of char c is for the next element in the values array
case ‘,’:
values[valueIndex] = tempValue;
//reset tempValue value
tempValue = 0;
//increment valuesIndex by 1
valueIndex++;
break;
//if the char c from Processing is character ‘n’
//which signals that it is the end of data
case ‘n’:
//save the tempValue
//this will b the last element in the values array
values[valueIndex] = tempValue;
//reset tempValue and valueIndex values
//to clear out the values array for the next round of readings from Processing
tempValue = 0;
valueIndex = 0;
break;
//if the char c from Processing is character ‘e’
//it is signalling for the Arduino to send Processing the elements saved in the values array
//this case is triggered and processed by the echoSerialData function in the Processing sketch
case ‘e’: // to echo
for (int i = 0; i < NUM_OF_VALUES; i++) {
Serial.print(values[i]);
if (i < NUM_OF_VALUES – 1) {
Serial.print(‘,’);
}
else {
Serial.println();
}
}
break;
}
}
}

Processing 

// IMA NYU Shanghai
// Interaction Lab

/**
* This example is to send multiple values from Processing to Arduino.
* You can find the arduino example file in the same folder which works with this Processing file.
* Please note that the echoSerialData function asks Arduino to send the data saved in the values array
* to check if it is receiving the correct bytes.
**/

import processing.serial.*;

int NUM_OF_VALUES = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/

Serial myPort;
String myString;

// This is the array of values you might want to send to Arduino.
int values[] = new int[NUM_OF_VALUES];

void setup() {
size(500, 500);
background(0);

printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);
// check the list of the ports,
// find the port “/dev/cu.usbmodem—-” or “/dev/tty.usbmodem—-”
// and replace PORT_INDEX above with the index of the port

myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
myString = null;
}

void draw() {
background(0);

// changes the values
values[0] = mouseX;
values[1] = mouseY;

// sends the values to Arduino.
sendSerialData();

// This causess the communication to become slow and unstable.
// You might want to comment this out when everything is ready.
// The parameter 200 is the frequency of echoing.
// The higher this number, the slower the program will be
// but the higher this number, the more stable it will be.
echoSerialData(200);
}

void sendSerialData() {
String data = “”;
for (int i=0; i<values.length; i++) {
data += values[i];
//if i is less than the index number of the last element in the values array
if (i < values.length-1) {
data += “,”; // add splitter character “,” between each values element
}
//if it is the last element in the values array
else {
data += “n”; // add the end of data character “n”
}
}
//write to Arduino
myPort.write(data);
}

void echoSerialData(int frequency) {
//write character ‘e’ at the given frequency
//to request Arduino to send back the values array
if (frameCount % frequency == 0) myPort.write(‘e’);

String incomingBytes = “”;
while (myPort.available() > 0) {
//add on all the characters received from the Arduino to the incomingBytes string
incomingBytes += char(myPort.read());
}
//print what Arduino sent back to Processing
print( incomingBytes );
}

Here is the video: 

Here is the circuit: 

I attended a Processing Basics Workshop hosted by IMA Fellow Leon. I decided to attend the workshop because I had very little experience with coding and wanted to catch up on basic commands that still seemed really unfamiliar to me, like boolean values, push and popMatrix, and how those commands relate to animation. 

We first learned how to make shapes using the ellipse() and rect() function, and how to use the stroke() and strokeWeight() to change the look of the shape. Secondly, we then learned how to use mousePressed() vs. keyPressed(), and learned how to control the animation using the mouse across the screen. 

The last thing we learned is pushMatrix and popMatrix, and how the commands relate to animation. I wanted to figure out how to make the circle then to bounce back to the middle of the screen, but could not figure out how to make it come back despite using negative x and y values to move it back. The final code and animation is attached below: 

//float: inbetween integers set in range

int sizeOfBall = 200;
float r, g, b;
int speedX= 10;
int speedY = 10;
int posX, posY;

void setup() { //this happens once
size (600, 600); //size (width, height)
background (0); //background (0-255), background (r,g,b)
//frameRate(100); //by default the frameRate is 60 fps
//rectMode(CENTER); //moves the middle of the rectangle the center point
colorMode(HSB); //(r,g,b) h = hue, s = saturation, b = brightness?
posX = width/2;
posY = height/2 ;
}

void draw() { // this happens again and again…
println(frameCount);

r= random (255); //random (lowerValue, upperValue)
g= random (10);
b = random (50);
fill(255, 0, 0, 150); //fill (r,g,b, alpha)
strokeWeight(5);
stroke(0,255,0);
noStroke();
ellipse(mouseX, mouseY, sizeOfBall, sizeOfBall); // ellipse (x, y, width, height)
//a ball should exist… ellipse
//a ball should go down…
// the ball should bounce back up when it hits the edge…

rotate(radians (5));
rect (width/2, height/2, sizeOfBall, sizeOfBall);

vertex (mouseX *100, mouseY/50);
vertex (mouseX – 500, mouseY+60);
vertex (mouseX +300, mouseY+50);

pushMatrix();
translate(width/2, height/2); // my 0,0 position is width/2, height/2
rotate(radians(45));
rect (0,0, sizeOfBall, sizeOfBall);
fill(255);
rect(500,500, 50, 50);

Overall I am really glad I went to this workshop, because it caught me up on a lot of basics I felt that a lot of people in the class already knew. I understood better how to plot points for each of the different shapes that allow for variables and animation, and it also taught me how to use int(), float(), and boolean values, which are really valuable once we start to get to harder material. I can see myself using the basics taught in this workshop to further my understanding of Processing, and will use what I’ve been taught to work towards a more comprehensive final project.