Category: Interaction Lab

My final project name is Cooperate! Partner! The problem my project want to address is the group work problem especially the two person group which is common when doing the IMA project. I noticed this problem from my personal experience. When I doing my first IMA project, I almost do it individually even I am in a two person group. Every time when I work in the studio, I am alone and have to deal with all the difficulties by myself. This increase my workload and also gave me great mental pressure that I nearly broke down. Then latter, when doing my research I interviewed some of my friends who are all two person group working on a project. Surprising to me that most of them had really bad experience when doing the group work. Person1 told me that she cried many times because her partner doing nothing and put all the work on her. She is busy with her work and she is under great pressure. Person2 told me that he always argues with his partner because his partner sticks to his own idea and never listen to him. He found some shortages of his partners idea and try to improve the idea but his partner always ignores his suggestions. Person3 told me that although her partner cooperates with her to finish the project but she hope her partner can devote more time to their project to make it better. She said although her partner participates but she thinks her partner doesn’t spent as much time and effort as she did on the project. These are three mostly happen situations I gathered from my research when unhappiness happens during group work. This inspires me to design a project to improve the group work experience. I hope by doing this can make partners cooperate better and prevent the situation that people emotionally break down when doing the project. After the presentation, after hearing others suggestions and ideas, I think I can make my audience broader to improve the cooperation between people who may not be familiar with each other. I hope for those who don’t know each other, after playing this game, they can get to know each other by doing the physical interaction cooperating with each other and this is also the significance of my project. Also, I hope the project can really improve the bad condition existing among the two person team.

For the two projects inspire me one is the Little Nightmare. This let me think of using sound interaction to make people feel happier when they finish the work which also let they realize team work is valuable and meaningful. For the Pluck it, I let me think that to make the concept stronger, I may set different level to help strengthen the user experiencing the importance of cooperation to better convey my idea. 

I plan to use the cube to represent the project. By separating the cube into two part, the team work divided into two part. I plan to put the two part on the screen using processing so the users can see the changing of the two cube when they rotate their palm. Their visual feeling is directly linked with their physical feeling. They have to find the right angle of the piece to form the cube. This also represents they have a control of their work. By controlling the angle, I hope people feel that you need to participate in the group work and devote your effort. If you don’t adjust the right angle of the piece, your partner have to spend more time and effort to change her angle to adjust to you. Then comes to the cooperate part. At first, I plan to use the key to control the position of the piece. Then when I read the article The Future of Design: When you come to a fork in the road, take it, I was really inspired by the sentence “It is action-oriented, learning by doing, through repeated iterations of making, testing, and observation.” Then I notice that using the key on the keyboard lack the continuous interaction between two players, then I change to use the distance sensors to do this part. This also meet my definition for interaction which is having a back and forth communication with thoughts and physical input from both sides. The distance sensor will sense the distance, if two people getting closer, the piece will be closer. Also, they can standing up and squatting down to move the piece up and down. I have two plan, one is to put the sensor in a helmet wearing on the head another is to put it on the leg to let the user move their legs to change the position of the piece. This add the physical interaction connecting to the processing. Then when the two piece get together, I plan to put a delightful music to let the players be more satisfied with the success and considered cooperation a great and valuable action.

I plan to borrow the accelerometer and learn how it works and how to do the coding of the accelerometer on Nov.22. Then on Nov.24, I plan to build the processing image on the screen. On Nov.26, I plan to set the distance sensor connecting to the processing. Then on Nov.27, I plan to do the prototype to test whether the helmet is better to put the sensors on the leg is better. After doing this, I plan to add different levels for my project to make it more cooperative which also meet the goal that I hope people can better cooperate with each other after they play this game. I may also keep changing just like the article Introduction to Physical Computing suggesting “A lot of solutions will appear in your peripheral vision”, I will keep improving to make the interaction stronger.

Cited:

1.“The Future of Design: When You Come to a Fork in the Road, Take It.” Jnd.org, 23Apr.2016, https://jnd.org/the_future_of_design_when_you_come_to_a_fork_in_the_road_take_it/.

2.Introduction to Physical Computing. by O’Sullivan and Igoe.

Exercise 1: Make a Processing Etch A Sketch

For the interaction for two steps in this exercise, both of these steps are using potentiometers to interact, the interaction given by us is just switching the knob. But the interaction given by the processing is different, the first one is simple which is just the ellipse changing its size, the second one is more interactive as I am drawing a line on the screen and it can go wherever I want in the window. Also I customize its color based on its position in the window so that I can draw a gradient line which is more interesting and more interactive. The codes are as follow.

Processing code

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
* Based on the readStringUntil() example by Tom Igoe
* https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
*/

import processing.serial.*;

String myString = null;
Serial myPort;

int NUM_OF_VALUES = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/
int[] preValues;

void setup() {
size(500, 500);
background(0);
setupSerial();
preValues = new int[2];
}

void draw() {
updateSerial();
printArray(sensorValues);
println("cr");
printArray(preValues);
println("pre");
//fill(sensorValues[0],sensorValues[1],sensorValues[2]);
//rect(100,100,300,300);
background(0);
//ellipse(sensorValues[0],sensorValues[1],100,100);
strokeWeight(1);
stroke(sensorValues[0],sensorValues[1],preValues[0]);
line(sensorValues[0],sensorValues[1],preValues[0],preValues[1]);

// use the values like this!
// sensorValues[0]

// add your code

//
}

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);
// WARNING!
// You will definitely get an error here.
// Change the PORT_INDEX to 0 and try running it again.
// And then, check the list of the ports,
// find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----"
// and replace PORT_INDEX above with the index number of the port.

myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
myString = null;

sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
for (int i =0; i < sensorValues.length; i ++){
preValues[i] = int(sensorValues[i]);
}
if (myString != null) {
String[] serialInArray = split(trim(myString), ",");
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}

Arduino code

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
// int sensor3 = analogRead(A2);

sensor1 = map(sensor1,0,1023,0,500);
sensor2 = map(sensor2,0,1023,0,500);
// sensor3 = map(sensor3,0,1023,0,255);

// keep this format
Serial.print(sensor1);
Serial.print(","); // put comma between sensor values
Serial.print(sensor2);
// Serial.print(",");
// Serial.print(sensor3);
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(200);
}

Exercise 2: Make a musical instrument with Arduino

For this exercise, we can make interaction with a buzzer according to our key-press or mouse-move. It gives more ways of interaction compared to using knobs that you can only switch it and is comparatively not interactive. I can modify the keys to become a tone player that each key has a corresponding sound frequency and when pressing the keys on the keyboard in random sequence will turn out different melodies. And I can even change the frequencies to make different kinds of tones. So I think it is playful and highly interactive. The codes are as follow.

Processing code

// IMA NYU Shanghai
// Interaction Lab

/**
* This example is to send multiple values from Processing to Arduino.
* You can find the arduino example file in the same folder which works with this Processing file.
* Please note that the echoSerialData function asks Arduino to send the data saved in the values array
* to check if it is receiving the correct bytes.
**/

import processing.serial.*;

int NUM_OF_VALUES = 3; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/

Serial myPort;
String myString;

// This is the array of values you might want to send to Arduino.
int values[] = new int[NUM_OF_VALUES];

void setup() {
size(500, 500);
background(0);

printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);
// check the list of the ports,
// find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----"
// and replace PORT_INDEX above with the index of the port

myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
myString = null;
}

void draw() {
background(0);

// changes the values
if(keyPressed){
if(key=='a'){
values[0] = 1;
values[1] = 1000;
}
if(key=='s'||key=='S'){
values[0] = 1;
values[1] = 1100;
}
if(key=='d'||key=='D'){
values[0] = 1;
values[1] = 1200;
}
if(key=='f'||key=='F'){
values[0] = 1;
values[1] = 1300;
}
if(key=='g'||key=='G'){
values[0] = 1;
values[1] = 1400;
}
if(key=='h'||key=='H'){
values[0] = 1;
values[1] = 1500;
}
}else{
values[0] = 0;
}
//for (int i=0; i<values.length; i++) {
// values[i] = i; /** Feel free to change this!! **/
//}

// sends the values to Arduino.
sendSerialData();

// This causess the communication to become slow and unstable1.
// You might want to comment this out when everything is ready.
// The parameter 200 is the frequency of echoing.
// The higher this number, the slower the program will be
// but the higher this number, the more stable it will be.
echoSerialData(200);
}

void sendSerialData() {
String data = "";
for (int i=0; i<values.length; i++) {
data += values[i];
//if i is less than the index number of the last element in the values array
if (i < values.length-1) {
data += ","; // add splitter character "," between each values element
}
//if it is the last element in the values array
else {
data += "n"; // add the end of data character "n"
}
}
//write to Arduino
myPort.write(data);
}

void echoSerialData(int frequency) {
//write character 'e' at the given frequency
//to request Arduino to send back the values array
if (frameCount % frequency == 0) myPort.write('e');

String incomingBytes = "";
while (myPort.available() > 0) {
//add on all the characters received from the Arduino to the incomingBytes string
incomingBytes += char(myPort.read());
}
//print what Arduino sent back to Processing
print( incomingBytes );
}

Arduino code

#define NUM_OF_VALUES 3 /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/

/** DO NOT REMOVE THESE **/
int tempValue = 0;
int valueIndex = 0;

/* This is the array of values storing the data from Processing. */
int values[NUM_OF_VALUES];

void setup() {
Serial.begin(9600);
// pinMode(13, OUTPUT);
pinMode(9, OUTPUT);
}

void loop() {
getSerialData();

// add your code here
// use elements in the values array
// values[0]
// values[1]
if (values[0] == 1) {
// digitalWrite(13, HIGH);
// } else {
// digitalWrite(13, LOW);
// }

// if (values[1] == 1) {
tone(9, values[1]);
// } else {
// noTone(9);
// }
}
if(values[0] == 0){
noTone(9);
}

}
//recieve serial data from Processing
void getSerialData() {
if (Serial.available()) {
char c = Serial.read();
//switch - case checks the value of the variable in the switch function
//in this case, the char c, then runs one of the cases that fit the value of the variable
//for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase
switch (c) {
//if the char c from Processing is a number between 0 and 9
case '0'...'9':
//save the value of char c to tempValue
//but simultaneously rearrange the existing values saved in tempValue
//for the digits received through char c to remain coherent
//if this does not make sense and would like to know more, send an email to me!
tempValue = tempValue * 10 + c - '0';
break;
//if the char c from Processing is a comma
//indicating that the following values of char c is for the next element in the values array
case ',':
values[valueIndex] = tempValue;
//reset tempValue value
tempValue = 0;
//increment valuesIndex by 1
valueIndex++;
break;
//if the char c from Processing is character 'n'
//which signals that it is the end of data
case 'n':
//save the tempValue
//this will b the last element in the values array
values[valueIndex] = tempValue;
//reset tempValue and valueIndex values
//to clear out the values array for the next round of readings from Processing
tempValue = 0;
valueIndex = 0;
break;
//if the char c from Processing is character 'e'
//it is signalling for the Arduino to send Processing the elements saved in the values array
//this case is triggered and processed by the echoSerialData function in the Processing sketch
case 'e': // to echo
for (int i = 0; i < NUM_OF_VALUES; i++) {
Serial.print(values[i]);
if (i < NUM_OF_VALUES - 1) {
Serial.print(',');
}
else {
Serial.println();
}
}
break;
}
}
}