Author: Sarah Waxman

INTERACTION LAB FINAL PROJECT SPRING 19

Tap the World – SARAH WAXMAN – PROFESSOR YOUNG

The concept for our project came from a certain understanding of human behavior; that people are more likely to do things that garner immediate results. Recycling is a very long process, from the moment a person places something in a recycling bin to the point when it is actually recycled. The people doing this therefore rarely see the results of their actions. This project therefore aims to provide a fun and alternative way to encourage recycling by creating a recycling bin (top) that gives users a glimpse into the future of their actions. The trees are a representation of the good they are doing for the environment, the whole project is a representation – albeit extremely simplified – of the overall process.

In terms of design, we understood that the top should fit the size of the regular recycling bins found around school (since we decided to localize the project) so that the intention would be clear simply from context of placement. The hole through  which users are expected to throw water bottles is also a circular hole that is roughly the same size as those normally found on recycling bins for plastic bottles. This is done to ensure that the user is familiar and associates it with plastic bottles. The trees are both aesthetically pleasing and relevant, and were part of a suggestion we received during the user testing. We used a laser cutter to create the circular top and the hole(s). The reason we had originally cut multiple holes is because we had previously planned to make it into a game where the size of the hole where a person’s recycled waste went through would account for different amounts of trees popping up on the Processing program. We ultimately decided that the game element was irrelevant to the overall purpose of our project, and went in a different direction. We therefore covered the holes with other laser-cut circles we found in the Fabrication Lab and painted them white, like the rest of our circular top. The circular top was also engraved with the map of the globe, and we covered this with paint as well when we decided to project the Processing program of the spinning globe and trees onto the top in order to further simplify the user experience. The white paint was used in order to make the projection more visible to the user. We used a tripod from the equipment room in order to hold up the projector at the correct height and angle to project onto the top. The trees were stuck on to the top very carefully to avoid blocking the projection.

In the earliest stages of our project and fabrication, we wanted to users to recycle plastic bottle caps with our recycling bin lid after being inspired by the Colombian nonprofit organization Tapas Para Sanar which collects these plastic caps and sells them to recycling companies, ultimately donating the proceeds to cancer research. However, after trying out a variety of sensors and consulting with lab fellows, we determined that the sensors available are not sensitive enough to detect the tiny lids flying through at such high speeds. We therefore changed it to plastic bottles which were much more consistently detected by the sensor. We were also originally going to have plastic bottle lids pop up around the globe, but after realizing that it kind of creates an image that defeats the purpose of our project, we decided to go with trees instead.

The processing program was mainly done during the workshop recitation at the media manipulation workshop. After being advised that 3D programs are very complex and go way beyond the scope of this course, we decided that a video of a rotating earth would work just as well and equally serve our purpose. We looped it and slowed down the pace. We then added the function to trigger trees popping up whenever the mouse was pressed to be later replaced with the function to trigger trees popping up whenever the sensor was triggered. The trees are a PNG image with a transparent background downloaded online. To make the trees move with the earth, we made the trees move in the same direction and adjusted the speed to match (as closely as possible) with the earth’s in the video. We also created a circular boundary (where the earth is) in which the trees are allowed to randomly pop up when triggered. Once the trees reach the edge, they disappear from view – as they would if they were actually attached to the globe. During the user testing session, we received some feedback saying we should allow the trees to come back on the other side asif they had made the full rotation. We considered doing this, but after thinking about the user experience we decided against it. We thought about the fact that most people do not normally stand watching a recycling bin, even one as exciting as ours, for more than a few seconds. And due to the fact that we also wanted our users to be able to tell whether anyone has used this device (and recycled) in the past couple minutes and therefore slowed down the whole Processing system (the video and speed of trees), we could not have it both ways. Moreover, if the trees did not disappear, then eventually the earth would have been to overloaded with trees that the addition of new trees would be insignificant and almost imperceptible to the user to the point that it would undermine the entire purpose of this project. In the physical, circuit aspect of this project, we needed only one Arduino and a few wires to connect it to the sensor, and we stuck this all under the lid in order for it to be hidden and have a clean look. Another idea we thought about adding and addressed during the user testing session was to add a sound to the Processing program for whenever the trees pop up / a bottle is thrown through the hole. However, after receiving constructive feedback, we decided this sound would be redundant as a sound was already being made by the bottle on impact when thrown into the bin. Nevertheless, we did receive a lot of positive feedback especially on the general concept of our project, which encouraged us to keep trying to further improve it and make it a feasible project that could viably be used in everyday life.

Overall, the idea we had for our project was to encourage the act of recycling by offering a new type of recycling bin that gives immediate user feedback as a direct result of their actions. This project thereby aligns with my definition of interaction, especially considering the fact that the entire purpose of the system we built cannot be achieved without the user’s input, and the speed of the communication/feedback between the system and the user. In the end, users interacted with our project in the way that we expected. Given more time, we would have tried to implement the idea given to us during our presentation to have the projection turn on only when there is a person nearby in order to avoid wasting energy on its constant projection. We may have also tried to add a sensor that could detect when the material of the waste going in is not plastic, and in such instances have the program fill the earth with garbage.

Given the fact that NYU Shanghai already has a recycling initiative, we thought this has the potential to be an interesting and good enhancement to that local effort. The many setbacks we faced showed me the value of focusing on one’s strengths, prioritization, and compartmentalization. In addition, through this project I learned how to create a good balance between the physical aspect of an interactive design project and its virtual (or on-screen) aspect, and the value and impact that a relatively simple system could have. 

Below is the evolution of our project through pictures, sketches, and videos. The programming codes are included below the media. 

Processing code:  Tab 1 “Earth”

import processing.serial.*;
String myString = null;
Serial myPort;
int valueFromArduino;

int NUM_OF_VALUES = 1; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/

import processing.video.*;
Movie earth;
PImage tree;
ArrayList<Tree> trees = new ArrayList<Tree>();

void setup() {
//size(640,360); //
fullScreen();//

earth = new Movie(this, “earth2.mp4”);
earth.loop();
earth.speed(0.5);
tree = loadImage(“tree.png”);
filter(INVERT);
myPort = new Serial(this, Serial.list()[ 24 ], 9600);
sensorValues = new int[NUM_OF_VALUES];
}

void draw() {
updateSerial();
printArray(sensorValues);
if (earth.available()) {
earth.read();
}
filter(GRAY);
background(255);
imageMode(CORNER);
image(earth, 0, 0, width, height);

if (sensorValues[0]== 1){
trees.add( new Tree() );
}

//float timeStamp = earth.time();
// println(timeStamp);

for(int i=0; i<trees.size(); i++) {
trees.get(i).display();
trees.get(i).move();
}

for(int i=0; i<trees.size(); i++) {
if(trees.get(i).offEarth == true) {
trees.remove(i);
}
}

println(valueFromArduino);//println(mouseX, mouseY);
println(trees.size());
}

//void mousePressed() {
// trees.add( new Tree() );
//}

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 24 ], 9600);
// WARNING!
// You will definitely get an error here.
// Change the PORT_INDEX to 0 and try running it again.
// And then, check the list of the ports,
// find the port “/dev/cu.usbmodem—-” or “/dev/tty.usbmodem—-”
// and replace PORT_INDEX above with the index number of the port.

myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
myString = null;

}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
printArray(serialInArray);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = 0;
println(sensorValues[i]);
sensorValues[i] = int(serialInArray[i]);
println(sensorValues[i]);
}
}
}
}
}

Processing code: Tab 2 “Trees” 

class Tree {
float x, y;
float speed;
boolean offEarth = false;

Tree() {
float centerX = width/2;
float centerY = height/2;
// while( dist(x, y, width/2, height/2) > 150) {
x = centerX + random(-200, 200);
//x = random(170, 200);
y = centerY + random(-200, 200);

// }
speed = 7;
}

void display() {
ellipse(x, y, 10, 10);
imageMode(CENTER);
image(tree, x, y);
}

void move() {
x += speed;
if(dist(x, y, width/2, height/2) > 300) {
offEarth = true;
}
}
}

Arduino Code: 

#define trigPin 2
#define echoPin 3
int sensor1;

void setup() {
SensorSetup();
Serial.begin(9600);
}

void loop() {

int actualDistance = MeasureDistance();

if (actualDistance < 15) {
sensor1 = 1;
}else {
sensor1 = 0;
}
// keep this format
Serial.println(sensor1);
}

void SensorSetup() {
pinMode(trigPin, OUTPUT);
pinMode(echoPin, INPUT);
}
int MeasureDistance() { // a low pull on pin COMP/TRIG triggering a sensor reading
long duration;
digitalWrite(trigPin, LOW); // Added this line
delayMicroseconds(2); // Added this line
digitalWrite(trigPin, HIGH);
delayMicroseconds(10); // Added this line
digitalWrite(trigPin, LOW);
duration = pulseIn(echoPin, HIGH);
long distance = (duration / 2) / 29.1;
return (int)distance;
}

For this recitation, I chose to participate in Leon’s workshop on media manipulation. He used the Game of Thrones intro/theme video as an example, and used different filters to change the look and feel of the video. Moreover, he transformed the overall atmosphere of the video by manipulating the speed, timing, and sound. 

The reason I chose this workshop in particular is because I thought it would be useful for my final project idea – and useful it was. Leon gave us the freedom to choose what we would work on in terms of media manipulation so I decided to use one that I may end up using for my project. The program includes a looped video of a rotating planet earth, and when one clicks the mouse, trees begin to pop up on the earth and move with it. It was extremely tricky to achieve this as working with 3D spaces is way beyond the scope of this course. The program therefore does not in fact work in 3D, instead it makes the trees disappear once they reach the other end of the earth. In addition, the trees’ movement speed had to be synced as accurately as possible to the speed of the earth’s rotation in the video. 

The code was split into two tabs. The first is titled “earth”: 

import processing.video.*;

Movie earth;

PImage tree;

ArrayList<Tree> trees = new ArrayList<Tree>();

void setup() {

  size(640,360);

  earth = new Movie(this, “earth2.mp4”);

  earth.loop();

  tree = loadImage(“tree.png”);

}

void draw() {

  if (earth.available()) {

    earth.read();

  }

  imageMode(CORNER);

  image(earth, 0, 0);

  

  //float timeStamp = earth.time();

  // println(timeStamp);

  

  for(int i=0; i<trees.size(); i++) {

    trees.get(i).display();

    trees.get(i).move();

  }

  

  

  for(int i=0; i<trees.size(); i++) {

    if(trees.get(i).offEarth == true) {

      trees.remove(i);  

    }

  }

  

  println(mouseX, mouseY);

  println(trees.size());

}

void mousePressed() {

  trees.add( new Tree() ); 

}

//make array with set amount of trees that will appear, use i and “for” function for big number

//put array, then make array[i] = 1 with “if” statement and make the processing draw a tree with “draw” function

The second tab is titled “trees”: 

class Tree {

  float x, y;

  float speed;

  boolean offEarth = false;

  

  Tree() {

    

    while( dist(x, y, width/2, height/2) > 150) {

      x = random(170, 470);

      //x = random(170, 200);

      y = random(30, 330);

      

    }

    speed = 1.2;

  }

  

  void display() {

    ellipse(x, y, 10, 10);

    imageMode(CENTER);

    image(tree, x, y);  

  }

  

  void move() {

    x += speed;

    if(dist(x, y, width/2, height/2) > 150) {

      offEarth = true;  

    }

  }

}

During this recitation, I was exposed to and analyzed three different projects besides my own.

Project 1: “Bongo with Bongo Cat” by Jackson Sayama

• This project creates a musical instrument where the cat’s paw acts as the interface where each “finger” will cause a different sound to be made on the screen. In the meantime, the Processing code will be playing a song that most people are familiar with, such as Queen’s “We Will Rock You” so that the objective of the user is to make the sounds at the correct time to complete the music.
• Cyclic process in which two or more actors affect one another is Jackson’s interpretation of interaction, and I believe this project fits into it well because of the continuous nature of the serial communication between the code as well as the user’s input.
• Some feedback:
  • Make it a two player game to make it even more interactive/entertaining  by adding some competition
  • Add an external speaker to amplify the sound, since otherwise the sound might be drowned out especially during user testing session when the room is pretty loud
  • Another solution to the sound issue would be to add headphones for players to use

Project 2: “Manamonster” by Jessica Xing

• Jessica’s “Manamonster” is inspired by the early 2000s popular game Tamagotchi but replaces the cute animal/creature with a monster. She aims to challenge social norms of beauty/cuteness. Interaction with a touch sensor, a button, and a potentiometer to control what takes place virtually for the monster.
• Clear purpose and expressed communication/meaningful and personal to the user is Jessica’s take on the definition of interaction. This project matches it well because the user is fully aware of the result or “output” they intend to see when triggering the sensors in a certain way, and they receive this every time.
• Some feedback/questions:
  • How to decide on what constitutes “monstrous” features in order to achieve the desired effect of creating an ugly creature that people have been conditioned by society to be repulsed by, as goes the premise of the project?
  • How will she determine whether people became enamoured by the monster at the end?
    • Suggestion: at the end of the game/the monster’s growth cycle, add a question/short survey asking the user if they love their creature/if they think it is cute — in this case, add another question at the beginning of the game as well asking if the user thinks the creature is aesthetically pleasing

Project 3: “Tip, Tap, Snap” by Justin Wu

• This project is inspired by the once wildly popular game of Guitar Hero but with a picture capturing program to capture the moment unbeknownst to the user. Uses a pressure sensor for the buttons to click according to the colors on screen
• Two parties bouncing ideas off of each other while creating 1+1>2 effect with creativity is Justin’s take on the definition of interaction. In this case, the project fulfills this as the output is greater than the inputs because not only are the colored “beats” pushed and met on time, the system also captures a photo of the user.
• This project is insteresting in that it adds the nostalgic effect by using old music that most people know, as well as incorporating the snap which gives the users a more authentic memory to hold onto the experience by catching them off guard during the game.
• Some feedback:
  • Add a filter to the picture, such as one for the winner indicating that they won and another one for the loser indicating that they lost
  • For the two player game, split the screen down the middle vertically instead of horizontally so that it is easier for the players to distinguish between their respective beats by each standing on one side of the screen as they would anyway instead of having it split between top and bottom which can potentially be more confusing to the users.

Feedback/suggestions received for my project: “Tap the World”

• Add sound effect – maybe a swish for every time the cap makes it through one of the holes
• Increase the degrees by which it rotates every time a cap makes it through a hole to give user a more immediate/substantial response
• Count the number of caps that have made it through on screen
• Making it into hole will evoke a message onto screen with a fun fact about climate change
• Add a count of the amount of full revolutions
• Or make each cap show up on the screen and rotate around the earth
• Prompt users to throw it, clarify that it is a game — put some simple instructions somewhere
• Make an LED light up whenever a cap makes it in

For this recitation, I uploaded an image of a field to Processing and used an image of a rose with a transparent background. This rose is to be controlled by two potentiometers in terms of its position on the field, to keep creating more roses on the field.

I ran into trouble a few times due to the sizing of the images compared to the size of the virtual ‘canvas’ but I was able to figure out how to resize the images to fit my canvas (I did not want to change the size of my canvas to fit the images as that would have made it far too large — probably bigger than my computer screen).

Here’s the Processing code:

// IMA NYU Shanghai

// Interaction Lab

// For receiving multiple values from Arduino to Processing

/*

* Based on the readStringUntil() example by Tom Igoe

* https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html

*/

PImage photo2;

PImage photo;

float posX2 = 0;

float posY2 = 0;

import processing.serial.*;

String myString = null;

Serial myPort;

int PORT_INDEX = 1;

int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/

int[] sensorValues;      /** this array stores values from Arduino **/

void setup() {

 size(500, 500);

 photo = loadImage(“yellowrosecopy.png”);

 photo2 = loadImage(“grassfieldcopy.png”);

 background(photo2);

 setupSerial();

}

void draw() {

 updateSerial();

 printArray(sensorValues);

 // use the values like this!

 // sensorValues[0]

 // add your code

 float posX = map(sensorValues[0], 0, 1023, 0, width);

 float posY = map(sensorValues[1], 0, 1023, 0, height);

 stroke(255);

 image(photo, posX2, posY2);

 posX2 = posX;

 posY2 = posY;

 

 

}

void setupSerial() {

 printArray(Serial.list());

 myPort = new Serial(this, Serial.list()[ PORT_INDEX ], 9600);

 // WARNING!

 // You will definitely get an error here.

 // Change the PORT_INDEX to 0 and try running it again.

 // And then, check the list of the ports,

 // find the port “/dev/cu.usbmodem—-” or “/dev/tty.usbmodem—-”

 // and replace PORT_INDEX above with the index number of the port.

 myPort.clear();

 // Throw out the first reading,

 // in case we started reading in the middle of a string from the sender.

 myString = myPort.readStringUntil( 10 );  // 10 = ‘\n’ Linefeed in ASCII

 myString = null;

 sensorValues = new int[NUM_OF_VALUES];

}

void updateSerial() {

 while (myPort.available() > 0) {

   myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’  Linefeed in ASCII

   if (myString != null) {

     String[] serialInArray = split(trim(myString), “,”);

     if (serialInArray.length == NUM_OF_VALUES) {

       for (int i=0; i<serialInArray.length; i++) {

         sensorValues[i] = int(serialInArray[i]);

       }

     }

   }

 }

}

Here is the Arduino code:

// IMA NYU Shanghai

// Interaction Lab

// For sending multiple values from Arduino to Processing

void setup() {

 Serial.begin(9600);

 //pinMode(7,INPUT);

}

void loop() {

 int sensor1 = analogRead(A0);

 int sensor2 = analogRead(A1);

// int sensor3 = analogRead(A2);

 // keep this format

 Serial.print(sensor1);

 Serial.print(“,”);  // put comma between sensor values

 Serial.print(sensor2);

 Serial.println();

// Serial.print(“,”);

// Serial.print(sensor3);

// Serial.println(); // add linefeed after sending the last sensor value

 // too fast communication might cause some latency in Processing

 // this delay resolves the issue.

 delay(100);

}

Tap the World

Over the past 3 decades, climate change has been an increasingly contentious issue, an imminent threat subject to intense scrutiny and debate. Arguments aside, it is extremely relevant at this moment which is why this project is inspired to try to actively add to the worldwide effort to mitigate the problem. From a recycling company’s perspective, plastic bottle caps (already separated from the bottle) are much more useful as they can be more efficiently categorized and recycled, partially due to their smaller size combined with the higher concentration of plastic. Moreover, this project aims to provide an alternate and entertaining way to participate in the global cause.

In order to fully understand our objective, the project needs a detailed plan. We plan to create a world (2-Dimensional earth) using a laser cutter, which will include three holes of difference sizes. The holes will be assigned different values depending on their size: the smallest one will be worth 5 points, the middle hole will be worth 2 points, and the biggest one will be worth 1 point. With Processing we will create a spherical planet earth (globe) that rotates. Each time a person is able to throw a plastic bottle cap successfully through one of the holes, the world will rotate. The degree of rotation will depend on the value of the hole the bottle cap went through; for example, if the bottle cap goes through the smallest hole, the program will translate this to 5 degrees and rotate the globe accordingly. The successful passage of the cap through the corresponding hole will be detected by motion sensors placed in each hole, which will send this information to Arduino which will then translate and communicate with Processing to make the world spin. Once the player has reached 360 degrees – has caused the world to make one full revolution – the player has achieved the ultimate goal of recycling the bottle caps.

In order to achieve this in time here is the plan in steps:

1. We will first laser cut the world and complete that design (April 27)
2. Begin to figure out the values for Arduino and Processing, and work with the sensors and circuits (April 28)
3. User testing session
4. Depending on user test results and feedback, make any changes that will improve the project.
5. Present “Tap the World” (May 14)

The project is also unique in that it provides a fun and interactive way for people to recycle, and more importantly offers positive reinforcement to encourage the continuation of this type of environmentally-friendly behavior. This is something I believe is severely lacking in this global cause: when a person places a plastic bottle in the correct recycling bin, nobody/nothing cheers. This would try to fill this void. Moreover, we plan to place this type of project next to already existing recycling bins for plastic bottles so that it’s not a hassle for a person to keep the bottle caps when throwing out a bottle and then having to go throw the cap out somewhere else. This could have a greater impact in NYU Shanghai’s recycling efforts, for example. Eventually it could be incorporated on a bigger scale and we can send the caps to organizations like the aforementioned one to help two causes at once.

Tap the World absolutely fulfills my criteria to be considered an interactive design or project, since it incorporates the users and provides a response in a timely manner to a particular action placed by the user. Moreover, the human action initiates the interaction between Arduino and Processing, therefore the interaction necessarily takes the user’s input into account in order for the system to fully function. This fully matches my interpretation of interaction.