“Save Me From Plastic-Lifan Yu-Inmi Lee

 Brief introduction:

This game focuses on the environmental problem of ocean plastic pollution. We designed an interactive game that encourages people to use less plastic product and find ways to treat plastic wastes that do less harm to our environment (and ourselves)–stop dumping them into natural environment! 

 CONCEPTION AND DESIGN:

In general, when I think of how my users are to interact with my project, I would like them to move around, be actively engaged and achieve the winning result with combined effort.

First, to move around. When we thought of how users can interact with images of falling plastic trash pieces on the screen, we first thought about pushing buttons. But this can’t get users to move around. So, we decided that half of the screen should have a camera-captured real-time image as a background. This way users’ image actually appears on the screen along with images of plastic trash. Users thus can move around, changing their places in the image and interact with image of falling pieces of plastic that’s laid on top of the camera image. To determine whether, in users’ image, they successfully touched the plastic pieces, a threshold is set. When falling images move pass a place whose colors are darker than the color this threshold represents, these images disappear. Thus the results looks like when users wear dark-colored gloves and their images on the screen “touches” the falling trash with those gloves, it counts as they have “successfully blocked the plastic pieces from falling into the sea”. (Dark-colored clothes can also work) This is effective to encourage users to move around.

Second, to be actively engaged. This game is a continuous, quick-paced game. Plastic trash pieces fall from the sky continuously and the already existing pieces of plastic in the ocean move around all the time. Users have to be focused on the game in order no to let the fish die. At first, we did not add the already existing plastic pieces images in the ocean. However, this is problematic because if one player is very good at “catching” all the falling plastic pieces, the other player don’t need to control the fish at all. Then we added extra plastic pieces in the sea to encourage two users to actively play this game from the beginning.

Third, to win this game with combined effort. We once thought of making a one-player game. If more plastic pieces fall into the sea, the color of sea water will change and there will appear dead fish in the water. When the dead fish reaches a certain amount, game over. However, we thought of offering players another experience in the fish’s viewpoint. Thus we added the fish part.

We wish to raise environmental awareness by providing our users with information about plastic pollution in ocean. There was an option that we rejected was a candy box that automatically opens when players win. We once thought of putting on the candy wrappers some sentences of call to actions and scientific facts about current environmental conditions. Therefore, users are more willing to take in all those information. However, our motor didn’t work well in our device and we just didn’t add it to our project.

FABRICATION AND PRODUCTION:

In user testing, we received suggestions to add a video of ocean in the background of the “fish” part. We also received suggestions that we should provide users with some gloves with colors that rarely appear on clothes. In our code, we can write some lines that detect the color of these gloves so that as soon as they appear on at the same place with falling plastic pieces,  the plastic pieces will disappear. Back then, we didn’t add the already-existing plastic pieces in the ocean, so all players just focused on “blocking” plastic pieces. Nobody went to control the fish’s movements.

We later added a video of ocean in the background. However, this didn’t work very well because our video showed fish swimming. During presentation, some users reported that they get confused about which fish they are controlling—fish in the background video or the hand-drawn fish.

We bought bright pink gloves and black gloves, but we ultimately chose black gloves to provide to users. However, this turned out to be a wrong choice because one of our users wore black clothes and can very easily “catch” the falling plastic pieces images by stretching out her hands.

CONCLUSIONS:

We tried to use a fun and interactive game to raise environmental awareness among our users. My definition of interaction is that people and device can receive information from and provide feedback to each other. This process is better if this communication can go on and on if both sides keep interacting. My project allows users to receive information by seeing the falling plastic pieces and try to block them from falling into the sea. When one piece of plastic is blocked, other plastic pieces will still keep falling down. People can keep interacting until the game ends. Referring to the game itself, a camera is used to detect users’ movement information and decide whether their images on the screen actually “touched” the plastic pieces or not. According to different actions the users make–successfully caught the trash pieces or accidentally let them fall into the ocean—the game itself also gives different feedbacks. When a plastic piece fall into the sea, it will start moving randomly in the “sea area” and threaten the fish. If users don’t control the fish and hide away from plastic pieces, it will die. Different actions done by users can lead to the game’s winning or game ending. The whole process is an active, interactive process.

If we had more time, we would change our “keypress” into pushbuttons. We can place several neat little buttons on laser-cut boards in front of users with captions like “start” “restart” etc.

We can also use a camera that connects to the computer and hide the computer away. This way, we can change the distance of the camera and the user instead of placing a computer in front of users (which made their image appear to be very large and they can too easily “catch” the plastic pieces.) After this, our whole project can look more simple and neat.

We can also add some sound effects so that as soon as users “catch” a piece of plastic trash, a sound can be played.

Finally, the page that contain scientific data and call to action can be shown to users before the game starts instead of after the game ends, because users are usually too busy restarting the game instead of stopping and taking a look at those words.

I’ve learned that if I want to raise awareness, I should focus more on how to make everyone feel like “I really should care”. Our project simply stated the current disastrous situation of ocean animals, but our game didn’t show how this affects us and wasn’t touching enough to encourage people to care. Moreover, it would be better if we made the whole thing look more pretty that people will be very interested in interacting with it as soon as they see it.

I was glad that some people really liked the “blocking the falling plastic pieces” part. They thought it was interesting. One person even did research into plastic pollution after user testing and decided to buy less drinks contained in plastic bottles and use less plastic products. I wish if I have another chance in the future, I can refine the whole game and make it look more attractive, so that we can let more people know about this serious environmental issue in a fun way. Taking actions to slow down our environment worsening can’t wait another day. However, if I directly call for everyone to take actions, no one will be willing. This project we made can be of some help to interest people in this often-ignored environmental problem that actually decide whether we and ocean animals live or die.

PART OF OUR CODE

(the part when fish is alive.

code that decide the winning and losing of this game)  

//PART OF OUR CODE
//(the part when fish is alive & code that decide the winning and losing of this game)   

 if (ok==true) {




      for (int k=0; k<plasticList.size(); k++) {
        Plastic temp=plasticList.get(k);
        temp.display();
        temp.move();
      }





      if (fishY<=height/2+10) {
        fishY=height/2+10;
      }

      if (fishY>=height-40) {
        fishY=height-40;
      }

      if (fishX<=30) {
        fishX=30;
      }

      if (fishX>=width-30) {
        fishX=width-30;
      }

      if(sensorValues[0]==1){
      fishY-=fishSpeed;
      }

      if(sensorValues[1]==1){
      fishY+= fishSpeed;
      }

      if(sensorValues[2]==1){
      fishX-= fishSpeed;
      }

       if(sensorValues[3]==1){
      fishX+= fishSpeed;
      }


      image (img1, fishX, fishY, 100, 100);
    }

    for (int j=0; j<plasticList.size(); j++)//When fish touches plastic, it dies
    {
      Plastic nmsl=plasticList.get(j);
      float dis = sqrt((fishX-nmsl.x)*(fishX-nmsl.x)+(fishY-nmsl.y)*(fishY-nmsl.y))-nmsl.size*0.4;
      if (dis<=0) {
        ok = false;
      }
    }


    if (ok==false ) {//if a fish dies, game over, users lose the game
      image(img6, width/2, height/2, width, height);
   //   println("show game over");
    }


    if (key == 'c' && ok == false ) {//see the facts page
      image(img8, width/2, height/2, width, height);
     //  println("show facts");
    }


    if ( key=='s' && ok==false ) {//restart after losing the game
      ok=true;
      win = 0;
      start = 0;
      int l = plasticList.size();
      for (int i = l-1; i>=0; i--) {
        plasticList.remove(i);
      }

      int r = fallingplastics.size();
      for (int s = r-1; s>=0; s--) {
        fallingplastics.remove(s);
      }
    
    }

    if (win == 1 ) {//this means when users win, the fish is alive whatsoever
      //myPort.write('1');
      ok = true;
      fill(255, 34, 899);
      rect(0, 0, width, height);
      textSize(40);
      fill(376, 678, 222);
      text("You Won!!Press 's' to restart", width/3, height*2/3);
      textSize(45);
      fill(255);
      text("You won the game.", width/2,height/5);
      textSize(30);
      text("But there are countless fish that survive by chance like this", width/2, height/4);
      text("We play a crucial role in deciding their lives and deaths",width/2,height/3);
      text("Less Plastic, less disaster", width/2, height/2);
    }

    if (fallingplastics.size()>=fallingplasticsWinning && ok==true) {//After a certain amount of time, users win
      win = 1;
      fill(255, 34, 899);
      rect(0, 0, width, height);
      textSize(40);
      fill(376, 678, 222);
      text("You Won!!Press 's' to restart", width/3, height/2);
      fill(255);
      text("Save our ocean!", width/2, height/4);
      text("Less Plastic, less disaster", width/2, height/3);
    }

    if (keyPressed && key=='s' && win==1) {//restart the game after winning
      ok = true;
      win = 0;
      start = 0;
      int l = plasticList.size();
      for (int i = l-1; i>=0; i--) {
        plasticList.remove(i);
      }

      int r = raindrops.size();
      for (int s = r-1; s>=0; s--) {
        raindrops.remove(s);
      }
    }
  }
}


class Plastic {
 
 
  float x,y;
  float size;
  float speedX;
  float speedY;
  Plastic(float startingX, float startingY){
    x=startingX;
    y=startingY;
    size=random(50,150);

    speedX=random(-10,10);
    speedY=random(1,15);
  }

    void display(){
      image(img2,x,y,size,size);
  }
 

  void move(){
    x+=speedX;
    y+=speedY;
    if(x<=0||x>=width){
      speedX=-speedX;
    }
   
    if(y<=height/2||y>=height){
      speedY=-speedY;
    }
   
  }
 
}

Recitation 10: Object Oriented Programming by Lifan Yu

Recitation 10: Object Oriented Programming by Lifan Yu (ly1164)

My object called “Plastic” contain all images of plastic bottles and their movements. This way, we can not only control the common ways of their movements (or their common parameters)  but also control their movements and parameters individually.

Source of the used image: Lifan Yu

Code:

ArrayList<Plastic> plasticList;
 PImage img2;
 
 void setup(){
   size(800,800);
 plasticList = new ArrayList<Plastic>();
  img2=loadImage("plastic.png");
 }
 
 void draw(){
   background(255);
   imageMode(CENTER);
    for(int k=0; k<plasticList.size(); k++){
    Plastic temp=plasticList.get(k);
    temp.display();
    temp.move();
   
  }
 }
 
 
   void keyPressed(){
  float x=map(mouseX,0,width,width/4,3*width/4);
  float y=map(mouseY,0,height,height/4,3*height/4);
  plasticList.add(new Plastic(x,y));
  
    
}

Create another tag called “Plastic”:

class Plastic {

  float x,y;
  float size;
  float speedX;
  float speedY;
  Plastic(float startingX, float startingY){
    x=startingX; 
    y=startingY;
    size=random(50,150);

    speedX=random(-10,10);
    speedY=random(1,15);
  }

    void display(){
      image(img2,x,y,size,size);
  }
  

  void move(){
    x+=speedX;
    y+=speedY;
    if(x<=0||x>=width){
      speedX=-speedX;
    }
    
    if(y<=0||y>=height){
      speedY=-speedY;
    }
    
  }
  
}

Recitation 9: Media Controller–Lifan Yu

Recitation 9: Media Controller

Lifan Yu (ly1164)

       In this recitation I changed the parameters of an image I painted, using Arduino and processing.

  1. I used the function “tint()” to change the colors. I used potentiometers to do this through serial communication.
  2. Also, when a button is pressed, my image starts moving and bounces when it touches edges.
  3. I used another potentiometer to control the blurring of this image using filter(BLUR, b);
  4. I used one more potentiometer to control zooming by adding the array of potentiometer data into my image’s size parameters:  float n=map(sensorValues[1],0,1023,0,500);  image(img1, x, y, n, n);

Zoom in and zoom out:

Tint:

Before applying these data, I mapped the original data from Arduino to suitable ranges for each parameter.

Code for Arduino:

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing

void setup() {
Serial.begin(9600);

}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
int sensor3 = digitalRead(6);
int sensor4 = analogRead(A3);

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.print(“,”);
Serial.print(sensor3);
Serial.print(“,”);
Serial.print(sensor4);
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Code for Processing:

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 4;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/
PImage img1;
float m=0;
float n=0;
float x=0;
float y=0;
float speedX=10;
float speedY=10;


void setup() {
  size(1000, 1000);
  background(0);
  setupSerial();
  img1 = loadImage("Witch.png");
}


void draw() {
  updateSerial();
  printArray(sensorValues);

  // use the values like this!
  // sensorValues[0] 
  imageMode(CENTER);
float m=map(sensorValues[0],0,1023,0,500);
  float n=map(sensorValues[1],0,1023,0,500);
  float b =map(sensorValues[3],0,1024,0,10);
    
  // add your code
  background(0);
  tint(m,m,m,m);
  
image(img1, x, y, n, n);
filter(BLUR, b);

if (sensorValues[2]==0 ){
  x=300;
  y=300;
}else{
x=x+1.7*speedX;
y=y+speedY;
if(x>width||x<0){
  speedX=-speedX;
}
if(y>height||y<0){
  speedY=-speedY;
}
}

  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 0], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

       The way that technology is used in my project is the serial communication that allows users to change parameters of a picture without actually typing in those data in a computer. Rather, this process is done by interacting with physical devices, changing their input values–indirectly but simply and conveniently changing parameters into whatever values we like. This only requires an extra step of mapping values into new ranges (that fit their parameters) after we have built up a serial communication between Arduino and Processing. Using this technology, even people who don’t know much about computer programming can easily interact with that image and change it’s features.

      

Final Essay by Lifan Yu

Final Essay

By Lifan Yu

A. PROJECT TITLE: “Save Me From Plastics”

 B. PROJECT STATEMENT OF PURPOSE:

      Me and Xinran focus on an environmental problem: plastic pollution. Plastic in the ocean harm marine animals. Animals die from accidentally eating plastic pieces because they stay in their bodies and cause them diseases. Also, plastic is difficult to decompose, they stay in natural environment for a long time.

According to a recent study, humans have produced 8,300,000,000 tons of plastic waste ever since the year of 1950. However, most of these plastic wastes were not properly treated. Only 9% of them are recycled, 12% are burned. However, as much as 79% of all these wastes were dumped into landfills or thrown into the ocean.

According to professor Jenna Jambeck from University of Georgia, most of these plastic wastes do not have any biodegradations. They will stay in these natural environments for hundreds or thousands of years. Not only will this harm wild animals but also lessen safe lands that we can live in.

Inspired by these, we try to design an interactive game that encourages people to use less plastic product and find ways to treat plastic wastes that do less harm to our environment. Our intended audience is people who are still not fully aware what a huge negative impact we have on our environment with all those plastic trash. We intend to raise environmental awareness, maybe people can have a more careful thought about this matter after enjoying this game and adopt a way of living with less use of plastic products.

C.PROJECT PLAN

      We plan to create an interactive two-players game that raises awareness on plastic pollution and promote environmental actions.

The game screen can be projected on a wall like this:

      Our game screen looks like this:

     Players stand in front of this wall. One player controls the motion of a person. Another player controls the motion of a fish in the sea. Plastic products like plastic bottles will fall from above (one at a time, but they fall from random places) , representing all the tempting plastic products we encounter each day and use naturally without thinking about their impact on our environment. If the person does nothing, the products are automatically being “accepted” for use by this person and soon thrown into the sea—just like what garbage treatment people do with plastic waste—dumping them into the sea. If the person moves around and purposefully “reject” using plastic products (in this game, by blocking them), those products will disappear, representing less usage of plastic products.

      The first player controls motion of the person by pushing four separately placed buttons in front of them. The person on the screen move to different places when different buttons are pushed. The player controls motions of the fish using a handle. This player tries to avoid touching all the randomly moving plastic pieces.

      Whenever the person successfully rejects the usage of a plastic product, a green light beside the screen will blink. The green light has a caption on “I used less plastic product today!” When they fail to reject, a red light with a caption of “used one more plastic product” will blink.

      There will be some already existing plastic pieces in the sea, but there will be more when more plastic pieces are thrown in to the sea by people. If the fish touches any pieces of plastic, it dies and people lose the game. If the fish is still alive after a certain period of time (such as two minutes), the players win this game.

      When they win the game, a sign (attached to a servo motor) will began to wave and instruct them to pick up a candy. they can pick up a candy from a candy box beside. Another computer is put beside the candy box. There will be a quiz question about our main theme: “About how many percentage of plastic waste is thrown into the ocean and landfills?” A. 10%, B. 30% C. 50% D. 80%. Four buttons represent A B C D. When D is pressed, a servo motor will more the candy box out and players can take candies from it. We will leave a piece of post-it note on each candy, we will write on them some actions people can take to minimize plastic pollution.

 

empathize with our intended audience:

      People usually don’t like it when they are told seriously about some distressing environmental problems. So this game introduces how we accept using lots of plastic products naturally and how hard life will be for those poor fish surrounded by plastic waste in a simple and fun way.

 

Our game is both interactive, fun and rewarding: players can not only gain knowledge of plastic pollution, be more aware of this environmental problem but also get delicious candy. Before they get candy, the quiz question will leave a deep impression on them. This will be a fun and effective way of promoting environmental actions.

Detailed Plan:

      We will finish building the candy box circuit and the code of the small candy device, and the code of the background image and the ‘person’ part of the game. In the second week, we will finish all code and the main device, the last week is for user testing and improvements.

D. CONTEXT AND SIGNIFICANCE:

Projects I’ve researched:

      Scott Snibbe’s “Arctic Ice”

      “Arctic Ice is an interactive immersive exhibit on global warming that tells the complicated story of feedback loops in melting arctic ice that are a major component of climate change.”

      People are actually experiencing “being part of this process”. They wave their hands to help “reflect” sunlight. At the same time, they see how the feedback loops warm up the ocean and melt ice—which greatly affect polar bears—by decreasing their living areas. This game, as is shown in the video, is highly interactive. Real-time feedback animations such as the melting of ice and the reflection of sun rays are happening all the time.

These inspired us to create a project also highly-interactive: plastics moving randomly in the sea, the player controls the fish to move around, these are all happening all through the whole game experience. People are also playing decisive roles: the amount of plastics in the “sea” is decided by the first player. Also, people really experience the hard lives of marine animals in an ocean environment full of plastics by trying to avoid plastic pieces in the “sea”.

Align with my definition of interaction:

      the process of an actor (a device, a person, an animal etc.) constantly communicating with the other “actor”. That is, it can detect the information of the other “actor” constantly. The other actor can also respond to the device’s movements, sounds etc. This type of communication can last for a long time instead of ending after a few simple movements.

      The project we will make aligns well with this. People move around and control movements of their game characters actively according to the moving images. And different things “such as the dying of the fish” will happen when people do certain things or fail to do certain things. This game lasts for a certain time. In the game process, feedbacks will be made constantly.

What’s unique and significant about our project/Values/Audience:

      Our project allows players to play a role in a serious environmental problem in a fun way.

      This can promote an environmentally-friendly value of using less plastic product (generating less plastic waste) and stop disposing them irresponsibly into natural environment.

      For people who are not yet aware of the seriousness and urgency of this environmental issue, this project, if successfully made, will lead them at least to think about plastic waste problems seriously, or more, inspiring them to use less plastic products in daily life and use more environmentally-friendly products. This will lead people to reflect on their own actions: using all those plastic products without thinking much about the environment. Also, this game leads people to empathize with the poor fish. Influencing people to take small and practical actions is more effective than only calling out blank slogans.

      Subsequent projects can be built upon the already existing game rule, but adapt a better form of interaction, such as using the people’s shadows on the screen instead of requiring people to push buttons. Or, there can be more physical models built that can move to give audiences a more real experience.

      Subsequent projects can also be better connected to the world in real time and make real changes in the world. Such as, the United Nation’s “Free Rice. Com”.

When we answer one question right on this website, they will donate one grain of rice to people in hunger all cross the world. I hope our future project can adapt a system like this. I would like my projects to focus on the themes of bettering our world, saving our environment and helping people in need. By presenting these values, these projects are truly meaningful. Targeted audience are those who not yet know much about a certain contemporary problem. We can raise awareness and open people’s eyes to things that they don’t usually observe about this world.

     

Recitation 8: Serial Communication by Lifan Yu

Recitation 8: Serial Communication

By Lifan Yu (ly1164)

       Exercise 1: Make a Processing Etch A Sketch

This one is my FAVOURITE WORK! Drawing the letters “NYU” using my potentiometers:

At first, my ellipse could only move up and down. It could not move to left and right. Then I found out that one of my potentiometers wasn’t connected well enough to the breadboard. I pressed on my potentiometers. Then the ellipse can move to every direction.

Then, when I wrote the code for the next step—drawing lines and ran my code, I saw dots, mot lines, like this:

(As is shown in the screenshot below) This is because I put “float m=map(sensorValues[0],0,1023,0,800);float n=map(sensorValues[1],0,1023,0,800);” and “m1=m; n1=n;” all in front of “line(m1,n1,m,n);”.

This is the same thing as sending the values of sensorValue[0]and [1] directly to m1 and n1. When these code are put together, m has the same value as m1, so do n and n1. However, to make a line, the values of m1 and n1 should be given values a step after m and n received their values. This ensures that point (m1, n1) and (m, n) are different points. (m1, n1) is a previous point, (m, n) is a new point.

However, when I put “m1=m, n1=n” after “line” function, I added “float” in front of “m1” and “n1”.

My lines then looked like this:

After I deleted “float”, I finally got this right. (As is shown below)

Schematic of my circuit:

Reflection on Interaction:

For the circuit, the input is the potentiometers’ rotation controlled by me. The “processing” process is how the motion was uploaded to my computer, collected in “sensorValues[0]” and “sensorValues[1]” and then sent to processing through serial communication. The output the ellipses and lines appeared on the canvas.

For the user: input: seeing lines on the canvas, seeing positions of the lines.

Thinking: according to what have been seen, deciding where to draw the next line.

Output: according to decisions, rotate the potentiometers to draw lines.

My code:

For Arduino

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing

void setup() {
Serial.begin(9600);
pinMode (11, OUTPUT);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
//int sensor3 = analogRead(A2);

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
//Serial.print(“,”);
//Serial.print(sensor3);
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

For Processing 

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/
float m=0;
float n=0;
float m1=0;
float n1=0;


void setup() {
  size(1000, 1000);
  background(0);
  setupSerial();
}


void draw() {
  updateSerial();
  printArray(sensorValues);

  // use the values like this!
  // sensorValues[0] 
  //fill(random(0,500), random(0,500),random(0,500));
  float m=map(sensorValues[0],0,1023,0,800);
  float n=map(sensorValues[1],0,1023,0,800);
   
 
  strokeWeight(3);
  stroke(255,255,255);
  line(m1,n1,m,n);
 m1=m;
 n1=n;
 
 
//ellipse(m,n,30, 30);
  // add your code

  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 0 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Exercise 2: Make a musical instrument with Arduino:

       I used an array whose values can be set to 1 or 0 to represent “mousePressed==true” and “mousePressed==false”. Like this:

       “mousePressed” can’t be used in Arduino, but this array can sent information of mouse interactions to Arduino through serial communication. Like this:

At first my code did not work. Tones could change according to my mouse’s X coordinate, but the buzzer kept working even when my mouse wasn’t pressed. Then I found out that I forgot to use double “=” in the if statement in Arduino. (As is shown below)

After adding another “=”, my circuit began to work properly.

Schematic of my circuit:

Reflection on Interaction:

For circuit: input: the X positions of the mouse and information of mouse interactions that are stored in arrays.

“thinking”: The process where values stored in arrays were sent to the circuit through serial communication.

Output: The buzzer works according to values sent to it.

For the user: input: hearing the buzzer’s sounds.

Thinking: according to the buzzer’s sounds, deciding on what tunes can be played next..

Output: according to decisions, move the mouse to different places on the canvas.

My code:

For Processing

// IMA NYU Shanghai
// Interaction Lab


/**
 * This example is to send multiple values from Processing to Arduino.
 * You can find the arduino example file in the same folder which works with this Processing file.
 * Please note that the echoSerialData function asks Arduino to send the data saved in the values array
 * to check if it is receiving the correct bytes.
 **/


import processing.serial.*;

int NUM_OF_VALUES = 2;  /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/


Serial myPort;
String myString;

// This is the array of values you might want to send to Arduino.
int values[] = new int[NUM_OF_VALUES];

void setup() {
  size(500, 500);
  background(0);

  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 0 ], 9600);
  // check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index of the port

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
}


void draw() {
  background(0);
  
if (mousePressed){
  values[1]=1;
  println("pressed");
}else{
  values[1]=0;
  println("not pressed");
}
values[0]=mouseX;
  // changes the values
  //for (int i=0; i<values.length; i++) {
  //  values[i] = i;  /** Feel free to change this!! **/
  //}

  // sends the values to Arduino.
  sendSerialData();

  // This causess the communication to become slow and unstable.
  // You might want to comment this out when everything is ready.
  // The parameter 200 is the frequency of echoing. 
  // The higher this number, the slower the program will be
  // but the higher this number, the more stable it will be.
  echoSerialData(200);
}

void sendSerialData() {
  String data = "";
  for (int i=0; i<values.length; i++) {
    data += values[i];
    //if i is less than the index number of the last element in the values array
    if (i < values.length-1) {
      data += ","; // add splitter character "," between each values element
    } 
    //if it is the last element in the values array
    else {
      data += "n"; // add the end of data character "n"
    }
  }
  //write to Arduino
  myPort.write(data);
}


void echoSerialData(int frequency) {
  //write character 'e' at the given frequency
  //to request Arduino to send back the values array
  if (frameCount % frequency == 0) myPort.write('e');

  String incomingBytes = "";
  while (myPort.available() > 0) {
    //add on all the characters received from the Arduino to the incomingBytes string
    incomingBytes += char(myPort.read());
  }
  //print what Arduino sent back to Processing
  print( incomingBytes );
}

For Arduino

// IMA NYU Shanghai
// Interaction Lab

/**
This example is to send multiple values from Processing to Arduino.
You can find the Processing example file in the same folder which works with this Arduino file.
Please note that the echo case (when char c is ‘e’ in the getSerialData function below)
checks if Arduino is receiving the correct bytes from the Processing sketch
by sending the values array back to the Processing sketch.
**/

#define NUM_OF_VALUES 2 /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/

/** DO NOT REMOVE THESE **/
int tempValue = 0;
int valueIndex = 0;

/* This is the array of values storing the data from Processing. */
int values[2];

void setup() {
Serial.begin(9600);
pinMode(13, OUTPUT);

}

void loop() {
getSerialData();

// add your code here
// use elements in the values array
// values[0] // values[1]

if (values[1] == 1) {
float m=map(values[0],0,800,200, 4000);
tone(13,m );
} else {
noTone(13);
}

}

//recieve serial data from Processing
void getSerialData() {
if (Serial.available()) {
char c = Serial.read();
//switch – case checks the value of the variable in the switch function
//in this case, the char c, then runs one of the cases that fit the value of the variable
//for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase
switch (c) {
//if the char c from Processing is a number between 0 and 9
case ‘0’…’9′:
//save the value of char c to tempValue
//but simultaneously rearrange the existing values saved in tempValue
//for the digits received through char c to remain coherent
//if this does not make sense and would like to know more, send an email to me!
tempValue = tempValue * 10 + c – ‘0’;
break;
//if the char c from Processing is a comma
//indicating that the following values of char c is for the next element in the values array
case ‘,’:
values[valueIndex] = tempValue;
//reset tempValue value
tempValue = 0;
//increment valuesIndex by 1
valueIndex++;
break;
//if the char c from Processing is character ‘n’
//which signals that it is the end of data
case ‘n’:
//save the tempValue
//this will b the last element in the values array
values[valueIndex] = tempValue;
//reset tempValue and valueIndex values
//to clear out the values array for the next round of readings from Processing
tempValue = 0;
valueIndex = 0;
break;
//if the char c from Processing is character ‘e’
//it is signalling for the Arduino to send Processing the elements saved in the values array
//this case is triggered and processed by the echoSerialData function in the Processing sketch
case ‘e’: // to echo
for (int i = 0; i < NUM_OF_VALUES; i++) {
Serial.print(values[i]);
if (i < NUM_OF_VALUES – 1) {
Serial.print(‘,’);
}
else {
Serial.println();
}
}
break;
}
}
}