Uncategorized

 

---

PROJECT TITLE – Fruit Ninja

MY NAME – Duorfan (with Kiki)

INSTRUCTOR’S NAME – Rodolfo Cossovich

---

 

Origin & Name

Garbage classification is part of our life. Inspired by the well-known game: fruit slice(fruit ninja), I wanna make a garbage slice to help us distinguish the garbage type quickly and clearly. We let 2D figures (garbage) show on the screen, then the player should distinguish the right category and pick up the right garbage bin, and then move it to catch the falling garbage. That’s why we give our project a interesting name: Garbage Ninja.

Final Idea & Concept 

Due to our immature our mid-term project, I realized that interaction was not only numerous pairs of input and output, continuous response and reaction are also playing an critical role. Besides the simple reactions, the loop of response is the essence of the concept of interaction. Great interaction project should not be a thing players can’t touch or experience, it should be intriguing instead of its deep concept which participants could not interpret by themselves. Processing could create beautiful image and interesting animation, while Arduino plays a critical role to help us connect our physical interaction with digital world. Also, the results of the loop should be various determined by different users. Because of these insights and my research, I decided to make an interactive game with educational meanings.

When my city, Wuxi, first adopted the Garbage Classification Policy, many citizens were not used to it. But after I learned the importance of garbage classification and the terrible consequences like the “garbage beaches”, where malnourished polar bears and innocent sea turtles died from consuming hundreds of plastic bags in their stomach, I wanted to make a change. Seeing how garbage classification is a necessity, I want to create a interactive game, like a the game fruit ninja, to help people practice garbage categories and raise awareness of environmental and animal protection.

Prototype

Based on the four types classified in Shanghai, we drew 12 garbage, 3 for each category. Our plan for this game changed from a Fruit Ninja game of cutting items to a random ball picking game, which match our action of throwing garbage much better. To illustrate our concept, I drew the animation by using Procreate and we split the game by first completing the Processing part. Instead of the using the sensors and Arduino, we choose to use Mouse-click() or Mouse-moved().

We originally planned to give these three outcome:

1. right bin and nice catch
2. right bin but miss the garbage
3. distinguish the wrong category, then fail.

However, we faced a lot of problems of coding in Processing since that was our first time programming for our own ideas. There are no similar example coding online, so we find the problems one by one through Google and the Github (this website is amazing). When we were spending our weekends working hard on those code, we also received help from our classmates (Siyuan and Kira, especially Siyuan). They helped us with the category part because we couldn’t find the proper function(I even wanted to use Matrix to solve it). Thanks to Processing’s comprehensive functions, we figure out how to pick and categorize items properly. The problems of defining win() to make the loop run was also solved when we came home and had a wonderful dinner, so our Processing part finally worked with mouse clicking and moving.

Code (Mouse version)

import processing.sound.*;
SoundFile sound;

import processing.video.*;
Movie movie;


PImage bottle;
PImage shoes;
PImage glass; 
PImage light;
PImage battery;
PImage medicine;

PImage banana;
PImage apple;
PImage cheese;

PImage mask;
PImage tissue;
PImage cige;

PImage bg;
PImage bg2;

float xPos = random( 0, 1000.0);
float yPos = 0.0;
float xVel=random(5, 6.0);
float yVel=random(4, 5.0);

float Xblue = 120.0;
float Yblue = 750.0;
float Xred = 340.0;
float Yred = 750.0;
float Xbrown = 560.0;
float Ybrown = 750.0;
float Xblack = 780.0;
float Yblack = 750.0;
float Xtemp;       
float Ytemp;       

float xWin;       
float yWin; 

int r;
int w = 1; 

PImage Go[];

boolean state = true;
boolean okState = false; 

//long lastTime = 0;

int level = 0;
int alpha = 255;

void setup() {
  //lastTime = millis();

  size(1000, 900);
  bottle =loadImage("bottle.png");
  bottle.resize(100, 100);
  shoes =loadImage("shoes.png");
  shoes.resize(100, 100);
  glass =loadImage("glass.png");
  glass.resize(100, 100);

  light =loadImage("light.png");
  light.resize(100, 100);
  battery =loadImage("battery.png");
  battery.resize(100, 100);
  medicine =loadImage("medicine.png");
  medicine.resize(100, 100);

  banana =loadImage("banana.png");
  banana.resize(100, 100);
  apple =loadImage("apple.png");
  apple.resize(100, 100);  
  cheese =loadImage("cheese.png");
  cheese.resize(100, 100);  


  mask =loadImage("mask.png");
  mask.resize(100, 100);
  tissue =loadImage("tissue.png");
  tissue.resize(100, 100);
  cige =loadImage("cige.png");
  cige.resize(100, 100);
  
  
  bg = loadImage("bg.jpg");
  
  bg.resize(1000, 900);

  bg2 = loadImage("bg2.jpg");
  bg2.resize(1000, 900);

  Go = new PImage[12];

  Go[0]=bottle;
  Go[1]=shoes;
  Go[2]=glass;

  Go[3]=light;
  Go[4]=battery;
  Go[5]=medicine;

  Go[6]=banana;
  Go[7]=apple;
  Go[8]=cheese;

  Go[9]=mask;
  Go[10]=tissue;
  Go[11]=cige;
  sound = new SoundFile(this, "win.aiff");
  
  movie = new Movie(this, "1080p.mp4");  
  movie.play(); 
}
void movieEvent(Movie m) {
  m.read();
}
  

void draw() {
  image(movie, 0, 0, width, height); 
    
  image(bg2,0,0); 
  tint(255, alpha);
  image(bg,0,0);
  println(level);
  tint(255,255);
  fang();        
  play1();
  textSize(50);  
  fill(#9D5630); 
  text("Point:"+level, 800, 50);
  
    
}



void fang() {     
  noStroke();
  fill(#0049D3);
  rect(Xblue, Yblue, 100, 200, 5);
  fill(#CC0303);
  rect(Xred, Yred, 100, 200, 5);
  fill(#9D5630);
  rect(Xbrown, Ybrown, 100, 200, 5);
  fill(#000000);
  rect(Xblack, Yblack, 100, 200, 5);
}

void ballmove() {                            
  image(Go[r], xPos, yPos);
  if (xPos + xVel > 950 || xPos + xVel< -50) {
    xVel = -xVel;
  }  
  if (yPos + yVel > 610 || yPos + yVel < -50) {
    yVel = -yVel;
  }
  xPos = xPos+xVel;
  yPos = yPos+yVel;
}

void play1() {
  if (state == true) {
    state = false;
    r = int(random(0, 12));
  }
  ballmove();            
  win();
  
  
}

void win() {
  if (r >=0 && r <=2) {
    Xtemp = Xblue;
    Ytemp = Yblue;
  }
  if (r >=3 && r <=5) {
    Xtemp = Xred;
    Ytemp = Yred;
  } 
  if (r >=6 && r <=8) {
    Xtemp = Xbrown;
    Ytemp = Ybrown;
  } 
  if (r >=9 && r <=11) {
    Xtemp = Xblack;
    Ytemp = Yblack;
  }

  if (Xtemp <= xPos + 50 && Xtemp >= xPos - 50 && yPos >= Ytemp -50 &&  yPos<= Ytemp + 50) {
    Xblue = 120;       
    Yblue = 750;
    Xred = 340.0;        
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;

    yPos = 1000;    
    level = level + 1;
    alpha = 255-level*20;
    //win
    yPos = 0.0;     
    state = true;
    sound.play(); 
  }
  if (w<60 && okState == true) {   
    fill(255, 255, 0);
    textSize(80);
    text("ok", xWin, yWin); 
  
    w=w+1;
  } else {
    okState = false;     
    w = 1;    
    
  }
  
  
   
  
}

void mouseClicked() {
  if (mouseX>120 && mouseX<220 && mouseY> 750&& mouseY<900) {
    Xblue = 120;
    Yblue = 600;         
    Xred = 340.0;        
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (mouseX>340 && mouseX<440 && mouseY> 750&& mouseY<900) {
    Xred = 340;
    Yred = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (mouseX>560 && mouseX<660 && mouseY> 750&& mouseY<900) {
    Xbrown = 560;
    Ybrown = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xred = 340.0;
    Yred = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (mouseX>780 && mouseX<880 && mouseY> 750&& mouseY<900) {
    Xblack = 780;
    Yblack = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xred = 340.0;
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
  }
  
}

void mouseMoved( ) {
  if (Yblue == 600) {
    Xblue = mouseX - 50;
  }
  if (Yred == 600) {
    Xred = mouseX - 50;
  }
  if (Ybrown == 600) {
    Xbrown = mouseX - 50;
  }
  if (Yblack == 600) {
    Xblack = mouseX - 50;
  }
}
 

By the way, we found it hard to add failed section, and we thought infinite mode could be more pleasant. So, we delete the third outcome. The wrong could only miss the rubbish, but it couldn’t stop the game. If the rubbish bin misses the rubbish, the game won’t stop as well, the rubbish would bounce back again.

Sensors  

After finishing the Processing part, what we need to do is to add the sensors to connect our physical interaction with digital world. But before the programing of Arduino, we drew different methods of interaction and asked the advice from our professor Rudi first.

We knew that simply picking the right bin is not intriguing while moving the bin left and right to catch the rubbish is a better idea. In addition, we got inspirations from Jintian and Andy: adding sound effects and attractive background would be more interactive. When thinking of implementing bingo background when right bin catches the garbage, I came up with the idea of transparency. Searching the function on Processing.com, I found that the value of alpha could be changed to control the transparency. Finally, we manipulated the transition to a suitable range, which players could discover it after several bingo sound. We were glad that many classmates was shocked on the background picture changed from gray city to green forest, which gave them a big surprise.

 

When deciding whether the light sensor or the pressure sensor, we found that photo resistor was more accurate. And we try two types of distance sensor, only the ultrasonic one could work properly.

We used laser cutter to make the four garbage bin, but we use 2.5mm instead of 3mm. When using glue to stick the boards together, we experience hotness and sturdiness of 502 glue.TT

Adaptations

First of all, because we need to limit communication with testers as much as possible and allow participants to try to find answers, some of them cannot reach the location of the trash can after picking up the trash can. In order to improve this problem, we can add a short animation at the beginning of the program, which includes the video part we just learned in class. In this way, our game instructions can be more clear and organized.

Secondly, our photo resistor is not sensitive enough, sometimes the cube on the screen will not change when different boxes are not picked up. Since the brightness is different, we absolutely need to re-measure and find the most suitable value, otherwise we can add another light to control the value detected by the sensor.

In addition, we original use separate paper boards for the installation, but some students found it hard to move the box on the paper board. So, we used laser cutter to make a complete installation with smooth board and proper holes for the ultrasonic distance sensor.

Last but not least, we reversed the positions of the trash can and the movable plate to avoid hand shadows, which will affect the accuracy of the light sensor. When the track approaches the trash can, it may also be more convenient for the players to move the bins. 

 

Finished Product

When the player picks up the correct trash can, the virtual trash can on the screen will appear in the middle of the screen. At the same time, if you move the trash can on the track, the virtual trash can on the screen will follow your movement so that we can quickly collect the correct trash ourselves.

Code (final version)

Processing:

import processing.sound.*;
SoundFile sound;
import processing.video.*;
Movie movie;

import processing.serial.*;
String myString = null;
Serial myPort;  // The serial port

PImage bottle;
PImage shoes;
PImage glass;

PImage light;
PImage battery;
PImage medicine;

PImage banana;
PImage apple;
PImage cheese;

PImage mask;
PImage tissue;
PImage cige;

PImage bg;
PImage bg2;

float xPos = random( 0, 1000.0);
float yPos = 0.0;
float xVel=random(5, 6.0);
float yVel=random(4, 5.0);

float Xblue = 120.0;
float Yblue = 750.0;
float Xred = 340.0;
float Yred = 750.0;
float Xbrown = 560.0;
float Ybrown = 750.0;
float Xblack = 780.0;
float Yblack = 750.0;
float Xtemp;       
float Ytemp;       

float xWin;       
float yWin; 

int r;
int w = 1; 

PImage Go[];

int NUM_OF_VALUES = 5;
int[] sensorValues; 

int a = 0;
int b = 0;
int c = 0;
int d = 0;
int e = 0;

boolean state = true;
boolean okState = false; 

int level = 0;
int alpha = 255;

void setup() {
  size(1000, 900);  
  setupSerial();
  bottle =loadImage("bottle.png");
  bottle.resize(100, 100);
  shoes =loadImage("shoes.png");
  shoes.resize(100, 100);
  glass =loadImage("glass.png");
  glass.resize(100, 100);

  light =loadImage("light.png");
  light.resize(100, 100);
  battery =loadImage("battery.png");
  battery.resize(100, 100);
  medicine =loadImage("medicine.png");
  medicine.resize(100, 100);

  banana =loadImage("banana.png");
  banana.resize(100, 100);
  apple =loadImage("apple.png");
  apple.resize(100, 100);  
  cheese =loadImage("cheese.png");
  cheese.resize(100, 100);  

  mask =loadImage("mask.png");
  mask.resize(100, 100);
  tissue =loadImage("tissue.png");
  tissue.resize(100, 100);
  cige =loadImage("cige.png");
  cige.resize(100, 100);
  
  bg = loadImage("bg.jpg");  
  bg.resize(1000, 900);
  bg2 = loadImage("bg2.jpg");
  bg2.resize(1000, 900);

  Go = new PImage[12];

  Go[0]=bottle;
  Go[1]=shoes;
  Go[2]=glass;

  Go[3]=light;
  Go[4]=battery;
  Go[5]=medicine;

  Go[6]=banana;
  Go[7]=apple;
  Go[8]=cheese;

  Go[9]=mask;
  Go[10]=tissue;
  Go[11]=cige;
  sound = new SoundFile(this, "win.aiff");
  movie = new Movie(this, "1080p.mp4");  
  movie.play();  
}

void draw() {
  image(movie, 0, 0, 1000, 800); 
  float md = movie.duration(); 
  float mt = movie.time();
  if (mt == md) {
  image(bg2,0,0);
  tint(255, alpha);
  image(bg,0,0);  
  println(level);
  tint(255,255);
  fang();

  binmove();
  play1();
  textSize(50);  
  fill(#9D5630); 
  text("Point:"+level, 800, 50);
  updateSerial();
  printArray(sensorValues);
  a = sensorValues[0];
  b = sensorValues[1];
  c = sensorValues[2];
  d = sensorValues[3];
  e = sensorValues[4];
 }
}


void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 1 ], 9600);
  myPort.clear(); // Throw out the first reading,
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
  sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length ==NUM_OF_VALUES){
        for (int i=0; i<serialInArray.length;i++){
          sensorValues[i]=int(serialInArray[i]);
        }
      }
    }
  }
}

void fang() {     
  noStroke();
  fill(#0049D3);
  rect(Xblue, Yblue, 100, 200, 5);
  fill(#CC0303);
  rect(Xred, Yred, 100, 200, 5);
  fill(#9D5630);
  rect(Xbrown, Ybrown, 100, 200, 5);
  fill(#000000);
  rect(Xblack, Yblack, 100, 200, 5);
}

void ballmove() {                            
  image(Go[r], xPos, yPos);
  if (xPos + xVel > 950 || xPos + xVel< -50) {
    xVel = -xVel;
  }  
  if (yPos + yVel > 610 || yPos + yVel < -50) {
    yVel = -yVel;
  }
  xPos = xPos+xVel;
  yPos = yPos+yVel;
}

void play1() {
  if (state == true) {
    state = false;
    r = int(random(0, 12));
  }
  ballmove();
  bin();
  binmove( );
  win();               
}

void win() {
  if (r >=0 && r <=2) {
    Xtemp = Xblue;
    Ytemp = Yblue;
  }
  if (r >=3 && r <=5) {
    Xtemp = Xred;
    Ytemp = Yred;
  } 
  if (r >=6 && r <=8) {
    Xtemp = Xbrown;
    Ytemp = Ybrown;
  } 
  if (r >=9 && r <=11) {
    Xtemp = Xblack;
    Ytemp = Yblack;
  }

  if (Xtemp <= xPos + 50 && Xtemp >= xPos - 50 && yPos >= Ytemp -50 &&  yPos<= Ytemp + 50) {
    Xblue = 120;       
    Yblue = 750;
    Xred = 340.0;        
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;

    yPos = 1000;    
    level = level + 1;
    alpha = 255-level*20;
    //win
    yPos = 0.0;     
    state = true;
    sound.play(); 
  }
  if (w<60 && okState == true) {   
    fill(255, 255, 0);
    textSize(80);
    text("ok", xWin, yWin);     
    w=w+1;
  } else {
    okState = false;     
    w = 1;               
  }
}

void bin() {
  if (a < 600 && e > 1000) {
    Xblue = 120;
    Yblue = 600;         
    Xred = 340.0;        
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (b < 800 && e > 1000) {
    Xred = 340;
    Yred = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (c < 750 && e > 1000) {
    Xbrown = 560;
    Ybrown = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xred = 340.0;
    Yred = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (d < 750 && e > 1000) {
    Xblack = 780;
    Yblack = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xred = 340.0;
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
  }
}

void binmove( ) {
  if (Yblue == 600) {
    Xblue = e - 50;
  }
  if (Yred == 600) {
    Xred = e - 50;
  }
  if (Ybrown == 600) {
    Xbrown = e - 50;
  }
  if (Yblack == 600) {
    Xblack = e - 50;
  }
}
 

Arduino:

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing
long duration; // variable for the duration of sound wave travel
int distance;

void setup() {
  pinMode(A4, OUTPUT); // Sets the trigPin as an OUTPUT
  pinMode(A5, INPUT); // Sets the echoPin as an INPUT
  Serial.begin(9600);
}

void loop() {
  int sensor1 = analogRead(A0);
  int sensor2 = analogRead(A1);
  int sensor3 = analogRead(A2);
  int sensor4 = analogRead(A3);
 // Clears the trigPin condition
  digitalWrite(A4, LOW);
  delayMicroseconds(2);
  // Sets the trigPin HIGH (ACTIVE) for 10 microseconds
  digitalWrite(A4, HIGH);
  delayMicroseconds(10);
  digitalWrite(A4, LOW);
  // Reads the  echoPin, returns the sound wave travel time in microseconds
  duration = pulseIn(A5, HIGH);
  // Calculating the distance
  distance = duration * 0.034 / 2; // Speed of sound wave divided by 2 (go and back)

  distance = map(distance, 0, 55, 100, 1000);

  // keep this format
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.print(",");
  Serial.print(sensor3);
  Serial.print(",");
  Serial.print(sensor4);
  Serial.print (",");
  Serial.print (distance);
  Serial.println (); 

  delay(100);
}

Conclusion & Improvements 

On the premise of mutual exchange, our goal is to make our works have a practical and deep significance on the basis of attraction. Garbage classification can promote environmental protection, while many people do not learn to separate garbage for convenience. I think our project did made some progress, but there remains a long way for us to improve, like the problem of light sensor. The goal of our project is to increase the awareness of citizens on garbage classification , and let them quickly understand the category of each type of garbage, so that daily garbage classification can be carried out more smoothly. But the sensor we chose is not sensitive enough to eject garbage bin and let the bin move smoothly on the screen. During the presentation, we received some recommenations of coding stategies to improve the fluency of the bin and the sensibility of the photo resistors. Though our coding is pretty rough now, through these improvisions, it would be more comprehensive and well-organized.

 

REALLY Thanks for this class, and every IMAer in our section💜.

Media Controller:

In this recitation, I created a Processing sketch that controls media (images) by manipulating both the size of the ellipse and the tint of the lights, using two physical controllers( 10k potentiometer) made with Arduino. 

The picture on the left is revised from the ex00_pixels_get we learned in class,

and the one on the right is revised from the ex03_tint.

Arduino:

I used two potentiometers to adjust two analog value. The circuit I used is similar with the one I made in last recitation.

Here’s the image of me controlling the two values.

 

Here’s the code:

void setup() {
  Serial.begin(9600);
}
  
void loop() {
  int theValue = analogRead(A0);
  int theMappedValue = map(theValue, 0, 1023, 0, 255);
  Serial.write(theMappedValue);
 
  theValue = analogRead(A1);
  theMappedValue = map(theValue, 0, 1023, 0, 255);
  Serial.write(theMappedValue);
 
  delay(20);  
}

 

Processing:

I may choose to use video, live video via webcam, or images as the media I will control in Processing.

By the way, in this recitation, I chose the images because my video library didn’t work well. However, it turns out to be working when assistant helped me restart the application. How silly I was. lol

I practice the use of map() since the two values I use work both in the tint and the size.

Here’s the code:

PImage img;
import processing.serial.*;
 Serial myPort;  // The serial port

int a = 0;
int b = 0;

void setup() {
  size(1200, 800);
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[1], 9600);
 
  noStroke();
  //img = loadImage("sunset.jpg");
  img = loadImage("colorful lights.jpg");
  img.resize(600, 800);
}

void draw() {
  while (myPort.available () > 1) {
    // read in the last byte sent from arduino (pot 2)
    a = myPort.read();
    // and the next byte (first one was discarded)
    b = myPort.read();
  for (int i=0; i<100; i++) {
    //int size = int( random(0.1, 50) );
    int x = int( random(img.width) );
    int y = int( random(img.height) );
    // get the pixel color
    color c = img.get(x, y);
    // draw a circle with the color
    //tint(0, 0, 255, 0);
    fill(c);
    float sizea = map( a, 0, 255, 0.1, 20);
    float sizeb = map( b, 0, 255, 0.1, 20);
    ellipse(x, y, sizea, sizeb);
    
  } 
   tint(a, b, 255, 155);
   image(img, 600,0);
  }
}

void mousePressed() {
  saveFrame("mySketch.png");
}

The computer is intelligent and sensible to receive input and output by using algorithmic method. The way computers create art is like the visualization and realization of people’s thoughts, while computer programs are like ways to actually imitate what happens inside the human brain. More specifically, when we view an image, we capture it with our eyes and then process it through the brain. I can directly use the imagination in my brain to change the attributes of the image, but then I need to use various visualization methods to display the image and how the computer will generate the image. After reading this week’s article, Computer Vision for Artist and Designers, I was inspired to numerous ways technology could be used in our project. The techniques used in reading (light sensor and motion sensor) are suitable for the examples. While I only use two potentiometers in my recitation project, my final project should include more interactions that require more user participation. Because the project in reading is interactive and emphasizes the connection between the user and the project, I need to adjust the idea of my own project interaction. 

Image resource:

• Pexels.com: https://www.pexels.com/

Garbage Ninja

Step 3: Essay

When my city, Wuxi, first adopted the Garbage Classification Policy, many citizens were not used to it. But after I learned the importance of garbage classification and the terrible consequences like the “garbage beaches”, where malnourished polar bears and innocent sea turtles died from consuming hundreds of plastic bags in their stomach, I wanted to make a change. Seeing how garbage classification is a necessity, I want to create a interactive game, like a the game fruit ninja, to help people practice garbage categories and raise awareness of environmental and animal protection.

 

Even in our NYUSH, there are posters about garbage classification. Based on the four types classified in Shanghai, we drew 12 garbage, 3 for each category. Our plan for this game changed from a Fruit Ninja game of cutting items to a random ball picking game, which match our action of throwing garbage much better. To illustrate our concept, I drew the animation by using Procreate and we could split the game by first completing the Processing part and add different sensors later.

 

After reading Ernest Edmonds’ article Art, interaction and engagement, I realized that a sophisticated work of interaction should have “dynamic interaction”, which changes with the audience behaviors. Any change that happens is unpredictable, which means real interaction is not about programming the code and giving users steps to go to the end based on what we design before. A benign interaction is the thing every user has the opportunity to choose, some people will succeed while others will fail. By playing games and interacting with them, players can improve their memory of losing knowledge. This not only facilitates their daily lives, but also promotes the popularization of environmental protection and friendly lifestyles.

 

Exercise 1: Make a Processing Etch A Sketch

Image from ebay.com    ( I actually bought one exactly the same when traveling !)

Image I drew

 

(A to P) use Arduino to send two analog values to Processing via serial communication

For this exercise, I used Arduino to send two analog values to Processing via serial communication. Just like an Etch A Sketch, the potentiometer on the left control the ellipse’s x-axis movement, and the other one on the right control the ellipse’s y-axis movement. Once I have it working with an ellipse, I modify the code to use a line instead by tracing the previous x and y values.

Processing

My first draft of the code:

// Interaction Lab
import processing.serial.*;

float penX;
float penY;
String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/

float newvalue1;
float newvalue2;

void setup() {
  size(800, 800);
  background(0);
  noStroke();
  fill(60, 120, 20 );
  setupSerial();
  
  penX = width/2; // starting x position of pen
  penY = height/2; // starting y position of pen
  
 
}


void draw() {
  penX = sensorValues[0];
  penY = sensorValues[1];
  updateSerial();
  printArray(sensorValues);
  newvalue1= map(sensorValues[0], 0, 1023,0, width);
  newvalue2= map(sensorValues[1], 0, 1023,0, height);
  ellipse( newvalue1,newvalue2, 15, 20 );
  ellipse( penX,penY, 15, 20 );

}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 1 ], 9600);
  myPort.clear(); // Throw out the first reading,
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

⬆️ My original coding from the example downloaded, 

but I find a more convenient way from the Internet⤵️

http://graysonearle.com/edu/physcom/etch-a-sketch-arduino-processing/

My new coding:

import processing.serial.*;
 
Serial myPort;  // The serial port
 
// a variable to store the incoming serial data to
int inByte1 = 0;
int inByte2 = 0;
 
// keep track of old line positions
float lastX = 0;
float lastY = 0;
 
void setup() {
  size(800, 800);
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[1], 9600);
 
  background(0);
  stroke(255); 
  strokeWeight(3);  
}
 
void draw() {
  while (myPort.available () > 1) {
    // read in the last byte sent from arduino (pot 2)
    inByte1 = myPort.read();
    // and the next byte (first one was discarded)
    inByte2 = myPort.read();
  }
 
  float x = map(inByte1, 0, 255, 0, width);
  float y = map(inByte2, 0, 255, 0, height);
 
  line(x, y, lastX, lastY);
   
  // reset the lastX and Y
  lastX = x;
  lastY = y;
}
 
void keyPressed() {
  background(0);
}

Arduino

void setup() {
  Serial.begin(9600);
}
  
void loop() {
  int theValue = analogRead(A0);
  int theMappedValue = map(theValue, 0, 1023, 0, 255);
  Serial.write(theMappedValue);
 
  theValue = analogRead(A1);
  theMappedValue = map(theValue, 0, 1023, 0, 255);
  Serial.write(theMappedValue);
 
  delay(20);  
}

Exercise 2: Make a musical instrument with Arduino

This processing sketch sends values to Arduino based on mouse’s x and y positions and/or keyboard interactions.  The corresponding Arduino code should read the serial values from Processing and translate them into frequency and duration for a tone, which will be sounded by the buzzer (hint: the tone() function).

Processing

import processing.serial.*;

int NUM_OF_VALUES = 2;  /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/


Serial myPort;
String myString;

// This is the array of values you might want to send to Arduino.
int values[] = new int[NUM_OF_VALUES];

void setup() {
  size(500, 500);
  background(0);

  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 1 ], 9600);

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
}


void draw() {
  background(0);
  values[0]=mouseX;
  values[1]=mouseY;
  // sends the values to Arduino.
  sendSerialData();
  echoSerialData(200);
}

void sendSerialData() {
  String data = "";
  for (int i=0; i<values.length; i++) {
    data += values[i];
    //if i is less than the index number of the last element in the values array
    if (i < values.length-1) {
      data += ","; // add splitter character "," between each values element
    } 
    //if it is the last element in the values array
    else {
      data += "n"; // add the end of data character "n"
    }
  }
  //write to Arduino
  myPort.write(data);
}


void echoSerialData(int frequency) {
  //write character 'e' at the given frequency
  //to request Arduino to send back the values array
  if (frameCount % frequency == 0) myPort.write('e');

  String incomingBytes = "";
  while (myPort.available() > 0) {
    //add on all the characters received from the Arduino to the incomingBytes string
    incomingBytes += char(myPort.read());
  }
  //print what Arduino sent back to Processing
  print( incomingBytes );
}
 

Arduino

#define NUM_OF_VALUES 1    /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/


/** DO NOT REMOVE THESE **/
int tempValue = 0;
int valueIndex = 0;

/* This is the array of values storing the data from Processing. */
int values[NUM_OF_VALUES];


void setup() {
  Serial.begin(9600);
  pinMode(13, OUTPUT);

}

void loop() {
  getSerialData();

  // add your code here
  // use elements in the values array
  // values[0]
  // values[1]

    tone(13, values[0]);
}

//recieve serial data from Processing
void getSerialData() {
  if (Serial.available()) {
    char c = Serial.read();
    //switch - case checks the value of the variable in the switch function
    //in this case, the char c, then runs one of the cases that fit the value of the variable
    //for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase
    switch (c) {
      //if the char c from Processing is a number between 0 and 9
      case '0'...'9':
        //save the value of char c to tempValue
        //but simultaneously rearrange the existing values saved in tempValue
        //for the digits received through char c to remain coherent
        //if this does not make sense and would like to know more, send an email to me!
        tempValue = tempValue * 10 + c - '0';
        break;
      //if the char c from Processing is a comma
      //indicating that the following values of char c is for the next element in the values array
      case ',':
        values[valueIndex] = tempValue;
        //reset tempValue value
        tempValue = 0;
        //increment valuesIndex by 1
        valueIndex++;
        break;
      //if the char c from Processing is character 'n'
      //which signals that it is the end of data
      case 'n':
        //save the tempValue
        //this will b the last element in the values array
        values[valueIndex] = tempValue;
        //reset tempValue and valueIndex values
        //to clear out the values array for the next round of readings from Processing
        tempValue = 0;
        valueIndex = 0;
        break;
      //if the char c from Processing is character 'e'
      //it is signalling for the Arduino to send Processing the elements saved in the values array
      //this case is triggered and processed by the echoSerialData function in the Processing sketch
      case 'e': // to echo
        for (int i = 0; i < NUM_OF_VALUES; i++) {
          Serial.print(values[i]);
          if (i < NUM_OF_VALUES - 1) {
            Serial.print(',');
          }
          else {
            Serial.println();
          }
        }
        break;
    }
  }
}
 

Additional Homework

Use two pushbuttons connected to Arduino to control the display of two stars in Processing.

Arduino:

void setup() {
  Serial.begin(9600);
}

void loop() {
  int sensor1 = digitalRead(7);
  int sensor2 = digitalRead(8);
  
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  
  Serial.print(sensor2);
  Serial.println(); // add linefeed after sending the last sensor value

  delay(100);
}

Processing:

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] Values;      /** this array stores values from Arduino **/
int previousValues0 = 0;
int previousValues1 = 0;

boolean star1 = false;
boolean star2 = false;

void setup() {
  size(600, 600);
  background(0);
  setupSerial();
}


void draw() {
  updateSerial();
  printArray(Values);
  background(0);
  
  if (Values[0] == 1 && Values[0] != previousValues0) {
    star1 = !star1;    
  }
  
  if (Values[1] == 1 && Values[1] != previousValues1) {
    star2 = !star2;   
  }
    
  if(star1 == true){
  pushMatrix();
  translate(width*0.3, height*0.3);
  rotate(frameCount / 400.0);
  star(0, 0, 30, 70, 5); 
  popMatrix();
  }
  
  if (star2 == true) {
  pushMatrix();
  translate(width*0.7, height*0.7);
  rotate(frameCount / -100.0);
  star(0, 0, 80, 100, 40); 
  popMatrix();
}
  
  previousValues0 = Values[0];
  previousValues1 = Values[1];

}


void star(float x, float y, float radius1, float radius2, int npoints) {
  float angle = TWO_PI / npoints;
  float halfAngle = angle/2.0;
  beginShape();
  for (float a = 0; a < TWO_PI; a += angle) {
    float sx = x + cos(a) * radius2;
    float sy = y + sin(a) * radius2;
    vertex(sx, sy);
    sx = x + cos(a+halfAngle) * radius1;
    sy = y + sin(a+halfAngle) * radius1;
    vertex(sx, sy);
  }
  endShape(CLOSE);
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 1 ], 9600);

  myPort.clear();
  
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  Values = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          Values[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

 

 

1. The married life of Cinderella

• Inspiration of the setting

I was dreaming of harvesting true love like those protagonists in fairy tales when I was a little girl. However, as I grow up, I’ve gradually realized that fairy tales are not the reality. That’s why I decided to develop the stories after Cinderella’s “happy” ending and I came up with the story through my observations and imagination. I wanted to make an interesting video game with special barrier techniques to foster interaction between the story and players, I wanted to tell the story in fun and surprising way.

• Challenges I came up with

2D maze game

Temple run+Music game

jump by amplifying sound level, dodge by tread button left and right.

• elements

the glass slipper

the pumpkin carriage

 

2. Garbage slice

• Inspiration of the setting

Garbage classification is part of our life. Inspired by the well-known game: fruit slice(fruit ninja), I wanna make a garbage slice to help us distinguish the garbage type quickly and clearly.

• Challenges I came up with

2D figures (garbage) show on the screen, then the player take up the right garbage bin and move it to catch the falling garbage.

There are three outcome:

1. right bin and nice catch
2. right bin but miss the garbage
3. distinguish the wrong category, then fail.

• elements

different types of garbage

text on them 

3. Snake

There are many codes like this online, we could revise them a little bit and make the classical video game: Snake. Given the Processing part, we just need to figure out what kind of sensor would be suitable for the Arduino part.

However, to be honest, I want to make the code myself instead of learning others from websites. Though it would be more time-consuming and unpredictable, only in this way could I receive a great sense of achievement.

 

 

 

Recitation Exercise

Task: use the coding concepts covered in this week’s class to create a sketch with many similar items at the same time. 

Step 1: 

displays some graphic of your own design.

I used 4 shapes to create a cute duck. Since the angle of the arc is unusual, I calculated it out. And the color of the duck’s face and peak should be fixed, otherwise, we could not figure out it is a duck.

Step 2:

Create a for loop in the setup() to display 100 instances of your graphic in a random variety of positions and colors. 

At first, I put the for loop in void draw(), but it was blinking, then I changed it to void setup(). Putting for loop in void setup() is like setting the location of those 100 ducks once, and it turns out to be a still graph. However, putting for loop in void draw() is like continuously set the location of those 100 ducks. void draw() is a loop, which means it repeats the action infinitely, so the graph is animated.

Step 3:

Create three Arrays to store the x, y, and color data. In setup(), fill the arrays with data using a for loop, then in draw() use them in another for loop to display 100 instances of your graphic (that’s two for loops total). 

At first, the color difference is made by random() in void draw(), and it turns out to be better in setup(). 

Step 4:

Add individual movement to each instance of your graphic by modifying the content of the x and y arrays.  Make sure that your graphics stay on the canvas (hint: use an if statement).

Here’s the final work:

Here’s the code:

color[] c = new color[100];
color[] d = new color[100];
float[] x = new float[100];
float[] y = new float[100];
float[] _x = new float[100];
float[] _y = new float[100];

void setup() {
  size(700, 700);
  for (int i = 0; i < 100; i++){
   c[i] = color(random(255), random(255), random(255));
   d[i] = color(random(255), random(255), random(255));
   x[i] = random(width);
   y[i] = random(height);
   _x[i] = random(-6,6);
   _y[i] = random(-1,1);
  }
  
}
void draw() {
  background(#755446);
  for (int i = 0; i < 100; i = i + 1){
   duck(200, 100, x[i], y[i], c[i], d[i]); 
   if (x[i] + _x[i] > width || x[i] + _x[i]< 0){
    _x[i] = -_x[i];
   }
   if (y[i] + _y[i] > height || y[i] + _y[i]< 0){
    _y[i] = -_y[i];
   }
   x[i] = x[i] + _x[i];
   y[i] = y[i] + _y[i]; 
  }
}

void duck(int x, int y,float a, float b, color c, color d) {
  //Use these parameters to create your graphics
  fill(#E26A45);
  arc(a+25, b+13, 50, 20, 1.5 *PI, 2.5*PI, OPEN);
  fill(#F7D484);
  stroke(c);
  ellipse(a, b, 60,80);
  triangle(a-5, b-40, a, b-55, a+5, b-40);
  fill(d);
  circle(a+10, b-5, 10);
  
}

I camp up with the idea to set new value x, y as _x, _y to let the ducks move, and the If statement set boundaries to prevent them from moving out of the canvas.

 

Additional Homework

This week assignment is so difficult, so I went to the workshop and learned it. While I could understand these steps through listening, I could not use them if I did it myself. Just as the instructor said, we just need to understand and memorize it. lol

video here:

My code here:

int row=25;
int col=25;
int totalNumber=row*col;
int padding=140;

int defaultSize=10;

float[] xPositionsOriginal = new float [totalNumber];
float[] yPositionsOriginal = new float [totalNumber];
float[] scaleFactors = new float [totalNumber];
float[] xPositions = new float [totalNumber];
float[] yPositions = new float [totalNumber];

float lerpFactors=0.5;

void setup(){
  size(800,800);
  for(int i=0;i<col;i++){
    for(int j=0;j<row;j++){
      int index=j+i*row;
      xPositionsOriginal[index]=map(i,0,col-1,padding,width-padding);
      yPositionsOriginal[index]=map(j,0,col-1,padding,height-padding);
      
      xPositions[index]=xPositionsOriginal[index];
      yPositions[index]=yPositionsOriginal[index];
      scaleFactors[index]=1.0;
    }
  }
}

void draw(){
   background(255);
   for(int i=0;i<totalNumber; i++){
      float circleRadius = dist(mouseX, mouseY, xPositionsOriginal[i], yPositionsOriginal[i]);
      circleRadius=max(circleRadius,0.001);
      
      float newX=xPositionsOriginal[i]-(25/circleRadius)*(mouseX-xPositionsOriginal[i]);
      float newY=yPositionsOriginal[i]-(25/circleRadius)*(mouseY-yPositionsOriginal[i]);
      
      
      xPositions[i]=lerp(xPositions[i],newX,lerpFactors);
      yPositions[i]=lerp(yPositions[i],newY,lerpFactors);
      
      scaleFactors[i]=100/circleRadius;
      scaleFactors[i]=max(scaleFactors[i], 0.4);
      scaleFactors[i]=min(scaleFactors[i], 2);
      //drawCircles(xPositions[i],yPositions[i],sizeFactors[i]);
      fill(0);
      circle(xPositions[i],yPositions[i],scaleFactors[i]*defaultSize);
      }
    }

Documentation

Question 1: In your own words, please explain the difference between having your for loop from Step 2 in setup() as opposed to in draw(). 
At first, I put the for loop in void draw(), but it was blinking, then I changed it to void setup(). Putting for loop in void setup() is like setting the location of those 100 ducks once, and it turns out to be a still graph. However, putting for loop in void draw() is like continuously set the location of those 100 ducks. void draw() is a loop, which means it repeats the action infinitely, so the graph is animated. 

Question 2: What is the benefit of using arrays? How might you use arrays in a potential project?     
The array plays the role to offer many storage space inside. In void setup(), each storage space was packed with one specific data, and then void draw() repetitively showed us the stored data. Since the data in the arrays are memorized, and it can’t be stored without arrays, the graph of step 3 is still but the last one is animated. In my future project, I could utilize the arrays to replace the number when I need to change, which save time and tidy our codes. 

PREPARATORY RESEARCH AND ANALYSIS

Read “Art, Interaction and Engagement” by Ernest Edmonds.

Inspiration of STYLE  

Processing could create beautiful image and interesting animation, while Arduino plays a critical role to help us connect our physical interaction with digital world. Then I come up with the idea of making a interactive game myself.

Here are some examples I found.

2D interactive game:

This is a 2D Interactive Multiplayer Maze game developed using Processing and Arduino. It inspires me to combine different functions I learned together to create a comprehensive project. Also, it is a video game, everyone could play it. So, the interaction here is better compared with the limited the mid-term project safety pants we created. Also, it is more practical than the space shuttle we created in the group project.

hand-gesture Flappy Bird:

This Arduino Game project is based on our hand gesture. Since we all have already played this flappy bird Game in different modes from our mobile phone, it is an opportunity for us to make Arduino hand gesture Game project using Arduino UNO. This novel experience would be extremely engaging and a good way for us to kill time. This project inspired me with more interactive ways to design my own work. Thinking old thing from a new light is surprising.

 

Other Inspirations:

The game controller:

 

Some visual elements I could learn:

https://www.manamana.net/video/detail?id=180231#!en

https://www.manamana.net/video/detail?id=54878#!en

https://www.manamana.net/video/detail?id=1767#!zh

There are countless interactions everywhere in our daily life. Drinking water is the interaction between you and the cup; turning on and off the lamp is the interaction between you and the buttons of the lamp. When using our mobile phone, we are doing more types of interaction. Although we just operate on the screen, there are indeed many “invisible” interactions inside. Interaction is a kind of dialogue, whether it is between people or between people and things. Ernest Edmonds said “the computer software made certain decisions that determined what graphical output would result.”Inspired by his words, I would define an interaction a kind of conversation that evoke thoughts and understanding. The several interactive art projects support my opinion. With the help of processing and Arduino, I could tell a story interesting, interactive, and inspiring.

 

Reference

https://www.manamana.net/explorevideo?type=1&id&videoOrderBy=3&searchTagForms=%5B%7B%22topGroupId%22%3A1,%22groupId%22%3A16,%22tagId%22%3A2%7D%5D#!zh

 

Welcome to Web Publishing @ NYU. This is your first post. Edit or delete it, then start creating your site!

Online help is available via the Web Publishing Knowledge Site (wp.nyu.edu/knowledge) and the ServiceLink knowledge base (www.nyu.edu/servicelink). Through ServiceLink, you can find step-by-step instructions, as well as tutorials.

Digital Accessibility

As content creators who create and publish text, images, video, and audio, you must adhere to the NYU Website Accessibility Policy (https://www.nyu.edu/digitalaccessibility/policy) when creating and publishing digital content.

Web Publishing-specific Digital Accessibility Best Practices and examples of how to ensure your content are compliant are available at https://wp.nyu.edu/digitalaccessibility

If you have additional questions, contact the IT Service Desk for assistance. Support is available 24/7/365. For more details, visit www.nyu.edu/it/servicedesk.