welcome to Duorfan' Blog

Final Blog Post: Fruit Ninja

by Leave a Comment

 

PROJECT TITLE – Fruit Ninja

MY NAME – Duorfan (with Kiki)

INSTRUCTOR’S NAME – Rodolfo Cossovich

 

Origin & Name

Garbage classification is part of our life. Inspired by the well-known game: fruit slice(fruit ninja), I wanna make a garbage slice to help us distinguish the garbage type quickly and clearly. We let 2D figures (garbage) show on the screen, then the player should distinguish the right category and pick up the right garbage bin, and then move it to catch the falling garbage. That’s why we give our project a interesting name: Garbage Ninja.

Final Idea & Concept 

Due to our immature our mid-term project, I realized that interaction was not only numerous pairs of input and output, continuous response and reaction are also playing an critical role. Besides the simple reactions, the loop of response is the essence of the concept of interaction. Great interaction project should not be a thing players can’t touch or experience, it should be intriguing instead of its deep concept which participants could not interpret by themselves. Processing could create beautiful image and interesting animation, while Arduino plays a critical role to help us connect our physical interaction with digital world. Also, the results of the loop should be various determined by different users. Because of these insights and my research, I decided to make an interactive game with educational meanings.

When my city, Wuxi, first adopted the Garbage Classification Policy, many citizens were not used to it. But after I learned the importance of garbage classification and the terrible consequences like the “garbage beaches”, where malnourished polar bears and innocent sea turtles died from consuming hundreds of plastic bags in their stomach, I wanted to make a change. Seeing how garbage classification is a necessity, I want to create a interactive game, like a the game fruit ninja, to help people practice garbage categories and raise awareness of environmental and animal protection.

Prototype

Based on the four types classified in Shanghai, we drew 12 garbage, 3 for each category. Our plan for this game changed from a Fruit Ninja game of cutting items to a random ball picking game, which match our action of throwing garbage much better. To illustrate our concept, I drew the animation by using Procreate and we split the game by first completing the Processing part. Instead of the using the sensors and Arduino, we choose to use Mouse-click() or Mouse-moved().

We originally planned to give these three outcome:

  1. right bin and nice catch
  2. right bin but miss the garbage
  3. distinguish the wrong category, then fail.

However, we faced a lot of problems of coding in Processing since that was our first time programming for our own ideas. There are no similar example coding online, so we find the problems one by one through Google and the Github (this website is amazing). When we were spending our weekends working hard on those code, we also received help from our classmates (Siyuan and Kira, especially Siyuan). They helped us with the category part because we couldn’t find the proper function(I even wanted to use Matrix to solve it). Thanks to Processing’s comprehensive functions, we figure out how to pick and categorize items properly. The problems of defining win() to make the loop run was also solved when we came home and had a wonderful dinner, so our Processing part finally worked with mouse clicking and moving.

Code (Mouse version)

import processing.sound.*;
SoundFile sound;

import processing.video.*;
Movie movie;


PImage bottle;
PImage shoes;
PImage glass; 
PImage light;
PImage battery;
PImage medicine;

PImage banana;
PImage apple;
PImage cheese;

PImage mask;
PImage tissue;
PImage cige;

PImage bg;
PImage bg2;

float xPos = random( 0, 1000.0);
float yPos = 0.0;
float xVel=random(5, 6.0);
float yVel=random(4, 5.0);

float Xblue = 120.0;
float Yblue = 750.0;
float Xred = 340.0;
float Yred = 750.0;
float Xbrown = 560.0;
float Ybrown = 750.0;
float Xblack = 780.0;
float Yblack = 750.0;
float Xtemp;       
float Ytemp;       

float xWin;       
float yWin; 

int r;
int w = 1; 

PImage Go[];

boolean state = true;
boolean okState = false; 

//long lastTime = 0;

int level = 0;
int alpha = 255;

void setup() {
  //lastTime = millis();

  size(1000, 900);
  bottle =loadImage("bottle.png");
  bottle.resize(100, 100);
  shoes =loadImage("shoes.png");
  shoes.resize(100, 100);
  glass =loadImage("glass.png");
  glass.resize(100, 100);

  light =loadImage("light.png");
  light.resize(100, 100);
  battery =loadImage("battery.png");
  battery.resize(100, 100);
  medicine =loadImage("medicine.png");
  medicine.resize(100, 100);

  banana =loadImage("banana.png");
  banana.resize(100, 100);
  apple =loadImage("apple.png");
  apple.resize(100, 100);  
  cheese =loadImage("cheese.png");
  cheese.resize(100, 100);  


  mask =loadImage("mask.png");
  mask.resize(100, 100);
  tissue =loadImage("tissue.png");
  tissue.resize(100, 100);
  cige =loadImage("cige.png");
  cige.resize(100, 100);
  
  
  bg = loadImage("bg.jpg");
  
  bg.resize(1000, 900);

  bg2 = loadImage("bg2.jpg");
  bg2.resize(1000, 900);

  Go = new PImage[12];

  Go[0]=bottle;
  Go[1]=shoes;
  Go[2]=glass;

  Go[3]=light;
  Go[4]=battery;
  Go[5]=medicine;

  Go[6]=banana;
  Go[7]=apple;
  Go[8]=cheese;

  Go[9]=mask;
  Go[10]=tissue;
  Go[11]=cige;
  sound = new SoundFile(this, "win.aiff");
  
  movie = new Movie(this, "1080p.mp4");  
  movie.play(); 
}
void movieEvent(Movie m) {
  m.read();
}
  

void draw() {
  image(movie, 0, 0, width, height); 
    
  image(bg2,0,0); 
  tint(255, alpha);
  image(bg,0,0);
  println(level);
  tint(255,255);
  fang();        
  play1();
  textSize(50);  
  fill(#9D5630); 
  text("Point:"+level, 800, 50);
  
    
}



void fang() {     
  noStroke();
  fill(#0049D3);
  rect(Xblue, Yblue, 100, 200, 5);
  fill(#CC0303);
  rect(Xred, Yred, 100, 200, 5);
  fill(#9D5630);
  rect(Xbrown, Ybrown, 100, 200, 5);
  fill(#000000);
  rect(Xblack, Yblack, 100, 200, 5);
}

void ballmove() {                            
  image(Go[r], xPos, yPos);
  if (xPos + xVel > 950 || xPos + xVel< -50) {
    xVel = -xVel;
  }  
  if (yPos + yVel > 610 || yPos + yVel < -50) {
    yVel = -yVel;
  }
  xPos = xPos+xVel;
  yPos = yPos+yVel;
}

void play1() {
  if (state == true) {
    state = false;
    r = int(random(0, 12));
  }
  ballmove();            
  win();
  
  
}

void win() {
  if (r >=0 && r <=2) {
    Xtemp = Xblue;
    Ytemp = Yblue;
  }
  if (r >=3 && r <=5) {
    Xtemp = Xred;
    Ytemp = Yred;
  } 
  if (r >=6 && r <=8) {
    Xtemp = Xbrown;
    Ytemp = Ybrown;
  } 
  if (r >=9 && r <=11) {
    Xtemp = Xblack;
    Ytemp = Yblack;
  }

  if (Xtemp <= xPos + 50 && Xtemp >= xPos - 50 && yPos >= Ytemp -50 &&  yPos<= Ytemp + 50) {
    Xblue = 120;       
    Yblue = 750;
    Xred = 340.0;        
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;

    yPos = 1000;    
    level = level + 1;
    alpha = 255-level*20;
    //win
    yPos = 0.0;     
    state = true;
    sound.play(); 
  }
  if (w<60 && okState == true) {   
    fill(255, 255, 0);
    textSize(80);
    text("ok", xWin, yWin); 
  
    w=w+1;
  } else {
    okState = false;     
    w = 1;    
    
  }
  
  
   
  
}

void mouseClicked() {
  if (mouseX>120 && mouseX<220 && mouseY> 750&& mouseY<900) {
    Xblue = 120;
    Yblue = 600;         
    Xred = 340.0;        
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (mouseX>340 && mouseX<440 && mouseY> 750&& mouseY<900) {
    Xred = 340;
    Yred = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (mouseX>560 && mouseX<660 && mouseY> 750&& mouseY<900) {
    Xbrown = 560;
    Ybrown = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xred = 340.0;
    Yred = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (mouseX>780 && mouseX<880 && mouseY> 750&& mouseY<900) {
    Xblack = 780;
    Yblack = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xred = 340.0;
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
  }
  
}

void mouseMoved( ) {
  if (Yblue == 600) {
    Xblue = mouseX - 50;
  }
  if (Yred == 600) {
    Xred = mouseX - 50;
  }
  if (Ybrown == 600) {
    Xbrown = mouseX - 50;
  }
  if (Yblack == 600) {
    Xblack = mouseX - 50;
  }
}
 

By the way, we found it hard to add failed section, and we thought infinite mode could be more pleasant. So, we delete the third outcome. The wrong could only miss the rubbish, but it couldn’t stop the game. If the rubbish bin misses the rubbish, the game won’t stop as well, the rubbish would bounce back again.

Sensors  

After finishing the Processing part, what we need to do is to add the sensors to connect our physical interaction with digital world. But before the programing of Arduino, we drew different methods of interaction and asked the advice from our professor Rudi first.

We knew that simply picking the right bin is not intriguing while moving the bin left and right to catch the rubbish is a better idea. In addition, we got inspirations from Jintian and Andy: adding sound effects and attractive background would be more interactive. When thinking of implementing bingo background when right bin catches the garbage, I came up with the idea of transparency. Searching the function on Processing.com, I found that the value of alpha could be changed to control the transparency. Finally, we manipulated the transition to a suitable range, which players could discover it after several bingo sound. We were glad that many classmates was shocked on the background picture changed from gray city to green forest, which gave them a big surprise.

 

When deciding whether the light sensor or the pressure sensor, we found that photo resistor was more accurate. And we try two types of distance sensor, only the ultrasonic one could work properly.

We used laser cutter to make the four garbage bin, but we use 2.5mm instead of 3mm. When using glue to stick the boards together, we experience hotness and sturdiness of 502 glue.TT

Adaptations

First of all, because we need to limit communication with testers as much as possible and allow participants to try to find answers, some of them cannot reach the location of the trash can after picking up the trash can. In order to improve this problem, we can add a short animation at the beginning of the program, which includes the video part we just learned in class. In this way, our game instructions can be more clear and organized.

Secondly, our photo resistor is not sensitive enough, sometimes the cube on the screen will not change when different boxes are not picked up. Since the brightness is different, we absolutely need to re-measure and find the most suitable value, otherwise we can add another light to control the value detected by the sensor.

In addition, we original use separate paper boards for the installation, but some students found it hard to move the box on the paper board. So, we used laser cutter to make a complete installation with smooth board and proper holes for the ultrasonic distance sensor.

Last but not least, we reversed the positions of the trash can and the movable plate to avoid hand shadows, which will affect the accuracy of the light sensor. When the track approaches the trash can, it may also be more convenient for the players to move the bins. 

 

Finished Product

When the player picks up the correct trash can, the virtual trash can on the screen will appear in the middle of the screen. At the same time, if you move the trash can on the track, the virtual trash can on the screen will follow your movement so that we can quickly collect the correct trash ourselves.

Code (final version)

Processing:

import processing.sound.*;
SoundFile sound;
import processing.video.*;
Movie movie;

import processing.serial.*;
String myString = null;
Serial myPort;  // The serial port

PImage bottle;
PImage shoes;
PImage glass;

PImage light;
PImage battery;
PImage medicine;

PImage banana;
PImage apple;
PImage cheese;

PImage mask;
PImage tissue;
PImage cige;

PImage bg;
PImage bg2;

float xPos = random( 0, 1000.0);
float yPos = 0.0;
float xVel=random(5, 6.0);
float yVel=random(4, 5.0);

float Xblue = 120.0;
float Yblue = 750.0;
float Xred = 340.0;
float Yred = 750.0;
float Xbrown = 560.0;
float Ybrown = 750.0;
float Xblack = 780.0;
float Yblack = 750.0;
float Xtemp;       
float Ytemp;       

float xWin;       
float yWin; 

int r;
int w = 1; 

PImage Go[];

int NUM_OF_VALUES = 5;
int[] sensorValues; 

int a = 0;
int b = 0;
int c = 0;
int d = 0;
int e = 0;

boolean state = true;
boolean okState = false; 

int level = 0;
int alpha = 255;

void setup() {
  size(1000, 900);  
  setupSerial();
  bottle =loadImage("bottle.png");
  bottle.resize(100, 100);
  shoes =loadImage("shoes.png");
  shoes.resize(100, 100);
  glass =loadImage("glass.png");
  glass.resize(100, 100);

  light =loadImage("light.png");
  light.resize(100, 100);
  battery =loadImage("battery.png");
  battery.resize(100, 100);
  medicine =loadImage("medicine.png");
  medicine.resize(100, 100);

  banana =loadImage("banana.png");
  banana.resize(100, 100);
  apple =loadImage("apple.png");
  apple.resize(100, 100);  
  cheese =loadImage("cheese.png");
  cheese.resize(100, 100);  

  mask =loadImage("mask.png");
  mask.resize(100, 100);
  tissue =loadImage("tissue.png");
  tissue.resize(100, 100);
  cige =loadImage("cige.png");
  cige.resize(100, 100);
  
  bg = loadImage("bg.jpg");  
  bg.resize(1000, 900);
  bg2 = loadImage("bg2.jpg");
  bg2.resize(1000, 900);

  Go = new PImage[12];

  Go[0]=bottle;
  Go[1]=shoes;
  Go[2]=glass;

  Go[3]=light;
  Go[4]=battery;
  Go[5]=medicine;

  Go[6]=banana;
  Go[7]=apple;
  Go[8]=cheese;

  Go[9]=mask;
  Go[10]=tissue;
  Go[11]=cige;
  sound = new SoundFile(this, "win.aiff");
  movie = new Movie(this, "1080p.mp4");  
  movie.play();  
}

void draw() {
  image(movie, 0, 0, 1000, 800); 
  float md = movie.duration(); 
  float mt = movie.time();
  if (mt == md) {
  image(bg2,0,0);
  tint(255, alpha);
  image(bg,0,0);  
  println(level);
  tint(255,255);
  fang();

  binmove();
  play1();
  textSize(50);  
  fill(#9D5630); 
  text("Point:"+level, 800, 50);
  updateSerial();
  printArray(sensorValues);
  a = sensorValues[0];
  b = sensorValues[1];
  c = sensorValues[2];
  d = sensorValues[3];
  e = sensorValues[4];
 }
}


void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 1 ], 9600);
  myPort.clear(); // Throw out the first reading,
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
  sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length ==NUM_OF_VALUES){
        for (int i=0; i<serialInArray.length;i++){
          sensorValues[i]=int(serialInArray[i]);
        }
      }
    }
  }
}

void fang() {     
  noStroke();
  fill(#0049D3);
  rect(Xblue, Yblue, 100, 200, 5);
  fill(#CC0303);
  rect(Xred, Yred, 100, 200, 5);
  fill(#9D5630);
  rect(Xbrown, Ybrown, 100, 200, 5);
  fill(#000000);
  rect(Xblack, Yblack, 100, 200, 5);
}

void ballmove() {                            
  image(Go[r], xPos, yPos);
  if (xPos + xVel > 950 || xPos + xVel< -50) {
    xVel = -xVel;
  }  
  if (yPos + yVel > 610 || yPos + yVel < -50) {
    yVel = -yVel;
  }
  xPos = xPos+xVel;
  yPos = yPos+yVel;
}

void play1() {
  if (state == true) {
    state = false;
    r = int(random(0, 12));
  }
  ballmove();
  bin();
  binmove( );
  win();               
}

void win() {
  if (r >=0 && r <=2) {
    Xtemp = Xblue;
    Ytemp = Yblue;
  }
  if (r >=3 && r <=5) {
    Xtemp = Xred;
    Ytemp = Yred;
  } 
  if (r >=6 && r <=8) {
    Xtemp = Xbrown;
    Ytemp = Ybrown;
  } 
  if (r >=9 && r <=11) {
    Xtemp = Xblack;
    Ytemp = Yblack;
  }

  if (Xtemp <= xPos + 50 && Xtemp >= xPos - 50 && yPos >= Ytemp -50 &&  yPos<= Ytemp + 50) {
    Xblue = 120;       
    Yblue = 750;
    Xred = 340.0;        
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;

    yPos = 1000;    
    level = level + 1;
    alpha = 255-level*20;
    //win
    yPos = 0.0;     
    state = true;
    sound.play(); 
  }
  if (w<60 && okState == true) {   
    fill(255, 255, 0);
    textSize(80);
    text("ok", xWin, yWin);     
    w=w+1;
  } else {
    okState = false;     
    w = 1;               
  }
}

void bin() {
  if (a < 600 && e > 1000) {
    Xblue = 120;
    Yblue = 600;         
    Xred = 340.0;        
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (b < 800 && e > 1000) {
    Xred = 340;
    Yred = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (c < 750 && e > 1000) {
    Xbrown = 560;
    Ybrown = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xred = 340.0;
    Yred = 750.0;
    Xblack = 780.0;
    Yblack = 750.0;
  }
  if (d < 750 && e > 1000) {
    Xblack = 780;
    Yblack = 600;
    Xblue = 120.0;
    Yblue = 750.0;
    Xred = 340.0;
    Yred = 750.0;
    Xbrown = 560.0;
    Ybrown = 750.0;
  }
}

void binmove( ) {
  if (Yblue == 600) {
    Xblue = e - 50;
  }
  if (Yred == 600) {
    Xred = e - 50;
  }
  if (Ybrown == 600) {
    Xbrown = e - 50;
  }
  if (Yblack == 600) {
    Xblack = e - 50;
  }
}
 

Arduino:

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing
long duration; // variable for the duration of sound wave travel
int distance;

void setup() {
  pinMode(A4, OUTPUT); // Sets the trigPin as an OUTPUT
  pinMode(A5, INPUT); // Sets the echoPin as an INPUT
  Serial.begin(9600);
}

void loop() {
  int sensor1 = analogRead(A0);
  int sensor2 = analogRead(A1);
  int sensor3 = analogRead(A2);
  int sensor4 = analogRead(A3);
 // Clears the trigPin condition
  digitalWrite(A4, LOW);
  delayMicroseconds(2);
  // Sets the trigPin HIGH (ACTIVE) for 10 microseconds
  digitalWrite(A4, HIGH);
  delayMicroseconds(10);
  digitalWrite(A4, LOW);
  // Reads the  echoPin, returns the sound wave travel time in microseconds
  duration = pulseIn(A5, HIGH);
  // Calculating the distance
  distance = duration * 0.034 / 2; // Speed of sound wave divided by 2 (go and back)

  distance = map(distance, 0, 55, 100, 1000);

  // keep this format
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.print(",");
  Serial.print(sensor3);
  Serial.print(",");
  Serial.print(sensor4);
  Serial.print (",");
  Serial.print (distance);
  Serial.println (); 

  delay(100);
}

Conclusion & Improvements 

On the premise of mutual exchange, our goal is to make our works have a practical and deep significance on the basis of attraction. Garbage classification can promote environmental protection, while many people do not learn to separate garbage for convenience. I think our project did made some progress, but there remains a long way for us to improve, like the problem of light sensor. The goal of our project is to increase the awareness of citizens on garbage classification , and let them quickly understand the category of each type of garbage, so that daily garbage classification can be carried out more smoothly. But the sensor we chose is not sensitive enough to eject garbage bin and let the bin move smoothly on the screen. During the presentation, we received some recommenations of coding stategies to improve the fluency of the bin and the sensibility of the photo resistors. Though our coding is pretty rough now, through these improvisions, it would be more comprehensive and well-organized.

 

REALLY Thanks for this class, and every IMAer in our section💜.

Reader Interactions

Leave a Reply

Your email address will not be published. Required fields are marked *