Final Blog Post

Survive the Plastic – Tracy and Elaine – Andy Garcia

 CONCEPTION AND DESIGN:

Our project aims to raise people’s awareness of protecting the Marine environment by reducing the immoderate use and production of plastic and trash. We searched online for the constitution of marine pollution and found that plastic pollution was a big part of it. So we also looked up online to search for relative information. We found that: Plastic pollution includes micro-plastic pollution; Micro-plastic can be consumed by fishes and they are deadly…

And with those information, we thought that it would be great if we can make it into a survival game to illustrate the idea that fish would die if touching the micro-plastic. We imagine that the interaction would be, players using some sensors to control the movement of fishes, for example potentiometers or joysticks. And for the computer’s respond, for example, if the fish touches a micro-plastic, the computer will respond by giving a sound and a visual effect that the fish will disappear as if it’s dead. However, initially, we wanted that if the player want to control the fish to eat shrimp, they should make sound to trigger the movement of eating, but after several tries by ourselves and our friends we found that it’s both quite difficult to use the potentiometer and the joystick to control the fish, and the players already are trying hard to control the fish’s position already, if they still need to make sound to trigger the movement of eating shrimp, it can be too hard and may reduce the fun of playing the game, so we decided not to add the sound triggering eating part. And by testing with Andy, he suggested that it would be better if the fish don’t disappear as soon as it touches the plastic, in order to give the players more chances and to have fun, we should give life counts to the fishes. So after receiving these suggestions, we removed the sound triggering part and added the life counts. They did have a better feedback from players we invited to play.

 FABRICATION AND PRODUCTION:

Some significant parts in our process is the following: The design of the elements in the game; Responds the computer give according to what the players(fishes) touch; The arrangement of the stages of the game. And I will introduce the difficulties we encountered in each part.

Firstly, the design of the elements. We have three elements in our game, the fishes(the players), the shrimps(fish can grow bigger by eating them), and the micro-plastic(fish will die if touching it). We designed these graphs by ourselves, we painted the shrimp and the fish using ps and gravit.

We coded the micro-plastic at first, however, after some office hours with LAs and Andy, they suggested that the coded plastic is a bit confusing, so we changed them into cartoon images of plastic bags and plastic bottles.

At first, we wanted there to be a settled number of plastic and shrimps, and we added a pollution layer like this 👇

that acts as count down by keep rising from the bottom of the screen and making the movement area smaller and smaller(the game can only be played in the blue area). However, after talking with Andy, we found that because of the density of micro-plastic varies, so they actually won’t all accumulate at the bottom of the sea. So we wanted to change the count down into the growing number of micro-plastic and the reduction of shrimp(Also to illustrate the idea that shrimps will die of micro-plastic as well and micro-plastic is increasing nowadays). However, we couldn’t successfully make the plastic increase according to time, which is a little pity in this project. But during this process we also learnt more about the micro-plastic and the fact that it’s spreading all over the sea. 

Secondly, the computer’s respond.

As was mentioned above, we added life counts for the fish. And other visual effects are 1, shrimps will disappear after the fish touches them 2, fish will disappear after touching the micro-plastic for three times. The first part was easy to code, but the second part was really challenging.

if (d4[i] < (5+size[i]/2) && touch1==false &&counter1 !=0) {//i
      touch1=true;
      counter1=counter1-1;
      lastCircle1 =i;
      sound1.play();
    }
    
    if (d2[lastCircle1]>(5+size[lastCircle1]/2)) {
      touch1 =false;
    }
    if (counter1 == 0 && fishalive1 == true) {
      x2=-200;
      y2=-200;
       heartx5=-100;
       hearty5=-100;
       photo4 = loadImage(“plose.jpeg”);
      sound2.play();
      fishalive1=false;
      break;
    }
    if (d4[lastCircle2]>(5+size[lastCircle2]/2)) {
      touch2 =false;
    }
    if (counter2 == 0 && fishalive2 == true) {
      x3=-200;
      y3=-200;
       heartx5=-100;
       hearty5=-100;
      sound2.play();
      fishalive2=false;
      break;
    }

The most difficult part was that the fish/the heart(stands for life counts) needs to stay disappear even if it no longer touches the micro-plastic, however the processing runs as a loop, so in order to do this, Winny and I tried Boolean. Also, the line “lastCircle1=I” and “d4[lastCircle2]” took us a long time to figure out. They both together enable the effect of the fish/the heart(stands for life counts) needs to stay disappear even if it no longer touches the micro-plastic,

Last but not least, the arrangement of the stages of the game. We had three stages: the first is the introduction. Following by the game, and then ending with a picture illustrating that as we eat fish, we eat plastic. At first we distinguished the three stages by the count of time, however, it didn’t work. Andy suggested the switch case to us, which, is life changing. This is very convenient and effective and make our code looks so logical as well.

Tracy and I did make a separate work sheet before we start, but basically we did everything together and helped each other the whole process.

CONCLUSIONS:

Our project wants to use the setting of the game, which is that the fish will die of micro-plastic, and eventually both fishes will die because there is too much of micro-plastic, to illustrate the idea that the micro-plastic we humans produced has caused great harm to the marine life and marine environment. Also, the final page of our project, the picture that shows a fish full of plastic placing on a plate, stating that those micro-plastic consumed by the fish will eventually be consumed by us, also shows that the harm we caused to the marine will eventually pay back on ourselves. We hope our project can arouse people’s awareness of reducing the production of trash and plastic, to protect the environment and to protect ourselves. I think our project did convey this idea, whether by the texts at the beginning and the picture at the end, or the game itself.

I think our project is interactive for it creates a communication platform: players controlling the fish, and computers giving responds accordingly. Friends we invited to play with our game did found that it’s a game that actually needs some strategy to play, for example, although eating shrimps can make the fish bigger, it makes it harder to avoid the plastic. Some friends would adopt the strategy of not eating any shrimps so that it can avoid the plastic more easily. However, if we have more time, we do want to make more improvement, for example, if the player doesn’t consume shrimp in a certain period of time, the fish will lose one life count, which is more related to the reality that fishes need food, and also makes the game a bit harder, that the players can’t avoid dying by not eating shrimps. Also, we didn’t manage to make the effect of micro-plastic increasing according to time, if we have enough time, maybe we can succeeded in making that effect come true and make the lose-lose situation more obvious. Also, according to some of our friends’ question like whether the game can be played by one person only, I think if we have more time, we can make the game more “professional” with a starting page, providing options of “single player” and “two players”, and by clicking on the specific option the player wants, they will play the game by themselves or with a partner.

I think the focus of our project is important. For it’s responding one of the most intense environmental issue nowadays. And people should care because the plastic pollution not only pollutes the marine environment as we shown in the game, but also the air, as we shown in the video, what’s more, it is affecting ourselves as well, by the air we breath and the fish we eat. It’s important to show this idea to everyone who gets to play our game because plastic pollution is accumulated by everyone’s daily life, one more person realizing this issue can contribute to some reduction of plastic pollution. Everyone counts and every realization matters. By this game, we hope to have more people realizing this issue and start with their everyday life, use plastic products properly and cut the plastic pollution gradually.

TECHNICAL DOCUMENTATION:

Code:

Arduino:

void setup(){
  Serial.begin(9600);
}
void loop(){
  int sensor1 = analogRead(A0);
  int sensor2 = analogRead(A1);
  int sensor3 = analogRead(A2);
  int sensor4 = analogRead(A3);
  int sensor5 =  digitalRead(9);
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.print(","); // add linefeed after sending the last sensor value
  Serial.print(sensor3);
  Serial.print(",");
 Serial.print(sensor4);
  Serial.println();
  delay(100);
}

Processing :
The beginning 1 page with rising texts:

import processing.sound.*;
int amount=20;
int count=20;

float startY;
float x[] = new float[count];
float yy;
float size[]= new float [count];
PImage photo1;
PImage photo2;
PImage photo3;

float[] Bx = new float[amount];
float[] By = new float[amount];
float[] Bsize = new float[amount];
float[] Bspeed = new float[amount];
SinOsc[] sine = new SinOsc[amount];
Env [] envelope = new Env [amount];
float [] freq = new float [amount];
int num=0;

long time;

void setup(){
  //size(800,800);
   
  fullScreen();
  background(0);
  photo1 = loadImage("shrimp.png");
  photo2 = loadImage("fish1.jpg");
  photo3 = loadImage("fish2.jpg");
   for (int i=0; i<amount; i++) {
    Bx[i] = random(width);
    By[i] = random(height, height * 2);
    Bsize[i] = random(10, 50);
    Bspeed[i] = map(size[i], 10, 50, 1, 5);
    sine[i] = new SinOsc (this);
    envelope[i] = new Env (this);
    freq[i] = map(Bsize[i], 10, 50, 500, 100);
  }
}


void draw(){
  
   switch(num) {
   case 0: 
   background(#3FA8C4);
    
    background(#3FA8C4);
    for (int i=0; i<amount; i++) {
      noStroke();
      fill(255, 120);
      circle(Bx[i], By[i], Bsize[i]);
      // Bubbles float to the top
      By[i] -= Bspeed[i];
      // if bubbles reaches the surface, create a new random bubble
      if (By[i] < 80+(Bsize[i]/2)) {
        Bx[i] = random(width);
        By[i] = random(height, height * 2);
        Bsize[i] = random(10, 50);
        Bspeed[i] = map(Bsize[i], 10, 20, 1, 2);
        sine[i].freq(freq[i]);
        envelope[i].play(sine[i], 0.0005, 0.05, 0.2, 1);
        freq[i] = map(Bsize[i], 10, 50, 500, 100);
        sine[i].play();
      }
  }
   startY=height;
 //startY=700;
 if (frameCount<10.0) {
    yy = startY - (frameCount * 2);
  }  
    startY=height-100;
   if ( frameCount  < 20.0) {
   
    yy = startY - ((frameCount-1) * 2);
  }  
   startY=height-200;
   if ( frameCount  < 30.0) {
   
    yy = startY - ((frameCount-2) * 2);
  }  
  startY=height-300;
   if ( frameCount  < 40.0) {
   
    yy = startY - ((frameCount-3) * 2);
  }  
   startY=height-400;
   if (frameCount  < 50.0) {
   
    yy = startY - ((frameCount-4) * 2);
  }  
   startY=height-500;
    if (frameCount  < 60.0) {
   
    yy = startY - ((frameCount-5) * 2);
  }  
  
 
  fill(255);
 textSize(30);
  text("Water popullution is a severe issue nowadays", 1/6*width, yy);
  
  textSize(30);
  text("Marine pollution is a combination of chemicals and trash", 1/7*width, yy+100);
  

  textSize(30);
  text("one of the 6 types of marine pollution is plastic pollution", 1/7*width, yy+200);
  
  textSize(30);
  text("which includes micro-plastic", 1/5*width,yy+300);
  break;
  
   }
}

 Following stages:

 

int num = 1;
import processing.serial.*;
import processing.sound.*;
SoundFile sound1;
SoundFile sound2;
SoundFile sound3;
SoundFile sound4;
int NUM_OF_VALUES_FROM_ARDUINO = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int sensorValues[];      /** this array stores values from Arduino **/
String myString = null;
Serial myPort;
int  count =10;
float x[] = new float[count];
float y[] = new float[count];
float u[] = new float[count];
float v[] = new float[count];
float  xspeed[] = new float[count];
float  yspeed[] = new float[count];
float  uspeed[] = new float[count];
float  vspeed[] = new float[count];
float size[] = new float[count];
float r[] = new float[count];
float g[] = new float[count];
float b[] = new float[count];
float d1;
float d2[]= new float[count];
float d3;
float d4[]=new float[count];
float pred2;
float pred4;
//float m=mouseX;
//float n=mouseY;
float x2, y2;
float x3, y3;
float heartx1=680;
float hearty1=25;
float heartx2=700;
float hearty2=25;
float heartx3=720;
float hearty3=25;
float heartx4=740;
float hearty4=25;
float heartx5=760;
float hearty5=25;
float heartX1=20;
float heartY1=25;
float heartX2=40;
float heartY2=25;
float heartX3=60;
float heartY3=25;
float heartX4=80;
float heartY4=25;
float heartX5=100;
float heartY5=25;
boolean touch1=true;
boolean fishalive1=true;
boolean touch2=true;
boolean fishalive2=true;
int counter1=5;
int counter2=5;
int lastCircle1;
int lastCircle2;
float fishSize=20;
color faceColor[] = new color[count];
PImage photo1;
PImage photo2;
PImage photo3;
PImage photo4;
void setup() {
  size(800, 600);
  //fullScreen();
  background(0);
  sound1 = new SoundFile(this, "broken-string-bounce.wav");
  sound2 = new SoundFile(this, "wah.mp3");
  sound3 = new SoundFile(this, "bite.wav");
  sound4 = new SoundFile(this, "lose.mp3");
  imageMode(CENTER);
  photo1 = loadImage("shrimp.png");
  photo2 = loadImage("fish1.jpg");
  photo3 = loadImage("fish4.png”");
  photo4 = loadImage("plose.jpeg");
  setupSerial();
  printArray(Serial.list());
  for ( int i=0; i < x.length; i++) {
    x[i] = random (100, width-100);
    y[i] = random (100, height-100);
    size[i]=random (50, 80);
    xspeed[i] = random (0.05, 0.1);
    yspeed[i] = random (0.05, 0.1);
  }
  for ( int i=0; i < u.length; i++) {
    u[i] = random (100, width-100);
    v[i] = random (100, height-100);
    uspeed[i] = random (0.05, 0.1);
    vspeed[i] = random (0.05, 0.1);
    r[i] = random(0, 255);
    g[i] = random(255);
    b[i] = random(255);
    faceColor[i] = color(r[i], g[i], b[i]);
  }
}//random size and speed for shrimps and the plastic, random color for the plastic
void draw() {
  getSerialData();
  // printArray(sensorValues);
  background(#3FA8C4);
  for ( int i=0; i < x.length; i++) {
    d1=dist(x[i], y[i], x2, y2);
    d2[i]=dist(u[i], v[i], x2, y2);
    d3=dist(x[i], y[i], x3, y3);
    d4[i]=dist(u[i], v[i], x3, y3);
    if (d1 < (2 + size[i]/2)) {
      fishSize+=10;
      sound3.play();
      x[i]=-100;
      y[i]=-100;
    }
    if (d3 < (2 + size[i]/2)) {
      fishSize+=10;
      sound3.play();
      x[i]=-100;
      y[i]=-100;
    }//the shrimp will disappear when the fish touches it, and the fish will grow bigger
    plastic(u[i], v[i], 10, faceColor[i]);
    if (d2[i] < (5+size[i]/2) && touch1==false &&counter1 !=0) {//i
      touch1=true;
      counter1=counter1-1;
      lastCircle1 =i;
      sound1.play();
      //heartx1=-100;
      //hearty1=-100;
    }
    if (d4[i] < (5+size[i]/2) && touch2==false &&counter2 !=0) {//i
      touch2=true;
      counter2=counter2-1;
      lastCircle2 =i;
      sound1.play();
    }
    //if(counter==4){
    //  heartx1=-100;
    //  hearty1=-100;
    //}
    //else if(counter==3){
    //  heartx2=-100;
    //  hearty2=-100;
    //}
    //else if(counter==2){
    //  heartx3=-100;
    //  hearty3=-100;
    //}
    //else if(counter==1){
    //  heartx4=-100;
    //  hearty4=-100;
    //}
    //else if(counter==0){
    //  heartx5=-100;
    //  hearty5=-100;
    //   fill (#DE4848);
    //heart(heartx1, hearty1, 0.3);
    //heart(heartx2, hearty2, 0.3);
    //heart(heartx3, hearty3, 0.3);
    //heart(heartx4, hearty4, 0.3);
    //heart(heartx5, hearty5, 0.3);
    if (d2[lastCircle1]>(5+size[lastCircle1]/2)) {
      touch1 =false;
    }
    //if (counter==0) {
    //if (counter == 0 && fishalive == true){
    //     sound2.play();
    //     break;
    //    }
    // fishalive=false;
    //sound2.play();
    //touch=true;
    //   }
    //  if (fishalive=false){
    if (counter1 == 0 && fishalive1 == true) {
      x2=-200;
      y2=-200;
      heartx5=-100;
      hearty5=-100;
      sound2.play();
      fishalive1=false;
      break;
    }
    if (d4[lastCircle2]>(5+size[lastCircle2]/2)) {
      touch2 =false;
    }
    if (counter2 == 0 && fishalive2 == true) {
      x3=-200;
      y3=-200;
      heartx5=-100;
      hearty5=-100;
      sound2.play();
      fishalive2=false;
      break;
    }
    if (counter1==0 && counter2 == 0) {
      switch( num ){
      case 1:
      size(800,600);
        photo4 = loadImage("plose.jpeg");
        break;
      }
    print(counter1);
   
    println(counter2);
    move();
    bounce();
    x2=sensorValues[0];
    y2=sensorValues[1];
    //  x3=sensorValues[2];
    // y3=sensorValues[3];
    x2=map(x2, 0, 1024, 0, width);
    y2=map(y2, 0, 1024, 0, height);
    x3=map(x3, 0, 1024, 0, width);
    y3=map(y3, 0, 1024, 0, height);
    if (fishalive1==true) {
      image(photo2, x2, y2, fishSize, fishSize);
    }
    if (fishalive2==true) {
      image(photo3, x3, y3, fishSize, fishSize);
    }
    image(photo1, x[i], y[i], 20, 20);
  }// the fish will disappear when touches the plastic five times.
  //pred2=d2[;
  if (counter1==4) {
    heartx1=-100;
    hearty1=-100;
  } else if (counter1==3) {
    heartx2=-100;
    hearty2=-100;
  } else if (counter1==2) {
    heartx3=-100;
    hearty3=-100;
  } else if (counter1==1) {
    heartx4=-100;
    hearty4=-100;
  }
  fill (#DE4848);
  heart(heartx1, hearty1, 0.3);
  heart(heartx2, hearty2, 0.3);
  heart(heartx3, hearty3, 0.3);
  heart(heartx4, hearty4, 0.3);
  heart(heartx5, hearty5, 0.3);
  //println(counter);
  if (counter2==4) {
    heartx1=-100;
    hearty1=-100;
  } else if (counter2==3) {
    heartx2=-100;
    hearty2=-100;
  } else if (counter2==2) {
    heartx3=-100;
    hearty3=-100;
  } else if (counter2==1) {
    heartx4=-100;
    hearty4=-100;
  }
  fill (#DE4848);
  heart(heartX1, heartY1, 0.3);
  heart(heartX2, heartY2, 0.3);
  heart(heartX3, heartY3, 0.3);
  heart(heartX4, heartY4, 0.3);
  heart(heartX5, heartY5, 0.3);
  //println(counter);
}
    }
void move() {
  for ( int i=0; i < x.length; i++) {
    x[i] += xspeed[i] ;
    y[i]  += yspeed[i] ;
  }
  for ( int i=0; i < u.length; i++) {
    u[i] += uspeed[i] ;
    v[i]  += vspeed[i] ;
  }
}
void bounce() {
  for ( int i=0; i < x.length; i++) {
    if ( x[i] <= 0 + size[i]/2 || x[i]>= width - size[i]/2-40) {
      xspeed[i] = -xspeed[i];
    }
    if ( y[i] <= 0+ size[i]/2 || y[i]>= height - size[i]/2-30 ) {
      yspeed[i] = -yspeed[i];
    }
  }
  for ( int i=0; i < x.length; i++) {
    if ( u[i] <= 0 + size[i]/2 || u[i]>= width - size[i]/2-40) {
      uspeed[i] = -uspeed[i];
    }
    if ( v[i] <= 0+ size[i]/2 || v[i]>= height - size[i]/2-30) {
      vspeed[i] = -vspeed[i];
    }
  }
}
void plastic(float u, float v, float size, color c) {
  fill(c);
  noStroke();
  circle(u, v, size);
  fill(c);
  noStroke();
  circle(u+10, v, size/2);
  fill(c);
  noStroke();
  circle(u-10, v-5, size-7);
  fill(c);
  noStroke();
  circle(u-10, v+4, size-8);
  fill(c);
  noStroke();
  circle(u-10, v+8, size-7);
}
void heart (float x, float y, float size) {
  pushMatrix();
  translate(x, y);
  scale(size);
  noStroke();
  beginShape();
  vertex(50, 15);
  bezierVertex(50, -5, 90, 5, 50, 40);
  vertex(50, 15);
  bezierVertex(50, -5, 10, 5, 50, 40);
  endShape();
  popMatrix();
}
void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 2 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port “/dev/cu.usbmodem----” or “/dev/tty.usbmodem----”
  // and replace PORT_INDEX above with the index number of the port.
  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = ‘\n’  Linefeed in ASCII
  myString = null;
  sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO];
}
void getSerialData() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Some videos we took during the process:

“🐟 can eat 🦐 and grow!!”

Struggles with shrimps and plastic:

More struggle:

Struggle with difficult controlling of the fish:

We’ve had a great time making this game, and I really learnt a lot during this semester. Although it’s a bit frustrating that we spent a big part of this class online, the class environment is still so good. I always look forward to interaction lab on Tuesdays and Thursdays, and being able to code some interesting things has been so stress-relieving. Every time I’m tired of writing essays or doing maths, I would always do a bit practicing of coding myself, which is so interesting and I had a lot of fun.
Thank you so much Andy for providing us so much help not only for the final but for the whole semester!! 🥰🎉

Recitation 10: Image & Video

During this recitation, we practiced the serial communication again and linked sound with image.

For the first exercise, surprisingly and not so surprisingly, the most difficult part turns out to be finding the picture haha. My nickname in middle school was Peppa pig, so I found a picture of it.

My code is below:

Arduino:

void setup()
{
  Serial.begin(9600);
}

void loop() 
{
  int sensor1 = analogRead(A0);
  int sensor2 = analogRead(A1);    
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.println(); // add linefeed after sending the last sensor value
  delay(100);
}

Processing:

import processing.serial.*;


int NUM_OF_VALUES_FROM_ARDUINO = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int sensorValues[];      /** this array stores values from Arduino **/

String myString = null;
Serial myPort;

PImage photo;


void setup() {
  size(800,800);
  background(#7BE5B1);
 
  photo = loadImage("Peppa pig.jpeg");
  printArray(Serial.list());
  setupSerial();
}

void draw() {
   background(#7BE5B1);
   getSerialData();
  printArray(sensorValues);
  
  float x= map(sensorValues[0],0,1023,0,width);
  float y= map(sensorValues[1],0,1023,0,height);
  
  image(photo, x, y);
  
  }
    
    
  void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[2], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO];
}

void getSerialData() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Video:

For the exercise2, it was quite tricky, and my processing got stuck thousands of times. I went to office hours to ask for helps, and Skye explains to me the function of abs(), R, PR and so on. I found the final result very interesting, the effect looks like the wall in swimming pool and the sound also somehow reminds me of that as well.

My code is below:

 

import processing.sound.*;
import processing.video.*;
SinOsc sine;
Env env;
String[] cameras = Capture.list();
Capture cam;
color myColor;
float attackTime = 0.001;
float sustainTime = 0.004;
float sustainLevel = 0.3;
float releaseTime = 0.4;
float d;
float f;
int s=20;
float R;
float PR;
void setup() {
  size(640, 480);
  cam = new Capture(this, cameras[1]);
  cam.start();
  printArray(cameras);
  // Create the sine oscillator.
  sine = new SinOsc(this);
  env  = new Env(this);
}
void draw() {
  background(0);
  if (cam.available()) {
    cam.read();
  }
  myColor=cam.get(width/2, height/2);
  R=red(myColor);
  f = map(R, 0, 255, 100, 600);
  float d = abs(R-PR);
  sustainTime = map(d, 0, 100, 0.001, 1);
  sustainTime = constrain(sustainTime, 0.001, 1);
  println(d);
  for (int x=0; x<width; x=x+s) {
    for (int y=0; y<width; y=y+s) {
      color c=cam.get(x, y);
      float red = red(c);
      float green= green(c);
      float blue = blue(c);
      float factor = map(red, 0, 255, 0.05, 2);
      noStroke();
      fill(red, green, blue, red);
      ellipse(x, y, s*factor, s*factor);
    }
  }
  // get the pixel color
  if (d>20) {
    sine.play();
    sine.freq(f);
    env.play(sine, attackTime, sustainTime, sustainLevel, releaseTime);
  }
  PR=R;
  //image(cam, 0, 0);
}

Video:

Recitation 9 – Sound in Processing

For this recitation, we use sound to do a lot of exercises. 

In the first exercise, we use keys to control the start of two sounds.

The code is below:

import processing.sound.*;

// declare three SoundFile objects
SoundFile kick;
SoundFile snare;


void setup() {
  size(640, 480);
  // create the objects and load sounds into them
  kick = new SoundFile(this, "kick.wav");
  snare = new SoundFile(this, "snare.wav");
 
}

void draw() {
  background(0);  
}

void keyPressed() {
  // when a key is pressed, use a switch statement
  // to determine which key was pressed
  // and play the appropriate sound file and set a background color
  switch(key) {
    case 'k':
      kick.play();
      background(255, 0, 0);
      break;
    case 's':
      snare.play();
      background(0, 255, 0);
      break;
    
  } 
}

 The video:

In the second exercise, we use the vibration motor to detect and show the beats of the song “Lost On The Free Way”.

The code is below:

For processing:

import processing.serial.*;


int NUM_OF_VALUES_FROM_ARDUINO = 1;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int sensorValues[];      /** this array stores values from Arduino **/

String myString = null;
Serial myPort;

import processing.sound.*;

// declare an AudioIn object
SoundFile sound;
// declare an Amplitude analysis object to detect the volume of sounds
Amplitude analysis;

void setup() {
  size(500, 500);
   sound= new SoundFile(this, "Lost On The Freeway.mp3");
   
   sound.play();
  // create the Amplitude analysis object
 
  // use the microphone as the input for the analysis
  analysis = new Amplitude(this);
  analysis.input(sound);

  setupSerial();
  
}

void draw() {
  background(0);
   println(analysis.analyze());
  background(0);
   float volume = analysis.analyze();
  // map the volume value to a useful scale
  float diameter = map(volume, 0, 1, 0, width);
  // draw a circle based on the microphone amplitude (volume)
  circle(width/2, height/2, diameter); 
  getSerialData();
 sensorValues[0]=int(map(analysis.analyze(),0,1,0,200));
}


void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list( )[6], 9600);

  myPort.clear();
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO];
}

void getSerialData() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Arduino code:

// IMA NYU Shanghai
// Interaction Lab

/**
  This example is to send multiple values from Processing to Arduino.
  You can find the Processing example file in the same folder which works with this Arduino file.
 **/
#define ZD 3
#define NUM_OF_VALUES_FROM_PROCESSING 1    /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/


/** DO NOT REMOVE THESE **/
int tempValue = 0;
int valueIndex = 0;

/* This is the array of values storing the data from Processing. */
int processing_values[NUM_OF_VALUES_FROM_PROCESSING];
#include <Servo.h>

Servo myservo;  // create servo object to control a servo

int potpin = A0;  // analog pin used to connect the potentiometer
int val;    

void setup() {
  Serial.begin(9600);
  pinMode(ZD,OUTPUT);
   
}

void loop() {
  getSerialData();
analogWrite(ZD,200);
delay(5000);
analogWrite(ZD,0);
delay(1000);
analogWrite(ZD,100);
delay(5000);
analogWrite(ZD,0);
delay(1000);
  // add your code here using elements in the values array

  //this is an example connecting a buzzer to pin 8
  /*
    if (processing_values[0] == 1) {
      //turn on an LED when the mouse is pressed
      digitalWrite(13, HIGH);
      // map values from mouseX to frequency from (0 - 500 pixels)
      //to the output pitch range (120 - 1500Hz)
      int f = map(processing_values[1], 0, 500, 120, 1500);
      // map values from mouseY to frequency from (0 - 500 pixels)
      //to the output duration range (10 - 2000 milliseconds)
      int d = map(processing_values[2], 0, 500, 10, 2000);
      // play the pitch:
      tone(8, processing_values[1], processing_values[2]);
      delay(1);        // delay in between reads for stability
    } else {
      digitalWrite(13, LOW);
    }
  */
  //end of example

}
//receive serial data from Processing
void getSerialData() {
  while (Serial.available()) {
    char c = Serial.read();
    //switch - case checks the value of the variable in the switch function
    //in this case, the char c, then runs one of the cases that fit the value of the variable
    //for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase
    switch (c) {
      //if the char c from Processing is a number between 0 and 9
      case '0'...'9':
        //save the value of char c to tempValue
        //but simultaneously rearrange the existing values saved in tempValue
        //for the digits received through char c to remain coherent
        //if this does not make sense and would like to know more, send an email to me!
        tempValue = tempValue * 10 + c - '0';
        break;
      //if the char c from Processing is a comma
      //indicating that the following values of char c is for the next element in the values array
      case ',':
       // processing_values[valueIndex] = tempValue;
        //reset tempValue value
        tempValue = 0;
        //increment valuesIndex by 1
        valueIndex++;
        break;
      //if the char c from Processing is character 'n'
      //which signals that it is the end of data
      case '\n':
        //save the tempValue
        //this will b the last element in the values array
       // processing_values[valueIndex] = tempValue;
        //reset tempValue and valueIndex values
        //to clear out the values array for the next round of readings from Processing
        tempValue = 0;
        valueIndex = 0;
        break;
    }
  }
}

Video:

The vibration motor is kind of scary hahaha, but the moving circle is very cool that gives a feel to the atmosphere and make the music even more popping and even interactive for it engages sights in the processes of enjoying music. It was a bit hard to use the map function for a proper extent, and it was also quite tricky when making the portfolio, for the music must be included in a portfolio called data with the processing sketch in the same portfolio.

In the homework part, we tried to use amplitude to draw something on Processing, I chose to draw a line that goes up and down accordingly.

The code:

 

import processing.sound.*;

// declare an AudioIn object
AudioIn microphone;
// declare an Amplitude analysis object to detect the volume of sounds
Amplitude analysis;

int x=0;
float prediameter;
 
void setup() {
   background(198,236,255);
  size(640, 480); 
  // create the AudioIn object and select the mic as the input stream
  microphone = new AudioIn(this, 0);
  // start the mic input without routing it to the speakers
  microphone.start();
  // create the Amplitude analysis object
  analysis = new Amplitude(this);
  // use the microphone as the input for the analysis
  analysis.input(microphone);
}      

void draw() {
  println(analysis.analyze());
 
  
  // analyze the audio for its volume level
  float volume = analysis.analyze();
  // map the volume value to a useful scale
  float diameter = map(volume, 0,1,  200, 0);
  // draw a circle based on the microphone amplitude (volume)
 
 
  x=x+1;
  strokeWeight(5);
  stroke(252,199,199);
  line(x, prediameter,x+1,diameter); 
  
  
  prediameter=diameter;
}

 Video:

It’s really fun to “see” your voice and to make different waves with it. However, it’s not so sensitive, I need to be very loud or very sharp can I make the line wave dramatically.

Essay

Our project is called Plastic Fish. It’s purpose is to convey the idea of protecting the marine environment by cutting down the amount of pollution people poured into the sea. According to Wekipedia, Marine pollution is a combination of chemicals and trash, most of which comes from land sources and is washed or blown into the ocean. This pollution results in damage to the environment, to the health of all organisms, and to economic structures worldwide. There are 6 types of marine pollution, they are pollution from marine debris, plastic pollution, including micro-plastics, ocean acidification, nutrient pollution, toxins and underwater noise.

 

We planned to build an interactive game involving two players. Each representing a fish. They can use sensors(potential meters/pressure sensors/sound/movement)to control the movement of the fish. The processing will have a background of blue ocean with  micro plastic(or actual plastic products) and fish food(maybe like little shrimps) bouncing around the whole page. For the micro plastic, it will start with one, and then double once 20 seconds. And the two players will need to avoid the micro plastic. By eating the little shrimps, they will grow bigger, and the player that is bigger at the end of the game will “win” the game. However, I add these quotation marks because that actually, there will be no winner in the game. We will also design a rising “pollution layer” that rises from the bottom of the sea and will reach 7/8 of the pages at the end. The fishes can only swim above this layer, so eventually the two players will find themselves stuck on the top. With the limited space and continuously doubling plastics, there’s no way that the fishes can still avoid touching them. So these two fishes will both die, no matter how much they struggle to survive. 

In the previous reading of 《Newton’s Sleep》, it describes a world that is “prototype of the Special Earth Satellite itself” because the earth is already over-polluted. This also stimulates us to develop the idea of reminding people we can’t consume the earth too wildly, it’s important that we appreciate all we’ve got and limit our pollution and harm to the earth. After all, what we’ve done to the earth will result on ourselves eventually, for the earth’s good, our own good and our latter generation’s good, we should be cautious of our actions now.

Recitation 8: Serial Communication

For this recitation, we did many exercises to get familiar with the connection between Arduino and processing.

Exercise 1:

The first exercise is using Arduino to send two analog values to Processing via serial communication. I find this one the most confident one for me to do. 
Below is the Arduino code:

void setup() {
  Serial.begin(9600);
}

void loop() {
  int sensor1 = analogRead(A0);
  int sensor2 = analogRead(A1);
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.println();
  
}

Processing:

import processing.serial.*;
float x;
float y;

int NUM_OF_VALUES_FROM_ARDUINO = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int sensorValues[];      /** this array stores values from Arduino **/
int sensorp[];
String myString = null;
Serial myPort;

void setup() {
  size(500, 500);
  setupSerial();
  sensorp = new int[NUM_OF_VALUES_FROM_ARDUINO];
  background(0);
}

void draw() {
  getSerialData();
  printArray(sensorValues);  
  x = map(sensorValues[0],0,1023,0,width);
  y = map(sensorValues[1],0,1023,0,width);

  stroke(255);
  line(sensorp[0], sensorp[1],x, y);
  sensorp[0]=int(x);
  sensorp[1]=int(y);
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[2], 9600);
  
  myPort.clear();
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO];
  
}
void getSerialData() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Video: 

Exercise 2:

For this exercise, I found two codes for the bouncing circle. The first one is like this:

/**
 * Bounce. 
 * 
 * When the shape hits the edge of the window, it reverses its direction. 
 */
 
int rad = 60;        // Width of the shape
float xpos, ypos;    // Starting position of shape    

float xspeed = 2.8;  // Speed of the shape
float yspeed = 2.2;  // Speed of the shape

int xdirection = 1;  // Left or Right
int ydirection = 1;  // Top to Bottom


void setup() 
{
  fullScreen();
  noStroke();
  frameRate(30);
  ellipseMode(RADIUS);
  // Set the starting position of the shape
  xpos = width/2;
  ypos = height/2;
}

void draw() 
{
  background(102);
  
  // Update the position of the shape
  xpos = xpos + ( xspeed * xdirection );
  ypos = ypos + ( yspeed * ydirection );
  
  // Test to see if the shape exceeds the boundaries of the screen
  // If it does, reverse its direction by multiplying by -1
  if (xpos > width-rad || xpos < rad) {
    xdirection *= -1;
  }
 
    ydirection *= 0;
  

  // Draw the shape
  ellipse(xpos, ypos, rad, rad);
}

However, I found it really hard to use this code to communicate with processing, I tried several times but don’t know how to use “if statement” in this case. And later Tracy and I figured another way to code the bouncing circle. My servo isn’t working, so I did the coding part with Tracy and she filmed how the servo interacts with the bouncing circle.

Arduino code:

#define NUM_OF_VALUES_FROM_PROCESSING 1    /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
#include <Servo.h>
/** DO NOT REMOVE THESE **/
int tempValue = 0;
int valueIndex = 0;
bool prevState = false;
bool currentState = false;
Servo myservo;
int processing_values[NUM_OF_VALUES_FROM_PROCESSING];
void setup() {
  Serial.begin(9600);
  pinMode(12, OUTPUT);
  myservo.attach(10);
}
void loop() {
  getSerialData();
  currentState = processing_values[0];
 
  if (currentState != prevState && currentState == 1) {
    myservo.write(90);
   
    myservo.write(0);
  } else {
    myservo.write(0);
  }
  prevState = currentState;
}
//receive serial data from Processing
void getSerialData() {
  while (Serial.available()) {
    char c = Serial.read();
    switch (c) {
      case '0'...'9':
        tempValue = tempValue * 10 + c - '0';
        break;
      case ',':
        processing_values[valueIndex] = tempValue;
        tempValue = 0;
        valueIndex++;
        break;
      case '\n':
        processing_values[valueIndex] = tempValue;
        tempValue = 0;
        valueIndex = 0;
        break;
    }
  }
}

Processing:

import processing.serial.*;
int NUM_OF_VALUES_FROM_PROCESSING = 1;  /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int processing_values[] = new int[NUM_OF_VALUES_FROM_PROCESSING]; /** this array stores values you might want to send to Arduino **/
Serial myPort;
String myString;
int x=30;
int speedX = 5;
void setup() {
  fullScreen();
  background(0);
  setupSerial();
}
void draw() {
  background(0);
  circle (x, 100, 100);
  x= x+speedX;
  if (x > width-40 || x < 0+30) {
    speedX = -speedX;
  }
  if (x > width-30) {
    processing_values[0] = 1;
  } else {
    processing_values[0] = 0;
  }
  sendSerialData();
}
void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[2], 9600);
  myPort.clear();
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
}
void sendSerialData() {
  String data = "";
  for (int i=0; i<processing_values.length; i++) {
    data += processing_values[i];
    //if i is less than the index number of the last element in the values array
    if (i < processing_values.length-1) {
      data += ","; // add splitter character "," between each values element
    }
    else {
      data += "\n"; // add the end of data character linefeed "\n"
    }
  }
  //write to Arduino
  myPort.write(data);
  print(data); // this prints to the console the values going to arduino
}

Video:

Additional homework:

I found the communication quite confusing for there are many sections and it’s sometimes even impossible to figure out which part goes wrong. So I participated in the workshop and learnt more about the whole logic behind this.

Arduino code:

// IMA NYU Shanghai
// Interaction Lab
// For recitation 8 homework, controlling the display of two stars with two buttons


void setup() {
  Serial.begin(9600);
}

void loop() {
  int button1 = digitalRead(9);
  int button2 = digitalRead(10);

  // keep this format
  Serial.print(button1);
  Serial.print(",");  // put comma between values
  Serial.print(button2);
  Serial.println(); // add linefeed after sending the last value

  // too fast communication might cause some latency in Processing
  // this delay resolves the issue.
  delay(100);
}

Processing:

// IMA NYU Shanghai
// Interaction Lab
// For recitation 8 homework, controlling the display of two stars with two buttons

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] buttonValues;      /** this array stores values from Arduino **/
int prevButtonValue0 = 0;
int prevButtonValue1 = 0;
boolean star1Display = false;
boolean star2Display = false;


void setup() {
  size(600, 600);
  background(0);
  setupSerial();
}


void draw() {
  updateSerial();
  printArray(buttonValues);

  background(0);

  if (buttonValues[0]==1 && buttonValues[0]!= prevButtonValue0) {
    star1Display = !star1Display;
  }
  if (buttonValues[1]==1 && buttonValues[1]!= prevButtonValue1) {
    star2Display = !star2Display;
  }

  if (star1Display) {
    pushMatrix();
    translate(width*0.3, height*0.3);
    rotate(frameCount / 200.0);
    star(0, 0, 30, 70, 5); 
    popMatrix();
  }

  if (star2Display) {
    pushMatrix();
    translate(width*0.7, height*0.7);
    rotate(frameCount / 400.0);
    star(0, 0, 80, 100, 40); 
    popMatrix();
  }

  prevButtonValue0 = buttonValues[0];
  prevButtonValue1 = buttonValues[1];
}



void star(float x, float y, float radius1, float radius2, int npoints) {
  float angle = TWO_PI / npoints;
  float halfAngle = angle/2.0;
  beginShape();
  for (float a = 0; a < TWO_PI; a += angle) {
    float sx = x + cos(a) * radius2;
    float sy = y + sin(a) * radius2;
    vertex(sx, sy);
    sx = x + cos(a+halfAngle) * radius1;
    sy = y + sin(a+halfAngle) * radius1;
    vertex(sx, sy);
  }
  endShape(CLOSE);
}




void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 6 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  buttonValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          buttonValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Video:

  

Research:

 For the coming classes we will be learning how to manipulate image, video and sound. I’m really excited about this, I think it can add a lot more interesting effects to the interaction. It also is so relevant to the final project as well. 

For our project, one of our ideas is to visualize the sound. Since originally, sound is a wave that comes from vibration. If it’s impossible for people with hearing problems to hear, it would be great for them to see the wave and feel the vibration, which are the two parts that builds the sound. By research we found that it’s possible to use processing to code wave or random shapes for different pitches, which I find really interesting.

Also, by searching for “processing image” on Google, I found that Processing currently works with GIF, JPEG, and PNG images. So it’s also possible to add actual pictures to the final result, which can also be very useful. I believe that using a GIF can have very funny effect.

Final Project Proposal

1, Circles

This project is designed for kids in kindergarten. It aims at teaching kids the three-primary colors, as well as how to properly compete and cooperate with one another. We were inspired by both the exercise we did during the class, the one that has a circle bouncing around the screen; and the project mood reflecting floor published by the “visualcraft” also gives me the idea of blending colors of each circle. We think the challenges may be that the switch of two mode as well as the connection of multiple sensors.

2, Etch-B-sketch

This is designed for people in all age and train their drawing skills as well as memory. This is inspired by the project “Etch-A- Sketch”, and it adopt the similar way of drawing like Etch_A_Sketch, switching the potentiometers, the processing will show a graph for 10s, and then the player will need to draw it. The challenge we think will be how to make the definition of a successful drawing and an unsuccessful one. 

3, Seeound

The specific audience is people with hearing impairments. This blog gave me the inspiration of this design choice that we can enable those special populations to “see” the sound. We decided to use processing to create a image that can change according to music pitch. And some LEDs can light according to the rhythm. It can help people with hearing impairments to see the shake of sound and how can we hear the sound, how does sound compose. When they use this project, it helps them learn how to live more vibrantly. It is the way they fuel their body, mind and spirit.

PREPARATORY RESEARCH AND ANALYSIS

I found two interaction projects that inspire me on YouTube. One is the solar tracker designed by Robot Greek in the video Top 10 Arduino projects all the time । Amazing Arduino school projects genius youtuber” . The other is the mood reflecting floor published by the “visualcraft”.  

For the first one, I think it’s successful because it can interact with the environment. While according to the “Art, Interaction and Engagement”, it mentions “infant behavior”, which is that infants will react according to the environment, I think this project can be viewed of a manifestation of the theory: the machine turn according to the light. From this project, it inspires me that, this project is very much like a sunflower, and maybe I can design a sunflower according to this idea but add more interaction with human, and make this interaction, continuous, according to the “Art, Interaction and Engagement”. And for the latter one, it provides a really good visual experience to me, and it’s also pretty entertaining and can interact with people for quite a long time. Also, I think I also reminds me of the interactive floor I saw in a hotel that there are many fishes projected on the floor, and they will “swim” away if you step on them. This inspires me that maybe I can also design my project with items that will change position or forms according to human movements.

I think a successful interactive experience should not only be containing the machine and people, as I said before in my definition of interaction. What’s more important in an interactive experience is that it should inspire the user to interact more, rather than just a one-round interaction. According to “Art, Interaction and Engagement”,  “the interaction [should be] continuous and fluid”. which, I think is also what I can improve for my midterm project as well. And also, the interactive experience should have a deeper intention. Just like what we did in our group project, by reading “The Winter Market” by William Gibson, we see the insight of how technology has negative influence on people, and integrated that in our design. So it is that we designed our midterm project to help people with eyesight disability.

 

Reference:

  • E. Edmonds, “Art, Interaction and Engagement,” 2011 15th International Conference on Information Visualisation, 2011, pp. 451-456, doi: 10.1109/IV.2011.73.
  • TGibson, W. (2000). The winter market. Burning chrome.

Recitation 7: Functions and Arrays

For this recitation, we tried to do more practices with Processing. It’s also a really good revision for Thursday’s lecture. By using parts of the codes we did during the lecture individually, I get to understand the codes more.

I first created a easy one based on what Professor Andy taught us during the lecture. I used the code and simply changed the graph and background. This graph and the color of the background were inspired by a nail art that I really like recently, but sadly due to the lockdown I’m not able to do the nails, I can only achieve the nails in this way😭

This is the nail art:

This is my code:

 

int count=10;

float x[] = new float[count];
float y[] = new float[count];
float size[] = new float[count];

float r[] = new float[count];
float g[] = new float[count];
float b[] = new float[count];
color flowerColor[] = new color[count];


float xspeed[] = new float[count];
float yspeed[] = new float[count];

void setup(){
  size(800,800);
  background(252,251,240);

 
 for ( int i=0; i < x.length; i++) {
   
    x[i] = random (100, width-100);
    y[i] = random (100, height-100);
    size[i]=random (50,80);


    r[i] = random(50,255);
    g[i] = random(100,255);
    b[i] = random(100,255);
  
    flowerColor[i] = color(r[i], g[i], b[i]);
     xspeed[i] = random (2, 5);
     yspeed[i] = random (2, 5);
 }
}
    void draw() {
      background(252,251,240);
      
    for ( int i=0; i < x.length; i++) {
    Flower(x[i], y[i], size[i], flowerColor[i]);
  }   
  move();
  bounce();
    }
    
    void move() {

  for ( int i=0; i < x.length; i++) {
    x[i] += xspeed[i] ;
    y[i]  += yspeed[i] ;
  }
}
    void bounce() {

  for ( int i=0; i < x.length; i++) {
    if ( x[i] <= 0 + size[i]/2 || x[i]>= width - size[i]/2 ) {
      xspeed[i] = -xspeed[i];
    }

    if ( y[i] <= 0+ size[i]/2 || y[i]>= height - size[i]/2 ) {
      yspeed[i] = -yspeed[i];
    }
  }
}

    void Flower(float x, float y, float size, color c) { 
 fill(0);
 ellipse(x, y, size, size);

  fill(0);
 ellipse(x+size/2, y-size/2, size, size);
 
 fill(0);
 ellipse(x+size, y, size, size);

 fill(0);
 ellipse(x+size/2-size/3, y+size/2, size, size);

 fill(0);
 ellipse(x+size/2+size/3, y+size/2, size, size);

 fill(c);
 ellipse(x+size/2, y+5, size, size);

  }

This is how it looks like:

However, I wanted to challenge myself more, so I wanted to make one with a heart shape and are listed in a grid with blinking colors. This was pretty hard. I first came up with something like this:

Though it was quite pretty, it wasn’t what I wanted. Then I wanted to study how exactly does the codes for a heart shape works and how it was composited. I reached out to Professor Andy for help and learnt how to code for a heart as well as how to control its size and position, and I also learnt how to make it blink.

After that, I attended the workshop about functions and arrays and learnt how to list the hearts in a grid.

This is my result:

 

int count=13;
int number =10;
float[] xPositions =new float[number];
float[] yPositions =new float[number];

float x[] = new float[count];

color random1;
color random2;
int delay = 1000;// ONE SEC
int now; 
//a flag
boolean red = false;


void setup(){
  size(800,800);
  background (203,253,255);
  
   for ( int i=0; i < x.length; i++) {
   
    x[i] = random (100, width-100);
    
    now = millis();
 random1 = color(random(255),random(255),random(255));
 random2 = color(random(255),random(255),random(255));
    
}
  for(int i=0; i<number; i++){
    for (int j=0; j<number;j++){
  xPositions[i]=i*100-150;
  yPositions[j]=j*100-90;
    }
  }
}

void draw() {
  background (203,253,255);

for(int i=0;i<number;i++){
    for (int j=0;j<number;j++){
  pushMatrix();
  translate(xPositions[i],yPositions[j]);
if (millis() - now > delay) { 

    //change flag
    red = !red;

    //reset counter
    now = millis();
  }


  if (red)
    fill(random1);
  else
    fill(random2);
  heart( 100, 100, 2);
  popMatrix();
 
    }
}
  

    }

void heart (float x, float y, float size) {

  translate(x, y);
  scale(size);
  
  noStroke();
  beginShape();
  vertex(50, 15);
  bezierVertex(50, -5, 90, 5, 50, 40);
  vertex(50, 15);
  bezierVertex(50, -5, 10, 5, 50, 40);
  endShape();
}

  • Q1: In the reading “Art, Interaction and Engagement” by Ernest Edmonds, he identifies four situations in an interactive artwork: ‘Static’, ‘Dynamic-Passive’, ‘Dynamic-Interactive’ and ‘Dynamic-Interactive(Varying)’. From the exercise you did today which situations you identify in every part you executed? Explain.

Static: The part that I design the graph. After I code for the graph and click play, I can see the graph, but it doesn’t change and it doesn’t interact with me.

Dynamic-Passive: It might be like the bouncing and moving movements I had in my first try. Here, the internal mechanism is the code.

Dynamic-Interactive:  I think it’s the part that we assigned random color to the graph. Because every time we click play, the color will be shown randomly. It’s a kind of interaction between the viewer and we do have “an active role in influencing the changes in the art project”(3).

Dynamic-Interactive(Varying): Because that the color is randomly assigned, it has the sense of unpredictability in it. Also, in my blinking hearts, the art object changes color as well. So I think it fits in the varying dynamic-interactive concept.

  • Q2: What is the benefit of using arrays? How might you use arrays in a potential project?

In my case, I think listing the graphs in a grid is really tidy and visually comforting. It also reminds me of the game of whack-a-mole, maybe I can design a similar game in a potential project with some of the graphs in the grid blinking and others having other movements. The grid gives a sense of unity and solidity despite the fact that there are a lot of changes and therefore provides balance to the project as a whole.

 

 

 

Recitation 6: Animating an Interactive Poster

For this recitation, we tried to make a animated poster by using processing. I find it really interesting to do posters in this way. Though it takes some time to learn how to apply certain formulas, this process still has a lot of fun. I integrated my homework with the recitation together into one poster, and it’s below.

Video:

My code is below:

void setup(){
  size(768,1024);
  background(234,161,233,100);
  

}
void draw() {

  background(234,161,233,100);

  
  background(108,37,95,100);
  fill(247, 241, 207); 
  ellipse(frameCount*2, frameCount*2, 350, 300);


  fill(224,93,209,100);
  triangle(590,0,0,0,0,500);
  
  fill(192,90,229,100);
  triangle(768,0,178,0,768,500);
  
  fill (247,183,5);
  textSize(300);
  text("I",width/2-35,250);
  

  fill (255,111,0);
  textSize(200);
  text("M",width/2-66,200);

  fill(255,38,0);
  stroke(0);
  rect(width/2+23,150,28,10,28);
 
  
  fill(0);
  textSize(25);
  text("IMA Spring 22 End-Of-Semester Show",width/2-195,360);
 
  fill(0);
  textSize(25);
  text("Location: 8th floor",width/2-50,410);

  fill(0);
  textSize(25);
  text("Date: Friday May 13",width/2-50,460);
  
  fill(0);
  textSize(25);
  text("Time: 6pm to 8pm",width/2-50,510);
  
  
  Float r=random(0,255);
  Float g=random(0,255);
  Float b=random(0,255);
     
   
  float d = dist(mouseX, mouseY, 125, 200);
  if(d < 100) {
    fill(r,g,b);
  } else {
    fill(167,205,255); 
  }
  ellipse(125, 200, 200, 200);
  
  float f = dist(mouseX, mouseY, 650, 200);
  if(f < 100) {
    fill(r,g,b);   
  } else {
    fill(267, 205, 255);
  }
  ellipse(650, 200, 200, 200);
 
  fill(232,232,232,100);
  ellipse(-50,700,250,250);
 
  fill(234,206,231,100);
  ellipse(50,700,250,250);
  
  fill(234,178,227,100);
  ellipse(150,700,250,250);
  
 fill(229,147,242,100);
  ellipse(250,700,250,250);
  
   fill(222,114,240,100);
  ellipse(350,700,250,250);
  
   fill(217,73,240,100);
  ellipse(450,700,250,250);
  
    fill(210,38,237,100);
  ellipse(550,700,250,250);
  
   fill(232,40,207,100);
  ellipse(650,700,250,250);
  
   fill(234,19,206,100);
  ellipse(750,700,250,250);
  
   fill(209,2,182,100);
  ellipse(850,700,250,250);
  
  fill(247,183,5,200);
  rect(0,700,768,700);
  }



A few parts where I found a bit struggling with, one is the locations for these elements, I need to try multiple times and change accordingly to find out the most suitable coordination. Another is that I find the sequence of the codes matters, for if I put some codes before others, they will be covered by the patterns whose codes come after them.

Nonetheless, it’s really interesting to add elements one by one and feel that I have more and more idea about this poster, and to add more animations and patterns. Sometimes some mistake codes can have unexpected effect as well, for example, the ellipse that slide from the top left to the bottom right was actually an accident, I didn’t mean to place it this way, I wanted to create the effect of many circle dropping from the top like rain, but this turn out to be like a spotlight, which I found also very cool so I kept it that way.

Girltar- Elaine – Tracy

Girltar- Elaine – Tracy

 CONTEXT AND SIGNIFICANCE 

For our group project, we made a human-bodypart-selling machine based on Winter Market. The process of preparing for the group project inspires my deeper understanding of interaction: It’s both about the interaction the machine has with people, and about the storytelling. Also, the interaction should be meaningful and designed for a purpose. For our previous group project, it was designed to remind people that we shouldn’t let our desire control us and that we need to control our desire. For this midterm project, we not only want to help people with eyesight disability learn guitar, but also want to let normal people experience the difficulty of  people with eyesight problems and to cherish our ability to live as a normal people. Our target audience is mainly those people with eyesight disability because it’s hard for them to learn guitar when they’re not able to use their sight, however, their sensitive touch and hearing have been developed. Our project focuses on these two senses as their advantages and transfer the sight part into pressing the button-touch, and hearing the changing of the tone of guitar-hearing.

CONCEPTION AND DESIGN:

We used two different kinds of buttons in our design, one is the small button and the other is a bigger button. This is based on our considering of those people with eyesight disability can tell the difference by feeling the different size of the buttons. We have thought of using cardboards with different designs to stick on the buttons and enhance the experience of different touch feeling, however it affects the sensitivity of the button itself so we had to abandon that idea. Also, we made the robot with some details like the little feet and hair, so that by touching at these parts can have a clear expression to the users that we have built a man-like girl robot. And we mainly used cardboards and multiple boxes to build. We chose a tissue box because it happens to have a mouth on one of its surface that enables us to hide the Arduino inside. And to make it echos to our poster, we painted it blue and yellow. For this project, we had three previous drafts, the first one was a glove that connects to an ultrasound sensor with a spring. When the user moves his/her hand, as the spring goes longer and shorter, the distance sensor can sense different distance so to make different tone of the particular instrument the user steps on, or two instruments, each hand’s sensor controlling each instrument. And we also wanted to have four paddles on the ground and each hiding a button, which are the beats. However we found it really hard to construct all elements in time, so we eased that idea into a blanket with 20 buttons on it, five as a group, playing Do Re Mi Fa So for one particular instrument. There will be leds beside the buttons. When the game begins, the computer will randomly play a combined tone with two to three notes for one of the instrument, the user has to find the instrument and the notes. For the first round, it will give the user ten seconds, the leds will fade as count down, if the user succeed in finding the notes in time, all leds will blink and play a cheerful music, if not, then all leds will fade and there will be a frustrating voice. However, we didn’t have enough wires as well as buttons and leds, and we didn’t dare to rely on the delivery, so we just choose our current project idea and started 

 FABRICATION AND PRODUCTION:

Tracy and I did the whole process together, including the beforehand design and the whole constructing and coding process. After we decided our project, we started with building the circuit. It was pretty hard to connect all five buttons on one small breadboard, there were a lot of wires that sometime confused us. As we uploaded the code for the first time, it didn’t work. So we dissembled the circuit and connected the buttons one by one and upload the code each time, and we found out that one of the buttons wasn’t connected properly. After the buttons all work, we connected the ultrasound to the circuit. It wasn’t working at all at first, however, after Andy reminded us to delete the // before the code for ultrasound, it worked. We wanted to change the guitar sound into other pieces of music, however we didn’t have the SD card and we looked online to see if we can play music without the SD card and sadly we didn’t find a way to do so. 
After we fixed the circuit and the code, we started to build the cardboard part. We painted a graph and just chose cardboards that have suitable sizes, we also used many boxes. And we painted them into yellow and blue. After the paint is dry, we sticked them together by the hot glue gun.

We finished the composition of the project pretty early, and we invited many people to try and play. I invited my mum and grandfather to try and interact with Girltar, what was really surprising to me is that they both found out how to interact with Girltar pretty quickly even without my introduction and made really cool music clips. Tracy invited her roommates, they also created some good music clips.

CONCLUSIONS:

Our project aims to enable people who are beginner at guitar learning or have eyesight issue to play guitar. As I written in previous blog, interaction is an entertaining communication between two actants, with either or none of them human, but doing man-like actions. Also, it’s consuming people’s senses like sight, hearing and so on. I think the project suits in my definition of interaction pretty nicely because the result of interacting with Girltar is a random clip of music that’s completely designed by the user, which is pretty entertaining, and the background of this robot having a name and body like human, last but not least, this project uses people’s sight, hearing and touching at the same time.

If we had more time and if the delivery is normal, I really wished that we could play more music, also we could add some LEDs on the project to make more visual effect. And we could also try more materials to make the buttons both have different touching experiences and to function properly. We’ve learnt that when connecting a complicated circuit with multiple ingredients, we should add on the components one by one. Also, it’s important to test the sensibility of the sensors before the idea for the project is fixed. And through this experience, I understand the coding skills more and am able to add codes independently now.

Although the construction of our project is pretty simple, just using cardboards and boxes, however, the idea behind the appearance is important. There are many people with eyesight disabilities, for them, even the simplest things can be of great difficulty. And it’s even harder for them to learn an instrument. But we think that everyone have the same right to pursue their dreams, and although we can’t really experience how their lives are like, at least we can help a little bit by building this project. 

Our poster:

Our presentation video:

Our code:

void setup() {
#define echoPin 2 // attach pin D2 Arduino to pin Echo of HC-SR04
#define trigPin 3 //attach pin D3 Arduino to pin Trig of HC-SR04
// defines variables
long duration; // variable for the duration of sound wave travel
int distance; // variable for the distance measurement
int echoNote = 220;
int buttonOne = 4;
int buttonTwo = 5;
int buttonThree = 6;
int buttonFour = 7;
int buttonFive = 8;
int tomOne = 196;
int tomOnePlay = 0;
int tomTwo = 784;
int tomTwoPlay = 0;
int soundOut = 13;
void setup() {
  // put your setup code here, to run once:
  pinMode(trigPin, OUTPUT); // Sets the trigPin as an OUTPUT
  pinMode(echoPin, INPUT); // Sets the echoPin as an INPUT
  Serial.begin(9600); // // Serial Communication is starting with 9600 of baudrate speed
  pinMode(buttonOne, INPUT);
  pinMode(buttonTwo, INPUT);
  pinMode(buttonThree, INPUT);
  pinMode(buttonFour, INPUT);
  pinMode(buttonFive, INPUT);
  pinMode(soundOut, OUTPUT);
}
void loop() {
  //Clears the trigPin condition
 digitalWrite(trigPin, LOW);
 delayMicroseconds(2);
// Sets the trigPin HIGH (ACTIVE) for 10 microseconds
digitalWrite(trigPin, HIGH);
delayMicroseconds(10);
  digitalWrite(trigPin, LOW);
  // Reads the echoPin, returns the sound wave travel time in microseconds
  duration = pulseIn(echoPin, HIGH);
 // Calculating the distance
  distance = duration * 0.034 / 2; // Speed of sound wave divided by 2 (go and back)
// Displays the distance on the Serial Monitor
  Serial.print("Distance: ");
  Serial.print(distance);
 Serial.println(" cm");

  if (digitalRead(buttonOne) == HIGH) {
    tone(soundOut, 110, 100);
  }
  if (digitalRead(buttonTwo) == HIGH) {
    Serial.println(digitalRead(buttonTwo));
    tomOnePlay = 1;
  }
  if (tomOnePlay == 1) {
    tone(soundOut, tomOne, 50);
    delay(10);
    tomOne -= 6;
    if (tomOne < 41)
    {
      tomOne = 196;
      tomOnePlay = 0;
    }
  }
  if (digitalRead(buttonThree) == HIGH) {
    tomTwoPlay = 1;
  }
  if (tomTwoPlay == 1) {
    tone(soundOut, tomTwo, 40);
    delay(12);
    tomTwo -= 28;
    if (tomTwo < 262)
    {
      tomTwoPlay = 0;
      tomTwo = 784;
    }
  }
  if (digitalRead(buttonFour) == HIGH) {
    tone(soundOut, 147, 30);
    delay(10);
    tone(soundOut, 220, 30);
    delay(10); 
    tone(soundOut, 294, 30);
    delay(10);
    tone(soundOut, 349, 30);
    delay(10);
    tone(soundOut, 440, 30);
    delay(10);
  }
 

  //
  //  if (digitalRead(buttonOne) == HIGH)
  //  {
  //    tone(soundOut, 220, 30);
  //    delay(40);
  //    tone(soundOut, 330, 30);
  //    delay(40);
  //    tone(soundOut, 440, 30);
  //    delay(40);
  //    tone(soundOut, 523, 30);
  //    delay(40);
  //    tone(soundOut, 659, 30);
  //    delay(40);
  //  }
  //
  //  if (digitalRead(buttonTwo) == HIGH )
  //  {
  //    tone(soundOut, 262, 30);
  //    delay(40);
  //    tone(soundOut, 392, 30);
  //    delay(40);
  //    tone(soundOut, 523, 30);
  //    delay(40);
  //    tone(soundOut, 659, 30);
  //    delay(40);
  //    tone(soundOut, 784, 30);
  //    delay(40);
  //  }
  //
    if (digitalRead(buttonFive) == HIGH)  {
      if (distance < 7)
      {
        if (distance > 1)
        {
          echoNote = 220;
        }
      }
      if (distance < 9)
      {
        if (distance > 6)
        {
          echoNote = 262;
        }
      }
      if (distance < 11)
      {
        if (distance > 8)
        {
          echoNote = 294;
        }
      }
      if (distance < 13)
      {
        if (distance > 10)
        {
          echoNote = 330;
        }
      }
      if (distance < 15)
      {
        if (distance > 12)
        {
          echoNote = 392;
        }
      }
      if (distance < 17)
      {
        if (distance > 14)
        {
          echoNote = 440;
        }
      }
      if (distance < 19)
      {
        if (distance > 16)
        {
          echoNote = 523;
        }
      }
      if (distance < 21)
      {
        if (distance > 18)
        {
          echoNote = 587;
        }
      }
      if (distance < 23)
      {
        if (distance > 20)
        {
          echoNote = 659;
        }
      }
      if (distance < 25)
      {
        if (distance > 22)
        {
          echoNote = 784;
        }
      }
      if (distance < 40)
      {
        if (distance > 24)
        {
          echoNote = 880;
        }
      }
      if (distance < 40)
      {
        tone(soundOut, echoNote, 200);
      }
      if (distance > 39)
      {
        tone(soundOut, 220, 200);
      }
    }
}