Interactive Fashion-recitation2

This week, we are making a soft switch. 

We first made the switch. It contains three layers: one conductive material,  one layer of velostat, and one thick layer of cloth in between.

This switch is very sensitive, which is a bit to our surprise. As you can see in the serial monitor, the contact of fabric is sensed very in-time and immediately.

We first connected it to the led to test:

Then we turned to the buzzer. We encountered some issues with the buzzer. The voice always came out on and off, it was very unsteady, but we examined the code including map function and constraint function, they all worked fine. We guessed maybe the buzzer is having a bad connection.

 

Interactive Fashion-recitation1

The recitation this week is to make a soft switch. Professor Marcele asked us to go wild with our imagination! Sooo… I thought of this year is the year of rabbit🐇… and rabbits love eating carrots, their eyes would “light up” for their satisfaction when eating carrots… So I decided to make a soft switch that is — the carrot touching the rabbit’s nose! (However I eventually changed the carrot idea into grass, because we only had green and grey cloth and I thought a green carrot, or a grey one, is so weird, so I decided to make a grass shape!)

So I decided to use 2 red leds (as the rabbit’s eyes) and drew this circuit:

I tested on the breadboard before I use the soft materials, and it worked well:

So I turned to sewing. I had experience sewing clothes in the past, however, the conductive thread is very thick and slippery, I encountered quite a few issues when trying to tie it (especially with my long nails..which I already cut and change😌)

Below is the pictures of my final result.

However, because I sewed those threads pretty randomly, maybe resulted by random threads contacting each other without being noticed, the leds didn’t light up as I wished. As Marcele and I tested various threads using the multimeter, we found two threads that’s piping out and causing the mistaken touch. However, after controlling those two threads, the leds still didn’t work. I think the main issue is definitely those messy threads, and I’ll definitely improve this aspect in future sewing.

Last but not least, to answer the questions:
  • What is fashion for you? and why are you interested in fashion?

To me, fashion is what gives people(who wear the clothes) confidence and impresses others(who see the clothes) with a clear idea of his/her identity. I’m interested in fashion because I feel like it’s a very personal thing but at the same time allows you to be known and know others, the special feeling of having echos with others who share the same interest in some fashion items always make me feel connected and I enjoy this feeling, also I’m very curious how many forms can fashion takes.

  • What kinds of things do your clothing say about you and your values?

I love things that are a bit exaggerative and strange, but in terms of color I like to keep the whole outfit in the same tune. I guess this shows that I have some attitudes and stick to some certain things and that I’m open to new things and adventures, but deep inside I still hold some traditional values and have some reservations.

  • What are your main learnings and take aways from the readings?

In the readings, Davis mentioned that fashion is not only a personal expression, but also a combined result of society and culture. Which explains part of the reasons why people have preferences and some groups of people share the same preference — they may share similar social/culture background.

Ying Gao’s art pieces remind me of the mocking jay outfit in the film “Hunger game” and inspired me to take fashion into considerations that are more out of the frame and to innovate.

Final Blog Post

Survive the Plastic – Tracy and Elaine – Andy Garcia

 CONCEPTION AND DESIGN:

Our project aims to raise people’s awareness of protecting the Marine environment by reducing the immoderate use and production of plastic and trash. We searched online for the constitution of marine pollution and found that plastic pollution was a big part of it. So we also looked up online to search for relative information. We found that: Plastic pollution includes micro-plastic pollution; Micro-plastic can be consumed by fishes and they are deadly…

And with those information, we thought that it would be great if we can make it into a survival game to illustrate the idea that fish would die if touching the micro-plastic. We imagine that the interaction would be, players using some sensors to control the movement of fishes, for example potentiometers or joysticks. And for the computer’s respond, for example, if the fish touches a micro-plastic, the computer will respond by giving a sound and a visual effect that the fish will disappear as if it’s dead. However, initially, we wanted that if the player want to control the fish to eat shrimp, they should make sound to trigger the movement of eating, but after several tries by ourselves and our friends we found that it’s both quite difficult to use the potentiometer and the joystick to control the fish, and the players already are trying hard to control the fish’s position already, if they still need to make sound to trigger the movement of eating shrimp, it can be too hard and may reduce the fun of playing the game, so we decided not to add the sound triggering eating part. And by testing with Andy, he suggested that it would be better if the fish don’t disappear as soon as it touches the plastic, in order to give the players more chances and to have fun, we should give life counts to the fishes. So after receiving these suggestions, we removed the sound triggering part and added the life counts. They did have a better feedback from players we invited to play.

 FABRICATION AND PRODUCTION:

Some significant parts in our process is the following: The design of the elements in the game; Responds the computer give according to what the players(fishes) touch; The arrangement of the stages of the game. And I will introduce the difficulties we encountered in each part.

Firstly, the design of the elements. We have three elements in our game, the fishes(the players), the shrimps(fish can grow bigger by eating them), and the micro-plastic(fish will die if touching it). We designed these graphs by ourselves, we painted the shrimp and the fish using ps and gravit.

We coded the micro-plastic at first, however, after some office hours with LAs and Andy, they suggested that the coded plastic is a bit confusing, so we changed them into cartoon images of plastic bags and plastic bottles.

At first, we wanted there to be a settled number of plastic and shrimps, and we added a pollution layer like this 👇

that acts as count down by keep rising from the bottom of the screen and making the movement area smaller and smaller(the game can only be played in the blue area). However, after talking with Andy, we found that because of the density of micro-plastic varies, so they actually won’t all accumulate at the bottom of the sea. So we wanted to change the count down into the growing number of micro-plastic and the reduction of shrimp(Also to illustrate the idea that shrimps will die of micro-plastic as well and micro-plastic is increasing nowadays). However, we couldn’t successfully make the plastic increase according to time, which is a little pity in this project. But during this process we also learnt more about the micro-plastic and the fact that it’s spreading all over the sea. 

Secondly, the computer’s respond.

As was mentioned above, we added life counts for the fish. And other visual effects are 1, shrimps will disappear after the fish touches them 2, fish will disappear after touching the micro-plastic for three times. The first part was easy to code, but the second part was really challenging.

if (d4[i] < (5+size[i]/2) && touch1==false &&counter1 !=0) {//i
      touch1=true;
      counter1=counter1-1;
      lastCircle1 =i;
      sound1.play();
    }
    
    if (d2[lastCircle1]>(5+size[lastCircle1]/2)) {
      touch1 =false;
    }
    if (counter1 == 0 && fishalive1 == true) {
      x2=-200;
      y2=-200;
       heartx5=-100;
       hearty5=-100;
       photo4 = loadImage(“plose.jpeg”);
      sound2.play();
      fishalive1=false;
      break;
    }
    if (d4[lastCircle2]>(5+size[lastCircle2]/2)) {
      touch2 =false;
    }
    if (counter2 == 0 && fishalive2 == true) {
      x3=-200;
      y3=-200;
       heartx5=-100;
       hearty5=-100;
      sound2.play();
      fishalive2=false;
      break;
    }

The most difficult part was that the fish/the heart(stands for life counts) needs to stay disappear even if it no longer touches the micro-plastic, however the processing runs as a loop, so in order to do this, Winny and I tried Boolean. Also, the line “lastCircle1=I” and “d4[lastCircle2]” took us a long time to figure out. They both together enable the effect of the fish/the heart(stands for life counts) needs to stay disappear even if it no longer touches the micro-plastic,

Last but not least, the arrangement of the stages of the game. We had three stages: the first is the introduction. Following by the game, and then ending with a picture illustrating that as we eat fish, we eat plastic. At first we distinguished the three stages by the count of time, however, it didn’t work. Andy suggested the switch case to us, which, is life changing. This is very convenient and effective and make our code looks so logical as well.

Tracy and I did make a separate work sheet before we start, but basically we did everything together and helped each other the whole process.

CONCLUSIONS:

Our project wants to use the setting of the game, which is that the fish will die of micro-plastic, and eventually both fishes will die because there is too much of micro-plastic, to illustrate the idea that the micro-plastic we humans produced has caused great harm to the marine life and marine environment. Also, the final page of our project, the picture that shows a fish full of plastic placing on a plate, stating that those micro-plastic consumed by the fish will eventually be consumed by us, also shows that the harm we caused to the marine will eventually pay back on ourselves. We hope our project can arouse people’s awareness of reducing the production of trash and plastic, to protect the environment and to protect ourselves. I think our project did convey this idea, whether by the texts at the beginning and the picture at the end, or the game itself.

I think our project is interactive for it creates a communication platform: players controlling the fish, and computers giving responds accordingly. Friends we invited to play with our game did found that it’s a game that actually needs some strategy to play, for example, although eating shrimps can make the fish bigger, it makes it harder to avoid the plastic. Some friends would adopt the strategy of not eating any shrimps so that it can avoid the plastic more easily. However, if we have more time, we do want to make more improvement, for example, if the player doesn’t consume shrimp in a certain period of time, the fish will lose one life count, which is more related to the reality that fishes need food, and also makes the game a bit harder, that the players can’t avoid dying by not eating shrimps. Also, we didn’t manage to make the effect of micro-plastic increasing according to time, if we have enough time, maybe we can succeeded in making that effect come true and make the lose-lose situation more obvious. Also, according to some of our friends’ question like whether the game can be played by one person only, I think if we have more time, we can make the game more “professional” with a starting page, providing options of “single player” and “two players”, and by clicking on the specific option the player wants, they will play the game by themselves or with a partner.

I think the focus of our project is important. For it’s responding one of the most intense environmental issue nowadays. And people should care because the plastic pollution not only pollutes the marine environment as we shown in the game, but also the air, as we shown in the video, what’s more, it is affecting ourselves as well, by the air we breath and the fish we eat. It’s important to show this idea to everyone who gets to play our game because plastic pollution is accumulated by everyone’s daily life, one more person realizing this issue can contribute to some reduction of plastic pollution. Everyone counts and every realization matters. By this game, we hope to have more people realizing this issue and start with their everyday life, use plastic products properly and cut the plastic pollution gradually.

TECHNICAL DOCUMENTATION:

Code:

Arduino:

void setup(){
  Serial.begin(9600);
}
void loop(){
  int sensor1 = analogRead(A0);
  int sensor2 = analogRead(A1);
  int sensor3 = analogRead(A2);
  int sensor4 = analogRead(A3);
  int sensor5 =  digitalRead(9);
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.print(","); // add linefeed after sending the last sensor value
  Serial.print(sensor3);
  Serial.print(",");
 Serial.print(sensor4);
  Serial.println();
  delay(100);
}

Processing :
The beginning 1 page with rising texts:

import processing.sound.*;
int amount=20;
int count=20;

float startY;
float x[] = new float[count];
float yy;
float size[]= new float [count];
PImage photo1;
PImage photo2;
PImage photo3;

float[] Bx = new float[amount];
float[] By = new float[amount];
float[] Bsize = new float[amount];
float[] Bspeed = new float[amount];
SinOsc[] sine = new SinOsc[amount];
Env [] envelope = new Env [amount];
float [] freq = new float [amount];
int num=0;

long time;

void setup(){
  //size(800,800);
   
  fullScreen();
  background(0);
  photo1 = loadImage("shrimp.png");
  photo2 = loadImage("fish1.jpg");
  photo3 = loadImage("fish2.jpg");
   for (int i=0; i<amount; i++) {
    Bx[i] = random(width);
    By[i] = random(height, height * 2);
    Bsize[i] = random(10, 50);
    Bspeed[i] = map(size[i], 10, 50, 1, 5);
    sine[i] = new SinOsc (this);
    envelope[i] = new Env (this);
    freq[i] = map(Bsize[i], 10, 50, 500, 100);
  }
}


void draw(){
  
   switch(num) {
   case 0: 
   background(#3FA8C4);
    
    background(#3FA8C4);
    for (int i=0; i<amount; i++) {
      noStroke();
      fill(255, 120);
      circle(Bx[i], By[i], Bsize[i]);
      // Bubbles float to the top
      By[i] -= Bspeed[i];
      // if bubbles reaches the surface, create a new random bubble
      if (By[i] < 80+(Bsize[i]/2)) {
        Bx[i] = random(width);
        By[i] = random(height, height * 2);
        Bsize[i] = random(10, 50);
        Bspeed[i] = map(Bsize[i], 10, 20, 1, 2);
        sine[i].freq(freq[i]);
        envelope[i].play(sine[i], 0.0005, 0.05, 0.2, 1);
        freq[i] = map(Bsize[i], 10, 50, 500, 100);
        sine[i].play();
      }
  }
   startY=height;
 //startY=700;
 if (frameCount<10.0) {
    yy = startY - (frameCount * 2);
  }  
    startY=height-100;
   if ( frameCount  < 20.0) {
   
    yy = startY - ((frameCount-1) * 2);
  }  
   startY=height-200;
   if ( frameCount  < 30.0) {
   
    yy = startY - ((frameCount-2) * 2);
  }  
  startY=height-300;
   if ( frameCount  < 40.0) {
   
    yy = startY - ((frameCount-3) * 2);
  }  
   startY=height-400;
   if (frameCount  < 50.0) {
   
    yy = startY - ((frameCount-4) * 2);
  }  
   startY=height-500;
    if (frameCount  < 60.0) {
   
    yy = startY - ((frameCount-5) * 2);
  }  
  
 
  fill(255);
 textSize(30);
  text("Water popullution is a severe issue nowadays", 1/6*width, yy);
  
  textSize(30);
  text("Marine pollution is a combination of chemicals and trash", 1/7*width, yy+100);
  

  textSize(30);
  text("one of the 6 types of marine pollution is plastic pollution", 1/7*width, yy+200);
  
  textSize(30);
  text("which includes micro-plastic", 1/5*width,yy+300);
  break;
  
   }
}

 Following stages:

 

int num = 1;
import processing.serial.*;
import processing.sound.*;
SoundFile sound1;
SoundFile sound2;
SoundFile sound3;
SoundFile sound4;
int NUM_OF_VALUES_FROM_ARDUINO = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int sensorValues[];      /** this array stores values from Arduino **/
String myString = null;
Serial myPort;
int  count =10;
float x[] = new float[count];
float y[] = new float[count];
float u[] = new float[count];
float v[] = new float[count];
float  xspeed[] = new float[count];
float  yspeed[] = new float[count];
float  uspeed[] = new float[count];
float  vspeed[] = new float[count];
float size[] = new float[count];
float r[] = new float[count];
float g[] = new float[count];
float b[] = new float[count];
float d1;
float d2[]= new float[count];
float d3;
float d4[]=new float[count];
float pred2;
float pred4;
//float m=mouseX;
//float n=mouseY;
float x2, y2;
float x3, y3;
float heartx1=680;
float hearty1=25;
float heartx2=700;
float hearty2=25;
float heartx3=720;
float hearty3=25;
float heartx4=740;
float hearty4=25;
float heartx5=760;
float hearty5=25;
float heartX1=20;
float heartY1=25;
float heartX2=40;
float heartY2=25;
float heartX3=60;
float heartY3=25;
float heartX4=80;
float heartY4=25;
float heartX5=100;
float heartY5=25;
boolean touch1=true;
boolean fishalive1=true;
boolean touch2=true;
boolean fishalive2=true;
int counter1=5;
int counter2=5;
int lastCircle1;
int lastCircle2;
float fishSize=20;
color faceColor[] = new color[count];
PImage photo1;
PImage photo2;
PImage photo3;
PImage photo4;
void setup() {
  size(800, 600);
  //fullScreen();
  background(0);
  sound1 = new SoundFile(this, "broken-string-bounce.wav");
  sound2 = new SoundFile(this, "wah.mp3");
  sound3 = new SoundFile(this, "bite.wav");
  sound4 = new SoundFile(this, "lose.mp3");
  imageMode(CENTER);
  photo1 = loadImage("shrimp.png");
  photo2 = loadImage("fish1.jpg");
  photo3 = loadImage("fish4.png”");
  photo4 = loadImage("plose.jpeg");
  setupSerial();
  printArray(Serial.list());
  for ( int i=0; i < x.length; i++) {
    x[i] = random (100, width-100);
    y[i] = random (100, height-100);
    size[i]=random (50, 80);
    xspeed[i] = random (0.05, 0.1);
    yspeed[i] = random (0.05, 0.1);
  }
  for ( int i=0; i < u.length; i++) {
    u[i] = random (100, width-100);
    v[i] = random (100, height-100);
    uspeed[i] = random (0.05, 0.1);
    vspeed[i] = random (0.05, 0.1);
    r[i] = random(0, 255);
    g[i] = random(255);
    b[i] = random(255);
    faceColor[i] = color(r[i], g[i], b[i]);
  }
}//random size and speed for shrimps and the plastic, random color for the plastic
void draw() {
  getSerialData();
  // printArray(sensorValues);
  background(#3FA8C4);
  for ( int i=0; i < x.length; i++) {
    d1=dist(x[i], y[i], x2, y2);
    d2[i]=dist(u[i], v[i], x2, y2);
    d3=dist(x[i], y[i], x3, y3);
    d4[i]=dist(u[i], v[i], x3, y3);
    if (d1 < (2 + size[i]/2)) {
      fishSize+=10;
      sound3.play();
      x[i]=-100;
      y[i]=-100;
    }
    if (d3 < (2 + size[i]/2)) {
      fishSize+=10;
      sound3.play();
      x[i]=-100;
      y[i]=-100;
    }//the shrimp will disappear when the fish touches it, and the fish will grow bigger
    plastic(u[i], v[i], 10, faceColor[i]);
    if (d2[i] < (5+size[i]/2) && touch1==false &&counter1 !=0) {//i
      touch1=true;
      counter1=counter1-1;
      lastCircle1 =i;
      sound1.play();
      //heartx1=-100;
      //hearty1=-100;
    }
    if (d4[i] < (5+size[i]/2) && touch2==false &&counter2 !=0) {//i
      touch2=true;
      counter2=counter2-1;
      lastCircle2 =i;
      sound1.play();
    }
    //if(counter==4){
    //  heartx1=-100;
    //  hearty1=-100;
    //}
    //else if(counter==3){
    //  heartx2=-100;
    //  hearty2=-100;
    //}
    //else if(counter==2){
    //  heartx3=-100;
    //  hearty3=-100;
    //}
    //else if(counter==1){
    //  heartx4=-100;
    //  hearty4=-100;
    //}
    //else if(counter==0){
    //  heartx5=-100;
    //  hearty5=-100;
    //   fill (#DE4848);
    //heart(heartx1, hearty1, 0.3);
    //heart(heartx2, hearty2, 0.3);
    //heart(heartx3, hearty3, 0.3);
    //heart(heartx4, hearty4, 0.3);
    //heart(heartx5, hearty5, 0.3);
    if (d2[lastCircle1]>(5+size[lastCircle1]/2)) {
      touch1 =false;
    }
    //if (counter==0) {
    //if (counter == 0 && fishalive == true){
    //     sound2.play();
    //     break;
    //    }
    // fishalive=false;
    //sound2.play();
    //touch=true;
    //   }
    //  if (fishalive=false){
    if (counter1 == 0 && fishalive1 == true) {
      x2=-200;
      y2=-200;
      heartx5=-100;
      hearty5=-100;
      sound2.play();
      fishalive1=false;
      break;
    }
    if (d4[lastCircle2]>(5+size[lastCircle2]/2)) {
      touch2 =false;
    }
    if (counter2 == 0 && fishalive2 == true) {
      x3=-200;
      y3=-200;
      heartx5=-100;
      hearty5=-100;
      sound2.play();
      fishalive2=false;
      break;
    }
    if (counter1==0 && counter2 == 0) {
      switch( num ){
      case 1:
      size(800,600);
        photo4 = loadImage("plose.jpeg");
        break;
      }
    print(counter1);
   
    println(counter2);
    move();
    bounce();
    x2=sensorValues[0];
    y2=sensorValues[1];
    //  x3=sensorValues[2];
    // y3=sensorValues[3];
    x2=map(x2, 0, 1024, 0, width);
    y2=map(y2, 0, 1024, 0, height);
    x3=map(x3, 0, 1024, 0, width);
    y3=map(y3, 0, 1024, 0, height);
    if (fishalive1==true) {
      image(photo2, x2, y2, fishSize, fishSize);
    }
    if (fishalive2==true) {
      image(photo3, x3, y3, fishSize, fishSize);
    }
    image(photo1, x[i], y[i], 20, 20);
  }// the fish will disappear when touches the plastic five times.
  //pred2=d2[;
  if (counter1==4) {
    heartx1=-100;
    hearty1=-100;
  } else if (counter1==3) {
    heartx2=-100;
    hearty2=-100;
  } else if (counter1==2) {
    heartx3=-100;
    hearty3=-100;
  } else if (counter1==1) {
    heartx4=-100;
    hearty4=-100;
  }
  fill (#DE4848);
  heart(heartx1, hearty1, 0.3);
  heart(heartx2, hearty2, 0.3);
  heart(heartx3, hearty3, 0.3);
  heart(heartx4, hearty4, 0.3);
  heart(heartx5, hearty5, 0.3);
  //println(counter);
  if (counter2==4) {
    heartx1=-100;
    hearty1=-100;
  } else if (counter2==3) {
    heartx2=-100;
    hearty2=-100;
  } else if (counter2==2) {
    heartx3=-100;
    hearty3=-100;
  } else if (counter2==1) {
    heartx4=-100;
    hearty4=-100;
  }
  fill (#DE4848);
  heart(heartX1, heartY1, 0.3);
  heart(heartX2, heartY2, 0.3);
  heart(heartX3, heartY3, 0.3);
  heart(heartX4, heartY4, 0.3);
  heart(heartX5, heartY5, 0.3);
  //println(counter);
}
    }
void move() {
  for ( int i=0; i < x.length; i++) {
    x[i] += xspeed[i] ;
    y[i]  += yspeed[i] ;
  }
  for ( int i=0; i < u.length; i++) {
    u[i] += uspeed[i] ;
    v[i]  += vspeed[i] ;
  }
}
void bounce() {
  for ( int i=0; i < x.length; i++) {
    if ( x[i] <= 0 + size[i]/2 || x[i]>= width - size[i]/2-40) {
      xspeed[i] = -xspeed[i];
    }
    if ( y[i] <= 0+ size[i]/2 || y[i]>= height - size[i]/2-30 ) {
      yspeed[i] = -yspeed[i];
    }
  }
  for ( int i=0; i < x.length; i++) {
    if ( u[i] <= 0 + size[i]/2 || u[i]>= width - size[i]/2-40) {
      uspeed[i] = -uspeed[i];
    }
    if ( v[i] <= 0+ size[i]/2 || v[i]>= height - size[i]/2-30) {
      vspeed[i] = -vspeed[i];
    }
  }
}
void plastic(float u, float v, float size, color c) {
  fill(c);
  noStroke();
  circle(u, v, size);
  fill(c);
  noStroke();
  circle(u+10, v, size/2);
  fill(c);
  noStroke();
  circle(u-10, v-5, size-7);
  fill(c);
  noStroke();
  circle(u-10, v+4, size-8);
  fill(c);
  noStroke();
  circle(u-10, v+8, size-7);
}
void heart (float x, float y, float size) {
  pushMatrix();
  translate(x, y);
  scale(size);
  noStroke();
  beginShape();
  vertex(50, 15);
  bezierVertex(50, -5, 90, 5, 50, 40);
  vertex(50, 15);
  bezierVertex(50, -5, 10, 5, 50, 40);
  endShape();
  popMatrix();
}
void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 2 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port “/dev/cu.usbmodem----” or “/dev/tty.usbmodem----”
  // and replace PORT_INDEX above with the index number of the port.
  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = ‘\n’  Linefeed in ASCII
  myString = null;
  sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO];
}
void getSerialData() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Some videos we took during the process:

“🐟 can eat 🦐 and grow!!”

Struggles with shrimps and plastic:

More struggle:

Struggle with difficult controlling of the fish:

We’ve had a great time making this game, and I really learnt a lot during this semester. Although it’s a bit frustrating that we spent a big part of this class online, the class environment is still so good. I always look forward to interaction lab on Tuesdays and Thursdays, and being able to code some interesting things has been so stress-relieving. Every time I’m tired of writing essays or doing maths, I would always do a bit practicing of coding myself, which is so interesting and I had a lot of fun.
Thank you so much Andy for providing us so much help not only for the final but for the whole semester!! 🥰🎉

Recitation 10: Image & Video

During this recitation, we practiced the serial communication again and linked sound with image.

For the first exercise, surprisingly and not so surprisingly, the most difficult part turns out to be finding the picture haha. My nickname in middle school was Peppa pig, so I found a picture of it.

My code is below:

Arduino:

void setup()
{
  Serial.begin(9600);
}

void loop() 
{
  int sensor1 = analogRead(A0);
  int sensor2 = analogRead(A1);    
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.println(); // add linefeed after sending the last sensor value
  delay(100);
}

Processing:

import processing.serial.*;


int NUM_OF_VALUES_FROM_ARDUINO = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int sensorValues[];      /** this array stores values from Arduino **/

String myString = null;
Serial myPort;

PImage photo;


void setup() {
  size(800,800);
  background(#7BE5B1);
 
  photo = loadImage("Peppa pig.jpeg");
  printArray(Serial.list());
  setupSerial();
}

void draw() {
   background(#7BE5B1);
   getSerialData();
  printArray(sensorValues);
  
  float x= map(sensorValues[0],0,1023,0,width);
  float y= map(sensorValues[1],0,1023,0,height);
  
  image(photo, x, y);
  
  }
    
    
  void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[2], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO];
}

void getSerialData() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Video:

For the exercise2, it was quite tricky, and my processing got stuck thousands of times. I went to office hours to ask for helps, and Skye explains to me the function of abs(), R, PR and so on. I found the final result very interesting, the effect looks like the wall in swimming pool and the sound also somehow reminds me of that as well.

My code is below:

 

import processing.sound.*;
import processing.video.*;
SinOsc sine;
Env env;
String[] cameras = Capture.list();
Capture cam;
color myColor;
float attackTime = 0.001;
float sustainTime = 0.004;
float sustainLevel = 0.3;
float releaseTime = 0.4;
float d;
float f;
int s=20;
float R;
float PR;
void setup() {
  size(640, 480);
  cam = new Capture(this, cameras[1]);
  cam.start();
  printArray(cameras);
  // Create the sine oscillator.
  sine = new SinOsc(this);
  env  = new Env(this);
}
void draw() {
  background(0);
  if (cam.available()) {
    cam.read();
  }
  myColor=cam.get(width/2, height/2);
  R=red(myColor);
  f = map(R, 0, 255, 100, 600);
  float d = abs(R-PR);
  sustainTime = map(d, 0, 100, 0.001, 1);
  sustainTime = constrain(sustainTime, 0.001, 1);
  println(d);
  for (int x=0; x<width; x=x+s) {
    for (int y=0; y<width; y=y+s) {
      color c=cam.get(x, y);
      float red = red(c);
      float green= green(c);
      float blue = blue(c);
      float factor = map(red, 0, 255, 0.05, 2);
      noStroke();
      fill(red, green, blue, red);
      ellipse(x, y, s*factor, s*factor);
    }
  }
  // get the pixel color
  if (d>20) {
    sine.play();
    sine.freq(f);
    env.play(sine, attackTime, sustainTime, sustainLevel, releaseTime);
  }
  PR=R;
  //image(cam, 0, 0);
}

Video:

Recitation 9 – Sound in Processing

For this recitation, we use sound to do a lot of exercises. 

In the first exercise, we use keys to control the start of two sounds.

The code is below:

import processing.sound.*;

// declare three SoundFile objects
SoundFile kick;
SoundFile snare;


void setup() {
  size(640, 480);
  // create the objects and load sounds into them
  kick = new SoundFile(this, "kick.wav");
  snare = new SoundFile(this, "snare.wav");
 
}

void draw() {
  background(0);  
}

void keyPressed() {
  // when a key is pressed, use a switch statement
  // to determine which key was pressed
  // and play the appropriate sound file and set a background color
  switch(key) {
    case 'k':
      kick.play();
      background(255, 0, 0);
      break;
    case 's':
      snare.play();
      background(0, 255, 0);
      break;
    
  } 
}

 The video:

In the second exercise, we use the vibration motor to detect and show the beats of the song “Lost On The Free Way”.

The code is below:

For processing:

import processing.serial.*;


int NUM_OF_VALUES_FROM_ARDUINO = 1;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int sensorValues[];      /** this array stores values from Arduino **/

String myString = null;
Serial myPort;

import processing.sound.*;

// declare an AudioIn object
SoundFile sound;
// declare an Amplitude analysis object to detect the volume of sounds
Amplitude analysis;

void setup() {
  size(500, 500);
   sound= new SoundFile(this, "Lost On The Freeway.mp3");
   
   sound.play();
  // create the Amplitude analysis object
 
  // use the microphone as the input for the analysis
  analysis = new Amplitude(this);
  analysis.input(sound);

  setupSerial();
  
}

void draw() {
  background(0);
   println(analysis.analyze());
  background(0);
   float volume = analysis.analyze();
  // map the volume value to a useful scale
  float diameter = map(volume, 0, 1, 0, width);
  // draw a circle based on the microphone amplitude (volume)
  circle(width/2, height/2, diameter); 
  getSerialData();
 sensorValues[0]=int(map(analysis.analyze(),0,1,0,200));
}


void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list( )[6], 9600);

  myPort.clear();
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO];
}

void getSerialData() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Arduino code:

// IMA NYU Shanghai
// Interaction Lab

/**
  This example is to send multiple values from Processing to Arduino.
  You can find the Processing example file in the same folder which works with this Arduino file.
 **/
#define ZD 3
#define NUM_OF_VALUES_FROM_PROCESSING 1    /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/


/** DO NOT REMOVE THESE **/
int tempValue = 0;
int valueIndex = 0;

/* This is the array of values storing the data from Processing. */
int processing_values[NUM_OF_VALUES_FROM_PROCESSING];
#include <Servo.h>

Servo myservo;  // create servo object to control a servo

int potpin = A0;  // analog pin used to connect the potentiometer
int val;    

void setup() {
  Serial.begin(9600);
  pinMode(ZD,OUTPUT);
   
}

void loop() {
  getSerialData();
analogWrite(ZD,200);
delay(5000);
analogWrite(ZD,0);
delay(1000);
analogWrite(ZD,100);
delay(5000);
analogWrite(ZD,0);
delay(1000);
  // add your code here using elements in the values array

  //this is an example connecting a buzzer to pin 8
  /*
    if (processing_values[0] == 1) {
      //turn on an LED when the mouse is pressed
      digitalWrite(13, HIGH);
      // map values from mouseX to frequency from (0 - 500 pixels)
      //to the output pitch range (120 - 1500Hz)
      int f = map(processing_values[1], 0, 500, 120, 1500);
      // map values from mouseY to frequency from (0 - 500 pixels)
      //to the output duration range (10 - 2000 milliseconds)
      int d = map(processing_values[2], 0, 500, 10, 2000);
      // play the pitch:
      tone(8, processing_values[1], processing_values[2]);
      delay(1);        // delay in between reads for stability
    } else {
      digitalWrite(13, LOW);
    }
  */
  //end of example

}
//receive serial data from Processing
void getSerialData() {
  while (Serial.available()) {
    char c = Serial.read();
    //switch - case checks the value of the variable in the switch function
    //in this case, the char c, then runs one of the cases that fit the value of the variable
    //for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase
    switch (c) {
      //if the char c from Processing is a number between 0 and 9
      case '0'...'9':
        //save the value of char c to tempValue
        //but simultaneously rearrange the existing values saved in tempValue
        //for the digits received through char c to remain coherent
        //if this does not make sense and would like to know more, send an email to me!
        tempValue = tempValue * 10 + c - '0';
        break;
      //if the char c from Processing is a comma
      //indicating that the following values of char c is for the next element in the values array
      case ',':
       // processing_values[valueIndex] = tempValue;
        //reset tempValue value
        tempValue = 0;
        //increment valuesIndex by 1
        valueIndex++;
        break;
      //if the char c from Processing is character 'n'
      //which signals that it is the end of data
      case '\n':
        //save the tempValue
        //this will b the last element in the values array
       // processing_values[valueIndex] = tempValue;
        //reset tempValue and valueIndex values
        //to clear out the values array for the next round of readings from Processing
        tempValue = 0;
        valueIndex = 0;
        break;
    }
  }
}

Video:

The vibration motor is kind of scary hahaha, but the moving circle is very cool that gives a feel to the atmosphere and make the music even more popping and even interactive for it engages sights in the processes of enjoying music. It was a bit hard to use the map function for a proper extent, and it was also quite tricky when making the portfolio, for the music must be included in a portfolio called data with the processing sketch in the same portfolio.

In the homework part, we tried to use amplitude to draw something on Processing, I chose to draw a line that goes up and down accordingly.

The code:

 

import processing.sound.*;

// declare an AudioIn object
AudioIn microphone;
// declare an Amplitude analysis object to detect the volume of sounds
Amplitude analysis;

int x=0;
float prediameter;
 
void setup() {
   background(198,236,255);
  size(640, 480); 
  // create the AudioIn object and select the mic as the input stream
  microphone = new AudioIn(this, 0);
  // start the mic input without routing it to the speakers
  microphone.start();
  // create the Amplitude analysis object
  analysis = new Amplitude(this);
  // use the microphone as the input for the analysis
  analysis.input(microphone);
}      

void draw() {
  println(analysis.analyze());
 
  
  // analyze the audio for its volume level
  float volume = analysis.analyze();
  // map the volume value to a useful scale
  float diameter = map(volume, 0,1,  200, 0);
  // draw a circle based on the microphone amplitude (volume)
 
 
  x=x+1;
  strokeWeight(5);
  stroke(252,199,199);
  line(x, prediameter,x+1,diameter); 
  
  
  prediameter=diameter;
}

 Video:

It’s really fun to “see” your voice and to make different waves with it. However, it’s not so sensitive, I need to be very loud or very sharp can I make the line wave dramatically.

Essay

Our project is called Plastic Fish. It’s purpose is to convey the idea of protecting the marine environment by cutting down the amount of pollution people poured into the sea. According to Wekipedia, Marine pollution is a combination of chemicals and trash, most of which comes from land sources and is washed or blown into the ocean. This pollution results in damage to the environment, to the health of all organisms, and to economic structures worldwide. There are 6 types of marine pollution, they are pollution from marine debris, plastic pollution, including micro-plastics, ocean acidification, nutrient pollution, toxins and underwater noise.

 

We planned to build an interactive game involving two players. Each representing a fish. They can use sensors(potential meters/pressure sensors/sound/movement)to control the movement of the fish. The processing will have a background of blue ocean with  micro plastic(or actual plastic products) and fish food(maybe like little shrimps) bouncing around the whole page. For the micro plastic, it will start with one, and then double once 20 seconds. And the two players will need to avoid the micro plastic. By eating the little shrimps, they will grow bigger, and the player that is bigger at the end of the game will “win” the game. However, I add these quotation marks because that actually, there will be no winner in the game. We will also design a rising “pollution layer” that rises from the bottom of the sea and will reach 7/8 of the pages at the end. The fishes can only swim above this layer, so eventually the two players will find themselves stuck on the top. With the limited space and continuously doubling plastics, there’s no way that the fishes can still avoid touching them. So these two fishes will both die, no matter how much they struggle to survive. 

In the previous reading of 《Newton’s Sleep》, it describes a world that is “prototype of the Special Earth Satellite itself” because the earth is already over-polluted. This also stimulates us to develop the idea of reminding people we can’t consume the earth too wildly, it’s important that we appreciate all we’ve got and limit our pollution and harm to the earth. After all, what we’ve done to the earth will result on ourselves eventually, for the earth’s good, our own good and our latter generation’s good, we should be cautious of our actions now.

Recitation 8: Serial Communication

For this recitation, we did many exercises to get familiar with the connection between Arduino and processing.

Exercise 1:

The first exercise is using Arduino to send two analog values to Processing via serial communication. I find this one the most confident one for me to do. 
Below is the Arduino code:

void setup() {
  Serial.begin(9600);
}

void loop() {
  int sensor1 = analogRead(A0);
  int sensor2 = analogRead(A1);
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.println();
  
}

Processing:

import processing.serial.*;
float x;
float y;

int NUM_OF_VALUES_FROM_ARDUINO = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int sensorValues[];      /** this array stores values from Arduino **/
int sensorp[];
String myString = null;
Serial myPort;

void setup() {
  size(500, 500);
  setupSerial();
  sensorp = new int[NUM_OF_VALUES_FROM_ARDUINO];
  background(0);
}

void draw() {
  getSerialData();
  printArray(sensorValues);  
  x = map(sensorValues[0],0,1023,0,width);
  y = map(sensorValues[1],0,1023,0,width);

  stroke(255);
  line(sensorp[0], sensorp[1],x, y);
  sensorp[0]=int(x);
  sensorp[1]=int(y);
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[2], 9600);
  
  myPort.clear();
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO];
  
}
void getSerialData() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Video: 

Exercise 2:

For this exercise, I found two codes for the bouncing circle. The first one is like this:

/**
 * Bounce. 
 * 
 * When the shape hits the edge of the window, it reverses its direction. 
 */
 
int rad = 60;        // Width of the shape
float xpos, ypos;    // Starting position of shape    

float xspeed = 2.8;  // Speed of the shape
float yspeed = 2.2;  // Speed of the shape

int xdirection = 1;  // Left or Right
int ydirection = 1;  // Top to Bottom


void setup() 
{
  fullScreen();
  noStroke();
  frameRate(30);
  ellipseMode(RADIUS);
  // Set the starting position of the shape
  xpos = width/2;
  ypos = height/2;
}

void draw() 
{
  background(102);
  
  // Update the position of the shape
  xpos = xpos + ( xspeed * xdirection );
  ypos = ypos + ( yspeed * ydirection );
  
  // Test to see if the shape exceeds the boundaries of the screen
  // If it does, reverse its direction by multiplying by -1
  if (xpos > width-rad || xpos < rad) {
    xdirection *= -1;
  }
 
    ydirection *= 0;
  

  // Draw the shape
  ellipse(xpos, ypos, rad, rad);
}

However, I found it really hard to use this code to communicate with processing, I tried several times but don’t know how to use “if statement” in this case. And later Tracy and I figured another way to code the bouncing circle. My servo isn’t working, so I did the coding part with Tracy and she filmed how the servo interacts with the bouncing circle.

Arduino code:

#define NUM_OF_VALUES_FROM_PROCESSING 1    /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
#include <Servo.h>
/** DO NOT REMOVE THESE **/
int tempValue = 0;
int valueIndex = 0;
bool prevState = false;
bool currentState = false;
Servo myservo;
int processing_values[NUM_OF_VALUES_FROM_PROCESSING];
void setup() {
  Serial.begin(9600);
  pinMode(12, OUTPUT);
  myservo.attach(10);
}
void loop() {
  getSerialData();
  currentState = processing_values[0];
 
  if (currentState != prevState && currentState == 1) {
    myservo.write(90);
   
    myservo.write(0);
  } else {
    myservo.write(0);
  }
  prevState = currentState;
}
//receive serial data from Processing
void getSerialData() {
  while (Serial.available()) {
    char c = Serial.read();
    switch (c) {
      case '0'...'9':
        tempValue = tempValue * 10 + c - '0';
        break;
      case ',':
        processing_values[valueIndex] = tempValue;
        tempValue = 0;
        valueIndex++;
        break;
      case '\n':
        processing_values[valueIndex] = tempValue;
        tempValue = 0;
        valueIndex = 0;
        break;
    }
  }
}

Processing:

import processing.serial.*;
int NUM_OF_VALUES_FROM_PROCESSING = 1;  /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int processing_values[] = new int[NUM_OF_VALUES_FROM_PROCESSING]; /** this array stores values you might want to send to Arduino **/
Serial myPort;
String myString;
int x=30;
int speedX = 5;
void setup() {
  fullScreen();
  background(0);
  setupSerial();
}
void draw() {
  background(0);
  circle (x, 100, 100);
  x= x+speedX;
  if (x > width-40 || x < 0+30) {
    speedX = -speedX;
  }
  if (x > width-30) {
    processing_values[0] = 1;
  } else {
    processing_values[0] = 0;
  }
  sendSerialData();
}
void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[2], 9600);
  myPort.clear();
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
}
void sendSerialData() {
  String data = "";
  for (int i=0; i<processing_values.length; i++) {
    data += processing_values[i];
    //if i is less than the index number of the last element in the values array
    if (i < processing_values.length-1) {
      data += ","; // add splitter character "," between each values element
    }
    else {
      data += "\n"; // add the end of data character linefeed "\n"
    }
  }
  //write to Arduino
  myPort.write(data);
  print(data); // this prints to the console the values going to arduino
}

Video:

Additional homework:

I found the communication quite confusing for there are many sections and it’s sometimes even impossible to figure out which part goes wrong. So I participated in the workshop and learnt more about the whole logic behind this.

Arduino code:

// IMA NYU Shanghai
// Interaction Lab
// For recitation 8 homework, controlling the display of two stars with two buttons


void setup() {
  Serial.begin(9600);
}

void loop() {
  int button1 = digitalRead(9);
  int button2 = digitalRead(10);

  // keep this format
  Serial.print(button1);
  Serial.print(",");  // put comma between values
  Serial.print(button2);
  Serial.println(); // add linefeed after sending the last value

  // too fast communication might cause some latency in Processing
  // this delay resolves the issue.
  delay(100);
}

Processing:

// IMA NYU Shanghai
// Interaction Lab
// For recitation 8 homework, controlling the display of two stars with two buttons

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] buttonValues;      /** this array stores values from Arduino **/
int prevButtonValue0 = 0;
int prevButtonValue1 = 0;
boolean star1Display = false;
boolean star2Display = false;


void setup() {
  size(600, 600);
  background(0);
  setupSerial();
}


void draw() {
  updateSerial();
  printArray(buttonValues);

  background(0);

  if (buttonValues[0]==1 && buttonValues[0]!= prevButtonValue0) {
    star1Display = !star1Display;
  }
  if (buttonValues[1]==1 && buttonValues[1]!= prevButtonValue1) {
    star2Display = !star2Display;
  }

  if (star1Display) {
    pushMatrix();
    translate(width*0.3, height*0.3);
    rotate(frameCount / 200.0);
    star(0, 0, 30, 70, 5); 
    popMatrix();
  }

  if (star2Display) {
    pushMatrix();
    translate(width*0.7, height*0.7);
    rotate(frameCount / 400.0);
    star(0, 0, 80, 100, 40); 
    popMatrix();
  }

  prevButtonValue0 = buttonValues[0];
  prevButtonValue1 = buttonValues[1];
}



void star(float x, float y, float radius1, float radius2, int npoints) {
  float angle = TWO_PI / npoints;
  float halfAngle = angle/2.0;
  beginShape();
  for (float a = 0; a < TWO_PI; a += angle) {
    float sx = x + cos(a) * radius2;
    float sy = y + sin(a) * radius2;
    vertex(sx, sy);
    sx = x + cos(a+halfAngle) * radius1;
    sy = y + sin(a+halfAngle) * radius1;
    vertex(sx, sy);
  }
  endShape(CLOSE);
}




void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 6 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  buttonValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          buttonValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Video:

  

Research:

 For the coming classes we will be learning how to manipulate image, video and sound. I’m really excited about this, I think it can add a lot more interesting effects to the interaction. It also is so relevant to the final project as well. 

For our project, one of our ideas is to visualize the sound. Since originally, sound is a wave that comes from vibration. If it’s impossible for people with hearing problems to hear, it would be great for them to see the wave and feel the vibration, which are the two parts that builds the sound. By research we found that it’s possible to use processing to code wave or random shapes for different pitches, which I find really interesting.

Also, by searching for “processing image” on Google, I found that Processing currently works with GIF, JPEG, and PNG images. So it’s also possible to add actual pictures to the final result, which can also be very useful. I believe that using a GIF can have very funny effect.

Final Project Proposal

1, Circles

This project is designed for kids in kindergarten. It aims at teaching kids the three-primary colors, as well as how to properly compete and cooperate with one another. We were inspired by both the exercise we did during the class, the one that has a circle bouncing around the screen; and the project mood reflecting floor published by the “visualcraft” also gives me the idea of blending colors of each circle. We think the challenges may be that the switch of two mode as well as the connection of multiple sensors.

2, Etch-B-sketch

This is designed for people in all age and train their drawing skills as well as memory. This is inspired by the project “Etch-A- Sketch”, and it adopt the similar way of drawing like Etch_A_Sketch, switching the potentiometers, the processing will show a graph for 10s, and then the player will need to draw it. The challenge we think will be how to make the definition of a successful drawing and an unsuccessful one. 

3, Seeound

The specific audience is people with hearing impairments. This blog gave me the inspiration of this design choice that we can enable those special populations to “see” the sound. We decided to use processing to create a image that can change according to music pitch. And some LEDs can light according to the rhythm. It can help people with hearing impairments to see the shake of sound and how can we hear the sound, how does sound compose. When they use this project, it helps them learn how to live more vibrantly. It is the way they fuel their body, mind and spirit.

PREPARATORY RESEARCH AND ANALYSIS

I found two interaction projects that inspire me on YouTube. One is the solar tracker designed by Robot Greek in the video Top 10 Arduino projects all the time । Amazing Arduino school projects genius youtuber” . The other is the mood reflecting floor published by the “visualcraft”.  

For the first one, I think it’s successful because it can interact with the environment. While according to the “Art, Interaction and Engagement”, it mentions “infant behavior”, which is that infants will react according to the environment, I think this project can be viewed of a manifestation of the theory: the machine turn according to the light. From this project, it inspires me that, this project is very much like a sunflower, and maybe I can design a sunflower according to this idea but add more interaction with human, and make this interaction, continuous, according to the “Art, Interaction and Engagement”. And for the latter one, it provides a really good visual experience to me, and it’s also pretty entertaining and can interact with people for quite a long time. Also, I think I also reminds me of the interactive floor I saw in a hotel that there are many fishes projected on the floor, and they will “swim” away if you step on them. This inspires me that maybe I can also design my project with items that will change position or forms according to human movements.

I think a successful interactive experience should not only be containing the machine and people, as I said before in my definition of interaction. What’s more important in an interactive experience is that it should inspire the user to interact more, rather than just a one-round interaction. According to “Art, Interaction and Engagement”,  “the interaction [should be] continuous and fluid”. which, I think is also what I can improve for my midterm project as well. And also, the interactive experience should have a deeper intention. Just like what we did in our group project, by reading “The Winter Market” by William Gibson, we see the insight of how technology has negative influence on people, and integrated that in our design. So it is that we designed our midterm project to help people with eyesight disability.

 

Reference:

  • E. Edmonds, “Art, Interaction and Engagement,” 2011 15th International Conference on Information Visualisation, 2011, pp. 451-456, doi: 10.1109/IV.2011.73.
  • TGibson, W. (2000). The winter market. Burning chrome.

Recitation 7: Functions and Arrays

For this recitation, we tried to do more practices with Processing. It’s also a really good revision for Thursday’s lecture. By using parts of the codes we did during the lecture individually, I get to understand the codes more.

I first created a easy one based on what Professor Andy taught us during the lecture. I used the code and simply changed the graph and background. This graph and the color of the background were inspired by a nail art that I really like recently, but sadly due to the lockdown I’m not able to do the nails, I can only achieve the nails in this way😭

This is the nail art:

This is my code:

 

int count=10;

float x[] = new float[count];
float y[] = new float[count];
float size[] = new float[count];

float r[] = new float[count];
float g[] = new float[count];
float b[] = new float[count];
color flowerColor[] = new color[count];


float xspeed[] = new float[count];
float yspeed[] = new float[count];

void setup(){
  size(800,800);
  background(252,251,240);

 
 for ( int i=0; i < x.length; i++) {
   
    x[i] = random (100, width-100);
    y[i] = random (100, height-100);
    size[i]=random (50,80);


    r[i] = random(50,255);
    g[i] = random(100,255);
    b[i] = random(100,255);
  
    flowerColor[i] = color(r[i], g[i], b[i]);
     xspeed[i] = random (2, 5);
     yspeed[i] = random (2, 5);
 }
}
    void draw() {
      background(252,251,240);
      
    for ( int i=0; i < x.length; i++) {
    Flower(x[i], y[i], size[i], flowerColor[i]);
  }   
  move();
  bounce();
    }
    
    void move() {

  for ( int i=0; i < x.length; i++) {
    x[i] += xspeed[i] ;
    y[i]  += yspeed[i] ;
  }
}
    void bounce() {

  for ( int i=0; i < x.length; i++) {
    if ( x[i] <= 0 + size[i]/2 || x[i]>= width - size[i]/2 ) {
      xspeed[i] = -xspeed[i];
    }

    if ( y[i] <= 0+ size[i]/2 || y[i]>= height - size[i]/2 ) {
      yspeed[i] = -yspeed[i];
    }
  }
}

    void Flower(float x, float y, float size, color c) { 
 fill(0);
 ellipse(x, y, size, size);

  fill(0);
 ellipse(x+size/2, y-size/2, size, size);
 
 fill(0);
 ellipse(x+size, y, size, size);

 fill(0);
 ellipse(x+size/2-size/3, y+size/2, size, size);

 fill(0);
 ellipse(x+size/2+size/3, y+size/2, size, size);

 fill(c);
 ellipse(x+size/2, y+5, size, size);

  }

This is how it looks like:

However, I wanted to challenge myself more, so I wanted to make one with a heart shape and are listed in a grid with blinking colors. This was pretty hard. I first came up with something like this:

Though it was quite pretty, it wasn’t what I wanted. Then I wanted to study how exactly does the codes for a heart shape works and how it was composited. I reached out to Professor Andy for help and learnt how to code for a heart as well as how to control its size and position, and I also learnt how to make it blink.

After that, I attended the workshop about functions and arrays and learnt how to list the hearts in a grid.

This is my result:

 

int count=13;
int number =10;
float[] xPositions =new float[number];
float[] yPositions =new float[number];

float x[] = new float[count];

color random1;
color random2;
int delay = 1000;// ONE SEC
int now; 
//a flag
boolean red = false;


void setup(){
  size(800,800);
  background (203,253,255);
  
   for ( int i=0; i < x.length; i++) {
   
    x[i] = random (100, width-100);
    
    now = millis();
 random1 = color(random(255),random(255),random(255));
 random2 = color(random(255),random(255),random(255));
    
}
  for(int i=0; i<number; i++){
    for (int j=0; j<number;j++){
  xPositions[i]=i*100-150;
  yPositions[j]=j*100-90;
    }
  }
}

void draw() {
  background (203,253,255);

for(int i=0;i<number;i++){
    for (int j=0;j<number;j++){
  pushMatrix();
  translate(xPositions[i],yPositions[j]);
if (millis() - now > delay) { 

    //change flag
    red = !red;

    //reset counter
    now = millis();
  }


  if (red)
    fill(random1);
  else
    fill(random2);
  heart( 100, 100, 2);
  popMatrix();
 
    }
}
  

    }

void heart (float x, float y, float size) {

  translate(x, y);
  scale(size);
  
  noStroke();
  beginShape();
  vertex(50, 15);
  bezierVertex(50, -5, 90, 5, 50, 40);
  vertex(50, 15);
  bezierVertex(50, -5, 10, 5, 50, 40);
  endShape();
}

  • Q1: In the reading “Art, Interaction and Engagement” by Ernest Edmonds, he identifies four situations in an interactive artwork: ‘Static’, ‘Dynamic-Passive’, ‘Dynamic-Interactive’ and ‘Dynamic-Interactive(Varying)’. From the exercise you did today which situations you identify in every part you executed? Explain.

Static: The part that I design the graph. After I code for the graph and click play, I can see the graph, but it doesn’t change and it doesn’t interact with me.

Dynamic-Passive: It might be like the bouncing and moving movements I had in my first try. Here, the internal mechanism is the code.

Dynamic-Interactive:  I think it’s the part that we assigned random color to the graph. Because every time we click play, the color will be shown randomly. It’s a kind of interaction between the viewer and we do have “an active role in influencing the changes in the art project”(3).

Dynamic-Interactive(Varying): Because that the color is randomly assigned, it has the sense of unpredictability in it. Also, in my blinking hearts, the art object changes color as well. So I think it fits in the varying dynamic-interactive concept.

  • Q2: What is the benefit of using arrays? How might you use arrays in a potential project?

In my case, I think listing the graphs in a grid is really tidy and visually comforting. It also reminds me of the game of whack-a-mole, maybe I can design a similar game in a potential project with some of the graphs in the grid blinking and others having other movements. The grid gives a sense of unity and solidity despite the fact that there are a lot of changes and therefore provides balance to the project as a whole.