Category: Communications Lab

Band Art – Sophia Wan and Stephanie Yip – Eric Parren

 

Our project band art utilizes the audience’s engagement and creativity to make both sounds and art that the user enjoys. We made our instruments/project self explanatory as most people already know of or have played drums and/or piano before. This way the users most likely know how to play our game without us having to explain it to them and have them test it out themselves as people did during user-testing. Each of the three drums make a different sound as well as each of the seven keys on our piano, this way, the user can remember what the sounds sound like and play something to their liking. Each instrument also corresponds to a different image displayed on the screen so the user can make art to their liking as well. During user testing, everything worked according to plan but some suggestions we got were to make the keys of our piano more sturdy and able to spring back better. Also, since we laser cutted our drums, there was a wood sound when the drumsticks hit the surface of the drums so one idea was to get rid of that. Another was to make the project so that the user can retrieve an image of their artwork after they are done playing. After user-testing, my partner and I worked on making the keys of the piano better, adding rubber bands to the tops of the drumstick to make them sound better and adding a saveImage() feature to our code so that after every minute, an image of the art is saved and can be accessed by the user.

The first thing we struggled with was the basis of our piano and our drums. Since we created the outline for our laser-cutted instruments on our own, we had to make sure that each piece would fit into each other given the kerf. After building our instruments, we struggled first with the drums. Since we were using knock sensors, at times the sensor would be too sensitive to the vibrations such as if there was a knock on the table. To solve this problem, we added a threshold to the sensorValues in Arduino. After we completed the drums, we had to think of a way to create our piano. Our first idea was to use buttons but that was difficult to go-through with since the keys were too wide for the breadboard and didn’t have steady footing since there would have to be wires underneath. Then we created our own makeshift button by soldering wires to copper paper to conduct electricity. This way each time one end of the key touches the bottom of the keyboard, it will send a value to the arduino and play a sound through processing. We wanted to make our keys be able to bounce back like normal piano keys do but we weren’t sure how to go about it. We first tried placing foam tape underneath but it didn’t bounce back like we wanted it to. Then we tried to hot glue springs underneath each key. This worked but it was a little difficult to push down each button and have them return to the same spot. After user testing, we decided to 3-D print tiny cylinders to put on the back of each key and stick a metal rod through all of the keys to make them stay at the same position and stay on the surface of each spring. After finally finishing the structure of our instruments, we then worked on the code. For the drums, each hit would put a splatter paint at a random position and color on the screen. If you hit above a certain sensorValue, you will trigger either a circle or a triangle to appear on the screen. Then for the piano, we place music notes that stay and move each time you play a key. 

I think overall our project succeeded in doing what we intended it to do. The user can play around with sound and art and their imagination to the best of their ability without having any prior knowledge to do so. It also aligned with my definition of interaction because it required the audience to be engaged with our project and try to learn about it as they played. It also is satisfying for the musically and visually to use. If I had more time, maybe I would make the piano keys work even better and get the right springs for them this time as well as adding more code for each key to do something different. Overall, this project taught me a lot about using sensors, laser-cutting, 3-D printing and mainly coding from the arduino to processing, adding sound and adding various images.

 

1. APPENDIX

Processing Code:

import processing.serial.*;

import processing.sound.*;

ArrayList<MusicNote> move = new ArrayList<MusicNote>();

Splat splatArray[] = new Splat[0];

Splat splatter;

MusicNote musicnote;

int oldValue = 0;

int m;

Serial serialPort; // Serial port

int NUM_OF_VALUES_FROM_ARDUINO = 10;

int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO];

int old_arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO];

SoundFile sounds[] = new SoundFile[10]; // Array to store sound files

boolean isPressed = false;

void setup() {

  size(1500, 800);

  // Load sound files

  sounds[0] = new SoundFile(this, “Drum sound 1.wav”);

  sounds[1] = new SoundFile(this, “Drum 2.wav”);

  sounds[2] = new SoundFile(this, “Drum 3.wav”);

  sounds[3] = new SoundFile(this, “do.wav”);

  sounds[4] = new SoundFile(this, “re.wav”);

  sounds[5] = new SoundFile(this, “mi.wav”);

  sounds[6] = new SoundFile(this, “fa.wav”);

  sounds[7] = new SoundFile(this, “so.wav”);

  sounds[8] = new SoundFile(this, “la.wav”);

  sounds[9] = new SoundFile(this, “ti.wav”);

  smooth();

  //splatter = new Splat();

  musicnote = new MusicNote();

  serialPort = new Serial(this, “/dev/cu.usbmodem11201”, 9600);

  m = millis();

}

void draw() {

  getSerialData();

  background(255);

  musicnote.move();

  if (splatArray.length >0) {

    for (int k = 0; k<splatArray.length; k++) {

      splatArray[k].display();

    }

  }

  

  // Draw drum pads

  //background(255);

  if (millis() >= m + 60000) {

    saveFrame(“arts-######.png”);

    m = millis();

  }

  //if (arduino_values[0] == 1 && arduino_values[0] != oldValue) {

  //  int = random(

  //   sounds[i].play();

  //   oldValue = arduino_values[0];

  //   splatter.splats();

  //}

  for (int i=0; i<NUM_OF_VALUES_FROM_ARDUINO; i++) {

    if (arduino_values[i] == 1 && arduino_values[i] != old_arduino_values[i]) {

      sounds[i].play();

      if (i < 3) {

        strokeWeight(0);

        //splatter.splats();

        float tempX = random(50, 1950);

        float tempY = random(50, 950);

        float tempR = random(0,255);

        float tempG = random(0,255);

        float tempB = random(0,255);

        for ( float j =3; j < 90; j++) {

          splatArray = (Splat[])append(splatArray, new Splat(tempX + cos(random(0, TWO_PI))*2*j,tempY+ sin(random(0, TWO_PI))*3*j,random(0, 50),tempR, tempG, tempB));

        //noLoop();

      }

      } else if (i > 2) {

        isPressed = true;

        musicnote.Notes();

      }

      //} else {

      //  sounds[i].stop();

      println(“Got 1”);

    } 

    else if (arduino_values[i] == 2 && arduino_values[i] != old_arduino_values[i]) {

      if (i < 3) {

       sounds[i].play();

      float x = random(width);

      float y = random(height);

      noFill();

      stroke(0);

      strokeWeight(random(5, 30));

      circle(random(x), random(y), random(80, 150));

      triangle(random(x), random(y), random(x+10), random(y+10), random(x+20), random(y+20));

      }

      println(“Got 2”);

    }

      

      old_arduino_values[i] = arduino_values[i];

      if (isPressed) {

        for (int j = 0; j < 10; j++) {

          move.add(new MusicNote());

         

        }

        println(“update”);

        for (int j = 0; j < 10; j++) {

          //move.get(j).display();

          move.get(j).move();

        }

        isPressed = false;

      }

    }

    //old_arduino_values[i] = arduino_values[i];

    //if (isPressed){

    //  for (int j = 0; j < 10; j++){

    //      move.get(j).display();

    //      move.get(j).move();

    //   }

    //   isPressed = false;

    //}

  }

  class Splat {

    float x;

    float y;

    float rad;

    float R;

    float G;

    float B;

    Splat(float tx,float ty,float trad,float tr,float tg,float tb) {

      x = tx;

      y = ty;

      rad = trad;

      R = tr;

      G = tg;

      B = tb;

      //x = random(50, 1950);

      //y = random(50, 950);

      //rad = random(0, 20);

      //R = random(0, 255);

      //G = random(0, 255);

      //B = random(0, 255);

      fill(R, G, B);

      noStroke();

      ellipse(x, y, rad, rad);

      //ellipse(x, y, rad, rad);

    }

    //void splats() {

      //x = random(50, 1450);

      //y = random(50, 750);

      //R = random(0, 255);

      //G = random(0, 255);

      //B = random(0, 255);

      //fill(R, G, B);

      //for ( float i =3; i < 90; i++) {

      //  float angle = random(0, TWO_PI);

      //  float splatX = x + cos(angle)*2*i;

      //  float splatY = y + sin(angle)*3*i;

      //  ellipse(splatX, splatY, rad-i, rad-i+1.8);

      //  //noLoop();

      //}

      

      void display() {

        fill(R,G,B);

        noStroke();

        ellipse(x,y,rad,rad);

      

      }

  }

  class MusicNote {

  //  float xPos;

  //  float yPos;

  //  float oldxPos;

  //  float oldyPos;

  //  float ySpd;

  //  String[] sample = {“♪”, “♫”}; // array of musical symbols

  //  int n;

  //  String note;

  // float wordSize;

  //  MusicNote() {

  //    xPos = random(width);

  //    yPos = random(height);

  //    oldxPos = xPos;

  //    oldyPos = yPos;

  //    ySpd = 2;

  //    n = int(random(0, 1));

  //    note = sample[n];

  //    wordSize = random(30, 150);

  //  }

  //  void move() {

  //    if (yPos < height) {

  //      // yPos += ySpd;

  //      yPos = yPos+random(-100, 100);

  //    }

  //    xPos = xPos + random(-100, 100);

  //    println(“moving”);

  //  }

    //void display() {

    //  //background(255);

    //  fill(0);

    //  textSize(wordSize);

    //  text(note, xPos, yPos);

    //}

  

  float xs[] = new float[10]; // int[] numbers = new int[3]

  float ys[] = new float[10];

  String[] sample = {“♪”, “♫”}; // array of musical symbols

  String[] notes =new String[10];

 MusicNote(){

    for(int i=0; i<xs.length; i++){

      int n = int(random(0,1));

      notes[i] = sample[n];

    }

     for (int i = 0; i < 10; i++) {

      xs[i] = random(width);

      ys[i] = random(height);

    }

  }

  void move() {

    textSize(random(30, 100));

    fill(0);

    for (int i = 0; i < 10; i =i+1) {

      text(notes[i], xs[i], ys[i]); //todo: notes index

      ys[i] = ys[i] + 2;

      if (ys[i] > 800) {

        ys[i] = 0;

      }

    }

  }

  void Notes() {

    move();

    int index = (int) random(0, 1);

    text(notes[index], xs[0], ys[0]);

  }

  }

  void getSerialData() {

    while (serialPort.available() > 0) {

      String in = serialPort.readStringUntil( 10 );  // 10 = ‘\n’  Linefeed in ASCII

      if (in != null) {

        //print(“From Arduino: ” + in);

        String[] serialInArray = split(trim(in), “,”);

        if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {

          for (int i=0; i<serialInArray.length; i++) {

            arduino_values[i] = int(serialInArray[i]);

          }

        }

      }

    }

  }

 

Arduino Code:

const int sensorPin0 = A0;  // Piezo vibration sensor connected to analog pin A0

const int sensorPin1 = A1; 

const int sensorPin2 = A2;

void setup() {

  Serial.begin(9600);  // Start serial communication

}

 

void loop() {

  int sensorValue1 = analogRead(sensorPin0);

  int sensorValue2 = analogRead(sensorPin1);

  int sensorValue3 = analogRead(sensorPin2);  // Read the sensor value

  //delay(100);                               // Delay for stability

 

  if (sensorValue1 > 5) {

    Serial.print(1);

  } else {

    Serial.print(0);

  }

  Serial.print(“,”);

  if (sensorValue2 > 5) {

    Serial.print(1);

  } else {

    Serial.print(0);

  }

  Serial.print(“,”);

  if (sensorValue3 > 5) {

    Serial.print(1);

  } else {

    Serial.print(0);

  }

  Serial.println();

  delay(10);

}

 

Below is a link for all the photos and videos documented in the process.

https://drive.google.com/drive/folders/1ZXZMz2v9v2aSFbjXWsre3Gf1JIdmYg8t 

 

What inspired the design of our project was Interaction Lab. We used components and mechanisms with different types of motors that were previously introduced in class and incorporated the designs into our prototype. For example, we used stepper motors, 130W motor and micro servo motors in each of our levels. Our targeted audience was mainly fellow students as they are more likely to understand how to complete the levels based on what we have done in class. We used the stepper motor mechanism we did during recitation to help move the ball along to the next level as well as the button game we played in class where we had to hold a button for a certain amount of seconds and the servo we used to build a meter that moved 180 degrees. I think our project is unique and uses a good amount of interaction because the audience has to complete the problems given in order to reach the end. However, since our project is more geared towards students in interaction class and have a good understanding of how the arduino and code works, it was harder to complete for those who didn’t understand the concepts taught in class. 

As previously mentioned, we used various types of motors for our project. When user testing, one of our components did not work as we would’ve liked (the fan motor) so we had to replace it and come up with another way to move the ball around. The fan was not powerful enough and didn’t move the ball no matter how much power we supplied. We assumed that it would have been easy for the user to build the circuit and fix the code for the first two levels of our game but it was proven to be difficult during user testing as they did not know what to fix. One solution was to provide a hint in the instructions telling them to look at the Arduino and the beginning part of the code for help. Other than that, the decisions we made worked pretty well as the stepper motor and the button game worked the way we wanted them to. 

During our process, coming up with the idea was not as hard as implementing it. What we struggled with the most was getting the fan to work and figuring out how to code the mechanisms. We had to ask teachers and learning assistants to help for these parts. For instance, there were moments when our motors weren’t working properly such as the servo motor since it had too much current flowing through so we needed an external power source. Another instance was since our fan wasn’t powerful enough, we tried connecting it to a 12V power supply but for only a few seconds and it still didn’t work the way we wanted so we scratched that mechanism and used the motor to make a spoon pusher that rotates in a circle to push the ball through a hole. During user testing, people found the game challenging because of the code and the button game had to be within .5 seconds of the intended “guessed” number to work so we increased the interval to a whole second and added hints. However it was still challenging to complete so I believe next time, to make the game more suited for a larger group of people beyond Interaction Lab, to make the challenges riddles easier to solve. 

I think our project had a good level of interaction involved and got the audience to be working to move the ball along. Our project correlates with my definition of interaction since the audience had equal participation in each level with each mechanism in order to play the game. I think in the future, I would change up the games to make it more as a riddle than a question about an interaction lab to open up the targeted audience range. Overall, I think our game, although challenging, received good feedback in that it would be a fun game to play in recitation and it was fun overall. Our game also gave us first-hand experience with coding and working with different types of motor and understanding which power source is needed to make them run. 

 

Below is a link to the instructions given to the players of our game:

 

https://docs.google.com/document/d/1OQ_vsZeYyzrebSi1Re6tmbrCwPBLc3beTObq_t6074c/edit?usp=sharing

 

Link to photos: https://drive.google.com/drive/folders/1DtoKkmZaUkgSMgiaoov1glkI-cRMlAWK

https://www.youtube.com/watch?v=2LQd8bidr0c

Concept:

The memory I chose for this project was the moment you wake up and realize you’re late for school. Your alarm keeps ringing as you have to quickly get ready for school but you also have to make your breakfast and finish your homework before you exit the room. The emotion I wanted to express in my soundscape was the feeling of anxiousness and panic one gets when they are running late.

Process:

The sounds I used were alarm, water, napkin dispenser, chopping, microwave, printer, highlighting a piece of text, typing on a keyboard, paper printing, slippers, door closing, door lock, zipper closing and a bag of chips opening. I mainly put these sounds in sequential order with overlapping so that first it would be the water and napkins sound with the sound of an alarm panning in the background and then it would be a bag of chips opening with chopping and the microwave sound in the background. Then the sound of highlighting, typing and printing to show someone doing their work quickly. Lastly, I added a zipper, door close and lock sound with slippers to show that the person is leaving. I had some difficulty using the Tascam recorder when recording my sounds because the recorder I had kept glitching out and turning off when trying to switch to shotgun mode so I mainly recorded with the internal recorder. To solve this problem, I had to keep playing with the recorder to switch it to shotgun mode and was able to use it after a while. Some effects I added were echo, reverb, and parameter equalizer to certain parts of the sound.  I added echo to the alarm sound to make the sound more significant and reverb to the printer sound to give the sound more dimension and that it was in a different room. I found the effects difficult to use and I was confused on how to show the emotion I wanted through the effects but I felt that the effects I used were able to enhance my sound in that there was a smoother transition in the final cut. 

Conclusion:

What I could improve on is focusing more on the feeling invoked by the sound rather than the narrative. I felt that the sounds were clear and the story behind it as well but it wasn’t very exciting or thought provoking to the audience. I could’ve used a different set of frequencies and sounds to make the emotions more clear in the soundscape. On presentation day, I receive a lot of useful feedback that I agree with and that is to focus more on the feeling and making the sound more versatile, not just sequential events. If I had more time I would’ve focused more on specific sounds and use more effects to emphasize the feeling behind it to show a more anxious sense of panic for the audience to feel and think about. 

The First Obstruction

• No single edit may be longer than 12 frames
• Answers to the questions (what is the man doing etc.)
• Cuba (location, budget?)
• No Set
• Man with cigarette, different views, use of same clips

The Second Obstruction

• A miserable place
• Don’t show it (how far are you willing to go)
• Jorgen Leth is the man
• The meal will be there

The Third Obstruction

• Complete freedom or back to Bombay

The Fourth Obstruction

• CartoonThe Fifth Obstruction
  • Lars Von Trier will make the last obstruction
  • Jorgen Leth will be credited as director

   

  Jorgen deals with the obstructions and completes them to the best of his ability but also leaves room for interpretation and originality. He makes the piece his own by interpreting the obstruction in his own way. The effect he applies on the movies is similar to the original film starring Claus Nissen where he adds his narration of the “perfect human” and cuts of different scenes to represent the concept he was given. 

            By “unbroken belief” in the truth of documentary images, Steyerl means that the authenticity of documentary cannot be questioned because it is open to interpretation by the public. The projection of the image shows how reliable the information is. What emotion and expression the image gives off to the viewer shows how authentic it is. The significance of the authenticity and representation of the truth in the media that we consume on a daily basis is that it can be manipulated in many different ways. For example, in social media nowadays people can post false information based on their personal beliefs that could be misleading for viewers. They can also manipulate images to shape what they want to put out there and misguide the viewers. However, in a live broadcast, information can be more reliable since it is live and no manipulation can be edited in to mislead the audience. There are a multitude of tools that can be used to alter what is put out there in the word which leaves the authenticity and representation open to the audience to judge since only the creator knows the truth.

 

The memory I am choosing for my memory soundscape project is the moment when you are running late but realize you still have stuff to do before you leave your dorm. There are often times I wake up late and only have like 10 minutes until I have to catch the shuttle. In the span of 10 minutes, I have to quickly get up and get ready, make lunch, finish homework and then catch the shuttle. The feeling of being rushed and the stress/anxiety of catching the shuttle is what I aim to represent in my project.

 

Stephanie Yip, The Harshness of Reality

           The concept I was going for with this piece is the feeling behind the thought of slowly losing your childhood memories and dreams to the harsh reality that time is always moving. The thoughts you have as a child are temporary as reality always kicks in and is inescapable. That is why I chose the objects: duck, origami stars, ribbon and dead roses to symbolize a sad, almost playful concept. 

           The process I had for this piece was at first just playing around with the composition and seeing how the objects work best with each other and what connection the objects have with each other can convey. I mainly liked the photos I took with a darker use of the white light and a higher ISO and aperture because it had better focus on the duck and less on the background as well as showed a sadder mood for the concept I was going for. When it came to the photoshop aspect of the project, I had difficulty using the techniques but after some time playing around with what I can do, I decided to use the lasso tool to add other elements to the photo. I added a bunch of arms/hands to represent that reality is inescapable and inevitable. I also added a blue/purple swirl in the background to symbolize the subject coming out of a dreamlike state and into reality. I would say that to me, the photoshopped image produced was able to convey the meaning I wanted it to convey.

            I think what I could do to improve my piece would be to firstly, figure out when adding my pictures onto photoshop, how to make it so that it has as clear a resolution as it was originally photographed. Another thing I could do is add more color to the photoshopped version to convey more of a dreamlike state and make it more clear what my concept was to the audience. 

               The concept I’m going for with this project combines a dream-like mood with a playful yet ominous one to depict the feeling of vulnerability in comfort. For the other image, I am thinking of capturing people/animals in exhaustion and for the photos of objects, I am thinking of using origami stars, dried roses, ribbon, and cookies and connecting the images with possibly the use of the ribbon and/or as a thought bubble/smoke. I was thinking of using the ribbon to represent the pulling back into reality or back into slumber. Through photoshop, I may alter the color/shade of the photo to a more dark image so the image looks more dark and unreal. Then I may change the arrangement of the objects to connect more with the subject of the other photo but I am not completely sure just yet on how I will do so. 

              By “Fluidity of the Digital”, Ritchin discusses how digital technology has now advanced with more capabilities to manipulate the image and changes the way it is perceived by the public. Digital technology refers to the changing of architecture of the photograph and changing the medium itself. This could alter the meaning behind the image and show a different perspective/course of events to those who perceive it. An example of digital photography could be photoshopping that changes the genetic appearance of an image as well as retouching images that creates a different aura/impression from its original picture. This changes the image from when it was originally taken to the way people see it when the image is published.

             Photography can capture any image of real events, however, there is an extent to which it is allowed to be published to the public eye. For example, there are limits since many photos can be negatively perceived and criticized for its impact on society that it may have such as violent images or images that could be perceived as racist or sexism that has a high possibility of receiving public hate. The same rules follow other forms of media such as texts and videos as the creator must make sure it is appropriate for its audience to prevent it from getting backlash. The publisher must think of the potential impact the image has on society and how the image represents reality. There are other forms of media that instead of capturing the reality of the world, express the imagination one may have due to the fictionality of the piece such as virtual reality and books. The creativity of the piece can be expressed but the reality of it is limited which limits the extent to which the image of what is imagined can be captured and what social limitations will possibly have on it as well.