Category: Interaction Lab

Vocal Art (With Kevin Nader)

This project will make artistic interpretations of the user(s)’ voices, and it will base the color’s saturation on the volumes of the voices. The project will ideally translate the vocal input (from a microphone) into colors and shapes (again, based on volume). After one user uses the project, the interpretation of their voice would remain drawn on screen, and will be added to by the next user. 

This project aims to interpret the voices of its users, and compound these interpretations between each user. The first user would start with a blank canvas, while each subsequent use would build on top of that. The Arduino code will process input data and send it to processing, which will generate the output. This project does not require that much physical fabrication, the only physical component being the microphone Arduino circuit. Once this part of the project is complete, my partner and I will write the Arduino and processing codes. Given the simplicity of the physical aspect of the project, this could allow my partner and me to design an aesthetically-pleasing physical apparatus to house the circuit. 

Over the course of the semester, we have had to define and redefine “interaction” numerous times. I originally defined it as “the communication between two actors” for the group project, then changing it to “the communication between two or more actors through mutually influential inputs and outputs” for the midterm project. As of now, my definition is “the communication between two or more organic or machine actors, facilitated by an interchange of inputs and outputs between actors”. Interactivity relies on the mutual exchange of inputs and outputs between human and non-human actors. These updates to my definition were influenced by Crawford’s “The Art of Interactive Design”, in which he defines interaction “in terms of a conversation: a cyclic process in which two actors alternately listen, think, and speak (1)”.

Vocal Art aligns with my updated definition of interaction because it consists of a machine actor and potentially several human actors that consistently communicate and build upon previous outputs. My partner may have different ideas about this project’s significance, but I see it as a collaborative art piece that complicates itself and constantly changes based on its perception of its audience. I don’t think there is a specific intended audience, simply anyone who wishes to add their voice to the collective canvas.

Part 1: Etch-A-Sketch

Arduino Schematic:

Arduino Code:

void setup() {
  Serial.begin(9600);
}

void loop() {
  int sensorValue1 = analogRead(A0);
  int sensorValue2 = analogRead(A1);
  int mappedvalue1 = map(sensorValue1, 0, 1023, 0, 500);
  int mappedvalue2 = map(sensorValue2, 0, 1023, 0, 500);
  Serial.print(mappedvalue1);
  Serial.print(",");
  Serial.print(mappedvalue2);
  Serial.println();
  delay(1);        
}

Processing Code:

import processing.serial.*;
String myString = null;
Serial myPort;
int x2;
int y2;
int NUM_OF_VALUES = 2;   
int[] sensorValues;      

void setup() {
  size(500, 500);
  background(0);
  setupSerial();
}

void draw() {
  updateSerial();
  printArray(sensorValues);
  stroke(255);
  line(x2, y2, sensorValues[0], sensorValues[1]);
  x2 = sensorValues[0];
  y2 = sensorValues[1];
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[5], 9600);
  myPort.clear();
  myString = myPort.readStringUntil( 10 ); 
  myString = null;
  sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Result:

Part 2: Musical Instrument
Arduino Schematic:

Arduino Code:

#define NUM_OF_VALUES 2

int tempValue = 0;
int valueIndex = 0;

int values[NUM_OF_VALUES];

void setup() {
  Serial.begin(9600);
  pinMode(13, OUTPUT);
}

void loop() {
  getSerialData();


  if (values[2] == 1) {
    tone(9, values[0]);
  } else {
    noTone(9);
  }

}


void getSerialData() {
  if (Serial.available()) {
    char c = Serial.read();
    switch (c) {
      case '0'...'9':

        tempValue = tempValue * 10 + c - '0';
        break;
      case ',':
        values[valueIndex] = tempValue;
        tempValue = 0;
        valueIndex++;
        break;
      case 'n':
        values[valueIndex] = tempValue;

        tempValue = 0;
        valueIndex = 0;
        break;

      case 'e':
        for (int i = 0; i < NUM_OF_VALUES; i++) {
          Serial.print(values[i]);
          if (i < NUM_OF_VALUES - 1) {
            Serial.print (",");
          }
          else {
            Serial.println();
          }
        }
        break;
    }
  }
}

Processing Code:

import processing.serial.*;

int NUM_OF_VALUES = 2;

Serial myPort;
String myString;

int values[] = new int[NUM_OF_VALUES];

void setup() {
  size(500, 500);
  background(0);

  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[5], 9600);

  myPort.clear(); 
  myString = myPort.readStringUntil(10); 
  myString = null;
}

void draw() {
  noStroke();
  fill(0, 30);
  rect(0, 0, 500, 500);

  strokeWeight(5);
  stroke(255);
  line(pmouseX, pmouseY, mouseX, mouseY);

  values[0] = mouseX;
  values[1] = mouseY;

  sendSerialData();
  echoSerialData(200);
}

void sendSerialData() {
  String data = "";
  for (int i=0; i<values.length; i++) {
    data += values[i];
    if (i < values.length-1) {
      data += ",";
    } else {
      data += "n";
    }
  }

  myPort.write(data);
}

void echoSerialData(int frequency) {
  if (frameCount % frequency == 0) myPort.write('e');

  String incomingBytes = "";
  while (myPort.available() > 0) {

    incomingBytes += char(myPort.read());
  }

  print(incomingBytes);
}

PROJECT TITLE

Are Bullied Emotional Trash bin?

 PROJECT STATEMENT OF PURPOSE

As can be seen from the project title, the purpose of this project is to make users feel like they are bullied. Through this project, I seek to shock people by giving them hands-on experience with the experience of viewing in first-person’s point-of-view of getting bullied. The feeling of scared, guilt, sadness, disgust. All of those emotions above would be intended for this project. 

PROJECT PLAN *

Explain in further detail what your project aims to do. Flesh out your project’s process through careful description, analysis, and with specifics.  What steps will you take to empathize with your intended audience and analyze their needs/requirements? Compose a detailed project plan that explains each of the steps you will take in the next few weeks to deliver your project. Be specific about when you need to finish which part of your project to finish it on time. 

Although we have grown up from the main bullied ages, we are still college “students” who are vulnerable to getting bullied. In fact, there are a lot of cyberbullying going on, which counts towards people of all ages. I hope through this project, people would walk away with a feeling of sadness and terror that they would be willing to stand up if they see the ones getting bullied. 

Because of the probable loudness within the room demonstrating this project, 1. Isolate the user in the small room (maybe we can use a black curtain to make up a small space where the user can go in?)

2. I am going to make the user use their earphone (Bluetooth earphone to ensure the user can move) listen to the audio. 

3. The room would in more or less in a dark hallway form so the user can walk straight. If there are mannequins standing in the sideways, it would be great. But if it is not available, I would try to film the video of walking in NYU hallway as if I am a transparent person where everybody is ignoring me (or looking at me but are spreading rumors about me). 

4. At the end of the hallway, there is a computer, which would work as a user’s computer. They would already feel strange by walking in the hallway. The distance of the user would be measured with the distance sensor. If the user walks up close to the computer, all of a sudden, there would be a notification on the side of the computer (through coding).

5. If the user clicks on the notification, there would be a short greeting message like “Yo you there?”.

6. The user will be able to click on the buttons to have a conversation with the fake person.

7. The computer starts receiving bullying messages.

8-1. If the user gets surprised and aims to walk away (also measured with the distance meter. if the user goes further than set amount of distance), there would be a message on the screen and the sound also through their bluetooth earphone saying “You might only feel it for these few minutes, the bullied have to go through this trauma all the time. A lot of people often still having difficulties from their childhood bullied experience.” and so on.

8-2. If the user successfully continues, the user will hear the voices through their earphone as he/she is receiving the messages as if they are speaking to them right next to the user.

9. After the user is finished with the messages, the user will be shown statistics about bullying. 

10. Outside the room would be an arduino-connected printer, a projector, and a 3D printed trash looking bin.

11. Through coding, the camera would take picture of the user (without them knowing as they go through the messages), the projector will screen the person’s face on the trash bin (because that is what the bullied are often used as – emotional trash bin where they can throw out their anger and negative emotions towards). As the user goes through messages, the printer will put papers (which has the bullied words that the user receives through the screen) in the trash bin.

CONTEXT AND SIGNIFICANCE *

Of course, there are three different types of people in the bullying scene: the bully, the bystander, and the bullied. The user is going to feel from the perspective of the bullied. I choose the bullied over anything else because the tragedy and the pain of bullying situation can only be felt through hand-on experience from the perspective of the bullied. The bully does not remember all the people he/she has bullied. If we were to make the hand-on experience for users with the first-person point-of-view of the bully, the purpose would be to make that user guilty for what the user has been doing. Although he/she may feel guilty, it would not be as effective as forcing the user to go on experience as if the user is bullied. 

This project is significant. Though there have been a lot of the people with the experience of getting bullied, there are still some people who have not got the experience of getting bullied just like my roommate. When I asked my roommate whether she knows how bad it is to be bullied, she would merely guess how bad would it be, but only has a vague idea about it. Through the first-person point-of-view in the perspective of the bullied, the user, hopefully, would walk away from my project with how bad it would be to be bullied by others. 

Although we have grown up from the main bullied ages, we are still college students who are vulnerable to getting bullied. In fact, there are a lot of cyberbullying going on, which counts towards people of all ages. I hope through this project, people would walk away with a feeling of sadness and terror that they would be willing to stand up if they see the ones getting bullied. 

This is related to our reading, Language of New Media by Manovich which describes the influence of computers on new media. We are currently living in a world where computers are everywhere. True, as I mentioned from the previous recitation blog posts, computers have had positive effect towards the human. Because we have the computers, we are able to communicate with people who are on the other side of the Earth, easily being able to find information and so forth. However, at the same time, they also had a negative impact towards human: it also has gotten easier for people to bully others. When people are online, they do not need to face the ones they are bullying. As a result, they would not feel guilty about saying mean things to other people online. 

We live in a consumerist world. Our products have short life spans- we buy them to throw them out, barely conscious of the environmental and social impact this has, such as deforestation, water pollution, child labor, and life-threatening working conditions. Some of us are aware, but indifferent. 

This ignorance and indifference magnifies the problems and injustices we face today as a society, as everyday people refuse to take an active roll in bettering the situation. Even worse, we are actively enabling the mistreatment of the less fortunate and the destruction of our ecosystems by buying from unsustainable companies. We feel good about ourselves for recycling or publicly disagreeing with child labor, but then we go ahead and pay companies that pollute our oceans, fill our landfills and make money off the back of those who can’t stand up for themselves.

I want my project to expose this issue, targeting those who actively make purchases without thinking of the consequences. I want to create an interactive art installation that makes the user think about what they stand for- what they promote. For this purpose, I will focus on fast fashion.

I will create a display where users must choose a product they want to buy or a store they want to shop from. Once they choose it, a projection will display quick-paced, shocking clips of the irresponsible actions that allow these companies to make as much money as they do.

I am still unsure as to the exact structure of the installation. Much testing and prototyping is still required to decide what form of display is both feasible and impactful. At the moment, I am considering various options for the user interface: a screen, a holograph, physical buttons and switches, etc.  I will order the prototyping material by Thursday November 28th. By Sunday, December 1st, I will have done research on the particular impact each company I choose has on the environment. On this day I will also order any material I need for the final design.  On December 2nd I will create the video content that will be projected, on the 3rd I will write the code I need and on the 4th, I will start assembling the installation.

I will look through news media, documentaries, and reports on fast fashion to inform myself more on the subject. One very strong inspiration for my project is the documentary “The True Cost” by Andrew Morgan, where he explores fast fashion and it’s effects on our society. Visually, I will draw inspiration from Daan Roosegaard’s work and Nick Verstand’s Aura Interactive Audiovisual Installation.

Our project is called “Who’s ordering your food?”. This project aims to help targeted users realize different factors affecting their choice of food and notice how easy it is for us to be misled by social media. We come up with this idea when noticing many female classmates check the nutrition chart very carefully before they buy foods and snacks. And with our research, we find many ways how social media influence people’s food choice. The most common existed scientific studies in this field suggests that social media influence people’s food choice in two ways. The first is the prevailing image of “beauty” and “health” and the trend to be on a diet and lose weight. This fit/slim/skinny being beautiful is embedded into people’s minds to make them conscious of calorie intake and contributes to a lot of eating disorder cases ^{1, 2,3,4} .

The second is the advertising and scientific reports revealed to the public (this is a tricky point because people assume what they see is true however it might only be partially real and there are chances that even scientific studies are sponsored by certain manufacture) ⁵.

Despite social media, who you eat with matters a lot in food choice subconsciously. For example, family dinner is often used as a therapy for  patients of eating disorder because eating with family usually means eating more healthy ⁶. Consider the different aspects mentioned above which affects people’s food choice, we created different scenarios which enables users to choose food for the character on the screen after watching a short video as “priming”. And as users realize how their choices differ after watching different videos, they will be more conscious when they later choose their foods and will not be influenced by social media that easily and this can avoid a lot healthy-harming behaviors. Since young girls are the most affected group in terms of changing diet because of external factors, they are the main target of this project although this project basically aims at everyone who want to improve their diet as target user.

Opening with the view of a single young girl in a restaurant, users are expected to choose from the menu board a dish or a meal for the girl. After this initial choice, users enter the first scenario where they will be presented with a short clip about diet and weight control (such as the following). We are considering whether to add a captured cam picture/live video afterward, might be and might not. Afterward, user will be asked to make the food choice again. The second scenario will use a food advertising clip or a food science news clip where the third scenario primes with family or friends. To make sure users fully get our message but trying to make this project more interactive and thought-provoking instead of lesson teaching, hints will be given during the experience. A note of random related information (an array will be created for that) will be presented at the end of each scenario (e.g. “An apple a day keeps the doctor away…but only an apple for a day sends you to the doctor”) and a sound response will be given consider the choice. We plan to have all the video clips ready (create the arrays if possible) and do the fabrication work next week and start coding after coming back from Thanksgiving Break.

The preparatory research influence the way how we give our message. While this is kind of an educational project, it would be reactive instead of interactive if we just try to have the same result for all inputs and that is a failure in our midterm project. So we are not trying to tell users directly how different factors might change people’s choice and the result but letting users discover that during their experience. It is possible that some users would be conscious and not affected by any of the factors. This is also good and our response given will only consider whether his/her choice is healthy and provide suggestion base on that. Therefore, each user has their personal experience and need to really think and get involved during the process. Professor Lin has recommended us a lot art projects related with food and these all put emphasis on personal experience and communication even with the same food so we also try to create more private responsive experiences. We hope this experience will raise users’ awareness when they choose food since once a person has awareness and is alert about what will affect his/her choice, he/she will result in more rational choices and this can help them getting healthy (less likely to be obese or have eating disorders).

Notes & Resources:

1. The following article (in Chinese) is a study on China’s losing fat industry which argues why many people choose to lose weight. Scholars argue that “losing weight” is promoted as a fashion in large cities like Shanghai and Beijing which however leads to bad consequences in terms of consumers’ health. https://wenku.baidu.com/view/1e3c8503bed5b9f3f90f1c2c.html

2.The following article (in Chinese) is a report which suggests many young girls are indulged in use of meal replacement powder/shake (e.g. the left image) influenced by advertisements who said these powders can substitute meals and are more healthy. However it only functions in losing weight and causes many hospitalized cases. http://www.xinhuanet.com/fortune/2018-06/27/c_1123040998.htm

3.Concurrent and Prospective Analyses of Peer, Television and Social Media Influences on Body Dissatisfaction, Eating Disorder Symptoms and Life Satisfaction in Adolescent Girls  https://link.springer.com/article/10.1007/S10964-012-9898-9

4. Body Image, Media, and Eating Disorders https://link.springer.com/article/10.1176/appi.ap.30.3.257 

5.Can Social Media Influence What Your Child Eats? – Study confirms social media can sway kids to eat junk food         https://health.clevelandclinic.org/can-social-media-influence-what-your-child-eats/

6. 9 Scientifically Proven Reasons to Eat Dinner as a Family https://www.goodnet.org/articles/9-scientifically-proven-reasons-to-eat-dinner-as-family