Author: Xueping

Codes: Arduino + Processing

We wish our users do not interact with our project simply by clicking some buttons  only but get more involved and think during the process when they are interacting with it. Therefore, we add more sensory experiences during the interaction to get users more focused and interested. Images, videos and sound responses are presented with this aim. We also tried to choose or create images which are bright-colored and kind of animated so that they look more inviting. Movement of images during the third scenario about news both help attract attention and resembles how people encounter news that massive information show up and change all the time. We could have use more words or one simple scenario but we think that might become more responsive instead of interactive and users will lose interest. The  mode that letting users go through three scenarios and make choices before teaching them anything makes results more convincing and provokes thoughts in this process too.

Video of the whole interaction process (shoot in the final show)

The most important part in our production is creating the three scenarios in processing and creating indications to tell users what they are supposed to do without being too direct. Since we want our users to make their food choice in a social context similar to reality to see how social media might have influenced their food choice so that they learn to be more aware in their later lives, we first let them choose dishes for a meal and then after being primed with video and news/advertisements, they are going to do the choices again. The menu is not made in processing but as fabrication also because we intended to make it look more like a real menu with images of the food combination which we assume can be more interesting and inviting. Nevertheless, since it is now a box-like thing, I personally feel it does not fit our plan and I do agree with some comments that maybe making it look more like a real menu or having other buttons all on this menu helps improving this project.

Our whole research & production process documented in doc

Because we spent too much time coding the third scenario and dealing with all images, videos and audios; we only have our third scenario for user test. We are very glad that most our users get the message we want to convey by this project before we explain anything to them. Their suggestion mainly fall into two categories: one is adding more instructions so they will not feel lost and the other is having a explicit feedback at the end. Although we already planned to do similar things, some detailed comments are really helpful. One advice we followed is to develop on the character Lisa we used originally for the front page only and to create a more conversational guidance so that all three scenarios are connected fluently as a full diet story of Lisa. The videos and “fake” news reports are also embedded into this story so users will not get a wired feeling why they need to do this before choice (since during the user test some users especially male users are impatient and skip everything they thought unnecessary before making their choice which ruins the experience and cannot get the whole idea of the project). 

This adaption seems to receive welcomed feedback as now most users report they feel the whole experience makes sense and they are willing to go through this project scenario by scenario. It also resembles how people interact with social media in reality and makes them more aware next time when confronted with these information relating to food choice. We also complete our “report” session and inserts a reset button as response to certain suggestions we get from user test.

After we fixed all bugs in codes and are almost complete, we invited some of our friends and classmates to test our project again and there are still some confusions about which key they should press to continue and sometimes they get lost where the menu is so we change some of the guidance again with red words hoping users can see the emphasized part at least and put on notes to inform them again. However, as we have seen in the presentation, the effect is limited.

The goals of this project is to call for awareness that our current food choice is so easily influenced by all kinds of social media and information we receive no matter it is in form of live-streaming, Youtube videos, news, scientific reports; or it is in essence for educational purpose, for advertising purpose or purely recommendations. I have friends who are easily influenced by the fashion trends to be fit or by advertising articles about new restaurants or popular snacks so some of them develop eating disorders or have obesity problems. So I wish this project would help arouse awareness and awareness certainly affects decisions. 

Our project overall align with my definition of interaction that it creates a personal experience since it is aimed to be used by a single person and his/her choices combined with the specific response considering those choices creates a unique experience for that user. And they need to take in various information and process different sensory experience into thoughts to make decisions. Something not so align with our definition of interaction is that we only involved buttons as a way to interact with the scenarios which is kind of a very simple physical interaction. We do wish more interesting ways of receiving information could be used such as having something similar to a POS to print out the result which is both interesting and creates the context of a restaurant too but that is hard to put into practice. 

Nevertheless, we are still glad that during most user tests and in the presentation users can go through with the scenarios with interest and after their experience, they all give some feedbacks about their current feelings on the impact of social media. Most of people who change their choices after being primed by videos or news/advertisements report that they never realize social media has such influence on them. And we are also happy to see some people are not influenced by social media. For these users, they are also satisfied to see there are responsive feedback for them. Since our project does not aim at telling users what information are correct and what are not valid but to let them be more aware of all information that might change their choice or behavior in some ways, it is great to see different results but all get the expected reflection on their previous choices. I am also surprised to learn some stories from users who had been impacted by the exact news or advertisement we choose to include here (three users report to have used the SlimFast shake because of its believed effect). And I even get several feedbacks in the final show that suggest me to use this project to have a social research in this area.

If we have more time, first thing we would do is to fix the problem in instruction to be clearer and apparent. We might create some more buttons on the menu so there is no need to ask people to press certain key which is somehow confusing and we will try to make something really look like a menu. Another thing we have wanted to do is to change the third scenario to have two versions. Some people report in the user test that there are too many information and they lose interest in the long “lectures” of news. So we were considering to have two different ways to read the news, one is the detailed version which is the current one and the other is a brief “skim” version which has all the news and big titles moving while the voiceover will be a combined soundtrack with varied voices telling different news or slogans. The “skim” version will try to create the effect of a “explosion of information” to  the user so that even if they are not patient to see the news one by one, they get the idea and some big titles. Also, if I have more time, I wish I could get the IRB approve and use this project as a device to observe and record people’s food choices after being primed with social media and research on this topic.

The most important thing we learn from the previous mentioned failures apart from coding is to think from the user side. Because of the fact that we are the people who have researched a lot in this topic and have a general knowledge what our project is and what is the aim, sometimes the “common knowledge” or “things easy to figure out” in our minds are not common sense at all for users which confuses them and make them feel lost what to do. Also an important thing is to target a specific audience group and keep that in mind while deciding certain interactive ways, the context where the project will be shown or presented is also something needs to be think of in order to create a coercive experience. But having a specific audience group does not mean they will perform or react in similar ways. We have observed how users interact with our project during the user test and be amazed at the different ways they interact. So if we do want everyone to behave in similar ways (e.g. in our project we want everyone at least to go through the three scenarios), there should be a explicit guidance or there should be ways to restrict users’ control over the project (e.g. set timer to restrict users from skipping everything). And differences should be allowed and valued so that there need to be “responses” toward the users’ different choice and decision.

Another really essential thing to keep in mind as learned from all work done in 826 is that never assume any job/work to be easy and simple. Whenever I assume some part of coding is easy and believe I can finish in minutes there turns out to be bugs… Like I think the second scenario will be the easiest to code because it is purely movie playing and the other interaction is the same as in the first scenario which I can copy. But there turns to be errors which later we find out is due to the fact that mov. does not work properly in Processing. And then more problems pop up such as being shown as an image instead of movie and that  images supposed to stay for a while just flashes.  So every part of the work needs to be carefully dealt with and when you arrange the plan there should be left enough time to accomplish each step.

Our project aims at raising awareness of impact of social media on food choice. Studies have shown that choice of food is essential for both physical health and psychology wellbeing. The blind choice of using some food to lose weight in a short time might lead to Bulimia or other eating disorders as well as depression or anxiety while the inordinate consumption of certain foods causes health issues like obesity or heart-diseases. In recent years, with the development of social media such as live streaming, photos, online short videos or other ways which might be used as ways to popularize some product or fashion; the ideology of regarding “thin” as pretty prevails and the massive information about certain food products and their pros and cons. These things ultimately alter people’s choices either in the way of leading to the expected way or causing opposite results. With our project, users experience themselves how some social media works to influence their choice unconsciously. And this awareness they get after this experience will help them follow their own heart for decisions instead of easily get affected by information around them.

Slide:

https://docs.google.com/presentation/d/1caNLRobmOTDkGvQKPCULgM2MbfE6JPNQP8-Zk-_fBnk/edit#slide=id.g5705e94f86_0_4

import processing.video.*;
Movie myMovie;
int previousX = 0;
int distance;

void setup() {
  background(0);
  size(640, 480);
  myMovie = new Movie(this, "music.mp4");
  myMovie.loop();
}
void movieEvent(Movie movie) {
  myMovie.read();  
}
void draw() { 
  if (myMovie.available()) {
//read my file when it can be read
    myMovie.read();
//read the current frame of the video
  }
  image(myMovie, 0, 0);   
  distance = mouseX - previousX;
  float newSpeed = map(distance, -width, width, 0.1, 5);
  myMovie.speed(newSpeed);
  previousX = mouseX;
}  

The way technology was used in my project is kind of negotiating with the physical conditions presented by the world although because the input I used are potentiometers whose output is already legible for the computer, it does not need to be modified again (except for the “map” process) to become more easily legible to vision algorithms. The inputs are later used as control of speed and color tone (tint) of the video clip being played. I tend to view my project as a very basic use of computer vision algorithms. From the input side, my project is very direct so the response is somehow more reactive while the use of other motion detective device or sensors might create better interaction. From the output side, since the video is existed while the input only changes the speed and the color tone, it reduces the creativity and degrades interactive experience.

Code for Arduino

https://github.com/LilyWang1997/Recitation-9-Media-Controller/blob/master/serial_multipleValues_AtoP_arduino.ino

Code for Processing

import processing.serial.*;
String myString = null;
Serial myPort;
int NUM_OF_VALUES = 2;   
int[] sensorValues;

import processing.video.*;
Movie myMovie;

void setup() {
  size(360, 640);
  myMovie = new Movie(this, "Lilly.mp4");
  myMovie.loop();
  setupSerial();
}

void draw() {
  if (myMovie.available()) {
    myMovie.read();
  }
  updateSerial();
  printArray(sensorValues);
  tint(sensorValues[0]/5, sensorValues[0]/1.5, sensorValues[0]/2); 
  image(myMovie, 0, 0);
  float newSpeed = map(sensorValues[1], 0, 1023, 0.5, 5);
  myMovie.speed(newSpeed);
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 5 ], 9600);
  myPort.clear();
  myString = myPort.readStringUntil( 10 );  
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Our project is called “Who’s ordering your food?”. This project aims to help targeted users realize different factors affecting their choice of food and notice how easy it is for us to be misled by social media. We come up with this idea when noticing many female classmates check the nutrition chart very carefully before they buy foods and snacks. And with our research, we find many ways how social media influence people’s food choice. The most common existed scientific studies in this field suggests that social media influence people’s food choice in two ways. The first is the prevailing image of “beauty” and “health” and the trend to be on a diet and lose weight. This fit/slim/skinny being beautiful is embedded into people’s minds to make them conscious of calorie intake and contributes to a lot of eating disorder cases ^{1, 2,3,4} .

The second is the advertising and scientific reports revealed to the public (this is a tricky point because people assume what they see is true however it might only be partially real and there are chances that even scientific studies are sponsored by certain manufacture) ⁵.

Despite social media, who you eat with matters a lot in food choice subconsciously. For example, family dinner is often used as a therapy for  patients of eating disorder because eating with family usually means eating more healthy ⁶. Consider the different aspects mentioned above which affects people’s food choice, we created different scenarios which enables users to choose food for the character on the screen after watching a short video as “priming”. And as users realize how their choices differ after watching different videos, they will be more conscious when they later choose their foods and will not be influenced by social media that easily and this can avoid a lot healthy-harming behaviors. Since young girls are the most affected group in terms of changing diet because of external factors, they are the main target of this project although this project basically aims at everyone who want to improve their diet as target user.

Opening with the view of a single young girl in a restaurant, users are expected to choose from the menu board a dish or a meal for the girl. After this initial choice, users enter the first scenario where they will be presented with a short clip about diet and weight control (such as the following). We are considering whether to add a captured cam picture/live video afterward, might be and might not. Afterward, user will be asked to make the food choice again. The second scenario will use a food advertising clip or a food science news clip where the third scenario primes with family or friends. To make sure users fully get our message but trying to make this project more interactive and thought-provoking instead of lesson teaching, hints will be given during the experience. A note of random related information (an array will be created for that) will be presented at the end of each scenario (e.g. “An apple a day keeps the doctor away…but only an apple for a day sends you to the doctor”) and a sound response will be given consider the choice. We plan to have all the video clips ready (create the arrays if possible) and do the fabrication work next week and start coding after coming back from Thanksgiving Break.

The preparatory research influence the way how we give our message. While this is kind of an educational project, it would be reactive instead of interactive if we just try to have the same result for all inputs and that is a failure in our midterm project. So we are not trying to tell users directly how different factors might change people’s choice and the result but letting users discover that during their experience. It is possible that some users would be conscious and not affected by any of the factors. This is also good and our response given will only consider whether his/her choice is healthy and provide suggestion base on that. Therefore, each user has their personal experience and need to really think and get involved during the process. Professor Lin has recommended us a lot art projects related with food and these all put emphasis on personal experience and communication even with the same food so we also try to create more private responsive experiences. We hope this experience will raise users’ awareness when they choose food since once a person has awareness and is alert about what will affect his/her choice, he/she will result in more rational choices and this can help them getting healthy (less likely to be obese or have eating disorders).

Notes & Resources:

1. The following article (in Chinese) is a study on China’s losing fat industry which argues why many people choose to lose weight. Scholars argue that “losing weight” is promoted as a fashion in large cities like Shanghai and Beijing which however leads to bad consequences in terms of consumers’ health. https://wenku.baidu.com/view/1e3c8503bed5b9f3f90f1c2c.html

2.The following article (in Chinese) is a report which suggests many young girls are indulged in use of meal replacement powder/shake (e.g. the left image) influenced by advertisements who said these powders can substitute meals and are more healthy. However it only functions in losing weight and causes many hospitalized cases. http://www.xinhuanet.com/fortune/2018-06/27/c_1123040998.htm

3.Concurrent and Prospective Analyses of Peer, Television and Social Media Influences on Body Dissatisfaction, Eating Disorder Symptoms and Life Satisfaction in Adolescent Girls  https://link.springer.com/article/10.1007/S10964-012-9898-9

4. Body Image, Media, and Eating Disorders https://link.springer.com/article/10.1176/appi.ap.30.3.257 

5.Can Social Media Influence What Your Child Eats? – Study confirms social media can sway kids to eat junk food         https://health.clevelandclinic.org/can-social-media-influence-what-your-child-eats/

6. 9 Scientifically Proven Reasons to Eat Dinner as a Family https://www.goodnet.org/articles/9-scientifically-proven-reasons-to-eat-dinner-as-family

Etch A Sketch (ellipse)

This device/toy basically turns movement into visualized figures which enables users to think and create drawings via spinning the knob. This interaction is meaningful and provides great individual experience so that users are involved in this game.

Code for Arduino – Etch A Sketch 

https://github.com/LilyWang1997/BeHappy/compare/master…LilyWang1997-recitation-8?expand=1#diff-b6f2cc785ca65c1eebad1297a4739d11

Code for Processing – Etch A Sketch (ellipse)

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/


void setup() {
  pixelDensity(2);
  size(500, 500);
  background(0);
  setupSerial();
}


void draw() {
  updateSerial();
  printArray(sensorValues);
  ellipse(sensorValues[0], sensorValues[1], 100,100);
  // use the values like this!
  // sensorValues[0] 

  // add your code

  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 5 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Etch A Sketch (REAL)

Code for Processing – Etch A Sketch (REAL)

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/
int[] psensorValues = {0,0};

void setup() {
  //pixelDensity(2);
  size(600, 600);
  background(0);
  setupSerial();
  printArray(sensorValues);
  //arrayCopy(sensorValues, psensorValues);
  
}


void draw() {
  updateSerial();
  printArray(sensorValues);
  strokeWeight(1);
  stroke (255);
  line(psensorValues[0], psensorValues[1], sensorValues[0], sensorValues[1]);
  psensorValues[0] = sensorValues[0];
  psensorValues[1] = sensorValues[1];
  // use the values like this!
  // sensorValues[0] 

  // add your code

  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 5 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Musical Instrument

This is kind of a new musical instrument though the interaction and outcome is pretty simple. By moving the mouse on the canvas, the tones change. It can be quite interesting, but for me this game is a bit boring as the interaction is too simple.

Code for Arduino – Musical Instrument

https://github.com/LilyWang1997/BeHappy/compare/master…LilyWang1997-recitation-8?expand=1#diff-b6f2cc785ca65c1eebad1297a4739d11

Code for Processing – Musical Instrument

import processing.serial.*;

int NUM_OF_VALUES = 2;  /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/


Serial myPort;
String myString;

// This is the array of values you might want to send to Arduino.
int values[] = new int[]{0,0};
int x;
int y;
void setup() {
  size(500, 500);
  background(0);

  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 5 ], 9600);
  // check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index of the port

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
}


void draw() {
  background(0);
  x = mouseX;
  map(x,0,600,0,255);
  y = mouseY;
  map(y,0,600,0,255);
   
  // changes the values
  for (int i=0; i<values.length; i++) {
    values[0] = x;  /** Feel free to change this!! **/
    values[1] = y; 
  }

  // sends the values to Arduino.
  sendSerialData();

  // This causess the communication to become slow and unstable.
  // You might want to comment this out when everything is ready.
  // The parameter 200 is the frequency of echoing. 
  // The higher this number, the slower the program will be
  // but the higher this number, the more stable it will be.
  echoSerialData(200);
}

void sendSerialData() {
  String data = "";
  for (int i=0; i<values.length; i++) {
    data += values[i];
    //if i is less than the index number of the last element in the values array
    if (i < values.length-1) {
      data += ","; // add splitter character "," between each values element
    } 
    //if it is the last element in the values array
    else {
      data += "n"; // add the end of data character "n"
    }
  }
  //write to Arduino
  myPort.write(data);
}


void echoSerialData(int frequency) {
  //write character 'e' at the given frequency
  //to request Arduino to send back the values array
  if (frameCount % frequency == 0) myPort.write('e');

  String incomingBytes = "";
  while (myPort.available() > 0) {
    //add on all the characters received from the Arduino to the incomingBytes string
    incomingBytes += char(myPort.read());
  }
  //print what Arduino sent back to Processing
  print( incomingBytes );
}