Final Project Reflection: Snack Facts by Eleanor Wade

Snack Facts – Eleanor Wade – Marcela Godoy

CONCEPTION AND DESIGN:

When considering how my users were going to interact with this project I took much of my research regarding the consumption of animals and animal products, as well as the typical experience of a grocery store in mind.  In order to recreate a similar feeling of the vast array options that consumers are presented with at a super market, I chose to use the color scanner and foods with various colored tags to allow for users to have the experience of checking out at a typical store. Following making their decisions and selecting different products from the shelves (specifically meats with red tags, animal products with blue tags, and plant-based foods with green tags) users would scan to see an overwhelming assortment of pictures and quick facts regarding the process from an industrialized factory farm to table, and the differing environmental impacts each of these actions have.  The color sensor was critical in my design and conception of this experience because it is not only a hands-on and interesting action, but also it can be very clearly linked to the overall feeling of checking out at a grocery store.  It is my hope that many of my users will associate this feeling of blindly making decisions with the pictures that appear on the screen.  While the shelves were made of cardboard, I also included many collected, plastic packages that are commonly used in grocery stores.  This helped to further explore the question of how we process our foods and package them for our convenience, without fully understanding what the consequences of these choices are. Other materials I used, such as real foods (carton of milk, jam, bread, sausages, cookies) were an effort to make the experience appear slightly more realistic.  Additionally, the few edible foods I provided were very beneficial in working to complete the experiences and add an interactive element of taste and smell to the project.  It was evident that these materials–particularly the real, edible foods– were central to the interactive aspect of my project because in addition to using the color sensor, being presented with both plant based and animal based products further made customers question the choices they make everyday.  In associating this specific taste with the exposed realities of the food systems, this project used levels of interactivity to educate people about the environmental impacts of their food choices.  

FABRICATION AND PRODUCTION:

The most significant steps in my production process started with building off of my previous research about animal products and talking with Marcela regarding the best ways to move forward with how to create an interactive and educational experience involving food.  After deciding to use the color sensor, I used my research from a previous recitation using this sensor to work through the Arduino to Processing communication.  In working on the coding of both, and furthering the project by adding a collage of photos from my research, Marcela was exceptionally helpful to me.  I definitely struggled with how to translate the specific numerical values associated with each color and how to connect this to groups of photos.  User testing proved to be very beneficial to me because I was able to engage with users as they were experiencing my project, as well as receive feedback such as the problems with clarity of the text (so later I changed this to only pictures, rather than facts) as well as the speed of the shifting pictures.  Users/”customers” also commented on the action of selecting individual products to scan, as well as the role that edible foods played in the entire understanding of interactivity in my project.  Because of this, I made an effort to select real foods that would be pertinent to the decisions that we make regarding our every meal.  In terms of justifying these aspects of the design, using sample sized foods also supported the various free samples that are commonly found at grocery stores. While the many changes I made to my project following user testing were effective, I think it would have been even better to clarify the images I used, in addition to fixing the distortion, however even after making many different alterations, this was especially difficult.

Digital Fabrication:

3D printing:  https://www.thingiverse.com/thing:2304545

I decided to create a 3D printed mushroom because it represents the produce that is commonly found at a grocery store or supermarket.  I wanted to 3D print rather than laser cut something because I found it relatively easy and beneficial to be able to make shelves out of cardboard, as well as the scanner that contains the Arduino and breadboard for the color scanner.  

CONCLUSIONS:

The primary focus of my project is to educate people on the larger consequences and implications of their food choices.  Through the interactive concept of using a scanner to trigger images specific to food production, I hope to demonstrate the consequences of dietary choices and the larger implications that surround industrialized agriculture and animal farming.  The results of my project align with my definition of interaction because not only do users get to engage with a supermarket-checkout-style scanner, but also they are presented with real, edible foods to further the understanding of what you eat matters. This response from seeing unpleasant or informative images helps to further the elements of interaction in that users both learn something new and associate these facts with the foods they consume regularly.  If I had more time, I would improve my project by fixing the distortion of the images, and by adding sound–specifically the screams of animals living on factory farms as well as a sound that is made after each scan to demonstrate the actions– in order to engage audiences in the experience of the project on a more complete level.  This project has taught me many valuable components, for example the potential that technology and design have for enhancing our understandings of the world and shifting ideologies on even the most basic aspects of life, such as food.  When users are able to experience projects that appeal to more than just one sense, it also enhances the project overall.  Regarding my accomplishments on this project, I am pleased to have been able to use creative technology to be able to introduce people to the realities of food systems that they may have otherwise been very disconnected from.  Ultimately, this project uses visual cues combined with senses such as taste and smell to demonstrate not only compelling methods of interaction, but also help to bridge the gap that we have from how our food is produced.  Audiences and customers should care about this project and experience because it demonstrates the exceptionally detrimental consequences of eating animals and animal products, and translates these very common interactions with food and at grocery stores into more tangible and straightforward pieces of information.  

BIBLIOGRAPHY OF SOURCES:

“5 Ways Eating More Plant-Based Foods Benefits the Environment.” One Green Planet, 21 Aug. 2015, https://www.onegreenplanet.org/environment/how-eating-more-plant-based-foods-benefits-the-environment/.
https://search.credoreference.com/content/entry/abcfoodsafety/avian_flu/0. Accessed 29 Oct. 2018.
“Dairy | Industries | WWF.” World Wildlife Fund, https://www.worldwildlife.org/industries/dairy. Accessed 4 Dec. 2019.
Eating Animals Quotes by Jonathan Safran Foer. https://www.goodreads.com/work/quotes/3149322-eating-animals. Accessed 3 Dec. 2019.
Flu Season: Factory Farming Could Cause A Catastrophic Pandemic | HuffPost. https://www.huffingtonpost.com/kathy-freston/flu-season-factory-farmin_b_410941.html. Accessed 29 Oct. 2018.
“Milk’s Impact on the Environment.” World Wildlife Fund, https://www.worldwildlife.org/magazine/issues/winter-2019/articles/milk-s-impact-on-the-environment?utm_campaign=magazine&utm_medium=email&utm_source=magazine&utm_content=1911-e. Accessed 4 Dec. 2019.
Moskin, Julia, et al. “Your Questions About Food and Climate Change, Answered.” The New York Times, 30 Apr. 2019. NYTimes.com, https://www.nytimes.com/interactive/2019/04/30/dining/climate-change-food-eating-habits.html, https://www.nytimes.com/interactive/2019/04/30/dining/climate-change-food-eating-habits.html.
Nijdam, Durk, et al. “The Price of Protein: Review of Land Use and Carbon Footprints from Life Cycle Assessments of Animal Food Products and Their Substitutes.” Food Policy, vol. 37, no. 6, Dec. 2012, pp. 760–70. DOI.org (Crossref), doi:10.1016/j.foodpol.2012.08.002.
Ocean Destruction – The Commercial Fishing Industry Is Killing Our Oceans. http://bandeathnets.com/. Accessed 3 Dec. 2019.
Siegle, Lucy. “What’s the Environmental Impact of Milk?” The Guardian, 13 Aug. 2009. www.theguardian.com, https://www.theguardian.com/environment/2009/aug/07/milk-environmental-impact.
“The Case for Plant Based.” UCLA Sustainability, https://www.sustain.ucla.edu/our-initiatives/food-systems/the-case-for-plant-based/. Accessed 4 Dec. 2019.
The Ecology of Disease and Health | Wiley-Blackwell Companions to Anthropology: A Companion to Medical Anthropology – Credo Reference. https://search.credoreference.com/content/entry/wileycmean/the_ecology_of_disease_and_health/0. Accessed 29 Oct. 2018.
“WATCH: Undercover Investigations Expose Animal Abusers.” Mercy For Animals, 5 Jan. 2015, https://mercyforanimals.org/investigations.
What Is The Environmental Impact Of The Fishing Industry? – WorldAtlas.Com. https://www.worldatlas.com/articles/what-is-the-environmental-impact-of-the-fishing-industry.html. Accessed 3 Dec. 2019.
Zee, Bibi van der. “What Is the True Cost of Eating Meat?” The Guardian, 7 May 2018. www.theguardian.com, https://www.theguardian.com/news/2018/may/07/true-cost-of-eating-meat-environment-health-animal-welfare.

Recitation 10: Workshops Documentation by Eleanor Wade

Serial Communication Workshop:

Using a button on the Arduino as a sensor to control the sketch on processing.  We used multiple value Arduino to Processing sample code.  

This recitation was very helpful in learning some necessary techniques in serial communication, with sensors that are beyond that of a potentiometer.  This will definitely be useful when moving forward with my final project as I will  definitely be using a color sensor to translate colored tags into specific factual information and images on the screen.  In this way, it was particularly beneficial to me to be able review the information that we previously had learned regarding arduino to processing.  

Processing Code: 

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/


void setup() {
  size(500, 500);
  background(0);
  setupSerial();
}


void draw() {
  updateSerial();
  printArray(sensorValues);

  // use the values like this!
  // sensorValues[0] : pot 0 - 1023
  // sensorValues[1] : button switch 0, 1

  // add your code
  float posX = map(sensorValues[0], 0, 1023, 0, 500); 
 int size;
 if (sensorValue[1] == 0) {
   size = 50;
 } else { 
   size = 200;
 }
 
  
  
ellipse (300, 300, size, size);
  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 1 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

One value Processing to Arduino Serial Communication: 

Arduino Code:

// IMA NYU Shanghai
// Interaction Lab
// This code receives one value from Processing to Arduino

#include <Servo.h>
Servo myservo;
int val;

char valueFromProcessing;
//int ledPin = 13;

void setup() {
Serial.begin(9600);
myservo.attach(9);
}

void loop() {
// to receive a value from Processing
while (Serial.available()) {
valueFromProcessing = Serial.read();
}

val = valueFromProcessing;
val = map(val, 0, 500, 0, 180);
myservo.write(val);
delay(15);

// if (valueFromProcessing == ‘H’) {
// digitalWrite(ledPin, HIGH);
// } else if (valueFromProcessing == ‘L’) {
// digitalWrite(ledPin, LOW);
// } else {
// something esle
// }

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(10);
}

In using the map function to map the values of canvas size, we specifically changed the code to this:

val = map(val, o, 500, 0, 180);

Recitation 9: Media Controller Documentation by Eleanor Wade

While in class we have used a potentiometer to control media we see on the screen, this recitation I tried to use a different sensor that will eventually be relevant to my final project.  

Working with a Color Sensor!

Because I have never worked with this sensor before and it is totally new to me, I pretty much entirely copied the code from this website and used the explanation of how to use this sensor from this site:

https://randomnerdtutorials.com/arduino-color-sensor-tcs230-tcs3200/

For the most part this link was helpful in both the coding and the putting together of the Arduino components, however I encountered several difficulties not mentioned on this site.  There are definitely challenges in reading different red, green, and blue values. It seems as if the blue colors are always very low, when compared with the others.  I will most likely need to take this into consideration when I am working with this sensor in the future.  Because I plan to use different colored tags, I will definitely need to plan to adjust the blue accordingly.  

Arduino Code: 

// TCS230 or TCS3200 pins wiring to Arduino
#define S0 4
#define S1 5
#define S2 6
#define S3 7
#define sensorOut 8

// Stores frequency read by the photodiodes
int redFrequency = 0;
int greenFrequency = 0;
int blueFrequency = 0;

void setup() {
// Setting the outputs
pinMode(S0, OUTPUT);
pinMode(S1, OUTPUT);
pinMode(S2, OUTPUT);
pinMode(S3, OUTPUT);

// Setting the sensorOut as an input
pinMode(sensorOut, INPUT);

// Setting frequency scaling to 20%
digitalWrite(S0,HIGH);
digitalWrite(S1,LOW);

// Begins serial communication
Serial.begin(9600);
}
void loop() {
// Setting RED (R) filtered photodiodes to be read
digitalWrite(S2,LOW);
digitalWrite(S3,LOW);

// Reading the output frequency
redFrequency = pulseIn(sensorOut, LOW);

// Printing the RED (R) value
//Serial.print(“R = “);
//Serial.print(redFrequency);
delay(100);

// Setting GREEN (G) filtered photodiodes to be read
digitalWrite(S2,HIGH);
digitalWrite(S3,HIGH);

// Reading the output frequency
greenFrequency = pulseIn(sensorOut, LOW);

// Printing the GREEN (G) value
//Serial.print(” G = “);
//Serial.print(greenFrequency);
delay(100);

// Setting BLUE (B) filtered photodiodes to be read
digitalWrite(S2,LOW);
digitalWrite(S3,HIGH);

// Reading the output frequency
blueFrequency = pulseIn(sensorOut, LOW);

// Printing the BLUE (B) value
//Serial.print(” B = “);
//Serial.println(blueFrequency);
delay(100);

Serial.print(redFrequency);
Serial.print(“,”); // put comma between sensor values
Serial.print(greenFrequency);
Serial.print(“,”);
Serial.print(blueFrequency);
Serial.println(); // add linefeed after sending the last sensor value

}

Processing:

In working on the processing code, I have selected photos that will most likely be relevant to my final project from this site:

https://unsplash.com/photos/amI09sbNZdE

And plan to switch between photos when the color sensor picks up on certain colored tags.  

Final Project Essay

PROJECT TITLE:  Snack Facts 

PROJECT STATEMENT OF PURPOSE :

The primary focus of my project is to educate people on the larger consequences and implications of their food choices.  This topic is meaningful because an individuals dietary choices have an even greater impact on green house gas emissions than their transportation choices and this is a problem that has extreme relevance and correlation to our influence on the environment and global warming.  The audience this project will be angled towards is primarily my classmates, but also the general public or those who frequently consume meats or animal products.  It is my belief that this will be relevant to everyone, because everyone eats food and everyone– in my opinion– should be aware of the consequences of their actions.  Because I am very passionate about starting a conversation regarding the impacts that our food choices have on not only the environment, but also vulnerable populations as it relates to global warming and food security, I think that my project has the potential to create an interactive (and edible) experience for audiences to learn more about where our food comes from and what this means to the rest of the world.  In terms of the research I have done that has shifted my conception of this plan, I hope to create an experience of “shopping” at a grocery store or supermarket that allows users to choose from a variety of options, just as they would in a store.  Additionally, I plan to carefully observe the ways that they market to consumers and keep the realities of where certain foods come from intentionally very well hidden.  

PROJECT PLAN :

My project aims to create a parallel experience to shopping at a grocery store in that the audience will be presented with choices that seem as if they have no larger impact on the world.  By using shelves that look similar to what would be found in a supermarket, and having packaged, edible products such as meat, fish, dairy, and vegan foods that are accessible to consumers, I hope to allow people to learn of the consequences of the choices they make.  Consumers will make their choice and then be presented with a number of interactive experiences, including but not limited to visual pictures of where the food came from (factory farms, as well as audio sounds of animals, etc.  Each packaged item will have a colored tag that can be scanned with a color sensor that is connected to my Arduino, which then translates to a processing display on my computer.  I have already begun using the color sensor with processing so that after recognizing a particular color (similar to scanning a barcode at the checkout line of a supermarket) consumers can see and interact with the visual and audio consequences of their choices, then decide if they wish to continue with their “purchase”.  In this way, I hope to allow for time between when the consumer finalizes their decision and sees the disturbing images associated with their choices.  I still need to further my research to have enough material so that each item can have food facts, images, sounds that are connected to the process of how this item got from “farm” to consumer.  I also need to continue to program particular images to be prompted after the scanning of different colors, as well as create digitally fabricated tags or other constructions that help to make consumers feel as if they are really interacting within the atmosphere of a traditional grocery store.  I will also need to finalize my decisions on which specific foods to feature and have available on my shelves, and find more information on the detailed processing systems of each food.  

In terms of emphasizing with the audience and adjusting to their needs in the experience overall, I hope to make this project as a way to start the conversation of why our food choices matter.  I am working on ways I can make it more of an educational experience and link the foods with which we are familiar to a greater societal and global significance, rather than simply telling someone that they should not eat something.  When further thinking about the consumer and my particular audience of classmates, I want to make sure that this does not come off too much like a lecture or a “do this, not that” but rather a “did you know where that comes from and what impact that has on the environment?”  It is my hope that much of this information will not necessarily be new to everyone, however, it may not be something we see and deal with on a regular basis, and I hope to change that.  It is my goal to bring the disconnect between our food systems and consumerism a tiny bit closer to a shared understanding, rather than a blindness of the results of industrial animal agriculture.  

CONTEXT AND SIGNIFICANCE :

The preparatory research helped to inspire my final conception of this project in that it forced me to consider different routes of interactivity and think about the things that really matter to me.  While the research I did following the assignment of this specific project was primarily design based (as I initially though I was going to create something as an art project or that uses colors to translate to other things) through this research I learned (or rather was forced) to narrow my focus to the things I genuinely like talking about and studying, and how I can use IMA and the forms of interaction we have studied to better understand or communicate this interest.  Following this research, I thought deeply about my senior capstone (obviously a interest and passion of mine), which focuses on nutritional shifts throughout history and how we can better motivate a shift towards plant based food systems.  Because of my research on this subject over the past several months and even the recent years– in addition to the fact I have been vegetarian for more than a decade and a vegan for 3 years– I was inspired to incorporate my knowledge and research in this area into a project that utilizes technology such as Arduino and Processing to better teach people about the challenges and complications with our dietary choices.  Because food is so inherently linked to our physical health, and well as incredibly meaningful to our social, traditional, and mental wellbeing, it is exciting and innovative to think about the ways that I can use interaction and technology to expand our understanding to why we eat what we eat (and what the consequences of these actions tangibly look like).  

Because my project is intended to help us better see the direct impacts– in the form of audio and visuals as well as text– of our dietary choices, I think anyone and everyone can be a meaningful and relevant target audience.  When specifically focusing on my classmates, and how to talk to friends and fellow students about what food means, I think using this level of edible and educational interaction very clearly helps to bridge the extreme distance we have when we are mindlessly shopping at stores and making choices about food.  I would consider this project to be a success if it sticks in people’s minds in the days following and maybe even if they reconsider a meal because of this.  While I am not trying to tell everyone to be vegan (even though I do somewhat want to; and in my opinion you can’t truly be an environmentalist if you’re not plant based so as we are quickly approaching irreversible climate change, being plant based is among the biggest and most significant changes one can make to take action), I am trying to force people to rethink their simple, everyday food choices and what it means in a global context.  The goal is not to be perfectly plant based; instead the goal is to think.  Every single 菜包, slice of pizza, and chicken nugget leaves an impact.  What consequences do you want to be responsible for?  Ultimately, if this project becomes successful on a broader scale I think it would be really cool if supermarkets were required to share this kind of horrifying, disgusting, and truly significant information with consumers, similar to the ways that cigarette packages put pictures of cancerous lungs on packs.  What if rather than a serene pasture that is seen on the package of raw beef we saw slaughter houses, heard the cries and screams of tortured animals, or considered the public health statistics on developing antibiotic resistance? 

Recitation 8: Serial Communication by Eleanor Wade

Exercise 1: Etch a Sketch

Creating this interactive Etch a Sketch was not too challenging, as it was fairly similar to something we learned to code in class when working through how to bring together Processing and Arduino.  In setting up this exercise (unlike the second one) it is very clear that the input information is coming from the Arduino and the two potentiometers, similar to that of a real etch a sketch.  In this way it makes sense that we are using Arduino to Processing, rather than the other way around because we can see the movements of the potentiometer being translated into designs on the screen.  

Processing Code:

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */


import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/
float preX;
float preY;

void setup() {
  size(500, 500);
  background(100);
  setupSerial();
  //map(sensorValues, 0, 1023, 0, 255);
}


void draw() {
  map(sensorValues[0], 0, 1023, 0, 255);
  updateSerial();
  printArray(sensorValues);
  fill(0);
  //ellipse(sensorValues[0], sensorValues[1], 20, 20);
  
  float posX = map(sensorValues[0], 0, 1023, 0, width);
   
  float posY = map(sensorValues[1], 0, 1023, 0, height);
  
  line(preX, preY, posX, posY);
  // save current posX and posY to preX and preY
  preX = posX;
  preY = posY;
  
  
  
  // use the values like this!
  // sensorValues[0] 

  // add your code

  //

}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[1], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Arduino Code:

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Exercise 2: Making a Musical Instrument with Arduino

In making this musical instrument with the buzzer, it is clear that the variations in sounds being produced will come from the movement on the mousepad of my computer.  In this way it makes sense that we will be using Processing to Arduino to set up this exercise.  

After setting up the buzzer and then working in processing to make sure that my mouse movement was being recorded in values, the biggest challenge was making sure there was communication from Processing to Arduino.  I definitely made mistakes in this step, as initially I had different numbers of values in each program.  

Processing code:

// IMA NYU Shanghai
// Interaction Lab


/**
 * This example is to send multiple values from Processing to Arduino.
 * You can find the arduino example file in the same folder which works with this Processing file.
 * Please note that the echoSerialData function asks Arduino to send the data saved in the values array
 * to check if it is receiving the correct bytes.
 **/


import processing.serial.*;

int NUM_OF_VALUES = 2;  /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/


Serial myPort;
String myString;

// This is the array of values you might want to send to Arduino.
int values[] = new int[NUM_OF_VALUES];

void setup() {
  size(500, 500);
  background(0);

  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 2 ], 9600);
  // check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index of the port

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
}


void draw() {
  background(0);
  
  values[0] = mouseX;
  values[1] = mouseY;
  //line(mouseX, 20, pmouseX, 80);   
  //println(mouseX + " : " + pmouseX);

  // changes the values
  //for (int i=0; i<values.length; i++) {
  //  values[i] = mouseX;  /** Feel free to change this!! **/
  //}

  // sends the values to Arduino.
  sendSerialData();

  // This causess the communication to become slow and unstable.
  // You might want to comment this out when everything is ready.
  // The parameter 200 is the frequency of echoing. 
  // The higher this number, the slower the program will be
  // but the higher this number, the more stable it will be.
  echoSerialData(200);
}

void sendSerialData() {
  String data = "";
  for (int i=0; i<values.length; i++) {
    data += values[i];
    //if i is less than the index number of the last element in the values array
    if (i < values.length-1) {
      data += ","; // add splitter character "," between each values element
    } 
    //if it is the last element in the values array
    else {
      data += "n"; // add the end of data character "n"
    }
  }
  //write to Arduino
   println(data);

  myPort.write(data);
}


void echoSerialData(int frequency) {
  //write character 'e' at the given frequency
  //to request Arduino to send back the values array
  if (frameCount % frequency == 0) myPort.write('e');

  String incomingBytes = "";
  while (myPort.available() > 0) {
    //add on all the characters received from the Arduino to the incomingBytes string
    incomingBytes += char(myPort.read());
  }
  //print what Arduino sent back to Processing
  print( incomingBytes );
}

Arduino code:

// IMA NYU Shanghai
// Interaction Lab

/**
This example is to send multiple values from Processing to Arduino.
You can find the Processing example file in the same folder which works with this Arduino file.
Please note that the echo case (when char c is ‘e’ in the getSerialData function below)
checks if Arduino is receiving the correct bytes from the Processing sketch
by sending the values array back to the Processing sketch.
**/

#define NUM_OF_VALUES 2 /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/

/** DO NOT REMOVE THESE **/
int tempValue = 0;
int valueIndex = 0;

/* This is the array of values storing the data from Processing. */
int values[NUM_OF_VALUES];

void setup() {
Serial.begin(9600);
pinMode(9, OUTPUT);
pinMode(13, OUTPUT);
}

void loop() {
getSerialData();
tone(9, map(values[0], 0, 500, 0, 1000), values[1]);
//Serial.println(values[0]);
// add your code here
// use elements in the values array
// values[0] // values[1] if (values[0] > 150) {
digitalWrite(13, HIGH);
} else {
digitalWrite(13, LOW);
}
//
// if (values[1] == 1) {
// tone(9, values[2]);
// } else {
// noTone(9);
// }

}

//recieve serial data from Processing
void getSerialData() {
if (Serial.available()) {
char c = Serial.read();
//Serial.println(“gjkg”);
//switch – case checks the value of the variable in the switch function
//in this case, the char c, then runs one of the cases that fit the value of the variable
//for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase
switch (c) {
//if the char c from Processing is a number between 0 and 9
case ‘0’…’9′:
//save the value of char c to tempValue
//but simultaneously rearrange the existing values saved in tempValue
//for the digits received through char c to remain coherent
//if this does not make sense and would like to know more, send an email to me!
tempValue = tempValue * 10 + c – ‘0’;
break;
//if the char c from Processing is a comma
//indicating that the following values of char c is for the next element in the values array
case ‘,’:
values[valueIndex] = tempValue;
//reset tempValue value
tempValue = 0;
//increment valuesIndex by 1
valueIndex++;
break;
//if the char c from Processing is character ‘n’
//which signals that it is the end of data
case ‘n’:
//save the tempValue
//this will b the last element in the values array
values[valueIndex] = tempValue;
//reset tempValue and valueIndex values
//to clear out the values array for the next round of readings from Processing
tempValue = 0;
valueIndex = 0;
break;
//if the char c from Processing is character ‘e’
//it is signalling for the Arduino to send Processing the elements saved in the values array
//this case is triggered and processed by the echoSerialData function in the Processing sketch
case ‘e’: // to echo
for (int i = 0; i < NUM_OF_VALUES; i++) {
Serial.print(values[i]);
if (i < NUM_OF_VALUES – 1) {
Serial.print(‘,’);
}
else {
Serial.println();
}
}
break;
}
}