Category: Interaction Lab

Posted on

Recitation 9: Media Controller – Zhao Yang

Code:

https://github.com/JooooosephY/Recitation-9

Introduction:

For this recitation class, we were required to use Arduino to send values to Processing to interact with some media, such as images or videos. I was interested in making some instant interaction so I chose the live camera as the media that I would change. Basically, it’s like an exercise that I used the Arduino to change the filter of the camera. Here is the video about how I interacted with the camera and an image of the circuits. 

Reflection:

I first looked up the filter function to check what kind of filter I can use. Then I found the filter POSTERIZE. Since it requires a  parameter to change the filter, I decided to use the potentiometer to change the value. I thought that if I only use the potentiometer as the tool to interact, it would be too boring. Thus, I added another button to control whether to use the INVERT filter. Fortunately, the output was pretty interesting. However, when I finished the exercise, I felt the way of interaction is still too simple. According to Golan Levin’s article, he notes that “Myron Krueger’s legendary Videoplace, developed between 1969 and 1975, was motivated by his deeply felt belief that the entire human body ought to have a role in our interactions with computers” (Levin 1). It is the first interactive artwork to incorporate computer vision. I really appreciate the Krueger’s idea that the entire human body ought to have a role in our interactions with computers. Since using buttons or potentiometer to interact are just the interaction with your fingers, the use of sensors could be a better way to create an interactive project. In my opinion, the values that the sensors can send are actually from the real world, while the values that some buttons send are the values inside the computer. So you can interact with sensors through your body. And some projects that use sensors as the input are usually more interesting than the projects that only use buttons as the input. Therefore, when I make the final project, I will try to make it a project that can involve the user’s whole body to interact with so that the project can be a good entertaining project. 

Posted on

Final Project Essay

PROJECT TITLE:  Snack Facts 

PROJECT STATEMENT OF PURPOSE :

The primary focus of my project is to educate people on the larger consequences and implications of their food choices.  This topic is meaningful because an individuals dietary choices have an even greater impact on green house gas emissions than their transportation choices and this is a problem that has extreme relevance and correlation to our influence on the environment and global warming.  The audience this project will be angled towards is primarily my classmates, but also the general public or those who frequently consume meats or animal products.  It is my belief that this will be relevant to everyone, because everyone eats food and everyone– in my opinion– should be aware of the consequences of their actions.  Because I am very passionate about starting a conversation regarding the impacts that our food choices have on not only the environment, but also vulnerable populations as it relates to global warming and food security, I think that my project has the potential to create an interactive (and edible) experience for audiences to learn more about where our food comes from and what this means to the rest of the world.  In terms of the research I have done that has shifted my conception of this plan, I hope to create an experience of “shopping” at a grocery store or supermarket that allows users to choose from a variety of options, just as they would in a store.  Additionally, I plan to carefully observe the ways that they market to consumers and keep the realities of where certain foods come from intentionally very well hidden.  

PROJECT PLAN :

My project aims to create a parallel experience to shopping at a grocery store in that the audience will be presented with choices that seem as if they have no larger impact on the world.  By using shelves that look similar to what would be found in a supermarket, and having packaged, edible products such as meat, fish, dairy, and vegan foods that are accessible to consumers, I hope to allow people to learn of the consequences of the choices they make.  Consumers will make their choice and then be presented with a number of interactive experiences, including but not limited to visual pictures of where the food came from (factory farms, as well as audio sounds of animals, etc.  Each packaged item will have a colored tag that can be scanned with a color sensor that is connected to my Arduino, which then translates to a processing display on my computer.  I have already begun using the color sensor with processing so that after recognizing a particular color (similar to scanning a barcode at the checkout line of a supermarket) consumers can see and interact with the visual and audio consequences of their choices, then decide if they wish to continue with their “purchase”.  In this way, I hope to allow for time between when the consumer finalizes their decision and sees the disturbing images associated with their choices.  I still need to further my research to have enough material so that each item can have food facts, images, sounds that are connected to the process of how this item got from “farm” to consumer.  I also need to continue to program particular images to be prompted after the scanning of different colors, as well as create digitally fabricated tags or other constructions that help to make consumers feel as if they are really interacting within the atmosphere of a traditional grocery store.  I will also need to finalize my decisions on which specific foods to feature and have available on my shelves, and find more information on the detailed processing systems of each food.  

In terms of emphasizing with the audience and adjusting to their needs in the experience overall, I hope to make this project as a way to start the conversation of why our food choices matter.  I am working on ways I can make it more of an educational experience and link the foods with which we are familiar to a greater societal and global significance, rather than simply telling someone that they should not eat something.  When further thinking about the consumer and my particular audience of classmates, I want to make sure that this does not come off too much like a lecture or a “do this, not that” but rather a “did you know where that comes from and what impact that has on the environment?”  It is my hope that much of this information will not necessarily be new to everyone, however, it may not be something we see and deal with on a regular basis, and I hope to change that.  It is my goal to bring the disconnect between our food systems and consumerism a tiny bit closer to a shared understanding, rather than a blindness of the results of industrial animal agriculture.  

CONTEXT AND SIGNIFICANCE :

The preparatory research helped to inspire my final conception of this project in that it forced me to consider different routes of interactivity and think about the things that really matter to me.  While the research I did following the assignment of this specific project was primarily design based (as I initially though I was going to create something as an art project or that uses colors to translate to other things) through this research I learned (or rather was forced) to narrow my focus to the things I genuinely like talking about and studying, and how I can use IMA and the forms of interaction we have studied to better understand or communicate this interest.  Following this research, I thought deeply about my senior capstone (obviously a interest and passion of mine), which focuses on nutritional shifts throughout history and how we can better motivate a shift towards plant based food systems.  Because of my research on this subject over the past several months and even the recent years– in addition to the fact I have been vegetarian for more than a decade and a vegan for 3 years– I was inspired to incorporate my knowledge and research in this area into a project that utilizes technology such as Arduino and Processing to better teach people about the challenges and complications with our dietary choices.  Because food is so inherently linked to our physical health, and well as incredibly meaningful to our social, traditional, and mental wellbeing, it is exciting and innovative to think about the ways that I can use interaction and technology to expand our understanding to why we eat what we eat (and what the consequences of these actions tangibly look like).  

Because my project is intended to help us better see the direct impacts– in the form of audio and visuals as well as text– of our dietary choices, I think anyone and everyone can be a meaningful and relevant target audience.  When specifically focusing on my classmates, and how to talk to friends and fellow students about what food means, I think using this level of edible and educational interaction very clearly helps to bridge the extreme distance we have when we are mindlessly shopping at stores and making choices about food.  I would consider this project to be a success if it sticks in people’s minds in the days following and maybe even if they reconsider a meal because of this.  While I am not trying to tell everyone to be vegan (even though I do somewhat want to; and in my opinion you can’t truly be an environmentalist if you’re not plant based so as we are quickly approaching irreversible climate change, being plant based is among the biggest and most significant changes one can make to take action), I am trying to force people to rethink their simple, everyday food choices and what it means in a global context.  The goal is not to be perfectly plant based; instead the goal is to think.  Every single 菜包, slice of pizza, and chicken nugget leaves an impact.  What consequences do you want to be responsible for?  Ultimately, if this project becomes successful on a broader scale I think it would be really cool if supermarkets were required to share this kind of horrifying, disgusting, and truly significant information with consumers, similar to the ways that cigarette packages put pictures of cancerous lungs on packs.  What if rather than a serene pasture that is seen on the package of raw beef we saw slaughter houses, heard the cries and screams of tortured animals, or considered the public health statistics on developing antibiotic resistance? 

Posted on

Recitation 9 Documentation

For recitation 9, we were to use Processing to manipulate a form of media, either an image or a video, with a physical controller corresponding with Arduino. Using the given code from the examples, we modified the code from the 01_oneValue_arduino_to_processing to correctly communicate the code from Arduino to Processing’s media output.

Using the functions pushMatrix() and popMatrix() from this week’s classes, I was able to make the image move and interact with the potentiometer from Arduino. These two functions allowed the image to be transformed, with pushMatrix() saving the current coordinate system to the stack, while popMatrix() restored the prior coordinate system. This allowed the image to jump from one place to another, in conjunction with the media processing from Arduino. 

Since I only used one potentiometer to control the input of the image, Arduino only reads one value to Processing. Ergo, I used the oneValue_arduino_to_processing example from our classes. I incorporated analog input via Arduino through the use of a potentiometer, then serially connected Arduino to Process to manipulate an image of my hamster. The potentiometer would adjust the image size by twisting it, reading the variables as an analog value and sending the input values as output values in Proccesing. 

Code for Processing:

import processing.serial.*;

Serial myPort;
int valueFromArduino;
PImage img1;
color mouseColor;
float mouseR, mouseG, mouseB;

void setup() {
  size(600, 600);
  img1 = loadImage("hamster.jpg");
  background(126, 137, 155);

printArray(Serial.list());

  myPort = new Serial(this, Serial.list()[3], 9600);
}


void draw() {
  while ( myPort.available() > 0) {
    valueFromArduino = myPort.read();
  }
  println(valueFromArduino);
  pushMatrix();
  translate(0, 0);  
  rotate(radians(map(valueFromArduino, 0, height, 0, 500)));
  
  image(img1, 100, 100, width/(valueFromArduino+1), height/(valueFromArduino+1));
  popMatrix();
  mouseColor = img1.get(mouseX, mouseY);
  mouseR = red(mouseColor);
  mouseG = green(mouseColor);
  mouseB = blue(mouseColor);
  println(mouseR+" "+mouseG+" "+mouseB);
  set(width/2, height/2,mouseColor);
}

Code for Arduino:

This week’s reading, Computer Vision for Artist and Designers, discussed the scope of applications that modern computer vision technologies has, especially in the field of military and law enforcement. But in recent years, an increase in the use of non-traditional computer programming applications such as Processing has contributed to the rise of a visual focus in the electronic arts and visual design communities. By using Processing, we are able to focus more upon the visual elements within coding, and therefore able to explore a multitude of technologies in order to execute such projects. Particularly in this recitation’s project, we used technology to serially connect images within physical hardware. Processing allows us to execute machine vision techniques within a digital environment, as it is extremely well­ suited to the electronic arts and visual design communities (Golan et al., 7). This application can be expanded into much more expansive projects that similarly connect media input and output, with these tools commonly seen “to create art installations, performances, commercial kiosks, and interactive industrial design prototypes” (Golan et al., 7). TeamLab Borderless’ exhibit is one such example, in which technology allowed for a much more visually emphasized production rather than the coding behind it.

Posted on

Recitation 9 by Jackson Pruitt

Moving image with a varied tint.

Processing:

import processing.serial.*;
import processing.video.*;
Movie myMovie;

Serial myPort;
int valueFromArduino;

void setup() {
  size(500, 500);
  background(0);

  myMovie = new Movie(this, “dancing.mp4”);
  myMovie.play();

  printArray(Serial.list());
  // this prints out the list of all available serial ports on your computer.

  myPort = new Serial(this, Serial.list()[ 12 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port “/dev/cu.usbmodem—-” or “/dev/tty.usbmodem—-”
  // and replace PORT_INDEX above with the index number of the port.
}

void draw() {
  // to read the value from the Arduino
  while ( myPort.available() > 0) {
    valueFromArduino = myPort.read();
    // fill (250);
    //ellipse(200,200,valueFromArduino,valueFromArduino);
  }

  if (myMovie.available()) {
    myMovie.read();
  }
  tint(valueFromArduino, 0, 0);
  image(myMovie, 0, 0);

  println(valueFromArduino);//This prints out the values from Arduino
}

Arduino:

// IMA NYU Shanghai
// Interaction Lab
// This code sends one value from Arduino to Processing

void setup() {
Serial.begin(9600);
}

void loop() {
int sensorValue = analogRead(A0) / 4;
Serial.write(sensorValue);

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(10);
}

Reflection:

In the reading Computer Vision for Artists and Designers, there is a quote that reads, “Processing is one such environment, which, through an abundance of graphical capabilities, is extremely well­suited to the electronic arts and visual
design communities. Used worldwide by students, artists, designers, architects, and researchers for learning, prototyping, and production, Processing obtains live video through a QuickTime­based interface, and allows for fast manipulations of pixel buffers with a Java­based scripting language.” Levin, Raes, and Fry are articulating the potential processing has in the visual media arts which is exactly what we took part in during the recitation exercise. Although I was only able to complete the task using one sensor to manipulate one aspect of the video, I feel that this could be further utilized in more advanced software technology such as automizing color grading in film or photography.

Posted on

Recitation 9: Media Controller–Lifan Yu

Recitation 9: Media Controller

Lifan Yu (ly1164)

       In this recitation I changed the parameters of an image I painted, using Arduino and processing.

  1. I used the function “tint()” to change the colors. I used potentiometers to do this through serial communication.
  2. Also, when a button is pressed, my image starts moving and bounces when it touches edges.
  3. I used another potentiometer to control the blurring of this image using filter(BLUR, b);
  4. I used one more potentiometer to control zooming by adding the array of potentiometer data into my image’s size parameters:  float n=map(sensorValues[1],0,1023,0,500);  image(img1, x, y, n, n);

Zoom in and zoom out:

Tint:

Before applying these data, I mapped the original data from Arduino to suitable ranges for each parameter.

Code for Arduino:

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing

void setup() {
Serial.begin(9600);

}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
int sensor3 = digitalRead(6);
int sensor4 = analogRead(A3);

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.print(“,”);
Serial.print(sensor3);
Serial.print(“,”);
Serial.print(sensor4);
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Code for Processing:

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 4;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/
PImage img1;
float m=0;
float n=0;
float x=0;
float y=0;
float speedX=10;
float speedY=10;


void setup() {
  size(1000, 1000);
  background(0);
  setupSerial();
  img1 = loadImage("Witch.png");
}


void draw() {
  updateSerial();
  printArray(sensorValues);

  // use the values like this!
  // sensorValues[0] 
  imageMode(CENTER);
float m=map(sensorValues[0],0,1023,0,500);
  float n=map(sensorValues[1],0,1023,0,500);
  float b =map(sensorValues[3],0,1024,0,10);
    
  // add your code
  background(0);
  tint(m,m,m,m);
  
image(img1, x, y, n, n);
filter(BLUR, b);

if (sensorValues[2]==0 ){
  x=300;
  y=300;
}else{
x=x+1.7*speedX;
y=y+speedY;
if(x>width||x<0){
  speedX=-speedX;
}
if(y>height||y<0){
  speedY=-speedY;
}
}

  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 0], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

       The way that technology is used in my project is the serial communication that allows users to change parameters of a picture without actually typing in those data in a computer. Rather, this process is done by interacting with physical devices, changing their input values–indirectly but simply and conveniently changing parameters into whatever values we like. This only requires an extra step of mapping values into new ranges (that fit their parameters) after we have built up a serial communication between Arduino and Processing. Using this technology, even people who don’t know much about computer programming can easily interact with that image and change it’s features.