Recitation 9: Media Controller (Katie)

I just went to my favorite singer’s concert  and I thought the lighting in the concert is very suitable for the tint function. I set the blue and green value to random and use one potentiometer to control the red value. An important thing is to map the values from the potentiometer to  (0,255).

This is the final result:

This is my processing code:

import processing.serial.*;
import processing.video.*;
Movie myMovie;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 1;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/


void setup() {
  size(1280, 1080);
  //myMovie = new Movie(this, "dancing.mp4");
  myMovie = new Movie(this, "hcy.MOV");
  myMovie.play();
  setupSerial();
  
}


void draw() {
  updateSerial();
  printArray(sensorValues);

float hcy = map (sensorValues[0],0, 1023, 0,255);

  // use the values like this!
  // sensorValues[0] 

  // add your code
if (myMovie.available()) {
    myMovie.read();
  }
  tint(hcy, random(255), random(255)); 
  image(myMovie, 0, 0);
  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 1 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}





this is my Arduino code

[code] // IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
//int sensor2 = analogRead(A1);
//int sensor3 = analogRead(A2);

// keep this format
Serial.print(sensor1);
//Serial.print(“,”); // put comma between sensor values
//Serial.print(sensor2);
//Serial.print(“,”);
//Serial.print(sensor3);
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}
[/code]

Reflection about the ways technology was used in your project:

After reading the texts, I realized that technology is just one approach to convey my meaning in the project, to enhance the interactive experience. technology itself is not the main part of the project, the most important things are first, the message behind. Second, the user’s experience. 

In the text, it introduced Rafael Lozano­ Hemmer’s installation Standards and Double Standards (2004). I like this project because first it involves full-body movement. Second, it’s thought-provoking. Are those belts a representative of humans? The first time encounter it, I was not impressed by how fancy technology is used in this installation but to think about the deeper meaning behind this project.

In our final project, we also want users to have full-body interaction so that we plan to use multiple sensors. Then, we want users to think about the relations between human and nature while experiencing this project.

Recitation 9: Controlling Media by ChangZhen from Inmi’s Session

Preview of project:

Player draws a picture by controlling potential meters. Arduino analogReads the two potentials and serially communicates them to Processing to be used as X and Y coordinates. Processing loads image, JuHunKim.jpg, gets color with random coordinate variations from and around (X, Y) of the image, and draws circle faces of random size with the color it just got respectively onto the canvas my project newly sets at the corresponding coordinates. Moreover, player can only draw the left half of the canvas since the right half will be mirrored from the image.

Arduino:

void setup() {
Serial.begin(9600);
pinMode(A0,INPUT);
pinMode(A1,INPUT);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
Serial.print(sensor1);
Serial.print(“,”);
Serial.println(sensor2);
}

Processing:

import processing.serial.*;
String myString = null;
Serial myPort;
int[] sensorVal = new int[2];
int x;
int y;

PImage img;

void setup() {
printArray(Serial.list());
size(870, 674);
noStroke();
img = loadImage(“JuHunKim.jpg”);
setupSerial();
}

void draw() {
updateSerial();
printArray(sensorVal);
x = sensorVal[0]*435/1023;
y = sensorVal[1]*674/1023;
for(int i=0; i<50; i++) {
int var = int(random(-50, -1));
int dev = int(random(-50, 50));
color c = img.get(x+var,y+dev);
fill(c);
int diam = int(random(1,10));
circle(x+var, y+dev, diam);
circle(870-x-var, y+dev, diam);
}
}

void mousePressed() {
saveFrame(“JuHunna.png”);
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 );
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == 2) {
for (int i=0; i<serialInArray.length; i++) {
sensorVal[i] = int(serialInArray[i]);
}
}
}
}
}

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 3 ], 9600);
myPort.clear();
myString = myPort.readStringUntil( 10 );
myString = null;
}

Electrical Diagram:

Final Effect:

Player may mousePress to save the drawn image as JuHunna.png, a higher ordered art piece derived from 김주훈’s model photo.

Final Project Essay—-Tao Wen

PROJECT TITLE

The Truman Show

PROJECT STATEMENT OF PURPOSE

As a filmlover, I’m very interested in new and creative way of telling a story, and I also find myself deeply attracted by interactive forms of storytelling. For example, Sony once launched a horror game called Until Dawn , in which the player keeps making choices that will decide life and death of the characters. My project intends to explore a new way of storytelling, which employs the idea that users are involved in making the story happen. It can also be seen as an art device considering that it delivers a philosophical idea once the user knows how to play it. For the fan of The Truman Show, this design provides them outlet to express their love for it; For those who haven’t watched the film, by interacting with my project, they could grasp the concept of the film without having to know the actual story; For general users, my project can be seen as an escape room game with a reinvented medium.

PROJECT PLAN

In the form of a minisized interactive escape room, my project intends to deliver the concept of the film The Truman Show. The film tells the story of a man who has been unknowningly playing the main role of a reality show since birth, since an island-sized shooting site is built specifically for it. Truman grows up and lives a normal life just like any other people, except that all the people around him are actors and follow a well-designed script. At the end of the film, Truman finds out that his whole life was a lie and was able to escape out of the fake world. The idea is to critique people’s desire to peek into other’s life so much so as to sacrifice a man’s freedom. The project is a minisized escape-room-like box. The user, standing at the viewpoint of the reality show audience, has no access to the interior except a camera installed inside, so he has to interact with the box to be able to see the inside, find the hints, solve 3-4 problems, and finally get the key to open the box and get Truman out of this world. To empathize with the user, I will keep doing research regarding these aspects: players’ feedback about a real escape room, filmlovers’ expectation of film-related products in general, and user feedback about puzzle boxes.

Here attached is a project timeline.

CONTEXT AND SIGNIFICANCE *

Projects employing creative storytelling forms:

Until Dawn

I watched some video bloggers playing this horror game long time ago, but I can never forget the experience. It’s as if you were actually deciding how the story would go on, and it inspires me to associate interaction with storytelling. I try to figure out why it attracts me so much, and an article answers the question, writing “Key decisions and moments have a ‘Butterfly Effect’, and you’re alerted to those moments that have unknown consequences on how things are going to play out. I couldn’t help but get curious when that happened: had I messed up? Had I consigned someone to a gory death? Did I actually have fair control over what was happening?”. Therefore, the interaction includes both sense of control and sense of insecurity, which makes the game a success.

Bandersnatch

This Netflix TV is very similar to Until Dawn, the former is a game-like film while the later is a film-like game.

Both are experimenting on the form of storytelling that invite users to engage with them. The use of interaction here helps tell the story to the audience, instead of telling at them. When the audience aren’t receiving the story passively, the experience might be more fun.

My project differs the ones above in two ways: Firstly, it takes the form of an object ( an escape room), showing a difference from linear way of storytelling. User has full control of the time and pace to explore the room. He can pause at any time to think about the implication of some detail in this project. Secondly, by engaging with the story of The Truman Show, the project is more profound in meaning. It criticizes the disrepect of people’s privacy and fundamental rights. Since only until the user shuts down the webcam installed inside can he successfully open the escape room, the answer to “What can set Truman free” is clear, which makes the project more than merely a game or a story that “looks fun”. My project faces a large range of audience. For the fan of The Truman Show, this design provides them outlet to express their love for it; For those who haven’t watched the film, by interacting with my project, they could grasp the concept of the film without having to know the actual story; For general users, my project can be seen as an escape room game with a reinvented medium.

My project, if completed successfully, opens more possiblities in this form of storytelling. By installing more devices and adding more details, there can be multiple endings—-Truman might get out, might not be able to get out even if he wants to, and he might even choose to stay in that world forever though well aware of what that is (like taking the blue pill in The Matrix). The targeted audience remain the same, but more people who have never watched the film would be able to know the plots just as much as those who have watched, if more game-like interactions are provided.

Reference

1. Until Dawn game review: https://www.denofgeek.com/games/until-dawn/37259/until-dawn-the-interactive-movie-and-storytelling

2. Bandersnatch vice report: https://www.vice.com/en_us/article/7xnq3a/netflix-more-interactive-movies-like-bandersnatch-rom-coms-todd-yellin-interview-vgtrn

Final Project Essay by Amy DeCillis

Statement of Purpose

Title: Line RiderS

With this project I hope to bring Line Rider to life and create an even more interactive experience for players. I want to get people on their feet, away from their personal laptops, and engaging with others. 

Project Plan

I hope to not only create a multiplayer version of Line Rider, but also one that incorporates two teams with an objective. The original online experience only requires one user to draw a line, or course, for the rider with no real goal. My version will include four players on each team. Each player will influence a different aspect of the game: the line, the speed of the rider, the size of the rider, and the x position of the rider. Each team will work together to place the rider on a target spot in the middle of the screen before the other team. 

I plan on using the following sensors:

    • joystick to draw the line
    • potentiometer for the speed of the rider
    • distance sensor for the size of the rider
    • a crank tool (need to find the specific name for this but essentially a potentiometer that rotates 360degrees) for the x value of the rider

Initially I will work on connecting different sensors to my Arduino and fleshing out the foundational code for the game like drawing the line, creating a target, etc. Then I will move on to designing the cases or covers for the different sensors and potentially a control board for each team. Finally, once everything is working smoothly and everything is put together, I’ll hopefully get some feedback from user testing and adjust accordingly. 

Project’s Context & Significance

I grew up watching my two older brothers take turns playing this game while I waited not so patiently beside them. I think that while online games are very engaging and fun, this is a unique experience to bring that game into the physical world and bring it to a new level of interaction. I still lean on Crawford’s understanding of analysis of interaction and its focus on the listening, thinking, and speaking aspects. In this experience, each user is not simply acting and reacting in one way, but rather collaborating with one another and engaging in thoughtful, continuous conversation.

Recitation 9: Media Controller by Ian (You Xu)

During this recitation, I intend to make a form of distortion of the capture of the live camera. Therefore, I referred to the caption sample code, processing website, and hooked up a potentiometer to achieve this goal.

  1. Green tint and low frame rate

When I was trying to run the sample code of capture, the screen has a green tint and refreshes very slow.

Green tint

Since the same code usually works in others’ devices, I was very confused. I thought it might because my computer – Surface Pro – has multiple cameras that work for different functions. I referred to the Processing website for capture: https://processing.org/reference/libraries/video/Capture.html and displayed all the available cameras for me to choose.

cameras

However, I tried nearly all of them, and no one worked. I looked it up online and found there is an “issue” under Processing’s Github repository https://github.com/processing/processing-video/issues/72. Many people who are using Surface with Windows 10 are encountering the same problem, and they do not have the solution to it. They believe it is the software problem of Windows 10 on Surface. Therefore, temporarily, I cannot fix it.

  1. Moving

By applying the sample code of capture to the serial communication sample code, I easily get the data from the potentiometer to the processing and use it to adjust the position of the capture. I use this step as a test of applying video function to serial communication. Again, one problem I found is that I need to be careful about the background refresh in the draw loop. This time, I do not want the previous images to leave on the screen, so I decided to draw the background in every loop.

  1. Scaling

I checked the capture mirror sample code and found out that it uses a “scale” function to make it the mirror effect. It gives me the insight to use the input from the potentiometer to control the scaling index. At first, I simply add “scale(sensorValues[0]);” to the code. However, I found out that the data from the potentiometer is ranging from 0 to 1023. It may scale the capture over 100 times bigger, which is too big.

Too big Scaling

Then, I realized I need to map it into a reasonable range first. I choose 0 to 2 here where 0 to 1 is shrinking, and 1 to 2 is amplification.

  1. Translate

However, this is still not what I expected. I want the capture always in the middle of the screen. Then I recalled another function “translate” that I used before to define that new origin. This required me to do math calculations. And I figure it out by geometric operation.

  1. Two dimensions

It works well. So, the final step is to hook up another potentiometer and make one of them scaling the x-axis and another to scale the y-axis.

2 potentiometer

Reflection

In the article “Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers” (Levin). it describes “new practitioners with an abundance of new application ideas, and the incorporation of computer vision techniques into the design vocabularies of novel artworks, games, home automation systems, and other areas.” I found two projects that use the capture function to build an authentic, interactive experience with users where they can see themselves. They are Cheese and LIMBOTIME. Therefore, I think to capture in the interactive project can increase the inclusiveness for the audience to engage their intention and communication with the project, which is correspondent to my definition of interaction that addresses the constant impact and inclusive communication environment. My work in this recitation would entertain the audience that they can see the distorted face in whatever dimension by interacting with the potentiometer. The only pity is the green tint bug that I am unable to fix.

Code: https://github.com/xuyou1999/InterLab_Fall_19/tree/master/Recitation_9_Code

Works Cited

Levin, Golan. “Computer vision for artists and designers: pedagogic tools and techniques for novice programmers.” AI & SOCIETY 20.4 (2006): 462-482.