Category: Rudi

During this week’s recitation we were required to control an image or video by manipulating that media’s attributes using a physical controller made with Arduino. I decided to choose my image from Creativity103.com which was given to use in the resource file. I chose a “Christmas Light” image and decided I would use the Filter example, specifically using the “blur” function. On my Arduino board, I used a Potentiometer to manipulate the “blurriness” of the picture I had chosen. 

I began by opening the provided Serial Communication example code given to us. I used the Serial One Value codes as I was only sending a single value.  I then opened up the “filter blur” example and typed in the code onto the ‘one value from AtoP processing’ code. I then went onto the Arduino Serial One Value code and included the map function to transmit my Potentiometer values from Arduino to Processing. To do this, I included an int value: int picture = map(sensorValue, 0, 1023, 0, 300). I then made sure I included ‘picture’ in the Serial.write code. On my Processing code, I had to include PImage and under void setup() I loaded the photo I had used which I named “colors.jpg.”. I wanted to control the range of blurriness depending on which values my Potentiometer was projecting so I included an If statement under void draw() which stated that if(valueFromArduino > 150) then it would filter(BLUR, 2); but if(valueFromArduino <100 then it would filter(BLUR, 5).  Find below video. 

Reading

The article Computer Vision for Artists and Designers draws light to the importance of computer vision through vision-based detection and tracking techniques to show how computer vision can be used in society to benefit its people. Most people assume “computer vision” has no greater aim or purpose and cannot really do anything that will contribute to the betterment of individuals in society, however this article proves this assumption wrong. One of the examples provided that I found was a really smart way to incorporate technology was Rafael Lozano­Hemmer’s installation: Standards and Double Standards (2004). This installation consisted of fifty leather belts which were suspended at waist height with the help of servo-motors located on the ceiling. Through computer vision tracking system, the belts would turn towards an individual depending on their movement. This project metaphorically illustrates the power of the patriarchy in an artistic manner. This had me thinking about how we could use Arduino and Processing to distort/illustrate images to reveal a deeper meaning and convey a deeper message to society. Art through computer vision is an interactive and engaging way to capture the attention of the public and allow them to see the pressing issues faced within society that need to be addressed. 

Resources:

Computer Vision for Artist and Designers

Materials: 

Arduino Kit 

Processing

Media Controller: 

In this recitation we had to have Arduino control a certain aspect of an image from processing. I used a pentameter to control the pixels in my webcam so that the lower the pentameter was the less pixels there were and the higher it was the more complete the webcam image would look. I had to use specific dimensions for the webcam to work with the amount of pixels I wanted to use, since each pixel had to be a certain size to fill the screen perfectly. 

Documentation: 

After reading Computer Vision I understand how computers and humans can interact in ways that both the computer and person can understand what each other are doing. This can be done with the computer being able to understand ones motion or how certain facial features move, and a human can understand the computer because of the visual that it projects. For example, in my recitation I knew that if I move the pentameter my image would become more pixelated, what the computer understood was that the more I moved the pentameter the more pixels it would have to add to the image. Through the visual aid of what the computer projected I was able to understand how the computer would react to something I did, and in turn the computer would react to what I inputed, causing interaction between me and the computer.  

IMG_4462

For the Arduino, I used two potentiometers to adjust an analog value.

For the code: I used the multipleValues code, sending from Arduino to Processing. 

For Processing:  I used Image function and the sensor values are used to change the position of the image.

• code in Arduino:

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.println(sensor2);
delay(100);
}

• code in Processing:

import processing.serial.*;

String myString = null;
Serial myPort;
PImage photo;

int NUM_OF_VALUES = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/

void setup() {
size(1300, 1300);
background(255);
setupSerial();
photo = loadImage(“1.jpg”);
}

void draw() {
updateSerial();
background(255);
printArray(sensorValues);
float x= sensorValues[0];
float y= sensorValues[1];
image(photo, x, y);
}

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);

myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
myString = null;

sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}

PROJECT TITLE

The Truman Show

PROJECT STATEMENT OF PURPOSE

As a filmlover, I’m very interested in new and creative way of telling a story, and I also find myself deeply attracted by interactive forms of storytelling. For example, Sony once launched a horror game called Until Dawn , in which the player keeps making choices that will decide life and death of the characters. My project intends to explore a new way of storytelling, which employs the idea that users are involved in making the story happen. It can also be seen as an art device considering that it delivers a philosophical idea once the user knows how to play it. For the fan of The Truman Show, this design provides them outlet to express their love for it; For those who haven’t watched the film, by interacting with my project, they could grasp the concept of the film without having to know the actual story; For general users, my project can be seen as an escape room game with a reinvented medium.

PROJECT PLAN

In the form of a minisized interactive escape room, my project intends to deliver the concept of the film The Truman Show. The film tells the story of a man who has been unknowningly playing the main role of a reality show since birth, since an island-sized shooting site is built specifically for it. Truman grows up and lives a normal life just like any other people, except that all the people around him are actors and follow a well-designed script. At the end of the film, Truman finds out that his whole life was a lie and was able to escape out of the fake world. The idea is to critique people’s desire to peek into other’s life so much so as to sacrifice a man’s freedom. The project is a minisized escape-room-like box. The user, standing at the viewpoint of the reality show audience, has no access to the interior except a camera installed inside, so he has to interact with the box to be able to see the inside, find the hints, solve 3-4 problems, and finally get the key to open the box and get Truman out of this world. To empathize with the user, I will keep doing research regarding these aspects: players’ feedback about a real escape room, filmlovers’ expectation of film-related products in general, and user feedback about puzzle boxes.

Here attached is a project timeline.

CONTEXT AND SIGNIFICANCE *

Projects employing creative storytelling forms:

Until Dawn

I watched some video bloggers playing this horror game long time ago, but I can never forget the experience. It’s as if you were actually deciding how the story would go on, and it inspires me to associate interaction with storytelling. I try to figure out why it attracts me so much, and an article answers the question, writing “Key decisions and moments have a ‘Butterfly Effect’, and you’re alerted to those moments that have unknown consequences on how things are going to play out. I couldn’t help but get curious when that happened: had I messed up? Had I consigned someone to a gory death? Did I actually have fair control over what was happening?”. Therefore, the interaction includes both sense of control and sense of insecurity, which makes the game a success.

Bandersnatch

This Netflix TV is very similar to Until Dawn, the former is a game-like film while the later is a film-like game.

Both are experimenting on the form of storytelling that invite users to engage with them. The use of interaction here helps tell the story to the audience, instead of telling at them. When the audience aren’t receiving the story passively, the experience might be more fun.

My project differs the ones above in two ways: Firstly, it takes the form of an object ( an escape room), showing a difference from linear way of storytelling. User has full control of the time and pace to explore the room. He can pause at any time to think about the implication of some detail in this project. Secondly, by engaging with the story of The Truman Show, the project is more profound in meaning. It criticizes the disrepect of people’s privacy and fundamental rights. Since only until the user shuts down the webcam installed inside can he successfully open the escape room, the answer to “What can set Truman free” is clear, which makes the project more than merely a game or a story that “looks fun”. My project faces a large range of audience. For the fan of The Truman Show, this design provides them outlet to express their love for it; For those who haven’t watched the film, by interacting with my project, they could grasp the concept of the film without having to know the actual story; For general users, my project can be seen as an escape room game with a reinvented medium.

My project, if completed successfully, opens more possiblities in this form of storytelling. By installing more devices and adding more details, there can be multiple endings—-Truman might get out, might not be able to get out even if he wants to, and he might even choose to stay in that world forever though well aware of what that is (like taking the blue pill in The Matrix). The targeted audience remain the same, but more people who have never watched the film would be able to know the plots just as much as those who have watched, if more game-like interactions are provided.

Reference

1. Until Dawn game review: https://www.denofgeek.com/games/until-dawn/37259/until-dawn-the-interactive-movie-and-storytelling

2. Bandersnatch vice report: https://www.vice.com/en_us/article/7xnq3a/netflix-more-interactive-movies-like-bandersnatch-rom-coms-todd-yellin-interview-vgtrn