Author: Kat Van Sligtenhorst

The development of the project involved a lot of back and forth, trying to balance the strong and sometimes abrasive message and the technological aspect, which needed to be simple enough to let the interaction speak for itself. For the survey, I originally had a list of 20 statements, which I edited down to 14 after some users said that the experience stretched on a little long. The goal was for them to be relatively short and simple, both so they could hold users’ attention and so they were answerable by the target demographic, most of whom could be assumed to have a basic working knowledge of the subjects. Within this process, I determined how the monitor should respond to certain answers, and the most effective ways of subtly giving users the sensation that they were being surveilled or that there were “right” or “wrong” responses. I wanted this idea to build over time to create feelings of unease and panic, without ever outright stopping a user from answering as they wished. Although the camera light comes on immediately, most users don’t notice, and the most dramatic reminder of surveillance is when a live video feed is actually flashed onscreen. This made people duck out of view and even want to stop the simulation. This is the danger of self-censorship–that varying methods and degrees of conditioning will push someone to omit, adjust, or change their opinion entirely. In the final step before beginning to code, I sourced images to flash in between statements, some of which were pulled from news coverage, and others that I took myself while in Hong Kong.

As far as other design choices, I knew early on that I wanted the response mechanism to be relatively simple, something where a user can choose only between a yes button and a no button. This is, first, to mimic casting a vote, although I wrapped the ballot box in trash bags to show that an individual opinion does not particularly matter when the CCP is creating policy. This is also to show the lack of much gray area in China in terms of what is acceptable to discuss. It seems that, more often than not, issues are clearly divided into safe topics and taboo topics, and there is very much a right or wrong stance to take. I originally coded the replies with keyPressed just to get everything working, then moved to the push buttons. I also had to adjust the code so that users could only record one answer for each statement, and added a counter so that the “safe” and “warning” tallies would correspond with the responses. For the processing interface, I used simple, black and white typewriter text as well as red flashes, photos, and live video feed (without really storing data) to, again, reinforce the “safe” or “warning” answers. The other consideration was to use more physical elements to drive the interaction, like placing little protestor or police figurines on pressure sensors in order to express an opinion. Ultimately, I decided that less was more, because I didn’t want to distract from the intensity of the statements and the need to make definitive, perhaps controversial, responses to them.

ILFinal

In the end, my project did align with my definition of interactivity as something that goes through multiple cycles of input and output between audience and product, and ultimately challenges the user in some way. Based on my observations of people using it, the experience was intuitive in its design and impactful in its message. It could be argued that the project is not interactive enough, and that the yes and no push buttons could be changed to something more meaningful or engaging. If I had more time to develop this idea, I would like to make it even more immersive, incorporating sound or light displays to draw attention to the user for “wrong” answers.

The most challenging moment of development was when my computer crashed and I lost the majority of my code. This happened on the Friday morning of user testing, which meant I was unable to get the full experience, although I did have a few people try my rewritten code on Monday and Tuesday. All in all, though, I learned a lot about the nuances of Processing, serial communication, and developing an interactive experience that blends technology and social issues in order to challenge an audience.

As I said in my earlier essay, I think this project is unique in that it addresses a particular group of people, who face all the nuances and challenges of attending the first joint US-Sino university. Our student body is in a position to both observe the affairs of China and to bring international perspectives and standards into our considerations of these issues. We have a distinct ability on our campus to discuss and debate topics that are taboo in wider Chinese society. Therefore, my goal was to take real-world current events and issues that are of huge concern to students in our position and force users to reconsider not only why they believe what they do, but how strong those beliefs really are when they are challenged, explicitly or implicitly. Watching people go through the survey, I believe my project did achieve its goal.

For this recitation, I wanted to use two potentiometers to change the size and location of an image in Processing. Here is my code for Arduino:

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);

Serial.print(sensor1);
Serial.print(“,”);
Serial.print(sensor2);
Serial.println();

delay(100);
}

And here is my code for Processing:

import processing.serial.*;

String myString = null;
Serial myPort;
PImage img;
int NUM_OF_VALUES = 2;
int[] sensorValues;

void setup() {
size(800, 800);
img=loadImage(“hongkong.jpg”);
setupSerial();
}

void draw() {
background(0);
updateSerial();
printArray(sensorValues);
float a = map(sensorValues[0],0,1023,0,800);
float b = map(sensorValues[1],0,1023,400,800);
image(img,a,200,b,b);
}

void setupSerial() {
printArray(Serial.list());
// myPort = new Serial(this, Serial.list()[1411], 9600);
myPort = new Serial(this, “/dev/tty.usbmodem1411”, 38400);
myPort.clear();
myString = myPort.readStringUntil( 10 );
myString = null;
sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 );
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}

While my final project is more focused on challenging users mentally rather than engaging them in physical motion and interaction, I still found Computer Vision for Artists and Designers to be a really interesting read. Christian Möller’s project, “Cheese,” was most useful to me because, if I were to incorporate the ideas of computer vision found in this text into my own project, it would be a really cool way to gauge users’ emotions as they went through the survey. He was focused on smiles, but if his “emotion recognition system” were also able to detect unease or discomfort, that would be an excellent addition to my project. The section on motion detection also gave me something to consider, as I could use this strategy to activate the live video feed whenever a user enters the voting booth. It’s fascinating that the reading mentions, “Techniques exist which can create real-time reports about people’s identities, locations, gestural movements, facial expressions, gait characteristics, gaze directions, and other characteristics,” all of which tie into China’s surveillance state. If I wanted to do a project further expanding on critiques of the Chinese government beyond the message of self-censorship, it would be really cool to give users the experience of all of the above, particularly for audiences in other countries that do not deal with quite so heavy surveillance in their day to day lives.

(I was having trouble uploading the screen recordings, so I will go back and add those later).

Credits:

Levin, G. “Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers”. Journal of
Artificial Intelligence and Society, Vol. 20.4. Springer Verlag, 2006.

For this recitation, we were instructed to pick a TV show or music video and recreate it in Processing by manipulating images and clips we found online. I chose The Office theme song. First, I needed to download it as an mp4, load it into the sketch, and set it to play on a loop. Then, to make it a little more fun, I changed the speed of the video so it would play three times as fast. I also used the pixel manipulation techniques that we learned in order to pixelate the video. Lastly, because we had to incorporate the map() function, I added three ellipses along the left-hand side that responded to the movements of the mouse. They are meant to give the video the impression of having been cut out with a three-hole punch, since the TV show is about a paper company.

Here is my code for Processing:

import processing.video.*;

Movie myMovie;
void setup() {
size(480, 480);
myMovie = new Movie(this, “officeIntro.mp4”);
myMovie.play();
myMovie.speed(3.0);
myMovie.loop();
}
void draw() {
if (myMovie.available()) {
myMovie.read();
}

if ( myMovie.available() ) {
myMovie.read();
myMovie.loadPixels();
}
int rectSize = 5;
int w = myMovie.width;
int h = myMovie.height;
for (int y = 0; y < h; y+=rectSize) {
for (int x = 0; x < w; x+=rectSize) {
int i = x + y * w;
fill( myMovie.pixels[i] );
rect(x, y, rectSize, rectSize);
}
}
myMovie.updatePixels();

tint(255, 50);
image(myMovie, 0, 0);

float x1 = map(mouseX, 0, width, 50, 150);
ellipse(x1, 30, 50, 50);

float x2 = map(mouseX, 0, width, 50, 150);
ellipse(x2, 130, 50, 50);

float x3 = map(mouseX, 0, width, 50, 150);
ellipse(x3, 230, 50, 50);
}

Here is the finished product:

theOffice (1)

Credits:

https://www.youtube.com/watch?v=0GXjAMnv1zs

The Self Censor

In my research for this project, I found several meaningful interactive exhibits whose designs or messages assisted me in developing my own. In terms of the message I wanted to convey, the idea was sparked by a project I found during the research stage. “The Listening Post” by Ben Rubin and Mark Hanson used of “a real-time visualization of thousands of ongoing online conversations” that appeared on numerous screens, giving the viewer the sense that they were being watched or monitored even in spaces they had thought were safe or anonymous. For the physical design, I drew on my research of an exhibit called “The Nautilus,” which utilized a sea of human-height, touch-activated poles to create music. I also incorporated takeaways from a conversation with Rudi and Marcela about a past student’s project, which attempted to recreate the experience of a foreigner going through the TSA at a US airport. I knew that I wanted my project to feel immersive, so that the physical space the interaction took place in added to the user’s experience. Therefore, I decided to construct a voting booth that users could step into, where they would then be faced with a monitor and a series of yes or no questions about Chinese politics and their impressions of censorship. Given the recent press on the topic of self-censorship at NYU Shanghai, the goal of this project is to push students to consider what they will or won’t say and why that is. The statement made by this interaction is particularly relevant and applicable to attendees of this university as we live and study in China and, in doing so, give up certain personal freedoms.

The interactive monitor will receive input from the user regarding their opinions on Chinese politics and censorship. They will be asked true/false or yes/no questions, and then instructed to press a key (T/F or Y/N) corresponding with their answer. In between questions, the screen will flash images of controversial events and human rights abuses in China that may be disturbing or unsettling, like riot police in Hong Kong or reeducation camps in Xinjiang. After several responses that are negative towards Chinese interests, the screen will display an excerpt from the student policy handbook that informs the user that NYUSH will not protect them in the case that their online activity violates PRC law. As detailed in the handbook, “The University shall not use its powers to interfere with the rights of a student beyond the University environment. Conduct that occurs off-campus, online, over social media, or outside the context of a University program or activity, should generally be subject only to the consequences of public authority and/or opinion.” In contrast, I will also include sections of NYUSH policy that guarantee students freedom of expression and the right to protest. Questions will be asked regarding collective action, such as, “You draft an Instagram post asking your peers to rally in support of Hong Kong in the campus cafe next week. Press Y to send and N to delete.” Finally, the camera will switch on and briefly display live video of the person taking the poll, so they are aware they are being surveilled. At the end, I will prompt them to record their name to see if people are willing to link themselves to the answers they have chosen, something like, “Thank you for your participation in this survey. Please type your first and last name and press enter. To remain anonymous, press the spacebar.” I want to observe how successful the deterrents are in conditioning the students to choose less inflammatory answers. So far, I have made my full list of questions for the survey, as well as the order of the images to go along with them. I have also written the code for both the questions and the images, and aim to have the live video portion worked into the code by this week. I will ask some people to test it out on December 6 so I can take the weekend to make adjustments based on their experiences.

While the technology behind this project is relatively simple, I think the message will be very impactful. It is unique in that it addresses a particular group of people, who face all the nuances and challenges of attending the first joint US-Sino university. Our student body is in a position to both observe the affairs of China and to bring international perspectives and standards into our considerations of these issues. We have a distinct ability on our campus to discuss and debate topics that are taboo in wider Chinese society. Therefore, this project is significant in that it takes real-world current events and issues that are of huge concern to students in our position and forces us to reconsider not only why we believe what we do, but how strong those beliefs really are when they are challenged, explicitly or implicitly. This project aligns with my definition of interactivity as something that goes through multiple cycles of input and output between audience and product, and ultimately challenges the user in some way. In subsequent iterations of this project, it could easily be tailored to different audiences and topics based on current sociopolitical issues. For example, if I were to present it in the US, I could focus it on impeachment, working in various quotes from Trump or from witness testimonies during the impeachment proceedings. I could also tailor it to the issue of voter fraud, making it difficult for members of certain demographics to make it through the survey successfully. All in all, I think the format is an effective way to push people to challenge their beliefs and convictions, as well as to realize the corruption of governments in any location.