Author: Bing Chen

My Heart Beats for the World – Bing Chen – Marcela

Partner: Danial Zamiri

Conception and Design: Going into the design phase of this project, we knew the user would interact with this project by mainly watching. In addition, we wanted to measure their heart rate as they watched a series of images, because of this, we wanted to reduce the amount of confounding variables as much as possible. So we made a setup that involves as little movement as possible for the user. This included: a chair to sit on, a box where the user would put the hand clipped to the heart rate sensor so they wouldn’t move it during the interaction, and a heart rate sensor with a clip so the readings would be more accurate. Originally, we were going to make our own heart rate sensor but due to material restraints (not having access to the right materials) and time restraints, we opted to use a sensor that already exists in the world so we can focus on other aspects of our project. We tried having the user stand during the interaction but the way they have to look down at the screen while watching would create strains on the neck, leading to stress and might affect the heart rate by increasing it as the interaction progresses, so we rejected that setup. We also chose the box for the hand because we could hide all the cables and circuit inside to make our project more presentable. Overall, we are satisfied with our design. 

Fabrication and Production: Step 1 of our project was to get the heart rate sensor up and running. First problem we ran into was material restraints and so we opted out of creating our own heart rate sensor and instead used one that already exists. Next we worked on the images to play for the slideshow. Our next problem came when we couldn’t get the images to display, turns out this was because Processing doesn’t update the canvas until the end of every draw() loop. And since the code for the images was inside a separate loop, the initial draw() loop never ends. We adjusted for that and the rest of it came together. We had a little problem with setting up a timer but that was easily fixed with some help from fellow Nick. Then we worked on displaying the results. After some feedback from Marcela, we decided that we wanted to print the result from an external printer. But because we didn’t plan on this from the very beginning, we didn’t consider how making this happen would be kind of difficult with the way our code and setup was at the moment. Because we were using serial communication to send the heart rate values from Arduino to Processing, we couldn’t send anything back (namely, the results) to Processing using the same port. With some help from fellow Tristan, we learned to use something called Call and Response and made the communication two-way. 

During user-testing, some feedbacks we got were concerning the results and the overall display. People were concerned that most people are getting the first image as their result, some people were distracted by their changing heart rate and paid more attention to that than the actual images themselves, some people suggested that we should use different quotes for each image. This lead to some adjustments: we got rid of the heart rate displays, made the grace period longer at the beginning, and put in more quotes for the results. This, however, led to another problem: the lack of memory space. Turns out, Arduino doesn’t really have a lot of storage. And all of the string bytes for the quotes brought us over 162% storage capacity. Because we were down to the last 2 hours we could work on this, we decided to shorten the number of quotes instead of utilizing an SD card. This was a bad decision… After we got the memory to less than 100% capacity, we retested our program and the serial communication ended up not working AT ALL. We theorized that it could be because we were using char values before but we changed it to int to better accommodate the results Arduino would be getting from Processing. Since before, Processing would just send 1 value to Arduino and it would print the same quote for everything. Now, Processing has to send the value that corresponds with the image so Arduino knows which quote to print. This led us to over 2 hours of reprogramming the entire code, changing everything regarding the image number to char. When we eventually finished, we tested it again… And the same thing happened. We spent 2 hours fixing nothing! Turns out the problem the entire time was the storage, because we were so close to max capacity (98%), the serial communication was actually still working but just so slowly that it didn’t work well enough for our project. So the lesson is, if given the chance to expand storage, always do it. 

Conclusions: The goal of my project was to create a slideshow of images regarding serious world issues while measuring the user’s heart rate as they watch the slideshow. The goal was to bring awareness to these issues and encourage users to think and talk about them even after they finish interacting with it. My project aligns with my definition of interaction in the way the user reacts to these images with their changes in heart rate and what they think about during the slideshow. It encourages further interaction by making the users think about these issues long after the time they watched the slideshow. It doesn’t align with the interaction in most of the other projects, however, because it isn’t the typical kind of physical interaction we see a lot. The user just sits there and watches and listens. Our audiences approves of our project, saying that it is “very deep” and “an interesting way of interaction.” 

If we had more time, we would have expanded the displaying of the results by adding a graph of what their heart rates were at the peak of every image. We would also increase our storage space with an SD card. We learned that taking the shortcut out of a problem may lead to even more problems. We also learned that no matter how early we start, or how much time we put into a project, there will always be problems to solve and ways to make it better. The difference just comes down to how dedicated and patient we are with our work. 

In this recitation, I attended the serial communications workshop. In this workshop, I learned how to send multiple values from Arduino to Processing. The values were read from 3 different inputs: 2 potentiometers and 1 push button. The potentiometers sent analog values and the push button sent digital values. I learned that the way to send both types of values is the same. Then I learned how to use these values sent from the sensors, send it to Processing, and apply in the sketch. The two potentiometers change the x and y value of an ellipse. Whenever the push button was pressed, a rectangle would show up where the mouse is. This is one of the many different applications of serial communications possible with Processing and I will use what I learned today to help me send analog values from a pulse sensor to Processing for my final project. 

Processing code:

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
* Based on the readStringUntil() example by Tom Igoe
* https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
*/

import processing.serial.*;

String myString = null;
Serial myPort;

int NUM_OF_VALUES = 3; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/

void setup() {
size(500, 500);
background(0);
setupSerial();
}

void draw() {
updateSerial();
printArray(sensorValues);

background(#ffffff);
fill(#000000);
ellipse(map(sensorValues[0], 0, 1023, 0, width), map(sensorValues[1], 0, 1023, 0, height), 50, 50);

if (sensorValues[2] == 1) {
fill(#ff0000);
rect(mouseX, mouseY, 100, 100);
}
}

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[3], 9600);
// WARNING!
// You will definitely get an error here.
// Change the PORT_INDEX to 0 and try running it again.
// And then, check the list of the ports,
// find the port “/dev/cu.usbmodem—-” or “/dev/tty.usbmodem—-”
// and replace PORT_INDEX above with the index number of the port.

myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
myString = null;

sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}

Arduino code:

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing

int buttonPin = 2; // digital pin 2
int buttonState = 0;
int ledPin = 13;

void setup() {
Serial.begin(9600);
pinMode(2, INPUT);
pinMode(ledPin, OUTPUT);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
buttonState = digitalRead(buttonPin);

if (buttonState == HIGH) {
// turn LED on:
digitalWrite(ledPin, HIGH);
} else {
// turn LED off:
digitalWrite(ledPin, LOW);
}

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.print(“,”); // put comma between sensor values
Serial.print(buttonState); // 1 or 0
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Julie’s project is about creating a tapping game that measures how precisely the player pushes a button at a specific time and for a specific length. This project is interesting because it sounds like a fun game, especially because it is based off a similar game that already exists that is already established as fun. The feedback we gave her was that she should make the game more interactive by making it multiplayer; instead of just one person playing, it should be a team player kind of game. This game is significant in the way we interact with technology because the original game it is based on involves tapping on a screen. Using physical buttons takes this game back to the retro period and brings a sense of nostalgia not many games bring. 

Kathy’s project is about making an interactive car that forces the user to get up and interact with it in order to keep them from a computer screen for a few minutes so they will have a healthier lifestyle. This project is interesting in its concept because it shows that interaction doesn’t have to be fun. The user will most likely become very annoyed with the car but the fact that they were willing to get such a product shows their desire to reduce their screen time even if it means giving up their freedom to a robot car. The project’s interaction with technology is different than the standard interaction with computers, phones, and other electronic devices where a person interacts with it while sitting still. This technology will interact with the person while keeping them active, allowing them to move around and getting some much needed exercise.

Jonathan’s project is about making a game modeled after the Greek mythology of Daedalus and the Minotaur. In this game, two players (one as the Minotaur and one as a runner trying to escape the maze) play against each other to see who can accomplish their goals first. The Minotaur has to catch the runner and the runner has to run away from the Minotaur and escape the maze. This project is interesting because of how many different layers it has: the heart beat mini-game to decide the speed of the avatars, the characters, the abilities, the goals, the Greek mythology backstory, etc. Some feedback we gave him was somehow teach the players the backstory of the myth before they play so they understand the purpose of the Minotaur (and what a Minotaur is) and players and why they have to do what they have to do. We also asked him to consider the difficulty as a super-sized 1000×1000 maze would be very difficult to solve while the player is occupied with escaping the Minotaur. This is different than our interaction with technology because once again, we are using retro joystick controllers instead of the modern computer keys that would normally be used to control characters on a screen. This brings back nostalgia of the pac-man arcade days. 

Overall, I didn’t discover a new definition for interaction but I did notice that people’s interactivity seems to increase the more people are involved. So something like a multiplayer games has “more” interaction than something that is not a game. This is something interesting I discovered from this recitation.

The feedback I received for my project was to include the other senses as stimuli to affect the heart-rate in addition to just sight and sound. Someone also said that we could make the images we show react to the user’s heart-rate. For example, if the user’s heart-rate is high, we could show a picture that could slow the heartbeat. We were also advised to create an environment where outside stimuli wouldn’t have an affect on the user’s heart-rate by either using headphones or making a soundproof containment for the project. After discussing the feedback with my partner, we decided that the last feedback was very helpful and we want to incorporate the headphones into our project. The feedback also pointed out that our project is too “sciency,” I think this is the least successful part of our proposal. We wanted to make something fun but it ended up sounding more like a science experiment than an interactive project. We will need to think deeply about the message we want to send with our project since it is no longer just a simple project involving a heart-rate monitor.