Introduction: Concept and Design
This blog post is going to discuss my Interaction Lab final project dubbed See Your Feelings. Albeit the name is a bit lackluster, the project itself is an interesting one.
See Your Feelings is a project that aims take a user’s story and give it a unique physical image. The inspiration behind See Your Feelings derived from a guest speaker hosted by the IMA department; this speaker talked about his interest in exploring the unorthodox concept of “seeing sound and hearing light”. Upon hearing his presentation, I was inspired to incorporate an idea similar to his in my own project. See Your Feelings, as the name suggests, is about seeing your feelings. I wanted to take a user’s feelings, as abstract as they may be, and provide them a way to see what their feelings as well as other users’ feelings looked like. In order to do this, a user must first choose their color. They create their desired color by twisting three potentiometers; each one correlating to red, green, and blue respectively. There is an RGB LED that provides the user with visual aid as to the color they are making. Once the user is satisfied with the color they have created, they press a button labeled “Done” and speak into a microphone. The user is encouraged to say whatever they want into the microphone. I’ve seen users sing songs, talk about their day, or even just scream into the microphone. Once the user is finished speaking, they then press a button labeled “Finish”. Upon pressing “Finish”, the user’s “feelings” will appear on a digital canvas in the form of what can best be described as a radiating circle. The circle’s size is dictated by the volume at which the user speaks; the louder the user talks, the louder their circle will be. One of my favorite aspects of See Your Feelings is how it compiles every user’s unique input into a vibrant picture that displays the diversity in our feelings.
Process:
This was the first stage of See Your Feelings. As you can see it was nothing but a single potentiometer and a button. This was to ensure the serial communication ran smoothly. Prior to user testing, the potentiometer would be replaced with a sound sensor and two more buttons would be added.
User Testing: Prior to user testing the project was extremely primitive. It consisted of three buttons that if pressed would display either a green, red, or blue pattern consisting of randomly generated shapes on a canvas with eight tiles. Once all eight tiles were full, the program would have to be restarted. Though the project presented at user testing was a poor representation of what the project aimed to be, I still received amazing feedback. (Unfortunately there are no videos)
Feedback:
- Allow the user to choose the color of their pattern. Emotions should not have associated colors.
- Forget the eight tiles, instead have the patterns occur in random places on the canvas.
- Alter the project so that it can run continuously. It is hard to display a project if you constantly have to restart the program.
- Make the pattern less ugly.
All of this feedback would eventually be implemented one way or another.
This was the first stage to constructing the project. As you can see, the only thing wired into the arduino was two buttons and a sound sensor.
Here you can see that improved buttons as well as potentiometers have had wires soldered to them. These will go on to be the potentiometers responsible for the user choosing their desired color. The improved buttons will serve as the aforementioned “Finish” and “Done” buttons.
This is an image of the hardware fitting into the newly fabricated case for the first time. The casing was made of a thin wooden material.
This video shows the potentiometers, sound sensor, buttons, and newly added RGB light all fitting into their allocated housing on what would soon become the lid of the casing.
This video displays how the wiring did not comfortably fit into the casing. Moreover, it was at this point that I realized all three of the potentiometers were not functioning.
This was the temporary rig I came up with in order to test the code. I resorted to using potentiometers from our Arduino kits in order to accomplish this.
These are the newly soldered potentiometers. Not only did they function, however the wires on these potentiometers were shorter compared to their predecessors which helped get around the issue of the hardware not fitting into its casing.
This image shows all of the hardware comfortable fitting into the casing. You can also see that a bulk of the M/M wires on the breadboard have been replaced with flat wiring in order to conserve more space.
This is an additional image of the hardware in its casing, as you can see the LED is working beautifully.
This video displays the final product!
Processing code:
// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing
/*
* Based on the readStringUntil() example by Tom Igoe
* https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
*/
import processing.sound.*;
SoundFile file;
import processing.serial.*;
ArrayList<Ring> rings = new ArrayList<Ring>();
ArrayList<Float> message = new ArrayList<Float>();
int nextSpot = 0;
//int speakerLow [] = new int [8];
////int speakerMid [] = new int [8];
//int speakerHigh [] = new int [8];
color colors[] = new color[8];
String myString = null;
Serial myPort;
int NUM_OF_VALUES = 6; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/
int[] prevValues;
float startx;
float starty;
float speakerLow;
float speakerHigh;
color colorRing;
boolean startMessage = false;
//float min = Float.MAX_VALUE;
//float max = Float.MIN_VALUE;
float min;
float max;
void setup() {
fullScreen();
//size(1000, 500);
background(0);
file = new SoundFile(this, "1.mp3");
file.play();
setupSerial();
for (int i=0; i<colors.length; i++) {
colors[i] = color(0, 0, 0);
}
}
void draw() {
updateSerial();
//printArray(sensorValues);
noStroke();
fill(0, 0, 0, 10);
rect(0, 0, width, height);
//background(0);
for (int u=0; u<rings.size(); u++) {
Ring r = rings.get(u);
r.drawShape();
}
if (sensorValues[1]==1 && prevValues[1]==0) {
colorRing = color(sensorValues[3], sensorValues[4], sensorValues[5]);
startMessage=true;
}
if (sensorValues[2]==0 && prevValues[2]==0 && startMessage==true) {
message.add(map(float(sensorValues[0]), 0, 1023, 0, height*0.8));
}
if (sensorValues[2]==1 && prevValues[2]==0) {
updateSpeaker();
speakerLow = min;
speakerHigh = max;
rings.add(new Ring(random(width), random(height), width/4, height/2, colorRing, speakerLow, speakerHigh));
}
if (sensorValues[1]==1 && sensorValues[2]==1) {
background(0);
rings.clear();
}
if (sensorValues[2]==1 && prevValues[2]==0) {
file.play();
}
arrayCopy(sensorValues, prevValues);
}
//void startMessage() {
// message.add(float(sensorValues[0]));
//}
void updateSpeaker() {
startMessage=false;
min = height*0.8;
max = 0;
println(message.size());
for (int i = 0; i < message.size(); i++) {
//Find the minimum value of the arraylist
if (message.get(i) < min) {
min = message.get(i);
}
//Find the maximum value of the arraylist
if (message.get(i) > max) {
max = message.get(i);
}
println("min " + min);
println("max " + max);
}
message.clear();
//int newValue = int(map(sensorValues[0], 0, 1023, 0, height/2.5));
//speakerLow = min(height*0.1, newValue);
////speakerMid[7] = __;
//speakerHigh = max(height*0.2, newValue);
}
void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[25], 115200);
// WARNING!
// You will definitely get an error here.
// Change the PORT_INDEX to 0 and try running it again.
// And then, check the list of the ports,
// find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----"
// and replace PORT_INDEX above with the index number of the port.
myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
myString = null;
sensorValues = new int[NUM_OF_VALUES];
prevValues = new int[NUM_OF_VALUES];
}
void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), ",");
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}
class Ring {
float startx;
float starty;
float wid;
float hei;
color clr;
float low;
float high;
Ring(float tempstartx, float tempstarty, float tempwid, float temphei, color tempclr, float templow, float temphigh ) {
startx = tempstartx;
starty = tempstarty;
wid= tempwid;
hei= temphei;
clr = tempclr;
low = templow;
high = temphigh;
}
void drawShape () {
float mid = (low+high)/2;
stroke(clr);
float p = sin( PI*millis() / (1000-high) );
float dia = mid + int ( p*(high-mid) );
ellipse(startx, starty, dia, dia);
}
}
Arduino code:
// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing
int red_light_pin = 11;
int green_light_pin = 10;
int blue_light_pin = 3;
void setup() {
Serial.begin(115200);
pinMode(red_light_pin, OUTPUT);
pinMode(green_light_pin, OUTPUT);
pinMode(blue_light_pin, OUTPUT);
}
void loop() {
int soundSensor = analogRead(A1);
int doneButton = digitalRead(8);
int finishButton = digitalRead(7);
int RedPot = analogRead(A5);
int GreenPot = analogRead(A4);
int BluePot = analogRead (A3);
analogWrite(red_light_pin, RedPot/4);
analogWrite(green_light_pin, GreenPot/4);
analogWrite(blue_light_pin, BluePot/4);
// keep this format
Serial.print(soundSensor);
Serial.print(","); // put comma between sensor values
Serial.print(doneButton);
Serial.print(",");
Serial.print(finishButton);
Serial.print(",");
Serial.print(RedPot);
Serial.print(",");
Serial.print(GreenPot);
Serial.print(",");
Serial.print(BluePot);
Serial.println(); // add linefeed after sending the last sensor value
// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}
Reflection:
Overall, I just wanted See Your Feelings to take the user’s feelings/story/whatever and provide them the opportunity to see whatever they said into that microphone with their own eyes. The project appeared to resonate with audiences of all ages. Though admittedly a little hard to use, See Your Feelings constantly put the user’s face once they were able to find their contribution to the intricate picture laid before them. I was glad that every step of the interaction appeared to be enjoyable; there were some users whose favorite aspect of the project was simply getting to create their own colors. I’m satisfied with the project’s level of interactivity as both using the project and witnessing the results appeared to be equally entertaining. I would improve my project by making the radiating circles produced by the user’s voice more unique. I would accomplish this by taking more aspects of the users voice into account when generating the radiating circles. For example, what if the user’s pitch influenced how fast their individual circle pulsed? I feel like additiones similar to this would greatly add to the symbolism of each user’s input; something that, in my eyes, would greatly improve the project. This leaves us with one final point to cover: why should anyone care about my project? To be quite honest, I don’t know the answer to that question. Would the world keep spinning had See Your Feelings never been created? Yes. With that being said, even if just a single user was able to see what they said and reflect on it in any capacity, then I consider the project a success. It’s been an extremely fun semester thanks in large part to Interaction Lab, and I’m glad I got to end it project I care so deeply about. Thanks for everything Marcela, see you around!
