Final Project – Victor Qiu

Conception and Design

The concept of our project is to represent how music is part of human nature, and we express ourselves through sounds and feel certain emotions when hearing them. Our idea was altered slightly compared to the one in the essay. Our initial idea is to allow people to feel the music by altering the speed, pitch and beat. This would be done by making certain physical movements so the user feels immersed in the melody. During the user, testing is when we realized we wanted to make something more unique. Therefore, we started to think of new ways we could display music and make people feel more connected to it. Our final project was a human head with sensors that will allow music to be altered. Moreover to make the user feel the music even more we created a visual using Processing. This visual will interact based on how the user interacts with the project. I believe displaying the project and adding sensors to a human head was very effective when transmitting our message.

Fabrication and Production

In our project, we decided to implement the distance sensor, the touch sensor, and a potentiometer to represent the physical part of the interaction. The first step to set up all the sensors was to drill a hole in the bottom part of the head, this will make the process of placing the sensors easier. Our goal for this part was to hide the sensors so that when the user interacts with them it feels more natural and instinctive instead of robotic. We implement this idea successfully as we made holes in the ears to include the distance sensor. The potentiometer is placed in the base of the mannequin head, our initial idea was to place the potentiometer somewhere on the face but we then decided that aesthetically it will look better there. Moreover, after some user testing, it was also more intuitive for the user to interact with it. The next part was to implement the LED lights that will shine through the eyes, we coded the LED lights through Arduino and the LED was synchronized to the music with to the code in Processing (Appendix). Additionally, we also included a touch sensor on the nose, when this is squeezed the graphics of the Processing change and produce a sound over the song played. During our initial idea, we implemented a vibration sensor on the right side of the face, when this sensor was triggered the song will switch. Due to some difficulties with the coding we weren’t able to include this sensor in our final project. The final outcome of our project is different from the idea we wrote in our essay, originally, we were going to create a box that will allow you to mix and alter music with different types of interaction. But shifting this idea to a more artistic and unique way to display the project was pretty successful.

Conclusions

The goal of my project was to represent the importance of music and how is it part of human nature. I believe the project was successful as we achieved to create the digital and the physical part as we have planned. The audience also reacted very positively to the project, they found the project very interesting and intriguing. Many testers also mentioned it was very fun to modify the music. This project aligns with the definition of interaction as it requires the user to utilize many human senses to make the project function. We implemented this interaction with both visuals and different sensors. In this case, there was more time we would have included more sensors around the head, and fix the vibration sensors so that instead of using the numbers on the keyboard to skip songs it can be made by slapping the mannequin. Overall, creating this project has helped me be more flexible and adapt if things don’t go as planned. The 3D printing part of our project wasn’t quite successful as the material we used wasn’t as flexible, but we still managed to implement the 3D printing into the project. After creating this project I have a deeper understanding of how Arduino and Processing work together and further understand how the sensors work as we had to asses many sensors to choose which one to implement.

Appendix

Code:

import processing.serial.*;
import processing.sound.*;

Serial serialPort;
SinOsc sine;
AudioIn in;

//arduino_values[0] is slider
//arduino_values[1] is Pressure
//arduino_values[2] is distance sensor
//arduino_values[3] is button

//LED LIGHTs
int NUM_LEDS = 60; // How many LEDs in your strip?
color[] leds = new color[NUM_LEDS]; // array of one color for each pixel

int NUM_OF_VALUES_FROM_ARDUINO = 4; /* CHANGE THIS ACCORDING TO YOUR PROJECT */
/* This array stores values from Arduino */
int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO];

int cols, rows;
int scl = 20;
int w = 2000;
int h = 1600;
float flying = 0;
float[][] terrain;

SoundFile music;
SoundFile sound;
SoundFile sound2;
SoundFile sound3;
SoundFile sound4;
SoundFile sound5;
SoundFile sound6;
SoundFile sound7;

Amplitude analysis;

void setup() {
size(1200, 800, P3D);
cols = w / scl;
rows = h/ scl;
terrain = new float[cols][rows];

printArray(Serial.list());
serialPort = new Serial(this, “/dev/cu.usbmodem1101”, 115200);

println(“Loading mp3…”);
sound = new SoundFile(this, “titi.mp3”); // load the seperate songs
sound2 = new SoundFile(this, “4eva.mp3”);
sound3 = new SoundFile(this, “kissit.wav”);
sound4 = new SoundFile(this, “loverboy.mp3”);
sound5 = new SoundFile(this, “thunder.mp3”);
sound6 = new SoundFile(this, “kungfu.mp3”);
sound7 = new SoundFile(this, “synth.mp3”);

music = sound;
sound.loop();
analysis = new Amplitude(this);
analysis.input(sound);
sine = new SinOsc(this);
sine.freq(map(arduino_values[1], 0, 1000, 70, 1000));
}

void draw() {

getSerialData();

float volume = analysis.analyze();
music.rate(map(arduino_values[0], 0, 1023, 0, 2)); //speed
music.amp(map(arduino_values[2], 0, 50, 0, 1)); // the volume

background(255, 105, map(arduino_values[0], 0, 1000, 150, 250));

//flying -= 0.1; // speed (original)
//flying -= map(mouseX, 0, width, 0, 1); // speed (mouse cursor)
flying -= map(arduino_values[0], 0, 1023, 0, .5); // speed (serial)

float yoff = flying;
println(arduino_values[0]);
for (int y = 0; y < rows; y++) {
float xoff = 0;
for (int x = 0; x < cols; x++) {
terrain[x][y] = map(noise(xoff, yoff), 0, 1, -100, map(volume, 0, 1, 0, 150)); // the
xoff += .2; //+ map(arduino_values[1], 0, 500, 0, .1); // the sharpness
}
yoff += .1 + map(arduino_values[1], 0, 500, 0, .1); // the jigglyness
}

stroke(255);
fill(0, map(arduino_values[1], 0, 500, 200, 250), map(arduino_values[1], 0, 500, 200, 250), 200);
translate(width/2, height/2+100);

if (arduino_values[1] >= 600) {
sound5.play(1, .05);
}

if (arduino_values[1] >= 600) {
background(0, 0, 0);
fill(250, 250, 0, 255);
stroke(0);
}

rotateX(PI/map(arduino_values[1], 40, 800, 2, 2.3));
translate(-w/2, -h/2);
for (int y = 0; y < rows-1; y++) {
beginShape(TRIANGLE_STRIP);
for (int x = 0; x < cols; x++) {
vertex(x*scl, y*scl, terrain[x][y]);
vertex(x*scl, (y+1)*scl, terrain[x][y+1]);
//rect(x*scl, y*scl, scl, scl);
}
endShape();
}
// LED STRIP

//LIGHT
for (int i=0; i < NUM_LEDS; i++) { // loop through each pixel in the strip
if (i < volume * NUM_LEDS) { // based on where we are in the song
leds[i] = color(0, 250, 250); // turn a pixel to red
} else {
leds[i] = color(255, 105, 255); // or to black
}
}
sendColors();
}

void keyPressed() {
music.stop();
if (key == ‘1’) {
music = sound;
} else if (key == ‘2’) {
music = sound2;
} else if (key == ‘3’) {
music = sound3;
} else if (key == ‘4’) {
music = sound4;
} else if (key == ‘5’) {
music = sound6;
} else if (key == ‘6’) {
music = sound7;
}
music.play();
analysis.input(music);
}

/*
void button() {
music.stop();
int s = 1;
music = sound;
if (arduino_values[3] == 1) {
s = s + 1;
if (s == 1) {
music = sound;
} else if (s == 2) {
music = sound2;
} else if (s == 3) {
music = sound3;
} else if (s == 4) {
music = sound4;
} else if (s == 5) {
music = sound5;
}
}
music.play();
analysis.input(music);
}
*/

void getSerialData() {
while (serialPort.available() > 0) {
String in = serialPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (in != null) {
print(“From Arduino: ” + in);
String[] serialInArray = split(trim(in), “,”);
if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
for (int i=0; i<serialInArray.length; i++) {
arduino_values[i] = int(serialInArray[i]);
}
}
}
}
}

void sendColors() {
byte[] out = new byte[NUM_LEDS*3];
for (int i=0; i < NUM_LEDS; i++) {
out[i*3] = (byte)(floor(red(leds[i])) >> 1);
if (i == 0) {
out[0] |= 1 << 7;
}
out[i*3+1] = (byte)(floor(green(leds[i])) >> 1);
out[i*3+2] = (byte)(floor(blue(leds[i])) >> 1);
}
serialPort.write(out);
}