A. Four Seasons- Joyce- Gottfried Haider
B. CONCEPTION AND DESIGN:
The concept of my project was heavily influenced by a traditional Chinese instrument that I started to learn this semester, the guqin. Everything from the overall shape and number of strings it has can be seen in my product. Leading to up the beginning of the project, we did some research and wrote an essay about our plans for development. The three projects that I have researched didn’t affect my project in any way, but they served as inspiration for me. They taught me to have my project presented cohesively and make everything look seamlessly connected. I utilized the essay in a way to help me serve as a way to help me organize my thoughts and create a timeline, to keep myself on track.
I had a feeling that users would pull each string individually, therefore for my original design, I had each string produce a different effect. However, after discussing it with my instructor, I changed it to have each string resume the video. Meaning that when the Arduino value was below 5, the video will be paused. This was the design I had during user testing, and I added a continuous soundtrack. However, even though many people said it was fun, I had a feeling something was missing. I remember Kevin told me to do something more with the video than just having it play and have the instrument as one whole, cohesive thing instead of multiple wooden planks. Professor Andy also said the same thing to me, he also mentioned how I should cover up my Arduino and all the wiring. Something that Andy said inspired me, and that was to have two videos fade in and out together when the strings are pulled. Taking that into account, after user testing, I decided to have four short videos, instead of one, and every string that’s pulled will allow a new one to play. I also decided to add a few notes that will play when certain strings are pulled because Andy said it will give more of a “user controlling the project” effect.
After trial and error, I think the changes I made were effective. Everything I changed made the project better than it previously was and I am grateful for all the feedback that I received. I also tried to follow a theme, courtesy of Andy’s suggestion, so I opted to follow a seasonal theme. Every video corresponds to a specific season, the Forbidden City is winter (hence the snow), spring is Guilin, Guangxi, fall is Suzhou, and summer is the Great Wall.
C. FABRICATION AND PRODUCTION:
Right after I submitted my Project Proposal Essay, I began to design my project, first on paper and then on Cuttle, to laser cut. I was not too sure what sensor I should use, because I had forgotten what sensors can pick up vibration/ sound, however during a meeting with my instructor, he informed me of using a Piezo sensor, which I ended up using. I chose not to use another because I tested this one out and it worked the way I wanted it to. I just needed a sensor that can pick up vibrations and I set a limit on Processing, so the Piezo sensor was a great choice.
For my fabrication portion of the project, I chose to laser cut using wood, because the instrument itself is made of wood, therefore I wanted to preserve that aspect. I chose a 5mm wood due to the sturdiness and the other wood options were too thin for my liking. My original laser-cutting design included six oval wooden planks with H-shaped stands on each side, to tie the strings. However, the finished look of that didn’t satisfy me and it wasn’t the overall ambiance I was aiming for. I considered Kevin’s suggestion and decided to laser-cut a cohesive piece of wood and I transferred the stands from the individual plank to the singular wood piece, and designed a little temple for my wires!
I believe the combination of four short videos was significantly better than one long video because the “interactive’ portion of my project comes from that. If the user was just to resume the video over and over, it would be boring. Also, the very first video I selected was very distracting because of the slow speed it was recorded and there were many people in the background, creating a chaotic vibe. The whole vibe of my project was “calm” and that video was not the most suitable. After the feedback from user testing, I went on Douyin to look for additional videos that can substitute the former and I found them. These new videos were quick and short and no people were present. It was also recorded in a very theatrical manner, which appeared very pleasant to me. I chose to include videos from the most popular Chinese tourist destinations because I very much enjoy traveling and I have been to all four of these places before. After my presentation on Thursday, someone, I forgot who said that it felt like they traveled to different parts of China and I appreciated that because this was, also, someone I had in mind during the rewriting of my code.
Asides from the video portion of the Processing code, I also had to incorporate sound into it. I wanted to include a cover of a popular modern Chinese pop song, but the instrument was just a guqin instead of the original ones used. I opted for “青花瓷” by Jay Chou because that is one of my favorite songs and my WeChat ringtone! Luckily, I found a guqin cover of it on Youtube and I used it as my consistent background music. I also found guqin sounds on another Chinese social media app and used those for five of the strings. I didn’t consider having the extra notes playing until after user testing and someone suggested it. After the presentation, Professor Marcela suggested that a better way to incorporate sound into my project was to have a nature-like sound in the background, such as wind blowing, etc, and then when the user plucks the string, it would play a note. She said because not everyone knows what a guqin sounds like, nor my song of choice, it would be slightly difficult for some people to differentiate the sounds. At the end of the day, even though I appreciated her feedback and she had a very valid point, I chose not to change my background sound because I wanted to persist with my original design.
Something that I appreciated about starting early was it provided me with the opportunity to go back to the drawing board many times. I had many setbacks, such as not knowing how to code what I wanted Processing to do, trivial things like not circuiting properly, and not knowing what exactly was missing from my overall project. However, I am very thankful to my instructor and the fellows for helping me out with all of these, as it may have taken me a long time to figure it out on my own. I did not have many feelings leading up to presentation day because I was not very confident with my project when I compared it to what the rest of my classmates have accomplished. However, I am aware of my understanding of the class and given my knowledge, I would say my project was a success, even if there were more room for improvement.
D. CONCLUSIONS:
The goal of my project was to create a calming touristy experience for the users and I would like to say my project did achieve that. Maybe to a certain extent it did, but, again, there was certainly a lot of room for improvement. Ultimately, the audience interacted with my project by plucking the strings one by one and many of the testers thought it was fun, but it could have been better. My definition of “interaction” is a call and response between a user and the project. I do think that my project aligns with my definition because the user “calls” the project, by plucking the strings, and the project “responds” by manipulating the video and adding a note. If I had more time, I would have liked to make my project more put together and more complex. I would have also liked to present the strings more sufficiently, however, since I bought guitar strings, I found out they are not as malleable as I assumed so I had a lot of trouble tying them around the stands. From the multiple setbacks I’ve encountered doing this project, I learned how to be more patient with myself, working things out slowly, and asking for help as a good thing instead of tiring myself out when I couldn’t find a solution after a long time. I truly believe that accomplishing this project is an accomplishment in itself, therefore one of the biggest takeaways is how important it is to believe in yourself and try to work it out yourself before asking for help.
To sum up, all that I have mentioned about my project and my progress throughout the whole thing, this paragraph will serve as a conclusion for all. The issues and setbacks I have mentioned in my documentation are trivial things, however, it reflects my creativity in the arts. Throughout the entirety of the project, I was hesitant about what I should do next and what I wanted to accomplish at the end. Although, at the end of it all, I was pleasantly satisfied with what I have created. To me, my ideas served as a stepping stone for my love and interest in traditional China, seen through the instrument, and the videos and sound files I chose to incorporate. The key elements of my project’s experience lay in the strings and how users can interact with them, they can be pulled one by one or all at the same time. Although, granted the sound combination of everything was not very pleasing to my ears. From this project, I dived deeper into my creativity and utilized some of the concepts I had learned this semester. If I were given a second chance to create a final project for Interaction Lab, I would perfect my original and not make a new one because the idea behind this project holds significance to me.
E. APPENDIX:
int SENSOR_PIN1 = A0; // analog pin per sensor
int sensorVal1;
int SENSOR_PIN2 = A1;
int sensorVal2;
int SENSOR_PIN3 = A2;
int sensorVal3;
int SENSOR_PIN4 = A3;
int sensorVal4;
int SENSOR_PIN5 = A4;
int sensorVal5;
int SENSOR_PIN6 = A5;
int sensorVal6;
void setup() {
Serial.begin(9600);
}
void loop() {
sensorVal1 = analogRead(SENSOR_PIN1); //values
sensorVal2 = analogRead(SENSOR_PIN2);
sensorVal3 = analogRead(SENSOR_PIN3);
sensorVal4 = analogRead(SENSOR_PIN4);
sensorVal5 = analogRead(SENSOR_PIN5);
sensorVal6 = analogRead(SENSOR_PIN6);
// print out the value in serial monitor:
Serial.print(sensorVal1);
Serial.print(","); // , separates the values in Processing
Serial.print (sensorVal2);
Serial.print(",");
Serial.print (sensorVal3);
Serial.print(",");
Serial.print (sensorVal4);
Serial.print(",");
Serial.print (sensorVal5);
Serial.print(",");
Serial.print(sensorVal6);
Serial.println( );
delay(50); // delay in between reads
}
import processing.serial.*; import processing.sound.*; SoundFile file; Serial serialPort; int NUM_OF_VALUES_FROM_ARDUINO = 6; int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO]; import processing.video.*; Movie myMovie[]; int curMovieIdx = -1; // index of the currently playing movie (0..3) int startTime = 0; // time when the current video started playing int lastActiveString = -1; // the string that triggered the last movie void setup() { size(1300, 750); myMovie = new Movie [4]; //myMovie.loop(); myMovie [0] = new Movie (this, "spring.mp4"); myMovie [1] = new Movie (this, "summer.mp4"); myMovie [2] = new Movie (this, "fall.mp4"); myMovie [3] = new Movie (this, "winter.mp4"); myMovie[0].pause(); myMovie[1].pause(); myMovie[2].pause(); myMovie[3].pause(); printArray(Serial.list()); serialPort = new Serial(this, "/dev/cu.usbmodem14201", 9600); file = new SoundFile(this, "guqin.mp3"); file.loop(); file = new SoundFile (this, "note 1.mp3"); file.amp(0.2); file.play(); file = new SoundFile (this, "note 2.mp3"); file.amp(0.2); file.play(); file = new SoundFile (this, "note 3.mp3"); file.amp(0.2); file.play(); file = new SoundFile (this, "note 4.mp3"); file.amp(0.2); file.play(); file = new SoundFile (this, "note 5.mp3"); file.amp(0.2); file.play(); } int pausedAt = 0; float amplitude = 0.0; float videoSpeed = 0.0; void draw () { getSerialData(); int activeString = -1; for (int i=0; i < 6; i++) { if (arduino_values[i] > 10) { activeString = i; } } if (activeString != -1 && (lastActiveString != activeString || curMovieIdx == -1)) { // check if another video is playing, and if so, pause it if (curMovieIdx != -1) { myMovie[curMovieIdx].pause(); println("pausing ", curMovieIdx); } // pick a random index 0-3 curMovieIdx = floor(random(4)); // play the new video myMovie[curMovieIdx].loop(); println("playing ", curMovieIdx); // make a note of the current time startTime = millis(); // make a note of the string that triggered this lastActiveString = activeString; } if (startTime != 0) { if (millis() - startTime > 3000) { myMovie[curMovieIdx].pause(); startTime = 0; println("pausing ", curMovieIdx); curMovieIdx = -1; } } if (arduino_values[0] > 20) { file = new SoundFile (this, "note 1.mp3"); file.play(); } if (arduino_values[1] > 20) { file = new SoundFile (this, "note 2.mp3"); file.play(); } if (arduino_values[2] > 20) { file = new SoundFile (this, "note 3.mp3"); file.play(); } if (arduino_values[3] > 20) { file = new SoundFile (this, "note 4.mp3"); file.play(); } if (arduino_values[4] > 20) { file = new SoundFile (this, "note 5.mp3"); file.play(); } if (curMovieIdx != -1) { if (myMovie[curMovieIdx].available() == true) { myMovie[curMovieIdx].read(); } image(myMovie[curMovieIdx], 0, 0, width, height); } } void getSerialData() { while (serialPort.available() > 0) { String in = serialPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII if (in != null) { //print("From Arduino: " + in); String[] serialInArray = split(trim(in), ","); if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) { for (int i=0; i<serialInArray.length; i++) { arduino_values[i] = int(serialInArray[i]); } } } } }