Context and Significance
During our group research project before, I defined interaction as the process of a device receiving signals from other subjects and reacting to it. Based on the definition, we designed an intelligent feeding robot “Sfeeder” and our core aim is to design a device which can facilitate users’ life through interaction. The idea of facilitating users’ life through interaction has been continued in my midterm project, meanwhile, my partner and I also added artistic elements to it.
Our basic idea of the midterm project is to make an instrument which can be easily played by the beginner and their hand movements while playing it can also be a part of the artistic performance which has ornamental value. The idea originates from my experience that I learned piano for years and I have seen many beginners give up learning it because they thought they lacked the coordination of hands and brain – they would either press the wrong key if pay attention to the standard hand movements, or play the correct melody with hard or weird hand movement. Thus, we wish we could design an instrument that everyone can easily make a melody through it. The significance of such design is to lower the threshold of the art world, making all the people who are interested in music have the ability to play an instrument. Meanwhile, the players’ hand movements can also be appreciated from the perspective of arts, which is also the unique point in our design.
Conception and Design
With the rough idea of making an interactive machine that makes melody based on the user’s movements it tracked, the basic components we chose for the project are 3 Arduinos, 3 breadboards, 4 sharp infrared distance sensors, and 8 buzzers. Each buzzer was coded with a ¾ beats of melody in reference to the theme song of Netflix series Daredevil. And each sensor was coded to detect movements within two sets of ranges: 4cm to 10 cm and 10 cm to 30 cm, and both ranges affect a unique ¾ beats of melody from two different buzzers. In this way, we expected that when a player moves his/her hands above the instrument like a music conductor, the sensors would detect hand movements and trigger different buzzers with different melodies; the melodies together would assemble a song created by the player himself/herself.
We chose buzzers to make the melody because they are small and can be easily hidden under the surface of the instrument. We also considered using MP3 players to replace the buzzers according to Professor Rudi’s advice, as the MP3 player could create more fair-sounding melody than buzzers. However, because we designed the instrument as a portable one, its volume should be as small as possible. At last, we gave up on the idea of MP3 player because it’s much bigger than buzzers and would complicate the circuit which had been already hard to be hidden in a shell; and if there were some parts of circuit naked before players’ eyes, they might feel uncomfortable. What’s more, the reason why we chose sharp infrared distance sensors as the core components is that they are a kind of basic sensors which can detect human movements; its usage exactly fits our vision of the project. Also, in the precious recitation, I once used the ultrasonic sensor, which provided me an experience of coding a sensor to detect the distance; comparing to it, sharp infrared distance sensor has a smaller volume which makes it a better choice for the appearance of our instrument. In addition, at first the components we used also included 8 LEDs as we planned to place the sensors on four different faces inside a box, and the LEDs were placed on the surfaces of the box so that they can indicate the relevant position of the sensors inside the box. But later we noticed when a user made movements inside the box, their attention would be attracted by the interior of the box instead of where the LEDs were, so the LEDs could hardly achieve the effect as we expected and even confused the users. Professor Rudi also pointed out this problem; as a result, we finally cut out the LEDs from our circuit.
Fabrication and Production
Step #1 – Circuit
Step #2 – User Testing Version
Step #3 – Final Version
The most significant steps in my production process should be coding and user testing. Because coding is the basis of our project and user testing offers really valuable advice for us to make the project better interacted with users and closer to our goal. I was mainly in charge of designing the circuit, code, and appearance for our project. At the very beginning, I found the most difficult part is to code the melody for different buzzers since I did not have much experience of coding. I had totally no idea about how to do the work. Then I searched a post of the process that a blogger made the little device which can play Happy Birthday Song for his girlfriend through Arduino and buzzer. This post helped me a lot to understand how to edit the notes and beats by codes. When I first completed the code of melodies, I didn’t know how to combine the code for melodies and the code for sensors together. Which means at first I could only make the buzzer ring once when I connected the power supply, instead of making them play the melodies following the data change of sensors. After Tristen and Nick’s help, I finally understood the key was to remove the loop for the melody beneath the “if; else” code which divided the distance value into two ranges, and successfully completed the code.
And we made some important changes after the user testing: one is cutting out the LEDs as I mentioned before; another one is changing the appearance of it and the position of the sensors (as pictures show). During the user testing, some users mentioned that the naked circuit made the instrument looked scary and messy and made people not want to get close to it, so in the final version we hid all the circuits under the shell; some users also mentioned that the look made them confused about its function and purpose, so we made the final version looked neater and drew some notes on its shell to remind people that it was an instrument; and because some sensors always detected something even when there were no hand movements in front it, some users also got confused about how to trigger the melodies, therefore, we arranged the sensors in an orderly manner on one surface of its shell for reducing such situation. However, the sensors sometimes still could not work perfectly after we made sure the circuit and code had no problem; according to others’ advice, I think maybe changing the sensors into another more stable kind of sensors is a good idea for future development.
Conclusions
The goal of our project is to design an interactive instrument that the player can create their own melody by combining different hand movements, while the melody and the hand movements can be both considered as parts of the artistic performance; we hope our project could encourage those people who failed to learn piano or other complicated instruments to still keep the passion for the music and build up the confidence of playing an instrument well with artistic hand movements. Our project aligns with my definition of interaction because it achieves the intelligent communication between other subjects and the device – it receives signals from the player’s hand movements and gives responses in the way of different melodies. However, the interaction was not perfect because the sensors we used sometimes misread the distance of obstacles in front of them. This was also what our audience expressed kind of disturbed their experience of interacting with our project. If I had more time, I would adjust the sensors or even change their kinds to make sure the sensors can work accurately. I would also make some changes to the interactive method. Our instrument now plays basically different melodies when the player changes the position of their hands, and some audiences expressed that sometimes the connection between the melody change and their hand movements was not clear. If I had more time, I would like to make the interactive method of how the player’s hand movements affect the change of melody clearer to observe and learn. So that it would also be easier for the player to create their own song through our instrument after having a clearer sense of how to change the basic melodies. For example, I could choose only one basic tone for each buzzer, but make their beats change responding to the distance between the hands and the sensors.
From my perspective, the most valuable thing I have learned from the setbacks and failures is that we should do everything step by step instead of expecting to achieve a lot of functions at the same time, otherwise, we might not make each function work well as we expected. The midterm project made me realize how hard it was to design a product and accomplish it. I appreciate all the interactive projects and their designers because they all try to facilitate our life or bring us a sense of beauty in some ways. Even the failed or imperfect project should be appreciated because without them we cannot achieve a better work. I will always carry the courage of starting from zero and standing up wherever I fall which grew inside me during the project. I believe such lessons are important to all the creators, designers and inventors in the entire world.