project2: Merry Christmas music box ā€“Katie ā€“ Inmi

how our project works:

our final product:

Context and significance:

In my previous research about interaction, inspired by the definition of interaction given by Crawford (“a cyclic process in which two actors alternately listen, think, and speak” (1)), I define interaction as a kind of conversation where two or more actors are involved. They listen (receive the information), think (process the information) and speak (give out the processed information). There is one project that both shaped my definition of actors and let me rethink the process of ā€œthinkingā€. The project is two computers play rock paper scissors together. In this case, the interaction has no human involvement but only two machines. But I see almost no cyclic process within the project: each computer runs its own random algorithm, choosing one of the three possible items. So, as we were designing our project, we kept in mind how can we enhance the process of ā€œthinkingā€ between two or more actors.

This summer I took a course called creative learning design. In that course we mainly focused on designing childrenā€™s music learning experience which has a very big market and lots of needs. So, I decided to make a project targeted to children and the goal is to in some extent help them be interested in learning music. Thatā€™s the starting point and we decided to think about how to enhance the music box experience of kids. In that course, one thing I remembered most is that children need instant response to their action, or they will lose interests sooner or later. The first thing we were going to design is of course the instant response to childrenā€™s action (for example, our first thought was to design several buttons and you press different buttons different songs will come out). But back to my research, we are not only making a responsive project but an interactive one. So, we are thinking about how children can ā€œthinkā€ after receiving this response from our project, and how they react to the feedback they received which then forms a circle of the clinic process of interaction.   There are many kinds of music box and the most common kind is the one that you wind up the spring and the music will come out itself.

CONCEPTION AND DESIGN:

The action of ā€œwinding up the springā€ gets the feedback of ā€œhearing musicā€ but itā€™s not interactive enough. Inspired by the winding up spring movement, I came to the idea of what if you turn the knob one step and one note will come out? the rhythm of the music is totally controlled by users. The user received the response from the music box and depending on the response one receives, he/she will then decide what to do next.

FABRICATION AND PRODUCTION:

Me and my partner cooperate very well. I took the responsibility of making the turning knob works, laser cutting the box as well as decoration. My partner took the responsibility of the two step motors and the potentiometers to control them. So, I will write some steps that I think is really challenging and important in this documentation.

Hook up the rotary encoder:

The first step and the most significant step is to find out and hook up the right component that can achieve my goal: easy to turn. The potentiometer is not the ideal one, so I search online and found one called Rotary Encoder. Our equipment room have this kind of rotary encoder but not the exact same type as the one I found on YouTube. The biggest challenge I think for me is to hook up the Rotary Encoder. The type in our equipment room is called ā€œdf robot ec11 rotary encoderā€. So, I searched it on google and wanted to find out the tutorial of how this works. But I did not find tutorial of how to hook up this, the only useful information I found was ā€œThe module has three signal terminals: terminals A & B are encoder output; terminal C is the button signal output.ā€ (from the official df robot website https://www.dfrobot.com/product-1611.html ). But I found a YouTube video of  how the similar rotary encoder works ( https://howtomechatronics.com/tutorials/arduino/rotary-encoder-works-use-arduino/). The names of the pins are different between the two: the one I got have five pins called: VCC, GND, A, B, C and the one I found online have the five pins called: CLK, DT, SW, +, GND. I then did some further research about the difference and hooked it up according to the YouTube video. (I forgot to take a picture of the wires L). I then hooked up the buzzer with the help of one of our recitation notes.  

Problems I faced when writing the code:

The next thing is the code, as we have done one making notes exercise in the recitation, itā€™s not that difficult compared to hooking up the encoder. In writing this code we need to define the output A and B first. I think the most important part is to define the two state (the current state of the encoder and the latest state. If the two states are not equal, then play notes. So, I will not explain in detail but only to note some problems I encounters. The first problem is that the rotary encoder kind of works after I uploaded the code: the music sometimes goes when I turned, sometimes thereā€™s no music when I turned and sometimes thereā€™s music even if I am not turning it. I asked for help from Dave, he thinks thereā€™s no problem with my code and the wires. So, he writes a println to only show the output of the encoder from serial monitor. It turns out that the value is not stable and changing itself without turning. So, he thinks is the connection problem of the encoder. By asking Tristan, I decided to solder it. After soldering it, it was totally worked by our movement of turning the knobs.  But then another problem occurs, it always plays two notes in one turn. Tristan suggested me to start a new Arduino document (see the picture below) to only monitor the encoder. It seems to work very well because both the output A and B changes between 0 to 1 when I turn. I told him Iā€™m going to double the notes in my code to solve the problem. But he suggests me not to do that but really think about how this algorithm works. I have no idea at that moment, but he suddenly knows what goes wrong: in each turn, the output changes from 0 to 1 then back to 0. In this case, every single turn makes the state of the output changes twice. In order to solve this, we add another statement in the if statement. (see the picture below).

soldering.

the new Arduino document to monitor the output of the encoder

the new document to monitor the output of the encoder

add a statement: only if the last state equal to 1, the note will come out.

And thereā€™s also some small problems like how to make the music loop: adding note count. How to make the note stop playing when thereā€™s no turning in more than 5 seconds: adding a timer (millis). How to make the note only start playing when someone turns it first rather than giving out sound as soon as itā€™s hood up to the computer: add a boolin. Jingtian really helps me a lot in these small problems. She writes down the structure of those code on a sheet of paper to let me figure it out first. If I have some problems, she will explain very patiently to me.

how the timer works.

Our biggest improvement comes from Inmiā€™s user test. She told us the users would just randomly turn the knob. We first came up with the solution that to write down an instruction but Inmi suggested that this is the last choice if we could not come up with a better one. She also said that she would made it a big plane to turn which is more difficult and involves more body movements. From that point, we started to rethink about our design and then directly led to the final product.

For the laser cutting process, we cut twice. The first time I used cardboard to test the size whole of the whole since the size can be changed by hand if we use cardboard.

Conclusion:

I think our final project align well with our definition of interaction the two actors (user and the music box) listen (receive the information: the music box listen to the instruction which is the turning of the knob and the user, and the user listen to the notes the music box made), think (process the information: after hearing the notes, the user decided what to act next to respond to the notes, different speed in turning results in different rhythm of the song ) and speak (give out the processed information: the action of the user then results in another action of the music box). In this way, the circle of interaction has completed.

For the audience, I think they act the way we expect: they turn a few notes and later figured out the melody and then enjoy the changing of rhythm by turning in different speed (they could not go crazily fast since the plane is quite big and we add a lot of weight on the plane so itā€™s not so easy to turn. One response we got is that a user said he gets the since of control of the melody which makes him very happy because he never manages to play an instrument. For some users who only turns in one direction, it makes me think of the long play CD player where you can see the CD is turning all the time while the music is playing. Some other users turn it in both ways which 

Improvement: The first thing is that we havenā€™t figured out is that as there are two step motors turning along with the big plate, the wires connecting the two motors get mass up when turning. So, we might come up with a solution to solve this if we have more time. Second, instead of randomly stick our decorations on to the plate, we could make three or four different scenes so that it might be more surprising when users are turning the plate. Also, if possible, we could allow the users to control the movement of the characters on the plate.

Learn from setbacks: when the rotary encoder doesnā€™t work very well, I almost want to change it to the potentiometer weā€™ve used in class and one learning assistant even asked me, why donā€™t you use the potentiometer? But I stick to the rotary encoder (part of the reason is that is takes me a long time to hook it up) and try to solve the problem. What I learn from this is whenever encounter a failure, try to define the problem and solve it first rather than to run away from it and bring up plan B (which is not as ideal as plan A).

Our project provides an interesting and playful way that kids can interact with music. Since the demands for cultivate childrenā€™s interests in music is really large and many parents really worried about this, our projects in some extent help solve part of the problem.

graphic score –Katie

WechatIMG200.jpeg

This graphic score shows how our three scenes develops as well as how the flashlight goes and the sounds. The natural figure of winds, trees mountains indicates where the natural sounds come into the scene and they are also figures of how the narrative develops: the journey of the king. The multiple circles at the beginning and the end give us hind of how the torch light goes: to go in circles away from or approaching the screen to open or close the show. The dots in the bottom indicates the drum beats of our background music. 

RAPS reading response Katie

When I watched the images of Whitney and Belsonā€™s work, the most direct similarity I would come up in mind is that they all place their settings onto a black background. Like one reading weā€™ve read before, the artist mentioned black when it linked to sounds, itā€™s a color of silence, itā€™s a color that can absorb all the sounds. And the biggest differences I think is Whitneyā€™s visuals are separate dots together forms a bigger shape and all the dots linked to each other. But in Belsonā€™s work, the images are quite distorted, itā€™s a whole, not consist of many dots.

After I read through the text, I came to realize some similarities and differences in deeper level: ā€œ[Whitneyā€™s] films do not refer to the actual world bet instead use optical effects to pluck at the musical inner mind and allow each viewer to become a synestheteā€ (Thames& Husdon 145). His visuals are pretty abstract, you almost canā€™t find reference in nature. But I could see Belsonā€™s work kind of get inspirations from nature especially the powerful image of solar eclipse.

In recent pop music concerts, the visual parts are also of great importance acting as a complement of the audio. Itā€™s very much like how Thames& Husdon  described the Vortex Concert series and the later light shows: ā€œ[it] offered up dreamy arrays of colors that both extended the hallucinogenic state of the spectators and evoked synesthetic responsesā€ (Thames& Husdon 167). For the pop music concerts, those light effects are sometimes creating some concrete figure images but more often, they are forming abstract shapes of lines with different colors in order to create a certain atmosphere of the song. Without those lights in concerts, the stage will become empty and dull.

Raps project1 documentation Jiayan Liu

https://gist.github.com/JiayanLiu27/d3e942ced3ee53e4855a59ff2c281e9f

Title: my happy little moments

The concept behind my project is colorful childhood. My intention is to recall some happy moment during the stressful midterm time. Whenever I play piano, I feel super relaxed, so I decide to use the piano sound for the audio.

My inspiration: We were learning Mozartā€™s twinkle twinkle little star variation called: ah,vous dirai-je maman which means ā€œtalk to me motherā€ in the piano class. When I was playing different variations, itā€™s just like little Mozartā€™s conversation with his mom. I felt that although they sound very different for each variation, they are conducted around the same theme and thereā€™s harmony within. Because they are all composed based on the same theme. So, I am wandering what will it sound like if I remix different variations. This is my starting point and intention behind this project.

Perspective and Context:

I really like how Oskar Fischinger use geometric shapes with primary colors dancing through piano music set against the black background. For me, when it comes to synesthesia, I would to depict the joyful mood in a white background of brighter colors, faster tempos and higher notes.

Development & Technical Implementation:

Audio: I first recorded myself playing two of the variations and the theme. Then I was experiencing how to remix them. I used Granular first to see how a music piece would be like if different small pieces are taken to conduct a new piece. It sounds chaotic but it provides me with idea to make an audio with those bright note piece. So, I picked another two variations with faster tempo and higher notes to create the brightness. And use the classroom samplr and audio mixer to mix them together and play around the settings to reach an ideal volume of the four audio sources.

Video: The problem I encountered first was that no matter how I changed the audio, the video output was always the variations of lines. I want to make geometric shapes like those weā€™ve watched in class. Another problem I encountered was those the video is only valid for two or three seconds, for the rest of time, it was only repeating what had shown before.

The solution to the both problems is using video mixers to mix different videos together and experiment with different mixing effects. In my previous version, I directly connected different video sources to the video output without a mixer, so Itā€™s invalid. As I mixed the videos together, the non-repetitive image became longer.

Presentation:

In my presentation I found that the sound quality is not very good as I recorded it by my phone. The video went quite well. I wasnā€™t nerves before class. but as I was the last to present, after I watched othersā€™ projects which are so good the sound and the image fits very nicely. I become nerves about my presentation and worried about the correlation of the audio and video. since my audio doesnā€™t have a strong structure, I think itā€™s harder to see the movement relation of the video compared to my classmatesā€™ project.

Conclusion:

I think I could add interactions on this project, as I watched one of the projects that involves piano keyboard playing on the computer keyboard. I can mix the variations and let the users play the theme, the very simple twinkle twinkle little star. It maybe can composed a clearer structure of the audio piece.

Week5: building drawing machine (Katie)

Materials:
For Steps 1 and 2

1 * 42STH33-0404AC stepper motor
1 * L293D ic chip
1 * power jack
1 * 12 VDC power supply
1 * Arduino kit and its contents

the thing to notice is that the side with the semicircle is the top side of the H-bridge.  Step one is easier but for step two, I encountered some problems. The first one is where to put the potentiometer and how to connect it. later I figured out for the side only have one pin, it should go to the ground on the ardurino board. For the side that has two pins, one should go to the positive side on the bread broad and one should go to the ground on the broad. The second problem is the potentiometer cannot correctly control the movement of the motor. So I opened up the serial monitor to check wether the potentiometer works. It turned out that it’s the feedback delayed. Whenever I turned it, the value would change after ten seconds. So I added a map function. And it worked well.

For step 3:

2 * Laser-cut short arms
2 * Laser-cut long arms
1* Laser-cut motor holder
2 * 3D printed motor coupling
5 * Paper Fasteners
1 * Pen that fits the laser-cut mechanisms

Q1:What kind of machines would you be interested in building?

The drawing machine is very fun. I am thinking about building a larger scale drawing machine that can paint grafitti on the wall with multiple colors.  Digital manipulation of art is very interesting because we used to separate “art” and “digital” , one is human and one is machine. The combination of the two which is the digital manipulation of art brings up many interesting features of the artwork.

Q2: Choose an art installation mentioned in the reading ART + Science NOW, Stephen Wilson (Kinetics chapter). Post your thoughts about it and make a comparison with the work you did during this recitation. How do you think that the artist selected those specific actuators for his project?

The robot drum player seems very appealing to me.  I have seen performance of my favorite singer cooperate with the robot player and conducted the show. Again is the human-machine contrast make this show interesting. This drum playing robots is different and similar to our drawing machine. The differences are first, they are robot shaped, second, they are playing drums. The similarities is both the drawing machine and the robots are to let machine to do something humans do, and the drawing machine can also be made in the shape of robot arms. The artist use different motor to achieve the move of the robot arm and fingers, to make the motion elegant, just like the human movement.