Exercise 1

In this exercise, I made an Etch A Sketch with Arduino and Processing. The Arduino reads the values of two potentiometers and sends to Processing via serial. Then Processing will draw lines based on the values from Arduino.

In the Processing code, I use line() to draw the pattern. Therefore, I have to keep track of the previous x and y values. However, when initiating the variables, previous x and previous y are by default 0. Consequently, one straight line will be drawn at the start, if not all potentiometers are at position 0. To get rid of it, I set previous x and previous y to the current values from potentiometers in the setup() function.

Arduino Code:

Processing Code:

The circuit diagram is as follow:

Here is the video of the final result:

Exercise 2

In this exercise, I made a Processing instrument. When I click on the Processing window, the buzzer will make a sound. And the frequency and duration of the sound are determined by mouse position.

Arduino Code:

Processing Code:

The circuit diagram is as follow:

Here is the video of the final result:

After spending a semester in the interaction lab, I have learned a lot about Arduino, Processing, laser printing and3D fabrication. Most importantly, through constant practice, I grasped a deeper experience and insight into the term “interaction.” During the group project, I gave a definition of “interaction”—interaction is the continuous cycle of information transferring between human and machine. What I want to emphasize here is the continuity and correlation of human action and machine feedback. A complete process of interaction must establish a loop of action and behavior—while the user action determines what feedback the machine will give, the machine feedback also encourages and affect the next action of the user. For the group project, we made a SHAKE TO QUAKE game. During the user test session, I received many helpful comments, such as “the response after pressing the button is delayed” and “there is no indication of the start of the game.” Therefore, now my understanding of “interaction” has evolved. I want to add that the interval between user action and machine response is usually short. In other words, interaction should be real-time and low latency. Moreover, an interaction usually starts from some suggestive behavior of a machine, which guides the user to interact in a proper way.

One project I found online which perfectly matches my definition of interaction is “Click Canvas.” (The project can be found here: https://www.hackster.io/natthakit-kim-kang/click-canvas-an-interactive-wall-04332c.) This project made a grid of clickable LEDs. The user can light up or turn off one LED by simply clicking it. Furthermore, since the LED has different colors, the user can create some pattern with this LED grid. I think this project is interactive. First, the LED is designed like buttons, which prompts the user to press it. Also, the feedback is quite timely. Most importantly, since the user can preview their work in real time, the user might adjust their idea each time a LED lights up or turns down. Thus, this project matches my core understanding of interaction, that the machine response is affecting the user’s next action.

Another project that I found online was “Interactive Mario Mushroom Block.” (The project can be found here: https://www.hackster.io/sclandinin/interactive-mario-mushroom-block-2235dd.) In this project, a box is decorated like the mushroom block in the Super Mario game. This box emulated the behavior of the mushroom block in the game, that it will eject enemy character when the player hit it by the head. When the user elevates the box up, the lid will open, and a stuffed toy will be ejected, with corresponding sound and light effects. Although this project is creative and interesting, I am afraid that it doesn’t perfectly fit my definition of interaction. The appearance of the box is an effective hint for the user to start interaction by lifting the box up. Also, the box responds immediately. Nevertheless, I think there is no loop of action and feedback. In this project, the action and feedback are one-time. Apparently, the machine responds to the user action with short latency, but then the cycle of action and feedback stops—the feedback doesn’t affect the user’s next action.

To conclude, interaction is a process of joint information transferring between human and machine. During this process, though the user doesn’t need to act immediately each time the machines respond to his/her previous action, the machine feedback must be timely. Also, the machine will give some hints or constraints to guide the user to interact properly. As Hornbæk mentioned in “What is Interaction?”, “interaction is joint computer-user behavior where a user pursues some goal or utility to the best of his/her capability within constraints posed by the UI, environment, and the task.” Apart from giving an understandable response to the user action, the machine has more to do in the process of interaction.

1. 1. http://users.comnet.aalto.fi/oulasvir/pubs/what-is-interaction-chi2017.pdf
   2. https://www.hackster.io/natthakit-kim-kang/click-canvas-an-interactive-wall-04332c
   3. https://www.hackster.io/sclandinin/interactive-mario-mushroom-block-2235dd

For the first part, I chose to continue working on my code which I wrote in recitation 6. I add some animation as well as mouse interaction into that code. Basically, I want the hexagons rotating, and the gradient color changes as the mouse moves. In order to implement the smooth gradient color effect, I used the function colorMode() to change the original RBG color mode to HSB color mode. With the HSB color mode, I can change the color simply by changing the value of H.

Here is my code:

 

And here is my final result:

As for the additional homework, I first made a circle that can change its size over time. Then I let the color change as well. Finally, I added keyboard interaction into this code. By pressing the navigation keys, we can change the moving direction of the circle. And when the circle reaches the border of the window, it will bounce back.  Here is my code:

 

And here is the video of the result:

In this recitation, I learned a lot about Processing:

1. colorMode(HSB); can make it much easier to draw gradient color
2. The math expression 10/3 returns an integer. If I want the result to be a float, I have to use 10/3f
3. Math expressions are able to accurately determine how the pattern changes over time

Here is the image I chose to be my motif:

Image from http://www.artnet.com/artists/victor-vasarely/radau-a-5ucK2BtOl-HuPfL5cHegZQ2

I chose this image because it is like a beautiful rose. It consists of a rotating nested hexagonal and gradient colors.

Drawing such a pattern in processing was not very easy, because the rotating is really a challenge. It took me a long time to figure out how to rotate the hexagons.

Here is my final code:

Here is my final work:

I think my work did well in the hexagon drawing and rotating, but the gradient color was different from the motif I chose.

Context and Significance

In this midterm project, my group (Jonathan Lin, Jennifer Cheung and me) created a Shake to Quake competitive game. My previous group project taught me what kind of thing is interactive. First, it must have an input which receives the outer world’s information and an output which respond back. Second, it must form a loop. While the input determines the output, output also changes users’ decision thus changes future input. Last, the time interval between input and output is usually small. We agreed that making a multi-user game is the best way because games perfectly match my interpretation of interaction and competitive games are entertaining. A competitive game is never a new idea, but we innovated to combine the earthquake with a competitive game. We expected users to shake our 3D-printed cities. And the faster will be the winner of the game. Apart from a game, our project is intended to kids. This game can teach them the different levels earthquake has, as well as the destructive nature of the earthquake.

Conception and Design

The sensors we chose to measure shaking motion was acceleration sensors, which is widely used nowadays. We want to install the sensors on two 3D-printed city model. And the users will shake the model to play the game. For the output, we initially decided to install several red LEDs on the 3D-printed city model as indicators. The goal of our game is to destroy the city as fast as possible. We use LEDs to represent buildings. The LED turns on when the corresponding building is down. We initially wanted to install LEDs on the 3D-printed buildings. Unfortunately, we quickly found that the city models are too small to contain that many LEDs. We considered using one RGB LED to replace several red LEDs, and we can use different colors to show different levels of earthquake. However, we realized that this indication might not be straightforward enough and might confuse our users. I suggested making a box and putting all the red LEDs in the box. Jennifer added that we can print a 2D Birdseye representation of the city on it. Eventually, we decided to make a box and use 5 red LEDs for each player.

Fabrication and Production

I was in charge of the coding part. And the biggest challenge I met was how to communicate between two Arduino. Our acceleration sensors connect to Arduino via I2C bus. And I knew that I2C bus protocol supports connecting more than one salve devices on one master device. However, the address of all the slave device must be different. Both our sensor has the same address. I searched online but still couldn’t change the address. I even asked Jonathan to borrow one Arduino MEGA2560 but still cannot solve this problem. My final idea is that I connect one sensor to one Arduino and combine two Arduino together. And here came my second challenge, which was how to communicate between two Arduino. Since the I2C bus was occupied by the sensor, I chose to use the serial I/O. But occupying the physical serial port will cause a lot of trouble in the debugging process. Fortunately, Arduino has one library named SoftwareSerial which can simulate serial port on almost all digital pins. And its interfaces are exactly the same as Serials. This helped me a lot. I quickly expanded the sensor code and created a simple communication protocol. It’s tricky that two Arduino don’t necessarily run synchronously. In the logic of sending a message to the other Arduino, I put some while loop to cope with the possible time mismatch. My first version program quantifies the shaking motion and lights up more LEDs when the user shakes harder. However, if the user shakes too hard, many LEDs will light up altogether in a sudden. Later we made a change: LEDs light up according to the accumulative amount of shaking motion. This change made the response much user-friendly. In the user test session, we found that everyone enjoyed the game, but they could hardly figure out when the game starts. One suggestion was to change the LEDs from respectively turning on to turning off. Jenifer said it’s like buildings losing their power. Another problem was that the button sometimes needs to be long pressed to take effects. After the session, I adjusted the code to fix these issues and add some start-up animations.

Conclusions

Our goal is to build an entertaining multiplayer game to tell which player’s earthquake is stronger. This project is interactive because the LEDs will turn off as a response to users, and users are also encouraged to continue shaking as the LEDs turn off. Our project is successful, as people were happy to play with it and we cooperated quite well. However, there is something we can do better. We can design more indication in the game. For instance, when the game starts, four LEDs may light up while one LED blinks. It may help users better understand that their goal is to turn off all the LEDs. Additionally, our project can be more educational. We can put digit dioxides on the box to display the level of the mini earthquake so that kids can have a clear understanding of the earthquake measurement. Throughout the whole process, I’ve learned a lot. I not only learned about advanced Arduino programming skills but also realized the importance of user tests.