Author: aaf425

Project Title: (Currently Undecided)

For me, the purpose of this project is to explore interaction and its connection to our senses. My project is in the form of a video game, utilizing sensors as the main force of interaction. However, whereas many video games are heavily based on visuals, this game is going to be based more off of audio effects. This way, the interaction fostered by this project is going to feel different and may perhaps be a bit strange for those testing it. However, as artists and designers, it is important for us to open our minds into the different ways which we interact with computers and how computers interact with us. By simply relying on visuals, we are leaving out many other factors which may aid in creating a worthwhile experience. From an artistic point of view, I feel that the way in which we are making users interact with this game is going to create an experience which may feel a bit out of place. However, I feel that this feeling is something which helps us to understand the effects of meaningful interaction.

For this project, I am working with a partner, and she has gone and fleshed out a storyline for our game. As I am well-acquainted with various coding languages, I am acting as the head of logistics in this project. However, my experience in the past has all been based on visuals, and therefore I am looking to broaden my experience by working with an audio-based game. Also, I have not worked closely with interactive project in the past and this game is my way of testing out opportunities for future work.

In my mind, the building of this project is separated into parts. The first part was to build a storyline for the game and plan out the choices which need to be made by the player. As my partner is basing this game off of her story, the choices are pretty much planned out and ready to be implemented. That being said, the choices may need to be revised as we try and implement the other aspects of the project.

The next part is the interaction. For this, we are using two distance sensors, one for each of the player’s hand. As the one in charge of the programming, I am planning on creating a boolean value as to whether or not there is hand covering the sensor and passing this value into processing. Once the values from the two sensors are in Processing, I can begin programming the choices. To do this, I am planning on using functions and conditionals. To avoid having too many loops inside of loops, I am going to trigger certain functions based on the sensor values along with how many of the other functions have been called already. Therefore, the programming is heavily based off of asynchronous callbacks within Processing. Though I am expecting the code to get a bit messy, I am confident that I will be able to work it out.

After the interaction is working, the next challenge is to figure out the sound and the visuals. Though this project is not heavily based off of visuals, I feel that the text should at least be displayed on the screen to create more accessibility and ease of use. For the audio, my partner and I still need to look into certain audio libraries and find a way to incorporate these into the code. We were planning on having the player wear headphones whilst they interacted with our game, and therefore we will likely process the audio files in Processing. From a logistical standpoint, this will also simplify the code as the values will only need to be sent from Arduino to Processing. This part of the project development will be the part which takes the longest, as it is the main framework behind everything else. Therefore, I intend to get it done within the next week so that the rest of the project may be built.

Lastly, we will need to combine all these elements together and run the project through user testing to ensure that it is at the state with which we intend it to be. In terms of audience, we were thinking mainly about those who attend the IMA show. As there are many projects within the show, we knew that we could not make our game too long, and that headphones would provide the best quality of audio (due to outside sounds during the show).  My idea was that the addition of sound will be the last thing which is added to the project, as the audio files can be collected while the rest of the project is being built, and the files can easily be integrated into the programming framework.

Though most of the projects which I have researched in the past have been largely based off of their aesthetic qualities, I would like to say that they have influenced my ideas of interaction and have hence allowed me to better explore the opportunities for interaction within my works. One of the projects I researched is a work in progress by Memo Akten, We Are All Connected. In this work, Memo works with interaction via sound and hand movement. In this project, he uses a particle animation and has the particles move along with the audio playing and the movement of the hand in front of a sensor. Due to there being no tangible controls, I would expect the type of interaction here to feel rather natural as the movement of the hands is unrestricted. I would say that my project fits my definition of interaction as can be considered as an exploration of natural interaction. Along with this, the interaction within this project is multilayered, and requires communication between the player and the computer. As this project is an exploration of interaction, and as it is based off of a story, it can be seen as a new medium of storytelling and information delivery. Along with this, this project will also help me to understand how computers sense our interaction, and how we can work to explore multiple ways of interacting with computers, and moving away from more traditional methods. Therefore, I am hoping that through the execution of this project, that in the future I will be able to utilize these new mediums of interaction. From a logistical standpoint, interaction doesn’t just have to be swiping across a screen or typing, and from an artistic stance, new forms of interaction could be use to create new experiences of art. With these values in mind, I would consider the largest significance to be the exploration into different modes of interaction.

My original definition of interaction could be defined by understanding the relation between two items, wherein one item gives a response to the other. Prior to the group project, my definition was relatively basic, where the only addition to it was that one of the actors must generate a response based on the input of the other actor. Originally, I did not consider much to how interaction should feel, how it should be designed, or how it can be evolved.

After completing my midterm project, I found that interaction should be something which occurs naturally and without much thought. Therefore, the ways in which we design interaction become incredibly important, especially when considering the development of technology and how we as humans choose to communicate with machines. My definition has also changed to include the idea that interaction should also flow back and forth between the actors involved.

One work which I would like to bring to light is Memo Akten’s We Are All Connected (WIP). While this is still a work in progress, there are videos of demos on his website (link here), and one can get the sense of how this project is meant to work. By observation, it looks like there are two methods of interaction here: sound and hand movements. The screen contains graphics created by particle generation and movement, and there is a sensor to read the motion of your hands. All demos are paired with music, in which case it appears that the particles move along to the rhythm. Without watching closely, it seemed as if this project was not interactive, as it appeared as if things were moving randomly across the screen. However, after looking closely, much of the interaction here is very implicit. A sound sample is given (though it is unclear whether or not the particles are reacting to the sound or are just programmed to match), but through the sensor and hand movement, people who are viewing the art can push the particles around using their hand. The interaction with this piece appears to be very natural, as one simply needs to move things with their hands or arms, and it appears similar to how we might move our hands while underwater. I choose to consider this project interactive as it is inviting to the viewer and one may easily figure out how to interact with it.

While doing research, I came across this project done by a Russian designer for the 2012 Architecture Biennale in Venice, named “i – city”. At the Russian pavilion in this biennale, designers Sergei Tchoban and Sergei Kuznetsov worked together to create an exhibition where the walls of the pavilion where covered in screens displaying QR codes of varying brightness. Before entering the display, people where handed a tablet which would be used to scan the QR codes which were lit up. Once scanned, the tablet would display information about a new city being built next to Moscow.

(photos courtesy of designboom, 2012)

To some, this project may be considered interactive as the viewers are required to bring in a device and scan a QR code. However, in my idea of interaction, the response given after the action must be more tailored to the action. I feel that the simple action of scanning a QR code could be considered interactive on a very basic level, but it lacks appeal to those who choose to interact with it. Visually, the project is well done and it is inviting to me in that sense, but after the QR code is scanned, the interaction ends and you are simply left to read a screen. Good interaction should take the form of an ongoing conversation between actors, and it should not unnaturally come to an end. Moreover, the medium for interaction is odd. It’s an effective way of getting people to read information, but it is not a way in which we would typically interact with something, and so to me it feels stagnant.

After reading A Brief Rant on the Future of Interaction Design, I feel that things which are interactive must also utilize different mediums of interaction and be more tailored to the things which use them. While scanning something may be a great interaction between two pieces of technologies, humans are not naturally made to “scan” things. Therefore, when redefining how I think about interaction, I feel that interaction should be natural. I tend to consider interaction as a conversation, but the way in which one actor converses with another is crucial in defining interaction. Conversations tend to natural and mindless. Therefore, in interaction, the way in which one thing interacts with another should feel natural and inviting. However, I do feel that there is something creative in purposefully making interaction unnatural. Aside from this, I would like to echo the idea that interaction should be a natural conversation between two or more actors. After one actor makes an action, the other should reply. There should be no loose ends and the conversation between the actors should flow smoothly.  In my definition, I would also include Crawford’s ideas of the levels of interaction, and I would say that there are tiers of complexity within types of interaction, but most importantly, interaction should be something which feels natural. Interaction is something which can occur between humans, between humans and machines, between animals, and between machines.

Sources Referred To

Art Goes Interactive – Brian Nadel (2013, ComputerWorld)

A Brief Rant on The Future of Interaction Design – Bret Victor

The Art of Interactive Design – Chris Crawford