Final Project Research Essay – Alison Frank

Project Title: (Currently Undecided)

For me, the purpose of this project is to explore interaction and its connection to our senses. My project is in the form of a video game, utilizing sensors as the main force of interaction. However, whereas many video games are heavily based on visuals, this game is going to be based more off of audio effects. This way, the interaction fostered by this project is going to feel different and may perhaps be a bit strange for those testing it. However, as artists and designers, it is important for us to open our minds into the different ways which we interact with computers and how computers interact with us. By simply relying on visuals, we are leaving out many other factors which may aid in creating a worthwhile experience. From an artistic point of view, I feel that the way in which we are making users interact with this game is going to create an experience which may feel a bit out of place. However, I feel that this feeling is something which helps us to understand the effects of meaningful interaction.

For this project, I am working with a partner, and she has gone and fleshed out a storyline for our game. As I am well-acquainted with various coding languages, I am acting as the head of logistics in this project. However, my experience in the past has all been based on visuals, and therefore I am looking to broaden my experience by working with an audio-based game. Also, I have not worked closely with interactive project in the past and this game is my way of testing out opportunities for future work.

In my mind, the building of this project is separated into parts. The first part was to build a storyline for the game and plan out the choices which need to be made by the player. As my partner is basing this game off of her story, the choices are pretty much planned out and ready to be implemented. That being said, the choices may need to be revised as we try and implement the other aspects of the project.

The next part is the interaction. For this, we are using two distance sensors, one for each of the player’s hand. As the one in charge of the programming, I am planning on creating a boolean value as to whether or not there is hand covering the sensor and passing this value into processing. Once the values from the two sensors are in Processing, I can begin programming the choices. To do this, I am planning on using functions and conditionals. To avoid having too many loops inside of loops, I am going to trigger certain functions based on the sensor values along with how many of the other functions have been called already. Therefore, the programming is heavily based off of asynchronous callbacks within Processing. Though I am expecting the code to get a bit messy, I am confident that I will be able to work it out.

After the interaction is working, the next challenge is to figure out the sound and the visuals. Though this project is not heavily based off of visuals, I feel that the text should at least be displayed on the screen to create more accessibility and ease of use. For the audio, my partner and I still need to look into certain audio libraries and find a way to incorporate these into the code. We were planning on having the player wear headphones whilst they interacted with our game, and therefore we will likely process the audio files in Processing. From a logistical standpoint, this will also simplify the code as the values will only need to be sent from Arduino to Processing. This part of the project development will be the part which takes the longest, as it is the main framework behind everything else. Therefore, I intend to get it done within the next week so that the rest of the project may be built.

Lastly, we will need to combine all these elements together and run the project through user testing to ensure that it is at the state with which we intend it to be. In terms of audience, we were thinking mainly about those who attend the IMA show. As there are many projects within the show, we knew that we could not make our game too long, and that headphones would provide the best quality of audio (due to outside sounds during the show).  My idea was that the addition of sound will be the last thing which is added to the project, as the audio files can be collected while the rest of the project is being built, and the files can easily be integrated into the programming framework.

Though most of the projects which I have researched in the past have been largely based off of their aesthetic qualities, I would like to say that they have influenced my ideas of interaction and have hence allowed me to better explore the opportunities for interaction within my works. One of the projects I researched is a work in progress by Memo Akten, We Are All Connected. In this work, Memo works with interaction via sound and hand movement. In this project, he uses a particle animation and has the particles move along with the audio playing and the movement of the hand in front of a sensor. Due to there being no tangible controls, I would expect the type of interaction here to feel rather natural as the movement of the hands is unrestricted. I would say that my project fits my definition of interaction as can be considered as an exploration of natural interaction. Along with this, the interaction within this project is multilayered, and requires communication between the player and the computer. As this project is an exploration of interaction, and as it is based off of a story, it can be seen as a new medium of storytelling and information delivery. Along with this, this project will also help me to understand how computers sense our interaction, and how we can work to explore multiple ways of interacting with computers, and moving away from more traditional methods. Therefore, I am hoping that through the execution of this project, that in the future I will be able to utilize these new mediums of interaction. From a logistical standpoint, interaction doesn’t just have to be swiping across a screen or typing, and from an artistic stance, new forms of interaction could be use to create new experiences of art. With these values in mind, I would consider the largest significance to be the exploration into different modes of interaction.

Leave a Reply