Name Rosie Gao
Instructor Eric Parren
Project Description
The goal of my project is to study how the image we see affect our feelings of items. I believe there is a relationship of synesthesia between visual and touching senses, and I would like to see how strong this relationship can be by showing users similar but different items visually and through touching at the same time.
My inspiration for the project comes from my own experience of some kind of the rubber hand illusion, that when the researcher sees a rubber hand being touched while being not able to see his or her own hand, he or she will feel like they are touched as well. I was fooled by a similar illusion while being touched in real life during playing horror VR games and got frightened by how I felt about a cold human’s hand while seeing ghosts around me. I am interested in how our brains get confused two different sources of senses information and I decide to play with this illusion, which might work in the entertainment fields like gaming, or other fields like dealing with patients’ fears against a treatment.
Perspective and Context
In his work The World of Perception, Merleau-Ponty describes how Descartes understand illusion created by senses interaction that:
When I assume I am seeing the wax, all I am really doing is thinking back from the properties which appear before my senses to the wax in its naked reality, the wax which, though it lacks properties in itself, is nonetheless the source of all the properties which manifest themselves to me. (Merleau-Ponty 42)
Such an illusion is the reason why Descartes makes the claim that we should release our intellect from the confusion and trust the leads of science, however, Merleau-Ponty holds the point of view that “we have rediscovered a way of looking at works of art, language and culture, which respects their autonomy and their original richness.”(Merleau-Ponty 101) by considering perception as part of our natural experience with life and art. It is an experience worth studying, researching and playing with, as we cannot separate the concept of an item from our perception of it as an animal. In my case, I would say the rubber hand illusion is an interesting topic to work on because it is not only a process to know about how our brain is fooled by our senses but also a way to learn the features of different items in different users’ experience. I believe such a process contains the beauty of variation and fun.
Development & Technical Implementation
In my initial design, I was going to put touched items in boxes with sensors and use serial communication between Arduino and Processing to call out images of seen items on the laptop screen to create a connection between two items. After proposing this idea in class, I changed my mind for I have a limited amount of time and also a more simplified lower-tech solution can actually work even more effectively as the user have more intimate experience with the items if I just put the image on the top of the box.
There are three main decisions I make which are very important for the final result of my project: the container of items, the pairing of items and the methods of presenting the items. For the containers, I first want something as a draw box with a hole on the top that the user can stick their hand in. However, I want the users to be able to directly look at the images while feeling the items, so I decided to cut the hole on the side of the container. I get the material from two 4L mineral water bottles, cut them in half and create holes on the side as shown in the below picture. Then I used hot-melt adhesive to stick unclear plastic cloth around the hole to block the users’ sight without causing trouble for them to touch the item. I also put a black foam paper below to make it neater, easier to set up and more stable.
For items matching, I had a couple of ideas before I narrowed them down to four. I mainly picked them according to my personal experience (bitten by a sea urchin once), cultural mixing (Ma Jiang and Toffee) or just simply by how interestingly gross the experience can be (bogger and sticky peach juice over your hand). I did some user test among my peers, friends, fellows, and faculties on whether these items can be the trigger of the items in the images, and then I collected them from my belongings or the internet. Then I printed out the photos and stuck them on the plastic cloths.
To experience the project, the users can just simply put their hands into the hole under the cloth while looking at the pictures. After feeling them, they can take out the items and feel them again while looking at them to compare two experiences. I was planning to have all the containers out at the same and ask the users to choose on their own of the order to feel, but prof Parren gave me the advice to think about present them one after another to keep the user focused and curious. It was helpful.
Presentation
(Taken by my peer Candy Bi, 2021)
You can tell from the video that the user appears to stay curious within the whole process. Some similarities were fully understood while some were not, which is in my initial design. However, I was pushing the process to actively, taking the time for the user to experience carefully, and using too many words to lead her imaginations. The process turns to be much shorter than I planned this way.
Conclusion
Generally, I am satisfied with the final work of my project. I built up the relationship between two senses as planned by using the concept of the rubber hand illusion, and I chose a low-tech method to make sure I get the device ready in time. I found some very interesting items to show based on my user survey and daily observation, and I made good use of materials I have to build the devices. The users do experience a sense of confusion and find it interesting, which proves my choice of containers and items matching was effective.
However, I was focusing too much on the outcome of the presentation rather than a result of my study, the user survey on how they feel about the items rather than how they feel throughout the whole experience. As a result, though I was expecting to have different feedbacks, I did not get a clear response on how users feel differently while looking at the image or the item itself, instead, they gave the answer of “it does feel interesting of being fooled by my vision”, which is unnecessarily bad, but not in my plans. Second, I did not design the presentation clean and effective enough by only giving instructions and leave the user to decide what they feel about the device. I chose to guide their feelings, which might take their attention away, and that is a part I did not prepare for and should approve in my future projects.