Category: capstone2020

 
Ishii suggests, “A Tangible User Interface gives physical form digital information and computation . . . and making them directly manipulatable by human hands” (Ishii, Tangible Bits: Beyond Pixels XV). The essence of this medium is to bring extra information and interactive dimension to the physical surroundings. AR as a technology that has been applied to most mobile devices fits well with the concept of TUI. Therefore, this paper explores potential solutions that could apply AR to bring the user experience into a new dimension.
Besides, T.L Taylor comments on live-streaming by saying, “It is a form that plays with the boundary lines between audience and producer”(2018, Taylor). The above suggests that in live-streaming, the content is the joint effort of both the streamer and the audience. However, according to Robinson’s research on live streaming interactions, we find that the interactions between audience and streamers are still mostly dependent on verbal comments (2020, Robinson). Moreover, Twitch, as the leading platform in the game streaming industry, tried to make the audience engaged in the game-streaming by allowing the audience to control the game characters rather than the streamer. Indeed such interaction allows the audience to participate in live-streaming content making; it does not bridge the audience and the streamers. Instead, it gives a double identity to all the audience, where they are both audience and streamers.
Therefore, AR can bring a shared and mutual virtual space and allows extra information overlays on the physical surroundings. This paper will further explore the possible solution of combining the AR with streaming experience.

 
Visual Emotions: A Multi-sensory Documentary Photography Series is an audiovisual installation that creates a sonic viewing experience for digital photography. Unlike many galleries’ traditional way of exhibiting photographic works in physical printed forms, this project displays a series of three dynamic documentary photos on three separate screens. The audience is encouraged to listen to the sound from each screen while observing the audio-reactive visuals from the photos.
In my photography practice, each shutter press is triggered by an emotional resonance between the self and the environment. The use of audio data, unique to each photograph, enhances access to the personal emotions that arise from each situation. This photo series was taken to document one of Harbin’s oldest and most vibrant morning markets while sound clips were recorded at the same time from the environments where the photographs were taken.
The audio clips, which can be heard by the viewer through headphones located near each photograph, are processed into frequency spectra, and the size and range of the spectra are manipulated in Touchdesigner with multiple mathematical algorithms. These audio data outputs affect positions, colors, brightness and sizes of the pixels in the rendered photos displayed on 25.0 cm x 16.5 cm screens mounted at eye-level. By engaging both visual and auditory sensations from the audience, this project proposes an alternative display for photographic works and aims at opening up space for imagination and interpretation.
This project fuses the two senses to embody personal emotional resonance, a response to how economic instability caused by COVID-19 has impacted common people’s living conditions. The audio clips reveal conversations about the immediate conditions in the market. Through diminished levels of market activity that can be experienced in both sensory media, the audience feels the implications of the situation. This project speaks for the calmness within the ordinary as well as the longing for hope within helplessness during the toughest time in human history.

 
Mirror is a digitally-drawn animated film depicting the inner struggle of a person who experiences extreme self-criticism. The initial idea comes from my personal experience of doing self-criticism in an extreme way and ending up with negative feelings. Self-criticism as an objective evaluation of oneself can lead to improvements. However, based on Sigmund Freud’s theory of personality structures, while experiencing extreme self-criticism, the unbalanced psychological parts of an individual can reduce mental well-being. The way I visualize the two conflicting minds of a character over self-criticism is based on how Freud divided one’s personality into psychological parts.
In my depiction, I let my character experience harsh criticism towards his/her self to show how extreme self-criticism can lead to strong depressed feelings. When I was struck by the anguish of criticism, the inner struggle was hard to convey verbally but I perceived it in a visual form. In this project, I turned my visions into a film to convey this chaotic inner state. The black and white visual style fits in the dark tone of the story. There are two conflicting parts inside the character: one part of the character does self-criticism, while the other tries to resist it. The conflict between the two parts is translated into the action of consuming and fighting between a figure outside the mirror and his/her reflection. The figures in my film don’t have distinguished characteristics as I intend to present the self-criticism concept in a universal way. The character can be understood as every person so that audiences can project their feelings onto the film character. The usage of a mirror in the film is inspired by Jacques Lacan’s concept of the mirror stage. As a mirror can reveal the psychological state, in the film, the mirror serves as a trigger for the process of self-criticism.