Category: capstone2022

 
This project explores the impact of haptic feedback on human emotion perception in virtual environments through offline experiments. The project begins by searching and integrating data from online databases. Since there are many kinds of research on VR, haptic feedback, emotion, etc., and the three are strongly correlated to a certain extent, the experimental design and process will refer to some previous papers. Reference papers are mainly from the APA (American Psychological Association) database and Google Scholar. After experimenting, I decided to go a step further and make a VR project that would help people play with their pets (whether dead or away from home) and get some emotional experience. Due to Covid-19, people have lost many opportunities for real-life contact. VR brings people out from lockdown. However, the interactive settings of many VR experiences are not coherent in the design of the operation. But many VR experience interaction settings are not coherent in the design of the operation. For example, player displacement in the Lab needs to use ray cast to select a point and then jump over instantly. There are currently very few pet-related VR experiences, and secondly, these projects rarely design haptic elements. For the virtual part, I first built a lifelike VR scene in Unity, then imported the animated animal models and added some AI and interaction scripts to the characters. For the haptic experience, I used fabric materials and rubber toys to simulate the touching experience. In this project, viewers sers can interact with virtual characters in a state of high fidelity and a high degree of freedom.

 
Entering Fuxing Park at seven in the morning you will hear the sound of multiple instruments mixing together. Going closer to a long and narrow trail in the southern part of the park, over 15 saxophone players are practicing a mix of 1980s pop and communist revolutionary music. The public spaces music activity typically happens in parks and pedestrian walkways in the city, and Fuxing Park is a typical representation of the activity.

Besides the onsite music activities, the emergence of online live streaming platforms incentivizes the online public music activities. Apps like TikTok or Kuaishou, hosts live streaming music performances that allow intense interactions with the audience. The platforms construct a virtual park for the audience to wander around and listen to various musical performances.

Confluence of Melody is a site-specific sound installation that links offline music with the virtual soundscape. Onsite public music recordings were collected from parks and open spaces in Shanghai, these are combined with recordings of virtual live streaming performances which were collected from multiple hosts in TikTok.

The physical installation consists of a park bench and seven speakers placed along a walking trail. The audience is invited to walk around the trail or sit on the park bench to listen to sounds generated from different speakers hung throughout the space. The speakers are similar to the one park musicians often use. The speakers, path, and bench recreate the environment of a public music park.

The project imagines that every source of sound forms a river, which runs and splashes into each other. The audience can walk into the confluence and feel the different stories and emotions embedded in the river.

 
Dance is far away from quotidian lives. Our bodies are so used to daily movements that become restrictions on the infinite possibilities of bodily movements. This project aims to inspire people to move freely and out of their movement comfort zone. With motion tracking technology and ground projection, the visual and body interactions are designed as choreographic objects.

 
With the faster development and increasing accessibility of new input systems, particularly that of Virtual Reality, it is time to reconsider what tools and processes may reap benefits from these new interfaces. Designing a digital 3-dimensional space has long relied on 2D data and more traditional input from a mouse and keyboard while displaying visuals on a monitor, of which the latter can create a discrepancy between the envisioned product and actual design. This project seeks to enhance the user’s ability to register their designs from a real-world, first-person perspective while utilizing the organic nature of VR’s input system to create- from early stage conceptualization through each iteration of ever finer editing splitting the scene into two general modes- one for editing the overall scene between a zone-editing layer, which uses primitive shapes to organize the scene, and a model-editing layer, to control the final design of the space, and the other for building new custom meshes by creating a closed shape then extruding its height to create a 3D object. While there are several existing products related to the topic, such as Google’s Tilt Brush, many market themselves as a novel tool, disregarding the process of conversion from what a potential user may be used to, and failing to insert themselves into current workflows which generally rely on rigid datasets, hence potentially limiting usage due to low benefits at the cost of the learning curve. Thus, for this project, collection of vector data is a key feature, allowing fine editing through the UI along with the natural controls in the 3D space using the controllers, while also creating the potential for porting creations to other programs.

 
An ARG can be defined as a game that is not restricted to one medium while relying on player interaction to develop a narrative that is presumed to be grounded in reality. Its open-ended format allows for creative freedoms other storytelling modes cannot offer. By contextualizing the narrative within the realm of an authentic space, the content of the story can be perceived as “real” by extension. ARGs are often made up of web-based media objects spread across space and time, both digitally and physically. It’s in this overlap of fiction and reality can a new level of intimacy between the audience and story can be formed. One that is perfect to be exploited in the realm of horror. It’s this invasive nature of the ARG that this project seeks to explore while introducing new audiences to the genre of ARG.

The ARG genre emulates the way a “real” situation would unfold through its object-oriented storytelling, a technique that relies on the media objects to convey narrative information, and fragmented plot, as there is a much bigger story beyond what Vi is explicitly saying to the audience. Audience members must dig deeper into the clues and hidden messages in the content to find answers and push the plot forward. Content reacts to audience input in real-time. Comments and audience input are acknowledged and play a part in how certain events play out. Reinforcing the audience’s feeling of involvement and stake in the narrative. However, this narrative structure requires precise planning and execution to create an authentic depiction of this “real” situation as it unfolds. Unlike some of the most popular ARGs, this one aims to appeal to an audience that is not familiar with the genre by overwhelming them with the typical elaborate puzzles and convoluted narrative. For what it lacks in technical complexity, it presents an interesting storyline with elements taken from the fictional SCP Foundation and places them in the NYUSH community.

Inspired by the SCP story of the same name, Star Signals the ARG follows Vi, a grad student visiting NYU Shanghai, who documents her increasingly strange experience living in Shanghai during the 2022 COVID lockdowns. Her Twitter and Instagram updates paint an unnerving situation wrapped in secrecy and otherworldly influence. Vi has enlisted the help of her followers to the bottom of this rabbit hole. Eventually, this leads up to a final message; a website she’s created to expose those responsible for the disappearance of a NYUSH professor. While also including specific instructions for those wholly devoted to fighting for their right to “become one with the stars”. But this doesn’t mean all secrets have been revealed. This ARG is still active even though Vi might be on the run.

So, are you willing to follow the signs and commit yourself to the church? Because after all, this is *not* a game, right?