Author: Chenlan Yao

Title

Social Media

Project Description

As the name suggested, my project is about social media. The socializing patterns have changed a lot due to the development of mobile phone and social media. In China, specifically, Wechat becomes the most frequent used chatting software and software for takeaway ordering such as ELEME (饿了么) become more and more popular. The development of social media and software changes the way we socialize a lot: they offer us convenience as well as more access to various information; however, they are also controlling us to some extent, as we rely and focus on them so much that we have neglected and lost some important aspects in life. Therefore, I want to embed the relationship between people and their socializing ways, in order to reflect the impact of social media nowadays critically.

Undoubtedly, the development of smartphones and social media created many conveniences in our daily life. In my project, I want to use some examples in real life (Wechat and ELEME) to reveal the connection between social media and people in a specific way. At the same token, I get inspiration from the topology theory to demonstrate the interaction between social media and people in an abstract way. As a geometric concept, the topology reveals the relationship among things with lines and its ramifications: arc, node, poly …, which can be a good example of abstracting the social network. The combination of specific (real-life examples) and abstract (topology) ways can reveal the complication of the interaction between people and social media.

On the other hand, social media has created problems, which is the other aspect that I want to include in my project. I got inspiration from a YouTube video called “Deleting Social Media for 30 Days Changed My Life” by Yes Theory. A critical question was raised: as we gain connectivity [from social media], what have we lost?”. To explore the reliance on social media for people nowadays, the host challenged himself to delete all social media software on his phone for 30 days and to see how he and his lifestyle had changed. The video aims at making people be aware of some aspects that have been neglected due to the dependence of social media; for instance, various activities including reading, working out, hanging out with friends without browsing one’s own phone and have no communication, can be easily achieved and enjoyed without social media. The host engaged more in his life instead of scrolling down the screen all-day and achieve nothing, which reflects the disadvantage that social media has brought to us. Therefore, I wanted to cover both the advantage and the disadvantage of social media to show the impact of it in an integrated and critical way.

Perspective and Context

Considering that the performance would be conducted in an underground club, I wanted to let the audience feel involved in my project and to do a typical VJ-ing performance. According to Eva Fischer in the chapter on VJing, “the viewer response [in VJing] plays an important role. Interaction with the audience, albeit on a subconscious level, has a big influence on the result of the performance, which never occurs in the same manner twice” (115). Under that certain atmosphere, I believe the more the audience gets involved in my performance, the better my project could work. Therefore, both audio and video I wanted to make an offer a sense of strong impact and shock: for the sound, popular trap beat and element of pop music would be contained; for the visual part, there would be simple patterns and shapes with strong-contrast color. Power and energy are the way I want to use in order to shorten the distance between my performance and the audience and to let them enjoy themselves during the performance.

For some specific patterns in the visual part, I gained inspiration from Ryoji Ikeda and his work Formula and Data Matrix:

Formula: https://www.youtube.com/watch?v=OwuI0sGm0rk&t=493s

Data Matrix: https://www.youtube.com/watch?v=k3J4d4RbeWc&t=132s

I really appreciate how he deals with simple lines and cubes. As the geometric concept of topology is an essential part of my project, the relationship and movement of different shapes in Ryoji Ikeda’s work offer me some stylistic ideas.

Development & Technical Implementation

In two to four paragraphs describe how you developed your project and how your final realtime audiovisual performance system works technically. What was your research and creation process like? Which things did you try out before arriving at your final setup? How was the audio created? How were the visuals created? What materials did you use to create them? Link to any outside source if you used them.

Describe the setup you used for the performance and how it works. What are the different components? How do they interact with each other? How do you interact with them? Include pictures, diagrams, and screenshots! Upload your Max patch, including a functioning snapshot, to Github as through the process described in Assignment 3 and link to it in this section.

Audio Patch: https://gist.github.com/Ellen25/29b33ef48300661610a88c68ca7475d5

Video Patch: https://gist.github.com/Ellen25/3954b13e9a25fcbbd96fa636a34e7724

Audio part: Because I want to use the visual part to reflect the geometric relationship, which is quite abstract, the audio part is more realistic. Several real-life sounds are included (Wechat, ELEME) in order to raise the resonance of the audience and to make them feel closer to the performance. I asked my friend to pretend to be the delivery man and record his voice. The content of his voice is the scenario of take-out delivery: “Hello, your take-out food is ready. I’m at the front door of the building. Can you come and get it? Wait what? Put it to the storage room? Ok sure. It will be in the storage room. Remember to pick it up. Thank you!”.I am sure that most people have encountered a situation like this, so the audience should be familiar enough to the content shown in the record. I want to keep the voice and words clear, so it is used as the very beginning of the performance. The real-life scenario is used as an intro leading the audience to the following performance and make sure they are engaged in the environment and atmosphere I created. At the same time, I asked my friend Bongani, who makes rap music, for one of his rap songs ELEME. I cut a part of the song with the lyrics “say 你好，你的外卖到了” (Say hello, your take-out food is ready) to add some rhythm to the delivery situation, and to make the audio more energetic. 

The following part is about Wechat. I recorded several sounds related to Wechat, including sounds of Wechat notification, Wechat phone call, Wechat account transfer, Wechat message sent and so on. These sounds are combined and composed in GarageBand, with some beats added to them to create a cadence. I also get help from my friend who helped me to edit a piece of the soundtrack with JL and I mixed several elements together. From simple to complicated, the density of the sounds changes as time goes by. It reflects the relationship between the users and Wechat: at the beginning, Wechat helps the user to communicate and offers convenience; after a while, the user becomes busier and busier with Wechat. Wechat starts to control its user’s daily life.

After the Wechat part, I added a record from a talk show Rock & Roast as a transition. The host used an example situation in a nightclub, in which a man used the advantage of the background music and the atmosphere to socialize. The situation the host performed is ironic as he wanted to show the contrast between him, a person who was too shy to socialize and could not step out of his comfort zone, and the man in the nightclub, and to reflect the problem in the socializing situation of the youth nowadays. I want to use this as the transition from the demonstration of some real-life situations to a reflective aspect. In the following part, I created some wavy and mysterious background soundtracks to lead the atmosphere of the performance deeper while adding some breaking trap beats to the basic layer to avoid being too dull. 

At the end of the performance, I made another melody based on my performance of Guzheng, a traditional Chinese instrument. The Chinese style and elements have been more and more popular nowadays in music and performance. It is special and cool to have Chinese elements combined with some modern techniques, which is undoubtedly one reason for me to keep it in my performance. What is more, I do not want my performance to end in a sad and deep way; in contrast, I want to leave some space for the audience to refresh and reflect for a while. Take their time to review what they have just experienced: how do they feel? What do they gain from social media? What do they lose?

Visual part: As I have mentioned before, topology is essential for my project. Jit.gl is the best choice for me to compose the visual things, as it offers an opportunity to draw and create geometric shapes. At the beginning, I was thinking about having various cool effects with various components in MAX; however, Eric suggested that jit.gl has already contained a lot for me to play with. Therefore, my MAX patch is quite easy to understand, as I copied and posted several 1D jit.gl worlds and made them overlapping with each other to create different effects. By changing the drawing model of the gridshape into lines and giving a close shape to it (sphere, cone, or circle), lines and shapes with lines were drawn.

In the Wechat part, I recorded several screen captures, including sending Wechat message and having a Wechat phone call and made them as the background. To create a chaotic effect as time goes by, I also combined two 1EASEMAPPERs to get a colorful and geometric background. The 1EASEMAPPERs can be added to several elements to make plain shapes colorful in some certain time. 

3D model has been contained as well. I asked my friend to make a 3D model of “ASSC”, referring to Anti Social Social Club, for the reflecting part (after transition); however, the final effect did not satisfy my expectation, which will be discussed further in the Performance part of this documentation.

Performance

Overall, the performance went well. Few mistakes had been made, which satisfied me a lot. Because I do not have enough time to manage too many knobs and variables, the audio part is mostly ready-made, and I focus more on generating the visual part during the performance.

The scale, dim and rotatexyz are 3 parameters that I generated most frequently during the whole performance. In the intro part, I generated the dim of the gridshape to change the shape from lines (1 side) to triangles (3 sides) and so, matching with the beat of the background music. For the following part, I generated the x and y scale and used the rotatexyz to move the shapes and make them overlapping with each other so that some special colors and shapes can be created. One thing I found out to be effective is the change of the MIXR mode, as the color and effects would have great differences with it. With the help of the MIXR mode, I turned the screen capture of Wechat phone call into the background of the performance, as it has a black background and a few colorful lines on it. When shapes from jit.gl overlapped with it, special shapes would come out. 

I planned a lot, but still quite a few problems came out during the performance. Eric said I was trying to do too many things at the same time. When I was practicing before the performance, the screen went fluent even I generate several knobs at the same time; however, the final effect was out of my expectation, as it kept on seizing up during the whole time. I was nervous and worried about that on the stage, but I just want to do something (or I don’t even know what I am doing at that time) to save my performance, so I kept generating the knobs and scrolling down the screen, which made the consequence worse. Fortunately, my friend commented on this and said “I didn’t find it. I thought you meant to do that as an effect.”

I tried to embed the 3D model into the project as well, as I have mentioned in the last part. However, because I failed to make special texture for it, the 3D model turned out to be plain and quite strange in the whole project, especially on a big screen. So I let it out for a while and removed it immediately during the performance.

Conclusion

At the starting point of this project I brain-stormed a lot of ideas that I wanted to put into it. I succeeded in focusing on some real-life examples (Wechat, ELEME) and to make them as specific as I can; however, I the narration of the whole performance needs improvement. The performance was divided into several parts, which did not move fluently enough as an integer. I tried to add transitions and phone call sounds to link these parts, but improvement is still needed for the emotional change during the whole process.I am not that confident with the final effect as my original thought is to make something beyond simply cool or lit, but conveys some points that can be further discussed.

For the technical part, I think I relied too much on software like GarageBand and JL rather than MAX when making the audio. I need to spend more time on developing audio with MAX and adding effects to them. In the visual patch, I have too many things (5 jit.worlds and several effects) and I believe there could be some ways to simplify it.

The experience of VJing is really interesting and I learned a lot, both technically and practically. A lot of accidents happened during the whole process: MAX breaks down frequently; some language on the objects that is theoretically correct does not work… but fortunately, I gained my first experience of doing VJ on the stage in a real club instead of simply being an audience as before. A music label planning to organize some performance even asked me to cooperate with them and to perform with them. The experience really taught me a lot and helped me to understand the VJ culture and their works better.

The concepts of VJ-ing, Live Cinema, and Live Audiovisual Performance are connected with each other in certain ways. They are all performed in real-time; they all need a good combination of audio and video. However, they still tend to show some differences in the way of performance and the content of a performance. Live Audiovisual Performance, firstly, is the broadest idea among them, which can be the summary of the other two. It can be audiovisual performance in any kind of scene, including clubs, museums, and exhibitions. VJ-ing, for me, represents random and free. It seldom relies on narration and careful preparation, as the performance is strongly related to the audience. The atmosphere it presents is reflected by the performance of not only the VJ but also the audience, which furtherly restricts its site to nightclubs or live houses where the audience tends to get more involved in the performance. If VJ-ing is shown in a cinema or museum setting, it would become a Live cinema. Live cinema is more like a combination of VJ-ing and cinema, as Menotti mentioned in Live Cinema“, it “encompasses forms of audiovisual performance that actively engage with traditional cinema to graphic conventions” (85). Based on the concept of VJ-ing which includes liveness and performance, Live Cinema shows a narration through its process of performance, which makes it more personal and artistic, as it is no longer random and can be remembered by its audience. Therefore, Live Cinema artists tent to have a kind of priority to VJs to some extent. According to Makela, a Live Cinema artist introduced by Menotti in his book Live Cinema, “Live Cinema creators feel the need to separate themselves from the VJ scene together, in order to establish their own artistic goals, which would rarely find an appreciative audience in a club environment” (93). Admittedly, the status of their audience are different, as the audience of VJ-ing has more interaction with the performer, while the audience of Live Cinema is quieter and sitting in a cinema scene. However, one can not easily separate them or even show a hierarchy between them, as they are both performed in real-time, they share some same methods, and most importantly, they both reflect and express the artist himself. They are art, but just with different content. 

Here is a sample of Live Cinema from the 2018 Live Cinema Festival:

It is easy to tell the narration in the video, as it gradually passed from the blossom of a flower to the connection between the flower and the person, which evolves step by step. 

The following is an example of VJ-ing from Pfadfinderei, a Berlin-based interactive media artist:

For VJ-ing, it is more about to create a kind of atmosphere that can bring the audience closer to the whole performance, instead of telling a certain story or idea. Comparing with the Live Cinema shown before, the Pfadfinderei live show is more like a collage, which contains all kinds of elements that can be used to bring the liveness to the whole scene, instead of showing something on the screen as the Live Cinema. 

Link to my gist: https://gist.github.com/Ellen25/5f3deafb41a829c51a9f8d5a19dd3447

Sample video:

Process:

I started this assignment from the 1D model I made in class:

I furtherly developed it by first adding another 2 dimensions to gl.multiple and adding attributes to gridshape. After editing the 3D model, I also used 1EASEMAPPR and 1PATTERNMAPPR to create a pattern with 2TONR to generate its color. Connected it with the 3D model created by jit.gl, the changing color pattern was added to the model. The final patch looks like:

I tried to use the 3D model downloaded from the Internet but I failed. No matter how I adjust the settings, the model didn’t show up clearly. I will try my  best to figure it out later.

Link to Gist: https://gist.github.com/Ellen25/5eb3f14f6e9c585cd079e719e1ec882e

Final audio effect:

Video:

Assignment4

I firstly added an audio clip from freesound.org as the bottom layer of the whole sound with a drug sequencer to control the rhythm of it.

The audio clip from freesound.org:

https://freesound.org/people/mauxuam/sounds/71106/

I selected this clip as I want to make the whole video more dynamic. The sound itself also conveys the basic idea of granular synthesis, as the rhythm of it is intensive and dense enough. 

Some effects have also been added to the original audios: chorus has been added to the second piece and the third piece has been added some delay on it. Both of these two effects are aiming at stripping the original human voice from the frames and making them abstract enough to represent the frames themselves. I want to let the abstract sound to show a kind of power and rhythm of the movement shown in the video. I kept the human voice from the first piece as I want there to be some sentences to be clear to create a contrast between the clearness and abstract sounds. I tried my best to make the effects close to the idea of granular synthesis. But I think there is still something I can do to improve and make the whole audio reflects a better understanding of granular synthesis.

VJing is strongly connected with DJ and music, helping and cooperating with each other. VJing is a combination of all kinds of fine art. With the accompany of music, VJing can show a better composition of rhythm and visual effects, which is a more concrete and diversified art form. As VJing has a relationship with the influences of music, VJing is a performative thing. It needs liveness. VJ artists long for the interaction and reaction from the audience. According to Eva Fischer, VJing “stands for liveness, transience, and uniqueness … VJing is pure improvisation” (108). The vivid visualization of music and the randomness is the charm of VJing. As is shown in the document viewed in class, many VJ artists suggested that they prefer to use some real, non-digital instruments such as DVDs to perform. Computer is the worst for them. Also, knobs are necessary for them. Both DVDs and knobs can be a little old-fashion for other people in this digital age, as everything can be done simply with a laptop. But for these artists, because they are VJs; they are doing some realtime performance in most times, it is better for them to have something in real as a kind of pure experience and feeling, and thus provide a better atmosphere to their audience.

Visual art is strongly connected with the music. At the same time, music is strongly relying on visual art, as it can help to create a richer and thicker atmosphere and thus to get the audience involved in the scene better. The following are two video clips I took from live houses: one with some VJing, and the other does not have VJing.

d7fe2d4943a9ae759d792a81a5488c

IMG_1426

It is obvious that the one with VJing can create a better atmosphere for the whole live house, and also make the music more interesting. VJing is a performance, which needs great involvement and connection with the audience, thus the most important thing for a VJ artist is to attract his or her audience into the rhythm he or she provided. In the videos I linked, the artist on the stage is originally and actually a DJ. I do not know whether he is controlling the VJ things, but the visual art does show a great combination with the music he produced, which is enough for us. As an audience, I am under the atmosphere he provided, and that is the purpose of the VJing.