Author: Joyce

Title

Butterfly Lovers

Project Abstract

By combining the physical device and digital effect from Max, our project tries to interpret Butterfly Lovers in a new way.

Project Description

Our project tells a traditional Chinese love story called Liangzhu (Butterfly Lovers). We use the original piano song Liangzhu played by the piano and the flute as the basic background music, and the blooming process of a flower to represent the love. The story happened in the East Jin Dynasty when the feudal thought was still strong. Pretending to be a male, Zhu Yingtai met Liang Shanbo on the way to school. They spent a few years together and both fell in love. However, Zhu was from a wealthy family and her father arranged her marriage with a rich man. Liang, who was a poor man died of disease of grief. On the way to her marriage, Zhu leaves the procession to pay her respects at Liang’s grave. She descends in bitter despair and begs for the grave to open up. Suddenly, the grave opens with a clap of thunder. Without further hesitation, Zhu threw herself into the grave to join Liang. Their spirits emerge in the form of a pair of butterflies and fly away together, never to be separated again. 

Our very first intention was to express a story like this since we are both familiar with the piano song of Butterfly Lovers, and this becomes one of the reasons why we wanted to present this story. Also inspired by the performance PANORAMICA 2011, we think to change different cut patterns (laser cut or paper cutting) to produce variable shadows is an amazing way to perform. Therefore, we decided to make shadows as the primary way of the performance, including the shapes of circles, lightings, waves, and rectangles. 

Perspective and Context

“[Light] is part of the universe of flux,” Wilfred said in the same interview, “and therefore motion is a necessary dimension, in fact, in any visual art involving light.” The small torchlight lying on a pile of books during the performance was indeed the most important thing. The light is the center of our whole performance and all the things that happened surround the light. The light went through the moving laser cut board and produce different kinds, layers of shadow. What’s more, we chose to use the partially reflective paper and stick them inside our laser cut frame, to produce the Lumia effect as Wilfred has described, also as the effect is shown in the LUMIA documentary (00:48). Combining the Lumia effect with the blooming of the flower endowed the performance with a feeling of the flower blooming in the universe of flux.

Lumia effect from the reflective paper

Development & Technical Implementation

From the very beginning, we decided to use laser cut to achieve the physical part of the performance device, so our development of the project is mainly divided into two parts: the physical laser-cut part and the effective treatment on Max. For our final setup, we have the physical device between the light source and the projection on the white paper. We have different patterns and components laser cut, for example, 3 parts of the flower, the butterflies, the bird, and four different patterns. On Max, we will change the mute mode of different parts of music during the progress, from rain sound, twitter sound, aeolian bells sound to the sound of thunder. Meanwhile, with the going of the story and music, adjust the effects like delayer, rotator, and Zoomer. Besides the physical part and digital part, we also consider other effects, for instance, the change of light color, to produce the Lumia effect and the use of smoke (dry ice) during the process. Therefore, we used partially reflective paper and paper that is reflective. As for how we developed the performance device, the interactions between us and the device are also divided into physical and digital parts. Based on the background music and the continuous promotion of the story, the projection and add of outsource music (e.g. the thunder) echo with the story plot. For example, when the basic layer of the song, the piano Liangzhu goes into the strongest part, we will add the sound of thunder sound, the video produced by EASEMAPPR will be a source of the final video since it will make the video flash quickly, while the laser cut board with lighting shapes in the front of the light source.  

 

Design of the whole project/ physical part

Final setup: flower component, frame, and effect of using partially reflective paper

Screenshot of our patch

Screenshot of the presentation mode

We also met some other problems. During the rehearsal, we found the shadow of the frame was annoying so we cut the board off. Moreover, our initial light source was some small LED lights, but their lights are too weak and hard to be controlled. We then borrowed the video light, but it contains a lot of LEDs so the shadow will be ambiguous. After that we tested the phone torchlight, the effect is good but the phone is hard to be fixed on the ground. We bought a torchlight from eleme but it also contains 6 little LEDs in it, so the result is the same as the video light. Our final light resource is the torchlight that contains only 1 light in it borrowed from another group. 

Cutting off the board

We bought a torchlight but sadly it is useless

For individual contributions, I am mainly responsible for the physical part. After testing the prototype of flowers and its shadow effect, I drew the draft of how flowers and the device frame will look like and turn them into ai file. One of the difficulties I have met is that the width of the petal contour is too small, so the laser may cause it to break directly. This also happened when cutting the wings of the butterflies.

Prototyping with paper

 

Screenshots of the design of flower

Adjusting and testing the width of petal contour 

 

Final Production from laser cut and their effects

 

I have also tested different distance of the track on the frame, when the position of the track on the left and right side was too low, the lower part of the board was super easy to be broken. I also contributed to the built of Max, when testing the visual effects, we found out several best effects that do not destroy the original video, also to use PRIMR instead of changing colorful pieces of paper in front of the light source.  

Performance

The final performance generally went well despite a few problems. However, during the performance based on my personal experience, it was a bit hectic. Since we only have two people and Tina is responsible for the MAX MID, I have to be responsible for all the changes in the physical device and the torchlight. One thing that really could go better is that we should have adjusted the distance between the camera and the wall so the NYU Shanghai wallpaper won’t be taken into the camera and the video would be much cleaner and purer. Another problem is that when I move the flowers on the track, due to its instability, the flowers were easy to fall out of the track, so before our final performance we asked Ellen for help. What’s more, since I was looking at the smaller television on the ground and there might be some reflections on the screen, so what I saw most of the time is really dark and in a mess, so I was really anxious and worried, while Tina told me later that it was fine and the Lumia effect can also be seen according to the audience, which was out of my expectation. 

Conclusion

Personally speaking, the most difficult part of the project is the research and design part, since we spent almost a whole week on it. At the same time when we wanted it to look great, we also had to make sure that it is achievable. However, the process is really meaningful. We not only learned how to produce a fantastic video with the effects on Max and to control and adjust Max on a physical controller but also how to figure out the best relationships between the story and the music, like when to add proper background music, also between the music and visual effects. But as Eric has pointed out, we failed to go further on the exploration of effect. Since we put too much effort into the built of the physical devices, we are tied too tightly with it, thus ignore to further discover the effects. The Eyeball from the first group gave me a lot of inspiration on the use of different visual effects and a better combination of video and audio. We built a pretty good shadow, but need improvements on the effects. 

Graphic score

Inspired by the graphic score from Concert for Piano and Orchestra by John cage, we started our graphic score from the top of the picture with a combination of piano and flute. The length and color changes of the rectangles reflect the change of rhythm and pitch made by the flute. Since our project audio is The Butterfly Lovers which is a traditional Chinese story, we came up with the idea of using Yin and Yang as the frame of our score. The main melody appears many times in the song, so our graphic score inside the Yingyang circle cannot be interpreted in a linear way anymore. With the repetition of the melody and mixture of sounds of rain, thunder, wind, stream, and birds, different stages of the story are able to be understood at the same time. The graphic score in Yinyang also shows a balance, even if music reaches its climax, it will eventually flatten out. When the chorus is over, it goes out of the circle on the graphic score. The extended calm line above indicates the quiet, ethereal end of the song, accompanied by the two endless curves, which represent the end of the song and the long echo.

Belson and Whitneys are all important pioneers in the industry or innovation of visual music. The shared the same essential, which is the Eastern metaphysics. Similar to the Whitneys, Belson desired the elimination of representational imagery and any association with the real world. The Whitneys created a completely new type of visual music, in both appearance and sound the works are electronic. What’s more, reenergized visual music by bringing three disparate vectors together. These two brothers had a great passion for creating new technology — they have invented Yantra and Lapis, which “not just a work of art but “an attempt to approximate mind forms.” Additionally, neither of them could have succeeded without each other. James’ work would not have been possible without John. John thinks no visual motion worked the way musical motion works. His work even inspired the special-effects artists of Hollywood. Whitney’s films seem to create a kind of ideal world, a universe of linked senses in which all elements — sound, shape, color, motion — are no longer merely related but are absorbed into one another.

While Whitney’s used contemporary technology to push visual music into new cosmic dimensions, Belson, who based in San Francisco, arrived at a similar place through the use of both old and new technology, including standard animation, optical printing, lasers, and liquid crystals. Unlike the Whitneys, whose small atomic globules seem to spotlight new technology, Belson never used an image that bespoke its origins. His work is trying to break down the boundaries between painting and cinema, fictive space and real space, entertainment and art. Belson and Jacobs were moving visual music from the screen out into three-dimensional space. The environmental aspect of the concerts and the breakdown of the rectangle of the film frame, which, as Jacobs suggested, “set visual music off in another direction, one that would attempt to involve the spectator in more fundamental ways, altering his or her consciousness to a greater extent.”

Brakhage, the Whitneys, and Belson were all interested in the transformative power of abstraction in motion and its correlation to music. According to Joshua White, who described the feeling of watching the Vortex Concerts, “everything gets into sync, and you sync it in your mind. …the very abstraction of the light show gave each person a personal experience.” Standing on an important transition of art during history, the films played a significant role as abstraction shifted from a concern with “depicting” things. Therefore, the distance between the viewer and the art object closed. These films must be seen as extensions of shifts in the general culture. Visual-music films are fundamentally bound with what Youngblood termed the “Paleo Cybernetic Age”, an era in which television was beginning to shrink the world, computers were beginning to extend the human mind, and space travel was suddenly possible. Though space travel is still not that easy today, those arts enable human beings to flourish in the field of human arts and keep up with the trend of the times. The light show is an art form derived from the exploding development of the visual show. Different from the Vortex Concerts or normal visual music performance, the light show drew together a wide variety of practices to create an immersive visual and sound experience. It offered a neutral place in which high art and popular culture, abstraction and representation, where all kinds of different arts could all be collaged together. As the ultimate synaesthetic experience, the interest in sacred geometry and cosmic archetypes finally burst out of the confines of art and flowed into the sea of mass culture. With the population of the light show, people around the world, especially those who are from London, New York, Los Angeles, and San Francisco, can have paintings, film, color organs, and music came together, enjoying the mixture of colorful art forms. 

 

A Fake La gloire à mes genoux

Project Description

My project is about a French song, La gloire à mes genoux from musical Le Rouge et Le Noir L’opéra Rock. From the very beginning, I thought that I could change the output of Max totally according to my thoughts, so I decided to make the melody and rhythm according to the song and also create a changing background that can be displayed (in my imagination) even in the theater. That’s how my idea came out.

Perspective and Context

Since we are required not to use any outside source video for our project, the video we produced is 100% from the audio input. Similar to Norman Mclaren https://www.nfb.ca/film/synchromy/ (though mine is much more ugly), our video and audio are one. Since my inspiration comes from musical, which is a form of artistic expression that shows strong use of visual music, synesthesia, and audiovisual systems, coordinating music and visual expression at the same time. Therefore, letting the audience see the change in visual with the change in rhythm is important for my project. 

Development & Technical Implementation

For the audio part, I produced the rhythm of the prelude, and I tried to have the melody of the first part but I failed, so I put the original song into the cell instead of an electronic video file I have downloaded. Therefore the audio output sounds a little more like the song La gloire à mes genoux that I am imitating. 

For the video part, Eric helped me put so much effort into the color style. Since the main color of the musical is red and black, I decided just to use those two colors. But later I found that the result is too monotonous, so I added more visual effects along with the red and black background from VIZZLE to make it look better. For the final video, the mode from 4MIXR helped me so much to change the visual style. When starting my final project, after twisting the piano roll sequencer and upload the music file in CELL, it will produce a changing pattern with dominant color schemes of red and black and the rhythm of the first part of the song. 

 

The p fader that helps the color in PRIMR change only between red and black  

The mode in 4MIXR I have chosen

What’s more, there were a few problems when I tried to build my patch.

 

Only with these overdrives on can I run my patch fluently

The SMOOTHR and DATASLIDR will slow down the signals

One of the most important components is the 3EASEMAPPR, which transfer my audio input into video output.

My gist: https://gist.github.com/Joyceyyzheng/e137b937b95d665e6968f0f8153451bb

(But it works only after you twist the sequence of the piano roll sequencer to 8 and put the song file into the CELL.)

https://youtu.be/7HZ8Yc4NTcI   

The video of my project but unfortunately it only has the video part.

Presentation

The presentation generally went well. But the period of my video is pretty long, so it would be better if I can have extra time. The suggestion of making my video brighter is really helpful, but since the background of my video will change into red, it might not work that well due to the change of brightness. I also failed to play the melody of the original song, which made my project sounded worse.

 

Conclusion

Fortunately, I went to the theater and watch Le Rouge et Le Noir L’opéra Rock before writing the project documentation. In the movie, the song being played is just the one I chose, and this musical has been praised by its amazing “ppt” background for many times. We can see how the shape in red and black changes. The movie I uploaded is the encore part, while during the musical, shapes on the background even changed more, like the black hole expanding and shrinking in the burning flames. That shocked me a lot especially when I started to think about my project.

https://youtu.be/bzuboFH1eWk

 But I still learned a lot from doing the project. I knew how to create an audio or a video before, but not know how to create them at the same time. Though I failed to create the real background of the song, what I made could be another translation of the song. For some future improvements on the project, I would add more patterns and changes to the video that looks cooler. I will also work on the audio input to find a better way that allows my patch to play the melody I want. 

 

My very first impression of LUMIA was aurora. No matter the static pictures or the dynamic ones, they all look like something from beyond the sky, as a reviewer for the Sun has pointed out, LUMIA is something that “changing and extending as far as the eye can reach”.  However, with the development of LUMIA art, people also gradually found the relationship and similarities between LUMIA and the aurora, or even the nebula. But still, as mentioned in the documentary, there would be no same LUMIA in the world. It is beautiful and unique. In the documentary there was a scene (00:48) where a simple LUMIA installation was working, it feels like when I put my glass/plastic bottle under the sunlight. It could produce magical light but the process is so easy, which was a little confusing. 

According to Earl Reiback, Wilfred “deserves far more fame than he got, cause he created a truly new art form”. Wilfred came up with the idea of light art when there were few art forms like that. Beginning in 1734,  a model for the clavecin ocu/aire (ocular harpsichord) appeared. The device “combined candles, mirrors, and colored papers to produce light reflections whose seven spectral hues correlated to the seven tones of the musical scale”. But what Wilfred thought about was an instrument that “solely for silent compositions radically differed from the earlier tradition”, which his audiences had never experienced before. That was the foundation of LUMIA art. Wilfred’s road of exploring was not that smooth, though he received so many praises from so many people in the society. For a period of time when he turned his attention to projects that could supplement his income, but we still cannot deny his great work and contribution. “Wilfred is important as scientist, inventor artist, even though he is being forgotten he is still important.” Having positioned LUMIA “as an art rooted in scientific underpinnings of the greatest importance to the human experience”, Wilfred can be said as a successful artist who created a truly new art form, though his success “appears to have depended largely on his ability to articulate a relevant framework for light itself, one that connected LUMIA in timely ways to the questioning of the essential truths of the broader universe”.

References: 

LUMIA