Assignment 4: Granular Synthesis – Tilted Over

Github Link

I named my project Tilted Over since it involves rapid shifts that flipped the color completely.

As in the context of Granular Synthesis, there are some basic features for categorizing this kind of art form. 

1. It uses naturally recorded video and audio as the source of the final video and audio stream.
2. It slices the sources to pieces and rearranges them by playing at a different mapping state, a different playback rate, or with different audiovisual effects on top of it.
3. It should be having connections between the visual elements and the audio elements.

Based on these principles and the sample patch we did during class time, I started by arranging several video sources in position. Aside from working with the sample videos, I added another video clip of a shadowy figure that works as the background layer of the video. 

Enlarge

From Left to right, except for the second one, are all sample videos from the class, with control panels linked with the HAP video player. I chose some of the frames that included the most frequent movements, set the starting frame and playback rate, and set them into a message box so that they can be stored once the patch is closed. The values with being sent again once the toggle connecting all of them starts again so that the values kept consistent. Also, for the CROPPER module in vizzie, it is hard to manage all the parameters well without introducing other message modules. Aside from the loadmess and unpack objects, I added a button to trigger the loading so that any changes can be applied immediately and manually. 

During the production of this assignment, I reviewed the instructions which contain the sentence “Take into account the stroboscopic / stuttering nature of the video and design your sound so that it fits with the video images.” This guides me in making effects that oriented towards stroboscopic nature.

I became interested in the pixel operators that mix the videos. It is extremely simple yet capable of making visible effects on the intersections of two videos. Hence, this is where the background layer of video functions with the other 3 videos.

Enlarge

I set the stroboscopic effect with a cycle of in total 8 seconds, 3 seconds for the line function, and 5 seconds of delay until the next iteration. In order to maintain the best visual effect, I choose the absdiff operation to be the ground layer of color. For every 2 seconds, the metro sets the operator back to absdiff so that it maintains a steady color.

For the audio part, to follow the instructions, I believe I must synchronize the audio elements with the control signals of visual elements. Hence, but building up a small module that enables the translation between “bang” and beap trigger, I succeeded in synchronization. 

Enlarge

Enlarge

I used the granular model, and a simple sampler module to store the audio samples. The granular model is powerful enough to create sounds that differ a lot from the original sample. By synchronizing them together, the stroboscopic functions matches with the sound very well.