Author: Phyllis Fei

I find myself particularly enjoyed McLaren’s Synchromy so I researched him and his work.  McLaren attempt[ed] “a ‘visual translation of the music,'” and what makes his work so special is that he considered films “‘a musical script’ instead of dialogue” (Chan).  His creativity and ambition drove him to produce music/sound visualization that we are appreciating now. Synchromy consists of “optically printed vertical bars with horizontal blocks appearing on them” in synchrony with the soundtrack (Valluri). I found out that he composed the soundtrack on the piano first before “print[ing] it onto the optical sound-strip section of 35mm film stock,” then he duplicated the optical sound patterns onto the stock’s visual frame (Valluri). 

According to Chan, McLaren was inspired by many of his peers and experimented with developing a deck of pitch cards that worked as a keyboard (which allowed the exact musical pitches of a piano to be photographed on to the soundtrack) with the filmmaker Evelyn Lambart. He marked up a lot on filmstrips with “an array of pencils, pens, brushes, razor blades and other tools” (Chan). By controlling the shape, thickness, and number of slashes, he was able to produce tone, volume and pitch variations (Chan). He arranged these shapes in sequences on the analog optical soundtrack to produce notes and chords, and then reproduced the sequence of shapes and colors in the image portion of the film (Synchromy). The well-syncing visual and sound make audiences feel like “hearing the shape,” or “seeing the sound.” What McLaren had achieved is not only about enhancing the listening/watching experience, but also proving the possibility that sound could be visualized in a smart way — the moving colorful visual actually comes from the soundtrack, “the part of the filmstrip that contains the audio” (Synchromy).

Works Cited

Chan, Crystal. “How to Write a Film on a Piano: Norman McLaren’s Visual Music: Sight & Sound.” British Film Institute, Apr. 2014, www.bfi.org.uk/news-opinion/sight-sound-magazine/features/how-write-film-piano-norman-mclaren-s-visual-music.

Millar, Gavin, director. The Eye Hears, the Ear Sees. Internet Archive, National Film Board of Canada, 1970, archive.org/details/theeyehearstheearsees/theeyehearstheearseesreel2.mov.

“Synchromy.” Wikipedia, Wikimedia Foundation, 30 Sept. 2019, en.wikipedia.org/wiki/Synchromy.

Valluri, Gautum. “The Eye Hears, The Ear Sees: An Investigation into the Visual Influences of Sound in Film.” lafuriaumana.it, www.lafuriaumana.it/index.php/60-archive/lfu-27/507-gautam-valluri-the-eye-hears-the-ear-sees-an-investigation-into-the-visual-influences-of-sound-in-film.

Here is the link to my gist.

Process

I started with experimenting with the first synth we created in class, planning to use it to generate the main melody. I had a vague idea of making the melody a bit vacant but still lively, so I chose relatively low-pitch notes and experimented their sequences in the SEQUENCER module. I set “steps” to 30 and “pulse” to 1/16 so that the melody sounds more energetic. Then  I basically played around with more parameters such as frequency, offset, feedback and so on to enrich the sound a bit more.

Continue reading “Assignment3: Second Synth (Phyllis)”

I was focusing on every single word Lucier said when the recording first began. There are a couple of times when he intentionally pronounces words weirdly, and I kept questioning his purpose of doing so throughout his 45-minute piece (even though I still haven’t found out in the end). At first, there was only his voice repeating the same content again and again, and I was able to tell where exactly his weird pronunciation appears. As the piece moves on, his voice starts to echo and becomes less clear. I felt his voice gradually moving farther away and I started to vaguely hear new sounds in low volume from the negative space (when Lucier’s voice is not in the air), which sounds more like white noise. The human voice becomes vaguer and vaguer while the echoing-like sound is more dominant. In the end, recorded sound from Lucier is completely removed and I could only hear new acoustic sounds generated from the space with a constant frequency.

Lucier’s idea of turning his own voice into an abstract sound art piece without any post-production manipulation really surprises me, and the result turns out to be really satisfying and unexpected. He challenges my previous understanding of music/sound production that producers select sound elements from databases and remix them in softwares. As is mentioned in Martha Joseph’s journal, Lucier treated “space as a component of sonic production,” and reveals a “radical reversal of the logic of musical composition.” This practice stretches sound production to a level that lets sound produce sound itself through engaging in its physical environment (in this case, the walls). It offers audience auditory experience through time and space.

Link to my gist.

Demo Video

Process and Explanation

I chose 2 sample videos (AR_8BIT_2 & AR_8BIT_5) from the provided folder. I used LUMAKEYR from Mix-Composite to mix them together, then adjusted the parameters (tolerance and fade) to have both videos be shown at the same time. Then I added FRACTALIZER to transform the visual by adding more objects horizontally and vertically and coloring each sub-video with tint mode on.
To make the visual more dynamic and random, I passed data from ATTRACTR to FRACTALIZER, which consistently changes the number of sub-videos in columns and rows.
Then I added FOGGR to give the combination a blurry/more pixelated effect at a set probability.I added WANDR and passed one dataset from ATTRACTR to give it more randomness. However, I don’t find a big visual difference…

To experiment, I added DELAYR and passed 2 datasets from ATTRACTR to create a delay and crossfade effect.

Synesthesia is a neurological phenomenon in which stimulation of one sensory involuntarily leads to experiences in a second one. A Japanese artist, Ryoichi Kurokawa, is one of my favorite artists who makes works that provide synesthetic experiences for the audience. One of his representative work, Unfold, investigates the audience’s ability to perceive noise as colors and shapes. It is an audio-visual installation that shocks our senses with uncomfortable static noise (Emrich, Neufeld, and Sinke mention in the article as “inducers”) and visual variations in shapes and colors ((Emrich, Neufeld, and Sinke mention in the article as “concurrents”).

Synesthesia helps build strong connection among objects that seem unrelated, which makes pure noise as an art form in Kurokawa’s case. On top of such a connection in artifacts, we as the audience think of both the sound and the visual as a whole, believing that they together form a coherent piece. In Unfold, Kurokawa matches less intense sounds and rhythms to less extreme visuals (colors and shapes), and vise versa. I see visual reaction coming along with noise, while others may see it as if noise is triggered by the visual. Noise is thus visualized through artists’ perception and is eventually presented to the audience. Synesthesia in  Kurokawa’s work materializes abstract notions/concepts and stimulates our visual sensations

Link to Unfold by Ryoichi Kurokawa.