Coded in collaboration with Lisa Jeong
Inspired by the classical Monty Hall problem the player must choose a door to open. Only instead of a car, they win a CAT!
Sometimes you win, sometimes you lose, and sometimes the consequences are unpredictable. Refresh to play again and see if you get a different outcome.
Last weekend, I went to the Whitney Biennial 2022, where at some point I noticed a group of people lying on their backs, enchanted by the movements on a set of ceiling mounted LED panels. Upon closer inspection I realized that they are watching an ever-changing pattern of shapes and colors that resembled nature’s textures like the ocean, jelly fish and blossoming flowers. The screens were interwoven with an LED net and surrounded by large aluminum panels on the walls, dripping with oil dents. The atmosphere in the room was otherworldly to say the least.
But what was truly fascinating about the installation by WangShui entitled Titration Print (Isle of Vitr⸫ous), Hyaline Seed (Isle of Vitr⸫ous), and Scr⸫pe II (Isle of Vitr⸫ous) is that this work was ‘co-authored’ by AI. The visuals displayed on LED panels are generated by Generative Adversarial Networks (GANs), which change based on the conditions of the surrounding environment that the AI ‘senses’. In particular, active sensors pick up light levels emitted from the LED screens and CO2 levels from the viewers and use that data to evolve the screen’s patterns. As a consequence, during the night, the piece enters into a form of a resting state. The ability to adapt to and sense the ‘self’ and the surrounding world reveals a conscious and living quality of the art piece. The aluminum panel paintings on the side walls are also created by using AI generated “new images from [artist’s] previous paintings which [the artist] then sketches, collages, and abrades into the aluminum surfaces.” [source].
Both the animation and the paintings mirror nature’s textures, colors and curves, giving us a hint into what the training data consisted of. In fact, during an interview with artnet News, WangShui mentions that the training dataset is a depiction of their research subjects and in a way is the artist’s journal. It includes ‘thousands of images that span deep sea corporeality, fungal structures, cancerous cells, baroque architecture, and so much more.’
As I laid under the screen, mesmerized by the evolving shapes above me, I thought about how important the artist is in curating and framing synthetic media. Generative algorithms like GANs sooner or later will become a commonplace in artists’ toolboxes and I look forward to being a part of the community experimenting with this medium.
This week I started developing circuits with the microcontroller using Arduino IDE. I also learned how to solder, which was new and exciting. Overall the labs went smoothly with a few hick-ups like figuring out that:
Some questions following the lab remain:
A few fun circuits that I constructed using potentiometer and a speaker:
and two Force-Sensitive Resistors (FSR’s) and two LED’s:
The second iteration of my self-portrait plays with the idea of the many human emotions and character traits, including ones that people often try to hide like anger and vindictiveness. As time passes, I change the coordinates of the key portraits elements (wings, eyebrows, and the flame) and the face color to transform my original self-depiction into an evil looking creature. The mouse cursor also controls the hair shape, letting the viewer add to the character’s maleficent aura. I also used a cyclic sine function to create the oscillating eyebrows, which add a feeling of uneasiness to the transformation. The p5.js code can be found here [link].
Creators: Maryia Markhvida, Dror Margalit, Peter Zhang
Voice: Zeynep Elif Ergin
If we are lucky, we are born into the loving arms of our parents and for many years they guide us through life and help us understand the events around us. But at some point, sooner or later, life takes a turn and we are all eventually thrown into the chaos of this world. In these moments things often stop making sense and we have a hard time navigating the day-to-day. Eventually though, most of us adapt and figure out some way to go on.
Sound, in a way, is a chaotic disturbance of the air, but somehow we learn how to make sense of it and even manipulate our voice and things around us to reproduce the strange frequencies. We start recognizing the patterns in the randomness and eventually derive deep meaning from it.
This work is an interactive sound journey that uses evolution of randomly generated frequencies to reflect the human experience with uncertainty and chaos. It tells Zeynep Elif Ergin’s story through a composition of computer-generated sounds, her voice, and interactive visuals :
After our first Hypercinema lecture, I got very intrigued by the composition of sound in terms of signal and superposition of sine waves. This is something I vaguely knew about from my engineering background (I worked a lot with earthquake wave signal in my PhD) but never got to actually play with, let alone create. I was also curious to hear what different sounds could be produced if I used different probability distributions (uniform, normal, lognormal) to generate the number of waves, the frequencies and the amplitudes.
Once we formed a group with Dror and Pete, we started talking about what uncertainty and randomness meant to each one of us. We discussed that when people face moments of high uncertainty they are first thrown into absolute chaos and then slowly they tend to embrace the uncertainty and adapt the chaos into something that feels more familiar. We eventually arrived at a question: What would ones journey through uncertain times sound like using randomly generated sounds?
All of us wanted this piece of work to be grounded in and driven by real human experience. In the words of Haley Shaw on creating soundscapes:
‘…even when going for goosebumps, the intended feeling should emerge from the story, not the design’
The final piece is presented in an interactive web interface ,which allows one to listen to Zeynep Elif Ergin’s story through a progression of randomly generated noises. The listener has the option of experiencing the story without the main subject (inspired by the removal of the main character in Janet Cardiff’s work) or overlaying her voice over the computer-generated soundscape. There is also a gradual evolution of the visuals.
The first question was how can one randomly generate sound starting from scratch, i.e. a blank Python Jupyter Notebook. After a quick conversation with my father about the physics of sound waves and super-positioning, I had an idea of what to do.
I started with generating a simple sine wave (formula below) and converting is into a .wav file.
$$ y = A sin(2\pi f t +\phi) $$
This is the equation of a sine wave with phase angle ($\phi$), frequency ($f$) and amplitude ($A$), all of which were eventually randomized according to different probability distributions. Below is a sample of the first python code and first sounds I generated with only one frequency:
Then I played around and generated many different sounds. Here are some things I tired:
Here is an example of one of these generated sounds:
In the meantime, Dror and Peter recorded the interview with Zeynep Elif Ergin as well as additional ambient sounds around NYC, and worked in Adobe Audition to compose the final piece using randomly generated sounds .
The last step was to create the interactive interface with p5.js [link to code] , which gives the listener the option of playing only the ‘chaos track’ or overlaying the voice when the mouse inside of the center square. As the track is played, the uncertainty and chaos slowly resolve, both sonically and visually… but they are never quite gone.
I cannot believe how much I learned about circuits in just two days. All my life I was intimidated by the amount of different parts that go on the breadboard, but alas, this week I started my journey into taming the beast. Over the last few days I created a short circuit, blew up an LED, and created a glove switch in tribute to the electric pop star, Michael Jackson. Here is a prototype of the Jackson Switch:
To create this switch, I used one of the circuit set-ups from the Labs with a 12V power source, a 5V voltage regulator, a 220 kOhms resistor and a white LED light. My father consulted me on my circuit and was my assistant in filming and doing the lighting for the video.
Below is the filming and circuit set-up:
Even when using latex gloves under the white gloves, my body still conducted electricity (albeit with low current and high resistance)! The LED would light up slightly when I put the gloves on, even though the ‘switch’ fingers weren’t touching. How would I prevent that?
Using the breadboard turned out to be much harder than I thought. The good news is that it got easier with practice. First of all, the working table became a mess very quickly:
There are a few important things that I learned:
Questions (and failures):
And finally, my first working circuit:
I wanted to attempt to incorporate some elements of a phoenix in my self-portrait as it symbolizes the cyclical nature of demise and rebirth – something I seem to identify with lately. So I made a quick sketch and started coding:
Using p5.js editor I first decided to create an outline of the portrait. I played around with shape of the flame, mirroring the side parts and creating the middle on top. The variables for the x and y position were extremely helpful in moving the entire image around without changing the curveVertex(). I followed a similar process for the wings although the sharp corners and change of direction was harder to achieve. For hair I used a combination of arc(), rect(), and quad(). For the face I used curveVertex() and quad() functions. The curve definition was easier this time due to experience and symmetry. For face features I experimented with line(), ellipse(), circle(), and triangle(). This is the intermediate result:
Then I added colors and rearranged the shape (I didn’t like having a crown on my head!). I wanted to keep a simple color palette, accentuating the fire aspect of the phoenix. Here is the final version [code can be found here]:
Reflections: one of the trickiest parts was to figure out how to create more complex curves like the claim and wings. While we didn’t learn this in class yet, I could work without using variables because I am too used to defining things parametrically and rapidly iterating (especially the positioning). I also really wanted to play around with curves so I ended up using curveVertex() function instead of curve(). I also realized that I cannot seem to get away from symmetry — something that I want to change in my next iterations.
There are way too many things I want to use computation for — it is hard to pick . Previously, I mostly used programming for data analysis, simulations, and calculations but finally I have a chance and space to go beyond that. I want to dive into the world of creative visualization, multi-sensory immersion, and human-digital interactivity. I also absolutely MUST try using machine learning and computer vision in creating interactive installations. This is something I always wanted do but never had the bandwidth nor a community to.
A recent memorable experience I had was attending an immersive night walk in a forest close to Whistler, Canada. Vallea Lumina took visitors on a 1km snowy night trail following the footsteps of two long-lost hikers into an enchanted forest. The multi-media night walk did a beautiful job in bringing a sense of magic into the forest, which made me feel like a child full of awe and wonder again. The light and sound installations throughout the walk really created a feeling of enchantment. I would love to create a similar immersive experience that supplements natural environment or day-to-day spaces with augmented elements (such as light, sound, projections, etc.) to create a new reality.
Describing sounds with words can be tricky. But what about describing words with sounds?
In collaboration with Dipika, Anvay, and Vera
Ticking:
Humming:
Airy Sound:
Sound of Heat:
Sound of Betrayal:
Sound of Roundness:
Sound of Loneliness:
Sound of Red:
Sound of Joy:
Deep listening: [link]
‘The ear hears, the brain listens, the body senses vibrations’
‘To hear is the physical means, to listen is to give attention to what is perceived both acoustically and psychologically.’
– Pauline Oliveros
Sound design: Haley Shaw [link]