RESEARCH PROJECT 1：Synesthesia Machine
The website of the project is shown here: https://www.manamana.net/video/detail?id=1816079#!en
Synesthesia Machine” is an artificial intelligence with synesthesia, it will emit corresponding unique scents while perceiving human facial expressions. Through the facial expression recognition algorithm, it can analyze the facial expressions of the audience and output the values of different emotions. At the same time, based on these emotional values, the installation is driven to extract different proportions of scent liquid and atomized and mixed to convert the audience’s facial expression into a unique smell.
This project uses cybernetics to present the results, which is consistent with the approach I took in the final project. This is in line with Ernest Edmonds’ statement,“Cybernetics, and the closely related study of Systems Theory, seemed to me to provide a rich set of concepts that helped us to think about change, interaction and living systems (Bertalanffy, 1950; Wiener, 1965)”.
Considering the interactions, the results of the scents and expressions corresponding to the pre-designed scents reflect the dynamic passivity. The diversity of human expressions is very important, as well as the unpredictability of expressions, creating different dynamic interactions. This prompted me to think more about how simple engagement, such as the interconnection of the senses, can be used to produce fascinating and creative results.
RESEARCH PROJECT 2：Colormatrics – Sonification-driven audiovisual works
The website of the project is shown here: https://www.manamana.net/video/detail?id=1816079#!en
Colormatrics is a set of three sonification-driven audiovisual works specially designed for the Cube at the Moss Arts Center. Based on my line-by-line image scanning method and additive synthesis techniques. Colormatrics converts generative visual patterns into sound and visualizes the sonification process in real time. Colormatrics_01 creates ambient- or pad-like sounds fitting in with the immersive atmosphere of the space. Colormatrics_02 creates beat music following self-changing graphic patterns that become complex. Colormatrics_03 creates timbre according to the colors and spatializes sound where the vertical and horizontal positions of graphics are mapped into 128-speaker systems in Cube.
In my final assignment, I also wanted to use sound to achieve human-machine interaction. Visualizing sound is a very innovative idea that makes me think that making things perceptible that we cannot visualize is a way to broaden the concept of interaction to a wider audience.
WHAT A SUCCESSFUL INTERACTION IS
It involves continuous communication between inputs and outputs, human involvement, intentions, and conveying unpredictable creative responses that can benefit from cybernetics.
I begin by arguing that interaction involves inputs, outputs, and the responses between them. Inputs and outputs can be provided by programs, people, or environmental factors. My mid-term project reminded me of the importance of human involvement as a user. I also grew to understand that interaction should provide and express intent, whether for entertainment or utility.
This reading and research further deepened my knowledge. As Edmonds explored the nature of the interaction, he explored how infants “without a shared language, just by prodding the world and looking at what happens: trying to find patterns (Bower, 1974)”. Interaction should involve more possibilities for connecting with the world. In addition, human-computer integration as well as art (Edmonds) support interaction. Cybernetics plays an important role in improving the variability of modern interaction.