Project Proposal:
The initial idea for my midterm project is called “Stars in the Waterfall”. This is a dynamic and interactive installation that allows the audience to be immersive in a virtual waterfall while interacting with it and generating colorful stars.
The main idea is that I want to create a scene of a waterfall where blue water drops fall down, and when the water drops touch my hands, they will immediately turn into stars in different colors and bounce out. I plan to use Object-Oriented Programming and PoseNet to realize my goal. The minimum viable product is still creating a waterfall scene but with hands acting like a filter and changing the water drops they catch into stars and following the original routes without bouncing out. The stretch goal is to program for the bouncing movements of the stars and make the color of stars change corresponding to the audience’s gesture.
For conceptual development, I have two inspirational projects: Crystal Universe and Universe of Water Particles. Crystal Universe utilizes LED tubes to create a light sculpture that can generate many different effects. Universe of Water Particles creates a waterfall and considers people’s position as obstructions to obstruct the flow of water like a rock and then change the flow of water. Flowers are also generated based on people’s position. I want to aesthetically visualize both projects into my project and create more interactions between the audience and the project by building upon the idea of Universe of Water Particles. As this inspirational project considers people’s position as a rock to obstruct the water flow and as a starting point for generating blooming flowers, I want to consider people’s hands as not only a rock, but also a convertor to transform water into stars. Unlike the unmovable location of “rock” and “flowers”, I want to make the interaction constantly changing and display in real-time. I’ll use OOP and PoseNet to detect the audience’s body segment positions.
Implementation Plan:
- Create the background of a cascading waterfall: determining the color of both the background and the water, the speed and the density of the waterfall.
- Visualize the emergence of stars: determine the dynamic moving path of stars.
- Use the mouse’s position and press to interact: use the mouse to prototype hands’ position and interact with the waterfall to generate stars. Also consider more interactions controlled by the audience.
- Implement PoseNet to detect hands’ position: detect both hands’ position to interact with the waterfall and try to also adopt more body segments into interacting with the project.
- Consider further conceptual ideas to improve the interaction.
By conducting this project, I hope to get more familiar with Object-Oriented Programming and PoseNet. I want to explore more approaches of utilizing and extending these two functions when creating interactive art pieces. I also want to gain more experience on creating animations by programming as I’m trying to mimic the effect of waterfall and stars bouncing around.
Insights from Peer Review:
From the comments of other classmates, I concluded that I should pay attention to the amount of the water drops since if there’re too many elements on the screen, it would slow down the speed and affect the interactive experience for the project. Also, consider the detail of the interaction.
From professor’s suggestion, I decided to change the concept a little bit and focus more on the deeper meaning of the project. Since professor mentioned about the characteristics of water which is the source of life, it made me consider if creating stars would be a good idea. So finally, I decided to take professor’s suggestion to generate butterflies.
Final Version of the Project: Aquatic Butterfly
“Aquatic Butterfly” is an interactive waterfall that breeds butterflies. Acting as a faceless man, the audience can stand in the middle of the waterfall to interact with the water drops. When you touch the water drops, it will break into butterflies and fly away.
Core Concept
- Water with vitality
- Cocoons of butterflies
- Audience as a cute, faceless soul
In the proposal, I didn’t think much about the meaning of turning water into stars. So after professor suggested me to create some living things like butterflies, I finally considered the characteristic of water that it’s actually the source of every living creature. So I implemented this concept into my project. Also, the water drops can be treated as cocoons of the butterfly. When you touch those water drops, you’ll break the cocoon and the butterfly will come out and fly away. It also has a meaning that without water, a caterpillar can never break the cocoon and become a butterfly. As for the image of the audience, I created a really cute and faceless translucent avatar so that you can clearly see your movement and interaction with the water drops.
Conceptual & Technical Development
- Concept
- Slightly different from the proposal (Butterfly, not the star)
- Water is the source of every living creature
- “No Face Man” from Spirited Away
- Techniques
- PoseNet: I use PoseNet to detect the positions of body segments like eyes, nose, ears, shoulders, elbows, and wrists.
- Butterfly: from “function” to “class”. Since the code for the butterfly was borrowed from an online open source, it was built as a “function” instead of a “class”. So professor helped me change the “function” into a class so that I can manipulate the butterfly with interaction.
- Water drops: “class”. I created another “class” to display the water drops.
- Sound effect: load a song in asset with “mousePressed”. By uploading a song file into the “asset”, I used the function “mousePressed” to loop the song in the project.
- Trail effect: “createGraphics” to separate two layers. Because I wanted to create a trail effect for the water drops but not the avatar, I decided to use “createGraphics” to create two separate layers, putting the water drops on one layer and the avatar on another. In this way, I can change the opacity value for the background of the water drop layer and not affect the avatar.
- Faceless avatar: “bezier” curve. I used “bezier” to create the avatar with aesthetically smooth curves.
What I struggled with
- Controlling a customized function. Since the code for the butterfly was borrowed from an online source, it originally uses both CSS and JavaScript to create the butterfly and customizes several “functions” instead of a “class” to generate butterflies. So I spent a lot of time trying to discover how to build the interaction between the audience and the butterflies, and how to control the “function”.
- Separating the water drops with the human’s reflection to create a trail effect. When I wanted to create a trail effect for the water drops, I tried to change the value of the background. However, it turned out that the image of the people also has a trail effect, which I don’t want it to be.
- Creating a proper human reflection in the scene. At first, there was no image but only a contour representing people in the scene, so it wasn’t so obvious to see where you are and how you interact with the water drops. Therefore, I decided to create an avatar to demonstrate people’s position and also their interaction with the water drops.
What I have learned
During the process, I’ve learned how to turn a customized function into a class and also how to play with the class. For the trail effect, I’ve learned to create different canvas/graphics to separate the objects and apply different effects individually. I’ve also learned to use bezier() to create smooth curves. For conceptual development, I’ve learned to consider the deeper meaning of my project and make it more meaningful.
How I would develop your project further
For further development, I want to use another technique to improve the trail effect so that it can be more smoother without any afterimage on the background. For the avatar, I want it to be continuously changing with the interaction: the more water drops it touches, the more butterflies come out, and the avatar would also have wings gradually generated on the back. When the wings are big enough, the avatar would turn into a butterfly and fly away.
Attributions: