Background
My final project is a further development of my midterm project. My midterm project was inspired by a movie called Nezha and it was an interactive painting. The users can trigger the motion of the still object by getting close to it. They can also hear the sound of animals or water by the same method. However, there were some features that didn’t work very well. For example, my thought was to let the users trigger the cloud by raising their hands. However, since I didn’t add any hint to inform them, the outcome was lower than my expectations. Besides, my interaction part was not that smooth and natural. The reason might be that I didn’t think about the logic of my project, I just added what I wanted to my project. I also received many excellent suggestions from my professor and other guests, and they really inspired me a lot on my final project.
Inspiration
One of the inspiration of my final project is Shenshen’s midterm project. I was surprised at how her project could let the user move the image through their eyes to see the whole picture. It looked like the user was actually walking through the gallery but not just seeing it. I also wanted to make my painting much larger than the canvas size so that it could be more immersive for the users. What’s more, I wanted my project to tell a story or to have logic but not just a pile of random elements. Since Christmas was around the corner, I chose this festival as the theme of my project and the story line was how Santa prepared and deliver presents.
My Projects
My final project is an interactive painting whose theme is Christmas. It has four different scenes. In the first scene, the user is a red glove and the position is determined by the position of user’s eyes. There is a closed book with an arrow pointing to it. The role of this arrow is to give hint to the user to get close to it. Once the user get close enough to the book, it will open and show a Christmas tree, which is the second scene. Then the user will notice the gingerbread since I add this GIF to emphasize it.
Once the user approaches to it, the glove will change into the gingerbread which means that the role of user in this painting is it. Then there will be a right arrow telling the user move to the right edge to go to the next part.
The next scene is a postcard in Christmas style I made by Photoshop to tell the user how to play this interactive painting. Then the user can go to the last scene by the same method. And the last one is an image as the follow:
This is the original image I found online and I also added some other images and GIFs to make it interactive, just like my midterm project. Besides, I found some sound files such as Jingle bells in elf tune and the sound of clock. If the user approaches to the right edge, he or she will see an arrow pointing to the right. And if the user goes further, the image will move to the left so that the user can see the rest part of the painting. They can also choose to go back to the left part by move to the left edge, just like pressing the arrow key on the game console.
Methodology & Difficulties
One of the most difficult part of my project is the coordinates. At first, I accepted Jamie’s method to let users move the image based on the position of their wrist. However, after several trials I found it was so tiring and the wrist usually blocked the eyes so it would influence the position of the gingerbread. Then I decide to use the coordinates of eyes to move the image. The solution is that I defined a value called xx, whose original value is 0, and let it be the x-coordinate of the image. Then I set some conditions to achieve my expected result. The reason to set two areas is to make the interaction in a gradual way, which is inspired by Tristan’s suggestion.
Here came another issue. The image I added to the large one couldn’t move with the main scene and the area I set as the condition couldn’t change with the image either. So I just applied the same logic to these still images and it worked.
The next one is taking control of the GIF.
My initial idea is that if the user touches the book, there will be a GIF displaying the opening process of the book. However, I couldn’t make the GIF play for only one time then stop. Thanks to Professor Mood, he sent me a sample code to solve this problem. Unfortunately, when I applied the method to my project, I got hundreds of errors, so I had to give it up. Now I know the reason is that the library for GIF was so advanced and I didn’t update my Atom so it couldn’t recognize the code.
The last one was not a problem but an essential method which is switch function recommended by Professor Moon. This function is so important for me because I have so many scenes, and they all require the position of the eyes. However, I cannot set different variables for each scene which is too much for me. This function solved the problem. I can set a condition to decide which mode I want to display and there is a break between each mode. I learned the importance of break from my extremely embarrassing final presentation. I didn’t add break between mode 3 and mode 4 so that my function didn’t work and I couldn’t see the images I added on my project. After seeking Professor Moon for help, I learned the reason and also the significance of break in switch function.
Further Development
For the further development, I want to focus on the transition part of my project. I really like Jessica’s project because she made some transitions in changing the original image to another one. I also want to apply fade away effect to the change of scenes in my project. Also, I can use fade function for the sound part to make smoother interaction. Last, I want to add some text bubbles around the characters in the painting such as Santa and elf. The content is about the background information about them and Christmas. In this way I can make my project more educational.
I sincerely appreciate everyone who help me with my project and who like my project. Especially Professor Moon, he really helped me a lot and made me realize that I can use knowledge related to machine learning to achieve some creative and artistic ideas. And I will definitely apply what I have learned from this course to the rest of my academic life in NYUSH and even after I graduate in the future.