Inspiration：
1. Photoshop software like MeituXiuxiu, TiantianPtu. These softwares could detect the user’s facial features in real-time and add decorations like cat ears, pig nose, eyeglasses according to the user’s choice. However, I’m going to make a different one. The special effect comes from arms/ hands instead of faces.
(Photo I shot from Meitu Xiuxiu using real-time detect technology)
2. In the game Arena Of Valor (王者荣耀), there is a kind of career Wizard/ Witch (法师). They could use magic to hurt the enemy. I intended to refer to some of the special effects from this Game. Users can have hearts, flowers or fires in their hands, on their head or somewhere else.
(Post from the game Arena of Valor)
Concept and the expected effects:
I want to make a camera that is “magic”. The users could have some special effects moving together with their body movement. So I made my original concept:
1. Use PoseNet to detect the key points of body/ hands.
2. Click the mouse to change the special effects. But it would be better if the change of special effect would be stimulated by specific action. It is difficult to realize though.
3. Try to find a way that the photo can be taken when the user pressed on the space and the photo would be saved locally.
User-testing:
I got some useful feedback in our user-testing process.
①According to my own using and testing experience, I find that people will be away from computer so it’s hard for them to reach the computer and press the “↑” button. This can be perfectly solved with makey-makey devices but sadly I didn’t borrow one in time.
②Also, some users think that I can add some sound to each effect and make the environment more real.
③Some female users also hope that this can provide them with “thinner face”(something like this) filters but sadly I am not able to make it.
Basic structure:
①Loading the poseNet and video.
②Create start page and start button.
③Get the position for the key points I need.
④Create 3 effects.
⑤Additional functions.
Working process:
Step 1. Preliminary conception. I want to use poseNet to get the position of the key points (like hands) and create effects.
Step 2. Referring to GitHub, I constructed the poseNet and the video. Then get the position of key points successfully.
Step 3. I use p5.js to separately create each effect and made some changes. You can see my work in the following pictures.
① Rotating box: This is a rotating box with some small planes rotate around it. I used directional light to make the 3D effect more real.
② “sin” curve: This is a transmitted sine wave. Its color would change when the position changes.
③Flame. I use class to create sparks and use Adobe Animate to create the gif of fire.
Step 4. Insert the effects into my project. Because the canvas is turning into 3D from 2D so the position for everything should be changed. I also load sound to different effects and some decorations like glasses. But different from the effect one eyes in the tutorial video we saw in class, I use the distance between two eyes to calculate the size of the glasses to suits the user better.
Step 5. Beautification and adding judgement(start page).
These are the beautified scene:
Step 6. Debugging and optimization. We overcame some challenges.
Challenge
- When I changed the scene, the sound effect for the previous scene would not stop immediately. It would stop until it ended. I add some xx.play and xx.stop in the if series to solve this problem.
- It takes me some time to translate the position as I use 3D canvas. Next time, I should have this planned in the very beginning and I won’t waste time on this.
My learning outcome:
I get to know how to use poseNet and is more familiar with the use of class. And I also learned how to create 3D canvas and 3D figures for the first time.
to use the data of the json file and create some visual effect. I used these data to determine some point of a person. If the user put the mouse on these points, they can see the point turns into red and its name appeared next to the point.
