For this final, I wanted to make a game that forced the user to behave in a way that I found silly, but which also wrapped in a lot of the things we learned about motion and mimicking natural forces. When I presented the idea to the class, people had concerns about what might be the best way to track the player. Making a game that tracks your whole body would be fun, but ultimately hard to set up in a way that’s playable given space restraints etc. PoseNet is pretty reliable at tracking your nose, but people were concerned that extended playtime might hurt people’s necks. Another option that was floated was the idea of making a printable “fish” for the camera to track.
Experiment one: Experiment with different kinds of tracking!
I started with doing the coding train’s steering mechanisms coding challenge to see how it felt swapping out different poseNet keypoints for the mouse control in the sketch.
After playing around on my own for a few hours and asking some friends to try it, I decided that the nose control wasn’t really a comfort issue, provided you weren’t playing for too long… and ultimately I want this game to be a reasonably quick experience.
Experiment two: Separate Flocks
Now that I had my control mechanism figured out, I wanted to start experimenting with how my creatures would move. After presenting my idea to class, I felt like swimming fish would be an appropriate kind of “animal” to be mimicking with my code…. but the last experiment wasn’t really communicating as fish motion. First, their static target wasn’t what I was looking for. Second, their tendency to sit on top of one another wasn’t an appropriate “fish” behavior. I shuffled through various class examples for something that felt closer to the motion I wanted to mimic and decided that Shiffman’s separate and seek example helped achieve the feeling of a “school of fish” that I was looking for. However, I realized as I played with separation settings, that once all the particles got on a path following the mouse, they stayed following exactly where the mouse was…. and the particles I set to avoid the user were literally impossible to I decided to give each particle a “range of vision”, so that it would only steer towards the player if it can “see” them. This worked really well with the “predators” steering toward the user…. but the “prey” steering away took a lot more trial and error to find the right range of vision to make the game of catching them challenging but achievable.
** NOTE: I pointed out in my presentation that sometimes touching one fish made too big of a jump in your health, and it was suggested that perhaps there was more than one fish in that location. At first, I didn’t think this was possible with the “separate” steering mechanism in place… but upon looking back at my code, the particles only separate IF they’re not already in the same location! Maybe some of the fish were being rendered on top of one another and therefore never separating?
I hooked these particles up with ml5 poseNet for playtesting… and it went fairly well! The most consistent piece of feedback I received was that the video was frustratingly NOT flipped to be a mirror image. I was expecting this… it wasn’t intentional but I hadn’t figured out yet how best to flip the recording for poseNet.
The second piece of feedback I heard a lot was that people didn’t like looking at their own face. This was an interesting challenge, because something I really wanted to achieve with this game was to foster a feeling of awkwardness or embarrassment at having to do something silly to steer in the game. After talking it out with a few play testers, I resolved to try making some kind of ellipse which would have a fill that was the video feed.
People seemed to respond positively to the swimming and steering motion so I decided to keep those as is moving forward.
After playtesting, I set about the task of trying to set up the “look” of the game. My initial intention was to draw my own fish or use creative commons to find fish I had the rights to, but I couldn’t find quite what I was looking for…. so while the background image is one I have the rights to use, the actual fish and nose are clip art that I’ll have to replace if I keep moving forward with the game. The font I chose to use is free for reuse so that part is safe….
Initially, the nose was meant to be a stand in while I figured out how to render a fish with a video-face… but as people passed me on the floor and I asked them to check out the game, the feedback was overwhelmingly PRO cartoon nose…. so I kept it!
Flipping the Video
The last major hurdle I faced was how to flip the posenet input to match a mirrored camera image. I was HELLBENT on making this work within ml5 because I had seen something in the documentation that made me think it was possible:
However, after hours of banging my head on the keyboard and asking friends who had more coding experience than me what I was doing wrong I finally found someone with the answer: THIS FEATURE IS UNDER CONSTRUCTION AND DOESN’T REALLY WORK RIGHT NOW!
Woof! However, almost as soon as I realized i couldn’t do “what I was trying to do” I realized that the whole issue had a much simpler solution than the one I was trying to use. I just needed to draw all the ml5 keypoints in the same flipped rectangle I was drawing the video! I just took the entire sketch and flipped it all, so that all the particles are being rendered in reverse:
|image(aquarium, 0, 0, width, height);|
Scoring and Dying
Once I had the game up and running, I wanted to experiment with how often you had to kill or be killed to succeed in the game… but I kept running into an issue where occasionally catching or getting caught by one fish registered in the code as being caught by anywhere from 2-5 different fish! Initially, I had calculated that for every 10 fish who caught you, the population of purple fish went up by one, and for every 5 fish you caught, the population of green fish would go down. However… this bug where it wasn’t clear how many fish you were touching was making it really hard to consistently check my math. As I tried to come up with a solution, I found that keeping the number of fish static was easier to do, and that while the jumps in numbers were unpredictable, they were happening evenly across purple and green fish. Ultimately, I decided to try a form of game play where instead of keeping the goal to survive and eat the MOST, the goal was to keep the equilibrium by being eaten and eating in even proportions. While this worked really well, I realized a bit too late that my existing health meter wasn’t really a clear way of communicating that your ideal state was somewhere in the center. I plan to take a pass at this game later this week with a health meter that looks more like this for clearer communication: