Body Flow

This week I developed a generative animation program utilizing BodyPix in ml5.js and p5.js.

Basically it reads data from BodyPix and stored points contained by the body. To improve performance, I chose grids of 20 pixels. It will draw white circles in areas covered by the body detected. If a circle area was covered by body in the previous frame but not the current frame, a falling circle of random color will be generated. In short, if you move fast in front of the camera, abundant colorful circles will appear on the screen. Also if the area covered by body is large enough, the canvas will be magnified.

Store BodyFix data

I created some classes to store the results. Instead of using a global variable to constantly track results from the model, I directly store the data into an instance of my class.

Here class BodyMap is a singleton, quite similar to the global variable in the class example. It can draw white circles representing the body, and at the same time keep track of changes of the body.  Every time it finishing updating the data, it will compute the ratio between body covered area and the uncovered area. If the index is large, a CSS transform: scale(n); attribute will be added.

Difference between sequential frames

A grid has a boolean attribute active . If the attribute is previously true and currently false, a colorful particle is generated. It will also draw a white circle to represent areas covered by body.

An array of particles is stored in a grid object. The way to clean particles is also very simple. Just check whether a particle’s y-position is larger than the canvas height.

Colorful particles

This is a very simple class. I used a constant number to simulate the gravity.

In practice, since I did not directly modify pixels of image, the rendering process is a little slow. That is the reason why I chose 20 as grid size.

Conclusion 

BodyPix is a very powerful tool. I found apart from drawing the whole body, it can also recognize different parts of the body. I think this is a useful feature to develop something interesting. I will learn more about it and maybe utilize it in my midterm project.

To complete this assignment I had to completely understand what the demonstration code was doing. 

To decipher the parts of the code I don’t understand I had to resort to the fellows at the IMA lab. 

The main part, I did not understand about the body pix code was how the rectangles were displayed not by the X and Y value of the canvas but by the index generated from body pix using the camera. 

After figuring that out I was able to brainstorm more ideas on how to make an interesting game based on it. 

As I played around more with the code, I realized that the white rectangles from the original code over a black background are almost like a mouse cursor moving on a black screen. That’s when I had the idea of making a “searching “game.

The goal of the game is to let the users find a red dot using their hands. 

For this week’s homework, I used bodyPix in p5 and created a project called “light”.

My idea for this project was, first, it would start off with a black background with nothing, once someone yelled, the background would suddenly appear in a yellow color and a hundred ellipses will show the shadow of the user. I was inspired by the sound control lamps I have in the hallways of my house. It was a common thing in our neighborhood were people just yell “a…” instead of clapping their hand to turn on the light. I thought it was very interesting.    

Through programming, I found that this was not an easy task because the audio input is very strict on using a local internet browser. It didn’ t work no matter how I tried. In the end, I had to change the interaction from audio to pressing a key. Since my plan failed to use a mic audio input, I thought I would just switch the background into continuing black and just change the color of the ellipse.   

Link to code