Body Flow

This week I developed a generative animation program utilizing BodyPix in ml5.js and p5.js.

Basically it reads data from BodyPix and stored points contained by the body. To improve performance, I chose grids of 20 pixels. It will draw white circles in areas covered by the body detected. If a circle area was covered by body in the previous frame but not the current frame, a falling circle of random color will be generated. In short, if you move fast in front of the camera, abundant colorful circles will appear on the screen. Also if the area covered by body is large enough, the canvas will be magnified.

Store BodyFix data

I created some classes to store the results. Instead of using a global variable to constantly track results from the model, I directly store the data into an instance of my class.

Here class BodyMap is a singleton, quite similar to the global variable in the class example. It can draw white circles representing the body, and at the same time keep track of changes of the body.  Every time it finishing updating the data, it will compute the ratio between body covered area and the uncovered area. If the index is large, a CSS transform: scale(n); attribute will be added.

Difference between sequential frames

A grid has a boolean attribute active . If the attribute is previously true and currently false, a colorful particle is generated. It will also draw a white circle to represent areas covered by body.

An array of particles is stored in a grid object. The way to clean particles is also very simple. Just check whether a particle’s y-position is larger than the canvas height.

Colorful particles

This is a very simple class. I used a constant number to simulate the gravity.

In practice, since I did not directly modify pixels of image, the rendering process is a little slow. That is the reason why I chose 20 as grid size.

Conclusion 

BodyPix is a very powerful tool. I found apart from drawing the whole body, it can also recognize different parts of the body. I think this is a useful feature to develop something interesting. I will learn more about it and maybe utilize it in my midterm project.

Cut The Ball

In this assignment, I combined ml5.js with p5.js and developed a simple interactive game, with the help of PoseNet.

Basically, the player should cut the circle with his/her nose. Position of player’s nose is tracked all the time. When player’s nose penetrate a circle, the circle will be destroyed.

Nose Detection

Nose is detected using PoseNet.

Among all body parts, I picked the nose and saved its position in an object. This object would also be used by circle objects to detect penetration.

Penetration

Penetration is divided into to parts. The first part is nose coming from outside the circle and the other part is nose leaving the circle. If both conditions are satisfied, a penetration is detected. Then, the ball will be destroyed and the system will record this action as a successful cut.

Limit Objects

There are two parts in this program that require object limit. First is cleaning the circle cut or expired. Since there will be at most 1 big circle on the screen and a new circle will be generated every 5 seconds. What I did was not displaying the circle when it was not available and when a new object was assigned to the variable, the old one would be cleaned automatically.

When a circle is cut, there will be an explosion effect. In fact it is hundreds of small circles. All circles will be removed after 1 second. Of course they are stored in an array first, and finally the array will be emptied.

I find that what PoseNet does is quite similar to Kinect SDK. I am looking forward to utilizing more interesting machine learning modals.

FireWorks

This week I developed a simulation of colorful fireworks using p5, in an object-oriented way.

Classes 

I used two classes to implement this effect, Spark and SparkManager .

The previous one is the particle itself and the latter one is the manager of particles.

Spark

In fact, they are circles filling with different colors.

The particle has properties position, velocity, size and color. The color is totally random. In order to produce the effect of explosion, sparks close to the center are larger than sparks with higher initial velocity.

As time goes by, sparks will fall more and more quickly and at the same time, they will become smaller and smaller. Finally all sparks will disappear.

SparkManager

Basically, this class is an array of sparks and some methods to manipulate the array. In practice, this class has only one instance.

Every frame, it will update all sparks and remove sparks that are not visible. Finally it will draw all sparks. Sparks are stored in an array.

Main Process

The main process is quite straightforward. This assignment is simple, however, object-oriented-programming is what we will keep using. OOP is a powerful way to program, especially program relating to visual effects.

With neural network, machines do not translate text separately, instead, they can analyze the context of the text and pick words that fit the context most. This technology can be used widely in language-related fields.

This week I developed a simple sketch using p5 basic shapes and some transform functions.

I coded these particles using class syntax. Since circles and squares in this case have a lot of properties in common, I used an abstract class Particle to include the common properties.