Week 4 MLNL – Interactive Game w/ PoseNet (Lishan Qin)

Overview

For this week’s assignment, I utilize the idea of “abstract” buttons and PoseNet with the help of oop concept to create an interactive screen game. In this game, the player is able to control the poke ball on screen by moving their face in front of the computer to try to catch the pikachu. When the poke ball is close to pikachu, the color of Pikachu will become lighter, indicating that the player is close to catching pikachu. When the player catches pikachu, a poke ball will appear under the specific pikachu. I used class() to design the “pikachu” object and wrote an array to hold the object. The number of Pikachu appearing on screen is limited and the pikachu will disappear before it reaches the border of the canvas. With the help of PoseNet, I’m able to get the coordination of player’s face, in my case, I used the coordination of the nose of the player to calculate the distance between the pokeball and the Pikachu and wrote the checkInteraction() function to design the game.

Technical Problem

The biggest problem I met during the process of creating the game is that I found it very difficult to get an accurate distance between the coordination of player’s nose and Pikachu. When I used MouseX and MouseY as condition, the distance between pikachu and the mouse is always very small and accurate. However, when I used Posenet, the number it gave is always above 100. I’m not sure why this is happening. In addition, there always seem to be a 0.5~1s delay when playing the game. I think it might be because the order of some of the code I wrote still needs improvement, but I can’t find what the errors are. The code of my project can be found here.

MLNI Week04: “Fruit Ninja” with PoseNet(Ronan)

For this week’s assignment, I want to create a simple version of Fruit Ninja with PoseNet, where the player will use their hands to cut the fruit directly.

Try it on p5 web editor:

https://editor.p5js.org/RonanChang/sketches/u5s3HUpyN

Steps and Difficulties that I’ve met:

1. The first step is to create a Fruit class and generate some fruits to cut.  In order to make the game similar to the real Fruit Ninja game, the fruits should shoot up from the bottom of the screen and then drop down because of the gravity. However, when I was playing around with acceleration, the objects never slow and drop down to the bottom. Since the acceleration is just a simple one pointing to the y-direction, I didn’t use the applyForce() function. Instead, I hard-coded a number in the constructor of the class. After close examination, I’ve found that I mistakenly multiplied the acceleration with zero every time the location is updated.

2. Second of all, when a fruit is cut, it should disappear from the canvas. In order to achieve this, I created a variable called “isCut” in the Fruit class. Whenever the mouse/hand’s position is within the circle, the fruit will disappear and be removed from the ArrayList.

Maybe in the future, I will try to create a better cutting effect instead of just letting the fruit disappear. For example, maybe I will generate two triangles to represents the cut and then make the circle disappear.

3. After I finished with the Fruit class, I started to create the three circles on the top right corner of the screen to indicate the number of fails in the game. Every time the player fails to cut a fruit, one circle will turn to red. In order to achieve this, I added another variable called “hitBottom” in the Fruit class to check the location of the circle. Besides, since all fruits are generated from the bottom of the canvas originally, we should be careful to check the location when the fruit starts to fall.

One thing that bothered me for a long time is that no matter how many fruits are failed to cut, all three circles will turn to red at the end.  After trying to console.log out the number of the fails, I found that the status of the fruits was still being checked even when the user can’t see anything on the screen, which means I should’ve deleted the fruits when they dropped to the bottom as well.

4. The next step is to generate new fruits. New fruits should be generated when the fails are less than 5 and there are no objects in the ArrayList. Therefore, I created another function called generateFruits() for this. And when the fails are equal to 5, the player will see a “game over” message on the screen.  However, when the game is over, I still want the rest of the fruits to fall until they disappear. Therefore, I checked whether the fails are larger than or equal to 5 because when the game is over, the rest of the fruits are still falling and the number of fails is also increasing.

5. The next step is to add poseNet in the game. I watched Daniel Shiffman’s video for help. 

One difficulty that I met is that the image in the webcam is opposite from reality and I didn’t really find a solution on the Internet.

6. For future development, I will add the sound effect!

Week 4 MLNI – Interactive Game w/ PoseNet (Cherry Cai)

Snow Controller

For this assignment, I created a screen-based interactive sketch that utilizes poseNet under the object-oriented programming concept

  • Inspiration

As I was reviewing my last assignment, I came up with the idea of creating a scene of the winter snow. Inspired by the example in p5js.org, I want to create a scene where the position of the hand can be a representative and simulation of the wind. (like the gif showed below⬇️)

  • Coding

By first using the poseNet  I input the position of the rightwrist: keypoints[10] and the leftwrist: keypoints[9] by referencing the keypoints list below.

To track the position of each new position, I recorded the x and y position for both the elements:

newrwX = poses[0].pose.keypoints[10].position.x;
newrwY = poses[0].pose.keypoints[10].position.y;

newlwX = poses[0].pose.keypoints[9].position.x;
newlwY = poses[0].pose.keypoints[9].position.y;

In order to reduce the noise and smooth the value of the changing value in the position, I used the function called 

lerp(x, y, amt) 
” x Numberthe x component
y Numberthe y component
amt Numberthe amount of interpolation; some value between 0.0 (old vector) and 1.0 (new vector). 0.9 is very near the new vector. 0.5 is halfway in-between” (p5js.org).
 
So what it really did is add the first two-element together and times the amt number and generated a new value in between.
 
After getting the position and successfully detected the movement of my wrist, I started to code for the interface of the falling snow.  First by creating a class and gives it a position, velocity, acceleration and a random number for the size. To actually simulating the snow in the real world, I added a special force, which will be multiplied by the size of the snow and then added to the acceleration. In this case, the bigger snow tends to fall faster than those smaller ones shown on the canvas.
 

// class for snowFlakes
class snowFlakes {
constructor(sx, sy) {
let x = sx || random(width);
let y = sy || random(-100, -10);
this.pos = createVector(x, y); //position
this.vel = createVector(0,0); //velocity
this.acc = createVector(); //acceleration
this.r = getRandomSize();
this.R = random(255);
this.G = random(255);
this.B = random(255);
this.isDone = false;
}

//add force to the acc
//make the bigger one fall more faster than smaller one
Force(force) {
let f = force.copy();
f.mult(this.r);
this.acc.add(f);
}

update() {
this.vel.add(this.acc);
this.vel.limit(this.r * 0.07);//limit the size associat with speed
this.pos.add(this.vel);
this.acc.mult(0);
if (this.pos.y > height + this.r) {
this.randomize();
}
}

//when they get to the bottom, use your rightwrist to add them into the canvas
randomize() {
let x = random(width);
let y = random(-100, -10);
this.pos = createVector(x, y);
this.vel = createVector(0, 0); //velocity
this.acc = createVector(); //acceleration
this.r = getRandomSize(); //size of the snow
}

display() {
stroke('#a3c5ff');
strokeWeight(this.r);
point(this.pos.x, this.pos.y);
}
}

The last step is to associate the position of the wrist with the snow. What I did was adding another force called the wind and using the map() function to input the detected position of the wrist and using it to control the snow in the x-axis. 
 
//the snow will move following the rightwrist position
let windX = map(rightwristX, 0, width, -0.7, 0.7);
let wind = createVector(windX, 0);
 
  • Conclusion

To sum up, I would say there is still a lot of function that can be associated with poseNet model and there must be more interaction that can be embedded. The assignment helped me to understand better how poseNet worked and again reinforced my ability coding using class/functions/for loops/if statements. 

Week 3-Generative Art (Kevin Xu)

For this assignment I wanted to go for a calligraphy style visuals. I didn’t want it simply to be a ‘paint tool’ where it simply draws where your mouse is. Instead I decided to make it so that when you clicked down, it would mark the mouse X/Y position then mark down the mouse X/Y position of where you release. I used this to map the general direction and velocity of the object it spawns as well as the speed of which it changes color. I originally had the balls going in straight lines but it seems very unnatural and not very pleasant so I added a random addition/subtraction to the change of the X/Y positions in order to make the balls travel a bit more fluidly. For controls I mapped w to clear objects, q to clear the background to black, and e to clear the background to white. As for controlling which direction the ball goes click and drag the opposite direction you want it to go.

https://editor.p5js.org/khx201/full/lTUgbUxk1

Source Code:

let ball = [];
function setup() {
createCanvas(1920 , 1080);
background(0)
ball.push(new Circle());
}

function draw() {
for (let i = 0; i < ball.length; i++) {
ball[i].move();
ball[i].display();
}
}
//clear screen
let keyb = 81;
let keyw = 69;
let keyc = 87;
function keyPressed() {
if (keyCode == keyb) {
background(0);
return false;
}
if (keyCode == keyw) {
background(255);
return false;
}
if (keyCode == keyc) {
ball = [];
}
}
//end clear screen
//Vars
let startX;
let startY;
let endX;
let endY;
//MousePressed/Release
function mousePressed() {
startX = mouseX;
startY = mouseY;
}
function mouseReleased() {
endX = mouseX;
endY = mouseY;
let b = new Circle();
ball.push(b);

}
//Ball Class
class Circle {
constructor() {
this.x = endX;
this.y = endY;
this.color = 0;
this.diameter = 35;
this.speedX = startX-endX;
this.speedY = startY-endY;
this.colneg = 1;
}
move() {
this.x += constrain(map(this.speedX,-200, 200, -5, 5), -4, 4) + random(-11,11);
this.y += constrain(map(this.speedY,-200, 200, -5, 5), -4, 4) + random(-11,11);
let colchange = constrain(abs((this.x+this.y)/2), 1, 3);
if (this.color >= 250) {
this.colneg = -1;
}
if (this.color <=0) {
this.colneg = 1;
}
this.color += colchange*this.colneg;
fill(this.color)
}
display() {
noStroke();
ellipse(this.x, this.y, this.diameter, this.diameter);
}
}

MLNI week 3 sketch

Shenshen Lei

sl6899

I created a sketch that detects the position of mouse. If the mouse is pressed, the brush in rotating polygon shape will follow the movement of mouse. When the mouse is released, the there will be a eraser that can cover the drawing.

This is the video record of the sketch:

https://drive.google.com/open?id=16ym1DuB8unpB11Mn-ib2QaON2JDwXxLC

I hope to create more complex sketch board in the future and combine some machine learning programs with it.