Machine Learning Assignment 3

1. The model that tries to detect commands by other people

link: https://teachablemachine.withgoogle.com/models/7hnLC0hB9/

I tried to train a model that could split the four pronunciations of my name. I had five friends record four versions for me. They were One male and four females (which might have an effect), all Chinese (which definitely had an effect), but one of them grew up in America (hope that can add some variety).

Each label has 205 samples. The epochs are 50. It’s great to see how their spectrograms are different from each other.

I also tried to add a random talking label, but it worked worse. It seemed like this label confused the model, so I deleted it in the end.

Fun phenomenon: when my male friends tested, I found that the least distinguishable pronunciation(Icelen) performed the best instead, while the most distinguishable(Icelean) performed the worst. I don’t know if it had anything to do with gender, but he was the only one with this phenomenon.

Regarding the training results, at least it had acceptable results for my female Chinese friends(quite similar when it works on just me). But maybe it’s only because we have very similar voices and accents.

Since the stress on the name is in front, the two labels with the same first syllable can easily be confused. For instance, when I say Icelen, Icelean is recognized when I read the first syllable, and Icelen is recognized only after I read the second syllable.

2. model and sketch

link: https://teachablemachine.withgoogle.com/models/OzZAftjHY/

link: https://editor.p5js.org/AislynnLi/sketches/Afv3tLOId

For the model, I had 5 labels: chicken, duck, can, curtain, and door. Because I needed to use the curtain in the classroom to test, it was not that well tested. The curtain and door were very likely to be detected with very high confidence even when there was only background noise. I guess it was because there’s a process in “closing the door” and “pull the curtain”,  they would have some silent moments in samples that would be wrongly detected. I tried to delete all the quiet samples, the door label was better, but maybe the sound of pulling the curtain is too small, it was still easily misidentified.

I’ve tried to change the overlapFactor. The phenomenon was that if it was very high (0.75), curtain and door were not easy to be incorrectly detected when it was very quiet, but the value of each label would jump more when there was a little sound. If it was very low, it was easier to appear 100% of the wrong detect, and the correct detect was not easy to maintain in the high confidence. I felt the application in the sketch effect was even worse. So I chose to adjust it to 0.75.

For the sketch, I just drew different parts and tried to make them interact with sounds. 

For duck and chicken label, I tried to add one more duck/chicken when the sound is detected. In order to just add one, here’s the code:

if(label == "door" || label == "chicken" || label == "can"){
duck_push = false
}
if (label == "duck" && duck_push == false) {
ducks.push(new duck(width, height / 2));
duck_push = true;
}

if(label == "door" || label == "duck" || label == "can"){
chicken_push = false
}
if (label == "chicken" && chicken_push == false) {
chickens.push(new chicken(width, height / 2));
chicken_push = true;
}

Because the curtain label was so easily detected, I didn’t put it in the condition. And in this case, I can’t add more than one duck in one turn, this is the part to improve.

For the curtain label, I tried to change the time, which is like closing the curtains when it’s dark for you. So I changed the sun to moon and overlapped a rectangle that changed the opacity to make it looks darker.

Here’s the code:

if(label == "curtain" && ySun<300){
ySun+=4;
}
if(label == "curtain" && ySun>100 && yMoon>50){
yMoon--;
}

if(label == "curtain" && opacity <100){
opacity+=0.5;
}
fill(0,0,0,opacity);
rect(0,0,400,450);

For the can label, I wanted to fill the trough and let the ducks eat the food. I replaced the image to “fill” the trough. Also, I tried to use code to make the ducks go back to the pool when there’s no noise of can anymore.

Here’s the code:

eatfood(){
this.dx = 300 - this.x;
this.dy = 400 - this.y;
this.len = dist(10, 10, this.dx, this.dy);

this.dx = this.dx / this.len;
this.dy = this.dy / this.len;

if(this.dx<0 && this.eat == true ){
this.scale = 0.3
}
if(this.dx>0 && this.eat == true){
this.scale = -0.3
}

if(label == "can"){
this.eat = true;
this.x = this.x + this.dx;
this.y = this.y + this.dy;
} else if(this.eat == true){
this.y = this.y - this.speed;
this.eatDone = true;
}

if(this.y<350 && this.eatDone == true){
this.eat = false;
this.eatDone = false;
}
}

For the door label, I wanted the chickens to go back to the chicken coop. And I delete the chickens that get in. The code is similar to the previous one.

To add to the fun, I let the chicken and duck move. The duck is moving in the range of the pool, while the chicken is only jumping in a certain area. 

Here’s the code:

this.jump = map(sin(frameCount*0.3+this.jumpRandom),-1,1,-1,1);

if(this.x>this.spaceRight || this.x<this.spaceLeft || this.x<10 || this.x>180){
this.xSpeed = this.xSpeed*-1
}
if(this.y>this.spaceRighty || this.y<this.spaceLefty || this.y<200 || this.y>400){
this.ySpeed = this.ySpeed*-1
}

if(this.xSpeed<0){
this.scale = 0.3
} else{
this.scale = -0.3
}

this.x = this.x+this.xSpeed;
this.y = this.y+this.ySpeed+this.jump;