Tag: aiarts03

Posted on

Week 03 Assignment–Crystal

Demonstration 

My project is called Good Omens. Here is a brief demonstration of my project.

Inspiration

After seeing all the examples, I was impressed by the ear one and the nose one. Then I reviewed the basic Posenet example to see what I can create based on it. For the basic Posenet example, I found that this model can position certain parts of the human body and facial parts, such as our left and right eyes. And the feedbacks are the small green ellipses and the name of the part. As for me the feedback is a little bit boring because the aim of it is only to give information. Therefore, my plan is to take advantage of this image recognition and location technology to create an interesting and interactive project, and the initial step is to replace the original feedback with some funny pictures.

The next step is to think about the thesis and the content of my project. I was inspired by a poster of Good Omens, which is a British TV series. The content of this poster is a combination of angle, Aziraphale and devil, Crowley. 

I think this poster is very interesting because it embodies the harmonious coexistence of the opposite side. So I want to make a kind of special effect to let the users act as an angel or a devil and they can switch the character only by a hit on the space bar. Also in order to show the harmonious coexistence, I have exchanged some elements of the angel and devil. For example, in my project, the angle has a devil aura while the devil has an angel aura.

Technical issues 

In the process of my programming, I met lots of difficulties and problems. At first my code did not work because it could not find the files that were some related pictures. After asking my friends for help, I knew that I should create a new folder in the project folder and put all my files in it. I also needed to mark the file path in the code to let the program know where they are. What’s more, the functions of images and GIFs are totally different. At first I just applied image() and loadImage() on GIFs and the website didn’t run. But now I know that I should use createImg() and [name].position(), [name].size() to display a GIF.

When the images can be shown on the screen, here comes another issue: When the ellipses become images, the position is slightly deviated. I have to test several times to adjust the coordinates of images to guarantee they are in the right position. 

And it is difficult for the model to identify the difference between the left and right wrists. My initial idea was to put different images on the left and right wrist but the outcome was not what I expected. Thus I had to make some changes and got my final version of my project.

Reflection & Development 

I have become more familiar with the functions of JavaScript and the principle behind the object or image classification through this assignment. It reminds me of some special effects of TikTok, Snapchat and beauty camera. Now I can make a special effect by myself and it is really exciting. However, this model still needs improving to solve the issues that I have mentioned before. My expectation of a mature version is a more interactive special effect with various forms of feedback. The user can change the character by some specific gestures or by saying certain words. Also the background and the lightness can be changed to fit the mood or style of music, and we can use Body pix to achieve this expectation. therefore, users will fully interact with this project by moving their bodies, hearing sound, shouting words and so on.   

link to my code

Posted on

Week03 Assignment: ml5.js Project —Ziying Wang

The following clip is a demonstration of my project: Nose & Notes

The user can strike notes by moving the position of the nose, the colored rectangles will appear on the screen to illustrate which area is being stroked. 

Nose & Notes is based on the PoseNet machine learning model that locates the user’s body parts and generates live coordinates for different body parts. My project uses the coordinates for the nose and dissects the canvas into nine sections, with eight sections each representing one note from the scale and the ninth playing note A in chorus. The user can then plays pieces of music by moving his head and getting his nose within different sections. It is also possible for multiple players to strike notes together. 

The initial plan was to have different body parts all able to play notes within the screen, so it can enable a single player to play chords. However, after testing “rightWrist” and “leftWrist”, these two body parts are often wrongly recognized by PoseNet and I would constantly strike on notes that I don’t want to strike. Even though the recognition conditions for those body parts have set to be more than 0.8, it would still wrongly recognize and eventually gets stuck. 

I ran into a few problems in programming. One major problem is that I’ve been using Atom to program and run my code and the sounds I store in my folder wouldn’t start preload in the page. Then I was advised that I should run my code on the local server to preload the sounds. I then downloaded the atom-live-server and successfully launched a trial live server to run the program. However, when I use Terminal to run my code, it wouldn’t run properly and the console reads as follows:

Another issue that affected my original plan was that since the sounds are programmed to play every time a section detects the nose, it will constantly start the mySound.play() function as long as the user’s nose is within the area, which is not very pleasant to listen to. Originally, I wanted to make it into 9 sections saying phrases or sentences when the nose is in those areas, however, if the code plays the beginning of each audio continuously, it wouldn’t sound like a complete word, just a syllable instead. Therefore, I used notes to make compensation for this drawback. It still sounds a bit ragged since the sound files themselves have background noises, but are definitely smoother than inserting other audios instead.

If more time is provided, I would turn it into a game which requires the user to use nose, right wrist and left wrist to participate. Three color blocks will randomly appear on the screen and the user will need to use the three body parts to fill in those three blocks, a countdown will also appear on the screen, the user gets one point if he/she finishes the goal within the few seconds.

The following is the link to the code:

https://drive.google.com/open?id=1pfMnVntYXmOIVGkaGhn_F9QUpq7Xrds_

Posted on

Week03 Assignment: ml5.js Project

Develop a simple project, utilizing any ml5.js models covered in class (or other available ones from their website).

  • It can be an experiment or exploration of the models introduced.
  • How about creating a drawing tool with a body part?  How about creating a simple application which loves or hates a certain object? How about creating a funny application which tells your attitudes? How about creating a poem with the object detected?
  • Post it on the IMA blog before Friday Midnight, 20th with tag: aiarts03
  • Supporting materials should be uploaded to NYU google drive or your GitHub if prefer. Make it public or at least grant the instructor access (aven@nyu.edu)  before Friday 11:59 pm, late submission will have influence on the mark.