Category: Machine Learning for New Interfaces

Posted on

MLNI: p5 Sketch – Tiger Li

https://editor.p5js.org/Tgr/sketches/L0yXJWHRG

When I saw a rotating object I immediately thought of a type of drawing tool I played with as a child. With that tool young children in grad school are able to draw complex geometric patterns. 

I really wanted to recreate it in p5 but I did not yet find the tools I need to make it. So I decided to make a flower instead. 

I used rotating lines to draw the patterns instead of points, unlike the children’s drawing tool. I wanted to add more complementary colors to it but my colors fill would not work in draw and I still am not sure of the reason. I hope to solve this issue and make my rose better looking tomorrow.

This image requires alt text, but the alt text is currently blank. Either add alt text or mark the image as decorative.

Posted on

MLNI – Interface with ML or AI (Billy Zou)

Interface with ML or AI

Neural Machine Translation

From wikipedia:

Neural machine translation (NMT) is an approach to machine translation that uses an artificial neural network to predict the likelihood of a sequence of words, typically modeling entire sentences in a single integrated model.

​

With neural network, machines do not translate text separately, instead, they can analyze the context of the text and pick words that fit the context most. This technology can be used widely in language-related fields.

Practical Application

Language Learning Helper

This technology is quite helpful in learning foreign languages. When people learn new languages, they tend to translate content direly from languages they are familiar with to the language they are learning. Therefore, some expression used by these language learners are like the text translated by a machine without neural network enhancement.

A straightforward solution to such problems are getting some native speakers to review the text, and that was what I did for my roommate when he learned Chinese in the freshman year. However, there are not always native speakers ready to help. With the help of NMT, we can develop an application to automatically review text for language learners and teach them how to organize sentences like native speakers.

Artistic Application

Slang Bot

With NMT, machines can understand slangs. I think the way people understand slangs are just like machines with neural network. Sometimes it is difficult to give an accurate definition of a slang, and people usually understand slangs with the contexts they are used.

The “Slang Bot” is a bot enthusiastic about slangs. When you say a sentence with slang, it can detect the slang you use in the sentence and reply you with another sentence including the same slang.

NMT introduces a new way for machines to process language. In the old days, machines understand language like a map that maps lexis to its definition. However, neural network helps machines understand language with context. In this way, machines can process language in an emotional way.

I believe it will be funny to see humans talking with robots using slangs. 

​
Posted on

MLNI: p5 Sketch – Sebastian Lau

http://imanas.shanghai.nyu.edu/~shl620/icecream/

Used translate to position the 3 ellipses. I think in the future I could just change the y-value.  I just made an ice cream cone, having the cone moving with the mouse and then having 3 scoops of ice cream on top that moved with the cone. 

Posted on

Week 2– Kevin Xu & Billy Zou

When looking up projects to study I had discovered an interesting report on algorithms used to detect gender in computer vision. As someone who is particularly interested in working with computer vision, I suggested we go from there, but Billy brought up a good point in that, although interesting, there was pretty limited usage for such a function. He, in turn, suggested us to study NMT or Neural Machine Translation. NMT is what most digital keyboards, translation services, etc use now to predict text. It is considered a significant improvement from its predecessor, Statistical Machine Translation, as it uses far less memory and predicts individual words based off context, intent, and relation between certain words as opposed to the statistical usage of words in phrases, which requires a lot of subcomponents to work together. As we discussed about NMT further, I started to see a lot of usage for this system, both practically and artistically. A project idea I had come up with was a robot that cuts you off as you try to talk, and attempts to finish your sentences for you. Because NMT tends to predict words based off it’s perceived intent and context of only a single sentence, chances are it would be very off-base with its prediction, though legible and grammatically correct. I’ve had many conversations with friends where it seems like we are on the same page but are actually in entirely different head-spaces, so when they finish my sentences I’m taken slightly aback by what they think I’m trying to say. That feeling is something I would try to replicate with this project idea.