Tag: aiarts06

Overview:

For this midterm project, I plan on creating a Meme-Generator. The user will interact with this machine by providing the machine with a quote or caption. The machine will then figure out the perfect photo for the given content. 

Background & Inspiration:

Meme culture is continuing to be a major factor for the new generation of netizens. What started out as a joke is now a worldwide phenomenon, with every group of people displaying their sense of humor through captions and photos. 

I found out about meme culture back in high school when it was still developing within the online forums. What amazed me was the versatility of memes in general. One picture could have multiple meanings depending on the context given by the caption. It was purely up to the creativity of the user.

Motivation:

Motivation for this project comes from the fact that everyone relates to memes and everyone has their own interpretation of memes. But I pondered a question: How can machine learning and AI help in the creation process of the meme? 

It seems to me that a majority of the people in the meme culture are purely consumers. Most people have never made a meme in their life but they find joy in consuming community-made memes. I found that people will likely interpret the meme from what is already given by the creator: a caption and picture. Most people do not stop to think that a different picture can give the same or more comedic value. I think that through machine learning we can test this theory. 

Another motivation is to try and test how well machine learning and AI can be used to provide humor or context. Humor is something that is inherently a human function. We see something as humorous when we can imagine the context. In other words, our brains and imagination count for a large portion of our perceived humor. I want to explore just how significant is our imagination in this process by allowing the machine to pick a portion of the context – the photo/image.

How to build and potential problems:

It is possible to create this machine if we can teach it to recognize certain words that are most commonly related to the specific meme image. It would be a process of assigning words to the photos and comparing the caption given by the user to see which photo fits best. 

Some potential problems that I can see is misinterpreting the content of the quote. During the earlier weeks in class, we were able to play around with a ml5 that detected the sentiment of a sentence. The technology wasn’t able to detect sarcasm and thus unable to correctly interpret the given sentence. Similarly, this model might run into the same issues. 

Ideal Project:

Ideally, I would like the project to be able to give correct images to the quotes.

For my midterm project I want to make progress towards my final project, which is a poster design system created with a machine learning algorithm. For the midterm I plan on presenting my method of gathering data in order to train the machine learning program in a particular style.

Background

My goal is to create a functioning design program where the users can input three sections of text, heading 1, heading 2, and heading 3, and have the program generate a poster based on Josef Muller-Brockmann’s grid system design style. I want to use Muller-Brockmann’s style because it has time-tested appeal and a more algorithmic approach to design, as he based his designs on grids with clean fonts and use of size and color to communicate. Many posters in this style use geometric elements, such as lines, rectangles and circles, instead of images, while employing creative placement of text across the canvas. I think it’s possible to break down these posters into data points that would allow me to train an AI on that data so that it can produce posters in this style.

Motivation

I’m very interested in design and I love creating posters ranging from the artistic to posters for parties, events, and various causes. A lot of the design work I do revolves around the same principles, such as layouts following certain rules or grids, certain fonts that go with the different styles I use, and placement of text and images on the canvas. I would love to create an algorithm for generating posters that can create simple, aesthetic designs following design principles by learning which elements go together. I think an algorithm could learn the principles a designer learns and follows, allowing it to create quality designs similar to that of a designer.

References

I’m inspired by Alibaba’s LuBan AI software, which generates banners and ads for Taobao and Tmall. The software was trained on thousands of well-designed ads and can generate 8,000 banners per second, freeing up Alibaba’s designers to work on more complex projects while the menial work of creating ads with similar layouts and designs is left to the AI. The AI was first trained on several different design elements: copy, product shot, background, logo and decorating artifacts. The team then used reinforcement training to teach the AI which types of designs were “good” and which were not. The designs are then evaluated by design agencies on aesthetics as well as success by how many clicks they get. The program was very successful and now many of Alibaba’s ads are created using this software.

https://medium.com/@rexrothX/ai-visual-design-is-already-here-and-it-wont-hesitate-to-take-over-your-petty-design-job-934d756db82e

Background

It’s interesting to think about the way how earliest Chinese characters were created, as they are visual representations of real objects rather than of symbols. 山 looks like a mountain, 木 a tree; we can clearly see the resemblance. Although modern Chinese characters have developed into a complex writing system that include both pronunciation and abstraction parts, early Chinese characters such as  Zhuànshū(篆书) and Jiǎgǔzì(甲骨字) are more inclined to be pictographic.

Project

I’d like to create a project that is based on these ancient Chinese characters: if a user uploads an image file as an input, trained algorithm will show the result of most resembling ancient Chinese character. Or it can trace contours of an image and create a new Zhuànshū character (I don’t think it requires artificial intelligence to perform such task though). If unsupervised learning is too hard, I may try supervised learning by labelling a few ancient Chinese characters.

Methodology

For the training data, I need to figure out where to get a data set that includes characters as many as possible. I’m thinking of extracting data from a Chinese font file, as it’s standardized and convertible in digital format; 64X64 or presumably 32X32 sizes will be sufficient to represent each character. It may require image classification but I still need a lot more research on how to substantialize this idea.

Examples

Below are the images that I found particularly resembling. It’d be interesting to see how algorithm might pair a given image with a similar traditional character.

Overview

For the midterm project, I’m planning on building an interactive 2-players battle game that requires each player to make certain specific body moves and hand gestures to control the action of his/her character, and the characters of the players in the game will make different moves to attack basing on the specific moves and gestures the players do. The outcome of each move will also be different accordingly. I called this project “Battle Like a Ninjia”.

Background & Inspiration

My idea for this form of interaction between the players and the game is to a large extent inspired by the cartoon “Naruto”. In Naruto, most of the Ninjias need to do a series of hand gestures before they launch their powerful Ninjitsu skills. In most of the existing battle games of Naruto today, players launch the character’s skills simply by pushing different button on joystick. I however want to put more emphasis on all the hand signs these characters do in the animation. Google’s pixel 4 that features with hand-gesture interaction also inspires me.

       

Motivation

I’ve always found that in most of the games today, the physical interaction between players and games is limited. Even though with the development of VR and physical computing technology, more games like “Beat Saber” and “Just Dance” are coming up, still, the number of video games that can give people the feeling of physical involvement is limited. Thus, I think it will be fun to explore more possibilities of diversifying the ways of the interaction between the game and players by riding of the keyboards and joysticks and having the players to use their body to control a battle game.

How to build it & Potential Technical Problem

Most of the PoseNet library don’t include detailed hand gesture recognition so I think I need another model to recognize those specific hand signs that will trigger the character to use different moves to attack. After research, I did find some projects that also involves hand gestures recognition like this and this. I’m probably going to use them as reference while developing this project. I might still be using PoseNet to detect the body movement of the players. One of the major difficulties I’m anticipating now is that there are two players in the game. I’m not sure if the trained model can tell which hand is which player. I think the coding might get a bit messy…

       

Ideal Final Output

The ideal result is that the model can successfully recognize the hand gesture and body movement of each player, thus allowing the character to battle accordingly in the game. The characters can use different skills, attack or dodge basing on the input the player gives, their HP will also change accordingly. And finally the game will give a winner.

I remember seeing Sofia Crespo’s Neural Zoo during the first week of class when I was perusing the AI Art Gallery website and being completely awestruck by the pieces she produced. There was something so beautiful about these organisms that a neural network created:

Inspired by Neural Zoo, I want to focus my midterm on creating a collection of generative art. I love the concept of a neural network being able to create realistic yet whimsical objects we have never seen before, and want to apply this to my project.

I did a little bit of research on AI generative art and found that many works use GANs (general adversarial networks), where one neural network tries to create increasingly “realistic” outputs while another neural network attempts to reject each of these outputs, the play between the two resulting in more and more convincing outputs (Source: Mike Tyka’s “Portraits of Imaginary People”).

Thus, I would like to try and train a GAN to create generative art, with a focus on cityscapes. If I feed a GAN pictures of cities from all around the world, what kind of new city would it produce? Would they look dystopian? Utopian? What kind of cultural elements would be present in each generated city? Many works of fiction try and depict what future cities look like, but these fictional worlds are all determined by humans. Blade Runner, for example, is really interesting to me because everything is extremely urban, dark, neon and partially inspired by Asian cityscapes.

If a neural network created this fictional world, however, what would it look like? This is the question I am curious to explore through my midterm project.