IML | Week10 Explore Deep Dream – Quoey Wu

This week, I chose to explore the Deep Dream because I think its visual effect is mysterious and cool. Firstly, I used the default parameters and tries it on my painting and a photo took before. The results turned out in this way:

  • ‘mixed2’: 0.2,
  • ‘mixed3’: 0.5,
  • ‘mixed4’: 2.,
  • ‘mixed5’: 1.5,
  • step = 0.01         # Gradient ascent step size
  • num_octave = 3      # Number of scales at which to run gradient ascent
  • octave_scale = 1.4  # Size ratio between scales
  • iterations = 20     # Number of ascent steps per scale
  • max_loss = 10.

 

Then I changed the parameters and wanted them to be more sharp. The former one is the result in the first case and the second one is the result in this case.

  • ‘mixed2’: 2,
  • ‘mixed3’: 0.5,
  • ‘mixed4’: 2.,
  • ‘mixed5’: 1.5,
  • step = 0.05         # Gradient ascent step size
  • num_octave = 3      # Number of scales at which to run gradient ascent
  • octave_scale = 1.4  # Size ratio between scales
  • iterations = 20     # Number of ascent steps per scale
  • max_loss = 5.

 

I notice that when mixed2 gets dominant and the step value goes up, the picture will have more lines and patterns, which I think is a good addition to picture of landscapes or scenery. Actually I also tried Deep Dream on some people’s face, but it turned out that the picture would get messy and the face’s outline would be stretched. I personally prefer its effect on landscapes so I remained these pictures. However, I feel like the use of Deep Dream is very random and sometimes to get better visual effect, what we can do is only keep trying. But without doubt it is a cool model which I enjoy a lot!

IML | Week09 Style Transfer – Quoey Wu

For this week’s assignment, we are asked to train the style transfer by ourselves and do some inference.

I chose a picture with colorful geometries so I named it as “colorfulgeo”.

Training

As for the training part, I simply followed the instructions. The obstacle I met is about the setup step because I originally operated in the login node. And there were also some problems with unzip which caused results like these.

Inference

After figuring out those problems, I successfully ran the task and finally got the training result. Firstly, I tried the style transfer in webcam and got screenshots of myself like these:

 

Later, I replaced the video input with some images and got these results:

I think when the input has a certain main object, it works better than when there is only some similar colors or patterns. Maybe it is also because my original image is not so large and has less information. However, at least I have a basic understanding of how style transfer works and there still remain a lot of aspects that we can explore to improve the visual effect at last.

NOC – Midterm Project – Keyin Wu (Quoey)

The Story of A Tree

Introduction

This project basically simulates the process of a tree’s growth throughout the year. I use some abstract shapes as well as some realistic outlines to present this tree by p5. It involves a lot of different objects, like green leaves, yellow leaves, flowers, fruits and some environmental elements. For the final video demo, I also add the background music to make this project more complete and strengthen the user experience.

Process

First of all, I sketched the timeline of different objects and movements. I decided to create the scenes one by one and finally combine them all together. And there are five periods in total.

  • Spring: The Season of Growing

In this period, the tree and leaves start to grow. I created two kinds of leaves in order to enrich the scene, and the two kinds of leaves will also disappear in different ways later.

  • Summer: The Season of Blooming

In this period, there will be flowers popping up, scaling in some frequency related to noise value. And the flower is consisted of several transparent long ellipses and a circle in the middle. 

  • Autumn: The Season of Bearing

This season has most transformations. Flowers turn into fruits and the majority of green leaves will shake and then “explode” out of the canvas. The remaining little leaves, as well as new leaves turn up later, gradually grow yellow. In the end, the fruits fall down the tree.

  • Winter: The Season of Withering

In this period, yellow leaves fade away slowly. The whole scene gets darker as if the cold winter is approaching. 

  • Recovering and Regrowing

With some white particles falling into the scene, the scene will get brighter. Those white particles can be imagined as either snow flakes or simply some light, which represent the signal of recovering. Therefore, after the white particles fall, there will be a couple of new green leaves popping up on the branches, and that’s just another round of story.

As for the debugMode, I mostly used it to check the outlines and the movements of my objects. Since I have different objects in different periods, I tried to keep it simple and obvious. I also added the variable of array length to the controller so that the user can manipulate the number of objects in real-time. I think debugMode tool is really useful and I would like to spend more time on how to combine it with my project better.

Demo

For a better experience, I added background music to my demo. To clarify, I did not make any change to my original recording of my project. I just adjusted the music a little bit to make it fit my project better.

The user can feel the music changes with the scene. Specially in the recovering period, I also added some birds’ singing. And here is the demo.

*BGM: EpicMusicVn – Last Reunion

Further Development

  1. Firstly, I think there can be more vivid creatures like little animals or insects playing with the tree. In fact, I tried to implement another creature class, but the result was not so coherent with my whole project and I did not use them in the end. However, I would like to spend more time on this part in future.
  2. Secondly, since I want to focus on what we learned in the first half semester like vectors, forces and oscillations, I did not spend too much time on the recursive tree. For now, the branches just move in the same way all the time and I think there can be more variations.
  3. Lastly, I think there can be some interaction between each object as well. And they can all interact with the branches in some way.

IML | Week07 Midterm – Quoey Wu

THE WEIGHT OF STROKE

Introduction

Basically, it is a drawing game where two players can draw by round and see whose drawing has higher “value”. In this case, I would prefer to refer to “value” as the complexity of the drawing, considering different people have various perspectives of the value of an artwork. And here I just pick the world-famous paintings as the highest value class. With machine learning technique, the drawings can be classified by machine based on the data set we provide in advance, so I think it is also an interesting try in terms of exploring how machine treats human’s artwork.

Process

1. KNN Classifier with Feature Extractor

Technically, I mostly made use of KNN Classifier with Feature Extractor to define my own classes. Originally, KNN Classifier with Feature Extractor connects with the webcam, so I broke its connection and rebuilt it with the canvas by createGraphics.

Here are my tests about how KNN works with the canvas input. I define three classes based on the number of the circles on the canvas, where the first class has few circles, the second one has some and the third one has a lot. The result shows that around ten to fifteen examples are enough for KNN to work. Besides, in the first three screenshots, examples I added to each class are partly repeated, and I found the confidence value would be easily stick to 100%. Therefore, I later made examples in one class also different from each other, and I then got the result of 66.666% as the fourth screenshot shows.

  

Then I replace three original labels with new examples drawn by myself.

  • First Label – Childish Painting

  • Second Label – Sketch

  • Third Label – Famous Painting

2. Front End by P5

Firstly, I created a balance that moves like this with sin value and vectors.

Then I created user interface with two parts. One part is the balance with the mass showing on its two sides. The other part is the drawing part where the two players can draw on the canvas by round.

Actually, I spent quite a lot time on this part, because it involves many details like different buttons and needs a lot of coding. And as the player is done with his drawing and clicks “Done”, there will be colorful balls falling down to the balance on their side. The ball’s size and mass are based on what the player drew on the canvas. It is because each label is given a range of value, where the first childish painting label has the lowest values and the third famous paintings label has the highest values. Then the confidence of each label will multiply the according value of label and generate the final result about the ball’s size.

 

How it Works

Here is an accelerated video demo.

When the mass on one side is 100 greater than the other or there have been 10 rounds, the game ends.

Further Development

  • Collect more data painted by different people and different styles;
  • Enrich the way of drawing;
  • Improve the user interface;