Category: Interactive Machine Learning

Posted on

IML | Week 12: CycleGAN Training – Yufeng

Introduction

For this week, I am training a CycleGAN model which transfers photographs to the style of Todd Hido, one of my favorite photographers.

Domain A

 For real life photographs, I used the Domain B from the ukiyoe2photo dataset. There are around 6300 images, which I probably should have trimmed down for faster training since the number is significantly larger than Domain B.

Domain B

The works of Hido has a mysterious and atmospheric vibe. I collected 400 photographs of him from a photographer community, including landscapes and portraits.

Result

I am (still) training the model on my workstation with a GTX 1080. The estimated time of 100 epochs is 50 hours, so it should be done by the end of May 7th.

Here are some samples generated while training at epoch 20. There is clearly some fogs added into the transfer real photo.

Posted on

Week 12: CycleGAN and Goblins?

This week’s CycleGAN homework had some interesting results in store, especially in the realm of hideous beasts. Wait and watch.

The training was done on colfax, but not by me. I had many problems with many things on my machine, so after many failed attempts, I got the training to work but only to realise that I would in no way finish in time. I cross checked with other people and Abdullah was training on the same dataset (Van Gogh) so I borrowed his checkpoints. 

After getting the checkpoints, I followed the steps to actually get the transfer working. To convert, we had to run the code in the file generate_model.sh. After a bit of peeking, I found that all the parameters were actually set to run with monet2photo and not with any dataset in the checkpoints folder. So I replaced the line python3 test_save_pb.py with python3 test_save_pb.py --dataset vangogh2photo and hoped that fixed the issue. 

However! the results that I got from putting and image into it were almost the same as the ones in monet2photo. I cross checked, found out that they were the same indeed and had to look through all the files individually, including freeze_graph.sh and convert_model.sh. The files actually were also set up to run the inference for monet2photo, so these had to be modified as well!

Here’s what the files look like after modification, in case you had dummy outputs as well: 

freeze_graph.sh: 

mkdir outputs
freeze_graph \
--input_graph=../../train/outputs/checkpoints/vangogh2photo/cyclegan_tf_vangogh2photo.pb \
--input_binary=true \
--input_checkpoint=../../train/outputs/checkpoints/vangogh2photo/Epoch_199_400of400.ckpt \
--output_node_names="a2b_generator/Tanh" \
--output_graph=./outputs/frozen_cyclegan_tf_vangogh2photo.pb

convert_model.sh: 


tensorflowjs_converter \
--input_format=tf_frozen_model \
--output_format=tensorflowjs \
--output_node_names='a2b_generator/Tanh' \
./outputs/frozen_cyclegan_tf_vangogh2photo.pb \
../tfjs/models
Results
 
After modifying the files and using the sample code to inference by passing in a few different kinds of images, here’s the effect you actually get, not much of Van Gogh though: 
 
 
 
 
 
And just to get the goblin-ness off myself, I ran it multiple times using the output as input. Here’s the series: 
 
Turns out that this cycleGAN turned out to be a negative filter after all. 
 
Posted on

iML Week 12: CycleGan Training – Thomas

Introduction

For this week, we had to train a cyclegan model on the Intel AI Devcloud. I created a photo to comic style transfer using a custom dataset collected on the internet. I was inspired by @ericcanete’s instagram artwork, so I created a model using his photos. I also used a dataset from the University of Illinois that was scraped from Flickr and includes portraits of people with a variety of facial expressions and backgrounds. Cyclegan allows us to make image to image translations using two types of training data. I ran the scripts provided to us with some modifications. Training took about 2 days with 112 epochs run. Below is a sample of the training and testing images I used for both domains.

Domain A (1,389 items):

Domain B (1,308 items):

Results

            Left Side: Input Image                          Right Side: Output Image

Conclusion

This was not really a comic style transfer as it was a line art style transfer. Ironically enough, this model does not work well for faces, the thing it was trained on. The black eyes turn into creepy white eyes and the portraits look somewhat spooky. This model is somewhat overkill as there are probably much better algorithms for converting images into black and white. However, it is interesting how we can teach a machine how to do a certain style of art without explicitly teaching it. 

Sources

Comic Art – https://www.instagram.com/ericcanete/?hl=en

Portrait Dataset – http://www.ifp.illinois.edu/~vuongle2/helen/

Training Docs – https://github.com/LynnHo/CycleGAN-Tensorflow-2

Instagram Scraper – https://github.com/rarcega/instagram-scraper

Posted on

Week 12 : CycleGAN – Abdullah Zameek

For this week’s assignment, I decided to use the vangogh2photo dataset that came with the cycleGAN model due to lack of time to procure a custom dataset. Not to mention the time taken it takes to train a GAN model.
That being said, this particular dataset took roughly 36-40 hours to train 200 epochs. 
The results were interesting to say the least. I tested it with 3 different images, and unironically, I chose 3 paintings by Van Gogh to see how a model trained with his style would respond to the cycleGAN.
Additionally, what I did was that I recursively fed back the outputs of the GAN back into the model to see what sort of outcome I’d get. 

Here are the results :

iris
Irises – Van Gogh

Here are the images generated by the model

vanggh

wh
White House at Night

The results of White House at Night is my personal favorite because of the fact that the artifacts that came about on the first image kept getting amplified as you progress down to the other images. 

SN
Starry Night