Title : Picture to Picture Style transfer using Cyclegan
For my midterm I wanted to build and reproduce the results of cyclegan. Originally I wanted to do a kind of histogram of gradients effect, but I saw that wasn’t trivial when wanting to choose a direction to warp when picking a feature, so I decided to shift my idea closer to style transfer. The effects of style transfer for cyclegan were really impressive to me, especially with the animation of horses I saw in the repo. The model I decided to train has not done before, however, I saw that existing data has already existed with it. The dataset I decided to train on was Vangogh2photo, which was a collection of 200 van gogh paintings, and 200 vacation style photos. Note that the training set does not have corresponding images for each painting, that is, we do not need specific 1:1 matching in order to train on the data. In addition to style transferring to that domain, cyclegan has a unique property in the way that it trains, allowing one to see a transferred photo being turned back into the original model.
Training
After training for about a week, ( I used checkpointing to make sure the model can incrementally train after the intel server’s walltime was over), I think I got decent results. The original artwork was a little too different from the artwork, so I think the results weren’t perfect, nonetheless, I cherrypicked some good examples which can be demo’d to prove that the model did indeed train well on the data.
Example Output
More photos: https://drive.google.com/open?id=1ErFCdAxX70y83rcGo3_gaQUIiFe_5cVp
Future work and Conclusion
Originally I wanted to build a site which can serve the users the model so that they can upload their own photo and see it style transfered. Overall, I think this project taught me a lot about style transfer and GANs in general. In the future, I will note to collect more and better data to make sure that the results of the generation are more realistic and interactive.