Author: Andrew Huang
For this week’s assignment we had to retrain the fast style transfer model so that we can use it with ml5js. Training the model was pretty straight forward, as I mostly just followed the instructions and results given by the professor. One thing I did have to do was run the generate weights? python file function afterwards, because of the 24 hour walltime, the python function didn’t actually reach 2 epochs through the very large dataset, luckily, it checkpointed throughout, so I just manually called the function afterwards to get the model saved. I didn’t really get the result I wanted, because I think I did not use a picture with enough contrast in the image. I used a picture of the desert with very little contrast and minimal sky in the background.
However, the results I got weren’t really to my satisfaction, but it was to be expected. Pretty much the webcam image just looks like theres a yellow and white sand filter, not really an example of style transfer, however, there seemed to at least be the proper texture, so in regards to that, the fast style transfer seemed to have worked well. Perhaps the model required more than 24 hours of training. Compared to my midterm project, I think the results of this gan aren’t as good as cyclegan, which transfers not just style but also content and context. It seems this fast style transfer seems to just transfer the texture and color of those photo but doesn’t really seem to understand the deeper meaning of objects in the photo. Also, this model seems to only train off one image only, while cyclegan and other gan architectures need an entire dataset in order to be effective. It seems like it trains faster with only one photo, but it doesn’t accurately represent an entire domain.
Screen Recording 2019-04-13 at 5.58.23 AM
Conclusion
Next time, I will train a style transfer model on a training image with more contrast, patterns, and textures, because I find the results to be most prominent and most visually pleasing. It really amazes me how fast style transfer can work so well in realtime, while traditional style transfer methods require very long training and other prerequisites.