Kenneth's documentation blog

  • Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • AI Arts
  • Critical Data Visulization
  • Nature of Code
  • Make Believe

Archives for May 2020

NOC-Final_Project-Lost in Dimension

May 11, 2020 by Haoquan Wang Leave a Comment

Inspiration

I was inspired by teamLab light show.

Light in my mind always represents hope and life. In their special time, we are in quarantine time and most people cannot go to see this fantastic show by themselves. This is my initial wish: build up a light project in digital form, and people can get access to it at home. Besides light, I was also inspired by the movie Interstellar.

It is shocking to see this fantastic multidimensional space in this movie. Since I tried to build up my beam system in WebGL, I thought that it is possible to create a similar incredible visual effect in my project.

Development

My project development is divided into three phases: the 3d grid system, the Beam system, and 3d Flowfield.

the 3d grid system

This is the first step of my project. Taking the idea from the 2d flowfield example code, I converted the 2d grid system setting into 3d mode. The first difficulty I encountered is how to deal with the Z-axis. In 2d mode, I only need to deal with x and y axes. But in WebGL, I need to code with the Z-axis. At the first beginning, I set up my 3d space by using Perspective() and Camera(), which are the content that we have not covered in the Nature of Code. These two “WebGL settings” are really tricky to play with. I spent almost one week to learn these things and it turned out that these efforts were meaningless and it resulted that I had less time for the rest of the project development. Another tricky thing is that the 3d’s axis is different from 2d’s. Even though I had learned that in the previous course, I still took the time to get used to the new setting.

As this image shows, the idea is really simple: based on the 2d grid system, I only add z for loop to go through the z-axis. I have to admit that something simple like this took most of my time throughout the whole project development process.

The beam system

The beam system is based on a 3d grid system. Basically, in the for loop that draws 3d grid system, every grid holds a beam

In order to make the line look more like a beam, I used changing RGB values to fill the line. Also, I added transparency into line. For the rotating angle, I used Perlin noise to control the rotation. The idea is as same as the previous flowfield code.

3d flowfield

Af first, I wanted to add a vehicle moving around the 3d space according to the flowfield angle, and then the beam system will track the vehicle. But I failed!

Technically, every time I draw a grid, there will be an angle value generated and stored into an angled array. When the vehicle reaches a certain grid, the vehicle will be affected by the vector which rotates with the assigned angle. However, when I brought vehicles into the flowfield system, it moved in the system but it would be stuck somewhere. I do not know what reason it is. However, even though I cannot achieve my initial goal, what I got, to some extent, is also satisfying.

It is really interesting to see the combination of 2d and 3d.

Interaction

Making a 3d space for exploration on the web is really cool! This is the part I am so proud of. I create an interactive web 3d project.

Demo:

Users can use Arrow keys and Key P and L to control the perspective. Also, QWASZX is the keys to rotation control. In GUI, users can build up their own 3d space by choosing to show boxes, lines, and shapes.

Final delivery

ConfiledChase Music video

Reflection

I am very proud of myself that I decided to get out of my comfort zone to learn relatively new things within this limited time. Learning WebGL is a new journey for me and it is quite challenging. During the process, everything supposed to be simple or as same as how it works in 2d becomes tricky. These features make me a lot of time to learn and get used to how 3d stuff work. Also, the wasted time on the wrong track(perspective() and camera()) caused that I did not have enough time for debugging and project development. I will conclude that my project is not a successful project, or at least it is not perfect. I appreciated all the things I have experienced on this fantastic journey. 

Eventually, big thanks to my instructor Moon! Thanks for his patience and teaching.

Filed Under: Nature of Code

AI Arts Final Project: SneakerGAN-Sketch2Sneaker translation

May 11, 2020 by Haoquan Wang Leave a Comment

Background

In recent years, sneaker culture is becoming more and more popular. We can see people wearing sneakers not only on a court but also in an office, on a street or even on a formal event. The group of people who love sneakers is expanding as well. Compared to the sneaker in the last century, which was mainly designed for sports, sneaker design experience a huge change: it becomes more casual and has a better design. They are unique and they have their own design language. In the era when people hope to pursue their own personality and style, many sports brands provide customizing service to their customers. For example, Nike ID and Nike Bespoke. However, that services are still limited, we cannot design our own sneaker. There fore, I hope to utilize AI to help the people who want to design their own sneakers but have no design skills to achieve dreams.

Model

The model I chose is cycleGAN, which can achieve style translation from categoryA to categoryB or vice versa. For my project purpose, I hope to use cycleGAN to translate the sneaker sketch(drawn by users) into a good sneaker design. The model was provided by Juyan Zhu

Github link: https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix

In order to train this model for my purpose, I prepare a dataset that contains approximately 1500 sneakers images from Nike and Adidas. For sketch data, I use the image2sketch sample code provided by Aven Zhou(AI Arts instructor) to translate my sneaker data into sketch data.

Sketch sample:

Sneaker sample:

Training

All the training process took place on Google collab, an online platform for people to train the AI model. This is the most difficult part of my project. I encountered many obstacles and failed so many times. Here are my training reports:

  1. My own laptop: stopped at 97/200 epoch (testing)
  2. Studio PC: stopped at 24/200 epoch (testing)
  3. Studio PC: stopped at 27/200 epoch (testing)
  4. My own laptop: stopped at 79/200 epoch (with edited dataset)
  5. Ed’s (peer) laptop: stopped at 79/200 epoch (with edited dataset)
  6. Studio PC: stopped at 50/200 epoch (final training version)
  7. Studio PC: stopped at 50.200 epoch (final training version)

For an unknown reason, Google won’t allow me to finish my training even if I activate the pro version. But I still got a training model.

Training process

Model testing

As we can see, this model can basically fill sneaker sketches with black, white, and grey colors. Also, it will show the lightness and greyness. But it is still my expected output, which can translate simple sneaker sketch (profile or layout) into fine design. The potential reason for this output might be:

  1. There is no common design language for sneaker color design.
  2. The dataset is not the perfect one for training.
  3. Training process didn’t completely finish

Supplementation in Processing and RunwayML

Back to my project purpose, I am about to make a project that can allow users to draw and receive a sneaker design output from the model. To achieve this, I decide to use Processing as a sketchbook, and RunwayML as the host for my model. 

  • Processing

This is the code screenshot. Basically, users can draw a sneaker sketch on Processing canvas. Press c, the programming will automatically save the sketch into the Processing folder, and the user needs to choose that image file and send it to RunwayML. After receiving input from Processing, RunwayML will process the input in the model and send output back to Processing.

Resolution testing

Suggested by Aven, I adjusted the resolution of input images in order to experiments whether the size of input images will affect the output quality. 

The offset setting:

 

256×256:

128×128

As we can see, the smaller the resolution is, the darker the output image is.

Future development

  • Dataset
    • Sneaker side images’ color varies too much, it should be simple for the AI model to extract features
    • Part of sneaker sketch data are not full sketch
  • Training
    • Train with a larger dataset
    • Go through all training process(200 epochs)

Reflection

Regarding the training, I should start training as soon as possible since there will be a bunch of issues I may encounter.

Prepare a perfect dataset according the goal I hope to achieve is really really important.

Filed Under: AI Arts

Primary Sidebar

Recent Posts

  • Make Believe Final Project: Autonomous Driving Taxi of the First Autonomous Work Group Company Service–Supportive Policy for Third Child Policy
  • Weekly assignment 5: TV Buddha by Nam June Paik
  • Make Believe Weekly Assignment 4–How to Explain Pictures to a Dead Hare 1965 by Joseph Beuys
  • Make Believe Weekly Assignment III: Uncommon Places The Complete Works By Stephen Shore
  • Make Believe Project II: Documentary “A Day with Black Chinese”

Recent Comments

  • A WordPress Commenter on Hello world!

Archives

  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020

Categories

  • AI Arts
  • Creative Game Design
  • Make Believe
  • Nature of Code
  • Uncategorized

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org
  • AI Arts
  • Critical Data Visulization
  • Nature of Code
  • Make Believe

Copyright © 2025 · Agency Pro on Genesis Framework · WordPress · Log in