Category: aiarts

Case study: Alibaba – ImgCook 

Motivation:

The aim of developing Imgcook is largely due to the heavy business code (duplicate view development). Honestly, most of the requirement and codes are redundant work which stimulates the demanding of directly generating codes based on the screenshot to free the labor that invests in redundant work.

Approaches:

In terms of the approaches, the Imgcook works in a way where transfers the sketch/screenshot to the middle structure and maps the middle structure into certain code paradigm like React or Vue.

Thus, the core part is to recognize components and patterns from a sketch and translates them into the middle representation. In this part, a bunch of technologies like OpenCV, edge detection, SSD and RNN are applied to ensure the outcome.

Current:

Right now, the Imgcook has involved in real production. And it has dramatically increased the efficiency of converting the sketch to real code.  Normally, a sketch that needs 6 hours to implement will be reduced to 1 or .5 hour. And they are trying to continuously develop it comprehensively for more functionalities like a logic generation and customized code style. 

Case Study: Edmond de Belamy, the First AI-created Artwork to be Sold at Auction

Written by Jonghyun Jee (John)

PowerPoint Slide: https://docs.google.com/presentation/d/1_eeYZaG6oL4r_ZzJkrk7BY5UhmNUHYYz8heThtWTZBU

Ten or more years ago, I was reading a book about futurology. I don’t remember most of its contents, but one thing still fresh in my memory is a short list in its appendix. In the list was a number of best jobs for the future, mostly consisted of the occupations I’ve never heard of. A single job caught my attention: an artist. It had seemed so random and unconvincing until I read the description. The author noted that AI and robots are going to replace most human jobs, therefore an artist—of which capability has been considered uniquely human—will probably survive and thrive even more. Ten years have passed since then and a lot has changed. A way more lot than the author might have thought.

Last October, an artwork named “Edmond de Belamy” was sold for $432,500 in a Christie’s auction. As you can tell from the image above, the portrait seems somewhat unfinished and uncanny. We’ve seen a myriad of abstract and puzzling artworks in recent years, so you may wonder what makes this portrait so special, worthy of hundreds of thousands of dollars.

If you look closely enough into the signature at the bottom right, you will notice a random math equation—which is extremely rare to find on top of a classical oil painting. This signature left by the creator of this artwork quite accurately states who painted it: an algorithm. 

That is, the painting we’re seeing now is the first-ever sold artwork created by not a human mind but an artificial intelligence. “Edmond de Belamy” is one of a series of portraits called “the Belamy family,” created by a Paris-based collective “Obvious.” They utilized “GAN,” which stands for Generative Adversarial Network (as far as I remember, Aven briefly introduced what GAN is and gave us a couple examples such as CYCLEGAN and DISCOGAN). 

The mechanism underneath this portrait is pretty straightforward. Hugo Caselles-Dupré, a member of Obvious, says that “[They] fed the system with a data set of 15,000 portraits painted between the 14th century to the 20th. The Generator makes a new image based on the set, then the Discriminator tries to spot the difference between a human-made image and one created by the Generator. The aim is to fool the Discriminator into thinking that the new images are real-life portraits.” 

At this point, humans were the one who fed the AI with a data set. In the future we may see an AI system that can somehow choose which sort of data it will be fed. Whether a robot Picasso is coming or not, I’m excited to use this entirely new medium to explore beyond and beneath the realm of art we currently have.

Sources:

Image 1 & 2 © Obvious (https://obvious-art.com/edmond-de-belamy.html)

Image 3 © https://allthatsinteresting.com/ai-portrait-edmond-belamy

The project I’ve found is Watson Beat, which is from IBM. IBM is an American multinational information technology company, and it mainly focuses on computer software and hardware. According to IBM’s website, IBM has created Watson, an open platform powered by machine learning. One of its main functions  is to let people automate the AI lifecycle. Watson Beat is a version of Watson and it focuses on composing melody.

Watson Beat can create complicated compositions on the basis of the input simple notes. What the user needs to do is to play a simple piece of melody as the input. Then Watson Beat will get the sound and generally analyze it in a short period. After that the user can choose different mood of the melody such as dark, romantic, and amped. Finally Watson will output a track based on what it heard from the users.

Here is an example showing how it works:

And the following videos display the compositions created by Watson Beat.

Watson Beat’s development is based on two methods of machine learning——Reinforcement Learning and Deep Belief Network. Generally reinforcement learning is used to get the best possible path in specific situations and maximize the reward. DBN is an unsupervised probabilistic deep learning algorithm, and it can create a function to achieve users’ goals. The role of reinforcement learning here in Watson Beat is using western music theory to build reward functions. DBN can train on a simple input and then create a complicated track.

Even though Watson Beat can create lots of rich and great melody, its main function is still to inspire composers but not to completely create melody and then replace the composers. When musicians or composers are struggling to build the first idea of a song, they can just randomly play some notes. And then Watson Beat will make the melody richer, which might give them inspiration. 

I’ve had a deeper understanding of how artificial intelligence and machine learning indeed benefit people in the creative area, especially of Reinforcement Learning and Deep Belief Network. Artificial intelligence is more like assistant for human in essence, at least in creative field. The full use of artificial intelligence can improve works of art and make them better. Because the analysis of AI is based on the professional theories and large data sets, which makes it more reliable. Human is still in the dominant position to decide what they want artificial intelligence to do to assist them. AI outputs an example or prototype and the artists will make the prototype a complete work. So there’s no need to worry about whether the development of artificial intelligence could replace human in creative activities. 

I’d like to quote Elizabeth Transier, IBM THINKLab Director, a sentence as the end of my blog, since I really believe his opinion about how artificial intelligence is benefitting us human. As he says, “Watson Beat is a great example of how IBM cognitive technologies are starting to augment human capabilities and help us reach new capabilities.”

The link of my presentation: 

https://docs.google.com/presentation/d/1-6eAQqNOhDy0j46je3fcbOvEIIyCvHCV8lTbsZAqhoY/edit#slide=id.p

Reference:

https://medium.com/@anna_seg/the-watson-beat-d7497406a202

https://www.ibm.com/case-studies/ibm-watson-beat

https://www.geeksforgeeks.org/what-is-reinforcement-learning/

https://medium.com/datadriveninvestor/deep-learning-deep-belief-network-dbn-ab715b5b8afc

【Link to presentation slides：https://docs.google.com/presentation/d/1HGNBJahn-no-A4c0an4ArfwE15fDsi-m95B06KopOjU/edit?usp=sharing】

Research：

The case I studied is the AI designer “Lubanner” from Alibaba. According to the annual report released by Alibaba, every year on November the eleventh (also known as the “double eleven festival”), about 40 million different posters and banners are called for for various promotion activities on Taobao. Such large amount of workload used to be extremely expensive and time-consuming for the company. However, since 2017, with the help of AI tech and machine learning, the artificial intelligence designer “Lubanner” created by Alibaba has improved the situation to a large extent.

  “Lubanner” is a self-improving artificial intelligence program designed to create a large number of advertising banners and commercial posters in a short time. With the help of big data and the AI tech, the programmers write functions to train “Lubanner” to receive inputs, process the similarities and patterns between different inputs, and give outputs that meet the requirement by imitating these patterns.  There are basically four things the programmers are doing. First, the digitalization of designing. They teach “Lubanner” to use image processing algorithm to identify and cutout the five major parts of each picture: the product, slogan, background, logo and decorations. Secondly, “Lubanner” archive those data. After that, programmers wrote function to teach “Lubanner” imitate those input posters and banners and create its own work. Finally, it evaluates its output before giving out the final design result. Here is a demo video presenting how “Lubanner” work in action.

The efficiency, low cost, and diversified outputs of “Lubanner” are all outstanding. According to Alibaba, it used to take a junior designer 20 minutes to work out a product banner. However, today, “Lubanner” is able to design up to 8000 different banners with various themes in 1 second. What’s more, the aesthetic of its works is also getting better and better, meaning that its works are getting more and more appealing to customers, sometimes even more so than works created by real humans. The reason why the machine is able to tell aesthetic is that it keeps updating itself by receiving and memorizing the most appreciated data about certain work and working out the key factors. It is clear that most commonly-used apps all have similar UIs, so are many banners and posters. AI designers can abstract and learn from these patterns by imitating them to create more appealing works for customers on their own. Thus, in the future, it’s only naturual that AI designer is going to take over more and more junior design work.

Even though it is clear that AI designers like “Lubanners” aren’t really producing any innovative art works,it’s actually more like a design factory that is able to produce diversified products. Still, these kinds of seemingly soulless yet effective design products have brought huge convenience to business company and are also becoming more and more well-received by the audience. The potential of the AI designer is numerous, and ot will most certainly one day revolutionize the way we define the word “design”.

source :https://www.alibabagroup.com/cn/ir/home; https://medium.com/@alitech_2017