Author: Yujia Liu

Posted on

UXD Documentation–Muse

Overview

Our project MUSE is an app gathering information about museums and exhibitions. It aims at providing accurate and comprehensive information. We provide users with a personalized tag-based classification system to help them find exhibitions instantly. For example, we have a tag called kid-friendly so parents who want to take their children to visit museums could select this tag and find educational exhibitions easily.  We place the focus on the visual information of exhibitions such as pictures from the official website to increase the credibility of the source.  In order to avoid being cheated by the over beautified pictures, all the photos on the introduction page are from official resources, and there will be the area of the museum in square feet and probably the amount of the collections to make up the limit of the picture.

Inspiration

The project idea actually originated from a personal experience. One day my friends and I were talking about going to exhibitions during the weekends, and they came up with several interesting exhibitions, but I had no idea what these exhibitions were and how they got the information. So I just nodded and agreed on the exhibition they finally chose. After several discussions with my group members, we found that it could be a universal problem for people finding an exhibition.

Market Research

According to the 2020 China Art Museum Industry Analysis Report, the number of visitors in 2019 was 41.36 million. The report also predicts that the number of visitors will continue to increase in the coming years as China recovers from COVID-19. On the one hand, this tells us we have a large potential user group as well as the exhibition-related app has a promising future. Then we conduct further market research, through our investigation, although the potential user group is huge, there are few apps providing users with a good channel to the exhibition. Compared to the total 41.36 million visitors, in 2019, the museum-related app with the largest user base: iMuseum has only been downloaded 586,123 times, counting for only 1.417% segmentation of the potential users’ group. This means the competitors’ rivalry is weak and is relatively easy for us to enter this market. Thus, we decide to create an app targeting museum visitors. 

Competitor Analysis

To know more about our competitor iMuseum, we analyze its download time in 2019. At first, we thought the decrease in download times was because of the virus. Surprisingly, we found that in 2019.2, iMuseum suffered a sharp decrease in download times. This is before the COVID-19 so the loss of users is caused by other factors.

To study why it fails to engage a large user group, we go through the user review in-app store during that period. We notice two issues. First is users sometimes lost in the wide range of information. iMuseum provides users with information, but they lack a good filter for the user to select. To improve their shortage, we design the tag function providing a user-friendly experience. If you are a mom, we provide a kid-friendly tag. If you are an office worker, we provide weekend-open and open-at-night options. Our slogan is “whenever you use our app, you can find the exhibition we like”. The second weakness is its pictures and text are separated. Users mentioned although there are a lot of photos, there is no text introducing the exhibits in these photos. To improve this, we integrate text with pictures helping users know more about the artifact. Our tag function and visual design are where we stand out.

Survey

Next comes the survey. We did a two-round survey. In the first-round result, the pinpoint was revealed again. First, visitors want to see the exhibition but cannot find the exhibition they want to see, because the exhibition information is too scattered. And again, we confirm the user’s need that 83% of them wish there can be a channel where they could find a summary of exhibition information. Second, they also hope the app can provide a credible but also easy-understanding guidance of the artifacts.

In the second-round survey, we collected 138 results. After information Integration, we found three problems. The first problem is the fragmentation of exhibition information. Official public IDs, self-published tweets, and friends’ recommendations all account for a large proportion of available information channels for users. This leads to the fact that users can only see part of the exhibition information in each platform, but cannot see the comprehensive information. Only 10.22% of users are quite satisfied with the existing platform. 60% of users feel that the content of the platform is not sufficient and some exhibitions cannot be searched on the platform.

To solve this problem, we decide to build a single platform to provide users with all the exhibition information. The second problem is users feel that the existing exhibition information is not that comprehensive enough. Because the existing profiles are mostly provided by a single channel, users respond that they need professional, objective, reference-oriented exhibition evaluation and introduction. Only 9.49% of users felt that the actual exhibition they saw was comparable to what they knew beforehand. Based on this, we intend to introduce the post function written by other users who have been to the exhibition before. This can provide more objective and comprehensive information about the exhibition for those who have not yet visited. The third issue is the classification and timely pushing of information. Only 14.49% felt that the existing search method could find the right exhibition quickly. We received feedback that information categorization is needed. We also received feedback indicating that information about exhibitions of individual interest needs to be pushed more actively. Based on this we decided to add a tag function and also alert users of upcoming exhibition information. 

How do you find the information about the exhibition?
What platform do you use to learn about the exhibition information?
What kind of the information is inaccurate?
To what extent do you think the platform can provide comprehensive information?
To what extent do you think the platform can help find the information efficiently?

Interview

After the survey, we also conducted 17 interviews. 13 of them have a demand for/already using an integrated informative museum app, 9 of them have a demand for a more personalized/filterable museum app. Below are some representative comments from our interviewees. 

Person one: “就其实我主要信息来源也就只有小红书啊点评上关注的博主或其他去过的人的贴文，但关于展本身的信息比较少（或者没有)”

Person two and three: “查找信息费时费力，没有时间。会希望有更多资讯，可以推出人流量数据，避免人太多” “希望更私人定制一些，不要虚假宣传，照片能更有全面性”

Person four: “平常应该一个月去一次吧，公众号我不会常看，但有时间去搜寻信息的时候，常常找不到合适的。至于app的话，希望推送能及时，也能推荐附近的展。”

Again this echoes the three problems: fragmentation, incompleteness, and lack of classification of exhibition information. Based on these findings, we start our further development. 

Iterations:

I–an application that integrates and categorize exhibition information

Based on our first round survey result and interviews, we came up with the conclusion that the pain point of our user lies in the fact that all the information about the exhibition is too scattered and sometimes not authentic enough. Building on our defined pain point, we came up with our first round solution– an application that integrates exhibitions from all sources (both the official channel and visitors reviews). In addition, since our target users don’t want to spend so much time on searching for exhibitions, we want to categorize the exhibitions properly so that they can find an exhibition that caters to their needs within several clicks.

II–in addition to integrating information, having precise tags to be more personalized.

After doing several rounds of user testing, we realized that the categories we originally defined were not user-friendly enough. Originally we were using conventional categorizing methods (solo/collective/retrospective exhibitions). Unfortunately, the users had no idea how to continue with them. After conducting and reviewing our second-round survey result, we found that what users think of when searching for an exhibition to go is actually the theme of it rather than the type. To categorize the exhibition that fit their needs, we searched for all the exhibitions on display and categorized them by themes( such as art/sci-tech/kid-friendly). In addition, to also shorten the time searching, we added a filter function that can filter by tags we defined that fits the need of our target users (open at night/on weekend). We also worked on the description page so as to make it simple enough for the users to make decisions quickly.

Nonetheless, since we skipped the low-fidelity digital version and turned to relatively high-fidelity one right away, it seems that we have been focusing too much on the detail of interaction rather than the general user flow itself. For the next iteration, we will conduct a new round of user testing so as to identify frictions within our current design.

Further Development

  • Modifying home page (simple and clear)
  • Posts and Review section (to create a sense of community)
  • Confirm and enrich the tags of exhibition
  • Highlight terms that will help strengthen the understanding of the exhibition, and make the highlighted text clickable. 

     

Citation:

2020年中国美术馆行业分析报告-行业运营现状与发展趋势预测. baogao.chinabaogao.com/qikantushu/398090398090.html. 

Posted on

AI Arts Final Project: A-Imitation — Crystal Liu

Inspiration

The inspiration of this project is quite ridiculous. At first, I didn’t have any ideas about my final project, so I just browsed some random things on the Internet. Then I found this picture:

Related image

And after I clicked it to learn more about it, I found that there were so many interesting and weird spoof paintings on the Internet. These paintings reminded me that I could build a project to let people add their creative ideas and their characteristics to the well known paintings. Besides, I noticed that people tend to imitate the signature moves of the characters in the painting, such as Mona Lisa and The Scream. The connection between the motion or poses and visual communication reminded me a project called Move Mirror. Here is the demonstration of it.

I really liked the connection and I wanted to apply such an idea to my project. In my case, if the users imitate the poses of the figures in the painting, there will be a corresponding painting beside the canvas to tell them which painting the machine think they are imitating. 

The last one is the artistic filters of Beauty Camera that can transfer the original camera style to the oil painting style. I wanted to use style transfer model to achieve such effect.

Related image 

My Project

Firstly, there are six paintings on the top of the web page working as the reference. The user can click the “Load Dataset” button and then click the “Start Predicting” button to play the project. If the users imitate the poses correctly, they can see the painting on the right side of the canvas. Also, if they press the spacebar, they can transfer the style to the painting style. You can see a brief demo video through this link:

https://drive.google.com/file/d/120leTwPVZvnTRMPI0Y-0_3AK9wFFSrUC/view?usp=sharing

Methodology & Difficulties

I use KNN &  PoseNet and Style transfer to build my final project. I have already used KNN&PoseNet in my midterm project so the logic is similar. I just need to define the classification result and the corresponding output. For this case, the output is the picture of the painting and the style of it. For the style transferring, I referred to Aven’s example about multi-style transferring and used the array to decide which kind of style it would display. 

One if the difficulties is the logic of my project. At first, my plan is to let the user take a screenshot and the result of style transfer will show on that image. However, I failed to take a screenshot directly. The only thing I could do was using saveCanvas() to download the screenshot and the outcome was not that good. Thus, I gave it up and tried to show the change through the live video rather than photos. And it worked well. 

The other problem is the outcome of style transfer models. I chose a lot of famous paintings as follows:

Meisje_met_de_parel.jpg (4095×4794)

   

However, the result of training models was lower than my expectations. Some of them were quite similar, so I just abandoned them and found different one. And these are some results of the models.

   

According to Professor Aven, it was because some of my images didn’t have vivid shape and color. Thus, the model couldn’t give style with strong features. I accepted his suggestions to choose the part with bright color and strong sense of geometry instead of the whole image. And the result was better than before.  For example, I only used a part of the portrait of Van Gogh to train the model.

What’s more, I learned how to train several models at the same time. Before I only changed the file but didn’t create new checkpoint and model folders. In this way, I can only get one model because the latest one could cover the previous models. Now I know how to create the checkpoints and models and download multiple models.

The last step is to beautify the UI. I chose a simple but classical image as the background to fit the characteristic of a gallery. Then I only kept “Start Predicting” and “Load Dataset” buttons and changed their original style to an artistic one. Next I made some cards with the basic information of the paintings and the paintings themselves. These cards were put on the top of the web page as a reference for the users.

Significance & Further development 

I have noticed that building this project helped me learn more about the paintings, especially their name and creators. Therefore, the significance of my project could be teaching the users some basic information about the famous paintings. For the further development, I wanted to enhance the educational function of my project. If the reference card contains more information such as the background or style of the paintings, the users will learn more about the paintings in a relatively interesting way. However, it’s essential to figure out how to draw the user’s attention to the reference. My idea is to add some text bubbles on the painting so that it seems like the figure in the painting is telling the information to the users. Also, I can add some sound files in my project to enrich the form of interaction. In addition, I plan to increase the types and the number of the paintings to enrich both input and output.

Posted on

MLNI Final Project Documentation–Crystal Liu

Background 

My final project is a further development of my midterm project. My midterm project was inspired by a movie called Nezha and it was an interactive painting. The users can trigger the motion of the still object by getting close to it. They can also hear the sound of animals or water by the same method. However, there were some features that didn’t work very well. For example, my thought was to let the users trigger the cloud by raising their hands. However, since I didn’t add any hint to inform them, the outcome was lower than my expectations. Besides, my interaction part was not that smooth and natural. The reason might be that I didn’t think about the logic of my project, I just added what I wanted to my project. I also received many excellent suggestions from my professor and other guests, and they really inspired me a lot on my final project.

Inspiration

One of the inspiration of my final project is Shenshen’s midterm project. I was surprised at how her project could let the user move the image through their eyes to see the whole picture. It looked like the user was actually walking through the gallery but not just seeing it. I also wanted to make my painting much larger than the canvas size so that it could be more immersive for the users. What’s more, I wanted my project to tell a story or to have logic but not just a pile of random elements. Since Christmas was around the corner, I chose this festival as the theme of my project and the story line was how Santa prepared and deliver presents.

My Projects

My final project is an interactive painting whose theme is Christmas. It has four different scenes. In the first scene, the user is a red glove and the position is determined by the position of user’s eyes. There is a closed book with an arrow pointing to it. The role of this arrow is to give hint to the user to get close to it. Once the user get close enough to the book, it will open and show a Christmas tree, which is the second scene. Then the user will notice the gingerbread since I add this GIF to emphasize it. 

Once the user approaches to it, the glove will change into the gingerbread which means that the role of user in this painting is it. Then there will be a right arrow telling the user move to the right edge to go to the next part.

The next scene is a postcard in Christmas style I made by Photoshop to tell the user how to play this interactive painting. Then the user can go to the last scene by the same method. And the last one is an image as the follow:

This is the original image I found online and I also added some other images and GIFs to make it interactive, just like my midterm project. Besides, I found some sound files such as Jingle bells in elf tune and the sound of clock. If the user approaches to the right edge, he or she will see an arrow pointing to the right. And if the user goes further, the image will move to the left so that the user can see the rest part of the painting. They can also choose to go back to the left part by move to the left edge, just like pressing the arrow key on the game console.

Methodology & Difficulties

One of the most difficult part of my project is the coordinates. At first, I accepted Jamie’s method to let users move the image based on the position of their wrist. However, after several trials I found it was so tiring and the wrist  usually blocked the eyes so it would influence the position of the gingerbread. Then I decide to use the coordinates of eyes to move the image. The solution is that I defined a value called xx, whose original value is 0, and let it be the x-coordinate of the image. Then I set some conditions to achieve my expected result. The reason to set two areas is to make the interaction in a gradual way, which is inspired by Tristan’s suggestion.

Here came another issue. The image I added to the large one couldn’t move with the main scene and the area I set as the condition couldn’t change with the image either. So I just applied the same logic to these still images and it worked.

 

The next one is taking control of the GIF. 

  

My initial idea is that if the user touches the book, there will be a GIF displaying the opening process of the book. However, I couldn’t make the GIF play for only one time then stop. Thanks to Professor Mood, he sent me a sample code to solve this problem. Unfortunately, when I applied the method to my project, I got hundreds of errors, so I had to give it up. Now I know the reason is that the library for GIF was so advanced and I didn’t update my Atom so it couldn’t recognize the code.

The last one was not a problem but an essential method which is switch function recommended by Professor Moon. This function is so important for me because I have so many scenes, and they all require the position of the eyes. However, I cannot set different variables for each scene which is too much for me. This function solved the problem. I can set a condition to decide which mode I want to display and there is a break between each mode. I learned the importance of break from my extremely embarrassing final presentation. I didn’t add break between mode 3 and mode 4 so that my function didn’t work and I couldn’t see the images I added on my project. After seeking Professor Moon for help, I learned the reason and also the significance of break in switch function.

Further Development 

For the further development, I want to focus on the transition part of my project. I really like Jessica’s project because she made some transitions in changing the original  image to another one. I also want to apply fade away effect to the change of scenes in my project. Also, I can use fade function for the sound part to make smoother interaction. Last, I want to add some text bubbles around the characters in the painting such as Santa and elf. The content is about the background information about them and Christmas. In this way I can make my project more educational.

I sincerely appreciate everyone who help me with my project and who like my project. Especially Professor Moon, he really helped me a lot and made me realize that I can use knowledge related to machine learning to achieve some creative and artistic ideas. And I will definitely apply what I have learned from this course to the rest of my academic life in NYUSH and even after I graduate in the future.

Related image

Posted on

Week12 Assignment: Final Concept Documentation–Crystal Liu

Background+Motivation

My final project is mainly inspired by the re-created famous paintings, especially the portraits. Some people replace the Mona Lisa’s face with the Mr. Bean’s face and the painting is really weird but interesting. 

Image result for mr bean and mona lisa

Also, I found that some people tend to motivate the poses of the characters in the painting, such as The Scream:

Image result for people imitate the screamImage result for people imitate the scream

Therefore, I want to build a project to let the users add their creativity to the famous painting and personalize the paintings to recreate these paintings. It reminds me my previous assignment for style-transfer. For that assignment I use a painting from Picasso to train the model, so that everyone or everything showing in the video can be changed into Picasso’s style. Even though the result is not that good, it still shows a way to personalize the painting or to let the users create their own version of paintings.

My idea is that the user can trigger a famous painting by imitating the pose of the characters in that painting. For example, if the user wants to trigger The Scream, he or she needs to make the pose like this: 😱. After the painting showing up, the user can choose to transfer the style of the live camera to the style of The Scream. If the users want to change to another painting, they just need to do the corresponding pose to trigger the expected painting.

Reference

My reference is the project called moving mirror. The basic idea is that when the user makes a certain pose, there will be lots of images with people making the same or similar pose.

What attracts me most is the connection between images and human poses. It displays a new way of interaction between human and computer or machine. Users can use certain poses to trigger things they want, and in my project it is the painting. 

The second one is style-transfer. It reminds me some artistic filters in Meituxiuxiu, a popular Chinese photo beautification application. These filters can change the style of the picture to sketch style, watercolor style or crayon style.

But the filter is only for still picture. I want to use style-transfer model to add this filter to the dynamic video so that the user can see their style-changed motions in a real time.

Posted on

Week11 Assignment: Exploration on deepDream — Crystal Liu

Problems:

This week we have learned BigGan and deepDream and I have done some exploration on BigGan in the class. Thus, for this week’s assignment I choose to explore more about deepDream. I met some errors when I train the model. One is that I ran the python script without installing keras and tensorflow so that I couldn’t run the script. The other one is that I didn’t change the a+=b to a=b+. After I changed that, the model could run successfully.

Process:

This is my original image:

The default version looks like this:

Then I changed the step from 0.01 to 0.1 and the result looked vague.

The next step I changed the number of scales from 3 to 6 and the result looked like this: it looked more like the original one except for the ring in the middle.

Then I changed the size ratio from 1.4 to 1.8 and the result looked like this:

Then I changed the max-loss and found this change doesn’t have great effect on the result: