Author: Like Yang

 Our user test video is here.

Conception and Design

At the very beginning, we would like to make a drawing machine for people with hand disabilities that could not hold a pen. After meeting with Young, he said that we would better focus on a smaller group of people instead of the broad audience of people with hand disabilities. So, we narrowed our scope to people without fingers. Based on this idea, we chose to use the largest buttons we can find in the ER and slide potentiometers instead of knobs to allow users to adjust the color and size of the brush. Such choices may ensure that people without fingers would (ideally) have no difficulty in using our drawing machine. For the part that controls the position of the brush, the original plan was to use a joystick but thanks to Zander’s recommendation, we replaced it with Leap Motion. Although Leap Motion could not compete with the joystick in terms of accuracy. We still think it would be a better choice because the joysticks we found in ER are too small in size, which would not be very friendly to our users. 

Another particular design we used, which few people have noticed, is our case for Magic Brush. When presenting out midterm project “Duet Beat”, Professor Marcela said that instead of ordinary boxes with squared shape, maybe we should come up with other designs to illustrate clearly how could the potential users use our project. This suggestion is of great value to us because accessibility and availability are something Guangbo and I have been always pursuing. To integrate this idea into our final project, I found the laser cut template from this webpage. It is originally a box clock but there is a slope on one side. Although this seems to be a very minor design difference than using a box from Makercase, I believe it could better hint the users that they need to press the buttons we provided, especially for children who are not quite tall but may use our project. This design enables them to see the buttons directly if their vision is not as high as the top of an ordinary box.

Fabrication and Production

From my perspective, the biggest success and failure are closely linked with each other in our project. I feel really happy that unlike our midterm project, we have enough time to fully realize our blueprint. Some of the aspects even attained better effects than we thought at the beginning (e.g. the use of Leap Motion instead of joysticks). However, it is very sad for us being unable to invite people with hand disabilities to test whether this project could really assist them in creating art. Although letting Magic Brush educate users about people with hand disabilities is still quite meaningful, to us, it is more like a compromise. The version we prepared for user testing session has a mandatory instruction that takes everyone 20 seconds to read at the beginning, most people did not like this design. Some suggest that we can show instructions under the main canvas so that users can refer to them while they paint. After consideration, we still decided to remove the instructions altogether because in Tom Igoe’s Making Interactive Art: Set the Stage Then Shut Up and Listen, he already said that designers “shouldn’t interpret their own work.” Meanwhile, our biggest concern as designers that people do not know how to interact with Leap Motion has been solved by using a red sticker to identify the brush and a physical wooden frame to identify the canvas. In our final version presented on Tuesday, there is a zone under the canvas used to show information such as “Saved Successfully!” and the painting the user just created. This suggestion comes from Professor Rudi. He said that we would better give users clearer responses after saving. Guangbo and I think this is indeed very important because it can increase the overall accessibility of our project. The responses have the same effect as the small light on our midterm project so the user would not be lost about what the machine is doing. Near the end of the user testing session, we invited Young to use our project, he said that if we insist regarding Magic Brush as something that can help people with accessibility to create artwork, we need to do some further improvements such as modifying the shape of the slide potentiometer. (which we did later by laser cutting a smaller box) He also mentioned that without real users to test, he could not say that Magic Brush is fully capable of helping people with hand disabilities to create art. If that is the case, turning it into an educational project would also be a good idea. Guangbo and I thought about this and have to admit that maybe positioning it as an educational project would be the most practical solution. In the end, for educational purposes, we prepared tapes in order to simulate the scenario to the best extent and added the QR Code function to further raise users’ awareness of people with hand disabilities.

Conclusions

The goal of Magic Brush is to create an educational experience about people with hand disabilities and raise everyone’s awareness on this group of people by letting them experience how difficult it is for disabled people to do some things we think are normal such as painting. This project is in line with our definition of interaction. It has two actors (the user and a computer), there are input (through buttons, potentiometer, and leap motion), processing (through Arduino and Processing) and output (visual and audio outputs). There are clear feedbacks both when the user is drawing or saving his or her work. The big buttons, design of the laser-cut box, adoption of leap motion and other factors revealed our effort of bringing accessibility and availability to everyone. However, to achieve the highest level of accessibility and availability, we should still invite real users with hand disabilities to test our project. Generally speaking, the audience interacted in the way we planned during our user tests. They could carry out the full process of interaction we have designed. If we have more time, we think we may consider the possibility of different brush textures (turning dots into lines, simulate the feeling of painting with a pencil, a pen, a sprayer etc.) Perhaps we will also make some small changes to the design of the box and turn it into a colorful one to identify that this project is used to create art. 

I used to think that since Guangbo and I have already gone through so much during midterms, we would not learn as much when making our final project. But of course, this idea is wrong. The process of designing something for people with disabilities helped me to realize how difficult it is to put the idea of “Making user the center of project design” into practice. It is indeed very difficult to design something for a group of people you do not belong to. Even though I think we are already very considerate on each designing details, we are still not confident that Magic Brush could really help people with hand disabilities to paint. Now I have an increasing feeling that in our daily lives, there may be so many details that we take for granted but other groups of people, whether they are different from us physically or psychologically, feel uncomfortable with. These details are often not self-evident. 

The example above, I think, can help to explain my thoughts. We would not find it weird if we see a handrail like the one in picture 2. But when we encounter a different kind of handrail like the one in picture 1, of course, we would regard it as a better choice because it considered the need of children. I hope that the production of Magic Brush will serve as the starting point for me to truly identify the need of people we do not often get in touch with. We should really work more on “the rest of us” and come up with solutions that are accessible for everyone to interact better with the rest of the world.

Media Manipulation with Leon @ Room 824

Video:

Process:

The workshop I attended was about media manipulation. In the end, I came up with a video featuring political celebrities and animation characters. There were not too many difficult tactics involved but it is actually very funny, at least from my perspective. The first thing I did was to cut the originally 2m08s long video into 30s and make it play again and again. To realize this effect, I simply used a ”if” function, together with the ”myMovie.jump” statement. I was afraid that running such a video over and over again will cause Processing to crash but actually, it did not. This made me realize that Processing can somehow replace some of the functions within other video-editing software, which is quite convenient. After that, I imported some images and make them appear at proper timing. Leon told us that we should import the pictures into Processing during the setup stage and then retrieve them in the drawing stage. This technique can make the project run more smoothly as the images do not have to load over and over again. The last functions I tried was tint and filters. They generally worked well but I was wondering whether I should consider using different colors in the tint function to make the frame clearer. Also, I may adjust the size of the stickers beforehand so it can fit into the entire frame better.

Sources:

We are Happy on Vimeo: https://vimeo.com/333893523

Kim Jong-Un Clapping: https://knowyourmeme.com/photos/444771-kim-jong-un

Donald Trump Clapping: https://gifer.com/en/72XG

Putin Dancing: https://tenor.com/search/putin-gifs

Xi Jinping Clapping: https://asia.nikkei.com/Economy/Xi-s-new-era-marred-by-old-problem-of-a-slowing-China2

Exercise: Media Controller

Video:

Component:

1 * Arduino Uno

1 * USB-B to A Cable

1 * 10K Potentiometer

Several Jumper Cables

Process:

The video clip I used was created by Chait Goli called A Tower Beside A River. You can find the link to this video here on pexels.com.  What I wanted to make in this recitation is a controller that can jump to different frames of the video. Therefore, users will have a feeling that as they rotate the potentiometer, the camera also moves towards the direction they are rotating. There are two steps involved to make this controller. At first, I need to set up the Arduino part and transfer the potentiometer readings from Arduino to Processing, which can be done by the template we currently have. However, I happened to find the original video was in 4K resolution so my laptop could not display the entire frame. I used VLC to lower its resolution into 1280 * 720 and it worked. The next step is to incorporate the readings to the Processing example for videos with the readings from Arduino. Although the codes worked well, the effect was not what I quite wanted. Every time I rotate the potentiometer, there is a delay for the transition of frames and as you can see in the video above, the transition between each frame is not very smooth. I asked Leon about this question and he advised me to increase the frameRate from 30 to 60. In the end, I increased it to 120 and the situation is slightly better. According to him, if I really want to realize my plan, I would better use photos instead of an entire film clip because Processing needs to load the entire video over and over again. This experience I had during this recitation made me know that sometimes our idealized effect are bound to the limitations of hardware and we need to figure out alternative ways to duplicate the effect. 

Reflection:

I find it amazing that the earliest interactive artworks appeared in the 1960s, which means that the idea of involving the entire human body into the interaction with computers is not new at all. Compared to the projects mentioned in this article, such as Messa di Voce and LIMBOTIME. I believe that the ways of interaction I used in my project could not bring a novel feeling to the users. Making use of potentiometers is not that different from the traditional keyboards or mouses because computers could only sense the human movement through their hands, while for computer vision projects, it is often the case that the entire human body would be captured as an input. What is more, the output of my project is only some changes within the frame itself. I am thinking that maybe we should make the users more engaged in the interaction process by presenting their bodies in the visual output. By using computer vision algorithms, the users would be able to get more customized feedbacks, which can really amaze them. That’s the effect I think would attract a wide range of audience and really reveal the power of interaction.

There were five people in our group so I critiqued on four other projects during recitation. The first project is a game inspired by Flappy Bird and The Avengers. Two players on different sides would control the Thanos and the Avengers separately. The Avengers have multiple arms to attack Thanos. If they shoot him down within 1 minute, then the Avengers win. Otherwise, Thanos wins. The second project is an immersive one which allows users to explore nature together with ancient Chinese music. By touching on the flowers, the user would be able to hear different Chinese ancient melodies. The aim of this project is to provide a recreational feeling for the people living in such a fast-paced society and raise people’s awareness of culture. The third project is also a game. Two players will control a whale and a garbage-collecting ship in the ocean separately. The whale has to stay away from all the garbages in the sea and the ship needs to help collect the garbage. If the whale was hit by any floating barriers, they both die. In the end, the fourth project is still a game. However, in this single-player entertainment, the user will only be able to see a small fraction of the map and for the most of the time, they need to judge their direction forward by hearing the sound positions from their headphones. If they can’t tell the difference in tones, they will meet a monster and dies. 

For the first project, I think it is a little bit too easy because the only way for players to interact in the game is either through buttons or joysticks. However, I really like the idea to combine the most popular movies with the project we are going to make. The second project has very good concepts and ways of interaction. Indeed, too many of us need to slow down our life paces and be nurtured with immersive cultural and natural experience. Meanwhile, they connected technology with living creatures, which could make traditional ways of interaction more close to human beings. The only problem is that they may not have too many time to achieve what they want since their plan is really ambitious. The concept of caring about our marine creatures in the third project is also something I like very much. But since they haven’t come up with very specific plans of what ways of interaction they would use in their project, I could not offer further comments on their interactive experience. The last idea of the last project is very sophisticated and I think it would bring the best experience to users if they could realize their proposal. It is just they have to overcome two difficulties that are central to the user experience. The first is the idea of making the sound come from different directions. The other is illuminating the areas around the player and make the illumination move as the player moves. 

In terms of interaction, I feel we do not have to stick around Processing by making a game. In this case, users’ interaction with the computer would be limited to joysticks, buttons, mouses or keyboard. Dating back to our midterm, we used many more ways for our interaction such as distance sensors, LED lights, etc. I don’t think we should focus that much on the content we present. Instead, we should still go back and pay our attention to the ways of interaction.

My group members offered me a lot of useful examples. They told me that based on making a paintbrush, I can add different styles of filters or make the stroke smoother. We should also add a function to save the paintings our users have created. Because we are making a piece of painting equipment for people with disabilities, it is also a good idea to add links or QR codes to raise money for charity organizations. To illustrate the potential of our project, we should first make some examples by ourselves to show others what amazing artwork can they create. The most successful part is that what we plan to make is a tool instead of a game, which means people will not get bored with it. The least successful thing is that they may not realize what can they do with the machine and it is our responsibility to tell them how to unlock the possibilities. I think what they said during the recitation is very useful and I will definitely integrate their ideas into our final project. We should make the mission, the usage, and the potential audience more obvious when we really begin to make the project.

A. Project Title

Magic Brush (Preliminary Name)

B. Project Statement of Purpose

Our midterm project Duet Beat is a machine that can generate customized music based on the users’ heartbeats and arm movements. Now that we have learned the Processing language, we would like to switch from the field of music into visual arts. In our final project, what we want to make is some equipment that can help users come up with their own paintings with the minimum effort. This new project would not only inherit the features favored by our users – accessibility and availability, but also integrate the new skill sets we learned after the midterm. We expect to come up with a drawing machine for the widest audience who do not necessarily need to have prior art training. By moving the joystick, pushing the buttons and adjusting knobs, everyone can create a painting with their own characteristics.

C. Project Plan

In our final project, there will be a box with multiple joysticks, buttons, and knobs. By adjusting the parameters, users will be able to come up with a customized brush to paint on the canvas in Processing. The brush can either be a normal one or in the form of pixels. We will also install some sensors, including but not limited to sound sensors and light sensors to reflect the environment the user is in when composing his or her artwork. When choosing components, we are going to use those in bigger size so it is more easy for our users to navigate through the control. Our box will be made by means of laser cutting. In the first stage, we are going to connect the components within Arduino and set the codes to transfer parameters into Processing. After that, we will decide in what form should the painting brush appear on the canvas and integrate it into our codes in Processing. In the end, we will explore ways to add features such as pixels and generative art in order for our users to create more sophisticated contemporary paintings.

D. Context and Significance

In my preparatory research, I found two projects featuring pixels to explain my definition of interaction. The beginning point is the project I really like – Cellular Automata and the Age of Chaos. However, from two aspects can this project be refined to enhance its accessibility to a wider audience. On one hand, we can input the parameters using means of physical computing instead of typing in numbers. On the other hand, the output is not so intuitive. More often than not, what users can get is just a bunch of pixels that is hard to interpret its artistic value. The biggest feature of our final project is accessibility so these two problems should be solved to allow the biggest audience to create their own artwork. Other ways to enhance our accessibility includes using bigger buttons/ knobs or even Leap Motion. In terms of interaction, this program has meet the following standards: it involves two actors, there are inputs through means of physical computing and visual outputs on the canvas. And it is usable to everyone. We hope by using this equipment. Those who did not have prior art training will also be able to find confidence of being an artist or even redefine his or her personal values. The bottom line for us is to make a paint brush on a white canvas but we will definitely find methods to integrate more aesthetic values for the sketches that our users create.