Author: yx1711

Posted on

Recitation 10: Workshops by You Xu (Ian)

During week’s recitation, we first reviewed the basics about mapping function. It refreshed my mind, and I also applied the map function in my later exercise.

Among all the three workshops, I chose to attend the media manipulation workshop since I think I will need to use some dynamic image and video in my final project, and I am already very familiar with serial communication and OOP during my previous computer sciences courses.

During the workshop, I learned many new built-in functions related to image, video, caption, and pixels. They are all listed in the slides. Therefore, I tried to apply them to a very simple exercise. I chose my favorite short online video on Vimeo: Watchtower of Turkey by Leonardo Dalessandri.

Since I learned how to start, pause, and loop the video, I tried to use my mouse input to control the playing of the video. Moreover, I also tried to move the video screen around the canvas by using my mouse as well. These functions are amazing. I applied the map function to the position of the mouse as well to make sure the video will not go outside the canvas.

I believe it will be helpful to my final project when coding the dynamic user interface.

Work Cited

Dalessandri, Leonardo. Watchtower of Turkey.

Code:

import processing.video.*;
Movie movie1;
void setup(){
  size(640, 640);
  movie1 = new Movie(this, "01.mp4");
  movie1.play();
}
void draw(){
  colorMode(HSB);
  float c = map(mouseX+mouseY, 0, 1280, 0, 255);
  background(c, 120, 255);
  if (movie1.available()){
    movie1.read();
  }
  if(mousePressed){
    movie1.pause();
  }else{
    movie1.play();
  }
  float x = map(mouseX, 0, 640, 0, 640-480);
  float y = map(mouseY, 0, 640, 0, 640-270);
  image(movie1, x, y);
}
Posted on

Assistive Technology Week 5 Assignment by You Xu (Ian)

a. Describe your partner
Throughout our conversation with him this time in class, we notice that he relies much on his mother about everything. When we are doing user testing of our prototype to him, he responded indirectly to his mother, and then his mother translates the sign language to us instead of trying to communicate directly to us. Most of the time, he is sitting and observing our communication with his mother as an observer. We then decide to have a conversation with him without his mother. In twenty minutes, we type in sentences on the computer and ask him to reply on paper. The conversation carries out very inefficient since it is hard for him to understand the grammar of a sentence. Therefore, we need to paraphrase a lot to confirm he understand our words. Also, he has limited capability of writing Chinese characters and link them into sentences. We spend much time guessing the meaning by applying body senses. After the conversation, we still think he has a strong inner desire to communicate and experience the outside world more. However, since his mother cares too much for him, it seems that he always wants his mother to help him overcome all the obstacles. Moreover, according to his mother, he likes to play computer games and watch online series at home when he is alone.

b. What are their desires/challenges/difficulties?
When we are communicating with him, we notice that since he is used to relying on his mother so much, he is not comfortable communicating with us directly. He also has little language sense, especially for reading and writing. In our WeChat chat, his mother records and describes how he uses the computer to play games and watch TV series. However, he can hardly control the keyboard using his left hand. We categorize the desires/challenges/difficulties for him in the following points:

1. Relies too much on his mother
2. Lack of language senses to communicate with others
3. Want to build a romantic relationship
4. Computer sometimes crashes
5. Difficult to click the keyboard with the left hand while playing the game

c. What is your project? 1 paragraph description + Pictures or Photos or Videos
We abandon our previous idea after the first prototype and user testing since it could not engage him to communicate independently with others, as he says. Therefore, by checking his needs listed above, we think addressing no.5 will be most helpful for him. We are thinking about to build a special keyboard for him to press the key using his feet instead of his hands. For most of the game his mother lists in the group chat, they require the use of four keys to move the figure. Therefore, we may design a five-key keyboard with four direction keys and one function key. We also want the user to customize the key function to make it applicable to more situations.

d. Development process. Include photos and video(s)
This is the prototype for our first idea last week. We abandon this one later.

Prototype 1

We have a conversation in the WeChat group. This is the website where he usually visits to play online games.

He Play Games

We then visit this website and plays for a while. Indeed, some games require using the left hand to press keys and the right hand to control the mouse.

game screenshot

We refer our idea to an interactive game Dance Dance Revolution. It demonstrates that feet are capable of controlling the game.

Dance Dance Revolution

Since we get the information and ideate it a little late, we have not started to prepare materials. However, we do want to test his performance while using the regular keyboard to play games on the computer, and his sensibility of using his feet press the bottom.

Posted on

Week 4 Assignment by Ian (You Xu)

Reflection of Field Trip:

We visited many of their club activities and one autism therapy class. By taking a look at their club activities, I think the school has already tried the best to maximize their personal capabilities and interests. Art activity leaves me the most impressive impression where some of the students are free drawing, and the others are doing some complex works. The coordinator in the school told us that they design the tasks differently for students with different skills and capabilities. I think it best fulfills different students’ needs by customizing the teaching plan for them. However, we got to know that the school has limited human resources. When we are watching the autism therapy class, even though there were only two children in the class, we still think the difference between the two children makes the class less efficient. It would be better to treat one child at one time. I think the iPad app that the teacher is helpful to different materials for different levels of learners. However, it focuses on how to reduce the burden of the teacher. I think it would better to design a technology that is customized to the learner. Therefore, it can minimize the effect of the recognition gap between peer learners. I think Young has a good idea for designing something to attract students’ intention so that they can stay more focused.

club

Moreover, while listening to the autism therapy class, I was also observing the classroom settings. I noticed that the school might intend to make a warm and safe room setting for students. However, it is not successful enough. Inevitably, some facilities that are out of line. Then how about change a perspective to design something applicable to students themselves to make them more comfortable in an unfamiliar environment? I do not have a specific idea, but I think such a design will be helpful for students especially with autism.

classroom setting

Describe your project idea

Describe your partner as a person

My partner is very quiet. When I first took a glimpse at him, I did not identify anything abnormal about him. He walks a little unbalanced, but that does not make him look strange. We started to ask questions and introduces ourselves to him. He seemed to have understood our words but only replies in limited voice and hand gestures. None of us understood it. Therefore, we found a pen and a piece of paper writing down our conversation. He writes comparatively slow, but the characters are clear to understand. After that, we had a long conversation with his mother. While we are chatting with his mother, I noticed that he is quietly listening and paying attention to our conversation. Later, his mother posted some questions on the paper, and he started writing the response. He even actively started writing a sentence to us while he is a little ignored by us. Therefore, I believed that he has a strong desire to join our conversation. However, the way to communicate is a big challenge for him.

What are their desires/challenges/difficulties?

He has many challenges. Throughout our conversation with his mother, we noticed that he never goes out by himself but also with his mother because of the potential dangers on the road. Since he lacks a certain degree of the ability to control the left side of his body, it is very hard for him to keep the balance. Also, he expressed his desire to have a romantic relationship with his mother two years ago, but his situation is not socially accepted. Furthermore, his mother is also concerned about his safety at home since he might open the room door for strangers. However, after all, we nearly forget that our target audience should be himself instead of his mother. We were chatting with his mother all the time instead of him. We then realized the most significant difficulty for him is the inconvenient environment of his communication with others. We could hardly communicate with him until his mother came.

What is your project?

We spent some time finding a pen and paper to write down words in communication. Also, we did not aware that he is unable to hear and speak at first. Therefore, we came up with an idea to address this problem. So, the purpose of the device should enable him to communicate with others without any barriers in any situation. We still think writing and reading is the most effective, stable, and safe way for his daily communication. We find the existing writing pad online. It is convenient to carry, write, and swipe the previous writing. However, it is not initially designed for disabled people. Therefore, we want to build a device that makes the writing pad accessible to him. We need to figure out the following details in design by prototype and test.

writing pad

  1. Notify others about his lack of ability in hearing and speaking. We may use the sound notification.
  2. Attached the writing pad to his body but should not block his daily movement and make him uncomfortable. We may design a special bracket for him.
  3. Avoid two people writing in opposite directions or make their writing readable to each other. We may want to make the device rotatable.
  4. Make sure the pen will not be lost while walking. We may attach the pen to the device with wires.
Posted on

Recitation 9: Media Controller by Ian (You Xu)

During this recitation, I intend to make a form of distortion of the capture of the live camera. Therefore, I referred to the caption sample code, processing website, and hooked up a potentiometer to achieve this goal.

  1. Green tint and low frame rate

When I was trying to run the sample code of capture, the screen has a green tint and refreshes very slow.

Green tint

Since the same code usually works in others’ devices, I was very confused. I thought it might because my computer – Surface Pro – has multiple cameras that work for different functions. I referred to the Processing website for capture: https://processing.org/reference/libraries/video/Capture.html and displayed all the available cameras for me to choose.

cameras

However, I tried nearly all of them, and no one worked. I looked it up online and found there is an “issue” under Processing’s Github repository https://github.com/processing/processing-video/issues/72. Many people who are using Surface with Windows 10 are encountering the same problem, and they do not have the solution to it. They believe it is the software problem of Windows 10 on Surface. Therefore, temporarily, I cannot fix it.

  1. Moving

By applying the sample code of capture to the serial communication sample code, I easily get the data from the potentiometer to the processing and use it to adjust the position of the capture. I use this step as a test of applying video function to serial communication. Again, one problem I found is that I need to be careful about the background refresh in the draw loop. This time, I do not want the previous images to leave on the screen, so I decided to draw the background in every loop.

  1. Scaling

I checked the capture mirror sample code and found out that it uses a “scale” function to make it the mirror effect. It gives me the insight to use the input from the potentiometer to control the scaling index. At first, I simply add “scale(sensorValues[0]);” to the code. However, I found out that the data from the potentiometer is ranging from 0 to 1023. It may scale the capture over 100 times bigger, which is too big.

Too big Scaling

Then, I realized I need to map it into a reasonable range first. I choose 0 to 2 here where 0 to 1 is shrinking, and 1 to 2 is amplification.

  1. Translate

However, this is still not what I expected. I want the capture always in the middle of the screen. Then I recalled another function “translate” that I used before to define that new origin. This required me to do math calculations. And I figure it out by geometric operation.

  1. Two dimensions

It works well. So, the final step is to hook up another potentiometer and make one of them scaling the x-axis and another to scale the y-axis.

2 potentiometer

Reflection

In the article “Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers” (Levin). it describes “new practitioners with an abundance of new application ideas, and the incorporation of computer vision techniques into the design vocabularies of novel artworks, games, home automation systems, and other areas.” I found two projects that use the capture function to build an authentic, interactive experience with users where they can see themselves. They are Cheese and LIMBOTIME. Therefore, I think to capture in the interactive project can increase the inclusiveness for the audience to engage their intention and communication with the project, which is correspondent to my definition of interaction that addresses the constant impact and inclusive communication environment. My work in this recitation would entertain the audience that they can see the distorted face in whatever dimension by interacting with the potentiometer. The only pity is the green tint bug that I am unable to fix.

Code: https://github.com/xuyou1999/InterLab_Fall_19/tree/master/Recitation_9_Code

Works Cited

Levin, Golan. “Computer vision for artists and designers: pedagogic tools and techniques for novice programmers.” AI & SOCIETY 20.4 (2006): 462-482.

Posted on

Final Project Process: Essay by Ian (You Xu)

Project Title: Our Planet

PROJECT STATEMENT OF PURPOSE

The insight of this project comes after the discussion with Vivien Hao about our research and our focuses on the definition of “interaction.” Vivien researches a project named bomb that addresses energy safety issues. It aims at revealing the truth of reality and arouse the public’s awareness. Therefore, we believe “impact” is very important for a project of interaction to be meaningful and significant. Then, we together visited the website of the United Nations and researched “Sustainable Development Goals” to find some insight for us to address a social issue. We notice that the description of “GOAL 13: CLIMATE ACTION” says that “Climate change is a global challenge that affects everyone, everywhere” (United Nations). However, apparently, not everyone is familiar and aware of the happening climate change. Therefore, we want to build something interactive that addresses this issue. Since it has to be an interactive project, according to my previous research about the less successful project in PSA and successful project in teamLab Borderless Shanghai where everything is creative and collaborate, I believe that an open communication environment that engages the audience to have continuous participation is necessary. We soon recalled a smartphone app “Forest.” In this app, when staying away from your phone for a certain amount of time, the user will grow up a virtual tree on the phone. We, therefore, think that the image of tree growth is straightforward and friendly to the audience to show its indication of the environment. By paying effort to plant the tree and save the planet, use is getting constant feedback and engagement to reinforce their behaviors. It may even have a further impact on their real daily lives. Besides, we also visited the NASA website (https://climate.nasa.gov/) to look for some climate change facts as a reference to make sure it is scientifically reasonable. In all, this project addresses the general public’s awareness of the urgent climate change problem around the globe. It intends to encourage uses to minimize the carbon emission and pay effort to make a greener planet.

PROJECT PLAN

       The project will have a projection screen on the middle, four positions with shovels, and masks. When the user puts on the mask, once breathe, it consumes oxygen and produces carbon dioxide. When planting the tree, vice versa, and the tree grows larger. The planet will boom if it lacks oxygen and filled with carbon dioxide. The sketch of it is as follows.

sketch_final

Since our audience is the general public, they could be anyone. Therefore, we will have user testing among ourselves and our friends throughout the process of project development to empathize with them. We have some specific plans as follows.

  1. We need to be sure about the detail of the design by prototyping and test it with the users. The user test should be in every step. The details include: whether or not to include a mask in our project, what types of sensors we decide to use, what material we will use, how will the user interface look like; sound, start and end, layout, and position. We will test with the comfortability and the cleanness of the mask. We will finish the discussion for all the details and have a blueprint by Sunday, Nov. 24.
  2. We need to build the circuit and the program that are both scientific and humanized. This will be practical work. We may encounter many problems. We will refer to online references and asking for help. Once when we finish the user interface, we will have someone to test playing with it to make sure it makes sense to the audience about the expression of our initial design. Finish before Dec. 1.
  3. We need to get the materials and build up the basic functional prototype. This is practical work, as well. We only need to make sure it can support our project to work. It can be done by Dec. 1, as well.
  4. We need to test with the accuracy of counting the movement of the shovel and breathe. Also bug fixing work. This process needs constant revising and test among ourselves and our users. We will list out all the bugs the project has and fix them one by one until it works exactly what it is supposed to be. It should be done by Dec. 6.
  5. Beatifying the project and adding something to improve user experience and accessibility. This step we will need to decorate everything, make the wires invisible, test the projection. After we think it works, we will invite many groups of people to interact with it. By hearing their feedback, we can make some little changes to make it more user-friendly and accessible for as many people as possible. It should be done by Dec. 8.

CONTEXT AND SIGNIFICANCE

I researched the project of teamLab. It connects every human and non-human factor together. Therefore, I believe it builds a reciprocal communication and relationship between human and human, human and environment, environment and environment. Moreover, it inspires and impacts the user continually so that communication is everlasting and inclusive to them. The keyword – communication – in my reshaped definition of “interaction” hints me to design something that is encouraging and inspiring to the audience. I hope to build an immersive user interface to engage their communication while getting constant feedback. The movement of the shovel and the changing scene on the screen fulfills this requirement. Users will start to try shovel harder to reach the goal in the authentic settings that are relative to them. I also researched an assistive technology Marco Cart. As I described before, this project is functioning as a tool that addresses the real need and has an impact on society. Also, cooperating with Vivien’s research and understanding about “interaction,” we think a significant impact is also a considerable surplus to the significance of an interactive project. Combing “communication” and “social impact” together, this project uniquely brings a significant social issue into the reciprocal personal conversation with the general public in an interactive way. Even though the form of our project may look similar to some commercial smartphone apps like “Forest” and “AntForest” in Alipay, the theory, logic, frame, design, outcome, purpose, and significance are different. We address more on impact and communication while they might be designed to trick consumers for more economic profits. Our project may have further implications on a personal level. It may bring more people’s attention to the issue of climate change. In this way, some social movement projects and activities about environment protection may be more meaningful and more influential to the general public. By cooperation together, every single person has the power to devote a remarkable effort to make our planet greener and to make our livings more sustainable.