Recitation 8: Serial Communication by Ian (You Xu)

Exercise 1:

       The first task is very similar to our in-class exercise. Therefore, by applying the sample code provided, I build up the circuit and modified the code quickly to make it work. For the circle task, I transmit two values from the Arduino to Processing. Then, I use these two values as the x and y of the center of the ellipse. Background refresh is a little tricky here. I will explain later. For some reason, it seems that my analog read A0 pin does not work… it took me a while to find this problem.

       When it comes to drawing a line, it gives me the hint that “you will need to keep track of the previous x and y values.” Therefore, I recall the strategy that I often use when coding with Python. I try to declare two more variables to store the previous x and y value. Then, I can simply use the previous values and the new read values to draw a line. However, when I tried to run it, the canvas is completely blank. I soon realized that to make my previous drawing stay on the canvas, I should not reset the background every time. When I removed that line of code, it finally works.

During this exercise, the interaction happens between my “input” to adjust the potentiometer and the “output” graphs on the screen. This process enables me to CREATE a picture as I wish that it impact me to build constant communication between my thoughts and the drawing.

Exercise 2:

First, I read through the code in the file that enables me to transmit multiple values from Processing to Arduino. It is similar to the one that transmits values from Arduino to Processing.

Therefore, it is not difficult for me to understand the code. I decided to transmit three values in total: mouseX, mouseY, and whether the mouse is pressed. For the third one, I use “1” to indicate pressed, and “0” to indicate not pressed. I believe this would enable me to decode the information in Arduino easier.

I applied if statement to make the buzzer sounding while I click the mouse on the screen. The only problem I encountered is that I am not familiar with the function tone(). I checked the reference and got the basic idea that the variable determines the frequency of the tone, which makes it sounds differently. Then how could both the mouseX and mouseY input impact the sound? I figured out one way to simply add two values together to decide the frequency of the sound. It has the potential to be more creative if I could be more expert on the basic musical fundaments in the future.

The interaction process in this exercise is similar to exercise 1. Both of the exercises enable the user to CREATE something in their mind to develop and express the limitless imagination. The difference is the form of input and output. For exercise 2, it inputs the position of the mouse and outputs a tone.

Circuit:

Circuit

Code

Assistive Technology Week 3 Assignment by Ian (You Xu)

  • What are the main challenges in applying mainstream mobile technologies to meet the needs of people who have autism or other cognitive disabilities?

“Chapter 15” in the book Assistive Technology introduces a GPS system that applies the mainstream mobile technologies. However, although it is designed to make disables lives easier, it may not be accessible enough for disables to use like other technologies. According to “Chapter 15” in the book Assistive Technology, people with cognitive disabilities lack some essential skills to build a connection to the external world. They include but not limited to “perception, attention” (378). Sometimes, applying technology and using technology requires a certain level of cognitive skills. For instance, to apply GPS mainstream mobile technologies, users may need to understand the different functions, may need to perceive the outcome of the technology (what will happen, what information will be sent out), may need to memorize the instruction of using the technology, may need to orient themselves to get familiar with when and where to use the technology, etc. Therefore, one of the main challenges is to eliminate the obstacles for people who have autism or other cognitive disabilities to normally INTERACT with mainstream mobile technologies as other users. This requires the careful design in considering every lack of skill that cognitive disables have.

This could be another challenge since it requires more advanced technological development and funding to reach the goal. Developers may not have enough incentives to do so if it does not increase their income significantly.

  • List three benefits of making mainstream mobile technologies accessible for people with autism or other cognitive disabilities.
    • Increase the safety level in the complex society that is not friendly to them.
    • Be accessible to the services/resources that they are unable to get.
    • Enable them to live indecently in society without others’ care.
  • How can technologies help to overcome stigma and discrimination for people with autism or other cognitive disabilities?

I believe the stigma and discrimination come from the significant additional human and material resources that are applied to serve for the disables. Most of the time, accessible facilities and services require large space, “unnecessary” funding, and normal humans to operate. This results in the stereotype that disables are troublemakers in society. Applying technologies may enable disables to independently overcome the obstacles by taking full advantage of their own skills without requesting much others’ help. Therefore, they get the chance to live and behave in society like other normal humans. They could live equally in society to enjoy social lives without discriminations.

Final Project Proposal by Ian (You Xu) and Vivien Hao

Proposal 1: Our Planet

As from the research Vivien has done for the first stage, she has realized unsustainability/climate change is one of the most vital issues that the United Nations has addressed lately. Vivien would want to work on something that illustrates the outcomes of unsustainability. But in the project, we would also like to show how humans could participate in the process of lessening the harms of unsustainability. Ian is highly interested in doing something that requires collaborations among people. He has proposed an idea of encouraging four people to make some movements simultaneously for the project to function. If only one person participates in the process, the project will not provide an outcome. Thus encourages collaborations among people, even among strangers. We both are highly interested in each other’s ideas. And we have worked together to combine these two ideas into one doable thought. Then we have figured out that we would want to create a project that requires movements among several people to lessen the unsustainable global issue. For the movement part, we want to make it very specific. The participants need to physically use a shovel and plant trees to push back the time of the earth explosion. The faster they plant trees, the longer the earth would stay away from the explosion. However, if the participant chooses to work alone, then there is no way for him to get fast enough to beat the accumulated speed for several people. We would set the number in our codes to be ridiculously high to make sure that no individuals could accomplish this goal by himself. In this project, we want to raise people’s awareness of the harmful outcomes of unsustainability. However, we would like to highlight the positive effect that we can bring forth through the collaboration processes. Our targeted group of the audience would be everyone on this planet. Overall, the earth is everyone’s plant. We all have to take part in helping to solve the unsustainable issue.

References

https://climate.nasa.gov/causes/

Proposal 2: Ourscraft – Creating a World of Ours

       The insight comes from the research Ian did for teamLab Borderless Shanghai and Vivien’s research that emphasize on impact to individuals. Ian went to teamLab last Friday. It is AMAZING. Ian wrote about the notion of “create the world together” in the research. Ian fully experienced it that every action of human movement will make some impact on the whole room. It indeed makes the audience connected with others, including all the human and non-human factors. It is super funny, creative, communicative, and inclusive.

Therefore, we want to build something that can make the participants interact to create their world. Then, a game called “Minecraft” hints at us. The player can use the blocks of material that the game set to build anything they want in the virtue world. How about making a 2D but a multiple user version of that? Here, we also get the inspiration from the “Dance pad” in the arcade and a fun small game called “Tetris.” We think these can be combined to build as an excellent form of interaction. Also, according to Vivien’s research, we believe that the project could also function as a tool for people who have a problem in social relationships to try getting in touch with others non-verbally, which is a significant social impact.

       How it may work: Four participants required. They will stand on each side of the screen. The random piece will appear on the screen. By stepping the bottom in front of them, they can make the shape move towards them a little. They can shake hands with corresponding people to reshape the pattern. Once a piece is confirmed, all four people put their hands together, and the next piece will appear. Therefore, after ten rounds, they may have built an interesting pattern!

       This project will give the audience the opportunity to cooperate to express their creativity. Their minds should not be limited by the project. Random pieces and colors with free choice to reshape and position enable them to free to make their crazy ideas shown on the device. Also, they are connected non-verbally to each other as a way of enhancing cooperation. It should be an enjoyable and creative experience for both the players and the audience.

Proposal 3: Where does your trash go?

       Vivien researched bomb. As described, it addresses a significant social problem that is worth people to pay attention and think. Also, Ian researched the project in PSA which works in response to the environment. However, both of us believe that these projects have a very significant core value but lack the application of interaction. Deriving from that, we start to focus on the issues that are around ourselves and try to figure out an engaging, interactive way to demonstrate it to the general audience by using a technological device. The recent hot topic about the trash sorting policy in Shanghai attracts our attention.

       We researched the existing technology that addresses the issue of trash soring in Shanghai, including the automatic trash-bin and the mini program in WeChat that tells the public how to sort the garbage. (http://tech.163.com/19/0821/22/EN4TIL6H00097U7T.html). However, we think that both of them are practical tools for daily use that do not build a communication environment, nor do they significantly arouse the public’s awareness. Therefore, we are thinking about creating a simple device that impresses the audience to think about a more fundament question: why should I sort the trash?

       How it may work: the audience can put different sample card, which indicates different types of trash into the trash bin. On the screen, it is demonstrating the different ways of dealing with different types of trash. When one trash is thrown into one of the bin, the animation will show the audience how it may impact the environment around them. Multiple motors and lights may be used to create an authentic experience for the audience and leave them a deep impression.

       Since many people in Shanghai do not understand the purpose and the importance of sorting the trash, this project brings impact to the general public by showing them the process and the effect of recycling garbage. It builds the connection between people’s daily lives and the changing environment around them. It may motivate them to reflect on their relationship between their living environment and take action to make themselves live greener with the environment.

Recitation 7: Functions and Arrays by Ian (You Xu)

The tasks in this recitation are very similar to our class practice which draws multiple smiling faces.

(I apologize for not using screen recording. I only have videos on my phone. I will remember to record the screen next time.)

Step 1:

I came up with the idea of the logo of Microsoft. Instead of drawing four rectangles, I found an easier way. I can only draw one rectangle and then draw a horizontal and a vertical line with heavy stroke weight. The steps of defining a function and call a function are very similar to what I learned for Python. A difference I notice is that I have to always define the data type of a new variable in Java. I will show the photo together with Step 2.

Step 2 and Question 1:

In the “for loop,” it draws ten random logos at random places. When I put it inside setup(), it draws 10 logos in the beginning and remains still since the code in setup() will only be executed once. Therefore, it only draws once.

step 2_in setup

When I put it inside draw(), it is constantly drawing logos and does not stop. Since draw() is looping, by every time it executes, 10 logos will be drawn on the screen. Therefore, it is drawing all the time without stop.

Step 3 and Question 2:

In step 2, I use “display(random(100, 500), random(100, 500), random(50, 300), color(random(255), random(255), random(255))).” However, if I want every object to have a series of fixed random values, this would not work. This statement will give a series of new values every time it executes.

So, instead of defining hundreds of variables to solve this problem, creating an array to store these values in sequence would be a convenient way to add and refer the data. Since every value in the array is assigned an index, by using for loop to add and get the value, it can simplify thousands of hundreds of repeating lines of code to a few lines, which is more readable and saves memory.

Since my function has four attributions: x, y, size, and color, I create four arrays to store each of the sequences of values. Then I add random values to the arrays in setup() and call them by index in the draw().

Step 4:

At first, I add a random value every time when a logo is drawn. Every time the same logo is drawn, I give it a random speed to move. However, this random speed is changing every time when I create a random value in draw(). So, it looks like this.

After this, I realize that I should make this random speed fixed for every logo. Therefore, I figure out that I need also to create another array to store the speed values in a sequence. Then I add “speedX” and “speedY” arrays that work similarly to other arrays. I add the “if-else statement” to reverse the direction of the movement when it touches the edge. This is covered in our class.

Optional:

I try to make the graph stop when I put my mouse on it and start to move when I move the mouse away. I achieved this by adding another “if-else statement” in the draw() to detect the position of the mouse. If the mouse is on the graph,  then it does not add “speed” to the corresponding logo in this loop. If not, it will normally add “speed” to make the graphs moving. It works. And I find it interesting to interact with it!

The answer to Question 1 and Question 2 are included in the documentation above.

Code:

int numbers = 100;
float[] moveX = new float[numbers];
float[] moveY = new float[numbers];
float[] x = new float[numbers];
float[] y = new float[numbers];
float[] size = new float[numbers];
color[] c = new color[numbers];
void setup(){
  size(600, 600);
  background(255);
  for(int i = 0; i < numbers; i++){
    x[i] = random(100, 500);
    y[i] = random(100, 500);
    size[i] = random(100, 300);
    c[i] = color(random(255), random(255), random(255));
    moveX[i] = random(-5, 5);
    moveY[i] = random(-5, 5);
  }
  //for(int i = 0; i < 100; i++){
  //  display(random(100, 500), random(100, 500), random(50, 300), color(random(255), random(255), random(255)));
  //}
}

void draw(){
  background(255);
  for(int i = 0; i < numbers; i++){
    display(x[i], y[i], size[i], c[i]);
    if(x[i] > width || x[i] < 0){
      moveX[i] = -moveX[i];
    }
    if(y[i] > height || y[i] < 0){
      moveY[i] = -moveY[i];
    }
    if(mouseX > x[i] - size[i]/4 & mouseX < x[i] + size[i]/4 & mouseY > y[i] - size[i]/4 & mouseY < y[i] + size[i]/4){
    }else{
      x[i] += moveX[i];
      y[i] += moveY[i];
    }
  }
  //for(int i = 0; i < 100; i++){
  //  display(random(100, 500), random(100, 500), random(50, 300), color(random(255), random(255), random(255)));
  //}
}

void display(float x, float y, float size, color c) {
  //Use these parameters to create your graphics
  noStroke();
  fill(c);
  ellipse(x, y, size, size);
  fill(255);
  rect(x - size/4, y - size/4, size/2, size/2);

  fill(c);
  stroke(c);
  strokeWeight(size/25);
  line(x - size/4, y, x + size/4, y);
  line(x, y - size/4, x, y+ size/4);

}

Final Project Process: Preparatory Research and Analysis by Ian (You Xu)

Chronus exhibition

Overall speaking, I think the Chronus exhibition is a collection of a series of great artworks but lacks the application of the notion of interaction regarding the definition we discuss in class. Comparing to the art galleries that I visited before. The similarity is that both of them are a form of art that is aiming at revealing or hinting the audience for certain ideas or arguments about the real world. For example, the artwork that Rechender Raum – Autonomous System is made of wooden frameworks with many motors that work together in building or executing certain object that happens in the middle.

While observing it for a while, I thought it might be revealing the complexity of the collaborate computing work. However, my interpretation turns out to be wrong. According to the project description, “on one hand by turning the machine inside-out its function is completely transparent, on the other hand, a strict self-referentiality and ignorance to the viewer is realized” (Baecker). Therefore, it has something to do with demonstrating the process of computer architecture and pointing out the problem of it, which is an interesting critique. This reminds me that when I was visiting Van Gogh museum in Netherland this summer (I do not have a picture since it’s not allowed), I could also hardly understand the hidden conceptuality about the art project by staring at it but to take a look at the description and interpretation. Therefore, understanding these forms of observational artworks requires a certain knowledge of background and explanation.

I think that the biggest difference for the work in the Chronus exhibition is that they are moving objects with the topics of “technology,” which is unique while tradition artworks are static. Taking Van Gogh’s art pieces as a comparison, they are not moving and does not require electricity to perform. For the Chronus exhibition, they are constantly moving objects that need electricity to run in front of the audience. However, I think they are not successful interaction works, not even responsive. For most of the works, the audience’s behaviors will have no effect on the performance of the project. Therefore, they do not even give the basic response to the audience, which does not fit in the most basic idea of interaction that consists of “input, processing, output” (Igoe and O’Sullivan XX). The only project that responds to the audience is Artificial Intuition. As described, it is supposed to “block people as they pass” (Zhang). However, when a group of people was interacting with the project, it did not give a corresponding response to block the way. Therefore, it does not bring any impact on the audience but make the audience confused.

Artificial Intuition

RESEARCH PROJECTS

I researched three projects. They are “Emerging Curators Project 2019: The Process of Art: Tools at Work – ‘ideas = prototype = the work itself’” held in Power Station of Art, “Marco Kart,” and “TeamLab Borderless Shanghai.”

  • The Process of Art: Tools at Work – “ideas = prototype = the work itself” (No online description, refer to PSA for detail: free entry)

The project consists of a series of Arduino and motors that are constantly moving. At first, I did not understand the pattern and logic behind this project.

ideas = prototype = the work itself

After reading the description, I found out that it “detects the radiation level in the atmosphere” and “it continues shaking while receiving the radiation” until “the radiation intensity … reaches the level before the Anthropocene” (Guo). Therefore, it is also a form of art that has a conceptuality to lead the audience thinking about human intervention to the natural world. I would define it as a responsive project, which is not successful enough to be defined as interactive. In terms of interaction, it is better than the Chronus exhibition since it receives information from the environment and makes the adjustment to give the response. Therefore, it does have “input, processing, and output” (Igoe and O’Sullivan XX). However, communication is not reciprocal. The information only goes from the environment to the project; the project does not impact the environment. Therefore, it fails to build and develop communication with the environment, which is like “two actors alternately listen, think, and speak” (Crawford 5). It only acts but not for the purpose of communication and making an impact. Moreover, even though the project is responding to the environment change, the design of the project is still to be exhibited to the audience who are human beings. So, it does not interact with the real audience at all.

This Italian project is an assistive technology done by Hackability. This project is a car that can facilitate a kid who has a certain disability to walk normally. It will automatically adjust to support the user in different environments. I think it is interactive since it connects the outside environment, the machine itself, and the behavior of the user together. It takes input from the environment and the user, then after processing, gives instant feedback to the user in supporting the kid. This process will enforce the user to adjust his walking gesture and body position. Therefore, it is impacting the user and building a connection with the user through physical movement. Most importantly, it is very useful and easy to use intuitively, which is user-friendly.

  • teamLab Borderless Shanghai (https://borderless.team-lab.cn/shanghai/en/)

Since I am planning to go there on the coming Friday, I only did research based on their website at this time. However, I visited another project “Universe of Water Particles in the Tank” (https://www.teamlab.art/e/tankshanghai2019/) earlier this year. I believe teamLab makes excellent interactive work in both projects with similar design theories. Two of their designs are very attractive to me. They are “Co-create the World Together” (https://v.youku.com/v_show/id_XNDM1NjM1MjQ3Ng==.html?spm=a1z3jc.11711052.0.0&isextonly=1) and “Artworks Influence Each Other” (https://v.youku.com/v_show/id_XNDM1NTk2MzA5Ng==.html?spm=a1z3jc.11711052.0.0&isextonly=1). As teamLab explains “people wander through the world, exploring with intention, creating and discovering a new world with others” (teamLab). They take full use of the light, sensors, and technologies in the space. In teamLab, when touching, standing, sitting, gesturing at a certain place, it will influence the world that is around you. Therefore, the audience is making the world upon their behaviors and creativity. Also, the work itself is moving so that the audience and the work are communicating and creating the world together. Even between the audience, they are able to creatively cooperate together to make a unique world for themselves.

teamLab

For “Artworks Influence Each Other,” teamLab describes that “Artworks move out of the rooms freely, form connections and relationships with people, communicate with other works, influence and sometimes intermingle with each other” (teamLab). According to the video, different works will have an impact on each other and making something new and amazing. It breaks the border and barrier of communication. I believe this is a remarkable work that brings everything into interaction. Not only people and the technology, but also between people and between the technologies themselves. Everything is communicating and connected. It makes me feel  in the project as a dream world.

“INTERACTION”

Therefore, I am reflecting on the complex ideas about my evolution of understanding “interaction.” I would summarize it to different stages below.

  • Input, processing, output.

According to Igoe and O’Sullivan that “interaction: input, output, and processing” (Igoe and O’Sullivan), this is the basis of interactive work. But only achieve this is not interactive but responsive. We applied this to our group project – “doglar.” It takes input from the dog, processes the information, and gives the response to the dog. However, as I wrote in the blog post, it is not interactive enough since it fails to build a continuous communication environment.

  • Function: art, entertaining, useful

Based on the project that I researched and I encountered in this course, I summary some possible successful forms of interaction work. They mostly function as art: teamLab, classmates’ midterm project; entertainment: my midterm project, teamLab, or useful tools: Marco Cart. This is also the basis of an interactive project that is “functional” (Igoe and O’Sullivan).

  • Continuous impact on each other, communication, connection

During our class discussion, we talked about the difference between “interactive” and responsive. Therefore, I updated my understanding of “interaction” based on Crawford’s argument that “interactivity that has some blood in its veins” (Crawford). I wrote that “an interactive project should also build a constant communication environment with the user or between users and learn new information from it.” Therefore, by applying it to the midterm project, I made a successful fun project that builds the communication environment between players and audiences. I think this is what Crawford means “blood.”

  • Inclusiveness

This is the new idea that I was inspired by teamLab project. During the midterm project presentation, I received a comment from the professor that “not only the people that are directly interacting with it feel fun, but also people who are watching it feel fun as well.” Therefore, it goes beyond building communication between two nodes but involving in more people together. They are connected to a network. I think teamLab demonstrates this idea perfectly that everything, including people, technology, environment, all of them are connected to each other. The project is unbelievably inclusive to all, which in response to “create a new world with others.” (teamLab).

Works Cited

“Borderless World.” teamLab. https://borderless.team-lab.cn/shanghai/en/

“Marco Kart.” Hackability. https://space.hackability.it/en/project/0XUTQSgCkQptIfaOI54P.

Crawford. “What Exactly is Interactivity.” The Art of Interactive Design,  pp. 1-5.

Igoe and O’Sullivan. “Introduction.” Introduction to Physical Computing.