Posts

Posted on

Week 4 Assignments – Iris

Reflection of the field trip

        This field trip made me feel that these kids are not different from normal people, and even more cute. Their school just looks the same as my primary school, which is not far away from the special school that we visited and gave me a familiar feeling. Through all the facilities, installations, classrooms, and many other details, I can feel the society and teachers’ concern for them. For example, the subway model donated by the Shanghai subway company, the Haibei Supermarket in which the kids can buy toys with Haibei coins that they earned by themselves, etc. These all help those special kids to grow up better. When observing the physical therapy class of a cerebral palsy kid, I felt that the teacher was extremely patient and I can feel her love for the kid. Although they are always repeating several simple movement therapies, they didn’t express any boredom or impatience. The kid’s enthusiasm for playing football made me felt he is so cute since although he failed a lot of times when kicking by his left foot, he never gave up and tried again and again. The music class with the seventh-grade students also made me feel that there are almost no differences between them and normal kids, and they are even more positive and innocent. They talked about the Peking Opera in class, which is not an easy content and surprised me a lot. Just like other seventh-grade students, they also raise their hands actively in class to answer the teacher’s question. I can feel that they pay attention to the teacher and their enthusiasm for the class. I also saw that the students had very different personalities in one class and the teacher always tried to help those who are introverted to speak more and is very fair to every student. Through this field trip, I became more respectful to the teachers working at the special school who are extremely patient to these kids. I also felt these kids are so lovely that I have more motivation to try my best to design some assistive technology to help their life.

Project Idea

  1. Describe your partner as a person (not PwD)

My partner Lamb is a seventeen-year-old girl. She lives in the Putuo district in Shanghai, where is far away from our school. She just graduated from the special school in Putuo last year. She has a disability of controlling her body well because of hypoxia at birth, but her intelligence is no different from others. She participated in the Hackability in NYUSH before and is enthusiastic about this kind of activity. She told me that she posted a video of the Hackability on his Weibo. She is an optimistic and outgoing girl. Although she can’t control her hands well, she still tried to write her name down on the paper in last Tuesday’s workshop. Although she can’t speak fluently, she is very willing to talk to other people. When we asked some questions about what difficulties or problems they met in daily life, she interrupted her mother and told us that there is a gap between the subway and the ground so that it’s hard for the wheelchair to get on the subway.

2. What are their desires/challenges/difficulties?       

Their main difficulty is transportation. Lamb’s mother told us that their home is far away from the subway station so that she has to push the wheelchair to walk for more than ten minutes to get to the station. Although there are passages for the disables in the station, they still get trouble in getting on the subway because of the gap between the subway and the platform and the wheel of the wheelchair will sometimes be stuck in the gap. Her mother told us that in this regard, the subway stations in Hangzhou even did a better job then Shanghai. When the staff at the station see people in a wheelchair, they will provide a board and put it above the gap to enable the wheelchair to pass easily. Another problem is that the buses are almost inaccessible to them because of two reasons. Firstly, it’s hard for the wheelchair-user to get on to the bus because of the stairs. Secondly, in most buses, there is no space for a wheelchair, not to mention facilities that help to fix the wheelchair. They said that if they could take buses, life would be much easier because there are bus stations just near their home. The third problem is that Lamb wants to go out by herself alone with her electronic wheelchair, but she can’t if it’s the rainy day because only her right hand has force so that if she holds an umbrella, she can’t control the electronic wheelchair. 

3. What is your project?

 We plan to help them to solve the issue of holding an umbrella since there are lots of rainy days in Shanghai. The umbrella can also be used to shade the sun. Since it’s difficult for Lamb to open the umbrella by herself, we plan to use an automatic opening and closing umbrella so that she can use the umbrella by simply press the button. When not in usage, the umbrella will be clinging to the handle of the wheelchair. When Lamb needs to use it, she just needs to pull it up and press the button. We will use an umbrella like this in case she will get wet when closing the umbrella. Our challenge would be how to fix the umbrella in that position after she pulled it up tightly and to make sure its safety.  

Posted on

Week 4 Assistive Technology Assignment – Elysia

Reflection on the field trip

Last Thursday I went to the Shanghai Pudong Special School. We went there with a taxi since the distance was very near, it only took us 15 minutes to get there. The building is very beautiful, it has a lot of trees and a large garden. When we went to the inside of the school, the first thing that we saw was the metro station look-alike. These are used to train the children to use the metro so that when they actually go there they won’t panic. After that, she took us around the school to see the other rooms. She told us that from 1-3rd grade they will learn about basic knowledge about taking care of themselves, from 4-6th grade they will learn about living with the community, and in the end, they can go to professional school.  The field trip was very informative, we get to see one of the teachers taught her students how to sharpen their kinetic abilities. It was really heartwarming to see how everyone really cared about the children and they are really passionate about what they’re doing. Another thing that I was really impressed with is how they structure the learning and the classes. They have a shop that the children can buy things from coins that they got if they do something good. It would motivate the students to be more involved in their learning. It really inspires me to know more about the students and how we can help them by learning how they learn. 

Describe your partner as a person

Our partner’s name is Siyi, she is very kind and she loves to help people. Initially, I was very nervous because I am not fluent in Chinese. Thankfully, my teammates were kind enough to translate what she’s saying to me. I feel like she is very eager to learn new things and share her experiences. I’m looking forward to meeting her again!

What were their challenges

At first, she told us that she has trouble getting in and out of the bus because most of the bus does not have a ladder that would allow wheelchair to pass. So we were thinking about making her a wheelchair ladder. However, it found out that is was too dangerous if she decided to use it on her own because the distance between the bus and the ground is pretty steep. Secondly, she told us that she has problems using umbrellas. Her hand is too weak to hold the umbrella, and she can only use her right hand. Therefore, we are very interested to know more about her opinions and experience of using the umbrella.

What is your project

After a lot of brainstorming, we decided to make a helpful tool for her to be able to use for the umbrella. So it would be like two sticks that are attached to each other but are able to be rotated 90 degrees. We will attach the automatic umbrella in one of the stick, and then let our partner rotate the stick whenever she wants to use the umbrella. There are a couple of great things from this design. First, it would be safe for her to use it because the umbrella would be taller than her head so that it won’t hurt her when she opens it. Second, it would be portable and easy to carry around since it can be pulled back and stored when it is not used. We are still thinking about our final design, yet we have made our prototype. The video is attached below. 

Posted on

Week 11: Training Deepdream – Jinzhong

The assignment for this week is to play around DeepDream, a GAN network to transfer the style (or pattern) of an image, for example, this one:

WORK

There are 5 parameters that are customizable in the step of generation in total. These are:


octave_n = 2
octave_scale = 1.4
iter_n = 15
strength = 688
layer = "mixed4a"
 
And today I am going to talk about my research and understanding of these parameters, as well as my tests and experiments.
 

octave_n

– Test Range: [1, 2, 3, 4, 5, 6]

– Test Outcome:

Form the test we can see the parameter determines the depth of deep dream. The larger octave_n becomes, the deeper the render/transfer process will be. When it is set to 1, the picture is only slightly changed, the color of the sheep remains almost the same as its original source. However, when the parameter becomes larger, the contrast colors become heavier and the picture loses more features.

octave_scale

– Test Range: [0.5, 1, 1.5, 2, 2.5, 3]

– Test Outcome:

This parameter controls the scale of the deep dream. Although the contrast colors are not as heavier as the first parameter octave_n, the size of each transfer point scales and affect a larger area. So, we can see from the last picture, the intersections of several transfers are highlighted.

iter_n

– Test Range: [10, 15, 20, 25, 30, 35]

– Test Outcome:

This parameter controls the number of iteration of the deep dream. In other words, it determines the times of image processing. When the number is smaller, the output woulld be more similar to its original input. When the number becomes larger, the output would be more ‘deepdreamed’.

strength

– Test Range: [300, 400, 500, 600, 700, 800]

– Test Outcome:

The strength determines the scalar condition of each deep dream process. As we may see from the pictures above, the 6 transforms of the original picture are almost the same while only differ in the strength of colors (patterns). The higher strength outputs the sharper result.

layer

– Test Range: [“mixed3a”, “mixed3b”, “mixed4a”, “mixed4c”, “mixed5a”]

– Test Outcome:

The layer gives different patterns of the deep dream. It is also the pattern GAN used to train. So, each pattern would render different shape of DeepDream.

Posted on

Recitation 9: Media Controller

For the Arduino, I used two potentiometers to adjust an analog value.

For the code: I used the multipleValues code, sending from Arduino to Processing. 

For Processing:  I used Image function and the sensor values are used to change the position of the image.

  • code in Arduino:

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.println(sensor2);
delay(100);
}

  • code in Processing:

import processing.serial.*;

String myString = null;
Serial myPort;
PImage photo;

int NUM_OF_VALUES = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/

void setup() {
size(1300, 1300);
background(255);
setupSerial();
photo = loadImage(“1.jpg”);
}

void draw() {
updateSerial();
background(255);
printArray(sensorValues);
float x= sensorValues[0];
float y= sensorValues[1];
image(photo, x, y);
}

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);

myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
myString = null;

sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}

Posted on

Recitation 9 Media Controller Chloe Wang

In this recitation, we incorporated the use of images in the interaction between Processing and Arduino. In this week’s class, we learned how to put an image in processing and manipulate the image using values in Arduino. This week, I built my recitation project based on last week’s project of Etch a Sketch. First, I took the first step from last week’s Etch a sketch where I got an ellipse to move according to me turning two potentiometers. 
Here is what I had last week:
Then in my codes, I added codes for PImage:  photo = loadImage(“cat.jpg”);

and dragged a picture of my cat in Processing.  At first, my goal was to have a blurred picture of my cat on top of the clear picture, and wherever the ellipse moves, that part of the canvas would show the photo beneath rather than the blurred picture on top. I used the maskImage function in doing this. At the time, blurring a photo would take some time, so I added another photo that has a similar size with

  maskImage = loadImage("mask.jpg");
  photo.mask(maskImage);
  maskImage.resize(652, 484);

But the program still did work as I wished. I did not screenshot the result, but it would show a picture of my cat with some colors distorted. The ellipse on the canvas would leave a faded trial behind as it moves around. Later I realized that the mask would only be used when the mask image needs to contain only greyscale data. So with some help, I changed the maskImage to black with maskImage.set(x, y,255); and a rectangle would move around to reveal the photo beneath. With set function, it changes the color of those pixels of the selected area.  For the rectangle that is moving around, I (with help) used two for functions

  for (int y=200; y<300; y++) {
    for (int x=200; x<300; x++) {
      maskImage.set(x, y,255);
   }
  }
 and 
  int a=round(map(sensorValues[1],0,1023,0,height));
  int b=round(map(sensorValues[0],0,1023,0,width));
  
  for (int y=a; y<a+50; y++) {
    for (int x=b; x<b+50; x++) {
      maskImage.set(x, y,255);
    }
  }
in set up and draw. 
This way, when the rectangle moves around the black canvas as I turned the two potentiometers on Arduino, the rectangle would only reveal the part of the image it is at. Using int a and int b and the map function, I finally gave a boundary to where the ellipse can freely move. Now the ellipse would be on the canvas at all times.
From doing this exercise in class, I also reflected on my final project. Rather than having a blurred picture representing the things we don’t know about, a black screen and just a small part of visible image would be more representative of our actual experience with modern media that we don’t see most of the things we comment on. After reading Computer Visions for Artists and Designers, it provided me with another idea with “simple object tracking”. If this rectangle was generated by an analog source of flashlight, maybe I could step up the interactivity of my project to have people “looking for the truth in the dark” with an actual flashlight and black screen. 
 
Here is the complete code to the final version:
 
Here is a video of my final result:
Here is another video of my final result that could lead to another direction for my project:
 
Just in case you want to know, this is the picture I used in this exercise.
Image used in Recitation 9
Image used in Recitation 9

Reference:

Levin, G. and Collaborators (2012) ‘Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers’. FLONG. Available at: https://drive.google.com/file/d/1NpAO6atCGHfNgBcXrtkCrbLnzkg2aw48/view.