Recitation 08: Serial Communication by Jiayi Liang(Mary)

During this recitation, we are told to make a processing Etch A Sketch and a musical instrument with arduino. The circuit is quite simple.  What we should do is only connecting two potentiometers. However, for me, writing the code is very challenging. Using the examples given, we try to send messages from arduino to processing and from processing to arduino.

Exercise 1

The first step is to use the potentiometers to control the position of a circle.  I succeeded at once, however, there is one problem that the circle will sometimes be out of the canvas.

Arduino:

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A2);

//int mappedsensor1 = map(sensor1,0,1023,0,255);
//int mappedsensor2 = map(sensor2,0,1023,0,255);

// keep this format
Serial.print(sensor1);
Serial.print(ā€œ,ā€); // put comma between sensor values
Serial.print(sensor2);

Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Processing:

 

So in the second step, the fellow tell me that maybe I can use the map() to adjust the position of the image. At the beginning I am stuck in “how to draw a line”, but soon after that I realize I just have to let the beginning point to be the previous sensorValue and the endpoint to be the current sensorValue.

Processing:

 

Exercise 2

In this exercise, we use mouse to control the sound of a buzzer. We use the sample code for multiple values from processing to arduino. The mouseX is the frequency of the tune and the mouseY is the duration.

Processing:

// IMA NYU Shanghai
// Interaction Lab


/**
 * This example is to send multiple values from Processing to Arduino.
 * You can find the arduino example file in the same folder which works with this Processing file.
 * Please note that the echoSerialData function asks Arduino to send the data saved in the values array
 * to check if it is receiving the correct bytes.
 **/


import processing.serial.*;

int NUM_OF_VALUES = 2;  /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/


Serial myPort;
String myString;

// This is the array of values you might want to send to Arduino.
int values[] = new int[NUM_OF_VALUES];

void setup() {
  size(500, 500);
  background(0);
 
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 9], 9600);
  // check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index of the port

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
}


void draw() {
  background(0);
    
  values[0]=mouseX;
  values[1]=mouseY;

  // changes the values
  // sends the values to Arduino.
  sendSerialData();

  // This causess the communication to become slow and unstable.
  // You might want to comment this out when everything is ready.
  // The parameter 200 is the frequency of echoing. 
  // The higher this number, the slower the program will be
  // but the higher this number, the more stable it will be.
  echoSerialData(200);
}

void sendSerialData() {
  String data = "";
  for (int i=0; i<values.length; i++) {
    data += values[i];
    //if i is less than the index number of the last element in the values array
    if (i < values.length-1) {
      data += ","; // add splitter character "," between each values element
    } 
    //if it is the last element in the values array
    else {
      data += "n"; // add the end of data character "n"
    }
  }
  //write to Arduino
  myPort.write(data);
}


void echoSerialData(int frequency) {
  //write character 'e' at the given frequency
  //to request Arduino to send back the values array
  if (frameCount % frequency == 0) myPort.write('e');

  String incomingBytes = "";
  while (myPort.available() > 0) {
    //add on all the characters received from the Arduino to the incomingBytes string
    incomingBytes += char(myPort.read());
  }
  //print what Arduino sent back to Processing
  print( incomingBytes );
}

Arduino:
int tempValue = 0;
int valueIndex = 0;

/* This is the array of values storing the data from Processing. */
int values[NUM_OF_VALUES];

void setup() {
Serial.begin(9600);
pinMode(11, OUTPUT);

}

void loop() {
getSerialData();

tone(11, values[0],values[1]*10);

// add your code here
// use elements in the values array
// values[0] // values[1] }

//recieve serial data from Processing
void getSerialData() {
if (Serial.available()) {
char c = Serial.read();
//switch ā€“ case checks the value of the variable in the switch function
//in this case, the char c, then runs one of the cases that fit the value of the variable
//for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase
switch (c) {
//if the char c from Processing is a number between 0 and 9
case ā€˜0ā€™ā€¦ā€™9ā€²:
//save the value of char c to tempValue
//but simultaneously rearrange the existing values saved in tempValue
//for the digits received through char c to remain coherent
//if this does not make sense and would like to know more, send an email to me!
tempValue = tempValue * 10 + c ā€“ ā€˜0ā€™;
break;
//if the char c from Processing is a comma
//indicating that the following values of char c is for the next element in the values array
case ā€˜,ā€™:
values[valueIndex] = tempValue;
//reset tempValue value
tempValue = 0;
//increment valuesIndex by 1
valueIndex++;
break;
//if the char c from Processing is character ā€˜nā€™
//which signals that it is the end of data
case ā€˜nā€™:
//save the tempValue
//this will b the last element in the values array
values[valueIndex] = tempValue;
//reset tempValue and valueIndex values
//to clear out the values array for the next round of readings from Processing
tempValue = 0;
valueIndex = 0;
break;
//if the char c from Processing is character ā€˜eā€™
//it is signalling for the Arduino to send Processing the elements saved in the values array
//this case is triggered and processed by the echoSerialData function in the Processing sketch
case ā€˜eā€™: // to echo
for (int i = 0; i < NUM_OF_VALUES; i++) {
Serial.print(values[i]);
if (i < NUM_OF_VALUES ā€“ 1) {
Serial.print(ā€˜,ā€™);
}
else {
Serial.println();
}
}
break;
}
}
}

Reflect on the interaction involved for each step

  1. In exercise one, there is interaction because the position of the circle or the line is changing according to the changes on the potentiometers. The first potentiometer determines the x-axis and the second determines the y-axis.
  2. In exercise two, there is interaction because the sound made by the buzzer is changing according to the mouse position on the canvas. If the mouseX turns big, the frequency will be higher, which means the tone will be higher. If the mouseY turns big, the duration will be longer, which means the length of the time of the sound will be longer.

Final Project: Project Proposal – Jiayi Liang(Mary)

 1 We Are Animals 

This project is a combination of a mirror that translate the users’ image into an animal figure and an illustrated handbook that collects all the images created for the users. When we look at the mirror in the first minutes, the mirror will reflect the real human face. However, after a few seconds, the image will gradually turn into an animal face. The animal figure is customized based on the users features such as gender, skin color and hair( even the body size). It’s a gradual process and the animal figure will act according to the users which make the audience understand that the animal in the mirror stands for themselves. Spontaneously, there will be a printer that print the image on a book with text descriptions. This book is being added with more and more images as audience keeping on using the mirror. The book is a record  of the various interesting images in the society.

I got this idea because recently I am really in favor of a Japanese Cartoon called <Beastars>. In this cartoon, all the main characters are having a manlike body while with an animal-like head. The story mainly tells about the conflicts between the herbivores and carnivoresā€“carnivores are not allowed to eat meat and being discriminated by their strength that may cause danger and herbivores are suffering from the threat from the carnivores and also being discriminated by their weakness. I think the cartoon is reflecting something happening in real life. The animals may stand for different social groups in society, such as male, female, the black, the white etc., which is a very common technique that is used in books, comics and films. For example, <Zootopia>,<Animal Farm> and<Maus>.

I find something related to my idea is one of the graduation project in our school called ā€œBook of Privilegesā€.

The project reflect peopleā€™s individual privileges into different colors. And what I want to do is to reflect someoneā€™s personal image into animals.

This kind of personification is a very useful technique in art creations, which uses a fun way to reflect things happening among people. Nowadays, many people are confusing with one simple but sophisticated question: who I am.  I think my project “We are animals” can help the confused users to have a clearer understanding of their personal identity that symbolizes their role in the big society by showing them the animalized image. The challenge in this project might be the facial recognition technology.

2 Happiness Vending Machine

The project will be a Vending Machine whose products are exhibited in a screen, and what it sells are not the real food or drink but the “happiness” from various people. The users can use one coin to buy one short paragraphs about the happy things happened in everyday life from a person.  For example, “I am so happy because the sun is very shiny today.” Or the text can be changed into only a picture. For example, a cute cat photo that makes one person feel joyful. The buyer can also upload their “happiness” into the vending machine and sell them to other users.

I have read a book written by Haruki Murakami. In the book, he raise a concept called “a little happiness in hand”. I think it is what I want to sell in my project. I get the inspiration from a Japanese novel called <Grief Grocery Store>. In this book, there is a grocery store. If you send letter to it to talk about your grief, the owner will write some advice and encouraging words as a response. Since peopleā€™s grief can be shared, how about sharing peopleā€˜s happiness? This is the reason why I make this proposal.

Nowadays, people are losing the ability of being happy. They think life is too hard, and they lose their hope towards life. I think the Happiness Vending Machine shares an optimistic attitude towards life to those people. There are countless things in the world are worth happy for, so why are you  keeping so sad every day? The challenge here may be that it is different for every individual in what they will be happy for. 

3 The Flowing Tao

The project shows an  image with black and white. The pattern on the picture will change accordingly to the movement and the voice made by the audience in a flowing way. The image will change from time to time, but when everything is over, it will change into its original pattern — a Taiji figure.

I get the inspiration when I was doing research on Taoism for my EAP project. Taoism raises a very interesting concept that Yin and Yang are contrast to each other, but at the same time, they are complementary, interconnected, and interdependent in the natural world. 

I want to use this project to embody the concept Yin and Yang. The pattern may change from time to time, but there is always 50% black and 50% white in the picture, which reflects a kind of harmony. This project aims at delivering the spirit of the Chinese traditional culture Taoism to those who don’t quite understand it. The challenge may be that the concept is too abstract for those who haven’t heard Taoism before to understand.

Recitation 7: Functions and Arrays — Jiayi Liang (Mary)

In this week’s recitation, we practice on how to use Functions and Arrays to create a lot of similar images at the same time. 

Step 1:

I design a graphic by my own.  It’s a heart with an “X” on it (a broken heart). It is a combination of “ellipse”ļ¼Œā€œlineā€ļ¼Œand a triangle made by “vertex”. A big circle forms the main body, two circles are the top part of the heart, the triangle forms the bottom part and two lines form an “X”.

My codeļ¼š

size(1000,1000);
background(255);
fill(0);
ellipse(300,300,100,100);
noStroke();
fill(#F51E1E);
ellipse(280,290,40,40);
ellipse(320,290,40,40);
beginShape();
vertex(262, 300);
vertex(300, 285);
vertex(338, 300);
vertex(300,340);
endShape();
stroke(0);
strokeWeight(8);
line(290,290,310,310);
line(310,290,290,310);

Step 2ļ¼†3ļ¼š

Next, to display 100 instances of my graphic in a variety of positions , I use the for loop. I combine step 2 and 3 together because I think it is easier to use arrays to display shapes of different positions and colors. I first define arrays of x, y and c as the position numbers and the color numbers,then I write the for loop to create 100 images. I first write the for loop into the draw(), and it works well. Then I move it to the setup(). A bad thing happens that my graphs don’t move.

My code:

int[] x = new int[100];
int[] y = new int[100];
color[] c= new color[100];
int[] speedX = new int[100];
int[] speedY = new int[100];

void setup(){
size(1000,1000);
colorMode(HSB,100);
for (int i=0;i<100;i++){
x[i]=int(random(1000));
y[i]=int(random(1000));
c[i]=color(int(random(0,100)),50,100);
}
}
void draw(){
background(0,0,100);
for(int i=0; i<100; i++) {
display(x[i],y[i],c[i]);
}
}
void display(float x,float y, color c) {

fill(0);
ellipse(x,y,100,100);
noStroke();
fill(c);
ellipse(x-20,y-10,40,40);
ellipse(x+20,y-10,40,40);
beginShape();
vertex(x-38, y);
vertex(x, y-15);
vertex(x+38, y);
vertex(x, y+40);
endShape();
stroke(0);
strokeWeight(8);
line(x-10,y-10,x+10,y+10);
line(x+10,y-10,x-10,y+10);

}

Step 4:

Then, to make it funnier, I add some movements like bouncing into the code. I add two new concepts– speedX and speedY to control the speed, which is learned form previous classes.

My code:

int[] x = new int[100];
int[] y = new int[100];
color[] c= new color[100];
int[] speedX = new int[100];
int[] speedY = new int[100];

void setup(){
size(1000,1000);
colorMode(HSB,100);
for (int i=0;i<100;i++){
x[i]=int(random(1000));
y[i]=int(random(1000));
c[i]=color(int(random(0,100)),50,100);
speedX[i]=int(random(-5,5));
speedY[i]=int(random(-5,5));
}
}
void draw(){
background(0,0,100);
for(int i=0; i<100; i++) {
display(x[i],y[i],c[i]);
x[i] = x[i] + speedX[i];
y[i] = y[i] + speedY[i];

if (x[i] > width || x[i]< 0) {
speedX[i] = -speedX[i];
}
if (y[i] > height || y[i]< 0) {
speedY[i] = -speedY[i];
}

}
}
void display(float x,float y, color c) {

fill(0);
ellipse(x,y,100,100);
noStroke();
fill(c);
ellipse(x-20,y-10,40,40);
ellipse(x+20,y-10,40,40);
beginShape();
vertex(x-38, y);
vertex(x, y-15);
vertex(x+38, y);
vertex(x, y+40);
endShape();
stroke(0);
strokeWeight(8);
line(x-10,y-10,x+10,y+10);
line(x+10,y-10,x-10,y+10);

}

Optional:

Finally, to make the process more interactive, I add some interactions involving keypress. If I press the keyboard, the moving speed will become faster.

Question 1:

In your own words, please explain the difference between having your for loop from Step 2 in setup() as opposed to in draw().

A: Setup() means “happens only once”.  Draw() means “loop”. So I guess the reason why my code works in draw() instead of “setup()” is that the movement of the graphs is the result of looping the code again and again. One loop can lead to only a movement that is really small, and many loops together result in the big movement. If I put the for loop in setup(), it only runs one loop, and then it stops, so what I see is only a motionless image.

Question 2:

What is the benefit of using arrays?  How might you use arrays in a potential project?

A: Arrays help me deal with a huge amount of variations. It is ok for me to write 3 variations without arrays. However, when it comes to 100 graphs, arrays is really necessary that I can store the 100 numbers at once. It can not only decrease the coding work load, but also clearly classify the different variations I need. The arrays also work as an index.

In the future, if I am doing a project that requires a huge amount of data, I will use arrays. For example, if I produce a game that contains 30 characters, and ach character has different scores on “intelligence”,”power” and “speed” etc,  I can use arrays to clearly customize the various characters.

Preparatory Research and Analysis by Jiayi Liang (Mary)

The visit to the Chronus Exhibition is a delightful experience which shows me various creative interactive designs and inspires me a lot. The exhibition is located in M50 where holds a lot of various art exhibitions, and I also get the chance to visit many more exhibitions besides the Chronus Exhibition. However, it is the experience of seeing other non-technology exhibits that let me realize the charm in the technology-based art pieces in the Chronus Exhibition. Compared to the traditional exhibitions that only display some pictures or sculptures to the audience, technology-based exhibits allow the viewers to make interactions with the exhibits. These exhibits are dynamic, rather than motionless, which gives art a brand new “moving” dimension that is more vivid and lively. Technology is realizing something more complex than images can do, and thus, express more information to the audience. And the completeness of some of the art pieces cannot exist without the intervention of the viewers. It means that the art pieces include the viewersā€™ interaction as part of its body. This kind of enhanced involvement increases viewers’ sense of participation, makes them more immersed in the visit and better understand what the air pieces want to deliver to them.

For example, these paintings are good, but I will simply miss them if I pass so quickly because the painting is just so “quiet”. It is not calling for my attention.

However, things go different when I see the first exhibit called “Artificial Intuition” in the the Chronus Exhibition. “Artificial Intuition” is a kinetic sculpture that interacts with human motions.  When we see these tentacles, we are curious about what will happen next. So we stop, we touch them, we observe how it works, and we get to feel the vitality inside the machines, which is exactly what the artist want to show to us — this kind of presence of living created by lifeless machines. I guess the working principle of the work is similar to what we learn in the class. The sculpture uses a kind of distance sensor to detect the existence of the audience, and then the computer controls the tentacles to get close to audience and interact with them.

Another interesting work called Beholding the Big Bang, we are just curious to see how fast will the gears rotate. In this case we are not part of the artwork, but the dynamic process just attract our attention. The last gear will take 13.82 billions years to rotate once, which is the estimated age of the universe, which indicates the immutability of the universe. I cannot imagine how this idea will be expressed in pictures without highlighting the stillness of the last gear by the help of technology.

After the field trip, I start to research more to find inspirations for my final project. In the midterm project, my biggest problems are the usefulness of the project and the lack of interaction. It is a little bit vague in the difference between interaction and responses. For the usefulness part, I start to consider that I used to simply define usefulness as ā€œ getting people’s life easier”, and I guess I am a little bit too pragmatistic. This kind of interpretation makes my project work harder because there are countless companies that are now designing products to fulfill people’s needs, and actually, as a student, I am not that capable to create a product that haven’t appeared in the market. I reckon, usefulness can be understood in other ways. How will my work change people’s mind? Can my work reflect something happened in the society? Can my work amuse people or cure their broken heart? Can the project express my ideas to others? Can they be art? These are all “useful meanings” that can exist in an interactive project. So, I start to find some interactive projects which are aesthetic rather than practical, and I want to see these projects focusing on engaging the audience into the process of producing to create a successful interactive experience.

The first project I find interesting is “Seeds of Hope”. It is a sort of interactive art installation. Each picture of the seed represents an audience. The art work wants to encourage immigrants to  share their wishes or dreams. In this case, new immigrants and those whose families have been here will be interwoven into one integrated whole. It is interesting that the project doesn’t use any digital technologies. But it is not just a simple display of a list of pictures as I have criticized before, it is interactive. The installation itself is created by the audience. When a person views the art work, he or she adds one more seed to it, which is an input, and the whole project is refreshed (processed) by the action. Then the project shows a new image to the observers as a response. Audience are involved in the creating process, and the project helps audience deliver their hopes or wishes to more audience, which makes the experience successful. I can see this project as a symbol of immigrants’ collective hope, which gives powers to those who are leaving their home and working hard for their future.

The second project I find is one of the graduation project in our school called ā€œBook of Privilegesā€ by Jianghao Hu. “Book of Privileges” maps individual privilege identities into different color patterns. The audience’s social identities (gender, race, etc.) was interpreted into a distinctive color pattern, which was then printed on semi-transparent paper and bound in a book by woodblock printing. The interactive experience sheds light on how interactive projects help interpret social identities. Recently, I learn from my GPS course that media has the function to reinforce social images. Hu’s work help us have a clearer understanding of individual differences, and intuitively conceive the diversity of the society by interacting with the book. What is very successful in the project is that it is the participation of the audience that reinforces the diversity.

However, there is one project that I am not so satisfied with.  Itā€˜s the “Crystal”. Although the installation will change its appearance when the audience come close., the audience are not changing the the project itself, which makes the process look much more like a response rather than interaction. From my perspective, the interactive experience is not so successful.

In my first group project, I state that interaction is a dynamic conversation between two actors or more, and interaction requires the process of input, process and output. After my research and personal experience on my midterm project, I find that the significant part of interaction is the participation of both the performer and the audience, especially the audience. And the process must be a continuous loop. If there is only one procedure of “input, process and outputā€ļ¼Œthe process will be responsive rather than interactive. That is exactly the problem of my project “Library Noise Sensor” . “Artificial Intuition” serves as a good example that the machine never stop interacting with the audience and the audience is the key factor that affects its movement.  “Book of privileges” is also good because the book keeps on evolving since there are more and more audience adding pages to it, and the audience is always receiving a brand new version of the book, thus having a deeper understanding on identity diversities shown by the book.

In “Making Interactive Art: Set the Stage, Then Shut up an Listen”, Tom Igoe writes that “So if youā€™re thinking of an interactive artwork, donā€™t think of it like a finished painting or sculpture.  Think of it more as a performance. Your audience completes the work through what they do when they see what youā€™ve made”. An interactive work is not just displaying something to the audience, it invites the audience’s involvement. Without it, the artwork is incomplete. The similar idea that emphasizes the involvement of the audience can be supported by a Chinese article called “The Future of Media Storytelling: Immersion, Interaction and Integration”. In this article, it is stated that “A more open and diverse two-way communication has been dominating the narrative mode, and the traditional author concept has been weakened”.  The producer and the audience share the same importance in an interactive process.

Reference:

https: //www.youtube.com/watch?v=WUbhFWJOYC4

https://www.sohu.com/a/282797095_677585

https://www.youtube.com/watch?v=RsOEOLnym9

Making Interactive Art: Set the Stage, Then Shut Up and Listenby Tom Igoe

https://www.ifanr.com/137901

Recitation 6: Processing Animation ā€” Jiayi Liangļ¼ˆMaryļ¼‰

What I did in class:       

       In this session, I am asked to create an animation by Processing. Having learnt how to create a canvas, draw images on it and decide their color, this week I learn how to make these pictures into animation. There is an extra requirement that I have to include keyboard or mouse in the interaction. I decide to adapt my code I did last week.

        My original code shows a girl’s sad face. I decide to make the image funnier by changing the girl’s eye position when the mouse is near the girl’s eye. When the mouse is far from the eye, the girl will fall asleep.

My code:

void setup(){
size(660,640);
}
void draw(){
background(#F8FAE8);
stroke(0,0,0);
fill(0,0,0);
ellipse(260,300,320,320);

fill(0,0,0);
if (mouseX<290 & mouseX>200 & mouseY<260 & mouseY>180){
fill(#F8FAE8);
ellipse(240,220,80,80);
fill(#F8FAE8);
noStroke();
ellipse(150,380,80,80);
fill(0);
ellipse(mouseX,mouseY,40,40);
}else{
background(200,100,255);
fill(0,0,0);
ellipse(260,300,320,320);
fill(200,100,255);
rect (200,220,100,10);
fill(200,100,255);
noStroke();
ellipse (150,360,150,80);
}
fill(0);
ellipse(110,230,100,100);

fill(0,0,0);
ellipse(380,190,40,40);
ellipse(415,190,40,40);
ellipse(450,190,40,40);
ellipse(485,190,40,40);
ellipse(520,190,40,40);
ellipse(555,190,40,40);
ellipse(555,225,40,40);
ellipse(555,260,40,40);
ellipse(555,295,40,40);
ellipse(555,330,40,40);
ellipse(555,365,40,40);
}

Additional Homework:

For the additional homework, I am asked to create a circle that keeps expanding and contracting periodically, whose position is decided by the mouse’s position. And the circle will change its color periodically too.

float a= 300;
float b=0;
float c=1;
void setup(){
size(600,600);
colorMode(HSB,100);
}
void draw(){
background(#FFFFFF);
stroke(b,100,100);
b=b+c;
if(b== 100){
c=-0.5;
}
if(b==0){
c=0.5;
}
strokeWeight(20);
ellipse(mouseX,mouseY,a=a+2,a=a+2);
if(a==400){
a=-a;
}if(a==-100){
a=-a;
}
}

Reflectionļ¼š

What I find difficult in the recitation is that if I change the background color at the end of the code, the whole image will be affected. To avoid this, I simply add a more background code at the beginning of the draw code. However, the fellow tells me that maybe I can use the pushStyle and popStyle to avoid this issue.

What I find hard in additional homework is how to make the circle contract. I find that when the size number is under 0, the size will be the absolute value of the number, so I simply make  “a=-a” in the code.

Interesting functions:

colorMode( )

mouseX/mouseY

if( )

“==” and “=” are different :  “==” is used for comparing two numbers, “=” is used for valuation