Author: Leah Bian

 For this recitation, we first had a quick workshop about the map() function. I reviewed when to use this function, and what the formats are. After this workshop finished, we were asked to choose a workshop to attend. The choices included media manipulation, serial communication, and object oriented programming. In our final project, we will focus on how to send data between Arduino and Processing, and how to draw and control the image of the marionette in Processing. Therefore, my partner attended the workshop about serial communication, and I chose the workshop about object oriented programming.

   In the workshop, Tristan gave us a detailed explanation about what “object” means, and what the parts of it are, which include class and instance. We thus went over the process of writing codes of object oriented programming from bigger parts (class, instance) to the smaller ones (variables, constructor, functions). Using the emoji faces as example, we started to work on the code together. Based on the code that we wrote during recitation, I started to create my own animation as exercise.

We needed to use classes and objects, and the animation should include some level of interactivity. As requirements, we needed to use the map() function and an ArrayList. I decided to use the mousePressed function as way of interaction. The shapes would be created at the mouse’s position. After searching a basic function for a star online, I modified the code to meet the requirement of using arrays. Creating a star needs to use the vertex() function for ten times. I let the stars to fall down to the bottom of the screen by setting the yspeed to random(3,7). I wrote a bounce function, so that if the stars hit the boundaries on the left and the right they will turn to the opposite direction. Finally, I used the map() function to limit the area where the stars can be created.

These are my codes in Processing ( 2 parts):

ArrayList<Stars> stars = new ArrayList<Stars>();

void setup() {
  size(800, 600);
  noStroke();
}

void draw() {
  background(130,80,180);
  for (int i=0; i<stars.size(); i++) {
    Stars s = stars.get(i); 
    s.move();
    s.bounce();
    s.display();
  }
  float x=map(mouseX,0,width,width/6,width-width/6);
  float y=map(mouseY,0,height,height/6,height-height/6);
  if (mousePressed==true) {
    stars.add( new Stars(x,y));
  }
}

class Stars {
  float x, y, size;
  color clr;
  float xspeed, yspeed;

  Stars(float tempX, float tempY) {
    x = tempX;
    y = tempY;
    size = random(10, 100);
    clr = color(255, random(180,255), random(50,255));
    xspeed = random(-3, 3);
    yspeed = random(3, 7);
  }

  void display() {
    fill(clr);
   beginShape();
   vertex(x,y);
   vertex(x+14,y+30);
   vertex(x+47,y+35);
   vertex(x+23,y+57);
   vertex(x+29,y+90);
   vertex(x,y+75);
   vertex(x-29,y+90);
   vertex(x-23,y+57);
   vertex(x-47,y+35);
   vertex(x-14,y+30);
    endShape(CLOSE);
  }

  void move() {
    x += xspeed;
    y += yspeed;
  }

  void bounce() {
    if (x < 0) {
      xspeed = -xspeed;
    } else if (x > width) {
      xspeed = -xspeed;
    }
}
}

For this week’s recitation, I created a Processing sketch to control the movement, size and color of an image. Data from Arduino are sent to Processing, which thus decides the image’s attributes. I used three potentiometers to adjust the analog values, and used the “serial communication” sample code as the basis for further modifications.

It was not hard to build the circuit, since we use potentiometers quite often. After finishing the circuit, I started to write the code. I adjusted the “Serial. print()” and “Serial. Println()” functions to send the data from Arduino to Processing smoothly. I chose an image from the famous cartoon “Rick and Morty” as my media. I decided to let the three potentiometers control the movement, size and color of the image respectively. I used the “map()” function to limit the analog values. It was not difficult to write the code for changing the image’s size and position, but changing colors was a bit complicated. I chose colorMode(RGB) to set the image with various colors, instead of only with white, grey and black. I used the “tint()” function to set the color. But since I only have one potentiometer controlling the color of the image, I can only set the image with analogous color schemes.

This is my code for Processing:

import processing.serial.*;

String myString = null;
Serial myPort;
PImage img;
int NUM_OF_VALUES = 3;   
int[] sensorValues;  

void setup() {
  size(800, 800);
  img = loadImage("rickk.png");
  setupSerial();
  colorMode(RGB);
}

void draw() {
  background(0);
  updateSerial();
  printArray(sensorValues);
  float a = map(sensorValues[0],0,1023,0,800);
  float b = map(sensorValues[1],0,1023,0,255);
  float c = map(sensorValues[2],0,1023,400,800);
  tint(b, 30, 255);
  image(img,a,200,c,c);
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[5], 9600);
  myPort.clear();
  myString = myPort.readStringUntil( 10 ); 
  myString = null;
  sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); 
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

This is my code for Arduino:

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
int sensor3 = analogRead(A2);
Serial.print(sensor1);
Serial.print(“,”);
Serial.print(sensor2);
Serial.print(“,”);
Serial.print(sensor3);
Serial.println ();
delay(100);
}

Reflection:

This week’s reading, Computer Vision for Artists and Designers, inspired me a lot. According to the article, Myron Krueger, the creator of Videoplace, firmly believed that the entire human body should have a role in people’s interactions with computers. In my previous definition of an interactive experience, I also mention this idea. Videoplace, an installation that captured the movement of the user, makes my hypothesis concrete and clear. In the project that I made for this recitation, the user can interact with the device only with the potentiometers, which makes the interactivity here quite low. Besides, the whole process is too simple and it does not convey any meaning implications, compared with the other art works mentioned in the reading, such as LimboTimeand Suicide Box. In conclusion, an interactive experience should let the user fully engaged, probably by physical interaction and building up a meaningful theme. The project that I made this time is not highly interactive due to various limitations, but I will try to create a satisfying interactive experience for the final project.

Reference:

Computer Vision for Artists and Designers: 

https://drive.google.com/file/d/1NpAO6atCGHfNgBcXrtkCrbLnzkg2aw48/view

Project Title: Puppet

In “The Future of Design: When you come to a fork in the road, take it”, the author discusses the significance of human-centered design. It requires a deep understanding of people and involves careful observations and analyses to determine needs. When working on the project proposal, I applied the main concept of human-centered design to our project. We first came up with the theme of puppet, which always represents those who are controlled and follow commands. We further developed the theme by extending it to the concept of social image. Based on this theme, we developed our design by thinking about how we can utilize Arduino and Processing to express this idea. We found that the Arduino servo is a perfect choice to control the movement of the marionette, and Processing can be used for the interaction, letting the user control the marionette in an indirect way. We then came up with our further plan and recorded it in the proposal. A miniature stage with stage curtains and lighting will be set, on which we will place a marionette. The marionette will be connected to the Arduino servos on the roof with strings, so that it can change poses. On the wall behind the miniature stage, we will cast the Processing image with a projector. In Processing, we will draw the shadow of the puppet. The users can control the moveable joints of the puppet by the mouse. Data of these gestures would be passed from Processing to Arduino. The real marionette on the stage would first quickly and randomly change a series of postures, and finally end up with the pose set by the user in Processing before. In addition, we will try to enhance the artistic appeal of the installation with music, lighting and so on. The potential challenges include how to accurately pass the data of the puppet’s poses from Processing to Arduino; how to connect servos with the puppet smoothly with the correct mechanism; and how we can convey the heavy topic in an implicit way.

After submitting the proposal, we made a plan to make our tasks clear. From Nov. 21 to Nov. 24, we will first collect general information. We will get a marionette to figure out how the mechanism works and how we can modify it to connect it with servos. In addition, we will make final decisions about how the image in Processing will look like to convey our theme; what else we can do to add aesthetic values; what the materials that may involve are… We will draw a draft of the final work as well. From Nov. 24 to Nov. 30, we will get the code done. During that week, we will also start to prepare the materials (start to do 3D printing). If necessary, we will make some modifications to the original plan. From Dec.1 to Dec. 10, we will assemble all the materials together and add some detailed decorations to enhance the aesthetic values, such as lighting, music, background settings and so on. We will invite some peers and IMA fellows to test our project, and collect suggestions from them to make final modifications.

In my preparatory research and analysis, I wrote my personal definition of a successful interaction experience. In my opinion, the process of interaction should be clear to the users, so that they can get a basic sense of what they should do to interact with the device. Various means of expression can be involved, such as visuals and audios. The experience could be thought-provoking, which may reflect the facts in the real life. Our project aligns with my definition of interaction well. It aims at those who intentionally or compulsively cater to the social roles imposed on them by the forces in the society. The user plays the role of the forces that decide our social images, while the puppet on the stage represents ourselves, who are constrained by the social expectations. The “struggle” that the puppet has before it stops with its final pose represents our mental struggles when pondering whether to meet social expectations. However, we ultimately make the passive or initiative choice that satisfies the forces just as what the puppet does. We expect that the user can be inspired by this artistic installation, gain new thoughts, or have reflection. The marionette on stage may be clownish, but it is more than that. We hope that we can find the balance between entertainment and artistic expression, and finally convey our theme to the audience.

Reference: 

“The Future of Design: When you come to a fork in the road, take it”:

https://jnd.org/the_future_of_design_when_you_come_to_a_fork_in_the_road_take_it/

Exercise 1: Make a Processing Etch A Sketch

Step 1: For this exercise, I needed to send two analog values from Arduino to Processing via serial communication. In Processing, I drew an ellipse, and its location was defined by the values from Arduino. I built a circuit with two potentiometers, controlling the ellipse’s x-axis and y-axis movement respectively. I used the A to P (multipleValues) code example and made some modifications. For example, I changed the number of the analog values in Processing, and used map function to limit the range. The interaction here is clear but quite simple, but it can be a single part of a bigger interactive process. 

Code (Processing):

import processing.serial.*;
String myString = null;
Serial myPort;
int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/

void setup() {
  size(500, 500);
  setupSerial();
}

void draw() {
  background(0);
  updateSerial();
  float x = map(sensorValues[0],0,1023,0,500);
  float y = map(sensorValues[0],0,1023,0,500);
  printArray(sensorValues);
  fill(255);
  ellipse(x,y,100,100);
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[5], 9600);
  
  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Code (Arduino):

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.println ();
// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Step 2: For step 2, I modified the code to draw a line. I needed to keep track of the previous x and y values to draw a line from there to the new x and y positions. Therefore, I used float() function for four times to define the variables, and then drew the line.

Code (Processing):

import processing.serial.*;
String myString = null;
Serial myPort;
int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/

void setup() {
  size(500, 500);
  background(0);
  setupSerial();
}

void draw() {
  float pX = map(sensorValues[0], 0, 1023, 0, width);
  float pY = map(sensorValues[1], 0, 1023, 0, width);
  updateSerial();
  float x = map(sensorValues[0], 0, 1023, 0, width);
  float y = map(sensorValues[1], 0, 1023, 0, height);
  printArray(sensorValues);
  stroke(255);
  line(pX,pY,x,y);
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[5], 9600);
  
  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Code (Arduino):

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.println ();
// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Exercise 2：Make a musical instrument with Arduino

For this exercise, I wrote a Processing sketch that sends values to Arduino based on the mouse’s x and y positions. A hint was provided that we should use the tone() function, so I studied it first. The syntax of the function should be “tone(pin, frequency, duration)”, and the function can only be used with pin3 and pin11. Therefore, I put a buzzer on the breadboard and connected it to pin11 and the ground. The serial values from Processing are sent to Arduino and are translated into frequency and duration for a tone, which will be sounded by the buzzer. Besides, I drew an ellipse to make the location of the mouse’s x and y positions clearer. The interaction here is clear but simple, since only two variables are involved and what the user needs to do is just to move the mouse. However, this simple musical instrument could be a basis for further development and the idea can be an interesting inspiration.

Code (Processing):

import processing.serial.*;
int NUM_OF_VALUES = 2;  /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
Serial myPort;
String myString;
int values[] = new int[NUM_OF_VALUES];

void setup() {
  size(500, 500);
  background(0);

  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[5], 9600);
 
  myPort.clear();
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;
}

void draw() {
  background(0);
  values[0]=mouseX;
  values[1]=mouseY;
  circle(mouseX,mouseY,80);
  sendSerialData();
  echoSerialData(200);
}

void sendSerialData() {
  String data = "";
  for (int i=0; i<values.length; i++) {
    data += values[i];
    if (i < values.length-1) {
      data += ","; // add splitter character "," between each values element
    } 
    else {
      data += "n"; // add the end of data character "n"
    }
  }
  myPort.write(data);
}

void echoSerialData(int frequency) {
  if (frameCount % frequency == 0) myPort.write('e');

  String incomingBytes = "";
  while (myPort.available() > 0) {
    incomingBytes += char(myPort.read());
  }
  print( incomingBytes );
}

Code (Arduino):

#define NUM_OF_VALUES 2

int tempValue = 0;
int valueIndex = 0;
int values[NUM_OF_VALUES];

void setup() {
Serial.begin(9600);
pinMode(11, OUTPUT);
}

void loop() {
getSerialData();
tone(11,values[0],values[1]);

}

//recieve serial data from Processing
void getSerialData() {
if (Serial.available()) {
char c = Serial.read();
switch (c) {
case ‘0’…’9′:
tempValue = tempValue * 10 + c – ‘0’;
break;
case ‘,’:
values[valueIndex] = tempValue;
tempValue = 0;
valueIndex++;
break;
case ‘n’:
values[valueIndex] = tempValue;
tempValue = 0;
valueIndex = 0;
break;
case ‘e’: // to echo
for (int i = 0; i < NUM_OF_VALUES; i++) {
Serial.print(values[i]);
if (i < NUM_OF_VALUES – 1) {
Serial.print(‘,’);
}
else {
Serial.println();
}
}
break;
}
}
}