Introduction

In this recitation we are required to control media with a Processing sketch through using a physical controller made with Arduino. First I used two potentiometers to control the location of the image I chose. Then I changed the function of one potentiometer to controlling the size of the image.

Controlling the Location:

Arduino Code (from the example):

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Processing Code (based on the example):

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */

import processing.serial.*;

String myString = null;
Serial myPort;

PImage img;

int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/

void setup() {
  size(500, 500);
  background(0);
  img=loadImage("figure0.jpeg");

  setupSerial();
}


void draw() {
  background(0);
  updateSerial();
  printArray(sensorValues);
float a = map(sensorValues[0], 0, 1023, 0, 280);
float b = map(sensorValues[1], 0, 1023, 0, 280);
tint(b, 180, 205);
image(img,a,b);
  // use the values like this!
  // sensorValues[0] 

  // add your code

  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 2 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

I used tint() in my code and connected the location with color. When the image moves up and down, its color changes simultaneously. 

Controlling Location as well as Size:

Arduino Code is the same as above.

Processing Code:

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */

import processing.serial.*;

String myString = null;
Serial myPort;

PImage img;

int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/

void setup() {
  size(500, 500);
  background(0);
  img=loadImage("figure0.jpeg");

  setupSerial();
}


void draw() {
  background(0);
  updateSerial();
  printArray(sensorValues);
float a = map(sensorValues[0], 0, 1023, 0, 255);
float b = map(sensorValues[1], 0, 1023, 0, 500);
tint(b, 180, 205);
image(img,a,100,b,b);
  // use the values like this!
  // sensorValues[0] 

  // add your code

  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 2 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

I changed the function of one potentiometer, and it is now controlling the size of the image. Maybe I should add one more potentiometer so that I can control both the size and the location of two directions.

Reading Reflection

This week’s reading refers to Myron Krueger’s legendary Videoplace, which was developed in the 1970s by his “deeply felt belief that the entire human body ought to have a role in our interactions with computers.” This examples inspires me in doing my final project. My partner and I are trying to create a new interactive experience, and the old way of keyboard or mouse interaction with the computer can no longer satisfy us. To expand the boundary of interaction, we look to the human body. We plan to use the motion of human bodies as input in our final project, and we think the game involving the entire human bodies (preferably >=2) would be more interactive and fun. In Krueger’s words, we can create a “multi­-person virtual reality.” It is fascinating to learn the history and pioneering work of human body interaction with the computer from the reading.

Work Cited

Golan Levin and Collaborators. “Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers.” Journal of Artificial Intelligence and Society, Vol. 20.4. Springer Verlag, 2006.