In this recitation we are asked to create a Processing sketch that controls media by manipulating that media’s attributes using a physical controller made with Arduino. For me I chose to use a physical controller to manipulate a video shown in our class. Two potentiometers are used in my Arduino part, one is to control the position of the video and the other is to control the speed of the video.
Code:
Arduino part is the same as the example:
// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing
const int buttonPin = 8;
void setup() {
Serial.begin(9600);
}
void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.println(); // add linefeed after sending the last sensor value
// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}
Processing:
// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing
/*
* Based on the readStringUntil() example by Tom Igoe
* https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
*/
import processing.serial.*;
String myString = null;
Serial myPort;
int NUM_OF_VALUES = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/
import processing.video.*;
Movie myMovie;
void setup() {
background(0);
size(800, 800);
myMovie = new Movie(this, "dancing.mp4");
myMovie.loop();
setupSerial();
}
void movieEvent(Movie movie) {
myMovie.read();
}
void draw() {
background(0);
updateSerial();
float c = map(sensorValues[1], 0, 1023, 0 , 800);
image(myMovie, c, c);
float newSpeed = map(sensorValues[0], 0, 1023, 0.1, 5);
myMovie.speed(newSpeed);
}
// use the values like this!
// sensorValues[0]
// add your code
//
void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[13], 9600);
// WARNING!
// You will definitely get an error here.
// Change the PORT_INDEX to 0 and try running it again.
// And then, check the list of the ports,
// find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----"
// and replace PORT_INDEX above with the index number of the port.
myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
myString = null;
sensorValues = new int[NUM_OF_VALUES];
}
void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), ",");
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}
Writing reflection:
A variety of computer vision techniques are introduced in the reading which inspire me the ways of how technology is used in my project. In the article, Myron Krueger, the creator of Videoplace, he states that technology should be used as supportive tools which resonates with my understanding of a great interactive project. As for my project, I use technology to make the experience and interaction of users and the project better and to make the audience identify with the theme of my project. And there is on particular project in the reading that really impressed me which is Messa di Voce’s interactive software. It can visualize the characteristic of water. Therefore , I think technology can be used to connect different senses together and this notion is used in my final project too.
Reference
Levin, Golan. “Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers.” AI & Society, vol. 20, no. 4, 2006, pp. 462-482.