Credit to Johannes Vermeer for Girl With a Pearl Earring painting.
I created a Processing sketch that controls the left right movement of the eyes of the image Girl With The Pearl Earring by manipulating an analog potentiometer setup made with Arduino. A user inputs potentiometer values by twisting the potentiometer knob. These values are transmitted as inputs to Processing via serial communication and then Processing converts those inputs to position outputs for the eyes.
At first, I was considering storing the color pixel information for the eyes into an array, then moving that array by using the translate function. However, Rudi suggested that I create a separate image for the eyes of the painting, with a transparent background, then move that image of eyes by changing the image’s position.
In photoshop, I selected the eyes and pasted in place into a new, transparent document of the same size. Next, I dragged the two images, the original and the eyes, into processing and began coding.
In arduino, I created an int sensor to receive information from analog read pin A0. Then I mapped that sensor value to the range 1-255 using the map function.
Arduino Code
// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing
void setup() {
Serial.begin(9600);
}
void loop() {
int sensor1 = analogRead(A0);
sensor1=map(sensor1,1,1023,1,255);
// keep this format
Serial.print(sensor1);
Serial.println(); // add linefeed after sending the last sensor value
// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}
In Processing, I created two PImage, one for the original image and one for the eyes. I created two int values to get one potentiometer value a bit before the next potentiometer value. I compared the two values to each other to determine whether to move the eyes left or right, then moved the image right or left one, and up or down .5
Code for first Processing Sketch
// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing
/*
* Based on the readStringUntil() example by Tom Igoe
* https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
*/
import processing.serial.*;
String myString = null;
Serial myPort;
PImage img1;
PImage img2;
int initialSensorVal;
int finalSensorVal;
int x=0;
int y=0;
int NUM_OF_VALUES = 1; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/
//240-266, 348-366 133-155 326-345
void setup() {
size(735, 1000);
background(0);
setupSerial();
img1=loadImage(“girlWithPearlEarring.jpg”);
img2=loadImage(“girlWithPearlEarringEyes.png”);
image(img1,0,0,width,height);
}
void draw() {
img1=loadImage(“girlWithPearlEarring.jpg”);
updateSerial();
printArray(sensorValues);
finalSensorVal=sensorValues[0];
//move to left
if(initialSensorVal<finalSensorVal){
image(img2,x++,y+=.5,width,height);
}
//move to right
if(initialSensorVal>finalSensorVal){
image(img2,x–,y-=.5,width,height);
}
initialSensorVal=sensorValues[0];
}
void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 3 ], 9600);
// WARNING!
// You will definitely get an error here.
// Change the PORT_INDEX to 0 and try running it again.
// And then, check the list of the ports,
// find the port “/dev/cu.usbmodem—-” or “/dev/tty.usbmodem—-”
// and replace PORT_INDEX above with the index number of the port.
myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
myString = null;
sensorValues = new int[NUM_OF_VALUES];
}
void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}
The problem with this processing sketch was that my potentiometer had to be set at 150 to be moving the right amount left or right. If it was set at 0, the potentiometer would only move the eyes to the right.
I created a second processing sketch where, instead of using if statements to determine left or right, I mapped the potentiometers’ values to the range I wanted the image to move in. However, the catch to this approach is that the potentiometer also dictates where the starting position of the eye is, so to have the original image line up with the eyes at the starting position, the potentiometer has to be set at a certain value too.
2nd Processing Sketch Code
// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing
/*
* Based on the readStringUntil() example by Tom Igoe
* https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
*/
import processing.serial.*;
String myString = null;
Serial myPort;
PImage img1;
PImage img2;
float initialSensorVal;
float finalSensorVal;
float x=0;
float y=0;
float leftBound=-10;
float rightBound=2;
float topBound=1;
float bottomBound=0;
int NUM_OF_VALUES = 1; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/
//240-266, 348-366 133-155 326-345
void setup() {
size(735, 1000);
background(0);
setupSerial();
img1=loadImage(“girlWithPearlEarring.jpg”);
img2=loadImage(“girlWithPearlEarringEyes.png”);
image(img1,0,0,width,height);
}
void draw() {
img1=loadImage(“girlWithPearlEarring.jpg”);
updateSerial();
printArray(sensorValues);
finalSensorVal=sensorValues[0];
x=map(sensorValues[0],1,255,leftBound,rightBound);
y=map(sensorValues[0],1,255,bottomBound,topBound);
image(img2,x,y,width,height);
initialSensorVal=sensorValues[0];
}
void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 3 ], 9600);
// WARNING!
// You will definitely get an error here.
// Change the PORT_INDEX to 0 and try running it again.
// And then, check the list of the ports,
// find the port “/dev/cu.usbmodem—-” or “/dev/tty.usbmodem—-”
// and replace PORT_INDEX above with the index number of the port.
myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
myString = null;
sensorValues = new int[NUM_OF_VALUES];
}
void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}
In my project, technology was used to allow physical inputs to manipulate digital images, which to me was unthinkable of before. While my project did not use computer vision, algorithms that allow computers to make intelligent assumptions about images and videos, the mere fact that processing and arduino exist are testament to the improvements in software development tools that allow student programmers to experiment artistically. Making tools with easier learning curves has enabled more people to be able to create interactive art with technology.