Category: Rudi

In this week’s recitation, I am asked to work individually to create a Processing sketch that controls media by  using a physical controller made with Arduino. I choose to use potentiometers to control an image. 

My Processing Code:

PImage img1;
import processing.serial.*;

String myString = null;
Serial myPort;

int NUM_OF_VALUES = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/

void setup() {
size(800, 800);
background(0);
setupSerial();
img1 = loadImage(“angel.png”);
imageMode(CENTER);
}

void draw() {
updateSerial();
printArray(sensorValues);
background(255);
image(img1,400,400,sensorValues[0],sensorValues[0]);
filter(BLUR, sensorValues[1]/100);
tint(255,sensorValues[0]/3);
}

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 3 ], 9600);

myPort.clear();
sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}

I use the first potentiometer to change the size and the transparency, and the second potentiometer to change the blur length.

Since I have practiced how to use arduino to control Processing, this week’s recitation task is quite simple. All I need to do is to use the Pimage to load an image and use tint, blur etc. to edit the image. I think if I have more time, I will try to load more pictures to let the characters seem like interacting with each other by changing their positions and sizes.

Reflection:

After reading  Computer Vision for Artist and Designers , I got a lot of inspirations.  The article introduces various types of computer vision techniques. The project mentioned in this article I am interested in most is Messa di Voce’s interactive software. It visualizes the sound. If the user is speaking, the sound he or she made will be transformed into an image. This makes me think of one writing skill I learned in my high school —synaesthesia. It inspires me that different senses can be associated with each other. I can comment  a song as blue to show that it is sorrowful, and I can define a girl’s smile as sweet to show that she is so cute. In my project, I can also use this kind of skill to use people’s different senses to fertilize the interaction process.

For this week’s recitation, we were assigned to manipulate media in Processing through Arduino. I decided to explore the live video webcam in processing and manipulate the tint function in live images with the help of a potentiometer and infrared distance sensor in Arduino. 

For the first attempt in the code, I was able to change the opacity in the tint applied to the live video through the potentiometer. However, as I wasn’t redrawing the background in processing, the opacity was just change upon itself, and led to the creation of an even more interesting effect of blurred image, that made any user on the screen have some kind of a ghostly look.

The second manipulation, was inspired by the readings in “Computer Vision in Artists and Designer”, as it mentioned how algorithms and computing media has been used in detecting motion, specially that which involved “the movements of people”. Therefore, I developed a code that changed the tint of an image from blue to red, depending on how close an individual was situated from the infrared distance sensor in Arduino. 

For the codes of both iterations of media manipulation, I was going to use the multiple values example so that both sensors in Arduino were connected to processing. However, I first did it with the one value example, and when changing it to multiple values, both sensors were not working in the most efficient way while working simultaneously, therefore I changed it back to one value codes for the time provided. Here I am attaching the codes I used for both Arduino and Processing.

I was inspired in the ways technology was used in my project, specially in the sense that I felt as if I created an object that helped enhance security systems in stores. Similar to the idea of the game system LimboTime, which was developed for participants to attempt and pass below an imaginary line. If such individual crosses above the line, the game rings an alarm. Similar to this idea, for my second media manipulation, if a person went passed the allowed distance, the colors started to change in the webcam.

Code from Arduino

void setup() {
Serial.begin(9600);
}

void loop() {
int pin1 = analogRead(A0);
int sensorValue = map(pin1,0,1023,0,255);
Serial.write(sensorValue);

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(10);
}

Code from Processing

import processing.serial.*;


Serial myPort;
int valueFromArduino;


import processing.video.*; 
Capture cam;


void setup() { 
  size(1280, 480); 
  cam = new Capture(this, 640, 480);
  cam.start(); 
  
  
  printArray(Serial.list());
  // this prints out the list of all available serial ports on your computer.

  myPort = new Serial(this, Serial.list()[4], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.
  
} 

void draw() { 
  //background(255);
  if (cam.available()) { 
   cam.read(); 
  } 
     
  int x = valueFromArduino;
  scale(-1,1);
  image(cam, -640, 0); 
  scale(-1,1);
  image(cam, 640, 0);
  tint(0, 153, 204, x); 
 

//  if (valueFromArduino < 100) {
//      tint(0, 153, 204);
//  } else { 
//    tint(255,0,0);
//  }
  
   // to read the value from the Arduino
  while ( myPort.available() > 0) {
    valueFromArduino = myPort.read();
  }
  println(valueFromArduino);//This prints out the values from Arduino
}

*Side Note: I have been trying to include images of the circuit and pictures of the media manipulation, however it is not allowing me to do so as it says there’s an error with the images, therefore I’ll try it again tomorrow and if not possible I’ll send them through e-mail directly to Rudi*

In my recitation, I planned to use two potentiometers to control the image in the Processing. The first potentiometer is to control the size of the image, and the second potentiometer is to control the degree of the blur of the image.

At first, after I read the Powerpoint in the folder, I used the function “resize” to  change the size of the image and the grammar of “Resize” is photo.resize( , ).  However, a mistake happened. When I resize the picture for several times, the picture blur a little. So I know the resize function will change the function, which leads to the change of the picture. Therefore, later I chose another function which do not  change the image: 

image(photo,0,0,a);

Second, when I first use the code that can transfer data from Arduino to Processing, the error said the ArrayException. At first, I thought it was because of the number from Arduino exceed the range that Processing. However, when I asked Rudi, he said the number of the blur function couldn’t be so big. So I used 

float a=map(sensorValues[1],0,1024,0,40);

to make the number from Arduino smaller.

Here is my arduino code.

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Here is my processing code

import processing.serial.*;

String myString = null;
Serial myPort;
PImage photo;

int NUM_OF_VALUES = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/

void setup() {
size(500, 500);
background(0);
photo=loadImage(“12deeaa706628e537518aa533d6c8658.jpeg”);
setupSerial();
}

void draw() {
background(0);
updateSerial();
printArray(sensorValues);

image(photo, 0, 0,sensorValues[0],sensorValues[0]);//,width,height);
float a=map(sensorValues[1],0,1024,0,40);
println(“senmap!”,a);
filter(BLUR,a);

//
//filter(BLUR, sensorValues[1]);

// use the values like this!
// sensorValues[0]

// add your code

//
}

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 4 ], 9600);
// WARNING!
// You will definitely get an error here.
// Change the PORT_INDEX to 0 and try running it again.
// And then, check the list of the ports,
// find the port “/dev/cu.usbmodem—-” or “/dev/tty.usbmodem—-”
// and replace PORT_INDEX above with the index number of the port.

myPort.clear();
// Throw out the first reading,
// in case we started reading in the middle of a string from the sender.
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
myString = null;

sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}

Here is my circuit:

Here is my video:

r9

Reading

In this week’s reading, the author talked about the development of Computer Image analysis. It showed that computer vision doesn’t only can be used in military and law-enforcement actions. One of the most impressive project based on the technology is called “Standards and Double Standards”, which can recognize the people in the room and make the belt rotate to face to the audience. The project used belts to represent people face to the audience using the tech.

What makes me think is that the game I want to make can use this technique to track the movement of the player in order to make them like moving the stick on the screen, but I’m not sure if I can manage to understand it.

   Source：Computer Vision for Artist and Designers