IMA Documentation

Vocal Art (With Kevin Nader)

This project will make artistic interpretations of the user(s)’ voices, and it will base the color’s saturation on the volumes of the voices. The project will ideally translate the vocal input (from a microphone) into colors and shapes (again, based on volume). After one user uses the project, the interpretation of their voice would remain drawn on screen, and will be added to by the next user. 

This project aims to interpret the voices of its users, and compound these interpretations between each user. The first user would start with a blank canvas, while each subsequent use would build on top of that. The Arduino code will process input data and send it to processing, which will generate the output. This project does not require that much physical fabrication, the only physical component being the microphone Arduino circuit. Once this part of the project is complete, my partner and I will write the Arduino and processing codes. Given the simplicity of the physical aspect of the project, this could allow my partner and me to design an aesthetically-pleasing physical apparatus to house the circuit. 

Over the course of the semester, we have had to define and redefine “interaction” numerous times. I originally defined it as “the communication between two actors” for the group project, then changing it to “the communication between two or more actors through mutually influential inputs and outputs” for the midterm project. As of now, my definition is “the communication between two or more organic or machine actors, facilitated by an interchange of inputs and outputs between actors”. Interactivity relies on the mutual exchange of inputs and outputs between human and non-human actors. These updates to my definition were influenced by Crawford’s “The Art of Interactive Design”, in which he defines interaction “in terms of a conversation: a cyclic process in which two actors alternately listen, think, and speak (1)”.

Vocal Art aligns with my updated definition of interaction because it consists of a machine actor and potentially several human actors that consistently communicate and build upon previous outputs. My partner may have different ideas about this project’s significance, but I see it as a collaborative art piece that complicates itself and constantly changes based on its perception of its audience. I don’t think there is a specific intended audience, simply anyone who wishes to add their voice to the collective canvas.

Part 1: Etch-A-Sketch

Arduino Schematic:

Arduino Code:

void setup() {
  Serial.begin(9600);
}

void loop() {
  int sensorValue1 = analogRead(A0);
  int sensorValue2 = analogRead(A1);
  int mappedvalue1 = map(sensorValue1, 0, 1023, 0, 500);
  int mappedvalue2 = map(sensorValue2, 0, 1023, 0, 500);
  Serial.print(mappedvalue1);
  Serial.print(",");
  Serial.print(mappedvalue2);
  Serial.println();
  delay(1);        
}

Processing Code:

import processing.serial.*;
String myString = null;
Serial myPort;
int x2;
int y2;
int NUM_OF_VALUES = 2;   
int[] sensorValues;      

void setup() {
  size(500, 500);
  background(0);
  setupSerial();
}

void draw() {
  updateSerial();
  printArray(sensorValues);
  stroke(255);
  line(x2, y2, sensorValues[0], sensorValues[1]);
  x2 = sensorValues[0];
  y2 = sensorValues[1];
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[5], 9600);
  myPort.clear();
  myString = myPort.readStringUntil( 10 ); 
  myString = null;
  sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Result:

Part 2: Musical Instrument
Arduino Schematic:

Arduino Code:

#define NUM_OF_VALUES 2

int tempValue = 0;
int valueIndex = 0;

int values[NUM_OF_VALUES];

void setup() {
  Serial.begin(9600);
  pinMode(13, OUTPUT);
}

void loop() {
  getSerialData();


  if (values[2] == 1) {
    tone(9, values[0]);
  } else {
    noTone(9);
  }

}


void getSerialData() {
  if (Serial.available()) {
    char c = Serial.read();
    switch (c) {
      case '0'...'9':

        tempValue = tempValue * 10 + c - '0';
        break;
      case ',':
        values[valueIndex] = tempValue;
        tempValue = 0;
        valueIndex++;
        break;
      case 'n':
        values[valueIndex] = tempValue;

        tempValue = 0;
        valueIndex = 0;
        break;

      case 'e':
        for (int i = 0; i < NUM_OF_VALUES; i++) {
          Serial.print(values[i]);
          if (i < NUM_OF_VALUES - 1) {
            Serial.print (",");
          }
          else {
            Serial.println();
          }
        }
        break;
    }
  }
}

Processing Code:

import processing.serial.*;

int NUM_OF_VALUES = 2;

Serial myPort;
String myString;

int values[] = new int[NUM_OF_VALUES];

void setup() {
  size(500, 500);
  background(0);

  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[5], 9600);

  myPort.clear(); 
  myString = myPort.readStringUntil(10); 
  myString = null;
}

void draw() {
  noStroke();
  fill(0, 30);
  rect(0, 0, 500, 500);

  strokeWeight(5);
  stroke(255);
  line(pmouseX, pmouseY, mouseX, mouseY);

  values[0] = mouseX;
  values[1] = mouseY;

  sendSerialData();
  echoSerialData(200);
}

void sendSerialData() {
  String data = "";
  for (int i=0; i<values.length; i++) {
    data += values[i];
    if (i < values.length-1) {
      data += ",";
    } else {
      data += "n";
    }
  }

  myPort.write(data);
}

void echoSerialData(int frequency) {
  if (frameCount % frequency == 0) myPort.write('e');

  String incomingBytes = "";
  while (myPort.available() > 0) {

    incomingBytes += char(myPort.read());
  }

  print(incomingBytes);
}