INTERACTION LAB RECITATION 9 – Annika’s Documentation Blog

Recitation 9: Media & Controls

Processing Code

import processing.sound.*;
import processing.serial.*;

int NUM_OF_VALUES_FROM_PROCESSING = 3;  /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int processing_values[] = new int[NUM_OF_VALUES_FROM_PROCESSING]; /** this array stores values you might want to send to Arduino **/

Serial myPort;
String myString;

SoundFile sound;
// declare an Amplitude analysis object to detect the volume of sounds
Amplitude analysis;

void setup() {
  size(640, 480); 
  sound = new SoundFile(this, "DRUM.mp3");
  // create the Amplitude analysis object
  analysis = new Amplitude(this);
  // use the microphone as the input for the analysis
  analysis.input(sound);
  sound.play();
  
  setupSerial();
}      

void draw() {
  println(analysis.analyze());
  background(0);
  
  // analyze the audio for its volume level
  float volume = analysis.analyze();
  // map the volume value to a useful scale
  float diameter = map(volume, 0, 1, 0, 255);
  // draw a circle based on the microphone amplitude (volume)
  circle(width/2, height/2, diameter); 
  delay(10);
  
  processing_values[0] = int(diameter);
  sendSerialData();
}

void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[0], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

}

void sendSerialData() {
  String data = "";
  for (int i=0; i<processing_values.length; i++) {
    data += processing_values[i];
    //if i is less than the index number of the last element in the values array
    if (i < processing_values.length-1) {
      data += ","; // add splitter character "," between each values element
    } 
    //if it is the last element in the values array
    else {
      data += "\n"; // add the end of data character linefeed "\n"
    }
    
  }
  //write to Arduino
  myPort.write(data);
  print(data); // this prints to the console the values going to arduino 
  
}

Arduino Code

#include <Servo.h>
Servo servoRight;

// IMA NYU Shanghai
// Interaction Lab

/**
  This example is to send multiple values from Processing to Arduino.
  You can find the Processing example file in the same folder which works with this Arduino file.
 **/

#define NUM_OF_VALUES_FROM_PROCESSING 3    /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/


/** DO NOT REMOVE THESE **/
int tempValue = 0;
int valueIndex = 0;
int pitch;

/* This is the array of values storing the data from Processing. */
int processing_values[3];


void setup() {
  Serial.begin(9600);
  pinMode(3, OUTPUT);
  pinMode(9, OUTPUT);

  servoRight.attach(9);

  servoRight.write(0);

}

void loop() {
  getSerialData();
  servoRight.write(processing_values[0]);
}
// add your code here using elements in the values array

//receive serial data from Processing
void getSerialData() {
  while (Serial.available()) {
    char c = Serial.read();
    //switch - case checks the value of the variable in the switch function
    //in this case, the char c, then runs one of the cases that fit the value of the variable
    //for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase
    switch (c) {
      //if the char c from Processing is a number between 0 and 9
      case '0'...'9':
        //save the value of char c to tempValue
        //but simultaneously rearrange the existing values saved in tempValue
        //for the digits received through char c to remain coherent
        //if this does not make sense and would like to know more, send an email to me!
        tempValue = tempValue * 10 + c - '0';
        break;
      //if the char c from Processing is a comma
      //indicating that the following values of char c is for the next element in the values array
      case ',':
        processing_values[valueIndex] = tempValue;
        //reset tempValue value
        tempValue = 0;
        //increment valuesIndex by 1
        valueIndex++;
        break;
      //if the char c from Processing is character 'n'
      //which signals that it is the end of data
      case '\n':
        //save the tempValue
        //this will b the last element in the values array
        processing_values[valueIndex] = tempValue;
        //reset tempValue and valueIndex values
        //to clear out the values array for the next round of readings from Processing
        tempValue = 0;
        valueIndex = 0;
        break;
    }
  }
}

Finally, I got a servo motor that taps (swings drumsticks) to the rhythm of musical drums.

Open Source Sample Sounds: https://www.freesound.org/browse/
Open Source Archive Sounds: https://archive.org/details/audio
Open Source Music: http://freemusicarchive.org/

Use this week’s reading, Computer Vision for Artist and Designers, as inspiration to write a reflection about the ways in which technology was used in your recitation project.

The reading talks about an algorithm, the computer vision algorithm, which is now often used in interactive media. It analyzes and compares every pixel of an image or video and uses it as a condition to trigger events such as recognizing facial expressions and tracking the movement. This is somewhat related to the research I did in recitation 7. I did not use these complex algorithms in my recitation project, but I think this will be used in my final project. I want to use face recognition to let the mechanical eye track people’s face or eye movements to show the effect of “surveillance”.

Recitation 9: Media & Controls

Leave a Reply Cancel reply