Recitation 10: Image & Video

During this recitation, we practiced the serial communication again and linked sound with image.

For the first exercise, surprisingly and not so surprisingly, the most difficult part turns out to be finding the picture haha. My nickname in middle school was Peppa pig, so I found a picture of it.

My code is below:

Arduino:

void setup()
{
  Serial.begin(9600);
}

void loop() 
{
  int sensor1 = analogRead(A0);
  int sensor2 = analogRead(A1);    
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.println(); // add linefeed after sending the last sensor value
  delay(100);
}

Processing:

import processing.serial.*;


int NUM_OF_VALUES_FROM_ARDUINO = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int sensorValues[];      /** this array stores values from Arduino **/

String myString = null;
Serial myPort;

PImage photo;


void setup() {
  size(800,800);
  background(#7BE5B1);
 
  photo = loadImage("Peppa pig.jpeg");
  printArray(Serial.list());
  setupSerial();
}

void draw() {
   background(#7BE5B1);
   getSerialData();
  printArray(sensorValues);
  
  float x= map(sensorValues[0],0,1023,0,width);
  float y= map(sensorValues[1],0,1023,0,height);
  
  image(photo, x, y);
  
  }
    
    
  void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[2], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO];
}

void getSerialData() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

Video:

For the exercise2, it was quite tricky, and my processing got stuck thousands of times. I went to office hours to ask for helps, and Skye explains to me the function of abs(), R, PR and so on. I found the final result very interesting, the effect looks like the wall in swimming pool and the sound also somehow reminds me of that as well.

My code is below:

 

import processing.sound.*;
import processing.video.*;
SinOsc sine;
Env env;
String[] cameras = Capture.list();
Capture cam;
color myColor;
float attackTime = 0.001;
float sustainTime = 0.004;
float sustainLevel = 0.3;
float releaseTime = 0.4;
float d;
float f;
int s=20;
float R;
float PR;
void setup() {
  size(640, 480);
  cam = new Capture(this, cameras[1]);
  cam.start();
  printArray(cameras);
  // Create the sine oscillator.
  sine = new SinOsc(this);
  env  = new Env(this);
}
void draw() {
  background(0);
  if (cam.available()) {
    cam.read();
  }
  myColor=cam.get(width/2, height/2);
  R=red(myColor);
  f = map(R, 0, 255, 100, 600);
  float d = abs(R-PR);
  sustainTime = map(d, 0, 100, 0.001, 1);
  sustainTime = constrain(sustainTime, 0.001, 1);
  println(d);
  for (int x=0; x<width; x=x+s) {
    for (int y=0; y<width; y=y+s) {
      color c=cam.get(x, y);
      float red = red(c);
      float green= green(c);
      float blue = blue(c);
      float factor = map(red, 0, 255, 0.05, 2);
      noStroke();
      fill(red, green, blue, red);
      ellipse(x, y, s*factor, s*factor);
    }
  }
  // get the pixel color
  if (d>20) {
    sine.play();
    sine.freq(f);
    env.play(sine, attackTime, sustainTime, sustainLevel, releaseTime);
  }
  PR=R;
  //image(cam, 0, 0);
}

Video: