Recitation 10: Workshops – Ariana Alvarez

For this week’s recitation, after the map() function workshop, I chose to attend the media manipulation workshop, as it was what aligned the most with my project. What I wanted to work on in this workshop, was to learn how to manipulate pixels in webcam. 

Initially, I attempted to change directly the RGB colors in the webcam, as during the workshop I was told that there may not be the possibility of adding a filter to it (similar with an image). As this process wasn’t being effective in creating a negative image effect,  I did some research and it was possible to add filters to webcam with cam.filter() function. 

After adding an inverse black and white filter effect on the webcam, I also attempted to make the image brighter and darker by manipulating the HSB values of the pixels. It was quite challenging, however this media manipulation workshop provided me with a better head-start towards my project and allowed me to explore further ways in which pixels could be manipulated in webcam through processing. 

The code was the following:

//int r = 50;
//int g = 50;
//int b = 50;

import processing.video.*; 
Capture cam;

//color invertColor( int r, int g, int b) {

//  return color(255 - r, 255 - g, 255 - b);
//}

void setup() {  
  size(640, 480); 
  colorMode(HSB);
  cam = new Capture(this, 640, 480);
  cam.start(); 
} 
 void draw() { 
   
   
   
  if (cam.available()) { 
   cam.read(); 
   image(cam, 0, 0); 
   cam.filter(GRAY);
   cam.filter(INVERT);
      //background(invertColor(r,g,b));
  } 

  cam.loadPixels();
       
//Pixels, code with Arduino Distance Sensor
  noStroke();
  int rectSize = 10;
  int w = cam.width;
  int h = cam.height;
  
  for (int y = 0; y < h; y+=rectSize) {
    for (int x = 0; x < w; x+=rectSize) {
      int i =  x + y * w;
      
      fill( cam.pixels[i] );     
      rect(x, y, rectSize, rectSize);
      
      
    //for (int y = 0; y < h; y++) {
    //  for (int x = 0; x < w; x++) {
    //    int i =  x + y*w; // *** IMPORTANT ***
    
        float b = brightness(cam.pixels[i]); 
        float s = saturation(cam.pixels[i]);
        float u = hue(cam.pixels[i]);
        float ch = map(mouseX, 0, 255, height, width);
        cam.pixels[i] = color(u, s, b+ch); 
     
      }
   
    cam.updatePixels();
  }
  }
 //   }
 //}

Recitation 10 media manipulation workshop (Katie)

In this recitation, I chose to attend the media manipulation workshop.  The thing I want to work out in this workshop is how to switch scenes. For example, to switch from the webcam to the video. I want to add a keypress function to achieve but the problem is that the scene only changes when I press the key. After I release the key, it turns back to the webcam. To solve this problem, I add a boolean to trigger the video.

Recitation 10: Workshops Documentation by Eleanor Wade

Serial Communication Workshop:

Using a button on the Arduino as a sensor to control the sketch on processing.  We used multiple value Arduino to Processing sample code.  

This recitation was very helpful in learning some necessary techniques in serial communication, with sensors that are beyond that of a potentiometer.  This will definitely be useful when moving forward with my final project as I will  definitely be using a color sensor to translate colored tags into specific factual information and images on the screen.  In this way, it was particularly beneficial to me to be able review the information that we previously had learned regarding arduino to processing.  

Processing Code: 

// IMA NYU Shanghai
// Interaction Lab
// For receiving multiple values from Arduino to Processing

/*
 * Based on the readStringUntil() example by Tom Igoe
 * https://processing.org/reference/libraries/serial/Serial_readStringUntil_.html
 */

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/


void setup() {
  size(500, 500);
  background(0);
  setupSerial();
}


void draw() {
  updateSerial();
  printArray(sensorValues);

  // use the values like this!
  // sensorValues[0] : pot 0 - 1023
  // sensorValues[1] : button switch 0, 1

  // add your code
  float posX = map(sensorValues[0], 0, 1023, 0, 500); 
 int size;
 if (sensorValue[1] == 0) {
   size = 50;
 } else { 
   size = 200;
 }
 
  
  
ellipse (300, 300, size, size);
  //
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 1 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

One value Processing to Arduino Serial Communication: 

Arduino Code:

// IMA NYU Shanghai
// Interaction Lab
// This code receives one value from Processing to Arduino

#include <Servo.h>
Servo myservo;
int val;

char valueFromProcessing;
//int ledPin = 13;

void setup() {
Serial.begin(9600);
myservo.attach(9);
}

void loop() {
// to receive a value from Processing
while (Serial.available()) {
valueFromProcessing = Serial.read();
}

val = valueFromProcessing;
val = map(val, 0, 500, 0, 180);
myservo.write(val);
delay(15);

// if (valueFromProcessing == ‘H’) {
// digitalWrite(ledPin, HIGH);
// } else if (valueFromProcessing == ‘L’) {
// digitalWrite(ledPin, LOW);
// } else {
// something esle
// }

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(10);
}

In using the map function to map the values of canvas size, we specifically changed the code to this:

val = map(val, o, 500, 0, 180);

Recitation 10 Media Manipulation Chloe Wang

In this recitation, I decided to do the media manipulation workshop. For the workshop, we needed to choose a music video or any video clip and recreate it with processing.  I chose a clip of my favorite Chinese TV show as the original video to work on. I downloaded the video and renamed it to liuxing.mp4. Liuxing is one of the main character’s name. At first, I was confused because I didn’t know how to incorporate the map() function in the video. So my first action was to loop the video to make it play over and over again. 
I also wanted to make the video pixelated when mouse is pressed because this is a quite old TV series with low resolution. But when I watched it while I was young, I remembered it to have rather a high resolution. 
To use the map function, I made the video to change speed according to where the mouse is placed on the canvas. To change the speed, I used a float i function, i is the speed, and it is changed according to the map. 
float i =map(mouseX,0,mouseY, 0,4);
myMovie.speed(i);
Here is the final result of my media manipulation project:
https://youtu.be/erkvH7dAsC4 
Here is my code:
import processing.video.*;

Movie myMovie;
void setup() {
  size(636, 360);
  myMovie = new Movie(this, "liuxing.mp4");
  myMovie.play();
  myMovie.loop();
  //define the state of the video
}
void draw() {
  if (myMovie.available()) {
    myMovie.read();
    //read the current frame of my video
  }
  image(myMovie, 0, 0);
  //draw the current frame of the video
float i =map(mouseX,0,mouseY, 0,4);
myMovie.speed(i);

if (mousePressed){
int rectSize = 10;
  int w = myMovie.width;
  int h = myMovie.height;
  for (int y = 0; y < h; y+=rectSize) {
    for (int x = 0; x < w; x+=rectSize) {
      int a =  x + y * w;
      fill( myMovie.pixels[a] );
      rect(x, y, rectSize, rectSize);
    }
}}
}

Source:
 

Recitation 10–Vivien Hao

For this recitation, I chose to do the media one. I have a lot of videos from different concerts. I have chosen one video and then insert it in this code. I want to use the videos that I have taken in the past. And by using potentiometer, I could control the speed of the video. The following codes are for Arduino and Processing. 

Arduino:

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing
void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
Serial.print(sensor1);
Serial.println();
delay(100);
}

Processing:

import processing.video.*;
Movie myMovie;
import processing.serial.*;
Serial myPort;
int valueFromArduino;

void setup() {
  size(400, 400);
  myMovie = new Movie(this, "ljj.mp4");
  myMovie.play();
   printArray(Serial.list());
  // this prints out the list of all available serial ports on your computer.

  myPort = new Serial(this, Serial.list()[ 2 ], 9600);
  //define the state of the video
}
void draw() {
  while ( myPort.available() > 0) {
    valueFromArduino = myPort.read();
  }
  println(valueFromArduino);
  float x = map(valueFromArduino,0,1023,0,93);
  if (myMovie.available()) {
    //read my file when it can be read
    myMovie.read();
    //read the current frame of the video
  }
  //draw the frame
  if (mousePressed){
   if (mouseX<width/2){
     myMovie.jump(myMovie.time()-x);
     myMovie.read();
   }else{
     myMovie.jump(myMovie.time()+x);
     myMovie.read();
   }
  }
  image(myMovie, 0, 0);
}