Recitation10: Documentation

This recitation asked us to build a project which uses physical controller to manipulate a video.  Instruction: https://wp.nyu.edu/shanghai-ima-interaction-lab/recitation-10-image-video/

I decided to focus on the .jump() and .pause() functions, which can generate an interaction effect that when users press the button, the video would pause, during which users can control the potentiometer and travel the frames of the video. 

Step1: Arduino Circuit and Codes:

The Arduino should send the state of button and potentiometer via SerialPort to the computer. All the big button were still embeded in our midterm project (we havn’t found a time to dismantle it) So I used the small button in Arduino Kit. The potentiometer was the same. I quickly built the circuit and wrote the code and tested it.

 

The First test showed that there’s something wrong with my button circuit. I’m sure that the circuit was same for the big button, but probably the big and small buttons are different innerly. I consulted Steve, my friend who are in different section and he corrected the circuit for me. After class I used an LED to test the inner part and found that they are different. Here is the video records the expected outcome:

Code:

 #include "SerialRecord.h"

SerialRecord writer(2);
//potentiometer, back, image1, image2
void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  pinMode(8,INPUT);
}

void loop() {
  // put your main code here, to run repeatedly:
  int value1 = analogRead(A0);
  int value2 = digitalRead(8);
  // Serial.println(value2);
  writer[0] = value1;
  writer[1] = value2;
  writer.send();
  delay(20);
}

Step2: Processing Coding:

I used the Jin’an.mp4 file in the lecture because I thought processing loads small files more quickly so helpful for debugging. 

I consulted professors and found that the jump() and read() function manipulate on time not frame number. 

I selected the code from example and merged it to the sketch. The video could play:

 

I wrote the code for pause. I added a boolean variable “pause” which shows whether the video is pause or not because I didn’t want users to control the video when it is playing. The first test was wrong because I intiated the “pause” as true and the logic would crash. 

 

Users still could not control the video with potentiometer. I modified the sample code and did experiment on the jump() function and found that the jump() function only works when video is playing. Here is the video for my experiment:

 

So I change the code in the if(!pause) module. It still looked weird because I wrote “jump(myVideo.read()+value)”. The myVideo.read() function would return the value synchronizely, so if the value is not 0, it would change in every draw() loop. I added a variable named “cur” to store the original time and I finally generate the interactive effect I want. Here is the final demo:

 

Processing Code:

import processing.video.*;
import processing.serial.*;
import osteele.processing.SerialRecord.*;


Serial serialPort;
SerialRecord serialRecord;
Movie myMovie;

void setup() {
  size(512, 288);
  myMovie = new Movie(this, "Jing'An.mp4");
  myMovie.loop();
  String serialPortName = SerialUtils.findArduinoPort();
  serialPort = new Serial(this, serialPortName, 9600);
  serialRecord = new SerialRecord(this, serialPort, 2);
}

int preb = 0, b;
float step;
boolean pause = false;
float cur;

void draw() {
  if (myMovie.available()) {
    myMovie.read();
  }
  serialRecord.read();
  int value1 = serialRecord.get(0);
  int value2 = serialRecord.get(1);
  
  b = value2;
  step = map(value1,0,1023,-0.5,0.5);
  
  if(b == 1){
    if(preb == 0){
      if(pause){
        myMovie.loop();
        pause = false;
      }else{
        myMovie.pause();
        pause = true;
        cur = myMovie.time();
      }
    }
    preb = 1;
  }else{
    if(preb == 1){
      preb = 0;
    }
  }
  
  if(pause){
    myMovie.play();
    myMovie.jump(cur+step);
    myMovie.pause();
  }
  image(myMovie, 0, 0);
  text(str(myMovie.time()),20.,200.);
}