Category: Interaction Lab

Posted on

Recitation 9: Media Controller-Azena

RESITATION:

Sorry for post so late. I realized my post was missing at Sunday, while those days I could not spare time to make it up. This is my second time recitation 9, hope you can forgive my late.

Arduino:

void setup() {
Serial.begin(9600);

}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
int sensor3 = digitalRead(9);

// keep this format
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);
Serial.print(“,”);
Serial.print(sensor3);
Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Processing:

import processing.serial.*;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 3;   
int[] sensorValues;     
PImage img;
float j=0;
float i=0;
float x=0;
float y=0;
float speedX=10;
float speedY=10;


void setup() {
  size(1000, 1000);
  background(0);
  setupSerial();
  img = loadImage("pumpkin.png");
}


void draw() {
  updateSerial();
  printArray(sensorValues);

 
  imageMode(CENTER);
float j=map(sensorValues[0],0,1023,0,500);
  float i=map(sensorValues[1],0,1023,0,10);
  
    
 
  background(0);
 
  
image(img, x, y, j*1.2, j*1.2);
filter(BLUR, i);

if (sensorValues[2]==0 ){
  x=300;
  y=300;
}else{
x=x+1.5*speedX;
y=y+0.8*speedY;
if(x>width||x<0){
  speedX=-speedX;
}
if(y>height||y<0){
  speedY=-speedY;
}
}

  //
}



void setupSerial() {
  printArray(Serial.list());
   myPort = new Serial(this, Serial.list()[ 3], 9600);
  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil(10); 
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

I’d like to use two the potentiometers and a Botton to control the movements of image. At the first, I checked the code of the button online and have no idea how to move the digital Read(); to the processing, after asking help from assistant, I found out using sensorValue(); can transform all into processing;

Posted on

Recitaition 9 by Tya Wang (rw2399)

For this week’s exercise, I created a program that changes the color of an image based on the value sent back from Arduino. The program works like this:

The color changes smoothly because I changed the colorMode into HGB and mapped value sent back by a potentiometer, which is between 0 and 1023, into between 0 and 360.

I think this program can be used in the future to make a digital color filling book, where you choose a section and then decide what color you want to fill in this section. This may help people reduce pressure when playing with it and creat their own art work.

Here is the code on processing attached:

import processing.serial.*;
PImage photo;


Serial myPort;
int valueFromArduino;


void setup() {
  size(500, 413);
  photo = loadImage("bob.jpg");
  photo.loadPixels();
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[0], 9600);
}


void draw() {
  tint(valueFromArduino, 100, 100);
  image(photo, 0, 0);
  
}
Posted on

Recitation 9—Vivien Hao

For this recitation exercise, I have decided to include one of my favorite song cuts. While I move the potentiometer, the image of the video would change a little bit. 

This is my Processing code:

import processing.serial.*;
import processing.video.*;
Movie myMovie;

String myString = null;
Serial myPort;


int NUM_OF_VALUES = 1;   
int[] sensorValues;      


void setup() {
  size(500, 500);
  myMovie = new Movie(this, "JJ Lin.mp4");
  myMovie.play();
  setupSerial();
}


void draw() {
  updateSerial();
  printArray(sensorValues);
  float hcy = map (sensorValues[0],255, 1023, 0,100);
  if (myMovie.available()) {
    myMovie.read();
  }
  tint(hcy, random(100), random(800)); 
  image(myMovie, 0, 0);
}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 1 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 5 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[NUM_OF_VALUES];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); 
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

This is my Arduino code

// IMA NYU Shanghai
// Interaction Lab
// For sending multiple values from Arduino to Processing
void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
Serial.print(sensor1);
Serial.println();
delay(100);
}

Posted on

Recitation 9: Media Controller – Ariana Alvarez

For this week’s recitation, we were assigned to manipulate media in Processing through Arduino. I decided to explore the live video webcam in processing and manipulate the tint function in live images with the help of a potentiometer and infrared distance sensor in Arduino. 

For the first attempt in the code, I was able to change the opacity in the tint applied to the live video through the potentiometer. However, as I wasn’t redrawing the background in processing, the opacity was just change upon itself, and led to the creation of an even more interesting effect of blurred image, that made any user on the screen have some kind of a ghostly look.

The second manipulation, was inspired by the readings in “Computer Vision in Artists and Designer”, as it mentioned how algorithms and computing media has been used in detecting motion, specially that which involved “the movements of people”. Therefore, I developed a code that changed the tint of an image from blue to red, depending on how close an individual was situated from the infrared distance sensor in Arduino. 

For the codes of both iterations of media manipulation, I was going to use the multiple values example so that both sensors in Arduino were connected to processing. However, I first did it with the one value example, and when changing it to multiple values, both sensors were not working in the most efficient way while working simultaneously, therefore I changed it back to one value codes for the time provided. Here I am attaching the codes I used for both Arduino and Processing.

I was inspired in the ways technology was used in my project, specially in the sense that I felt as if I created an object that helped enhance security systems in stores. Similar to the idea of the game system LimboTime, which was developed for participants to attempt and pass below an imaginary line. If such individual crosses above the line, the game rings an alarm. Similar to this idea, for my second media manipulation, if a person went passed the allowed distance, the colors started to change in the webcam.

Code from Arduino

void setup() {
Serial.begin(9600);
}

void loop() {
int pin1 = analogRead(A0);
int sensorValue = map(pin1,0,1023,0,255);
Serial.write(sensorValue);

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(10);
}

Code from Processing

import processing.serial.*;


Serial myPort;
int valueFromArduino;


import processing.video.*; 
Capture cam;


void setup() { 
  size(1280, 480); 
  cam = new Capture(this, 640, 480);
  cam.start(); 
  
  
  printArray(Serial.list());
  // this prints out the list of all available serial ports on your computer.

  myPort = new Serial(this, Serial.list()[4], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.
  
} 

void draw() { 
  //background(255);
  if (cam.available()) { 
   cam.read(); 
  } 
     
  int x = valueFromArduino;
  scale(-1,1);
  image(cam, -640, 0); 
  scale(-1,1);
  image(cam, 640, 0);
  tint(0, 153, 204, x); 
 

//  if (valueFromArduino < 100) {
//      tint(0, 153, 204);
//  } else { 
//    tint(255,0,0);
//  }
  
   // to read the value from the Arduino
  while ( myPort.available() > 0) {
    valueFromArduino = myPort.read();
  }
  println(valueFromArduino);//This prints out the values from Arduino
}

*Side Note: I have been trying to include images of the circuit and pictures of the media manipulation, however it is not allowing me to do so as it says there’s an error with the images, therefore I’ll try it again tomorrow and if not possible I’ll send them through e-mail directly to Rudi*

Posted on

Final Project Essay

PROJECT NAME: 侬说啥？(What do you say?)

PROJECT STATEMENT OF PURPOSE 

I was thinking about doing my final project about learning Chinese dialects and having these native speakers of the dialect be able to have a visual for the learners. I want to do this project as well to raise awareness about the dying population of local dialect speakers. Fewer and fewer children are learning their local dialect because schools are only teaching Mandarin. Many parents are also not teaching or talking to their children in the local dialect because they want to make sure their children have the best chance in school with no issues of getting confused with words. The Chinese section of the gaokao also only tests on the standard Mandarin, so it further promoting the killing of the local dialects. As I was brainstorming how to address this issue, with the help of the presentation, I decided to focus this project on children, therefore making specific design choices that would entice children to play with the project.

 PROJECT PLAN 

This project aims to intrigue children to learn the dialect of their families. To do this, I will create a puzzle of China with the different provinces being able to be taken out so the child user can hear the dialect of that region of China. However, instead of randomly them picking, I also want to use a joystick so when they move into the province on the screen, they will hear the dialect in whispers (a lower volume) and looped. When they lift the piece up, the volume will be louder and can clearly hear the dialect. To get the dialects for this project, I have enlisted the help of native speakers from NYU Shanghai’s Class of 2022 and 2023 to be recorded. Interestingly, while finding native speakers for the recordings, finding people who spoke the dialect was hard because of this issue of only learning Mandarin. At the time of submission of this essay,  I have found native speakers from Sichuan, Chongqing, Shanghai, Guangdong, Fujian, and I am waiting on a reply from confirmed speakers from Beijing and Hebei. I am working on finding native speakers from other areas to try to get as many recordings as possible. In addition, I have finished designing the base of the puzzle and the puzzle pieces for the China puzzle. 

CONTEXT AND SIGNIFICANCE 

Talking with people about the pressure they faced when preparing for gaokao for the Chinese section and where they’re from inspired my want to address this issue. It was very interesting when I was doing preparatory research for this project after submitting the original proposal because when I asked friends who came from certain regions if they spoke the native dialect of the region, it proved difficult because oftentimes, they did not speak their regional dialect. For me, I thought this was weird because I speak my American Northeast regional dialect which isn’t as different as Chinese dialects are, but when we were taught English in school, we were taught specifically to the region. I think it’s just a cultural difference, but it seems very different for me. I think this project achieves interaction because it forces the user to give input to the project and the project will give output in which the user then acts on. I think this project is significant because I think this is one of the topics swept underneath the rug when talking about the unification of the Chinese language. This project is intended for young children so I wanted to make the interface as easy as possible for them to interact with, hence the puzzle. This is of special value to Chinese children to give them an interest in learning about their native region’s dialect. After successful completion, I would like to find out how this has affected culture because culture is often tied to language. I think this will help to address the idea of culture wiping that China has been doing with its minorities and the strength that comes from keeping your culture.