Category: Interaction Lab

Posted on

Connor S. – Project Proposals

  1. “Beat Orchestra” 

“Beat Orchestra” gives aspiring hip-hop producers the opportunity to make rap beats in a fun and interactive way through the use of multiple sensors that serve as different instruments (eg. a physical kick drum on the ground one can stomp on to add a kick drum to their beat). This project would bring the fun of making music without forcing the monotony of the sole use of a monitor, mouse, and keyboard. This project could be used by people of all ages who are able to use their hands, arms, feet, etc. to engage with sensors to produce music for their and others’ enjoyment. After some research into music production, it turns out that most producers just use a monitor, mouse, and often an electric keyboard for all the instruments that go into a beat. This method sounds boring, repetitive, and alienating to those who don’t necessarily want to slouch over a computer to make music. With “Beat Orchestra,” anyone from small children to serious producers can start to see music production as an active activity. 

  1. “Soccer Skills Assister” 

    Imagine Dance Dance Revolution but for learning how to do stepovers and other soccer skills. Various sensors are placed around a stationary soccer ball which light up depending on where the user is supposed to place his/her feet to perform the intended skill. In front of the user is a screen with a person demonstrating how the move should be performed in real life. The user can change difficulties by selecting different skills, and deciding at what speed they want to perform them. By altering the delay between the different light up sensors, the speed at which the user performs the skill can change. I was not able to find anything like this after some research online, so this is probably a trillion dollar idea that I should probably patent sooner rather than later. The intended audience is essentially anyone with function in their legs. It’s pretty much like Guitar Hero but for soccer, but “Soccer Skills Assister” can actually teach you something useable on a soccer field unlike Guitar Hero which just teaches you how to push buttons. 

  1.  â€œSecurity Sensor”

Everyone has some concern for the safety of their personal belongings, and the “Security Sensor” is marketable to anyone with drawers containing valuables they wish to protect from stealers. One just places the “Security Sensor” inside any drawer that contains valuables, and the light sensor on the “Security Sensor” detects that the drawer has been opened, and sends a message to the owner, and snaps a photo of the perpetrator as well. When you want to open the drawer yourself, simply turn off the “Security Sensor” from your phone. After initially learning about light sensors, I immediately thought of a fridge, and how the light (allegedly) is off when you close it, and on when you open it. A light sensor would know for sure whether the light is on or off, so why not apply this to a closed drawer or cabinet?  

Posted on

Connor S. – Final Project Essay

“Soccer Skills Assister”

Physical interaction has seemed to have taken the gaming world by storm — systems like the Nintendo Wii and Xbox Kinect became massively popular upon their release, and demonstrate how marketable physically interactive systems can be. I really enjoy the idea of taking an experience or activity that would normally require a large amount of space, and shrink it down into something much more accessible, and equally stimulating. I was initially inspired by the soccer free kick simulator I referenced in a previous post as it played into my love of soccer, and sparked my interest in the concept of creating small-scale versions of things that require more space. My project follows a similar trajectory, which incorporates my love of playing soccer with the idea that people can have fun learning and playing the sport without leaving their home, and let alone going to a soccer field. The “Soccer Skills Assister” would give people the opportunity to learn soccer skills to better themselves as players, or get a bit of fun out of an exercise that may operate differently than pretty much anything else on the market right now. 

My plan for the project is essentially to first recreate the game Dance Dance Revolution using Arduino and Processing, and transposing it to be soccer related. My idea would of course include a soccer ball, and four cardboard sensor pads around the ball to the upper right, upper left, lower right, and lower left of the ball. After selecting the skill move the user wishes to learn from a menu on a computer in front of them, a video of someone doing the move appears on the screen, and replays as many times as the user needs to understand how the move is performed. When the user is ready, a countdown timer starts on the screen, prompting the user to get ready to attempt the move. When the countdown reaches zero, the pads on the ground will light up, directing the user to step on them in the same sequence they saw in the video on the screen. My idea is after a certain skill move is selected from the menu, the speed at which the user has to perform the move increases each time, until the fastest viable speed to perform the move is reached. The first round will be fairly slow, to allow essentially any able-bodied person to get  the basics of it, regardless of skill level. By the third or fourth round, the speed will have increased significantly, leaving only skilled players, or people who played the game a lot, a chance of succeeding. I plan on spending a substantial chunk of the next two weeks making sure my sensor pads work properly, because if people are unable to step on them or the sensors are not sensitive enough, then I think the whole thing is a total loss. Once I make sure the sensors are working properly for the project’s intended purpose, I will write the code for the game itself, creating the menu, making sure the pads light up correctly depending on the chosen skill, etc. 

The research I have done so far in relation to this project has essentially been limited to my admiration for games like Dance Dance Revolution in that they take the activity of learning how to dance and make it both easily accessible and fun. I thought a lot about my definition of interaction when drafting the idea for my final project, and concluded that out of the ideas I brainstormed, this one would be the most interactive. I personally think a soccer ball sitting on the ground would be reason enough for someone to approach my project, just out of curiosity. When the potential user approaches the project, my goal is to get them to say: “Hey, let me see if I can move my feet quickly like a pro soccer star” rather than “Eh it’s just a sports game and I don’t even like soccer, I’m not interested.” I think what makes my project particularly unique/significant is the fact that it can attract both people with an interest in soccer/athletics, as well as those without a strong interest in sports, but who are nonetheless somewhat interested in the idea of a quickness game where they can get some exercise and have fun. I’d say this project is also significant in that it can be made into both a casual, arcade style game, and a more serious soccer drill for players to develop their skills. If I were to continue improving upon this idea, I would probably make two versions of it: A casual, arcade soccer skills game, and a serious tool for coaches to buy for their players to use at practice.           

Posted on

Recitation 9: Media Controller – Elysia

In this week’s recitation, we were told to use Arduino and processing to manipulate the media attributes using the physical controller in Arduino. I decided to use an image to be manipulated. At first, i was using the position of the x and y of the mouse to change the size of the image, but i decided that it does not fit what i was going to do. I want to use two potentiometers to control two different things in the image. One potentiometer to control the length of the snow on the bottom of the image, the other one is for the snow that falls. I made the snow with for loops that we learned in class, but i made some modifications to make it seem smaller and use the value from the potentiometer to control the frequency of the for loop.

The code is attached below:

Arduino

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
/// int sensor3 = analogRead(A2);

// map(sensor1, 0, 1023, 0, 255);

// keep this format
Serial.print(sensor1);

Serial.print(“,”); // put comma between sensor values
Serial.print(sensor2);

// Serial.print(“,”);
// Serial.print(sensor3);

Serial.println(); // add linefeed after sending the last sensor value

// too fast communication might cause some latency in Processing
// this delay resolves the issue.
delay(100);
}

Processing

import processing.serial.*;
PImage photo;
String myString = null;
Serial myPort;


int NUM_OF_VALUES = 2;   /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues;      /** this array stores values from Arduino **/
//float psensorValue0;
//float psensorValue1;

void setup() {
  size(800, 800);

  photo = loadImage("image.jpeg");
  setupSerial();
}


void draw() {
    background(#74CFD8);
  updateSerial();
  printArray(sensorValues);
  float m = map(sensorValues[0], 0, 1023, 0, 100);
   float n = map(sensorValues[1], 0, 1023, 0, height); //left
  image(photo, 0, 0, width, height);
 for (int i=0; i<m; i++) {
    int size = int( random(1, 15) );
    int x = int( random(photo.width) );
    int y = int( random(photo.height) );
    // get the pixel color
    color c = photo.get(x, y);
    // draw a circle with the color
    fill(#FFFFFF);
    noStroke();
    ellipse(x, y, size, size);
    rectMode(CENTER);
    rect(400, 800, 800, n);
  //  filter(BLUR, m);


}}



void setupSerial() {
  printArray(Serial.list());
  myPort = new Serial(this, Serial.list()[ 3 ], 9600);
  // WARNING!
  // You will definitely get an error here.
  // Change the PORT_INDEX to 0 and try running it again.
  // And then, check the list of the ports,
  // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" 
  // and replace PORT_INDEX above with the index number of the port.

  myPort.clear();
  // Throw out the first reading,
  // in case we started reading in the middle of a string from the sender.
  myString = myPort.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
  myString = null;

  sensorValues = new int[2];
}



void updateSerial() {
  while (myPort.available() > 0) {
    myString = myPort.readStringUntil( 10 ); // 10 = '\n'  Linefeed in ASCII
    if (myString != null) {
      String[] serialInArray = split(trim(myString), ",");
      if (serialInArray.length == NUM_OF_VALUES) {
        for (int i=0; i<serialInArray.length; i++) {
          sensorValues[i] = int(serialInArray[i]);
        }
      }
    }
  }
}
Posted on

Recitation 9: Media Controller

For the Arduino, I used two potentiometers to adjust an analog value.

For the code: I used the multipleValues code, sending from Arduino to Processing. 

For Processing:  I used Image function and the sensor values are used to change the position of the image.

  • code in Arduino:

void setup() {
Serial.begin(9600);
}

void loop() {
int sensor1 = analogRead(A0);
int sensor2 = analogRead(A1);
Serial.print(sensor1);
Serial.print(“,”); // put comma between sensor values
Serial.println(sensor2);
delay(100);
}

  • code in Processing:

import processing.serial.*;

String myString = null;
Serial myPort;
PImage photo;

int NUM_OF_VALUES = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/

void setup() {
size(1300, 1300);
background(255);
setupSerial();
photo = loadImage(“1.jpg”);
}

void draw() {
updateSerial();
background(255);
printArray(sensorValues);
float x= sensorValues[0];
float y= sensorValues[1];
image(photo, x, y);
}

void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);

myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
myString = null;

sensorValues = new int[NUM_OF_VALUES];
}

void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}

Posted on

Recitation 9 Media Controller Chloe Wang

In this recitation, we incorporated the use of images in the interaction between Processing and Arduino. In this week’s class, we learned how to put an image in processing and manipulate the image using values in Arduino. This week, I built my recitation project based on last week’s project of Etch a Sketch. First, I took the first step from last week’s Etch a sketch where I got an ellipse to move according to me turning two potentiometers. 
Here is what I had last week:
Then in my codes, I added codes for PImage:  photo = loadImage(“cat.jpg”);

and dragged a picture of my cat in Processing.  At first, my goal was to have a blurred picture of my cat on top of the clear picture, and wherever the ellipse moves, that part of the canvas would show the photo beneath rather than the blurred picture on top. I used the maskImage function in doing this. At the time, blurring a photo would take some time, so I added another photo that has a similar size with

  maskImage = loadImage("mask.jpg");
  photo.mask(maskImage);
  maskImage.resize(652, 484);

But the program still did work as I wished. I did not screenshot the result, but it would show a picture of my cat with some colors distorted. The ellipse on the canvas would leave a faded trial behind as it moves around. Later I realized that the mask would only be used when the mask image needs to contain only greyscale data. So with some help, I changed the maskImage to black with maskImage.set(x, y,255); and a rectangle would move around to reveal the photo beneath. With set function, it changes the color of those pixels of the selected area.  For the rectangle that is moving around, I (with help) used two for functions

  for (int y=200; y<300; y++) {
    for (int x=200; x<300; x++) {
      maskImage.set(x, y,255);
   }
  }
 and 
  int a=round(map(sensorValues[1],0,1023,0,height));
  int b=round(map(sensorValues[0],0,1023,0,width));
  
  for (int y=a; y<a+50; y++) {
    for (int x=b; x<b+50; x++) {
      maskImage.set(x, y,255);
    }
  }
in set up and draw. 
This way, when the rectangle moves around the black canvas as I turned the two potentiometers on Arduino, the rectangle would only reveal the part of the image it is at. Using int a and int b and the map function, I finally gave a boundary to where the ellipse can freely move. Now the ellipse would be on the canvas at all times.
From doing this exercise in class, I also reflected on my final project. Rather than having a blurred picture representing the things we don’t know about, a black screen and just a small part of visible image would be more representative of our actual experience with modern media that we don’t see most of the things we comment on. After reading Computer Visions for Artists and Designers, it provided me with another idea with “simple object tracking”. If this rectangle was generated by an analog source of flashlight, maybe I could step up the interactivity of my project to have people “looking for the truth in the dark” with an actual flashlight and black screen. 
 
Here is the complete code to the final version:
 
Here is a video of my final result:
Here is another video of my final result that could lead to another direction for my project:
 
Just in case you want to know, this is the picture I used in this exercise.
Image used in Recitation 9
Image used in Recitation 9

Reference:

Levin, G. and Collaborators (2012) ‘Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers’. FLONG. Available at: https://drive.google.com/file/d/1NpAO6atCGHfNgBcXrtkCrbLnzkg2aw48/view.