Author: Mimi Cai

Conception and Design
Basically, our project is a sound game, which requires players to get out of a darkness maze through the sound instructions, and through those sound instructions only. It appears to us nowadays, more and more visual art projects are created and people rely heavily rely on their eyes to achieve the process of interaction. The rationale is often that people use their eyes to take in information, hence making the decision. To this extent, my partner and I think it would be very interesting if we could create a game in which decision making is totally based on the sounds rather than visual pictures. We originally thought about using a joystick to control the movement in the game, but we found it can be confusing for the players to figure out the correct direction of the joystick. In contrast, directly using the arrow keys is much more convenient, hence we decide to replace the joystick by the arrow keys. However, to add to the level of interaction and sort of externalize the sound instructions from the computer, we decided to use a button to play the instructional melodies. If the player is in the correct direction, the volume of the music would grow louder. The players also need to pay attention to the sound of snaps, which represent the monsters. If they go toward the direction of the finger snap sound, there will be a jump scare and it will show that they fail the game. To some extent, I think this game is very unique, and it serves its purpose, which is to draw importance not only to our eyes but also to our ears as a sensory organ. Additionally, the game requires players to closely focus only on the sound, which can be a little bit hard, as the world today has so many distractions that reduce our concentration.

Desigen of the map: (the red area means the monster, if the player goes into the red area, he will lose the game)

Fabrication and Production
During the production process, I found the design of the game is quite difficult. Firstly, we need to decide whether this is a horror game or a music game. As we decided that designing a horror game will be more exciting and impressive, we also need to come up with a way that the players might fail the game. After we decided that there will be monsters that can make players fail the game, the design of the map become the next tricky thing. Since we are only able to separate left and right soundtrack, we initially decided that the players will only need to make their decisions on whether to turn left or turn right, whereas the straight road will be automatically controlled. However, we later found that the game is not challenging, hence not entertaining enough. Therefore, we decided to make things more tricky by letting players to find all their ways out in pure darkness. The only instructions they will have are the sounds of both the melody (which indicates the correct direction) and the finger snap (which indicates the direction of monsters). Although we decided that this would be a horror game, we try to involve some humor and abstraction into it, which is why we did not use the real sound of beasts but rather the sound of a finger snap. We also designed the game screen to be purely black apart from the spot where the player is, as we want to add to the difficulty of the game and simplize the idea of the maze.

During the user testing process, we discovered that using earphones might not be the most efficient way to engage multiple people with the game. We originally chose to use earphones so as to enable users to better perceive the soundtrack. However, we also discovered during the user testing that when one person is playing, others would have no idea what was going on. Also, since our game might take a long time for a single player, it might be boring for those who are waiting to test our game. Therefore, after the user testing, we decided to replace the earphones by stereophony, which can engage multiple people into the game at the same time.

The biggest problem that appears during the user test, however, is that people cannot understand the rules of this game without our explanation, even after we added a tutorial after the first user testing. Additionally, because there are two types of sounds that serve different purposes, people are easily confused by what they are for. People also tend to avoid the sound of the monster. As long as they hear the sound of the monster, they stop going toward that direction anymore. But the truth is, instead of avoiding the monsters, players have to go past them to get to the exit, meaning that you will definitely hear the sound of the monster constantly, but you can choose your way not to bump into them. Therefore, before the ima show, we polished the tutorial to make it simpler and changed the description to make it easier to understand.

For the game itself, we designed nine separate screens including information, warning, jumpscare, game and winning screens. We used the scale function to achieve the jump scare effect, which is essential to our game. When playing the game, the player can see a small ellipse on the screen that represents their current position. We use the X position and Y position of the center of the ellipse to decided the state of the player and play different screens accordingly. For the sound of the monster and instruction, we need to adjust it to the single soundtrack in advance, and then use the pan function to decide which soundtrack will it be played. We also need to keep balancing the volume of the sounds, as sometimes the sounds are very light but the sounds of jumpscare are very loud. But although we had encountered many challenges, we managed to finish the coding part with the help of those brilliant fellows in the lab.

Conclusion
The goal of our project is to use hearing rather than eyesight as the main way of interaction, which is quite unique as most of the interaction are based heavily on visual arts. Also, using the sound like the only instruction to help players get out the darkness actually helps people to concentrate and ignore all the distractions from the environment. To some extent, this project can even stimulate the way that those blinds perceive the world. I think this project result aligns with my definition of interaction which includes entertainment and communication between human and computer. However, currently, our project does not involve the other parts of the body much and is only limited to hearing. Although our purpose is to focus on the hearing, we also hope that the project can engage other parts of the body to be more interactive. Therefore, if I have more time to do this project, I would like to consider how to engage the rest parts of the body into this game as well. Through the challenges we met during this process, as well as the adjustments we have made, I learned that when designing a project, it is always important to take the environment into consideration, so as to check whether the idea works out effectively not only for the users but also for those people around it. It is also important to conduct more user testing, so as to keep moderate the difficulty level of the game and figure out a balance between entertaining and challenging.

Videos

Code

Arduino

Processing:

allGenerated:

import processing.serial.*;

Serial myPort;
int val;

import processing.sound.*;
SoundFile file1;
SoundFile file2;
SoundFile file3;
SoundFile file4;
SoundFile file5;
SoundFile file6;
SoundFile file7;
SoundFile win;
SoundFile mons;
SoundFile scream;
SoundFile tut1;
SoundFile tut2;
SoundFile tut3;

int sentinel = 0;
int rad = 10;
int PosX= rad;
int PosY = 750 – rad;
int PosXt= 720;
int PosYt = 770;
float speed = 4;
color b = color(0);
PImage map;
PImage map2;
PImage tardis;
PImage freddy;
PImage chicken;
PImage spring;
PImage foxy;
PImage toyBonnie;
PImage BB;
float size = 0.6;

void setup() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 2 ], 9600);

size(1440, 790);
map = loadImage(“map.jpg”);
map2 = loadImage(“map2.jpg”);
tardis = loadImage(“tardis.jpg”);
freddy = loadImage(“freddy.jpg”);
chicken = loadImage(“chicken.jpg”);
spring = loadImage(“springtrap.jpg”);
foxy = loadImage(“foxy.jpg”);
toyBonnie = loadImage(“toyBonnie.jpg”);
BB = loadImage(“BB.jpg”);
file1 = new SoundFile(this, “sent1.aif”);
file2 = new SoundFile(this, “sent2.aif”);
file3 = new SoundFile(this, “sent3.aif”);
file4 = new SoundFile(this, “sent4.aif”);
file5 = new SoundFile(this, “sent5.aif”);
file6 = new SoundFile(this, “sent6.wav”);
file7 = new SoundFile(this, “sent7.wav”);
win = new SoundFile(this, “Seven Seas Of Rhye.wav”);
mons = new SoundFile(this, “knock.aif”);
scream = new SoundFile(this, “scream.wav”);
tut1 = new SoundFile(this, “breakFree.wav”);
tut2 = new SoundFile(this, “underPressure.wav”);
tut3 = new SoundFile(this, “noStop.wav”);
}

void draw() {
if (sentinel == 0) {
initScreen();
} else if (sentinel == 1) {
gameScreen();
} else if (sentinel == 2) {
bearjump();
} else if (sentinel == 3) {
chickenjump();
} else if (sentinel == 4) {
springtrap();
} else if (sentinel == 5) {
foxjump();
} else if (sentinel == 6) {
gameOverScreen();
} else if (sentinel == 7) {
winningScreen();
} else if (sentinel == 8) {
warnScreen();
} else if (sentinel == 9) {
infoScreen();
} else if (sentinel == 10) {
tutorial();
} else if (sentinel == 11) {
rabbitjump();
} else if (sentinel == 12) {
bbjump();
} else if (sentinel == 13) {
TutOverScreen();
} else if (sentinel == 14) {
TutContinue();
}
}

void initScreen() {
background(0, 0, 0);
textAlign(CENTER);
fill(#F5453B);
textSize(70);
text(“Darkness Rhapsody”, width/2, height/2);
textSize(15);
text(“Click to start”, width/2, height-30);
}

void warnScreen() {
background(0, 0, 0);
textAlign(CENTER);
fill(#F5453B);
textSize(70);
text(“Warning:”, width/2, height/3);
textSize(50);
text(“This game contains jumpscares that”, width/2, height/3 + 60);
text(“some may find disturbing.”, width/2, height/3 + 120);
textSize(30);
text(“Click to go on”, width/2, height/3 + 200);
}

void infoScreen() {
background(255);
textSize(50);
fill(#CB7C2B);
textAlign(CENTER);
text(“How to get out of the maze:”, width/2, height/6);
textAlign(CORNER);
textSize(35);
text(“Use arrowkeys to move;”, width/10, height/5+60);
text(“Listen to the sound of finger snap,”, width/10, height/5+120);
text(“Do not head for the direction it comes from.”, width/10, height/5+170);
text(“Press the button at every crossroads”, width/10, height/5+230);
text(“Follow the direction where the melody comes from.”, width/10, height/5+280);
textSize(20);
text(“Tips”, width/10, height/5+330);
text(“You CANNOT AVOID the sound of finger snap.”, width/10, height/5+360);
text(“But you can choose the correct direction to avoid danger.”, width/10, height/5+390);
textSize(40);
textAlign(CENTER);
text(“Click to continue”, width/2, height/5+480);
}

void gameScreen() {
loadPixels();
map.loadPixels();
flashlight();
fill(0);
direction();
musicsound();
ellipse(PosX, PosY, rad*2, rad*2);
monsterSound();
decide();
}

void gameOverScreen() {
background(0);
textAlign(CENTER);
fill(255);
textSize(30);
text(“Game Over”, height/2, width/2 – 20);
textSize(15);
text(“Click to Restart”, height/2, width/2 + 10);
}

void winningScreen() {
winning();
background(255);
textAlign(CENTER);
fill(0);
textSize(30);
text(“Congratulations! You’ve found your way out!”, width/2, height/2);
textSize(15);
text(“Click to go back”, width/2, height/2 + 40);
}

void tutorial() {
loadPixels();
map2.loadPixels();
flashlightT();
fill(0);
directionT();
tutorialSound();
ellipse(PosXt, PosYt, rad*2, rad*2);
tuMonSound();
tuDecide();
tuText();
}

void TutOverScreen() {
background(255);
textAlign(CENTER);
fill(#F5453B);
textSize(70);
text(“Told you not to go there…”, width/2, height/2);
textSize(15);
text(“Click to restart”, width/2, height-30);
}

void TutContinue() {
background(255);
textAlign(CORNER);
fill(#CB7C2B);
textSize(40);
text(“Great! You’ve finished the tutorial.”, width/10, height/4);
text(“Now let’s keep moving.”, width/10, height/4+70);
text(“Click to continue”, width/2, height/4+400);
}

void mouseClicked() {
if (sentinel == 0) {
sentinel = 8;
} else if (sentinel == 6 ||sentinel == 7 || sentinel == 13) {
win.stop();
sentinel = 0;
PosX= rad;
PosY = 750 – rad;
PosXt = 720;
PosYt = 770;
} else if (sentinel == 8) {
sentinel = 9;
} else if (sentinel == 9) {
sentinel = 10;
} else if (sentinel == 10) {
sentinel = 14;
}else if (sentinel == 14) {
sentinel = 1;
}
}

Jumpscares

void bbjump(){
mons.stop();
imageMode(CENTER);
translate(width/2, height/2);
if (size < 2.4) {
scream.play();
scale(size);
image(BB, 0, 0, BB.width, BB.height);
size += 0.3;
image(BB, 0, 0);
}
if (size >= 2.4 && scream.isPlaying() == false){
delay(1000);
sentinel = 13;
size = 1;
}
}

void bearjump() {
mons.stop();
imageMode(CENTER);
translate(width/2, height/2);
if (size < 1.6) {
scream.play();
scale(size);
image(freddy, 0, 0, freddy.width, freddy.height);
size += 0.2;
image(freddy, 0, 0);
}
if (size >= 1.6 && scream.isPlaying() == false){
delay(1000);
sentinel = 6;
size = 1;
}
}

void chickenjump(){
mons.stop();
imageMode(CENTER);
translate(width/2, height/2);
if (size < 1.2) {
scream.play();
scale(size);
image(chicken, 0, 0, chicken.width, chicken.height);
size += 0.2;
image(chicken, 0, 0);
}
if (size >= 1.2 && scream.isPlaying() == false){
delay(1000);
sentinel = 6;
size = 1;
}
}

void foxjump() {
mons.stop();
imageMode(CENTER);
translate(width/2, height/2);
if (size < 1.6) {
scream.play();
scale(size);
image(foxy, 0, 0, foxy.width, foxy.height);
size += 0.2;
image(foxy, 0, 0);
}
if (size >= 1.6 && scream.isPlaying() == false){
delay(1000);
sentinel = 6;
size = 1;
}
}

void rabbitjump() {
mons.stop();
imageMode(CENTER);
translate(width/2, height/2);
if (size < 2) {
scream.play();
scale(size);
image(toyBonnie, 0, 0, toyBonnie.width, toyBonnie.height);
size += 0.2;
image(toyBonnie, 0, 0);
}
if (size >= 2 && scream.isPlaying() == false) {
delay(1000);
sentinel = 13;
size = 1;
}
}

Decide game status

void decide(){
if (PosX >= 120 && PosY >= 650 && PosX <= 300){
sentinel = 2;
}
else if (PosX >= 0 && PosX <= 100 && PosY <= 180){
sentinel = 3;
}
else if (PosX > 550 && PosX <= 650 && PosY <= 180){
sentinel = 4;
}
else if (PosX >= 950 && PosX <= 1350 && PosY <= 600 && PosY >= 250){
sentinel = 5;
}
else if (PosX >= width && PosY <= 100){
sentinel = 7;
}
}
// 950,250,400,350

void tuDecide() {// decide the player’s statuses in the tutorial
if (PosXt <= 400 && PosYt >= 500) {
sentinel = 11;
} else if (PosYt <= 200 && PosXt >= width/2 – 100 & PosX <= width/2 + 100) {
sentinel = 12;
} else if (PosXt <= 0 && PosYt >= 0 & PosYt <= 400) {
sentinel = 14;
}
}

Direction control

void direction(){
//control
if (keyPressed){
if (key == CODED) {
if ((get(PosX,PosY-10) == color(b) || PosY-rad <= 0) && keyCode == UP){
PosY-=0;
}else if ((get(PosX,PosY+10) == color(b) || PosY+rad >= height) && keyCode == DOWN){
PosY+=0;
}else if ((get(PosX-10,PosY) == color(b) || PosX-rad <= 0) && keyCode == LEFT){
PosX-=0;
}else if ((get(PosX+10,PosY) == color(b) || PosX+rad >= width && PosY >= 100) && keyCode == RIGHT){
PosX+=0;
}else if(keyCode == UP) {
PosY -= speed;
} else if (keyCode == DOWN){
PosY += speed;
} else if (keyCode == LEFT){
PosX -= speed;
} else if (keyCode == RIGHT){
PosX += speed;
}
}
}
}

void directionT(){
//control
if (keyPressed){
if (key == CODED) {
if ((get(PosXt,PosYt-10) == color(b) || PosYt-rad <= 0) && keyCode == UP){
PosYt-=0;
}else if ((get(PosXt,PosYt+10) == color(b) || PosYt+rad >= height) && keyCode == DOWN){
PosYt+=0;
}else if ((get(PosXt-10,PosYt) == color(b) || PosXt-rad <= 0 && PosYt >= 400) && keyCode == LEFT){
PosXt-=0;
}else if ((get(PosXt+10,PosYt) == color(b) || PosXt+rad >= width) && keyCode == RIGHT){
PosXt+=0;
}else if(keyCode == UP) {
PosYt -= speed;
} else if (keyCode == DOWN){
PosYt += speed;
} else if (keyCode == LEFT){
PosXt -= speed;
} else if (keyCode == RIGHT){
PosXt += speed;
}
}
}
}

Draw the map

void drawMap(){
size(1500,790);
background(255);
noStroke();
fill(0);
rect(100,0,450,150);
rect(100,250,450,550);
rect(550,500,250,300);
rect(650,0,100,350);
rect(750,250,200,100);
rect(850,100,650,50);
fill(255,0,0);
rect(100,650,200,150);
rect(0,0,100,200);
rect(550,0,100,200);
rect(950,250,400,350);
}

Flash light effect

void flashlight(){

// We must also call loadPixels() on the PImage since we are going to read its pixels. map.loadPixels();
for (int x = 0; x < map.width; x++ ) {
for (int y = 0; y < map.height; y++ ) {

// Calculate the 1D pixel location
int loc = x + y*map.width;

// Get the R,G,B values from image
float r = red (map.pixels[loc]);
float g = green(map.pixels[loc]);
float b = blue (map.pixels[loc]);

// Calculate an amount to change brightness
// based on proximity to the mouse
float distance = dist(x, y, PosX, PosY);

// The closer the pixel is to the mouse, the lower the value of “distance”
// We want closer pixels to be brighter, however, so we invert the value using map()
// Pixels with a distance of 50 (or greater) have a brightness of 0.0 (or negative which is equivalent to 0 here)
// Pixels with a distance of 0 have a brightness of 1.0.
float adjustBrightness = map(distance, 0, 140, 8, 0);
r *= adjustBrightness;
g *= adjustBrightness;
b *= adjustBrightness;

// Constrain RGB to between 0-255
r = constrain(r, 0, 255);
g = constrain(g, 0, 255);
b = constrain(b, 0, 255);

// Make a new color and set pixel in the window
color c = color(r, g, b);
pixels[loc] = c;
}
}

updatePixels();
}

void flashlightT(){ //flashlight effect in the tutorial

// We must also call loadPixels() on the PImage since we are going to read its pixels. map.loadPixels();
for (int x = 0; x < map2.width; x++ ) {
for (int y = 0; y < map2.height; y++ ) {

// Calculate the 1D pixel location
int loc = x + y*map2.width;

// Get the R,G,B values from image
float r = red (map2.pixels[loc]);
float g = green(map2.pixels[loc]);
float b = blue (map2.pixels[loc]);

// Calculate an amount to change brightness
// based on proximity to the mouse
float distance = dist(x, y, PosXt, PosYt);

// The closer the pixel is to the mouse, the lower the value of “distance”
// We want closer pixels to be brighter, however, so we invert the value using map()
// Pixels with a distance of 50 (or greater) have a brightness of 0.0 (or negative which is equivalent to 0 here)
// Pixels with a distance of 0 have a brightness of 1.0.
float adjustBrightness = map(distance, 0, 90, 8, 0);
r *= adjustBrightness;
g *= adjustBrightness;
b *= adjustBrightness;

// Constrain RGB to between 0-255
r = constrain(r, 0, 255);
g = constrain(g, 0, 255);
b = constrain(b, 0, 255);

// Make a new color and set pixel in the window
color c = color(r, g, b);
pixels[loc] = c;
}
}

updatePixels();
}

The sound of Monster

void monsterSound() {
boolean isPlaying = false;
boolean wasPlaying = false;
// freddy
if (PosX <= 100 && PosY >= 550) {
isPlaying = true;
mons.pan(1);
mons.amp(map(PosY, height, 550, 1, 0));

//chica
} else if (PosX <= 100 && PosY < 450 ) {
isPlaying = true;
mons.pan(0);
mons.amp(map(PosY, 450, 200, 0, 1));

//chica 2
} else if (PosX >= 100 && PosX <= 250 && PosY <= 250) {
isPlaying = true;
mons.pan(-1);
mons.amp(map(PosX, 100, 250, 1, 0));

//springtrap 1
} else if (PosX >= 400 && PosX <= 650 && PosY <= 250) {
isPlaying = true;
mons.pan(map(PosX, 400, 650, 1, 0));
mons.amp(map(PosX, 400, 650, 0, 1));

//springtrap 2
} else if (PosX > 550 && PosX <= 650 && PosY <= 350) {
isPlaying = true;
mons.pan(0);
mons.amp(map(PosY, 250, 350, 1, 0));

//foxy 1 (L shape) area: 950,250,400,350
} else if (PosX >= 650 && PosX <= 950 && PosY >= 350 && PosY <= 600) {
isPlaying = true;
mons.pan(1);
mons.amp(map(PosX, 650, 950, 0, 1));

//foxy 2
} else if (PosX >= 800 && PosX <= width && PosY > 600 && PosY <= height) {
isPlaying = true;
mons.pan(map(PosX, 800, width, 1, -1));
mons.amp(map(PosY, 600, height, 1, 0.3));

//foxy 3
} else if (PosX >= 1350 && PosX <= width && PosY <= 600 && PosY >= 250) {
isPlaying = true;
mons.pan(-1);
mons.amp(map(PosX, 1350, width, 1, 0.3));

// foxy 4
} else if (PosX >= 750 && PosX <= width && PosY <= 350 && PosY >= 150) {
isPlaying = true;
mons.pan(map(PosX, 750, width, 1,-1));
mons.amp(map(PosY, 350,150, 1,0.3));
} else {
isPlaying = false;
}
if (isPlaying == true && wasPlaying == false) {
if (!mons.isPlaying()) {
mons.play();
}
} else {
mons.stop();
}
wasPlaying = isPlaying;
}

void musicsound() {

while (myPort.available() > 0) {
val = myPort.read();
println(val);
}

if (val == 1) {
//if (keyPressed) {
// if (key != CODED) {
// if (key == ‘l’) {
if (PosX <= 100 && PosY >= 250) {
file1.amp(map(PosY, height, 250, 0.01, 1));
file1.play();
//delay(6000);
}
if (PosX >= 0 && PosX <= 550 && PosY <= 250) {
file2.pan(1); // map(PosX, 0,550,1,0)
file2.amp(map(PosX, 0, 550, 0, 1));
file2.play();
//delay(6000);
}
if (PosX > 550 && PosX <= 650 && PosY >= 150 && PosY <= 350) {
file3.amp(map(PosY, 150, 500, 0, 1));
file3.play();
//delay(3000);
} else if (PosX > 550 && PosX <= 800 && PosY > 350 && PosY <= 500) {
file4.amp(map(PosX, 550, 800, 0, 1));
file4.pan(map(PosX, 550, 800, 1, 0));
file4.play();
//delay(3000);
} else if (PosX > 800 && PosY > 350 && PosY <= height) {
file5.amp(map(dist(PosX, PosY, width, 500), dist(800, height, width, 500), 0, 0, 1));
file5.pan(map(PosX, 800, width, 1, 0.5));
file5.play();
} else if (PosX <= width && PosX >= 750 && PosY <= 350 && PosY >= 150) {
file6.amp(map(dist(PosX, PosY, 750, 150), dist(width, 350, 750, 150), 0, 0, 1));
file6.pan(-1); // map(PosX, width, 750, -1, 0)
file6.play();
} else if (PosX <= width && PosX >= 750 && PosY <= 150 && PosY >= 0) {
file7.amp(map(dist(PosX, PosY, width, 0), dist(750, 150, width, 75), 0, 0.1, 1));
file7.pan(map(PosX, width, 750, 1, 0.1));
file7.play();
} else if (sentinel == 7) {
win.play();
}
}
}

void tuMonSound(){ //sound of the monster in the tutorial
boolean isPlaying = false;
boolean wasPlaying = false;
// bonnie
if (PosXt > 400 && PosXt <= 800 && PosYt >= 500) {
isPlaying = true;
mons.pan(-1);
mons.amp(map(PosXt, width/2, 450, 0.5, 1));

//BB
} else if ( PosYt <= 400) {
isPlaying = true;
mons.pan(map(PosXt, 0,width,1,-1));
mons.amp(map(PosYt, 400, 200, 0.5, 1));
} else {
isPlaying = false;
}
if (isPlaying == true && wasPlaying == false) {
if (!mons.isPlaying()) {
mons.play();
}
} else {
mons.stop();
}
wasPlaying = isPlaying;
}

void springtrap(){
mons.stop();
imageMode(CENTER);
translate(width/2, height/2);
if (size < 1.4) {
scream.play();
scale(size);
image(spring, 0, 0, spring.width, spring.height);
size += 0.2;
image(spring, 0, 0);
}
if (size >= 1.4 && scream.isPlaying() == false){
delay(1000);
sentinel = 6;
size = 1;
}
}

Tutorial text

void tuText() {
if (PosXt <= 1200 && PosYt >= 500) {
textAlign(CORNER);
fill(255);
textSize(40);
text(“Now that you are at a crossroads.”, 20,370);
text(“Hear the finger snap? It’s from the left side.”, 20,410);
text(“Now Press the button.Identidy where the melody comes from.”, 20,450);
text(“Head for that direction and you are safe!”, 20,490);
println(“text”);
//delay(6000);
} else if (PosXt > 1200 && PosYt >= 400 && PosYt <= height) {
textAlign(CORNER);
fill(255);
textSize(40);
text(“Now go straight up, along the road.”, 20,410);

//delay(6000);
} else if (PosYt < 400) {
textAlign(CORNER);
fill(255);
textSize(40);
text(“Now that you are at another crossroads.”, 20,410);
text(“Press the button and listen.”,20, 450);
text(“Head for the direction of the melody you hear.”, 20,490);
text(“If the volume goes up, then you are in the correct direction.”, 20,530);
}
}

Tutorial sound

void tutorialSound() {
while (myPort.available() > 0) {
val = myPort.read();
println(val);
}

if (val == 1) {
//if (keyPressed) {
// if (key != CODED) {
// if (key == ‘l’) {
if (PosXt <= 1200 && PosYt >= 500) {
tut1.pan(1);
tut1.amp(map(PosXt, width, width/2, 1, 0.1));
tut1.play();
//delay(6000);
} else if (PosXt > 1200 && PosYt >= 400 && PosYt <= height) {
tut2.pan(0);
tut2.amp(map(PosYt, 400, height, 1, 0.1));
tut2.play();
//delay(6000);
} else if (PosYt < 400) {
tut3.pan(-1);
tut3.amp(map(PosXt, width, 0, 0.05, 0.5));
tut3.play();
}
}
//delay(3000);
}

Winning 

void winning() {
boolean isPlaying = false;
boolean wasPlaying = false;
if (sentinel == 7) {
isPlaying = true;
} else {
isPlaying = false;
}
if (isPlaying == true && wasPlaying == false) {
if (!win.isPlaying()) {
win.play();
}
} else {
win.stop();
}
wasPlaying = isPlaying;
}

Since our project will need to send values created in the joystick to the processing, it is necessary for us to know how to transfer multiple values from Arduino to Processing, I attended the serial communication workshop. During the recitation, we use two potentiometers to control the movement of an ellipse and the button to control the advent of a rectangular. Although we used potentiometers instead of the joystick during the recitation, the essence of transferring values is to the same. Important requirements include: finding the correct ports, define the number of the values that are to be transferred and make sure the titles of the values are the same.

Circuit

Code

Video

Circuit:

Video:

Code:

During this recitation, we need to create a processing sketch that can control a certain aspect of a media such as an image or a video. For me, I think it would be interesting if the level of one of the filters of an image can be manually controlled and manipulated. To realize, the first step is to find the image and I decided to use the image of light, which makes more sense as I intended to use the blur function. Then I built the circuit using a potentiometer that can control the change of the degree of blur. Finally, I finished the coding that aims to transfer one value from Arduino to Processing.

It is really interesting that the article “Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers” points out the generalization of computer vision technologies from military and law-enforcement purposes to more general and artistic uses. Due to the time and ability limitation, my project only realizes the simple manipulation of the degree of blur function. However, with the development of computer technology, it is possible to change different parts of the image. For instance, people might be able to manipulate the light from a lantern, they can change the brightness of the sun or the growing stage of a tree by pushing buttons or turning knobs. In this way, people can actually create their unique art even with the same settings and background, which I believe is one of the most amazing and interesting aspects of what computer vision technologies can achieve.

Resources:

The image of the lantern: https://www.google.com.hk/search?hl=en&tbm=isch&source=hp&biw=1269&bih=586&ei=mwLJXMDRFoHS8wXEp4QY&q=lantern&oq=lantern&gs_l=img.3..35i39j0l9.796.1585..1857…0.0..1.186.1196.0j7……1….1..gws-wiz-img…..0.8bMz37lsDIc#imgrc=MzoNpEsQE7pnNM: 

Background image: https://www.google.com.hk/search?hl=en&biw=1269&bih=586&tbm=isch&sa=1&ei=ngLJXKnrHq60mAX44q7QCw&q=dark+room&oq=dark+room&gs_l=img.3..35i39j0l9.44268.46514..46883…0.0..0.137.1004.4j6……1….1..gws-wiz-img…..0..0i67.Ensu1RXxUQA#imgrc=fMnk-bq02oj5DM:

Step 1

During this recitation, I read three very interesting ideas for the final project. The first idea is about creating a game that imitates the Snake. However, the difference is that in this game, players need to pay extra attention because all the direction control are the opposite. For example, to move the snake downwards, players need to push the up button. The aim of the game is to control the snake to eat as many apples as possible while keeping not touching the frame. It is also interesting that the student actually had a sample already, so we were able to try this game out. Our feedback for this idea is that while it is reasonably difficult and challenging to play, it might be better if they can create different levels by changing the speed of the snake. Also, they might need to ponder on the meaning of this game. The second idea I read is to create a game that targets people who are interested in exploring space. The aim of the game is to transport supply through a space battle and players need to avoid being hit by bombs or rocks. The interesting thing about this idea is that players will have the first-person vision as if they are a pilot sitting in a spaceship and facing the universe. Our comment for this idea is that it can include more interaction between players and the game by fully using Arduino, for it seems that the game can be realized without using Arduino. The third idea I read is about creating an Avengers’ related game. It targets at the fans of the superheroes and has a lot to do with the latest blockbuster Avengers 4. As a huge fan of Marvel and its superheroes, I find the idea itself really interesting. Nevertheless, during our group discussion, we also agree that they need to find a way to balance the two sides of the players so that one side won’t be greatly more powerful than the other side. They also need to figure out the meaning of their game. By looking at all the other group members’ ideas, I find that one of the most obvious commonplaces is that all of us decide to create a game as our final project. It might because the game is also an excellent way to engage people and machine together to realize interaction. Also, games are always highly entertaining and funny to play, which also add to people’s motivation to create games as their final project.

Step 2:

As my idea for the final project is about creating a game that requires players to use hearing to get out of a dark building, I received several very educational advice from my group members, as well as my partner’s group members as well. One of my group members suggests that we can ask the player to wear an eyeshade when playing the game, while others can actually see how he/she is doing. The purpose is to enable normal people to experience how difficult and challenging it can be for blind people to walk. Also, someone from my partner’s group suggests that we can create multiple levels and settings, so as to make the game more changeable. For my group members, the most successful part of the game is that people can merely depend on their hearing instead of eyesight to get out of a sort of maze, which is quite rare. I strongly agree with them and I think that the meaning of the whole project is based on such a setting. However, they also mention that it might be hard to control the difficulty of the game, meaning that the game can easily be either too easy or too hard for people to play. I agree with is criticism as well, and I think we may be able to fix that problem by conducting more users’ tests. All in all, I really appreciate all the advice that my group members have given me, which are all of the great use and quality. I will definitely take these pieces of advice into consideration when creating our project.

Project Title

Darkness Rhapsody

The project statement of purpose

For our final project, we intend to create a game that requires players to find their way out of the darkness by listening to the sound of the wind. During the game, people will come to crossroads and need to decide which way to go. It is when they need to tell which direction is the sound of wind come from and head for that direction. We also intend to simulate how it might feel like to be a blind person. Although such simulation is not very close to the real situation, it more or less gives people the experience of not using eyes but using ears to find the way. Moreover, by emphasizing the importance of hearing, this project also serves as a reminder to those who always fix their eyes on smartphones while walking on the street, showing them walking without looking at one’s way is rather dangerous and they should always pay attention to the sound on the street to keep safe.

Project Plan

The first step we plan to take is to use processing to create the main scene of the game. We will need to realize the effect of walking by moving the background. Although we haven’t decided what should be the background yet, we are thinking about design it as a two-dimension map of a building that includes an entrance, an exit, and several rooms. As there are around three weeks left, We intend to finish this part of the project during the first week. For the second week, we will focus on the Arduino part. We need to find a way to enable players to control which direction to go in processing. We are currently thinking about using a joystick to realize it. We will also need to prepare the sound of footsteps and the sound of wind and connect them to earphones so that people can hear the sound better. What’s more, we intend to add background music to this game, which will also serve as instructions or hints to some extent. As this part is more difficult, we intend to spend one and a half week to finish this part. For the rest of the time before the due date, we will run user tests and try to improve the project according to the advice we can get.

Context and Significance

According to my preparatory research and analysis, I think there are two efficient ways to engage people in the interaction. The first approach is to make sure that people have to use their sensation during the process of interaction. In our project, it is very important for people to use their ears to find the way out. The second way is to make the game enough entertaining and changeable. If the game requires people to pay a little effort to figure out, it would be more interactive and have a better user experience. Therefore, we decided that there will be enough crossroads for players to explore. It is also possible that people cannot make it get out of the map because they turned into the wrong direction. By adding choices and possibility to fail the game into our project, we intend to make it more entertaining and interactive. Although there is no specific segment of people we are targeting at, we think it is important for people to experience how it feels like without eyesight while having fun playing the game.