Music Drum Game – Sid Tuladhar – Rudi

 

CONCEPTION AND DESIGN:

One of the biggest influences that influenced was the type of project that I wanted to build was the book by Ernest Edmonds because the different types of interactions that exist between an object and the user was a crucial idea to making a project that met my definition of interaction. Additionally, his book also contained the concept of different types of engagement with the user and creating long term engagement, which I believe is a really important aspect to include in my project. After looking for projects that would be able to do this, I came across Moon Quest by Pen Porter which is a video game that is has procedurally generated levels. All video games are engaging and have constant and dynamic interaction with the user, but not all video games create long term engagement. Moon Quest creates long term engagement because the levels are random and you can come back to the came and always have a different type of experience. Hence, that’s why I wanted to create a video game that would have long term engagement.

While brainstorming with my partner Jessie, we decided that we would take a common game and add a unique type of interaction to the game. After discussing different possibilities, we came up with basing our game off the Japanese rhythm came called Taiko No Tatsujin. This video game is a rhythm game where a player hits a drum when the game indicates when they need to hit it. To put our own twist to it, we wanted to use multiple drums and have the drums be associated with the primary colors. And, our game with also have secondary colors, where the user would have to hit the two primary color drums to remove on the screen. Additionally, to have long term engagement, we decided to make it a multiplayer game because if you’re working with someone, your experience will always be different because the person is  different or you’re improving your teamwork with the person. So in the end we decided that there would be three drums, and each player would have one drum stick and they would have to work together to hit the drums according to the colors on the screen. They would have to have proper timing because for the secondary colors, each of them would have to hit a drum at the same time.

During the User Testing Session, there were many different types of feedback we received.

 

The most significant feedback was changing the GUI of the game because the game was too difficult, which would hinder the teamwork and cooperation between the players. We initially had six different lines (three for primary and three for secondary) where the colored boxes would come down, this would be very difficult for the user to try to concentrate on six different places. We were suggested to change the secondary colored lines into one line so we would have four lines in total. This would make it easier for the player and the GUI would also look cleaner. Also, we got advice that the boxes shouldn’t disappear so early if the player misses otherwise they would be confused whether they removed the box because they hit the drum or if they actually missed. Also, to make the game easier, I increased the hit box for when the player hits the drum and the box disappears. But to reward players that are more accurate, the closer they hit the drum when the colored box is in the middle of the hit box, they would be rewarded with more points. Another complaint we got was that there should be a tutorial or explanation on how to play, thus we added a short video to show how the game works before the game starts. Additionally to make it easier, we added three different difficulties to the game. All of these small improvements drastically improved the engagement and playability of the game. Also, it improved the teamwork and communication between players.

 

FABRICATION AND PRODUCTION:

During the Fabrication and Production part, I worked on the coding part while Jessie worked on the fabrication part. The coding was challenging and I learned a lot during the process. I looked for different Processing code examples online that I could use as a template to the game. Initially, I found one that was coded using p5.js but I decided against it since I didn’t think I would have enough time to learn the code in p5.js and I assumed there would be multiple additional challenges that would occur once I connected it with the Arduino. So I kept looking until I found one that was in Java. I had to learn the syntax for Object Oriented Programming in Java. Then with trial and error, I slowly made modifications to the code to fit the idea that I had in mind. The biggest challenge I faced while coding was making the sensors work properly with the Processing code. Sometimes they would trigger the wrong colored box, or sometimes they wouldn’t work at all. The sensitivity of the vibration sensors that were used was very difficult to control. The Arduino part was easy since I just used the analog signal example code and modified it for three sensors.

 

Some of the significant additions that I included was the music, text boxes, and difficulty in the game. I used the Processing Sound library to include three different songs that were related to the Christmas. The songs start when the level starts. Text boxes were added using Object Oriented Programming, which was my first time trying to implement something like this. To implement difficulty, I created variables that would change the speed of the colored boxes. The easy difficulty would only have the primary colored boxes so that the users are understand the timing of the drums. The medium difficulty would include the secondary colored boxes so the users have to work together to get points. The hard difficulty would be at twice the speed of the medium difficulty.

One of the functions that I wanted to implement that I wasn’t able to was to manually place each of the colored boxes wherever I wanted, but I wasn’t able to implement that. Instead they always come down randomly, which unintentionally  would help the game with long term engagement. Another function that I wanted to include after the User Testing Session was to incorporate a video into the beginning and end of the game. I was able to include the video at the beginning, but wasn’t successful in implementing it at the end. If I had more time, I’m sure that I would’ve been able to figure it out. The video at the beginning of the game was created by Jessie. 

 

For the Fabrication part, Jessie did most of the work. She worked on creating the drum and the laser cutting the plastic on top of the drum.

  

She added her own personal touch to make the drum look nice. She also came up with the idea of using a vibration sensor. We used a vibration sensor since when the drum would get hit, the sensor would output an analog signal. One of the unexpected changes that we had was making our controller look like a donut. We realized that the drum needs to be hit with something that has a large surface area otherwise sometimes the vibration sensors sometimes wouldn’t detect the hit, and using drum sticks wouldn’t work because they don’t have a large surface area when you hit the drum. Making our controllers a donut would make it large surface area that would hit the drum.

The drumstick was made by Jessie by laser cutting wood into a donut shape, sticking them together, and gluing fabric on top of it.

I worked on the wiring part between the sensors and the Arduino.  The wiring was very simple and straightforward since it was a connection with three different vibration sensors. 

 

CONCLUSIONS:

The main goals for this project was to create a multiplayer video game that had long term engagement. Our project was able to achieve this goal because we created a game that changes every time you play because it is based on randomness, and including another player also makes the experience different every time. We expected the audience to interact it with the drums by hitting them and coordinating with their partner on how to work together.

 

Ultimately, they did interact with the game how we expected, but we didn’t expect the way they would coordinate and work together. We discovered that different people have different ways of working together depending on how they communicated. Some people don’t communicate and realize then they realize afterwards that they have to come up with a plan. Others shout which color they’re going to hit and which color their partner should hit. At the very least, both users are highly engaged with the game. My project aligns with my definition of interaction because the project is reacts dynamically with the user because it’s always different for the user, and there’s also long term engagement. If I had more time, there are many different improvements that can be implemented. I would work on the improving the graphics and adding a video at the end when the user closes the game. Also, adding more songs and more levels would all be feasible with more time. 

I’ve learned so much because of this project. I couldn’t understand how Object Oriented Programming worked at all in Java before this project, but I was able to learn and implement my own objects and classes in Processing. Also, learning about the sensors and their sensitivity was a very good learning experience. I hadn’t worked with analog sensors to this extent before. Using the vibration sensor took so much time because our initial vibration sensors were damaged or didn’t work properly so I was initially very doubtful if the vibration sensor was the right decision. However, using different vibration sensors really made me hopeful when they worked. The coordination between the physical inputs and digital interactions was a very important connection that I spent a lot of time trying to work flawlessly. Additionally, understand the draw function and how other libraries react to the draw function was something I learned. I had no idea how to implement the video without making the game start because the game and the video starts because of the draw function. But after asking for help, I used different states so that the video would start first and once it ends the game would start. 

Besides learning from the code, I learned that interaction between users is something that I can’t predict but I can only give subtle hints towards a mutual goal. It wouldn’t be fun if I told them exactly how they should work together, but it’s about how they discover to work together and interaction between each other and the game. The mutual goal of trying to get a high score aids in the coordination and teamwork between the users. Also, it’s not expected that they would be successful the first time they play. Failure or doing bad is a part of the learning experience that leads to better coordinated interaction and more engagement with the game. It becomes a loop of interacting with each other to improve the interaction with the game, which creates long term engagement.

In conclusion, when it comes to creating a object or project that needs to be interactive, it’s important to focus on what type of interaction that is expected with the user, and see if the user actually interacts with it as you intended. Especially when it comes to devices that take a physical interaction and displays a virtual reaction, it’s important to make sure that the user understands what their physical interaction causes on the screen. Otherwise, the user becomes confused and the physical interaction feels uneventful because the physical interaction doesn’t have any other way to react. Additionally, if the project requires interaction that requires focus, the project should physical slowly guide the user towards this type of interaction instead of forcing them into this interaction all at once. These aspects were crucial in my project and something I reflect on when looking back at the development of my project. This is important because creating an engaging and interactive project is essential for any product whatever the purpose of the product is. 

 

ANNEX

Processing Code (based on this Jeppe Brown Richardson’s code):

import processing.serial.*;
import osteele.processing.SerialRecord.*;
import processing.sound.*;
import processing.video.*;

// declare move object
Movie movie1;
Movie movie2;

// declare a SoundFile object
SoundFile sound1;
SoundFile sound2;
SoundFile sound3;

Serial serialPort;
SerialRecord serialRecord;

ArrayList tile = new ArrayList();
static int gameWidth = 1200, gameHeight = 600;
int score=0;
int misses=0;
String questionMessage1;
String questionMessage2;
boolean first_time = true;
boolean display_question = false;
int blue;
int yellow;
int red;
int difficulty_num;
float difficulty_speed = 1.5;
float duration;
int gamestatus = 0;

void setup() {
  String serialPortName = SerialUtils.findArduinoPort();
  serialPort = new Serial(this, serialPortName, 9600);
  serialRecord = new SerialRecord(this, serialPort, 3);

  fullScreen();
  noStroke();
  questionMessage1 = "Do You Want To Play Again?";
  questionMessage2 = "Choose Your Difficulty";
  askingQuestion = true;
  textSize(18);

  sound1 = new SoundFile(this, "christmas.mp3"); // It's Beginning To Look A Lot Like Christmas - Perry Como
  sound2 = new SoundFile(this, "tree.mp3"); // Christmas Tree Farm - Taylor Swift
  sound3 = new SoundFile(this, "santa.mp3"); // Santa Tell Me - Ariana Grande

  movie1 = new Movie(this, "opening.mp4");
  movie1.play();

  //image(movie, 0, 0, width, height);

  //beginning video or instruction video
}

void draw() {

  serialRecord.read();
  blue = serialRecord.values[1]; //blue
  yellow = serialRecord.values[2]; //yellow
  red = serialRecord.values[0]; //red

  background(50, 150, 200, 200);

  if (gamestatus == 0) {
    if (movie1.available()) {
      movie1.read();
      print(questionMessage2);
    } else if (movie1.available() == false) {
      print(questionMessage1);
      gamestatus = 1;
    }
    image(movie1, 0, 0, width, height);
  } else if (gamestatus == 1) {




    //image(movie, 0, 0, width, height);


    tiles til = new tiles(int(random(difficulty_num)), int(random(3)));

    fill(#aa0000, 50);
    rect(0, 600, width, 100);

    if (frameCount%50==6) {            //amount of notes generates per second, frameCount%10=6 notes/second
      tile.add(til);
    }

    fill(#aaaaaa);
    rect(200, 0, 20, height);
    rect(450, 0, 20, height);
    rect(700, 0, 20, height);
    rect(950, 0, 20, height);

    for (int i=0; i<tile.size(); i++) {

      tiles ta = (tiles) tile.get(i);
      ta.run();
      ta.display();
      ta.move(difficulty_speed);
      if ( red > 10 && ta.location.y > 550 && ta.location.y < 750&& ta.location.x == 160.0) { //red
        ta.gone=true;
        serialRecord.read();

        red = serialRecord.values[0];
      }
      if (blue > 10 && ta.location.y > 550 && ta.location.y < 750 && ta.location.x==410.0) { //blue
        ta.gone=true;
        serialRecord.read();
        blue = serialRecord.values[1];
      }
      if (yellow > 10 && ta.location.y > 550 && ta.location.y < 750 && ta.location.x==660.0) { //yellow
        ta.gone=true;
        serialRecord.read();
        yellow = serialRecord.values[2];
      }
      if (yellow > 30 && blue > 30 && ta.location.y > 550 && ta.location.y < 700 &&ta.location.x==910.0) { //green
        ta.gone=true;
        serialRecord.read();
        
        yellow = serialRecord.values[2];
        blue = serialRecord.values[1];
      }
      if (red > 30 && blue > 30 && ta.location.y > 550 && ta.location.y < 700&&ta.location.x==910.0) { //purple
        ta.gone=true;
        serialRecord.read();
        
        red = serialRecord.values[0];
        blue = serialRecord.values[1];
      }
      if (red > 30 && yellow > 30 && ta.location.y > 550 && ta.location.y < 700 &&ta.location.x==910.0) { //orange
        ta.gone=true;
        serialRecord.read();
        
        red = serialRecord.values[1];
        yellow = serialRecord.values[2];
      }


      if (ta.location.y>1000) {
        tile.remove(i);
        misses++;
      }
      if (ta.gone==true) {
        score+=ta.location.y>650?30:ta.location.y>600?20:10;    //scoring system(you get more points if you do better)
        tile.remove(i);
      }
    }

    fill(#0000aa); // score display
    textAlign(CENTER);
    textSize(50);
    text(score, 1300, 130);

    if (sound1.isPlaying() == false && sound2.isPlaying() == false && sound3.isPlaying() == false) {
      if (first_time == true) {
        drawDifficulty();
        first_time = false;
      } else if (display_question == false) {
        display_question = true;
        askingQuestion = true;
        for (int i=0; i<tile.size(); i++) { // removes all tiles after song stops
          tile.remove(i);
        }

        drawScore();
      } else if (display_question == true) {
        askingQuestion = true;
        display_question = false;
        drawDifficulty();
        score = 0;
        misses = 0;
      }
    }
  } else if (gamestatus == 2) {
    movie2 = new Movie(this, "ending.mp4");
    movie2.play();
    if (movie2.available()) {
      movie2.read();
      println("Game Over! Thanks for playing, please refresh the page when you are ready to PLAY AGAIN!");
    } else if (movie2.available() == false) {
      println("bug!");
      gamestatus = 3;
    }
    image(movie2, 0, 0, width, height);
  } else if (gamestatus == 3) {
    endGame();
  }
}

void mousePressed() {
  int check1 = checkAnswer1();
  if (check1 == YES) {
    loop();
  }
  if (check1 == NO) {

    // add end video
    gamestatus = 2;
  }

  int check2 = checkAnswer2();
  if (check2 == EASY) {
    print(check2);
    difficulty_num = 3;
    difficulty_speed = 1.5;
    sound1.play();
    duration = sound1.duration();
    loop();
  } else if (check2 == MEDIUM) {
    difficulty_num = 4;
    difficulty_speed = 1.5;
    print(check2);
    sound2.play();
    duration = sound2.duration();
    loop();
  } else if (check2 == HARD) {
    difficulty_num = 4;
    difficulty_speed = 2.8;
    sound3.play();
    duration = sound3.duration();
    loop();
    print(check2);
  }
}


class tiles {
  PVector location;
  Boolean gone=false;
  color tile_color;

  tiles(int i, int j) {
    color[] colors = {#19ad05, #9504de, #fc8403};
    location = new PVector((i*250) + 160, 0);
    tile_color = colors[j];
  }


  void run() {
    display();
    move(difficulty_speed);
  }

  void display() { //#19ad05, #9504de, #edd602
    fill(location.x>=0 && location.x<200 ?#de0404: location.x>200 && location.x<=410 ?#0416de:
      location.x>410 && location.x<=660 ?#edd602: location.x>660 && location.x<=910?tile_color: location.x>900 && location.x<1200?#fc7600:#fafafa);
    rect(location.x, location.y, 100, 50, 40);
  }

  void move(float speed) {
    location.y+=speed;
    //note speed, changing this will up the difficulity, putting it too high will make
  }                                  //it literally impossible
}


static boolean askingQuestion = false, answer = false;
static int questionX=gameWidth/2-150, questionY=gameHeight/2-40, questionWidth=370, questionHeight=100;
final static int NO_ANSWER = 0;
final static int YES = 1;
final static int NO = 2;
final static int EASY = 3;
final static int MEDIUM = 4;
final static int HARD = 5;

int checkAnswer2() { //Check to see if user clicked which difficulty
  if (mouseX >= questionX && mouseX <= questionWidth + 650 && mouseY >= questionY + 50 && mouseY <= questionY+questionHeight && askingQuestion == true) {
    loop();
    askingQuestion = false;
    answer = true;
    return EASY;
  }
  if (mouseX >= questionX && mouseX <= questionWidth + 650 && mouseY >= questionY + 100 && mouseY <= questionY + 140 && askingQuestion == true) {
    loop();
    askingQuestion = false;
    answer = true;
    return MEDIUM;
  }
  if (mouseX >= questionX && mouseX <= questionWidth + 650 && mouseY >= questionY + 150 && mouseY <= questionY + 190 && askingQuestion == true) {
    loop();
    askingQuestion = false;
    answer = true;
    return HARD;
  }

  return NO_ANSWER;
}

int checkAnswer1() { //Check to see if user clicked yes or no
  if (mouseX >= questionX + 600 && mouseX <= questionX + 783 && mouseY >= questionY + 150 && mouseY <= questionY + 190 && askingQuestion == true) {
    loop();
    askingQuestion = false;
    answer = true;
    return YES;
  }
  if (mouseX >= questionX + 783 && mouseX <= questionX + 950 && mouseY >= questionY + 150 && mouseY <= questionY + 190 && askingQuestion == true) {
    loop();
    askingQuestion = false;
    answer = true;
    return NO;
  }

  return NO_ANSWER;
}


void drawScore() {
  stroke(225);
  fill(50);
  //rect(questionX - 2, questionY - 2, questionWidth + 10, questionHeight + 4);
  rect(questionX + 600, questionY, questionWidth, questionHeight - 50);
  rect(questionX + 600, questionY + 50, questionWidth, questionHeight - 50);
  rect(questionX + 600, questionY + 100, questionWidth, questionHeight - 50);
  rect(questionX + 600, questionY + 150, questionWidth, questionHeight - 50);
  rect(questionX + 783, questionY + 150, questionWidth - 183, questionHeight - 50);
  stroke(225);
  fill(225);
  textSize(30);
  text(questionMessage1, questionX + 790, questionY + questionHeight - 66);

  text("Score: " + score, questionX + 685, questionY + questionHeight - 18);
  text("Misses: " + misses, questionX + 682, questionY + questionHeight + 34);
  text("Yes", questionX + 684, questionY + questionHeight + 83);
  text("No", questionX + 872, questionY + questionHeight + 83);
  noLoop();
}

void drawDifficulty() {
  stroke(225);
  fill(50);
  //rect(questionX - 2, questionY - 2, questionWidth + 10, questionHeight + 4);
  rect(questionX, questionY, questionWidth + 160, questionHeight - 50);
  rect(questionX, questionY + 50, questionWidth + 160, questionHeight - 50);
  rect(questionX, questionY + 100, questionWidth + 160, questionHeight - 50);
  rect(questionX, questionY + 150, questionWidth + 160, questionHeight - 50);
  stroke(225);
  fill(225);
  textSize(30);
  text(questionMessage2, questionX + 250, questionY + questionHeight - 66);
  textSize(20);
  text("Easy: It's Beginning To Look A Lot Like Christmas - Perry Como", questionX + 267, questionY + questionHeight - 18);
  text("Medium: Christmas Tree Farm - Taylor Swift", questionX + 190, questionY + questionHeight + 34);
  text("Hard: Santa Tell Me - Ariana Grande", questionX + 158, questionY + questionHeight + 81);
  noLoop();
}

void endGame() {
  noLoop();
  exit();
}

 Arduino Code:

#include "SerialRecord.h"

SerialRecord writer(3);

void setup() {
  Serial.begin(9600);
}

void loop() {
  
  int value1 = analogRead(0);
  int value2 = analogRead(2);
  int value3 = analogRead(4);
  writer.send();
  //if (value1 > 0) {
  writer[0] = value1;
  writer[1] = value2;
  writer[2] = value3;
    //writer.send(0);
      //
  //}
  //if (value2 > 0) {
  
  
      //
  //}
  
  // This delay slows down the loop, so that it runs less frequently. This can
  // make it easier to debug the sketch, because new values are printed at a
  // slower rate.
  delay(20);
  
}