Interaction Lab Final Project Blog

1. Soaring Over the Sound City – Junhao Zhu & Jiaxi Zhang – Gottfried Haider

Here is a picture of our project.

2. Conception And Design

In this project, we tried to experiment with the methods of players interacting with a game.  We conducted research on the forms of interactivity that go beyond the monotonous keyboards and mice input. We found some amateur projects from profound companies including Kinect from Microsoft, a project that features multiple sensors for gesture and motion recognition, which add immersion for users playing games;

From Reliance Digital [@reliancedigital]. (2011, December 20). Xbox 360 Kinect. Youtube. https://www.youtube.com/watch?v=oyjNqksc-m8

and Labo, a toys-to-life concept developed by Nintendo. This project encourages users to invent new ways of interacting with their gaming devices with cardboards and imaginations.

Create new ways to play with Nintendo Labo! (n.d.). Nintendo of Europe AG. Retrieved May 8, 2024, from https://www.nintendo.com/en-gb/Nintendo-Labo/Nintendo-Labo-1328637.html

In conventional game interactions, players usually passively receive feedbacks from the game itself and then give out their responses. In comparison, the two projects above actively inviting the players to contribute to the interaction process. This gives us the thought of utilizing the user’s voice as the medium of interaction. 

After making the decision of designing a voice-based multiplayer games, we looked up to possible applications of sounds. “YASUHATI / With your voice!” caught our attention.

YASUHATI / With your voice! (n.d.). Google.com. Retrieved May 9, 2024, from https://play.google.com/store/apps/details?id=jp.ne.freem.YASUHATI&hl=en&gl=US

Above is a game developed by Freedomcrow, an indie game developer in Japan. Players can bypass the need for traditional, complex control schemes. Instead, the game simplifies interaction by allowing players to use their voices to direct a musical note character. As the note navigates through various levels, players control its movements by altering the pitch and volume of their voices, enabling it to jump across a series of platforms and avoid obstacles. 

Based on the previous researches, we developed our project of “Soaring Over the Sound City”. In this project, two players will act as the character of “Navigator” and “Builder”. The “Builder” will build the scene with his or her voice, the heights of all the blocks will be decided based on their vocal inputs. The “Navigator” will try to navigate through this vocally-built landscape with a controller. The “Builder” will have certain kinds of special skills to create obstacles in the scene and make the game more challenging.

Considering the game physics, there are two decisions that needed us to address in the design. The first one is how to interact with this game with sound, and the second one is finalizing the gameplay. 

Sound can be measured with verious criterias including velocity, pitch, energy and loudness. In our initial design, we want to design a “pitch” based game to encourage users play with music pieces by tuning the pitchs and pausing the pieces. However, the existing pitch detecting algorithms in Processing is not powerful enough to support our idea, and in addition, lots of players are too shy to sing out in their own voice. Instead, we change it into a “loudness” control game. It enables us to develop a more inclusive and fun gaming experience.  

For the gameplay, there were lots of existing games that inspired us. Besides “YASUHATI / With your voice!”, which is mentioned above, another game called “Line Rider” was also included in our early gameplay design. Below is a map design in Line Rider.

DoodleChaos [@DoodleChaos]. (2017, October 7). Line Rider – Mountain King. Youtube. https://www.youtube.com/watch?v=RIz3klPET3o

We originally proposed that the “builders” could draw these lines by changing their pitches, and generate blanks if they stop making sounds. The “navigators” need to control the “rider” to navigate through the scene with the controllers, and the controllers would have four functions: rotate clockwise, rotate counterclockwise, jump up and dive down.

Here is an early demo with the pitchs from a piece of music.  This demo indicated a major drawback in the original design: the lines drawed with pitchs were too dense that the “rider” were impossible to ride on.

This is the user test version of “Soaring Over the Sound City.” 

In the user testing, we recieved positive feedbacks in the gameplay and the multiple-player ideas. Testers also gave us suggestions on the gaming experience and guidance. They hope we can control the difficulties of the game as in certain circumstances, the blocks will be too close to the boundaries for the “navigator” to travel through. Another improvement that we adopted was a better instruction in the tutorial. In this version we only implemented some simple guidances on the functions of each key, and the players might be confused about the actual gameplay if we did not inform them. Our change is adding a full tutorial section for users to freely explore the interactions assisted by our minimal guidances.

Below is the video of us interacting with our project:

 

3. Fabrication And Production

Our project is a coding based project, and our fabrication is focused on the controller with a 3D printer. 

In this project, our objective was to pay tribute to retro arcade games, which hold a special place in the history of gaming and in the hearts of many enthusiasts. Its nostalgia looking fits the 8-bit theme inside our game comparing to other designs like a modernized joystick. By building a controller that mimics the style and functionality of retro arcade game controllers, we aim to capture the essence of classic gaming experiences. 

Below are the sketches for the arcade controller:

Case

Cover

3D-Printed Cover

When assembling the controller, we started by integrating buttons into the pre-drilled holes on the cover. Each button is carefully aligned with a corresponding hole on the cover, ensuring a snug and secure fit. This prevents any wobbling or misalignment during gameplay, which is essential for precise control.

After placing the buttons into their designated spots on the controller’s cover, the next step involves soldering and wiring these buttons to ensure they are functionally integrated with the controller’s electronics.  The arduino is used to detect whether each button is pressed down or not.

The controller with the circuits

This is the code for Arduino:

void setup() {

Serial.begin(9600);

pinMode(2,INPUT);

pinMode(3,INPUT);

pinMode(4,INPUT);

pinMode(5,INPUT);

pinMode(6,INPUT);

pinMode(7,INPUT);

}

void loop() {

int button1State = digitalRead(2);

int button2State = digitalRead(3);

int button3State = digitalRead(4);

int button4State = digitalRead(5);

int button5State = digitalRead(6);

int button6State = digitalRead(7);

Serial.print(button1State);

Serial.print(",");

Serial.print(button2State);

Serial.print(",");

Serial.print(button3State);

Serial.print(",");

Serial.print(button4State);

Serial.print(",");

Serial.print(button5State);

Serial.print(",");

Serial.print(button6State);

Serial.println();

delay(20);

}

Then I will give an overview of key components and functionalities within the code as we adopted the methodology of object-oriented programming.

Main Game Loop (void draw())

This function is the heart of the game loop, responsible for updating all aspects of the game state:

Handles different game modes using a switch-case structure (modes for intro, tutorial, game start, map loading, and game over).

Calls specific functions based on the current game mode to handle game logic, drawing, and interactions.

Game Modes

Each mode in the game has a specific function:

Intro Mode (void Intro()): Displays initial instructions and allows the player to select a game mode.

Tutorial Mode (void Tutorial()): Guides the “builder” and “navigator” through game mechanics using a step-by-step tutorial.

Constructor Mode (void constructor()): Allows the “builder” to influence the game map using sound input, creating platforms of varying heights based on the volume detected.

Navigator Mode (void Navigator()): The “navigator” navigates the character through the map, attempting to collect coins and avoid obstacles.

Map Loading (void Map_load()): Handles selection and loading of pre-defined or random maps.

Interaction with Arduino

The script reads data from the Arduino via serial communication to get input from physical buttons:

Serial Input (void getSerialData()): Reads serial data to get button presses from Arduino, influencing game state for jumping and state selection.

Utility Functions

Several helper functions manage game dynamics:

Map Generation (void Map_generate()): Dynamically generates the game map based on sound input or pre-loaded configurations.

Character States (void state0(), void state1(), etc.): Manage different states of the character such as moving, jumping, or falling.

This is the code for Processing:

Minim minim;
AudioInput in;
SoundFile soundFile;
SoundFile music;
AudioIn input;
SoundFile sound;
import processing.sound.*;
import ddf.minim.*;
import ddf.minim.analysis.*;
import processing.serial.*;

PImage bgImage, cImage, treeImage, rbImage, transparentImage, vmImage, bdImage, coinImage; // Background, character
PImage warnImage;
PGraphics pg, pg2;//pg2 for coin
float volume;
Serial serialPort;

//Game initialization
int mode = 0;//1 for tutorial, 2 for direct start, 3 for load map, 4 for gameover(win)
int state = 0;//For Mode2/Mode3-Gamer1
int cur_width = 1200;//should be changed accordingly
int cur_height = 800;//should be changed accordingly
boolean gameStarted = false;
boolean gameOver = false;//mode 0-3 false, mode 4 true
boolean sound_exist = false; //For (from gamer0_0 to gamer0_1)
int move_state = 1;//1 for moving right, 2 for moving left in Intro
int death_count = 0;


//Variable initialization for the characters
int size = 40;  // character size
int speed = 15;  // character speed
int speed_fall = 18; // character fall speed
float x, y; // The character's x and y position
int lastStateChange;
int cur_i;
int new_cur;

//Map initialization with rectangle
int rec_x = 50; //The width of each rectangle on the x axis
int hole_size = 5;
int numRects = cur_width / rec_x;  // Calculate the total number of rectangles
float[] rect_height = new float[numRects]; //The height for each rectangle
float[] rect_starting_x = new float[numRects];
float[] rect_height_tuto = new float[numRects];
float[] rect_starting_x_tuto = new float[numRects];
int j = 0;
int mini_height = 2;

//Map initialization with images
boolean pg_drawn = false;
boolean pg2_drawn = false;

//Map Memory
int map_numbers = 6;
int map_idx = 0;
int check_idx = 100;//For checking the available index
boolean loading_map = false;
float[][] Memo_rect_height = new float[map_numbers][numRects];
float[][] Memo_rect_starting_x = new float[map_numbers][numRects];
PImage[][] Image_for_x = new PImage[map_numbers][numRects];
PImage[] Image_for_cur_map = new PImage[numRects];
boolean[][] Memo_coin = new boolean[map_numbers][numRects];
int[] Memo_current_coin = new int[map_numbers];
int current_coin = 0;
boolean[] coin_for_cur_map = new boolean[numRects];
boolean[] Map_saved = new boolean[map_numbers];
boolean map_loaded = false;

//Other Initialization
int NUM_OF_VALUES_FROM_ARDUINO = 6;
int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO];
boolean[] keys = new boolean[128];//For Key Press
int time = 1; //It is useless right now
float perc = 0.3; //Fault tolerance Percentage
int gamer = 0; //0 is constructor, 1 is navigator
int cons_tutor_state = 1;
boolean special_ability_used = false;
int force_exit= 0;
long force_exit_startTime = 0;
boolean music_play = true;

boolean[] press_button_exit = new boolean[3];
boolean exit_botton_pressed = false;
boolean exit_botton_previous =false;
//red:a[0] or  keys[H]


void setup() {
  String[] portNames = Serial.list();
  for (String port : portNames) {
    println(port);
  }
  serialPort = new Serial(this, "/dev/cu.usbmodem11201", 9600);
  size(1200, 800);
  pg = createGraphics(1200, 800);
  pg2 = createGraphics(20, 20);
  cur_width = width;
  cur_height = height;
  numRects = cur_width / rec_x;
  bgImage = loadImage("bg3_2.jpg");
  cImage = loadImage("character.png");
  treeImage = loadImage("tree.png");
  rbImage = loadImage("Roadblock.png");
  transparentImage = loadImage("transparent.png");
  vmImage = loadImage("VendingMachine.png");
  bdImage = loadImage("bd.png");
  warnImage = loadImage("warning.png");
  coinImage = loadImage("Coin.png");
  music = new SoundFile(this, "music.mp3");
  music.play();
  music.loop();

  x = 0;
  y = height - size;
  minim = new Minim(this);
  in = minim.getLineIn(Minim.MONO);

  //Tutorial Design For Navigator
  for (int i = 0; i < numRects; i++) {
    Memo_current_coin[0] = 12;
    Memo_rect_height[0][i] = 0;
    Memo_rect_starting_x[0][i] = rec_x * i;
    if (i % 4 == 0) {
      Memo_rect_height[0][i] = i*20;
      Memo_rect_height[0][i] = constrain(Memo_rect_height[0][i], 10, height*0.5);
      if (Memo_rect_height[0][i]<=mini_height) {
        Image_for_cur_map[i] = transparentImage;
      } else if (Memo_rect_height[0][i]<=30) {
        Image_for_cur_map[i] = rbImage;
      } else if (Memo_rect_height[0][i]<=100) {
        Image_for_cur_map[i] = vmImage;
      } else if (Memo_rect_height[0][i]<=200) {
        Image_for_cur_map[i] = treeImage;
      } else if (Memo_rect_height[0][i]<=300) {
        Image_for_cur_map[i] = bdImage;
      } else {
        Image_for_cur_map[i] = treeImage;
      }
      Image_for_x[0][i] = Image_for_cur_map[i];
    } else {
      Memo_rect_height[0][i] = 0;
      Image_for_cur_map[i] = transparentImage;
    }
    Image_for_x[0][i] = Image_for_cur_map[i];
  }
  Map_saved[0] = true;
}

void draw() {

  frameRate(60);
  stroke(0);
  image(bgImage, 0, 0, width, height);
  getSerialData();
  if (force_exit == 1) {
    mode = 0;
    return;
  }
  println(force_exit);
  println(music_play);

  if (music_play) {
    if ((mode == 1 && gamer == 0) || (mode == 2 && gamer == 0))
    {
      music.stop();
      music_play = false;
    } else {
    }
  } else if (!music_play) {
    if (!((mode == 1 && gamer == 0) || (mode == 2 && gamer == 0)))
    {
      music.play();
      music.loop();
      music_play = true;
    } else {
    }
  }


  //Intro
  if (mode == 0) {
    Intro();
    return;
  }
  //Tutorial
  else if (mode == 1) {
    Tutorial();
  } else if (mode == 2) {
    if (gamer == 0) {
      constructor();
    } else if (gamer == 1) {
      Map_generate();
      Navigator();
    }
  } else if (mode == 3) {
    if (!map_loaded) {
      Map_load();
      return;
    }
    if (!loading_map) {
      frameRate(60);
      Map_generate();
      Navigator();
    }
  } else if (mode == 4 || gameOver) { //When the Navigator wins
    win();
  }
}



void Intro() {
  //Initialization
  pg = createGraphics(1200, 800);
  pg2 = createGraphics(20, 20);
  j = 0;
  state = 0;
  special_ability_used = false;
  pg_drawn = false;
  loading_map = false;
  map_loaded = false;
  pg_drawn = false;
  gameStarted = false;
  gameOver = false;
  sound_exist = false;
  pg2_drawn = false;
  cons_tutor_state = 1;
  current_coin = 0;
  gamer = 0;
  map_idx = 0;
  death_count = 0;


  //Move of the character
  image(cImage, x, height-100, size, size);
  if (move_state % 2 == 1) {
    x = x + speed;
    if (x + size >= width-1) {
      move_state ++;
      x = width - size;
    }
  } else {
    x = x - speed;
    if (x <= 0) {
      x = 0;
      move_state ++;
    }
  }
  //Mode choosing
  if ((keys['T']||keys['t']) ) {
    mode = 1;
  }
  if ((keys['K']||keys['k']) ) {
    gameStarted = true;
    mode = 2;
  }
  if ((keys['L']||keys['l']) ) {
    gameStarted = true;
    mode = 3;
  }
  //Instruction Text
  fill(255, 255, 255, 200);
  stroke(255);
  textSize(46);
  textAlign(CENTER);
  String text = "Press T to Begin Tutorial for Beginners!\nPress K to Directly Start a New Game!\nPress L to Load Maps!";
  float textWidth = textWidth(text);
  float textHeight = 46;
  float textx =  width / 2;
  float texty = height / 2;
  rect(textx - textWidth / 2-10, texty - textHeight / 2-20, textWidth+20, textHeight*3+25);
  fill(255, 0, 0);
  text(text, textx, texty);
  return;
}


void Tutorial() {
  if (gamer == 0) {
    if (cons_tutor_state == 1) {
      constructor_tuto1();
    } else if (cons_tutor_state == 2) {
      constructor_tuto2();
      return;
    }
  } else if (gamer == 1) {
    coin_for_cur_map[12] = true;
    current_coin = 12;
    map_idx = 0;
    mode = 2;
    if (!map_loaded) {
      map_idx = 0;
      for (int i = 0; i < numRects; i++) {
        rect_height[i] = Memo_rect_height[map_idx][i]; //The height for each rectangle
        rect_starting_x[i] = Memo_rect_starting_x[map_idx][i];
        Image_for_cur_map[i] = Image_for_x[map_idx][i];
      }
      map_loaded = true;
    }
    Map_generate();
    Navigator();
  }
}

void constructor_tuto1() {
  volume = in.left.level() * 100;
  float each_rectHeight_tuto = 800 / 20;
  float rect_height_tuto = volume * each_rectHeight_tuto;
  fill(255);
  rect_height_tuto = constrain(rect_height_tuto, 0, height*0.5);
  rect(575, height-rect_height_tuto-50, 50, rect_height_tuto);
  fill(255);
  textSize(32);
  text("This is the block generated by your current volume, continue by pressing RED BUTTON", 590, 790);
  if (keys['M']|| keys['m']||arduino_values[0]==1) {
    cons_tutor_state = 2;
  }
}

void constructor_tuto2() {
  //println(gamer);
  if (keys['N']|| keys['n']||arduino_values[3]==1) {
    gamer = 1;
    return;
  }
  if (keys[ENTER]) {
    cons_tutor_state = 1;
    j = 0;
    sound_exist = false;
    current_coin = 0;
  }

  image(bgImage, 0, 0, width, height);
  frameRate(4);
  fill(255);
  textSize(40);
  textAlign(CENTER);
  text("As a Constructor, try to make the map by changing your volume.\nYou can put a coin (for the navigator) by pressing RED BUTTON\n(Press BUTTON 4 if you are Navigator)", 590, 100);
  volume = in.left.level() * 100;
  if ((volume <= 0.5) && (!sound_exist)) {
    return;
  } else {
    sound_exist = true;
    if (j >= numRects) {
      int j_ = numRects-1;
      //gamer = 1;
      for (int i = 0; i <= j_; i++) {
        fill(255);
        //Attention0
        image(Image_for_cur_map[i], rect_starting_x_tuto[i]+hole_size, height - rect_height_tuto[i], rec_x-hole_size*2, rect_height_tuto[i]);
        if (coin_for_cur_map[i]) {
          image(coinImage, rect_starting_x_tuto[i]+hole_size+0.5*size, height - rect_height_tuto[i]-0.5*size, 0.5*size, 0.5*size );
        }
      }
    } else {
      float each_rectHeight = 800 / 20;
      int m = j * rec_x;
      rect_height_tuto[j] = volume * each_rectHeight;
      rect_starting_x_tuto[j] = m;
      fill(255);
      rect_height_tuto[j] = constrain(rect_height_tuto[j], 0, height*0.5);
      if ((!special_ability_used) && (keys['B']||keys['b']||arduino_values[0] == 1)&&rect_height_tuto[j] > mini_height) {
        coin_for_cur_map[j] = true;
        current_coin = j;
        special_ability_used = true;
      }
      if (rect_height_tuto[j]<=mini_height) {
        Image_for_cur_map[j] = transparentImage;
      } else if (rect_height_tuto[j]<=30) {
        Image_for_cur_map[j] = rbImage;
      } else if (rect_height_tuto[j]<=100) {
        Image_for_cur_map[j] = vmImage;
      } else if (rect_height_tuto[j]<=200) {
        Image_for_cur_map[j] = treeImage;
      } else if (rect_height_tuto[j]<=300) {
        Image_for_cur_map[j] = bdImage;
      } else {
        Image_for_cur_map[j] = treeImage;
      }
      println(j);
      //Attention1
      image(Image_for_cur_map[j], rect_starting_x_tuto[j]+hole_size, height - rect_height_tuto[j], rec_x-hole_size*2, rect_height_tuto[j]);
      if (coin_for_cur_map[j]) {
        image(coinImage, rect_starting_x_tuto[j]+hole_size+0.5*size, height - rect_height_tuto[j]-0.5*size, 0.5*size, 0.5*size );
      }
      //rect(rect_starting_x_tuto[j]+hole_size, height - rect_height_tuto[j], rec_x-hole_size*2, rect_height_tuto[j]);
      //println(height -rect_height[j]);
      for (int i = 0; i <= j; i++) {
        println(j);
        println(i);
        fill(255);
        //Attention2
        image(Image_for_cur_map[i], rect_starting_x_tuto[i]+hole_size, height - rect_height_tuto[i], rec_x-hole_size*2, rect_height_tuto[i]);
        //rect(rect_starting_x_tuto[i]+hole_size, height - rect_height_tuto[i], rec_x-hole_size*2, rect_height_tuto[i]);
        if (coin_for_cur_map[i]) {
          image(coinImage, rect_starting_x_tuto[i]+hole_size+0.5*size, height - rect_height_tuto[i]-0.5*size, 0.5*size, 0.5*size );
        }
      }
      j ++;
    }
  }
}

void constructor() {
  //j = 0;
  frameRate(4);
  fill(255);
  textSize(32);
  textAlign(CENTER);
  text("Use Your Voice to Create Blocks", width / 2, height / 2);
  textAlign(CENTER);
  text("Press RED BUTTON to Generate A Coin", width/2, height/2+30);
  fill(255, 0, 0, 100);
  rect(0, 0, width, height*0.34);
  image(warnImage, 0, 0, width, height*0.34);

  volume = in.left.level() * 100;
  if ((volume <= 0.5) && (!sound_exist)) {
    return;
  } else {
    sound_exist = true;
    if (j == numRects) {
      gamer = 1;
      return;
    }
    float each_rectHeight = 800 / 20;
    int m = j * rec_x;
    rect_height[j] = volume * each_rectHeight;
    rect_starting_x[j] = m;
    fill(255);
    rect_height[j] = constrain(rect_height[j], 0, height*0.5);
    if ((!special_ability_used) && (keys['B']||keys['b']||arduino_values[0] == 1)&&rect_height[j] > mini_height) {
      coin_for_cur_map[j] = true;
      current_coin = j;
      special_ability_used = true;
    }
    if (rect_height[j]<=mini_height) {
      Image_for_cur_map[j] = transparentImage;
    } else if (rect_height[j]<=30) {
      Image_for_cur_map[j] = rbImage;
    } else if (rect_height[j]<=100) {
      Image_for_cur_map[j] = vmImage;
    } else if (rect_height[j]<=200) {
      Image_for_cur_map[j] = treeImage;
    } else if (rect_height[j]<=300) {
      Image_for_cur_map[j] = bdImage;
    } else {
      Image_for_cur_map[j] = treeImage;
    }
    //Attention3
    //rect(rect_starting_x[j]+hole_size, height - rect_height[j], rec_x-hole_size*2, rect_height[j]);
    image(Image_for_cur_map[j], rect_starting_x[j]+hole_size, height - rect_height[j], rec_x-hole_size*2, rect_height[j]);
    if (coin_for_cur_map[j]) {
      image(coinImage, rect_starting_x[j]+hole_size+0.5*size, height - rect_height[j]-0.5*size, 0.5*size, 0.5*size );
    }
    for (int i = 0; i <= j; i++) {
      fill(255);
      //Attention4
      image(Image_for_cur_map[i], rect_starting_x[i]+hole_size, height - rect_height[i], rec_x-hole_size*2, rect_height[i]);
      if (coin_for_cur_map[i]) {
        image(coinImage, rect_starting_x[i]+hole_size+0.5*size, height - rect_height[i]-0.5*size, 0.5*size, 0.5*size );
      }
    }
    j ++;
  }
}


void Map_load() {
  image(bgImage, 0, 0, width, height);
  String trueIndices = "Map Saved With Specific Index:";
  String falseIndices = "Generate Random Map: 6";
  for (int i = 1; i < map_numbers; i++) {
    if (Map_saved[i]) {
      trueIndices += (trueIndices.endsWith(":") ? "" : ", ") + i;
    } else {
      falseIndices += (falseIndices.endsWith(":") ? "" : ", ") + i;
    }
  }
  fill(255, 255, 255, 200);
  stroke(255);
  textSize(46);
  textAlign(CENTER);
  String text = trueIndices + "\n" + falseIndices + "\nPlease Press The Corresponding Button To Load The Map!";
  float textWidth = textWidth(text);
  float textHeight = 46;
  float textx =  width / 2;
  float texty = height / 2;
  rect(textx - textWidth / 2-10, texty - textHeight / 2-20, textWidth+20, textHeight*3+25);
  fill(255, 0, 0);
  text(text, textx, texty);

  loading_map = true;
  for (int i = 0; i <= 5; i++) {
    if (arduino_values[i] == 1) {
      map_idx = i+1;
    }
  }

  if (map_idx != 0 && loading_map) {
    if (!(map_idx == 6)) {
      if (Map_saved[map_idx]) {
        for (int i = 0; i < numRects; i++) {
          current_coin = Memo_current_coin[map_idx];
          rect_height[i] = Memo_rect_height[map_idx][i]; //The height for each rectangle
          rect_starting_x[i] = Memo_rect_starting_x[map_idx][i];
          Image_for_cur_map[i] = Image_for_x[map_idx][i];
          coin_for_cur_map[i] = Memo_coin[map_idx][i];
          coin_for_cur_map[current_coin] = true;
        }
      } else if (!Map_saved[map_idx]) {
        for (int i = 0; i < numRects; i++) {
          float cur_height = (int) random(0, height*0.5);
          if (cur_height<=mini_height) {
            Image_for_cur_map[i] = transparentImage;
          } else if (cur_height<=30) {
            Image_for_cur_map[i] = rbImage;
          } else if (cur_height<=100) {
            Image_for_cur_map[i] = vmImage;
          } else if (cur_height<=200) {
            Image_for_cur_map[i] = treeImage;
          } else if (cur_height<=300) {
            Image_for_cur_map[i] = bdImage;
          } else {
            Image_for_cur_map[i] = treeImage;
          }
          current_coin = (int)random(0, numRects);
          coin_for_cur_map[current_coin] = true;
          Memo_rect_height[map_idx][i] = cur_height;
          Memo_rect_starting_x[map_idx][i] = i * rec_x;
          Image_for_x[map_idx][i] = Image_for_cur_map[i];
          Memo_coin[map_idx][i] = coin_for_cur_map[i];

          Map_saved[map_idx] = true;
          rect_height[i] = cur_height; //The height for each rectangle
          rect_starting_x[i] = i * rec_x;
        }
        check_idx = map_idx;
      }
      loading_map = false;
      map_loaded = true;
      gamer = 1;
    } else {
      for (int i = 0; i < numRects; i++) {
        float cur_height = (int) random(0, height*0.5);
        if (cur_height<=mini_height) {
          Image_for_cur_map[i] = transparentImage;
        } else if (cur_height<=30) {
          Image_for_cur_map[i] = rbImage;
        } else if (cur_height<=100) {
          Image_for_cur_map[i] = vmImage;
        } else if (cur_height<=200) {
          Image_for_cur_map[i] = treeImage;
        } else if (cur_height<=300) {
          Image_for_cur_map[i] = bdImage;
        } else {
          Image_for_cur_map[i] = treeImage;
        }
        rect_height[i] = cur_height; //The height for each rectangle
        rect_starting_x[i] = i * rec_x;
        current_coin = (int)random(0, numRects);
        coin_for_cur_map[current_coin] = true;
      }
      loading_map = false;
      map_loaded = true;
    }
  }
}



void Map_generate() {
  //println(coin_for_cur_map);

  //Don't Jump into this area
  if (!pg_drawn) {
    pg.beginDraw();
    pg.fill(255, 0, 0, 100);
    pg.rect(0, 0, width, height*0.34);
    pg.image(warnImage, 0, 0, width, height*0.34);
    for (int i = 0; i < numRects; i++) {
      fill(255);
      //Attention5
      pg.image(Image_for_cur_map[i], rect_starting_x[i]+hole_size, height - rect_height[i], rec_x-hole_size*2, rect_height[i]);
    }
    pg.endDraw();
    pg_drawn = true;
  }
  if (!pg2_drawn) {
    pg2.beginDraw();
    for (int i = 0; i < numRects; i++) {
      if (i == current_coin) {
        pg2.image(coinImage, 0, 0, 0.5*size, 0.5*size);
      }
    }
    pg2.endDraw();
    pg2_drawn = true;
  }

  image(pg, 0, 0);
  image(pg2, rect_starting_x[current_coin]+hole_size+0.5*size, height - rect_height[current_coin]-0.5*size, 0.5*size, 0.5*size);
}


void Navigator() {
  //println(coin_for_cur_map[current_coin]);
  if (coin_for_cur_map[current_coin]) {
    image(pg2, rect_starting_x[current_coin]+hole_size+0.5*size, height - rect_height[current_coin]-0.5*size, 0.5*size, 0.5*size);
  } else {
    pg2 = createGraphics(20, 20);
  }
  frameRate(60);
  //Intro
  if (state == 0) {
    state0();
  }
  //Wait
  if (state == 1) {
    state1();
  }
  //Jump
  if (state == 2) {
    state2();
  }
  //Falling Down
  if (state == 3) {
    state3();
  }
  if (state == 4) {
    state4();
  }
}


void state0() { //Intro for M2/3_G1
  image(cImage, rect_starting_x[0], height- rect_height[0] - size, size, size);
  x = rect_starting_x[0];
  y = height- rect_height[0] - size;
  fill(255);
  textSize(24); // Set text size
  textAlign(TOP); // Align text to center and bottom of the character image
  fill(0, 0, 0, 90);
  stroke(0, 0, 0, 90);
  rect(5, 5, 450, 200);
  fill(255);
  text("Press YELLOW BUTTON To Start This Turn!", 10, 30); // Display the prompt above the character
  text("Press LEFT BLUE BUTTON To Move Backward.", 10, 60);
  text("Press RIGHT BLUE BUTTON to Jump Higher.", 10, 90);
  text("Your Death Count is: "+ death_count, 10, 120);
  text("Notice that you need to approach \nthe coin (if any) once to pass this map.", 10, 150);
  if (keys['l']||keys['L']||(arduino_values[2]==1)) {
    state = 1;
  }
}

void state1() { //Game_Wait for M2/3_G1
  int numRects = width / rec_x;
  int cur_i = (int)(x / rec_x);
  if (cur_i < numRects-2) {
    cur_i ++;
  }

  coin_for_cur_map[cur_i-1] = false;
  image(cImage, x, y, size, size);
  if (keys['W']||keys['w']||arduino_values[1] == 1) {
    state = 2;
  }
}

void state2() {
  //The charac1
  image(cImage, x, y, size, size);

  //Don't move forward
  if (!(keys['A']||keys['a']|| arduino_values[5] == 1) ) {
    x += 5;
  } else {
    x -= 1;
  }
  //Jump, or you will fall
  if (keys['W']||keys['w']||(arduino_values[1] == 1)) {
    y -= speed;
  } else {
    y += speed_fall;
  }
  cur_i = (int)(x / rec_x);

  //Achieving the end
  if (x+size >= width ) {
    state = 0;
    if (coin_for_cur_map[current_coin]) {

      death_count ++;
    } else if (!coin_for_cur_map[current_coin]) {
      mode = 4;
      gameOver = true;
      return;
    }
  }
  //This is Lose
  if (y >= height || y <= height*0.3) {
    death_count ++;
    state = 0;
    return;
  }
  new_cur = cur_i;
  if (cur_i < (numRects-1)) {
    new_cur = cur_i + 1;
  }

  if (
    (x + size  >= rect_starting_x[new_cur] + hole_size && height-rect_height[new_cur] <= y)
    ||
    (x <= rect_starting_x[cur_i] - hole_size&& height-rect_height[cur_i] <= y)
    ||
    (rect_height[new_cur] <= mini_height && y + size >= height-mini_height)
    ||
    (rect_height[cur_i] <= mini_height && y + size >= height-mini_height)
    )
  {
    state = 4;
    lastStateChange = millis();
    return;
  }
  boolean Condi_1 = x + perc * size >= rect_starting_x[new_cur]
    && x + size <= rect_starting_x[new_cur] + rec_x
    ;
  boolean Condi_1_1 =
    y + size >= height-rect_height[new_cur] - 10
    ;
  boolean Condi_2 = x + size - perc * size <= rect_starting_x[cur_i] + rec_x
    &&
    x >= rect_starting_x[cur_i]
    ;
  boolean Condi_2_1 =
    y + size >= height-rect_height[cur_i] - 10
    ;

  if ((Condi_1 && Condi_1_1)) {
    state = 1;
    y = height-rect_height[new_cur]-size;
    ;
  }
  if (Condi_2 && Condi_2_1) {
    state = 1;
    y = height-rect_height[cur_i]-size;
  }
}


//Game_Fall

void state4() {
  image(cImage, x+5, y+2, size, size);
  image(Image_for_cur_map[cur_i], rect_starting_x[cur_i]+hole_size, height - rect_height[cur_i], rec_x-hole_size*2, rect_height[cur_i]);
  image(Image_for_cur_map[new_cur], rect_starting_x[new_cur]+hole_size, height - rect_height[new_cur], rec_x-hole_size*2, rect_height[new_cur]);
  if (millis()-lastStateChange > 500)
  {
    state = 3;
    death_count++;
  }
}

void state3() { //Game Fall for M2/3_G1
  image(cImage, x+5, y+2, size, size);
  image(Image_for_cur_map[cur_i], rect_starting_x[cur_i]+hole_size, height - rect_height[cur_i], rec_x-hole_size*2, rect_height[cur_i]);
  image(Image_for_cur_map[new_cur], rect_starting_x[new_cur]+hole_size, height - rect_height[new_cur], rec_x-hole_size*2, rect_height[new_cur]);
  y += 10;
  if (y-40 >= height) {
    state = 0;
    return;
  }
}

void win() {
  //println(Map_saved);
  String text;
  float textHeight = 46;
  float textx =  width / 2;
  float texty = height / 2;
  fill(255, 255, 255, 200);
  stroke(255);
  textSize(46);
  textAlign(CENTER);
  if (check_idx >= 6) {
    check_idx = (int)random(1, 6);
    for (int i = 1; i < Map_saved.length; i++) {
      if (!Map_saved[i]) {
        check_idx = i;
        break;
      }
    }
  }

  if (!(keys['S'] || keys['s'])) {
    String dc = "You Win with a death count of: " + death_count;
    text =dc + "!\nPress RED BUTTON to Play This Map Again!\nPress BUTTON 5 to Go Back to the Home Page\nPress S to Save this Map to Index " + check_idx;
    float textWidth = textWidth(text);
    rect(textx - textWidth / 2-10, texty - textHeight / 2-20, textWidth+20, textHeight*3+80);
    fill(255, 0, 0);
    text(text, textx, texty);
    if (keys[ENTER]||arduino_values[0]==1) {
      coin_for_cur_map[current_coin] = true;
      pg2_drawn = false;
      mode = 2;
      gamer = 1;
      state = 0;
      gameOver = false;
      death_count = 0;
    }
    if (keys['H'] || keys['h']||arduino_values[4]==1) {
      mode = 0;
      gamer = 0;
      state = 0;
      gameOver = false;
    }
  } else {
    for (int i = 0; i < numRects; i++) {
      Memo_rect_height[check_idx][i] = rect_height[i];
      Memo_rect_starting_x[check_idx][i] = rect_starting_x[i];
      Image_for_x[check_idx][i] = Image_for_cur_map[i];
      Memo_current_coin[check_idx] = current_coin;
    }
    Map_saved[check_idx] = true;
    check_idx ++;
    mode = 0;
    gamer = 0;
    state = 0;
    gameOver = false;
  }
  return;
}

//Additional
void keyPressed() {
  if (key < 128) keys[key] = true; // When the key is pressed, update
  if (key >= '1' && key <= '6') {
    int index = key;
   if (index >= 49) {
      index = index - 48;
    }
    if (mode == 3 && loading_map) {
      map_idx = index;
    }
  }
}

void keyReleased() {
  if (key < 128) keys[key] = false; // When the key is released, update
}

void getSerialData() {
  while (serialPort.available() > 0) {
    String in = serialPort.readStringUntil( 10 );
    if (in != null) {
      //print("From Arduino: " + in);
      String[] serialInArray = split(trim(in), ",");
      if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
        for (int i=0; i<serialInArray.length; i++) {
          arduino_values[i] = int(serialInArray[i]);
        }
      }
    }
  }
}

4. Conclusions

Our goal is to experiment with the methods of players interacting with a game, and we want to build a game that everyone can enjoy. And we choosed sounds as the medium of interaction. I believe that from the feedbacks that we received from the IMA final show, our project attracted people of different ages. Their laughter and encouragement proves that we are successful.

Building upon the success and positive feedback from the IMA final show, our next steps are to refine and expand the game’s accessibility and interaction capabilities. The enthusiasm and enjoyment expressed by players from diverse age groups underscore the potential of our game. Here’s how we can move forward:

1. Gameplay

Navigator:

In the user test, some testers had proposed that same as the “builder”, the “navigator” can be voice-controlled. Mimicing your friends’ voices to jump through the map can be great fun.

Builder:

It can be better if we are able to engage more users, or even a whole audience to come and enjoy the game together.

I have an idea of building a bridge for a character to walk through by referring to the joint sounds from a large crowd, so everyone is engaged in the game and involved in the progress of the character. This interactive gameplay concept could transform individual audience reactions into a collective, real-time game control system. By using sound recognition technology, the game would analyze the volume and pitch of the crowd’s reactions to determine the actions of the character on the screen. For instance, when the crowd claps loudly, the character might jump, while laughter could cause the character to dance or perform a special move. This would not only make the gaming experience more immersive and entertaining but also foster a sense of community and collaboration among the participants. As the crowd learns how their actions directly influence the game, they become more engaged, creating a dynamic and exciting atmosphere that enhances the overall experience.

2. Devices

The controller that we used can be confusing as the instructions of it is not intuitive, we can minimize it by only saving the components for example four buttons for from directions and perform the rests with other creative devices.

To conclude, the journey of developing this interactive game, beginning with its inception and feedback at the IMA final show, has been a testament to the power of community engagement in gaming. Our innovative approach in using sound recognition to integrate audience interaction not only enhances the gameplay but also fosters a unique sense of friendship among players. The enthusiastic participation from people of various age groups underscores the universal appeal of our game, it breaks the bias that  an experimental work is progressive and condescending, it can be something that everyone can enjoy.

5. Disassembly

6. Appendix

In the appendix I wanted to show some texture maps in our game.

Interaction Lab Midterm Project Blog

1. Tic-Tac-Mole! – Junhao Zhu & Jiaxi Zhang – Gottfried Haider

Here is a picture of our project.

2. Context & Significance

In our project, we are trying to reimagine how the classical game in our childhood can be more enjoyable to modern players. Among many candidates that we spent a lot of time on when we were children, we chose Whac-A-Mole, an arcade game in which multiple holes in the play area top are filled with small, plastic, cartoonish moles, which pop up at random. Points are scored by whacking each mole with the hammer as it appears. The faster the reaction, the higher the score. The game demands quick reflexes, hand-eye coordination, and timing from the player to successfully hit the moles as they appear.

By sa_ku_ra / sakura – *Source: Flickr image., CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=36174230

To add interactivity to the original design, we applied the rules of Tic-Tac-Toe, a paper-and-pencil game for two players who take turns marking the spaces in a three-by-three grid with X or O. The player who succeeds in placing three of their marks in a horizontal, vertical, or diagonal row is the winner.

By Jkca Newton – *Source: Board Game Wiki, https://board-games.fandom.com/wiki/Tic-tac-toe

Since Tic-Tac-Toe is a solved game, which means the first player will always win if he or she follows an optimal strategy, we think the randomness from the Whac-A-Mole can make the simple Tic-Tac-Toe challenging. Similarly, the rules from Tic-Tac-Toe can make the project more strategic as it involves predicting the necessity of each move within a limited time. Altogether, the project involves a blend of unpredictability, skill, and certain thinking, making the project interactive for players.

3. Conception & Design

In our original design, we were trying to build a multiplayer game with a combination of Tic-Tac-Toe and Whac-A-Mole, the concept is shown in the image below.

The box behind would act as an “arcade”. We designed that the sticks attached to servos would indicate moles, and they would randomly flip up in every round of the game. Two players would take turns to hit the corresponding square on the cardboard in front of them. Capacitive sensors were hidden behind to sense whether the current player hit the mole with their hands correctly, and if so, lit up the player’s LED on the “arcade”.  We also designed a rule book in our initial design.

In each round, the current player can choose either to “stay”, which is not to hit the mole, or to “move”, which is to hit. If he or she “stays”, the position changes to the other player. If he or she “moves”, there will be 2 circumstances: he or she hits the “right” mole and hits the “wrong” mole. If he or she hits the “right” mole, the LED on the corresponding square will be lit and the game will move to the next round. If he or she hits the “wrong” mole, there will be a penalty round. The other player will be offered an extra round for hitting the mole without any further penalty if he or she also hits the wrong mole.

However, several drawbacks existed in this design made us need to compromise and enhance the project. 

  1. The complicated rule book. The users could not easily understand how to interact with the projects without our detailed explanations and will require a long period to be familiar with the gameplay process. That is to say, our program is not intuitive and is of poor interactivity.
  2. The multiplayer design. Within our current design, it is hard for the Arduino to track which player is currently making the move. Different issues will occur if the users do not follow the instructions correctly, and they might confuse the users in eventually enjoying playing this game.
  3.  The efficiency in mechanism. To specify, this issue refers to the excessive emphasis on displaying the status on the “arcade”. If we want to control nine servos individually, the power supply would be an issue, and it would also take up too many digital pins on the Arduino. 

We solved these three questions in our final design. The game is now in single-player mode which the user plays against the Arduino. In the code, the Arduino will randomly pick a square that lights up the LED and will earn that point if the user does not hit the right square or not hit at all. For the rules, we just maintain those essential components of Tic-Tac-Toe: the player who succeeds in placing three of their marks in a horizontal, vertical, or diagonal row is the winner. This alternative plan ensures the project is more easy to play with without sacrificing much delight.

To enhance efficiency, we choose neopixels in both displaying the moles’ positions and the chessboard position by changing the individual LED’s blinking status. This option will significantly reduce the over-consumption of digital pins and reach our purpose.

4. Fabrication & Production

In the first part, I will introduce the main structure of this project: a display board and a control panel.

Display Board:


The image shown above is a double-layered cardboard with embedded neopixels in our prototype. This cardboard will show the current status of the “mole” and the chessboard.  The video shows the process when we tested whether it worked properly in the prototype of a 2×2 version of Tic-Tac-Mole!  

Here is a video of us testing the feasibility of neopixels.

In our initial design, we decided to drill holes in one cardboard and thread the neopixels through them as the basic structure of the display board. However, we found that the pads on neopixel LEDs were extremely vulnerable because they could easily be dragged down when we were threading.

To solve this issue, we came up with an alternative plan to protect the pads. We soldered nine LEDs into a long strip, first securing them on one piece of cardboard, then drilling holes at corresponding positions on another piece of cardboard to expose the LEDs, and finally, sticking the two pieces of cardboard together. The double-layered board will act as a cushion from the outside force. 

This new design had proven its accessibility and durability in the user test, and we kept this design in our final result.

Control Panel:

This image is the circuit of the capacitive sensors. The 9 sensors will take the digital pins 2 to 10 on the Arduino. The capacitive sensors are soldered to the copper tapes that are stuck to the panel in front of the users. 

We also used a hammer to enhance immersiveness for users. The hammer is wrapped up with copper tapes so that when the user holds it with their hands and smashes it on the panel, the electric field changes, and the capacitive sensor can sense their movements with the DigitalRead functions, then light up the corresponding LED with the pre-assigned addresses in the code. 

Code:

#include FastLED.h
#include time.h
int cur_pos; //The current position of the mole
#define NUM_LEDS 9// How many LEDs are on your strip?
#define DATA_PIN 11

CRGB leds[NUM_LEDS];
int chessboard[9] = {0,0,0,0,0,0,0,0,0}; //0 for no one, 1 for player, 2 for mole
int sensor_array[9] = {0,0,0,0,0,0,0,0,0};
int state = 1;//control state
long startTime;
int WoL_state = 0;

void setup() {
  Serial.begin(9600);
  pinMode(2, INPUT);
  pinMode(3, INPUT);
  pinMode(4, INPUT);
  pinMode(5, INPUT);
  pinMode(6, INPUT);
  pinMode(7, INPUT);
  pinMode(8, INPUT);
  pinMode(9, INPUT);
  pinMode(10, INPUT);
  randomSeed(analogRead(A0));
  FastLED.addLeds<NEOPIXEL, DATA_PIN>(leds, NUM_LEDS);
  FastLED.setBrightness(50); 
}

void loop() {
  // Serial.println(state);
  if (state == 1) {//game time
    state1();
  } else if (state == 2) {// win/lose/draw time
    state2();
  }
}


// main game
void state1() {
  for (int i = 0; i <= 8; i++){
    sensor_array[i] = digitalRead(i + 2);
  }
  cur_pos = random(0,9);
  while (chessboard[cur_pos] != 0) {
  cur_pos = random(0,9);
  }
  chessboard[cur_pos] = 2;
  for (int i = 0; i <= 3; i++){
    leds[cur_pos] = CRGB(255, 165, 0);
    FastLED.show();
    if (sensor_array[cur_pos] == 1) {
      chessboard[cur_pos] = 1;
      leds[cur_pos] = CRGB(0, 255, 0);
      FastLED.show();
    }
    delay(100);
    if (sensor_array[cur_pos] == 1) {
      chessboard[cur_pos] = 1;
      leds[cur_pos] = CRGB(0, 255, 0);
      FastLED.show();
    }
    leds[cur_pos] = CRGB(0, 0, 0);
    FastLED.show();
    if (sensor_array[cur_pos] == 1) {
      chessboard[cur_pos] = 1;
      leds[cur_pos] = CRGB(0, 255, 0);
      FastLED.show();
    }
    delay(100);
    if (sensor_array[cur_pos] == 1) {
      chessboard[cur_pos] = 1;
      leds[cur_pos] = CRGB(0, 255, 0);
      FastLED.show();
    }
    for (int i = 0; i <= 8; i++){
    sensor_array[i] = digitalRead(i + 2); 
  }

    if (sensor_array[cur_pos] == 1) {
      chessboard[cur_pos] = 1;
      leds[cur_pos] = CRGB(0, 255, 0);
      FastLED.show();
      break;
    }
  }
  if (chessboard[cur_pos] == 2){
      leds[cur_pos] = CRGB(255, 0, 0);
      FastLED.show();
  }
  delay(1000);
  if (chessboard[2] == 1 && chessboard[1] == 1 && chessboard[0] == 1 || 
  chessboard[3] == 1 && chessboard[4] == 1 && chessboard[5] == 1 || chessboard[8] == 1 && chessboard[7] == 1 && chessboard[6] == 1 || chessboard[2] == 1 && chessboard[3] == 1 && chessboard[8] == 1 || chessboard[1] == 1 && chessboard[4] == 1 && chessboard[7] == 1|| chessboard[0] == 1 && chessboard[5] == 1 && chessboard[6] == 1||chessboard[2] == 1 && chessboard[4] == 1 && chessboard[6] == 1||chessboard[0] == 1 && chessboard[4] == 1 && chessboard[8] == 1){
    state = 2;
    WoL_state = 1;
  }
  else if (chessboard[2] == 2 && chessboard[1] == 2 && chessboard[0] == 2 ||
   chessboard[3] == 2 && chessboard[4] == 2 && chessboard[5] == 2 || chessboard[8] == 2 && chessboard[7] == 2 && chessboard[6] == 2 || chessboard[2] == 2 && chessboard[3] == 2 && chessboard[8] == 2 || chessboard[1] == 2 && chessboard[4] == 2 && chessboard[7] == 2|| chessboard[0] == 2 && chessboard[5] == 2 && chessboard[6] == 2||chessboard[2] == 2 && chessboard[4] == 2 && chessboard[6] == 2||chessboard[0] == 2 && chessboard[4] == 2 && chessboard[8] == 2){
    state = 2;
    WoL_state = 2;
  }
  else if (chessboard[0] != 0 && chessboard[1] != 0 && chessboard[2] != 0 && chessboard[3] != 0 && chessboard[4] != 0 && chessboard[5] != 0 && chessboard[6] != 0 && chessboard[7] != 0 && chessboard[8] != 0){
    state = 2;
  }
}
// outro
void state2() {
  
  leds[0] = CRGB(0,0,0);
  leds[1] = CRGB(0,0,0);
  leds[2] = CRGB(0,0,0);
  leds[3] = CRGB(0,0,0);
  leds[4] = CRGB(0,0,0);
  leds[5] = CRGB(0,0,0);
  leds[6] = CRGB(0,0,0);
  leds[7] = CRGB(0,0,0);
  leds[8] = CRGB(0,0,0);

  if (WoL_state == 1){
    leds[0] = CRGB(255,0,0);
    FastLED.show();
    delay(50);
    leds[1] = CRGB(255,165,0);
    FastLED.show();
    delay(50);
    leds[2] = CRGB(255,255,0);
    FastLED.show();
    delay(50);
    leds[3] = CRGB(0,255,0);
    FastLED.show();
    delay(50);
    leds[4] = CRGB(0,255,255);
    FastLED.show();
    delay(50);
    leds[5] = CRGB(0,0,255);
    FastLED.show();
    delay(50);
    leds[6] = CRGB(128,0,255);
    FastLED.show();
    delay(50);
    leds[7] = CRGB(255,0,255);
    FastLED.show();
    delay(50);
    leds[8] = CRGB(255,0,128);
    FastLED.show();
    delay(50);
    leds[0] = CRGB(0,0,0);
    FastLED.show();
    delay(50);
    leds[1] = CRGB(0,0,0);
    FastLED.show();
    delay(50);
    leds[2] = CRGB(0,0,0);
    FastLED.show();
    delay(50);
    leds[3] = CRGB(0,0,0);
    FastLED.show();
    delay(50);
    leds[4] = CRGB(0,0,0);
    FastLED.show();
    delay(50);
    leds[5] = CRGB(0,0,0);
    FastLED.show();
    delay(50);
    leds[6] = CRGB(0,0,0);
    FastLED.show();
    delay(50);
    leds[7] = CRGB(0,0,0);
    FastLED.show();
    delay(50);
    leds[8] = CRGB(0,0,0);
    FastLED.show();
    delay(50);
  }
if (WoL_state == 2){
    leds[0] = CRGB(255,0,0);
    leds[2] = CRGB(255,0,0);
    leds[4] = CRGB(255,0,0);
    leds[6] = CRGB(255,0,0);
    leds[8] = CRGB(255,0,0);
    FastLED.show();
    delay(200);
    leds[0] = CRGB(0,0,0);
    leds[2] = CRGB(0,0,0);
    leds[4] = CRGB(0,0,0);
    leds[6] = CRGB(0,0,0);
    leds[8] = CRGB(0,0,0);
    FastLED.show();
    delay(200);
}
if (WoL_state == 0){
  leds[0] = CRGB(255,255,255);
  leds[1] = CRGB(255,255,255);
  leds[2] = CRGB(255,255,255);
  leds[3] = CRGB(255,255,255);
  leds[4] = CRGB(255,255,255);
  leds[5] = CRGB(255,255,255);
  leds[6] = CRGB(255,255,255);
  leds[7] = CRGB(255,255,255);
  leds[8] = CRGB(255,255,255);
  FastLED.show();
}
}

Since this project is strongly coding-based, I will explain a bit about how we conduct the coding. At the very beginning, we divided the program into two states: state 1 was for recording the movements of the users, indicating the places of moles, and examining whether the winner won, lost, or drew the game by continuously measuring all possible lines horizontally, vertically, and diagonally. We adopted a random seed function to make sure that the mole would appear at a completely random place at the beginning of each game since if removed, the LEDs would be lit on in the same pattern. This was a precious experience that we learned from our user test.

After several rounds of the game, it would finally end in one of the three statuses: win, lose, and draw. In this situation, we would move to status 2, a status that indicates the result to the users. Here are three clips for different results:

Win:

Lose:

Draw:

Experience from User Test:

In the user test, we exhibited a prototype of a 2×2 version of Tic-Tac-Mole!  as the previous video showed. We received valuable feedback from our testers. Compared to the prototype, we have added guides on the display board to help users better understand the game rules. We also installed a servo motor on the upper side of the display board to control a hammer for hitting the mole, increasing the fun factor. Additionally, we are using a box to cover the breadboard and Arduino on the control panel, making the entire project look like an arcade machine.

In the process of building this project, I was responsible for the coding and the electronics while my teammate, Jiaxi Zhang, put her effort into the construction and improvements of our design. Thank her for the great job!

5. Conclusion

This project is a reimagination of a more enjoyable classical game. We generated our inspiration from two games that people of our generation are familiar with in our childhood: Tic-Tac-Toe and Whac-A-Mole to create a new game that everyone can enjoy. The project involves unpredictability, skill, and certain thinking from the users, and these are the features that generate interactivity. We believe that after our advancements, the game itself is easy to start with but will take some time to fully master it, and that’s why it is attractive.

However, this project is absolutely not that perfect, and more improvements can be made. The first is the choice of sensors. Due to the working principle of touching capacitive sensors, such sensors will monitor inputs even if the user is not placing the hammers on the panel since the electric field can change even when the copper tapes are at a distant place above the panel. The touching capacitive sensors are oversensitive for this project. 

Another issue is the continuity of the projects. In the current design, if the user wants to start again, he or she can only press the reset on the Arduino. We can add a button to the “arcade” so the user can press it to restart, and we should add certain codes to record the winning/losing status for the user and display it if the user is curious.

The process of this project taught me that the initial idea can vary a lot from the final result. When we conceived the idea for the project, we put a lot of emphasis on the gameplay. We tried to make it special, to make it challenging, to make the player laugh out loud when interacting. However, when we started to build it and listened to our testers, we figured out that many compromises needed to be made due to the limitations of our abilities or the materials that we chose to build the project. For example, the sensitivity of the capacitive sensors posed unexpected challenges, requiring adjustments to ensure accurate gameplay; the complicated game rules that we thought were interesting enough seemed to be non-intuitive. By actively hearing from others, can we make our project better.

6. Disassembly

7. Appendix

Code for the prototype:

#include FastLED.h
#include time.h
int cur_pos, sAudioPin = 7; //The current position of the mole
#define NUM_LEDS 5// How many leds on your strip?
#define DATA_PIN 6
#define NOTE_A5  880
#define NOTE_B5  988
#define NOTE_C5  1047
#define NOTE_D5  1175
#define NOTE_E5  1319
#define NOTE_G4  392
#define NOTE_C4  262

CRGB leds[NUM_LEDS];
int chessboard[9] = {0,0,0,0,0}; 
int sensor_array[5] = {0,0,0,0,0};
int state = 2;
int button;
long startTime;
char WoL_state = "Draw";

void setup() {
  Serial.begin(9600);
  pinMode(13,INPUT);
  pinMode(2, INPUT);
  pinMode(3, INPUT);
  pinMode(4, INPUT);
  pinMode(5, INPUT);
  pinMode(sAudioPin, OUTPUT);
  randomSeed(analogRead(A0));
  FastLED.addLeds<NEOPIXEL, DATA_PIN>(leds, NUM_LEDS);
  FastLED.setBrightness(50); 
  for (int i = 0; i <= 8; i ++ ){
    leds[i] = CRGB(0, 0, 0);
    FastLED.show();
  }
}

void loop() {
  // Serial.println(state);
  if (state == 1) {
    state1();
  } else if (state == 2) {//game time
    state2();
  }
}

void state1() {
  button = digitalRead(13);
  if (button == HIGH){
    state = 2;
  }
}

void state2() {
  for (int i = 0; i <= 4; i++){
    sensor_array[i] = digitalRead(i + 1); 
  }
  cur_pos = random(1,5);
  Serial.println(cur_pos);
  while (chessboard[cur_pos] != 0) {
  cur_pos = random(1,5);
  }
  Serial.print(cur_pos);
  chessboard[cur_pos] = 2;
  for (int i = 0; i <= 3; i++){
    leds[cur_pos] = CRGB(255, 165, 0);
    FastLED.show();
    delay(100);
    leds[cur_pos] = CRGB(0, 0, 0);
    FastLED.show();
    delay(100);
    for (int i = 0; i <= 4; i++){
    sensor_array[i] = digitalRead(i + 1);
  }
    if (sensor_array[cur_pos] == 1) {
      chessboard[cur_pos] = 1;
      leds[cur_pos] = CRGB(0, 255, 0);
      FastLED.show();
      break;
    }
  }
  if (chessboard[cur_pos] == 2){
      leds[cur_pos] = CRGB(255, 0, 0);
      FastLED.show();
  }
  delay(1000);
  if (chessboard[1] == 1 && chessboard[2] == 1 || chessboard[3] == 1 && chessboard[4] == 1 || chessboard[1] == 1 && chessboard[3] == 1 || chessboard[2] == 1 && chessboard[4] == 1 || chessboard[1] == 1 && chessboard[4] == 1 || chessboard[2] == 1 && chessboard[3] == 1){
    state = 3;
    WoL_state = "Win";
  }
  else if (chessboard[1] == 2 && chessboard[2] == 2 || chessboard[3] == 2 && chessboard[4] == 2 || chessboard[1] == 2 && chessboard[3] == 2 || chessboard[2] == 2 && chessboard[4] == 2 || chessboard[1] == 2 && chessboard[4] == 2 || chessboard[2] == 2 && chessboard[3] == 2){
    state = 3;
    WoL_state = "Lose";
  }
} 

The illustration for the multiplayer gameplay in initial design: