Conception & Design
Soaring Over the Sound City is designed to be a novel multiplayer game that enables users to interact with the computer in a creative way, specifically voice and volume in our project. It is inspired by YASUHATI / With your voice!, an arcade game in which users use their voice to help the avatar travel across platforms and obstacles by controlling its jumping height. The basic structure of our project has two parts which would be completed by two users respectively. The first part is for “Constructor”, while the second is for “Navigator”. The constructors can use their voice to create a unique city in 4 seconds with 24 different components throughout the screen from left to right. Different volume levels will lead to different components with different heights such as vending machines, trees, and buildings at each time point. Then the navigator will use buttons to control the movement of the avatar in the second part to soar over the sound city created by the constructor without bumping into the items or falling down.
The concept is slightly different from the jumping game mentioned before because the contribution of voice is not limited to reacting to the information provided by the computer, but also creating something unique with the users’ own wisdom. Originally, the choice of controlling the character using voice is also considered. However, during the user testing session, we found out that Processing has an annoying delay when dealing with sound input, and it is easily influenced by environment noice as well. While the navigator part was somehow challenging for many users, we decided to give up this less controllable choice and still utilize the traditional button control for our final project .
Some other changes took place after the user testing session as well. The most significant change is that we decided to create a system for playing a map multiple times by saving and loading maps, which makes it similar to real-life video games. We made this decision because, during the user testing session, there were some users who complained about the inconvenience and impossibility of creating the same map, and each map can only be cleared once. They could not compete with their friend to see who was the best navigator. The Home pages (Intro) and Gameover pages (Victory) were changed. Moreover, a Map Loading page was created as well.
Moreover, as can be seen in the map loading page. When a save point is not used, our approach is to generate a new random map and save the map to it, and typing 6, which is not an index of the saving array, will also lead to the result of generating a new map for a Navigator to play. I suggested this decision because when I was writing the code for the Navigator part, I also used a random map to generate the result for my own testing without a pre-generated sound city. It was really a time killer of great fun since the jumping rule was quite tricky. It made this project a normal jumping game even though there might only be one player in a situation where it is inappropriate to make a sound. As a result, when combining my code with my partners’, I insisted on merging this random map part with the map loading system. We also add a death count which is reset to 0 every time a new map is loaded to encourage the competition between different players. Overall, we are happy to see that our system design aligns with Norman’s theory which activate the users enthusiasm with these competitive feedbacks. Below is a video which shows two kids trying to get the opportunity to play the navigator part.
We also made a constructors’ special ability possible. When using their voice to create a map, constructors could press a specific button (set to be the only red button in our project) to add a coin on the top of the item they have just created, and the navigator will have to get the coin to pass the map. I think this choice makes the map more challenging and interesting as well.
Fabrication & Production
Coding
I would like to talk about our code structure after the user testing instead of from the very beginning since the main structure of our project became very complicated after adding the map-saving-and-loading system. The original code before the user testing is all written in the draw() function, but after that, I reorganized the piles of code to call specific functions to make the code more readable. Now the main draw() function contains only a few lines with a variable to indicate the current mode (mode 0: Homepage(Intro), mode 1: Tutorial, mode 2: Game [mode 2 + gamer 0: constructor, mode 2 + gamer 1: navigator], mode 3: Map Loading, mode4: Win). I made this choice because different mode might be changed during the specific function call and might repeat in different situation. For example, for both normal game and tutorial, when a navigator has completed the map, the mode will be directly changed to 4 to call the win() function, and I don’t need to write the same code twice in this situation.
void draw() {
frameRate(60);
stroke(0);
image(bgImage, 0, 0, width, height);
getSerialData();
//Set the background music pause when microphone is used
//Continue when constructor is not building a map
if (music_play) {
if ((mode == 1 && gamer == 0) || (mode == 2 && gamer == 0))
{
music.stop();
music_play = false;
}
} else if (!music_play) {
if (!((mode == 1 && gamer == 0) || (mode == 2 && gamer == 0)))
{
music.play();
music.loop();
music_play = true;
}
}
//Intro
if (mode == 0) {
Intro();
return;
}
//Tutorial
else if (mode == 1) {
Tutorial();
} else if (mode == 2) {
if (gamer == 0) {
constructor();
} else if (gamer == 1) {
Map_generate();
Navigator();
}
} else if (mode == 3) {
if (!map_loaded) {
Map_load();
return;
}
if (!loading_map) {
frameRate(60);
Map_generate();
Navigator();
}
} else if (mode == 4 || gameOver) { //When the Navigator wins
win();
}
}
In fact, this is the most challenging part for code writing because the original code had about 300 lines and most of them did not have comments. Even though I wrote them in a week before, I still spent about over 3 hours to identify different functions of different part of my code and more hours to reconstruct the whole system. One lesson I learnt here is that when dealing with a project of high complexity, we need to first decide on the overall structure before going into the details. The sketch below is an overall structure I planned to construct after user testing.
Another challenge I met was that the game set was normal for the first play, but after returning to the home page, many details might go wrong. For example, the coin might be drawn even though a special ability is not used. This because my structure for map saving is using a temporary array to store the current map, and Memo_arrays to save others. I used an index to indicate to where a coin should be put. The code below is part of the memory initialization before setup().
int map_numbers = 6; int map_idx = 0; int check_idx = 100;//For checking the available index boolean loading_map = false; float[][] Memo_rect_height = new float[map_numbers][numRects]; float[][] Memo_rect_starting_x = new float[map_numbers][numRects]; PImage[][] Image_for_x = new PImage[map_numbers][numRects]; PImage[] Image_for_cur_map = new PImage[numRects]; boolean[][] Memo_coin = new boolean[map_numbers][numRects]; int[] Memo_current_coin = new int[map_numbers]; int current_coin = 0; boolean[] coin_for_cur_map = new boolean[numRects]; boolean[] Map_saved = new boolean[map_numbers]; boolean map_loaded = false;
To address this problem, I re-initializaed every possible variable for any state judging everytime get back to the home page (Intro()).
void Intro() {
//Initialization
pg = createGraphics(1200, 800);
pg2 = createGraphics(20, 20);
j = 0;
state = 0;
special_ability_used = false;
pg_drawn = false;
loading_map = false;
map_loaded = false;
pg_drawn = false;
gameStarted = false;
gameOver = false;
sound_exist = false;
pg2_drawn = false;
cons_tutor_state = 1;
current_coin = 0;
gamer = 0;
map_idx = 0;
death_count = 0;
The last thorny issue I want to discuss regarding coding is the use of image. All of the items were designed to be blocks (white rectangles with black strokes) at user testing. However, after I finished drawing the .png images for all of the items, I found that drawing about 24 pictures at the same time made the framerate lower than 15 per seconds, which made the movement of the navigator less smooth. I used another layer pg for the constructed city and a boolean variable to prevent redundant calls to graphics by setting it to true when drawn once. The appearance and disappearance of the coin are controlled in this way as well (using pg2). The following code is about the use of pg.
pg = createGraphics(1200, 800);
if (!pg_drawn) {
pg.beginDraw();
pg.fill(255, 0, 0, 100);
pg.rect(0, 0, width, height*0.34);
pg.image(warnImage, 0, 0, width, height*0.34);
for (int i = 0; i < numRects; i++) {
fill(255);
//Attention5
pg.image(Image_for_cur_map[i], rect_starting_x[i]+hole_size, height - rect_height[i], rec_x-hole_size*2, rect_height[i]);
}
pg.endDraw();
pg_drawn = true;
}
Fabrication
Since our project is more like a Processing-based arcade game, the physical fabrication is comparatively less challenging than the coding part. The main composition contains two parts: a microphone and a few buttons. The microphone we used has a knob and we could change the threshold to make sure the environment noise will not disturb the user voice too much. Moreover, we utilized 3-D printing fabrication to fix the buttons’ position and prevent possible damage using appropriately sized holes to hold the buttons. As I was more responsible for the coding part, my partner Mike was responsible for the 3-D model box design and adjustment. We used buttons of medium size and tried to use the corresponding colors for different functions.
Conclusion
Our project goal is to provide a new and creative way for the user to interact with the computer. I am convinced that our project makes it possible and inspires the users to use their voice to contribute to a competing game. Our project also gives appropriate feedback and inspires competitiveness among players which is similar to Norman’s theory. If time permits, I have several plans to improve our project. First, I want to add the function of controlling the avatar using voice in the navigator part since it is of great importance in the creative interaction field. More effort should have been put into the methods of dealing with noise, setting thresholds, and addressing data processing delays. Moreover, our game in fact lacks guidance for beginners, and that’s why we have a tutorial section to choose from. However, it is not enough because for average users (not gamers), the system is still way to complicated. I hope more interactive and intuitive design could be applied to make everyone’s use more smooth. The arcade design, for example, the choice of buttons could be improved as well. The experience during my final project gives me a lesson that I need to have a macro and overall perspective and the awareness to communicate with my peers in all aspects. In this way, can we improve the efficiency and save the time which originally used on ineffective extra work. Moreover, we cannot pre-assume that every user is familiar with our project. We need to improve the guidance and make it more interactive and understandable instead of only written words on the screen. In brief, this is the first video game I made using computer software that can be displayed on a computer screen. It may not be perfect, but it’s the best start I can think of. I hope in the future, I could still own the courage to enbrace and love my projects exactly like this one.
Disassembly
Appendix
Code-Processing
Minim minim;
AudioInput in;
SoundFile soundFile;
SoundFile music;
AudioIn input;
SoundFile sound;
import processing.sound.*;
import ddf.minim.*;
import ddf.minim.analysis.*;
import processing.serial.*;
PImage bgImage, cImage, treeImage, rbImage, transparentImage, vmImage, bdImage, coinImage; // Background, character
PImage warnImage;
PGraphics pg, pg2;//pg2 for coin
float volume;
Serial serialPort;
//Game initialization
int mode = 0;//1 for tutorial, 2 for direct start, 3 for load map, 4 for gameover(win)
int state = 0;//For Mode2/Mode3-Gamer1
int cur_width = 1200;//should be changed accordingly
int cur_height = 800;//should be changed accordingly
boolean gameStarted = false;
boolean gameOver = false;//mode 0-3 false, mode 4 true
boolean sound_exist = false; //For (from gamer0_0 to gamer0_1)
int move_state = 1;//1 for moving right, 2 for moving left in Intro
int death_count = 0;
//Variable initialization for the characters
int size = 40; // character size
int speed = 15; // character speed
int speed_fall = 18; // character fall speed
float x, y; // The character's x and y position
int lastStateChange;
int cur_i;
int new_cur;
//Map initialization with rectangle
int rec_x = 50; //The width of each rectangle on the x axis
int hole_size = 5;
int numRects = cur_width / rec_x; // Calculate the total number of rectangles
float[] rect_height = new float[numRects]; //The height for each rectangle
float[] rect_starting_x = new float[numRects];
float[] rect_height_tuto = new float[numRects];
float[] rect_starting_x_tuto = new float[numRects];
int j = 0;
int mini_height = 2;
//For each rectangle, to draw it:
//rect(rect_starting_x[j]-hole_size, height - rect_height[j],
//rec_x-(hole_size*2), rect_height[j]);
//Map initialization with images
boolean pg_drawn = false;
boolean pg2_drawn = false;
//Map Memory
int map_numbers = 6;
int map_idx = 0;
int check_idx = 100;//For checking the available index
boolean loading_map = false;
float[][] Memo_rect_height = new float[map_numbers][numRects];
float[][] Memo_rect_starting_x = new float[map_numbers][numRects];
PImage[][] Image_for_x = new PImage[map_numbers][numRects];
PImage[] Image_for_cur_map = new PImage[numRects];
boolean[][] Memo_coin = new boolean[map_numbers][numRects];
int[] Memo_current_coin = new int[map_numbers];
int current_coin = 0;
boolean[] coin_for_cur_map = new boolean[numRects];
boolean[] Map_saved = new boolean[map_numbers];
boolean map_loaded = false;
//Other Initialization
int NUM_OF_VALUES_FROM_ARDUINO = 6;
int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO];
boolean[] keys = new boolean[128];//For Key Press
int time = 1; //It is useless right now
float perc = 0.3; //Fault tolerance Percentage
int gamer = 0; //0 is constructor, 1 is navigator
int cons_tutor_state = 1;
boolean special_ability_used = false;
int force_exit= 0;
long force_exit_startTime = 0;
boolean music_play = true;
boolean[] press_button_exit = new boolean[3];
boolean exit_botton_pressed = false;
boolean exit_botton_previous =false;
//red:a[0] or keys[H]
void setup() {
String[] portNames = Serial.list();
for (String port : portNames) {
println(port);
}
serialPort = new Serial(this, "port_name", 9600);
size(1200, 800);
pg = createGraphics(1200, 800);
pg2 = createGraphics(20, 20);
cur_width = width;
cur_height = height;
numRects = cur_width / rec_x;
bgImage = loadImage("bg3_2.jpg");
cImage = loadImage("character.png");
//height: 0-2 transparentImage,3-40 rbImage, 41-100, 100-150, 151-400 tree
treeImage = loadImage("tree.png");
rbImage = loadImage("Roadblock.png");
transparentImage = loadImage("transparent.png");
vmImage = loadImage("VendingMachine.png");
bdImage = loadImage("bd.png");
warnImage = loadImage("warning.png");
coinImage = loadImage("Coin.png");
music = new SoundFile(this, "music.mp3");
music.play();
music.loop();
x = 0;
y = height - size;
//background(255);
//image(bgImage, 0, 0, width, height);
minim = new Minim(this);
in = minim.getLineIn(Minim.MONO);
//Tutorial Design For Navigator
for (int i = 0; i < numRects; i++) {
Memo_current_coin[0] = 12;
Memo_rect_height[0][i] = 0;
Memo_rect_starting_x[0][i] = rec_x * i;
if (i % 4 == 0) {
Memo_rect_height[0][i] = i*20;
Memo_rect_height[0][i] = constrain(Memo_rect_height[0][i], 10, height*0.5);
if (Memo_rect_height[0][i]<=mini_height) {
Image_for_cur_map[i] = transparentImage;
} else if (Memo_rect_height[0][i]<=30) {
Image_for_cur_map[i] = rbImage;
} else if (Memo_rect_height[0][i]<=100) {
Image_for_cur_map[i] = vmImage;
} else if (Memo_rect_height[0][i]<=200) {
Image_for_cur_map[i] = treeImage;
} else if (Memo_rect_height[0][i]<=300) {
Image_for_cur_map[i] = bdImage;
} else {
Image_for_cur_map[i] = treeImage;
}
Image_for_x[0][i] = Image_for_cur_map[i];
} else {
Memo_rect_height[0][i] = 0;
Image_for_cur_map[i] = transparentImage;
}
Image_for_x[0][i] = Image_for_cur_map[i];
}
Map_saved[0] = true;
}
void draw() {
frameRate(60);
stroke(0);
image(bgImage, 0, 0, width, height);
getSerialData();
//fill(255);
//textSize(16);
//text("Current frame rate: " + nf(frameRate, 0, 2), 10, 30);
if (force_exit == 1) {
mode = 0;
return;
}
//music.loop();
println(force_exit);
println(music_play);
if (music_play) {
if ((mode == 1 && gamer == 0) || (mode == 2 && gamer == 0))
{
music.stop();
music_play = false;
} else {
}
} else if (!music_play) {
if (!((mode == 1 && gamer == 0) || (mode == 2 && gamer == 0)))
{
music.play();
music.loop();
music_play = true;
} else {
}
}
//Intro
if (mode == 0) {
Intro();
return;
}
//Tutorial
else if (mode == 1) {
Tutorial();
} else if (mode == 2) {
if (gamer == 0) {
constructor();
} else if (gamer == 1) {
Map_generate();
Navigator();
}
} else if (mode == 3) {
if (!map_loaded) {
Map_load();
return;
}
if (!loading_map) {
frameRate(60);
Map_generate();
Navigator();
}
} else if (mode == 4 || gameOver) { //When the Navigator wins
win();
}
}
void Intro() {
//Initialization
pg = createGraphics(1200, 800);
pg2 = createGraphics(20, 20);
j = 0;
state = 0;
special_ability_used = false;
pg_drawn = false;
loading_map = false;
map_loaded = false;
pg_drawn = false;
gameStarted = false;
gameOver = false;
sound_exist = false;
pg2_drawn = false;
cons_tutor_state = 1;
current_coin = 0;
gamer = 0;
map_idx = 0;
death_count = 0;
//Move of the character
image(cImage, x, height-100, size, size);
if (move_state % 2 == 1) {
x = x + speed;
if (x + size >= width-1) {
move_state ++;
x = width - size;
}
} else {
x = x - speed;
if (x <= 0) {
x = 0;
move_state ++;
}
}
//Mode choosing
if ((keys['T']||keys['t']) ) {
mode = 1;
}
if ((keys['K']||keys['k']) ) {
gameStarted = true;
mode = 2;
}
if ((keys['L']||keys['l']) ) {
gameStarted = true;
mode = 3;
}
//Instruction Text
fill(255, 255, 255, 200);
stroke(255);
textSize(46);
textAlign(CENTER);
String text = "Press T to Begin Tutorial for Beginners!\nPress K to Directly Start a New Game!\nPress L to Load Maps!";
float textWidth = textWidth(text);
float textHeight = 46;
float textx = width / 2;
float texty = height / 2;
rect(textx - textWidth / 2-10, texty - textHeight / 2-20, textWidth+20, textHeight*3+25);
fill(255, 0, 0);
text(text, textx, texty);
return;
}
void Tutorial() {
if (gamer == 0) {
if (cons_tutor_state == 1) {
constructor_tuto1();
} else if (cons_tutor_state == 2) {
constructor_tuto2();
return;
}
} else if (gamer == 1) {
coin_for_cur_map[12] = true;
current_coin = 12;
map_idx = 0;
mode = 2;
if (!map_loaded) {
map_idx = 0;
for (int i = 0; i < numRects; i++) {
rect_height[i] = Memo_rect_height[map_idx][i]; //The height for each rectangle
rect_starting_x[i] = Memo_rect_starting_x[map_idx][i];
Image_for_cur_map[i] = Image_for_x[map_idx][i];
}
map_loaded = true;
}
Map_generate();
Navigator();
}
}
void constructor_tuto1() {
volume = in.left.level() * 100;
float each_rectHeight_tuto = 800 / 20;
float rect_height_tuto = volume * each_rectHeight_tuto;
fill(255);
rect_height_tuto = constrain(rect_height_tuto, 0, height*0.5);
rect(575, height-rect_height_tuto-50, 50, rect_height_tuto);
fill(255);
textSize(32);
text("This is the block generated by your current volume, continue by pressing RED BUTTON", 590, 790);
if (keys['M']|| keys['m']||arduino_values[0]==1) {
cons_tutor_state = 2;
}
}
void constructor_tuto2() {
//println(gamer);
if (keys['N']|| keys['n']||arduino_values[3]==1) {
gamer = 1;
return;
}
if (keys[ENTER]) {
cons_tutor_state = 1;
j = 0;
sound_exist = false;
current_coin = 0;
}
image(bgImage, 0, 0, width, height);
frameRate(4);
fill(255);
textSize(40);
textAlign(CENTER);
text("As a Constructor, try to make the map by changing your volume.\nYou can put a coin (for the navigator) by pressing RED BUTTON\n(Press BUTTON 4 if you are Navigator)", 590, 100);
//fill(255);
//textSize(32);
//text("As a Constructor, try to make the map by changing your volume", width/2, height/2);
volume = in.left.level() * 100;
if ((volume <= 0.5) && (!sound_exist)) {
return;
} else {
sound_exist = true;
if (j >= numRects) {
int j_ = numRects-1;
//gamer = 1;
for (int i = 0; i <= j_; i++) {
fill(255);
//Attention0
image(Image_for_cur_map[i], rect_starting_x_tuto[i]+hole_size, height - rect_height_tuto[i], rec_x-hole_size*2, rect_height_tuto[i]);
if (coin_for_cur_map[i]) {
image(coinImage, rect_starting_x_tuto[i]+hole_size+0.5*size, height - rect_height_tuto[i]-0.5*size, 0.5*size, 0.5*size );
}
//rect(rect_starting_x_tuto[i]+hole_size, height - rect_height_tuto[i], rec_x-hole_size*2, rect_height_tuto[i]);
}
} else {
float each_rectHeight = 800 / 20;
int m = j * rec_x;
rect_height_tuto[j] = volume * each_rectHeight;
rect_starting_x_tuto[j] = m;
fill(255);
rect_height_tuto[j] = constrain(rect_height_tuto[j], 0, height*0.5);
if ((!special_ability_used) && (keys['B']||keys['b']||arduino_values[0] == 1)&&rect_height_tuto[j] > mini_height) {
coin_for_cur_map[j] = true;
current_coin = j;
special_ability_used = true;
}
if (rect_height_tuto[j]<=mini_height) {
Image_for_cur_map[j] = transparentImage;
} else if (rect_height_tuto[j]<=30) {
Image_for_cur_map[j] = rbImage;
} else if (rect_height_tuto[j]<=100) {
Image_for_cur_map[j] = vmImage;
} else if (rect_height_tuto[j]<=200) {
Image_for_cur_map[j] = treeImage;
} else if (rect_height_tuto[j]<=300) {
Image_for_cur_map[j] = bdImage;
} else {
Image_for_cur_map[j] = treeImage;
}
println(j);
//Attention1
image(Image_for_cur_map[j], rect_starting_x_tuto[j]+hole_size, height - rect_height_tuto[j], rec_x-hole_size*2, rect_height_tuto[j]);
if (coin_for_cur_map[j]) {
image(coinImage, rect_starting_x_tuto[j]+hole_size+0.5*size, height - rect_height_tuto[j]-0.5*size, 0.5*size, 0.5*size );
}
//rect(rect_starting_x_tuto[j]+hole_size, height - rect_height_tuto[j], rec_x-hole_size*2, rect_height_tuto[j]);
//println(height -rect_height[j]);
for (int i = 0; i <= j; i++) {
println(j);
println(i);
fill(255);
//Attention2
image(Image_for_cur_map[i], rect_starting_x_tuto[i]+hole_size, height - rect_height_tuto[i], rec_x-hole_size*2, rect_height_tuto[i]);
//rect(rect_starting_x_tuto[i]+hole_size, height - rect_height_tuto[i], rec_x-hole_size*2, rect_height_tuto[i]);
if (coin_for_cur_map[i]) {
image(coinImage, rect_starting_x_tuto[i]+hole_size+0.5*size, height - rect_height_tuto[i]-0.5*size, 0.5*size, 0.5*size );
}
}
j ++;
}
}
}
void constructor() {
//j = 0;
frameRate(4);
fill(255);
textSize(32);
textAlign(CENTER);
text("Use Your Voice to Create Blocks", width / 2, height / 2);
textAlign(CENTER);
text("Press RED BUTTON to Generate A Coin", width/2, height/2+30);
fill(255, 0, 0, 100);
rect(0, 0, width, height*0.34);
image(warnImage, 0, 0, width, height*0.34);
volume = in.left.level() * 100;
if ((volume <= 0.5) && (!sound_exist)) {
return;
} else {
sound_exist = true;
if (j == numRects) {
gamer = 1;
return;
}
float each_rectHeight = 800 / 20;
int m = j * rec_x;
rect_height[j] = volume * each_rectHeight;
rect_starting_x[j] = m;
fill(255);
rect_height[j] = constrain(rect_height[j], 0, height*0.5);
if ((!special_ability_used) && (keys['B']||keys['b']||arduino_values[0] == 1)&&rect_height[j] > mini_height) {
coin_for_cur_map[j] = true;
current_coin = j;
special_ability_used = true;
}
if (rect_height[j]<=mini_height) {
Image_for_cur_map[j] = transparentImage;
} else if (rect_height[j]<=30) {
Image_for_cur_map[j] = rbImage;
} else if (rect_height[j]<=100) {
Image_for_cur_map[j] = vmImage;
} else if (rect_height[j]<=200) {
Image_for_cur_map[j] = treeImage;
} else if (rect_height[j]<=300) {
Image_for_cur_map[j] = bdImage;
} else {
Image_for_cur_map[j] = treeImage;
}
//Attention3
//rect(rect_starting_x[j]+hole_size, height - rect_height[j], rec_x-hole_size*2, rect_height[j]);
image(Image_for_cur_map[j], rect_starting_x[j]+hole_size, height - rect_height[j], rec_x-hole_size*2, rect_height[j]);
if (coin_for_cur_map[j]) {
image(coinImage, rect_starting_x[j]+hole_size+0.5*size, height - rect_height[j]-0.5*size, 0.5*size, 0.5*size );
}
//println(height -rect_height[j]);
for (int i = 0; i <= j; i++) {
fill(255);
//Attention4
image(Image_for_cur_map[i], rect_starting_x[i]+hole_size, height - rect_height[i], rec_x-hole_size*2, rect_height[i]);
if (coin_for_cur_map[i]) {
image(coinImage, rect_starting_x[i]+hole_size+0.5*size, height - rect_height[i]-0.5*size, 0.5*size, 0.5*size );
}
//rect(rect_starting_x[i]+hole_size, height - rect_height[i], rec_x-hole_size*2, rect_height[i]);
}
j ++;
}
}
void Map_load() {
image(bgImage, 0, 0, width, height);
String trueIndices = "Map Saved With Specific Index:";
String falseIndices = "Generate Random Map: 6";
for (int i = 1; i < map_numbers; i++) {
if (Map_saved[i]) {
trueIndices += (trueIndices.endsWith(":") ? "" : ", ") + i;
} else {
falseIndices += (falseIndices.endsWith(":") ? "" : ", ") + i;
}
}
fill(255, 255, 255, 200);
stroke(255);
textSize(46);
textAlign(CENTER);
String text = trueIndices + "\n" + falseIndices + "\nPlease Press The Corresponding Button To Load The Map!";
float textWidth = textWidth(text);
float textHeight = 46;
float textx = width / 2;
float texty = height / 2;
rect(textx - textWidth / 2-10, texty - textHeight / 2-20, textWidth+20, textHeight*3+25);
fill(255, 0, 0);
text(text, textx, texty);
loading_map = true;
for (int i = 0; i <= 5; i++) {
if (arduino_values[i] == 1) {
map_idx = i+1;
}
}
if (map_idx != 0 && loading_map) {
if (!(map_idx == 6)) {
if (Map_saved[map_idx]) {
for (int i = 0; i < numRects; i++) {
current_coin = Memo_current_coin[map_idx];
rect_height[i] = Memo_rect_height[map_idx][i]; //The height for each rectangle
rect_starting_x[i] = Memo_rect_starting_x[map_idx][i];
Image_for_cur_map[i] = Image_for_x[map_idx][i];
coin_for_cur_map[i] = Memo_coin[map_idx][i];
coin_for_cur_map[current_coin] = true;
}
} else if (!Map_saved[map_idx]) {
for (int i = 0; i < numRects; i++) {
float cur_height = (int) random(0, height*0.5);
if (cur_height<=mini_height) {
Image_for_cur_map[i] = transparentImage;
} else if (cur_height<=30) {
Image_for_cur_map[i] = rbImage;
} else if (cur_height<=100) {
Image_for_cur_map[i] = vmImage;
} else if (cur_height<=200) {
Image_for_cur_map[i] = treeImage;
} else if (cur_height<=300) {
Image_for_cur_map[i] = bdImage;
} else {
Image_for_cur_map[i] = treeImage;
}
current_coin = (int)random(0, numRects);
coin_for_cur_map[current_coin] = true;
Memo_rect_height[map_idx][i] = cur_height;
Memo_rect_starting_x[map_idx][i] = i * rec_x;
Image_for_x[map_idx][i] = Image_for_cur_map[i];
Memo_coin[map_idx][i] = coin_for_cur_map[i];
Map_saved[map_idx] = true;
rect_height[i] = cur_height; //The height for each rectangle
rect_starting_x[i] = i * rec_x;
}
check_idx = map_idx;
}
loading_map = false;
map_loaded = true;
gamer = 1;
} else {
for (int i = 0; i < numRects; i++) {
float cur_height = (int) random(0, height*0.5);
if (cur_height<=mini_height) {
Image_for_cur_map[i] = transparentImage;
} else if (cur_height<=30) {
Image_for_cur_map[i] = rbImage;
} else if (cur_height<=100) {
Image_for_cur_map[i] = vmImage;
} else if (cur_height<=200) {
Image_for_cur_map[i] = treeImage;
} else if (cur_height<=300) {
Image_for_cur_map[i] = bdImage;
} else {
Image_for_cur_map[i] = treeImage;
}
rect_height[i] = cur_height; //The height for each rectangle
rect_starting_x[i] = i * rec_x;
current_coin = (int)random(0, numRects);
coin_for_cur_map[current_coin] = true;
}
loading_map = false;
map_loaded = true;
}
}
}
void Map_generate() {
//println(coin_for_cur_map);
//Don't Jump into this area
if (!pg_drawn) {
pg.beginDraw();
pg.fill(255, 0, 0, 100);
pg.rect(0, 0, width, height*0.34);
pg.image(warnImage, 0, 0, width, height*0.34);
for (int i = 0; i < numRects; i++) {
fill(255);
//Attention5
pg.image(Image_for_cur_map[i], rect_starting_x[i]+hole_size, height - rect_height[i], rec_x-hole_size*2, rect_height[i]);
}
pg.endDraw();
pg_drawn = true;
}
if (!pg2_drawn) {
pg2.beginDraw();
for (int i = 0; i < numRects; i++) {
if (i == current_coin) {
pg2.image(coinImage, 0, 0, 0.5*size, 0.5*size);
}
}
pg2.endDraw();
pg2_drawn = true;
}
image(pg, 0, 0);
image(pg2, rect_starting_x[current_coin]+hole_size+0.5*size, height - rect_height[current_coin]-0.5*size, 0.5*size, 0.5*size);
}
void Navigator() {
//println(coin_for_cur_map[current_coin]);
if (coin_for_cur_map[current_coin]) {
image(pg2, rect_starting_x[current_coin]+hole_size+0.5*size, height - rect_height[current_coin]-0.5*size, 0.5*size, 0.5*size);
} else {
pg2 = createGraphics(20, 20);
}
frameRate(60);
//Intro
if (state == 0) {
state0();
}
//Wait
if (state == 1) {
state1();
}
//Jump
if (state == 2) {
state2();
}
//Falling Down
if (state == 3) {
state3();
}
if (state == 4) {
state4();
}
}
void state0() { //Intro for M2/3_G1
image(cImage, rect_starting_x[0], height- rect_height[0] - size, size, size);
x = rect_starting_x[0];
y = height- rect_height[0] - size;
fill(255);
textSize(24); // Set text size
textAlign(TOP); // Align text to center and bottom of the character image
fill(0, 0, 0, 90);
stroke(0, 0, 0, 90);
rect(5, 5, 450, 200);
fill(255);
text("Press YELLOW BUTTON To Start This Turn!", 10, 30); // Display the prompt above the character
text("Press LEFT BLUE BUTTON To Move Backward.", 10, 60);
text("Press RIGHT BLUE BUTTON to Jump Higher.", 10, 90);
text("Your Death Count is: "+ death_count, 10, 120);
text("Notice that you need to approach \nthe coin (if any) once to pass this map.", 10, 150);
if (keys['l']||keys['L']||(arduino_values[2]==1)) {
state = 1;
}
}
void state1() { //Game_Wait for M2/3_G1
int numRects = width / rec_x;
int cur_i = (int)(x / rec_x);
//float cur_i = x / rec_x;
if (cur_i < numRects-2) {
cur_i ++;
}
coin_for_cur_map[cur_i-1] = false;
image(cImage, x, y, size, size);
if (keys['W']||keys['w']||arduino_values[1] == 1) {
state = 2;
}
}
void state2() { //Game_Jump for M2/3_G1
//The charac1
image(cImage, x, y, size, size);
//Don't move forward
if (!(keys['A']||keys['a']|| arduino_values[5] == 1) ) {
x += 5;
} else {
x -= 1;
}
//Jump, or you will fall
if (keys['W']||keys['w']||(arduino_values[1] == 1)) {
y -= speed;
} else {
y += speed_fall;
}
cur_i = (int)(x / rec_x);
//Achieving the end
if (x+size >= width ) {
state = 0;
if (coin_for_cur_map[current_coin]) {
death_count ++;
} else if (!coin_for_cur_map[current_coin]) {
mode = 4;
gameOver = true;
//Write ("You Win") or something else with LED
return;
}
}
//This is Lose
if (y >= height || y <= height*0.3) {
death_count ++;
state = 0;
//Write ("You Lose") or something else with LED
return;
}
new_cur = cur_i;
if (cur_i < (numRects-1)) {
new_cur = cur_i + 1;
}
if (
(x + size >= rect_starting_x[new_cur] + hole_size && height-rect_height[new_cur] <= y)
||
(x <= rect_starting_x[cur_i] - hole_size&& height-rect_height[cur_i] <= y)
||
(rect_height[new_cur] <= mini_height && y + size >= height-mini_height)
||
(rect_height[cur_i] <= mini_height && y + size >= height-mini_height)
)
{
state = 4;
lastStateChange = millis();
return;
}
//println(cur_i);
//cur_i = max(cur_i,23);
boolean Condi_1 = x + perc * size >= rect_starting_x[new_cur]
&& x + size <= rect_starting_x[new_cur] + rec_x
;
boolean Condi_1_1 =
//y + size <= height-rect_height[cur_i]
//&&
y + size >= height-rect_height[new_cur] - 10
;
boolean Condi_2 = x + size - perc * size <= rect_starting_x[cur_i] + rec_x
&&
x >= rect_starting_x[cur_i]
;
boolean Condi_2_1 =
//y + size <= height-rect_height[cur_i]
//&&
//y + >= height-rect_height[max(0,cur_i-1)] - 10
y + size >= height-rect_height[cur_i] - 10
;
if ((Condi_1 && Condi_1_1)) {
state = 1;
y = height-rect_height[new_cur]-size;
;
}
if (Condi_2 && Condi_2_1) {
state = 1;
y = height-rect_height[cur_i]-size;
}
}
//Game_Fall
void state4() {
image(cImage, x+5, y+2, size, size);
image(Image_for_cur_map[cur_i], rect_starting_x[cur_i]+hole_size, height - rect_height[cur_i], rec_x-hole_size*2, rect_height[cur_i]);
image(Image_for_cur_map[new_cur], rect_starting_x[new_cur]+hole_size, height - rect_height[new_cur], rec_x-hole_size*2, rect_height[new_cur]);
if (millis()-lastStateChange > 500)
{
state = 3;
death_count++;
}
}
void state3() { //Game Fall for M2/3_G1
image(cImage, x+5, y+2, size, size);
image(Image_for_cur_map[cur_i], rect_starting_x[cur_i]+hole_size, height - rect_height[cur_i], rec_x-hole_size*2, rect_height[cur_i]);
image(Image_for_cur_map[new_cur], rect_starting_x[new_cur]+hole_size, height - rect_height[new_cur], rec_x-hole_size*2, rect_height[new_cur]);
y += 10;
if (y-40 >= height) {
state = 0;
return;
}
}
void win() {
//println(Map_saved);
String text;
float textHeight = 46;
float textx = width / 2;
float texty = height / 2;
fill(255, 255, 255, 200);
stroke(255);
textSize(46);
textAlign(CENTER);
if (check_idx >= 6) {
check_idx = (int)random(1, 6);
for (int i = 1; i < Map_saved.length; i++) {
if (!Map_saved[i]) {
check_idx = i;
break;
}
}
}
if (!(keys['S'] || keys['s'])) {
String dc = "You Win with a death count of: " + death_count;
text =dc + "!\nPress RED BUTTON to Play This Map Again!\nPress BUTTON 5 to Go Back to the Home Page\nPress S to Save this Map to Index " + check_idx;
float textWidth = textWidth(text);
rect(textx - textWidth / 2-10, texty - textHeight / 2-20, textWidth+20, textHeight*3+80);
fill(255, 0, 0);
text(text, textx, texty);
if (keys[ENTER]||arduino_values[0]==1) {
coin_for_cur_map[current_coin] = true;
pg2_drawn = false;
mode = 2;
gamer = 1;
state = 0;
gameOver = false;
death_count = 0;
}
if (keys['H'] || keys['h']||arduino_values[4]==1) {
mode = 0;
gamer = 0;
state = 0;
gameOver = false;
}
} else {
for (int i = 0; i < numRects; i++) {
Memo_rect_height[check_idx][i] = rect_height[i];
Memo_rect_starting_x[check_idx][i] = rect_starting_x[i];
Image_for_x[check_idx][i] = Image_for_cur_map[i];
Memo_current_coin[check_idx] = current_coin;
}
Map_saved[check_idx] = true;
check_idx ++;
//fill(255, 255, 255, 200);
//stroke(255);
//textSize(46);
//textAlign(CENTER);
//text ="You Win!\nPress Enter to Play This Map Again!\nPress H to Go Back to the Home Page\nSuceessfully Saved at Index" + check_idx;
//float textWidth = textWidth(text);
//rect(textx - textWidth / 2-10, texty - textHeight / 2-20, textWidth+20, textHeight*3+80);
//fill(255, 0, 0);
//text(text, textx, texty);
mode = 0;
gamer = 0;
state = 0;
gameOver = false;
}
return;
}
//Additional
void keyPressed() {
if (key < 128) keys[key] = true; // When the key is pressed, update
if (key >= '1' && key <= '6') {
int index = key;
if (index >= 49) {
index = index - 48;
}
if (mode == 3 && loading_map) {
map_idx = index;
}
}
}
void keyReleased() {
if (key < 128) keys[key] = false; // When the key is released, update
}
void getSerialData() {
while (serialPort.available() > 0) {
String in = serialPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
if (in != null) {
//print("From Arduino: " + in);
String[] serialInArray = split(trim(in), ",");
if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
for (int i=0; i<serialInArray.length; i++) {
arduino_values[i] = int(serialInArray[i]);
}
}
}
}
}
Code- Arduino
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
pinMode(2,INPUT);
pinMode(3,INPUT);
pinMode(4,INPUT);
pinMode(5,INPUT);
pinMode(6,INPUT);
pinMode(7,INPUT);
}
void loop() {
// put your main code here, to run repeatedly:
int button1State = digitalRead(2);
int button2State = digitalRead(3);
int button3State = digitalRead(4);
int button4State = digitalRead(5);
int button5State = digitalRead(6);
int button6State = digitalRead(7);
Serial.print(button1State);
Serial.print(","); // put comma between sensor values
Serial.print(button2State);
Serial.print(",");
Serial.print(button3State);
Serial.print(",");
Serial.print(button4State);
Serial.print(",");
Serial.print(button5State);
Serial.print(",");
Serial.print(button6State);
Serial.println(); // add linefeed after sending the last sensor value
// too fast communication might cause some latency in Processing
// this delay resolves the issue
delay(20);
}
Art Galary
The whole project is inspired by Cyberpunk 2077 and Cyberpunk Edgerunners. Some items used for constructing the city is based on some 3-D modeled items in Cyberpunk 2077, and the character that the navigator can control is Rebecca in Cyberpunk Edgerunners.
Character & Items [drawn by I (Jiaxi Zhang)]
Background
Works Cited
Background Image, ChatGpt, version 4, OpenAI, 8 Mar. 2023, https://chatgpt.com/.
Cyberpunk 2077. CD Projekt, 2020. Steam.
Cyberpunk: Edgerunners. Directed by Hiroyuki Imaishi, Trigger, 2022. Netflix, https://www.netflix.com/hk/title/81054853.
Norman, Donald A. The Design of Everyday Things. 1st Basic paperback, Basic Books, 2002.