A. Submarine – Peirong Li – Professor Gottfried Haider
(Image: game logo)
B. CONCEPTION AND DESIGN
This project is an arcade-like game in which an idiosyncratic control panel (a wheel) is used to steer the submarine to descend in the ocean. My definition of interaction is “an intuitive set of actions that occurs between systems (non-human) and participants (human), in which both the system and the participants are equally involved, through facilitated or ambient physical elements that invite responses in a continuous loop.” My project aims to align with this definition by creating a system that reacts but responds directly to audience action.
At the beginning of this project, a key design decision was to use the rotary encoder as the main sensor and input instrument. This decision is the foundation of making a smooth experience for the game. On the graphic side of things, I used Processing to create a virtual world. The main mechanism is using a long image scrolling up to give the user a feeling that the game is progressing downwards. Techniques such as pixel color detection were used to detect collisions and also detect the “bufferzone”, which sends a vibration signal whenever the user gets too close to the border. During the earlier phase of the project, I used a simple, small wheel for control. User testing feedback gave me the idea to have a much bigger wheel to allow the user more movement and make the game experience more physical and amplified. Importantly, I also came up with the idea of a “secondary control”, in which the rest of the screen is black, and a second wheel controls the “lighting”, to make the game harder and more interesting.
C. FABRICATION AND PRODUCTION
Before User Testing:
1. Fabrication & Arduino
The very first thing I did was try out the rotary encoder and write the code to make sure that this main sensor works as expected. After a few unsuccessful tries, Professor Haider provided some very helpful guidance on using the “interrupt” method in Arduino to count how fast the wheel is turning, and which direction it is turning. Next, I fitted the rotary encoder to a laser-cut box and installed a small wheel that I purchased online and modified.
(image: rotary encoder with the small wheel)
(image: pre-user testing package fitted iside a wood box)
(video: testing with a circle )
2. Processing
After the physical mechanism was working, I turned to Processing.
(image: the submarine logo used in the game)
At first, I used Adobe Firefly, a text-to-image generative platform, to generate some concept art. I started out from the overall code structure that has an initial screen, a game screen, and a game over screen. The main game asset is a long png image that includes rocks on the side, a bright yellow line that the user needs to avoid, and deep blue background. I also used Adobe Firefly to generate “rocks” and incorporated them into game assets. Collision detection is based on pixel detection. Whenever the pixel at the edge of the submarine is yellow, which is also the color of the borderline, the game ends. At this stage of development, the game was still very short and not integrated.
The game asset were drew using Procreate and Adobe Photoshop.
(image: the main game asset)
(video: user testing session)
After User Testing:
1. Fabrication & Arduino
The most important feedback from User Testing was to make the experience more physical and increase the size of the installation. I purchased a wooden wheel, some bearings (ended up not using these), and some lightstrips for decoration from online platforms. I picked some leftover wooden panel and sticks, and with help from Professor Garcia, I fitted the rotary encoder to a standing wooden stick and installed the wheel on the encoder. Then, I decided to utilize the previous small box/wheel to set up a secondary control to manipulate the “lighting” function. Arduino code was altered accordingly to accommodate two rotary encoders, which was the maximum number of encoders that could be connected to the Arduino Uno board. Moreover, for decorative purposes, I did three things to improve the overall design: 1) a big laser-cut wood box behind the wheel to house all the cables and Arduino board, and also acts as a computer stand; 2) two buttons for starting the game and returning to menu; 3) light strips installed on the side of the base platform to improve aesthetics; 4) a vibration motor that reacts when the user is too close to the edge.
(image: the big wheel)
(image: final look)
(image: final look from another angle)
2. Processing
There were major extensions on the Processing side as well. First of all, the main game asset was extended 5 times longer, and a second level was added, making the game experience around 3 minutes long. Two screens were added, a “ScreenB” for the second level, and an “end screen” for users that completed the game. I used Framerate() to control the speed of the game, and found that 120 frames was the best setting for users to play at. An extremely important modification was to call the resize function during setup() instead of draw(), and this drastically improved efficiency and saved the game from lagging severely. Button presses and vibration signals through serial communication were also added. For the second level, I added a method called “masking”, in which the loadPixels() function was used to scan the screen pixel by pixel and black out the entire screen except a circle controlled by the user using the same mechanism as the wheel. Finally, a reset() function was added to re-initialize all the global variables to their original values when the blue (return to main menu) button was pressed.
(image: the final main asset)
D. CONCLUSIONS
The goal of this project is to create an interactive experience that involves the installation and the user equally, in the form of a game that utilizes actuators and sensors. The audience interacted with the project just as expected, since the control mechanism was very self-explanatory. I believe that the project aligns with my definition of interaction. There are many considerable improvements can be made if there are more time, for example: adding sounds and music, with beeps going off when the user is close to the edge, complementing the vibration motor; fitting the vibration motor to the wheel; fabricating a single button mechanism that is more intuitive for navigating the menus; fabricating a wood panel that resembles a “submarine window” and fit the screen behind it; add floating things to make the game environment more vibrant; using alpha values for collision detection instead of color; clearer instructions, etc. Overall, I believe this project is a success. There were many setbacks and difficulties that I encountered during the process, but each time when I resolved it, I gained from it and my problem-solving skills improved. Just as Professor Haider said, “programming (and electronics) is to a large degree a matter of practice,” and I learned not just fabrication or coding skills, but the ability to engage in the creative process, conceive ideas, and apply skills to make these ideas a reality.
(image: IMA show)
E. APPENDIX
Credit: Professor Gottfried Haider (Arduino Code), Professor Andy Garcia (Fabrication), Adobe Firefly (Assets)
(Video: Full Demo Video at the IMA show)
FULL CODE
Arduino
#define PIN_A 2
#define PIN_B 4
#define PIN_C 3
#define PIN_D 5
#define NUM_OF_VALUES_FROM_PROCESSING 1
/* This array stores values from Processing */
int processing_values[NUM_OF_VALUES_FROM_PROCESSING];
int delta_A = 0;
int delta_B = 0;
void setup() {
Serial.begin(115200);
pinMode(8, OUTPUT);
pinMode(PIN_A, INPUT_PULLUP);
pinMode(PIN_B, INPUT_PULLUP);
pinMode(PIN_C, INPUT_PULLUP);
pinMode(PIN_D, INPUT_PULLUP);
pinMode(11,INPUT);
pinMode(12,INPUT);
attachInterrupt(digitalPinToInterrupt(PIN_A), shaft_moved_1, FALLING);
attachInterrupt(digitalPinToInterrupt(PIN_C), shaft_moved_2, FALLING);
}
void loop() {
getSerialData();
// add your code here using incoming data in the values array
// and print values to send to Processing
int buttonState1 = digitalRead(12);
int buttonState2 = digitalRead(11);
// example of using received values and turning on an LED
if (processing_values[0] == 1) {
digitalWrite(8, HIGH);
} else {
digitalWrite(8, LOW);
}
int old_A = delta_A;
delta_A = 0;
int old_B = delta_B;
delta_B = 0;
Serial.print(10000 + old_A);
Serial.print(",");
Serial.print(10000 + old_B);
Serial.print(",");
Serial.print(buttonState1);
Serial.print(",");
Serial.println(buttonState2);
delay(100);
}
void shaft_moved_1() {
int pinb_value = digitalRead(PIN_B);
if (pinb_value == HIGH) {
delta_A = delta_A - 1;
} else {
delta_A = delta_A + 1;
}
}
void shaft_moved_2() {
int pind_value = digitalRead(PIN_D);
if (pind_value == HIGH) {
delta_B = delta_B - 1;
} else {
delta_B = delta_B + 1;
}
}
/* Receive serial data from Processing */
/* You won't need to change this code */
void getSerialData() {
static int tempValue = 0;
static int valueIndex = 0;
while (Serial.available()) {
char c = Serial.read();
// switch - case checks the value of the variable in the switch function
// in this case, the char c, then runs one of the cases that fit the value of the variable
// for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase
switch (c) {
// if the char c from Processing is a number between 0 and 9
case '0' ... '9':
// save the value of char c to tempValue
// but simultaneously rearrange the existing values saved in tempValue
// for the digits received through char c to remain coherent
// if this does not make sense and would like to know more, send an email to me!
tempValue = tempValue * 10 + c - '0';
break;
// if the char c from Processing is a comma
// indicating that the following values of char c is for the next element in the values array
case ',':
processing_values[valueIndex] = tempValue;
// reset tempValue value
tempValue = 0;
// increment valuesIndex by 1
valueIndex++;
break;
// if the char c from Processing is character 'n'
// which signals that it is the end of data
case '\n':
// save the tempValue
// this will b the last element in the values array
processing_values[valueIndex] = tempValue;
// reset tempValue and valueIndex values
// to clear out the values array for the next round of readings from Processing
tempValue = 0;
valueIndex = 0;
break;
}
}
}
Processing
/********* VARIABLES *********/
// 0: Initial Screen
// 1: Game Screen
// 2: Game-over Screen
// 3: Game Screen B
// 4: End Screen
import processing.serial.*;
Serial serialPort;
PImage[] images = new PImage[10];
int NUM_OF_VALUES_FROM_ARDUINO = 4;
/* This array stores values from Arduino */
int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO];
int NUM_OF_VALUES_FROM_PROCESSING = 1; /* CHANGE THIS ACCORDING TO YOUR PROJECT */
/* This array stores values you might want to send to Arduino */
int processing_values[] = new int[NUM_OF_VALUES_FROM_PROCESSING];
boolean buttonPressedRed = false;
boolean buttonPressedBlue = false;
float speedX;
float x = 250;
float accelX;
float speedX_mask;
float x_mask = 250;
float accelX_mask;
float maskSize = 250;
float title_y = -150; //title init pos
float bg_y = 0;
float portal_y = 4075+170+1909+800;
int fade_y = 4380+170+1909+800;
float bg2_y = 0;
float opacity_4 = 0;
float op1 = 0;
float boundaryLeft; // left boundary
float boundaryRight; // right boundary
color left_color, right_color;
float leftR, leftG, leftB;
float rightR, rightG, rightB;
int gameScreen = 0;
/********* SETUP BLOCK *********/
void setup() {
//fullScreen();
size(600, 900);
frameRate(120);
noSmooth();
boundaryLeft = 0; // set the left boundary
boundaryRight = width - 100; // set the right boundary
printArray(Serial.list());
serialPort = new Serial(this, "/dev/cu.usbmodem14401", 115200);
// Load all the images into an array
for (int i = 0; i < images.length; i++) {
images[i] = loadImage(i + ".png");
}
//pre-resizing to improve performance
images[0].resize(600, 900);
images[1].resize(550, 0);
images[2].resize(600, 7600);
images[3].resize(100, 100);
images[4].resize(400, 100);
images[5].resize(600, 825);
//images[6].resize(600, 5691);
images[7].resize(600, 5691);
}
/********* DRAW BLOCK *********/
void draw() {
if (arduino_values[2] == 1) {
buttonPressedRed = true;
} else {
buttonPressedRed = false;
}
if (arduino_values[3] == 1) {
buttonPressedBlue = true;
} else {
buttonPressedBlue = false;
}
// Display the contents of the current screen
getSerialData();
if (gameScreen == 0) {
initScreen();
} else if (gameScreen == 1) {
gameScreen();
} else if (gameScreen == 2) {
gameOverScreen();
} else if (gameScreen == 3) {
gameScreenB();
} else if (gameScreen == 4) {
endScreen();
}
// send the values to Arduino
sendSerialData();
}
/********* SCREEN CONTENTS *********/
void initScreen() {
// codes of initial screen
background(255);
tint(255, 255);
image(images[0], 0, 0);
tint(255, op1);
image(images[1], 25, 200);
op1 += 2;
if (buttonPressedRed == true) {
if (gameScreen==0) {
reset();
startGame();
}
}
}
void gameScreen() {
if (bg_y < -4430.0 - 1909.0 - 800.0) {
gameScreen = 3;
}
// code of game screen
image(images[2], 0, bg_y);
bg_y -= 1;
//submarine init
noStroke();
fill(243, 247, 12);
image(images[3], x, 150);
//x speed processing
x = x + speedX;
speedX = speedX + accelX;
speedX = speedX * 0.9;
accelX = map(abs(arduino_values[0]), 11000, 9000, -1, 1);
//boundry setting
if (x <= boundaryLeft || x >= boundaryRight) {
speedX = -speedX;
// adjust the position of the circle so that it stays within the boundary
if (x < boundaryLeft) {
x = boundaryLeft;
} else if (x > boundaryRight) {
x = boundaryRight;
}
}
//collision detection, r = 50
left_color = get(round(x)+5, 215);
//circle(round(x),215,10);
right_color = get(round(x)+90, 215);
//circle(round(x)+80,215,10);
//yellow border detection
leftR = red(left_color);
leftG = green(left_color);
leftB = blue(left_color);
rightR = red(right_color);
rightG = green(right_color);
rightB = blue(right_color);
//println(leftR,leftG,leftB," ",rightR,rightG,rightB);
if ((leftR > 140 && leftR < 192) && (leftG > 130 && leftG < 140) && (leftB > 48 && leftB < 60)) {
gameScreen = 2;
} else if ((rightR > 140 && rightR < 200) && (rightG > 128 && rightG < 142) && (rightB > 45 && rightB < 62)) {
gameScreen = 2;
} else if ((leftR > 7 && leftR < 40) && (leftG > 46 && leftG < 58) && (leftB > 80 && leftB < 94)) {
processing_values[0] = 1;
} else if ((rightR > 7 && rightR < 40) && (rightG > 46 && rightG < 58) && (rightB > 80 && rightB < 94)) {
processing_values[0] = 1;
} else {
processing_values[0] = 0;
}
println(bg_y);
//-4075
image(images[9], 0, portal_y);
portal_y -= 1;
image(images[8], 0, fade_y);
fade_y -= 1;
}
void gameScreenB() {
if (bg2_y < -5540) {
gameScreen = 4;
}
background(0);
// Draw the image with an alpha value of 100
image(images[7], 0, bg2_y);
bg2_y -= 1;
//submarine init
noStroke();
fill(243, 247, 12);
image(images[3], x, 150);
//x_mask speed processing
x_mask = x_mask + speedX_mask;
speedX_mask = speedX_mask + accelX_mask;
speedX_mask = speedX_mask * 0.95;
accelX_mask = map(abs(arduino_values[1]), 11000, 9000, -1, 1);
// Masking
loadPixels();
for (int x1 = 0; x1 < width; x1++) {
for (int y = 0; y < height; y++) {
float d = dist(round(x_mask), 215, x1, y);
if (d > maskSize/2) {
int index = x1 + y * width;
pixels[index] = color(0);
}
}
}
updatePixels();
//x speed processing
x = x + speedX;
speedX = speedX + accelX;
speedX = speedX * 0.95;
accelX = map(abs(arduino_values[0]), 11000, 9000, -1, 1);
//boundry setting
if (x <= boundaryLeft || x >= boundaryRight) {
speedX = -speedX;
// adjust the position of the circle so that it stays within the boundary
if (x < boundaryLeft) {
x = boundaryLeft;
} else if (x > boundaryRight) {
x = boundaryRight;
}
}
//collision detection, r = 50
left_color = get(round(x)+5, 215);
right_color = get(round(x)+90, 215);
//yellow border detection
leftR = red(left_color);
leftG = green(left_color);
leftB = blue(left_color);
rightR = red(right_color);
rightG = green(right_color);
rightB = blue(right_color);
if ((leftR > 140 && leftR < 192) && (leftG > 130 && leftG < 140) && (leftB > 48 && leftB < 60)) {
gameScreen = 2;
} else if ((rightR > 140 && rightR < 200) && (rightG > 128 && rightG < 142) && (rightB > 45 && rightB < 62)) {
gameScreen = 2;
} else if ((leftR > 7 && leftR < 40) && (leftG > 46 && leftG < 58) && (leftB > 80 && leftB < 94)) {
processing_values[0] = 1;
} else if ((rightR > 7 && rightR < 40) && (rightG > 46 && rightG < 58) && (rightB > 80 && rightB < 94)) {
processing_values[0] = 1;
} else {
processing_values[0] = 0;
}
println(bg2_y);
}
void gameOverScreen() {
// codes for game over screen
background(255);
image(images[4], 100, 420);
tint(255, opacity_4);
if (opacity_4 <= 255) {
opacity_4 += 1;
}
if (buttonPressedBlue == true && gameScreen == 2) {
gameScreen = 0;
}
}
void endScreen() {
image(images[5], 0, 0);
if (buttonPressedBlue == true && gameScreen == 4) {
gameScreen = 0;
}
}
/********* INPUTS *********/
public void keyPressed() {
// if we are on the initial screen when clicked, start the game
/*
if (gameScreen==0) {
startGame();
}
if (key == 'b' || key == 'B') {
gameScreen = 3;
}
*/
}
/********* OTHER FUNCTIONS *********/
void startGame() {
gameScreen=1;
}
void getSerialData() {
while (serialPort.available() > 0) {
String in = serialPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII
if (in != null) {
//print("From Arduino: " + in);
String[] serialInArray = split(trim(in), ",");
if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
for (int i=0; i<serialInArray.length; i++) {
arduino_values[i] = int(serialInArray[i]);
}
}
}
}
}
void sendSerialData() {
String data = "";
for (int i=0; i<processing_values.length; i++) {
data += processing_values[i];
// if i is less than the index number of the last element in the values array
if (i < processing_values.length-1) {
data += ","; // add splitter character "," between each values element
}
// if it is the last element in the values array
else {
data += "\n"; // add the end of data character "n"
}
}
// write to Arduino
serialPort.write(data);
//print("To Arduino: " + data); // this prints to the console the values going to Arduino
}
void reset() {
speedX = 0;
x = 250;
accelX = 0;
speedX_mask = 0;
x_mask = 250;
accelX_mask = 0;
maskSize = 250;
title_y = -150; //title init pos
bg_y = 0;
portal_y = 4075+170+1909+800;
fade_y = 4380+170+1909+800;
bg2_y = 0;
opacity_4 = 0;
op1 = 0;
left_color = color(0, 0, 0);
right_color = color(0, 0, 0);
leftR = 0;
leftG = 0;
leftB = 0;
rightR = 0;
rightG = 0;
rightB = 0;
}