Project Title: FinFriend: A Virtual Aquarium Adventure
Name: Evelyn Yao
Instructor: Andy Garcia
Concept & Design:
We keep our concept all the way. “FinFriend” is an immersive virtual experience that explores the therapeutic aspects of caring for virtual fish. Informed by research on the positive effects of interaction with aquatic environments, this project aims to create a captivating game where users engage in activities such as feeding, accompanying, and cleaning their virtual fish. The design choices are influenced by the idea that nurturing a digital aquatic companion can provide a sense of calm and joy. The intended impact is to offer users a delightful and relaxing escape, promoting well-being through the simulated care of virtual aquatic life.
Fabrication & Production:
Our project are consist of two main parts, the coding aspect and the visual design. I took charge of developing the entire codebase while also contributing to the visual design.
The visual design concludes the game page design and external decorations. In terms of the game page design, I made all the animations such as food pouring and sweeping are made by PR, using my knowledge from the communication lab. Beatrice and I also found online sources for fish images and sound effects. For the external decorations, through PS and Illustrator, I also got the path of the decorative design and Beatrice used laser cutting to cut it out. She also using 3D printing to print the decoration of shells, starfish and fish food jar. I cut a small box and a broom for the ultimate decoration.
For the coding part, we established the electronic circuits as the project’s foundation, accompanied by basic code to ensure the functionality of each sensor. Initially, we only used buttons to trigger “playing”, “feeding” and used pressure sensor to trigger “cleaning”. However, to enhance the project’s interactivity, we sought to mimic real-world movements. We used distance sensor to replace the button for “feeding”, accompanied by a 3D-printed fish food model. When participants pour the fish food, the sensor is triggered, adding a realistic touch to the game. Similarly, the pressure sensor was swapped with a tilt switch sensor, activating when users perform a sweeping action.
In user-testing section, we added a button for restart and add more visual elements for it, use to the difficulty of the game. Before the final presentation, we elevated the ambiance by incorporating an LED strip. However, the Arduino got confused and was unable to deal with the new LED data it received from processing. Following discussions with instructors, a pragmatic solution emerged: integrating a new Arduino into the project to effectively handle the additional LED data.
In the process of coding, I met many problems, such as over-triggering, repeated-counting and game stuck. Thanks to guidance from instructors, I successfully resolved each issue, ensuring the seamless operation of the entire system.
Conclusions:
In conclusion, our project met our initial expectations for interactivity. By engaging multiple senses, including sight, sound, and touch, we successfully immersed the audience in a dynamic and participatory experience. The diverse reactions from the audience during the IMA show further emphasized the project’s success.
At the project’s inception, our aim was to evoke a sense of calmness and companionship. However, the outcome transformed into a recreational and challenging endeavor. The IMA show revealed a spectrum of emotions among the audience—some expressed sadness over the simulated death of their virtual fish, while others found excitement in experimenting with various feeding methods.
If we have more time, I think there are two distinct possible paths for further improvements. The first path involves a comprehensive transformation, steering the project toward a challenge-oriented game with multiple stages. On the other hand, an alternative approach could involve scaling back the intensity of the project’s goals, shifting its focus towards a more soothing and companionable experience.
Appendix:
Here is the code for processing:
import processing.serial.*;
import processing.sound.*;
SoundFile eating;
SoundFile playing;
SoundFile clean;
SoundFile music;
SoundFile music2;
SoundFile music3;
Serial serialPort;
int NUM_OF_VALUES_FROM_ARDUINO = 5;
int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO];
Serial serialPort2;
import processing.video.*;
Movie moviefood;
Movie moviecleaning;
Movie movieintroduction;
boolean introductionPlaying = true;
boolean foodPlaying = false;
boolean foodFalling = false;
boolean cleaningPlaying = false;
boolean game = true;
boolean celebration = false;
PImage ghost;
PImage fish;
PImage display;
PImage background;
float scaleFactor = 1.0;//for fish
boolean[] mirror;
int q=1;
float[] x = new float [q];
float[] y = new float [q];
float[] xspeed = new float [q];
float[] yspeed = new float [q];
int qf=30;
float[] xf = new float [qf];
float[] yf = new float [qf];
float[] xspeedf = new float [qf];
float[] yspeedf = new float [qf];
float satiety =5;
float cleanness=5;
float happiness =0;
int gameOverReason = 0;
int startTime;
float angle= 0;
boolean animate = false;
int startTimeText;
boolean showText= false;
int displayTextTime =1000;
String satietyText ="satiety+1";
boolean satietytrigger=false;
boolean cleaningtrigger=false;
boolean playingtrigger=false;
PImage happy;
PImage sad;
PImage[] images = new PImage[5];
int randomIndex;
PFont font;
int NUM_LEDS = 60; // How many LEDs in your strip?
color[] leds = new color[NUM_LEDS]; // array of one color for each pixel
void setup() {
font = createFont("font.TTF", 30);
textFont(font);
size(1280, 720, P2D);
background(255);
printArray(Serial.list());
serialPort = new Serial(this, "/dev/cu.usbmodem1401", 115200);
serialPort2 = new Serial(this, "/dev/cu.usbmodem1101", 115200);
moviefood = new Movie(this, "food.mov");
moviecleaning = new Movie(this, "cleaning.mov");
movieintroduction = new Movie(this, "introduction.mp4");
movieintroduction.play();
ghost = loadImage("ghost.png");
//fish = loadImage("fish.png");
background = loadImage("1401700540245_.pic.jpg");
//display = fish;
for (int i = 0; i < images.length; i++) {
images[i] = loadImage("fish" + i + ".png");
}
randomIndex = int(random(images.length));
display = images[randomIndex] ;
sad = loadImage("sad.png");
happy = loadImage("happy.png");
eating = new SoundFile(this, "eating.mp3");
playing = new SoundFile(this, "playing.mp3");
clean = new SoundFile(this, "clean.mp3");
music = new SoundFile(this, "music.mp3");
music2 = new SoundFile(this, "music2.mp3");
music3 = new SoundFile(this, "music3.mp3");
music.loop();
music2.loop();
music3.loop();
music.amp(0);
music2.amp(0);
music3.amp(0);
mirror = new boolean[q];
for (int i=0; i<q; i++) {
x[i] = random(0, width);
y[i] = random (0, height);
xspeed[i] = random(-3, 3);
yspeed[i] = random(-1, 1);
mirror[i] = false;
}
for (int i=0; i<qf; i++) {
xf[i] = random(0, width);
yf[i] = -10;
xf[i] = xf[i] + xspeedf[i];
yf[i] = yf[i] + yspeedf[i];
fill(#F0C879);
strokeWeight(1.3);
circle (xf[i], yf[i], 15);
}
}
void draw() {
background(255);
//
getSerialData();
if (movieintroduction.available()) {
movieintroduction.read();
}
image(movieintroduction, 0, 0, width, height);
print("movie play");
if (movieintroduction.time() >= movieintroduction.duration()*0.99) {
introductionPlaying = false;
movieintroduction.stop();
startTime = millis();
}
if ( introductionPlaying == false) {
if (arduino_values[2] == 1) {
animate = true;
}
if (animate) {
angle += 0.9;
}
if (angle >= TWO_PI) {
animate = false;
angle = 0;
}
int elapsedTime = millis() - startTime;
int Time = 45000-elapsedTime;
imageMode(CORNER);
image(background, 0, 0, width, height);
float cleannessVal = map(cleanness, 0, 15, 0, 200);
fill(255);
rect(150, 20, 200, 40, 50);
if (cleanness<6) {
fill(255, 0, 0);
} else {
fill(0, 255, 0);
}
rect(150, 20, cleannessVal, 40, 50);
textSize(25);
fill(0);
text("cleanness:"+ cleanness, 155, 50);
float satietyVal = map(satiety, 0, 15, 0, 200);
fill(255);
rect(550, 20, 200, 40, 50);
if (satiety<7) {
fill(255, 0, 0);
} else {
fill(0, 255, 0);
}
rect(550, 20, satietyVal, 40, 50);
textSize(25);
fill(0);
text("satiety:"+ satiety, 570, 50);
//text("happiness:"+ happiness, 950, 20);
float timeVal = map(Time, 45000, 0, 200, 0);
fill(255);
rect(950, 20, 200, 40, 50);
if (Time<10000 && Time>0) {
fill(255, 0, 0);
rect(950, 20, timeVal, 40, 50);
textSize(25);
fill(0);
text("time:"+Time / 1000, 990, 50);
} else if (Time>=10000) {
fill(0, 255, 0);
rect(950, 20, timeVal, 40, 50);
textSize(25);
fill(0);
text("time:"+Time / 1000, 990, 50);
} else {
textSize(25);
fill(0);
text("time:0", 990, 50);
}
for (int i=0; i<q; i++) {
x[i] = x[i] + xspeed[i];
y[i] = y[i] + yspeed[i];
if (x[i]>width-80) {
xspeed[i]=random(-3, 0);
mirror[i] = false;
}
if (x[i]<40) {
xspeed[i]=random(0, 3);
mirror[i] = true;
}
if (y[i]>height-100) {
yspeed[i]=random(-1, 0);
}
if (y[i]<40) {
yspeed[i]=random(0, 1);
}
if (mirror[i] == true) {
pushMatrix();
scale(-1, 1);
imageMode(CENTER);
translate(-x[i], y[i]);
rotate(angle);
image(display, 0, 0, 90 * scaleFactor, 80 * scaleFactor);
popMatrix();
}
if (mirror[i] == false) {
pushMatrix();
imageMode(CENTER);
translate(x[i], y[i]);
rotate(angle);
image(display, 0, 0, 90 * scaleFactor, 80 * scaleFactor);
popMatrix();
}
}
if (game == false) {
float progress2 = music2.position() / music2.duration();
for (int i=0; i < NUM_LEDS; i++) { // loop through each pixel in the strip
if (i < progress2 * NUM_LEDS) { // based on where we are in the song
leds[i] = color(255, 0, 0); // turn a pixel to red
} else {
leds[i] = color(0, 0, 0);
}
}
sendColors();
music.amp(0);
music2.amp(1);
//music3.amp(0);
display=ghost;
textSize(135);
fill(0);
filter(GRAY);
text("YOUR FISH DIED!", 155, height/2+80);
textSize(25);
text("CLICK THE RED BUTTON TO RESTART!", 700, 590);
// Stop the entire game
if (gameOverReason == 1) {
textSize(135);
fill(0);
text("TOO HUNGRY!", 250, height/2-80);
} else if (gameOverReason == 2) {
textSize(135);
fill(0);
text("TOO DIRTY!", 280, height/2-80);
} else if (gameOverReason == 3) {
textSize(135);
fill(0);
text("TOO CLEAN!", 280, height/2-80);
} else if (gameOverReason == 4) {
textSize(135);
fill(0);
text("NOT HAPPY ENOUGH!", 120, height/2-80);
} else if (gameOverReason == 5) {
textSize(120);
fill(0);
text("TOO MANY REASONS!", 120, height/2-80);
} else if (gameOverReason == 6) {
textSize(135);
fill(0);
text("TOO FULL!", 280, height/2-80);
}
}
if (game == true) {
music.amp(1);
music2.amp(0);
display = images[randomIndex];
//display=fish;
float transparency = map(cleanness, 0, 15, 250, 0);
//fill(180, 180, 120, transparency);
fill(80, 150, 230, transparency);
rect(0, 0, width, height);
if (mirror[0] == true) {
if (happiness>=3) {
image(happy, x[0], y[0]-50, 30, 30);
} else {
image(sad, x[0], y[0]-50, 30, 30);
}
}
if (mirror[0] == false) {
if (happiness>=3) {
image(happy, x[0], y[0]-50, 30, 30);
} else {
image(sad, x[0], y[0]-50, 30, 30);
}
}
if (showText==true) {
if (satietytrigger==true) {
textSize(20);
fill(0);
text("satiety+1", x[0]-30, y[0]-140);
}
if (cleaningtrigger==true) {
textSize(20);
fill(0);
text("cleanness+1", x[0]-30, y[0]-120);
}
if (playingtrigger==true) {
textSize(20);
fill(0);
text("happiness+1", x[0]-30, y[0]-100);
text("satiety-0.5", x[0]-30, y[0]-80);
}
}
if ( millis() -startTimeText > displayTextTime) {
showText = false; // Hide the text
satietytrigger= false;
playingtrigger= false;
cleaningtrigger= false;
}
if (arduino_values[1] == 1) {
moviefood.play();
foodPlaying = true;
foodFalling = true;
cleanness = cleanness -0.5;
for (int i=0; i<qf; i++) {
xspeedf[i] = random(-0.5, 0.5);
yspeedf[i] = random(2, 3);
}
}
if (foodPlaying == true) {
if (moviefood.available()) {
moviefood.read();
image(moviefood, 180, 150, 300, 300);
if (moviefood.time() >= moviefood.duration() *0.95) {
foodPlaying = false;
moviefood.stop();
}
}
}
if (foodFalling == true) {
for (int i=0; i<qf; i++) {
circle (xf[i], yf[i], 15);
fill(#F0C879);
strokeWeight(1.3);
yf[i] = yf[i] + yspeedf[i];
if (dist(x[0]+40, y[0]+45, xf[i], yf[i]) <= 40) {
satiety = satiety +1;
showText= true;
satietytrigger=true;
startTimeText=millis();
yf[i]=-50;
yspeedf[i] = 0;
scaleFactor *= 1.1;
scaleFactor = max(1.0, scaleFactor);
eating.play();
}
if (yf[i]>height) {
yf[i]=-50;
yspeedf[i] = 0;
}
}
}
if (foodFalling == false) {
for (int i=0; i<qf; i++) {
xspeedf[i]=0;
yspeedf[i]=0;
}
}
if (arduino_values[0] == 1 && showText==false ) {
startTimeText=millis();
showText=true;
cleaningtrigger=true;
cleanness = cleanness +1;
clean.play();
moviecleaning.play();
cleaningPlaying = true;
}
if (cleaningPlaying == true) {
if (moviecleaning.available()) {
moviecleaning.read();
image(moviecleaning, 900, 420, 500, 500);
if (moviecleaning.time() >= moviecleaning.duration() *0.95) {
cleaningPlaying = false;
moviecleaning.stop();
}
}
}
if (arduino_values[2] == 1 && showText==false) {
startTimeText=millis();
showText=true;
playingtrigger=true;
playing.play();
happiness = happiness +1;
satiety = satiety -0.5;
}
// Stop the entire game
if (satiety < 0) {
gameOverReason = 1;
game =false;
} else if (satiety > 15) {
game =false;
gameOverReason = 6;
} else if (cleanness < 0) {
gameOverReason = 2;
game =false;
} else if (cleanness > 15) {
cleaningPlaying = false;
gameOverReason = 3;
game = false;
}
}
if (arduino_values[4] == 1) {
startTime = millis();
scaleFactor = 1.0;
game = true;
satiety =5;
cleanness=5;
happiness =0;
celebration=false;
randomIndex = int(random(images.length));
display = images[randomIndex] ;
}
if (elapsedTime/1000 == 45) {
if (satiety >= 7 && happiness >=3 && cleanness >=6) {
game = true;
celebration = true;
} else {
game = false;
if (satiety < 7 && happiness >=3 && cleanness >=6) {
gameOverReason = 1;
} else if (happiness <3 && cleanness >=6 && satiety >= 7) {
gameOverReason = 4;
} else if (cleanness <6 && satiety >= 7 && happiness >=3) {
gameOverReason = 2;
} else {
gameOverReason = 5;
}
}
}
// fill(0);
//text(frameRate, 50, 50);
if (celebration==false && game==true) {
float progress = map(frameCount % 100, 0, 99, 0, 1);
//float progress = music.position() / music.duration();
for (int i=0; i < NUM_LEDS; i++) { // loop through each pixel in the strip
if (i < progress * NUM_LEDS) { // based on where we are in the song
leds[i] = color(0, 0, 255); // turn a pixel to red
} else {
leds[i] = color(0, 0, 0);
}
}
}
if (celebration==true && game==true) {
float progress3 = map(frameCount % 500, 0, 499, 0, 1);
for (int i=0; i < NUM_LEDS; i++) { // loop through each pixel in the strip
if (i < progress3 * NUM_LEDS) { // based on where we are in the song
leds[i] = color(0, 255, 0); // turn a pixel to red
} else {
leds[i] = color(0, 0, 0);
}
}
}
sendColors();
if (celebration == true) {
music3.amp(1);
textSize(115);
fill(0);
text("CONGRATULATIONS!", 145, height/2-60);
text("GOAL ACHIEVED!", 200, height/2+60);
} else {
music3.amp(0);
}
}
}
void getSerialData() {
// Read the cleaning value from the Arduino
while (serialPort.available() > 0) {
String in = serialPort.readStringUntil( 10 );
if (in != null) {
print("From Arduino: " + in);
String[] serialInArray = split(trim(in), ",");
if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) {
for (int i=0; i<serialInArray.length; i++) {
arduino_values[i] = int(serialInArray[i]);
}
}
}
}
}
void sendColors() {
byte[] out = new byte[NUM_LEDS*3];
for (int i=0; i < NUM_LEDS; i++) {
out[i*3] = (byte)(floor(red(leds[i])) >> 1);
if (i == 0) {
out[0] |= 1 << 7;
}
out[i*3+1] = (byte)(floor(green(leds[i])) >> 1);
out[i*3+2] = (byte)(floor(blue(leds[i])) >> 1);
}
serialPort2.write(out);
}
Here is the code for the first Arduino dealing with sensors:
const int trigPin = 9; //feeding
const int echoPin = 10;
float duration, distance;
long startTime = 0;
int prevDistance;
int val2; // for playing
int prevVal2;
int lastTiltState2 = LOW;
long lastDebounceTime2 = 0; // the last time the output pin was toggled
long debounceDelay2 = 200; // the debounce time; increase if the output flickers
int val; // for restart
int prevVal;
int sensorPin = 4; // for cleaning
int sensorValue;
int lastTiltState = LOW;
long lastDebounceTime = 0; // the last time the output pin was toggled
long debounceDelay = 200; // the debounce time; increase if the output flickers
int feedingtrigger;
int cleaningtrigger;
int playingtrigger;
int tiggertimes = 0;
bool gamePaused = false;
void setup() {
Serial.begin(115200);
pinMode(trigPin, OUTPUT);
pinMode(echoPin, INPUT);
pinMode(3, INPUT);
pinMode(sensorPin, INPUT);
digitalWrite(sensorPin, HIGH);
pinMode(11, OUTPUT);
}
void loop() {
digitalWrite(trigPin, LOW);
delayMicroseconds(2);
digitalWrite(trigPin, HIGH);
delayMicroseconds(10);
digitalWrite(trigPin, LOW);
duration = pulseIn(echoPin, HIGH);
distance = (duration * .0343) / 2;
if (gamePaused == false) {
if (distance < 10 && distance > 0 && prevDistance > 20) {
feedingtrigger = 1;
} else {
feedingtrigger = 0;
}
prevDistance = distance;
sensorValue = digitalRead(sensorPin);
// If the switch changed, due to noise or pressing:
if (sensorValue != lastTiltState && millis() - lastDebounceTime > debounceDelay) {
if (sensorValue == HIGH) {
cleaningtrigger = 1;
} else {
cleaningtrigger = 0;
lastDebounceTime = millis();
}
Serial.print(cleaningtrigger);
}
lastTiltState = sensorValue;
}
if (cleaningtrigger == 1 || playingtrigger == 1 || feedingtrigger == 1) {
tiggertimes = tiggertimes + 1;
}
val = digitalRead(2);
if (prevVal == LOW && val == HIGH) {
if (gamePaused == true) {
}
}
prevVal = val;
Serial.print(",");
Serial.print(feedingtrigger);
Serial.print(",");
val2 = digitalRead(3);
if (lastTiltState2 == LOW && val2 == HIGH) {
playingtrigger = 1;
} else{
playingtrigger = 0;
}
lastTiltState2 = val2;
Serial.print(playingtrigger);
Serial.print(",");
Serial.print(tiggertimes);
Serial.print(",");
Serial.print(val);
Serial.println();
delay(20);
}
Here is the code for the second Arduino dealing with LED strip:
#include
#define NUM_LEDS 60 // How many LEDs in your strip?
#define DATA_PIN 11
CRGB leds[NUM_LEDS];
int next_led = 0; // 0..NUM_LEDS-1
byte next_col = 0; // 0..2
byte next_rgb[3];
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
FastLED.addLeds<NEOPIXEL, DATA_PIN>(leds, NUM_LEDS);
FastLED.setBrightness(50); // external 5V needed for full brightness
leds[0] = CRGB::Red;
FastLED.show();
delay(1000);
leds[0] = CRGB::Black;
FastLED.show();
}
void loop() {
// put your main code here, to run repeatedly:
while (Serial.available()) {
char in = Serial.read();
if (in & 0x80) {
// synchronization: now comes the first color of the first LED
next_led = 0;
next_col = 0;
}
if (next_led < NUM_LEDS) {
next_rgb[next_col] = in << 1;
next_col++;
if (next_col == 3) {
leds[next_led] = CRGB(next_rgb[0], next_rgb[1], next_rgb[2]);
next_led++;
next_col = 0;
}
}
if (next_led == NUM_LEDS) {
FastLED.show();
next_led++;
}
}
}