Author: Gloria Liu

Concept and Design:

Our project is an informative game to raise people’s awareness of marine pollution, which is largely contributed by the daily garbage. We wanted the users to interact with our game by moving their head and even the position where they stand at to change the main character, the whale, in the game. In order to construct this interaction method for users:

1.    We used the OpenCV of the computer, so that the webcam can track the RGB color, which is the red area on the helmet that the user is going to wear. The reason why we chose this helmet was that it happens to have a red part in the front, whose color can be tracked by the computer. The other reason we used the helmet was that it was solid enough that it can hold some circuit on the top.

2.    The second thing we did for constructing this interaction method for users was to build a circuit, which includes two vibration motors so that the user can get informed if the whale eats the plastic. The reason why we chose the vibration motors was that the vibration is easier to be sensed especially in a noisy situation, where the sound effect could hardly be heard.

3.    The third thing we did was that we made a whale costume out of some Ikea shopping bags, so that it is easier for users to put themselves into the position of the whales, and it can also cover the red if the user is wearing red clothes. The reason why we chose the Ikea shopping bags as the material was that the color blue is similar to the color of the whale body. What is more, the recycled material corresponds to our topic. Originally, another choice of the material for the whale costume was to buy a whale raincoat online, but that betrayed the purpose of the game, which is to reduce the use of plastic.

 

Fabrication and Production:

The most significant steps in our production process were the process of coming up with the general concepts, the process of building up the physical interaction, and the process of building the visual settings for the game. Here are these three most significant steps in detail:

1.    The general concept of the game is an important guideline for the whole creation of our project. The successful part was that our project was not something merely has the sense of aesthetics or entertainment, instead, we set the link between our project and a worldwide problem, the marine pollution. However, the unsuccessful part in the step was that we restricted ourselves by sticking on thinking about how to make the game more educational at first, which made it hard for us to come up with new ideas of how to make the game more interactive.

2.    For the process of building up the physical interaction, the failure was that the instant feedback was not obvious enough to let the user know that they did something wrong since the circuit for the vibration motor sometimes can go wrong. The other failure was that the helmet was too heavy to carry, making it extremely difficult for the user to stand up, to squat down, and to move horizontally. The successful part was that this interaction method is really creative and immersive because the user has to move the head and even the body to change the position of the whale.

3.    For the process of building the visual settings for the game, the successful part was that we used Photoshop to modify the picture of the whale and put them into an array to make an animation so that the whale in this game can be more vivid. We used the background from Ecco the Dolphin, used PS to flip it, put the original and the flipped together, and made the background rolling infinitely, instead of simply using a stable background. The unsuccessful part was that when the whale moves to a brand-new stage, which means the pollution is getting worse, there is no notification to clarify that. What is more, the objects in the game was too tiny, and the food, the jellyfish, was not typical enough as food for the whale, and it may cause confusion for the users, although we have instruction at the beginning. Another failure was that we did not have a score rank, which serves as an incentive for the user to play the game again.

During the user testing session, we received the following suggestions:

1.    Objects:

INSTANT and CLEAR rewards or warnings should be added to tell the user whether they have done the right thing or not. Originally, we didn’t have any feedback for our users. With this suggestion, we added two vibration motors so that the user can be informed if he or she eats the plastic.

Background music & sound effect should be added. We added the background music later to make the game more immersive.

Notification such as “pollution is getting worse” should pop up when you move forward to a new stage

2.    Bar:

There should be a bar representing the life of the whale, to replace the method of showing the value of living or the number of the score.

The position of the bar should be at the bottom of the screen. Try to use multiple patterns of the whale to composite the bar.

We do add a bar at the bottom representing the life of a whale to replace the original value that we used. But we did not put multiple patterns of the whale to composite the bar, because we did not have enough time for modifying this…

3.    Interaction:

Draw an area or calibration on the floor for the user to stand in.

We stick some yellow tape on the floor to show the area in which the user can stand.

4.    Beginning:

Add a brief instruction

We added an instruction page at the beginning telling people what to eat and what to dodge, using some brief explanation and the PNGs of each object.

5.    Ending:

Telling people why the whale died

We added an ending page telling people the whale died of eating too much garbage and showing a the question “Plastic or Planet” to spur people to think about the educational the information behind the game.

6.    Costume:

In the sleeve, there should be a place to grab

I glued some air-cells for package inside the sleeve so that the user can have somewhere to grab at, and the sleeve would look more like the wing of the whale.

Stick the hood with the helmet

I put a Velcro between them.

Replace the helmet with a mask

We did not change this part because a mask has nowhere to hold the heavy circuit.

Get rid of the Ikea trademark

I removed the Ikea cloth and put some Velcro instead, so that the tightness of the costume can be adjusted as well.

 

Sketches and drawings:

 

 

 

Sewing Machine

Final Demo on the IMA Show

Conclusions:

The goal of my project is to make an informative game about marine pollution, in which people can change the position of the whale by making certain gestures. My project aligns with my definition of interaction that it has some social utilities, such as raising people’s awareness about marine pollution. The part not aligning with my definition was that we did not give the audience enough space to create something by themselves, and maybe the tuition part was still not clear enough, which may lead to confusion. During the IMA show, the audience had fun playing our game, especially for those kids. But they did complain that the helmet was too heavy, and they did not know in which way they should move so that the position of the whale can be changed efficiently. If I had more time, I would improve the animation part, so that every time the whale eats the plastic, the whale would get puffy, and maybe swim slower. I would also improve the instruction part so that the user can be taught step by step to learn how to play the game. I would also think about changing the objects in different periods, which correspond with the developing process of marine pollution. I learned from the unsuccessful part that sometimes it is more important to make the game more intuitive and convenient to play for the audience. It does not always work if you try to make the game more complex in order to make it look fancy. I have learned from our accomplishments that it is important to support the game with a significance which can be found in reality and figure out ways to attract the audience.

To sum up, our project has great educational significances not only for those little kids but also for everyone. We would like to tell people that each of them is actually a contributor to the marine pollution either in a direct way or an indirect way since that daily garbage like bubble tea bottle and water bottle can actually do great harm to those marine creatures. However, I guess most of the audience would be willing to pay attention to this game, since the interaction way of moving their head is fun, and the costume is cute as well.

I attended Leon’s recitation about media manipulation. After going through the basic codes for using images or videos to make media manipulations, I downloaded a video called The Whale which tells a story that a man was looking for whales to kill using a telescope, and did not realize that he was stepping right on a whale, which was hiding itself in this way to escape from being killed. I also downloaded a Pepe frog meme saying “Be Nice Man” to put into this exercise.

After importing the video and the image using “import processing.video.*” and “PImage”, I adjusted the speed into the 1.2 times of the original. The new function that I have learned from this recitation which is very useful is video.time(), which is counted by seconds. With the help of this function, the image appears at the 30^th second and disappears at the 34^th second.

Apart from doing the recitation exercise, I also asked a question about my final project that how I can slow down an animation array without touching the framerate. Thanks to Jingyi’s help, I learned that instead of using the Photoshop to create series of pngs of animation, using After Effect to create an mp4 video is a better choice, before which I should cut more slices for the animation.

Code:

import processing.video.*;

Movie whale;

PImage frog;

void setup(){

  size(1000,600);

  whale=new Movie (this, “The Whale.mp4”);

  whale.loop();

  frog= loadImage(“nice.png”);

}

void draw(){

 if(whale.available()){

  whale.read();

  whale.speed(1.2);

 }

 image(whale,0,0,1000,600);//////////

 float timeStamp = whale.time();

 println(timeStamp);

 if(timeStamp >=30 && timeStamp <=34){

 image(frog,400,250,200,200);

 }

}

For this recitation, I wanted to pre-explore a little bit about the animation part for my final project, with this given opportunity of using the Processing and the Arduino to control a media. Therefore, I used a sketch drawn by myself with my laptop.

First of all, I opened the given model for connecting the Arduino to the Processing. Then I uploaded the picture into the Processing, set the size, the background, but later found out that I forgot to define the image using “whale = loadImage(“whale.jpg”) after loading the image. Then I connected the circuit using one potentiometer and wrote the code for the Arduino part, which is pretty simple since I only need to write the analog in pin for the potentiometer. Then I went back to the Processing and modified the image with “if” and “else if” so that the size and the filter of my image can be changed in terms of the value from the potentiometer, from 0 to 1023.

Then I ran the code, but it seemed that the printed number for the potentiometer did not change in the Processing. With Tristan’s help, I changed the “Serial.print” into “Serial.println” in the Arduino part, otherwise, the system would be unable to recognize the new value given by the potentiometer.

I ran the code again, but the white background in my image was ugly, Therefore I used the Photoshop to delete the background of my original image with Nick’s help.

Finally, my whale can get bigger or smaller or would be added with a filter as I twisted the potentiometer.

According to Computer Vision for Artist and Designers, the way that technology was used in my project is the computer vision algorithms, through which the digital image can be modified into different art genre enabled by the vision functionalities. With this technology, the variability of creation can be generated, since a fixed prototype of an artwork can be modified into different versions by different audience, through the interaction with the computer. This method empowered my project since I can turn a fixed artwork into different versions in terms of the settings that I want to put my project in.

Code:

Processing part:

PImage whale;
import processing.serial.*;
String myString = null;
Serial myPort;
int NUM_OF_VALUES = 1; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/
int[] sensorValues; /** this array stores values from Arduino **/
void setup() {
size(1023, 1023);
background(255);
setupSerial();
whale = loadImage(“whale.jpg”);//////
}
void draw() {
background(255);
updateSerial();
printArray(sensorValues);
println(sensorValues);
if (sensorValues[0]<400) {
image(whale, 0,0,400, 400);
} else if (sensorValues[0]>=400 && sensorValues[0]<700) {
image(whale, 0, 0, sensorValues[0], 0.8*sensorValues[0]);
} else if (sensorValues[0] >=700 && sensorValues[0]<1023) {
image(whale, 0,0, sensorValues[0], 0.8*sensorValues[0]);
filter(INVERT);
}
}
void setupSerial() {
printArray(Serial.list());
myPort = new Serial(this, Serial.list()[ 0 ], 9600);
sensorValues = new int[NUM_OF_VALUES];
}
void updateSerial() {
while (myPort.available() > 0) {
myString = myPort.readStringUntil( 10 ); // 10 = ‘\n’ Linefeed in ASCII
if (myString != null) {
String[] serialInArray = split(trim(myString), “,”);
if (serialInArray.length == NUM_OF_VALUES) {
for (int i=0; i<serialInArray.length; i++) {
sensorValues[i] = int(serialInArray[i]);
}
}
}
}
}

Arduino Part:

void setup() {
Serial.begin(9600);
}
void loop() {
int sensorValues = analogRead(A0);
Serial.print(sensorValues);
Serial.println(“,”); //print the new ln!!!
delay(10);
}

1. Luke’s “Magic Brush” is a drawing machine for the one who does not have much prior art skill or training. People can control what they want to draw by moving the joystick and clicking the buttons. I suggested that he can refer to some projects which are designed for the disabled who are unable to draw, and they can input the signal using different gestures. Other group members suggested that body gestures can be used, and there should also be interactions between the user and the computer to change the genres of the painting, for example, the options of ink painting or oil painting. The interesting point about this project’s concept was that it is a useful tool to help people generate their art creativity, without being constrained by their ability to draw. This project reinforces the exploration of how we interact with technology by creating a language system for drawing, deciding which kind of pattern or genre to draw.
2. Julia was going to make a maze game based on the change of the sound effect for the player, instead of simply by the visual guidance. For this project, we gave suggestion about making some backed-up plans since this project sounds really challenging for controlling the change of the music in a proper range, according to the player’s order. I think it is interesting that people use their judgment of the volume of the music and the direction in which the music came from to decide which way to go in that game. This project creates an interaction between people and technology that it combines the 3D visual signal and sound effect to give people experiences from various aspects.
3. Linhui’s project is an interactive installation, combining the sound of traditional Chinese musical instrument, and the scenery of nature. The music can be played as the user interact with the flower. My advice was that they should make sure that the component to create the sound should be sensitive enough, and try to close the gap between the user’s action of interacting and the sound effect. Other groups members suggested that they should add more musical instruments so that this installation can create varied sounds. The concept of this project intrigued me because it is not simply a musical instrument, instead, it is an art installation which can give people a sense of happiness both visually and aurally. This project fosters the interaction between the humans and the technology by adopting the elements from nature, such as flowers, and from traditional Chinese culture, such as Bianzhong (a musical instrument).
4. Eugene’s project is a game about Avengers. Players will play the role of Marvel superheroes and Thanos.  Our group members suggested that he can add more fancy effects for the movements and attacks according to the specific role in this game.

After seeing their project proposals, I added something new to my definition of interaction. To get the audience fully involved, the process of interaction should contain different forms through vision, audio, or body gesture. This new definition gotten from my group members’ projects focuses more on the variability and possibilities of the interaction, while I simply paid attention to the content and the form of what the interaction should be like which is a qualitative thought and ignored the problem about the quantitative.

Feedback:

1. Set the background of the game a dynamic one, which moves backward as the whale moves forward.
2. The changes after the whale got touched with the garbage should be obvious. For example, the whale can get darker and darker or more and more swollen as it touches the garbage in the ocean.
3. There should be a yardstick to judge whether the player has passed or failed, otherwise, the game would turn out to be an infinite loop.
4. An informative end should be added to the game, such as showing people the picture of how the marine creatures got threatened by marine pollution.

Evaluation:

The most successful part: The concept of the game is good. Instead of merely being an entertaining game, it has educational significance for the audience. I agree with this because this is the fundamental inspiration which supports our game.

The least successful part: The way of how the players are going to interact with the computer was not clear. I agree with this as well, because we have not decided in which way would either the player 1 or player 2 interact with the computer in order to make the players get fully involved in the background of the game, and put themselves in the whale’s position.

The feedback above has a profound influence on my project that it makes me clarify and pay more attention to the details of the game, instead of simply coming up with a grand concept with a vague structure of the game.

Improvements:

1. Changing the background from a planar graph of a maze into a dynamic one, which moves backwards as the whale moves forward.
2. There should be a visible difference after the whale touches the garbage to show the negative effect from ocean pollution.
3. The yardstick of judging whether the players passed or failed the game should be clear, and the game should not be an infinite loop.
4. Pictures and data about marine pollution would be added at the end of the game, and the players would learn this information through an impressive and interactive way.

Project Title:

Ocean, Whales, Us

Project Statement of Purpose:

The topic and insight of our project is the severe situation that the marine creatures are in, caused by ocean pollution. The direct inspiration we got was news that I saw from the website of National Geographic (https://www.nationalgeographic.com/environment/2018/11/dead-sperm-whale-filled-with-plastic-trash-indonesia/). In November 2018, people found 13.2-pound plastic contents in the stomach of a dead sperm whale, which was washed ashore in eastern Indonesia. We also found more research data provided by the National Geographic that “192 coastal countries contribute a combined total of 8.5 million tons of plastic waste to the oceans every year”. Some other researches (https://www.sohu.com/a/260503756_172952) have suggested that plastic particles have been found in the salt that people eat.

Inspired by those heartbroken news and researches, we plan to make a two-player game, which can make people realize the severe situation that those marine creatures are in and raise their awareness that if the ocean pollution cannot be relieved, the negative impacts by the pollution would also go back to impact people’s lives. The aimed audience is everyone, since everyone has contributed to polluting the ocean, either directly or indirectly, and either by misconducting the daily garbage or emitting huge amount of industrial pollution. To make experiencer get a deeper understanding of the bad situation those marine creatures are in, we decided to set two players in this game, one of which plays the role of a whale which has to not only stave off the garbage in the ocean but also breathe above the water for every several minutes, while the other plays the role of a salvage ship, which has to remove as much garbage from the ocean as possible.

Project Plan:

Our project aims to raise people’s awareness of the bad status that those marine creatures are in, by letting them save a whale in the game. In order to let the audience experience a sense of crisis and tension, one of the experiencer would play the role of a whale trying to survive in a polluted ocean, while the other would serve as a salvage ship removing as much garbage as possible to save the whale. Here is the detailed timeline:

April 26- April 27:

The basic design for the whole game and detailed compositions

Research on how to write the code for the game

The basic design for the animation

Decide and start to prepare the necessary materials

Decide the ways of interaction and console for the two players (adjustable later)

April 28-May 9:

Finishing the code

Finishing the animation

Finishing the circuits

Finishing the physical part

 

May 10-13:

User testing with our friends, our classmates, and IMA fellows. We will not only listen to their comments but also ask them questions initiatively in terms of the points we are concerned about.

Make final adjustments according to the feedback

(the sensitivity of interaction for the audience, debugging the circuit and codes, thinking about whether the informative significance of the game is clear enough for the audience, ways of displaying the game for presentation)

May 14-17:

Preparation for the Presentation & Presentation

By May 22:

Blog Post

Context and Significance:

My preparatory research and analysis have a decisive impact on my proposal for the final project. I defined interaction as an informative conversation between several actors. Inspirations, creations, and many other utilities can be generated, and the relationship between the actors should be clear. Without this definition from my preparatory research, my final project would probably turn out to be an aesthetic art-piece with no informative inspiration for the audience or no connection with the problems in our life. The project I’ve researched on, Le Vent Nous Portera by Alexander Letcius and Bulat Sharipov, also gave me some inspirations. In that project, the experiencer took a decisive place for which scenery this project is going to form. Therefore, I was also thinking about whether there is a way to let the experiencers maximize their own subjective initiatives to reach different results for the game, depending on what aim they want to achieve in the settled game background, which is the ocean pollution for my project. Therefore, my project aligns with my definition of interaction that my game is an informative conversation between the computer and the experiencers, and people would be inspired to think about the severe threat faced by the marine creatures. It is not only about a sense of aesthetics, but it can also actually generate some social utilities. The uniqueness about my project is that those two players collaborate with each other to achieve the goal of saving the whale, instead of competing with each other. This form would not only add a relationship of interaction for the actors in this process but also reveal a lesson for the humans that different individuals should work together to achieve the goal of avoiding the marine creatures from the ocean pollution in an effective way. Hopefully, our project can make people realize how bad the situations that those ocean creatures are in, and how it is possible that people can actually change the current situation if they collaborate together.