Author: Justin Wu

Reflection:

In this week’s recitation, I decided to join Leon’s group that focused on Media Manipulation through images and videos. As I intend to build a game for my final project, I wanted to figure out how I can manipulate images, videos that I can incorporate into my game.

Leon demonstrated how to apply filters, pixels to different images, videos and I decided to get a headstart on my final project.  Using what Leon demonstrated in class, I wanted to start creating my starting screen and a countdown video for my game.  As my game heavily revolves around music and rhythm, it is important to give users a countdown before immediately throwing them into the game. First, I began by searching for a suitable starting screen. I found a simple starting screen to begin with and included a mousePressed function after it. I intended to create a three step process 1) Starting Screen 2) Countdown 3) Game

After finding a starting screen and writing the mousePressed function, I found a popular countdown gif with the help of Professor Rudy.  By using a gif and not a video, it helped made my coding process easier. Processing identifies a gif as an image so I could continue using my original code from before.  As of right now, the gif will end after the countdown and bring you to a white screen. We intend for our game to replace the white screen and will replace it after we finish coding our music rhythm game.

Video:

Code:

import processing.video.*;
Movie myMovie;
void setup() {
size(720, 480);
photo = loadImage(“StartScreen.jpg”);
myMovie = new Movie(this, “giphy.mp4”);
//myMovie.play();
}
void draw() {
image(photo, 0, 0,width,height);
if (myMovie.available()) {
myMovie.read();
}
image(myMovie, 0, 0,width,height);
// Draws a line on the screen
// when the movie half-finished
float md = myMovie.duration();
float mt = myMovie.time();
if (mt >= md) {

background(255);

}
}

void mousePressed(){myMovie.play();}

Reflection:

In this recitation, I decided to use a potentiometer to control the filter of an image. Depending on how much I twisted the potentiometer, it would cast a different filter for the image. I only used one potentiometer, that controls the range of the values. The filter is defined by different RGB color values. In my project, I believe I did not fall for the fallacy that computer vision should be limited to military or law-enforcement purposes. Instead, I believe I experimented with blurring the lines of computer reality with reality. I used a picture captured by a professional camera that illustrates the sanctuary of nature but I also decided to use computer coding to enhance user experience. Technology was used to capture the image I chose but it is also employed to help improve the overall experience by allowing users to interact with the image through filter technology.

Processing Code:

Image:

Step 1

During recitation this week, I discussed our final project with Jackson Sayama, Sarah Waxman, and Jessica Xing. All three of my classmates had a very different definition of interaction and final projects.

First, we discussed Jackson Sayama’s project “Bongo with Bongo Cat.” Jackson Sayama’s project is quite similar to my final project as we both define it as a music rhythm project. However, instead of offering instructions Jackson’s project encourages users to freelance and do not follow instructions. As my final project will not work if users do not follow instructions, it is refreshing and interesting to see Jackson promote a no boundary project. Unlike most modern technology, “Bongo with Bongo Cat” creates a meaningful experience for users as it does not provide users with a maze to get out of and advocates for users to interact with technology as if it is a blank piece of paper that is waiting to be drawn on. As Jackson already had a half working prototype, I offered him advice on hardware issues instead of software issues. Such as using a three finger paw instead of a four-fingered paw to make user testing easier, while other group members advised him to include another paw so it could be a collaborative music rhythm game.

Next, we talked about Jessica Xing’s project “Manamonster.” Inspired by the Japanese gadget game Tamagotchi, Jessica Xing aims to create a virtual pets game that will test users idea of prettiness and ugliness. She aims her project at young kids and promotes the idea that love should not be based on aesthetic features. I found Jessica’s idea interesting as it uses a childhood game in a new purpose, instead of just fostering care and building personal relationships with your pets, Jessica wants users to grow comfortable with their pets as they grow older and “uglier.” Jessica’s project uses technology to create a meaningful partnership between the users and their pets, instead of just relying on technology to help us achieve our goals, “Manamonster” wants to change people’s appreciation of beauty. I gave Jessica some feedback regarding her presentation of her project, as the original Tamagotchi game used pixelated creatures to depict user’s pet, I suggested Jessica to use clear images to outline indeed what the creature’s appearance to help define the ugly and the pretty. As a group, we also told Jessica how she could use a reverse cycle of her game, and it essentially means that when the users feed and care for their pets, it will stop the pets from growing instead of catalyzing their growth. This way, users will attend extra care for the pet and learn to appreciate the pet.

Finally, we discussed Sarah Waxman’s project “Tap the World.” In this project, Sarah Waxman aims to provide an alternative yet entertaining way to participate in the global cause. She will build a bottle recycling gadget that will help NYU Shanghai’s recycling effort. She plans to add a trash bin with three different-sized holes on the cover; users should toss plastic bottle caps into the holes. Depending on whether the user makes the shot or which hole they successfully throw the caps into, the screen next to the trash can will show a globe revolving at a similar rate. The smaller the mess that the cap goes through, the more the world will rotate. Like Jessica’s project, Sarah’s project helps establish and educate users on the urging need to address global warming. Sarah’s project also explores how humans do not always have to hesitate to interact with technology as we are afraid it will add fire to the global warming movement. Instead, her project helps us understand how technology could work hand in hand with our effort to combat global warming. I offered Sarah some advice regarding the hardware, I was skeptical about having a screen next to the bin, so I advised her to have a globe on top of the bins. The world should rotate every time a user makes a shot. On the other hand, other group members also suggested Sarah display fun facts about global warming on the screen when users make a shot.Jackson defines interaction as two or actors affecting one another in a psychic process.

Jessica Xing defines interaction as expressing an idea that should be translatable to human emotion. Jackson Sayama defines interaction as two or actors affecting one another in a psychic process. Sarah Waxman defines interaction as a process that incorporates users and provides responses promptly to a particular action placed by the user. Unlike my definition of interaction, my group members define interaction more broadly. However, Jessica’s definition of interaction is particularly interesting as users do not necessarily regard an action to translate into emotions.

Step 2

According to my group members, the most successful part of my proposal is the competitive and nostalgic component of my game. The short game fits well with my targeted users as it does not take up much of their time and the competitive aspect adds a different element to the experience. I agree with my group members as Julie, and I spent quite some time trying to integrate multiple users into our project, and I see my group member’s approval as validation. The least successful part of my project, according to my group members, is the incorporation of the photo aspect. They said the purpose of the photo-taking aspect was unclear and seemed redundant. I partially agree with their feedback as I can see how users may be confused about why our project captures a picture of their experience. However building onto my group member’s feedback, Julie and I will have to discuss how we want to incorporate the photo taking experience. We could either lose the photo taking or keep it and add a “winner” and “loser” filter to help indicate which user was more skillful. Based on the feedback from my group members, I believe I can draw some inspiration from Jackson’s project and feedback. Jackson advised me to shy away from the Tap Tap Revenge format (which requires a lot of visual) as it might take more time than we have. I will discuss with Julie about the possibility of making our game less dependent on visual cues and instead rely more on audio cues.

Project Title:

Tip Tap Snap

Project Statement of Purpose:

For our final project, Julie and I want our users to have an experience of teenage nostalgia, forget about college stress and recall what it feels to have a good time and create memories that last a lifetime. Growing up, we all played favorite games such as “Tap Tap Revenge” or “Guitar Hero”; easy games that incorporated music and rhythm. There is no learning curve for these simple music rhythm games and will resonate with our mission of creating a project that will easily impress our users. Julie and I started brainstorming for ideas and realized we should resume our theme from the midterm project. We wanted to capitalize on our overlapping memories with our potential users while simultaneously introduce a fun interactive game to users who are not familiar with music rhythm games. While new users may present a challenge as they do not understand the purpose of our game, we are confident we will create an easy game that will not require much learning.

Project Plan:

We aim to create an interactive, competitive and exciting gaming experience for our users by combining people’s love for music and competitive nature. Instead of the traditional single-player games, we integrate a friendly sense of competition so our users can test their music sense and be put up against their peers. We intend to make use of makercase.com and the laser cutter to help fabricate a vintage arcade gaming box. On top of that, we also plan to use the laser cutter to cut out three ellipses on the cover of the box so we can place three different colored buttons. The three different colored buttons will be the main gaming interface of our project; users are expected to tap the corresponding colors on the panel when different colored balls reach the bottom of their screen. We plan to use Arduino to Processing and Processing and Arduino to help us connect our buttons to a computer screen. Whenever the button is pressed, it will trigger a pressure sensor that will light up the corresponding button, therefore providing the user feedback and when the users stop pressing the button, the feedback stops. Last but not least, in order to implement a surprise element Julie and I plan to add a photo capturing code. By taking a picture of the users during or after their game, we will be able to capture a moment of their experience when they least expect us to.

We plan to use Processing to code the main menu that will present users a selection of songs and also create the notes necessary to compose the song. We expect to start with the fabrication process, by measuring the ellipses for the buttons and finding space to store our Arduino to create a tidy appearance for our project while also allowing the users to focus on the arcade box. Having our user’s undivided attention on the arcade buttons will be an essential first step as it should grasp the user’s attention immediately while also conveying what our project could be. While working on the fabrication, we will also be dividing our time on the coding process, and we will focus on coding once we complete fabricating. The coding process should take the bulk of our time, but Julie and I already have a shortlist of catchy and well-known songs that should easily catch on with our users. Our game will consist of two separate scores, depending on how well each player fares. The score will reflect the individual player’s overall accuracy percentage and serve as our scoring system. We plan to have a blueprint of our project by the start of 4/28 and proceed to the fabrication process. During the coding process, we will most likely bump into a user testing session. We plan to have a good bit of our project completed by the user testing session as we value what our peers have to comment about our game. We expect the coding and testing process to take up about two week’s time maximum, but we should be able to work around our schedule in case of an urgent situation. The overall game will be our main priority before we proceed to create the photo taking element of our game. As the project is due on May 13th to May 17th, we want to be ready to start coding no later than May 5th. This means we will have to finish planning and to fabricate by then, and we plan to do so as we believe testing our project will be a crucial determinant. Julie and I want to be able to check our game as many times as possible before submitting

Context and Significance:

During my preparatory research and analysis, I realized the need to incorporate more creativity into our final project. In between my midterm project and now, my definition of interaction changed, I realized besides two parties bouncing ideas off each other, having a (1+1>2) effect, it should also be out of the blue. Therefore Julie and I decided to recreate our childhood game with a twist. Our final project draws inspiration from Tap Tap Revenge and Guitar hero, but instead of creating a single player game, we wanted users to compete against each other. We believe having one user experience this music rhythm game would be mundane but implementing friendly competition would create the (1+1>2) effect. The two players will be able to talk about their experience and make the game more collaborative than before. While I was researching for possible ideas, I found many creators borrowed past concepts, but there were only a few cases of creators furthering the borrowed ideas or even utilizing their inspiration in a different form. I wondered if it was possible to connect a nostalgic gaming experience from my childhood days with a new skill we learned in class. Professor Marcela was teaching us how to use our built-in video cameras on our laptops through Processing and Arduino earlier this week, and it got me thinking about how we could incorporate that into our game. Then, I recalled the article “Alexa, How Will You Change Me” I had read about Amazon and Google’s trailblazing experiment with smart speakers. I realized we should not have to force to combine two ideas together as it might become a misfit. If we want to combine a music rhythm game with photo capturing technology, we should aim to make it as seamless as possible. We want to further the original idea of a music rhythm game by capturing the moment that means the most.  I was inspired by Tim Deagan’s idea of merging aspects of our lives to create a project after reading about his project “Light Leather Arm Braces” By developing our own music rhythm game and picture taking technology, we combine two common aspects of our lives, entertainment, and memory, together. We do not have to create the most complex project ever made, we just have to understand how to connect to our users like Google and Amazon did. Our plan should bring joy and nostalgia out of our audience while they are experimenting with our game yet at the same time offer a particular value to them by showing them their emotions when the game reveals the score, this is a lot like riding a rollercoaster in an amusement park. While we ride a rollercoaster park, we are focused on bracing ourselves for impact, but in the end, it is always exciting to find out through the pictures how we looked during the ride. If our project becomes a full range of success, I can see arcades implementing this idea as well. Games are competitive itself, and like a sporting event, it elicits exclusive emotions, by capturing these emotions we create stories worth a lifetime.

Similarly, it should not just stop at arcades, as people begin to embrace photo taking opportunities and start to snap pictures of their daily lives, we can push the idea of capturing emotions at a surprising time to a whole new extent. Why not sell this to when an electronic version of a college decision is announced? The possibilities that can arise from our project is immense, but for now, we want to keep it simple and delicate, create a game and camera that will record and recall old and new memories.

Overall, Tip Tap Snap perfectly embodies my interpretation of interaction. Not only will our game require players who bounce ideas off the game itself by carefully observing how well they are doing but it also creates the (1+1>2) effect as users compete against themselves and friends to elevate this gaming experience. Last but not least, Tip Tap Snap glues an arcade game with photo capturing technology to employ a creative twist for our users, we want our users to be delighted when we present to them their gaming face. Most importantly we also want our users to truly interact with our project.