Category: Research Post

Coastline Paradox

The Coastline Paradox is a project that visualizes the different sea levels over the years. This was my inspiration to tackle an environmental issue in my final project. Timo Aho and Pekka Niittyvirta, the artists behind this climate data experiment, present data that is hard to grapple with in a visually pleasing way. The audience is faced with the reality of future sea levels laid on top of the streets they know. The exact moment that inspired me while going through the project is the screenshot shown below as it shows an area that’s become part of my daily life here.

Dear Data 

Dear Data is a year-long project created by Giorgia Lupi and Stefanie Posavec. Throughout this time, they kept track of a type of data for a week, visualized it, and mailed it to each other on postcards. Before seeing this project, my way of thinking about visualizing the environmental data was very literal. This inspired me to look at abstract and minimalist designs.  

DOHMH Indoor Environmental Complaints

While scrolling through possible API’s or datasets to use, I came upon one that consists of indoor environmental complaints received through the 311 number. This caught my attention because asbestos, indoor air quality, indoor sewage, and mold are not usually given as much attention when it comes to casual discussions on the environment. This dataset inspired my choice of the categories “asbestos, indoor air quality, indoor sewage, and mold” as possible design parameters for different visualizing designs. The complaints are categorized by boroughs as well.

Resources: 

• https://experiments.withgoogle.com/coastline-paradox
• http://www.dear-data.com/theproject
• https://data.cityofnewyork.us/Health/DOHMH-Indoor-Environmental-Complaints/9jgj-bmct

My final project concept for my game Falling Bird was initially inspired by an art competition that I entered my junior year of high school. The competition was run by a foundation called Bow Seat. They provide an innovative space for teens to create, connect and communicate for our oceans through their ocean awareness programs.

This inspired me to make an educational game, but I wanted to make a game surrounding birds instead of the ocean. I was inspired by the once very popular game, Flappy Bird. I wanted the aesthetic of the game to be a mixture of vector and pixel art. I hope to attract nostalgic users of the game to my game as well because I did not know anyone at the time who did not like the game just those who couldn’t play for whatever reasons they had. 

When I came up with the concept of falling birds, I needed to have a reason they were falling. There could not simply just fall out of the sky This brought my mind to air pollution, a topic less talked about, so I thought it would be interesting to promote talking about it and hopefully bring further awareness through my game.  

Sign Language Using Teachable Machine

Schools and jobs encourage their students or workers to learn a new language to be able to communicate with a larger amount of people. However, I feel that it is a better use of people’s time to learn sign language because it would be more accessible to all people. This source discusses the idea of using deep learning and coding language to educate others on sign language. This site primarily uses python to teach and use hundreds of images. The concept of using code to teach a language reminded me of the app Duolingo. I believe that Duolingo should work with a teachable machine to teach American Sign Language as well as other sign languages. These ideas along with the concept of a game inspired me to use a teachable machine to try and make a sort of prototype of teaching sign language. I think this class provides an opportunity to be inspired and creative which I have enjoyed, but I also like the idea of taking that creativity and combining it with something to make it purposeful. I also believe that the teachable machine has a lot of potential and will be very present in future technology. 

https://www.startasl.com/sign-language-game/

https://www.funbrain.com/games/sign-the-alphabet

https://debuggercafe.com/american-sign-language-detection-using-deep-learning/

For my final project, I will make a Riddle Game. A monster will pop up on the screen and ask the player a riddle, and the player will have three guesses. If the answer is correct, the player will proceed to the next level, otherwise, they will lose. One of my main inspirations for this project is the story of Oedipus and the Riddle of the Sphinx. I like the idea of Oedipus trying to enter the town of Thebes and being stopped by the Sphinx, and that he could only enter if he answered the Sphinx’s riddle. It helped me come up with the idea of having a monster ask the riddle instead of it just popping up randomly on the screen. Another inspiration I had was Gods of Egypt (2016) – The Riddle of the Sphinx Scene. The atmosphere of the scene was very eery which inspired me to make my game on the scarier side. I also remember this scene very vividly since I found the idea and sound of the Sphinx monster very cool. 

One more main inspiration I had was The Riddler from DC comics. The Riddler, a riddle-obsessed supervillain, is one of my favorite characters in the show and I always wanted to make a project that had to do with riddles. The idea of having riddles be a timed thing that could lead to consequences was very unique and intriguing to me.  

Oedipus and the Riddle of the Sphinx

Gods of Egypt Riddle Scene

The Riddler (Picture Source)

The 3 major inspirational references that inspired my project were keyboard instrument patatap, dot piano, and jazz computer. Keyboard instrument patatap is a project where the computer is acting as an instrument, and different notes will be played when different keys are pressed. The reason for me finding this project interesting is that this project gives its users many freedom to compose unique music of their own, and this is the reason for me to decide to use keyboards and mouses for playing different notes/different instruments in my project.

Here is the link to keyboard instrument patatap: https://patatap.com/

Another project that inspired me was the dot piano. I really like the visualization of this project. This dot piano was showing lines with different colors at different positions when the soundtrack that is being played has different notes. The reason for me finding this project interesting is that it provides a visualization of music, and this is the reason for me to decide to display random geometric shapes with different colors when different keys and mouse are pressed, so that the players are not only composing their special music, but also creating their own art work.

Here is the link to the dot piano: https://yotammann.info/dotpiano

The third project that I found inspiring was the jazz computer. This project allows the temple of playing jazz to  change based on the speed of scrolling down the screen. The one aspect that I really liked about this project was also the visualization, and that is also the reason for me to add random generated geometric shapes with different colors when pressing the keys.

Here is the link to the jazz computer: http://jazz.computer/ 

The main inspiration for my project are the GIFs that are made to be anxiety relieving. From my own experience with anxiety and panic attacks, I wanted to create a game that actually keeps the user engaged, because sometimes in those moments of intense stress and emotion, it’s hard to stay grounded.  

I didn’t want to just create breathing exercises like the GIFs. So, I researched more grounding methods. The website I used is Medical News Today.

The other methods the website suggested that stuck out to me were the 5-4-3-2-1 method, the repetition of mantras, and muscle relaxation exercises. Personally, I have been advised to use the 5-4-3-2-1 method, but it’s never worked on me because I find it difficult to keep myself engaged, so turning it into a game so the goal of progression is there would be helpful for me and potentially other users. The goal of progressing in a game is also helpful with mantra repetition. I, and hopefully other people, find it uneasy to repeat positive affirmations about ourselves, so needing to verbally say the mantras to progress in the game would be helpful once again.

This is the website I’m going to use for a list of mantras. 

https://www.healthline.com/health/mind-body/mantras-for-anxiety

A very general inspiration are those artistic websites, they are very single-purposed and show nothing besides the work itself. They’re usually aesthetic and calming. These are some examples:

http://www.intotime.com/

https://codepen.io/neave/full/WNvGNez

Image Analysis and Face Recognition

The face is presumed to be an essential element in the body of a human being, vide the fact that it is the primary feature of identifying ourselves. The Facial Action Coding System (FACS) used in determining faces is said to use forty-six dimensions of the face for recognition. The significant modifiers regarding the topic of Image analysis and face recognition include recognition, detection, and tracking, all of which connote different meanings. Detection is described as a binary classification process wherein the task involves deciding on whether something is a face or not. Recognition, on the other hand, connotes the ability to distinguish between two different faces. Lastly, tracking in so far as this topic is concerned means the liaison of both detection and recognition to help know the same face recognized prior as the same as the one recognized afterward (McDonald, 2017).

The application of facial recognition is believed to have existed for some decades. Over the past years, it has been sophisticated through various advancements to arrive at better solutions in our day-to-day lives. This includes being used for security purposes, as illustrated by the actor Denzel Washington in the movie ‘Dejavu’ (Facial Recognition: Last Week Tonight with John Oliver (HBO), 2020). The application has also since evolved to incorporate better technology used in identifying and recognizing not only human beings but also sea animals like fish.

Despite these pronounced achievements on the use of facial recognition, the issue of algorithmic bias is an issue worth concern (How I’m Fighting Bias in Algorithms | Joy Buolamwini, 2017). The computer facial recognition system is believed not to be accurate and efficient enough in recognizing faces because they all depend on the designed machine language. Consequently, this renders the use of facial recognition as being inaccurate and their legitimacy in usage such as by the security in identifying criminals may be rendered inadmissible as evidence. Through research, it is a major concern that established facial recognition software often fail to recognize black people’s faces (Richardson, 2017).

References

Facial Recognition: Last Week Tonight with John Oliver (HBO). (2020, June 15). YouTube. https://www.youtube.com/watch?v=jZjmlJPJgug 

How I’m fighting bias in algorithms | Joy Buolamwini. (2017, March 29). YouTube. https://www.youtube.com/watch?v=UG_X_7g63rY 

1. (2017). Week 2 · kylemcdonald/AppropriatingNewTechnologies Wiki. GitHub. https://github.com/kylemcdonald/AppropriatingNewTechnologies/wiki/Week-2

Richardson, W. J. (2017, November 29). Against Black Inclusion in Facial Recognition. Digital Talking Drum. https://digitaltalkingdrum.com/2017/08/15/against-black-inclusion-in-facial-recognition/ 

The idea behind Clearview AI is one that is very concerning to me. As Ton-That demonstrates the use of the software to a journalist and a picture of the journalist when he was 16 years old shows up is not just worrying, but also terrifying. In the video, John Oliver also mentions how Ton-That has some even more worrying history with his previous companies.

The second concept that needs much more attention is the inclusion of everyone in facial recognition software. The software works off of data sets that are fed into it and can only recognize faces that are similar to what it was given. This is why we need software to encompass a full spectrum. The software itself is not to blame, but the humans feeding the data sets into the software.

“The struggle for liberation is not a struggle for diversity and inclusion — it is a struggle for decolonization, reparations, and self-determination” (Hassein)

I think this quote is extremely powerful. I realized during summer 2020, that a lot of corporations’ plans to end systemic racism consisted of creating 1 diversity and inclusion program and hiring a few black people in their attempt to be anti-racist. While I think diversity and inclusion are important, I think the way companies are going about is not productive. Placing black employees in corporate places where they still have to combat racism is not the answer. When companies or schools just hire more minorities without doing any work to change their racist, sexist or homophobic culture, they are simply subjecting these minorities to violent spaces where they will have to fight for their rights/lives every day. Then when the minority has had enough mental trauma from working in such a violently oppressive environment, the company will turn around and say “see we tried to recruit *insert group of marginalized people* they just don’t want to work”. It’s not about hiring more minorities to be forced to navigate these oppressive systems, it’s about destroying the current system and rebuilding it with people of all backgrounds in mind. That is the real work. That is the real solution. However, in order for this to happen, certain people in high places will lose money, and then power. 

” It is equally clear to me that no technology under police control will be used to hold police accountable or to benefit Black folks or other oppressed people” (Hassein)

I’m quite troubled by the increase in surveillance technology given to the police. It’s extremely problematic knowing that this technology is going into the hands of crooked and racist cops who will use this technology to harass oppressed people. Once again, you cannot throw new shiny technologies at century-old problems and expect them to not be used oppressively when work hasn’t been done to deconstruct the issues in our society. 

The title to the video “Facial Weaponization” is a very appropriate and accurate way of talking about face recognition bias. As Kyle McDonald’s article pointed out, faces are a way we associate with identity. By having programmed inequalities in face recognition, we are invalidating those who suffer from these prejudices. The flaws of algorithms do not just happen miraculously, they are programmed in, or failed to be noticed in the first place due to lack of representation. Programmed inequalities do not just affect technology, but real life mentality as well. 

In the video “Facial Recognition: Last Week Tonight with John Oliver,” he talks about how facial recognition is mainly used for law enforcement. I thought that it was good at first, but then he asks, “but at what cost?” and continues by explaining that facial recognition in law enforcement has been utilized in places that don’t necessarily need law enforcement (for example, identifying people at protests, interfering with the American right to protest). Furthermore, governments have used facial recognition to track people – which, when explained, seems like a very advanced degree of invasion of privacy. This video has taught me that facial recognition has gone too far and doesn’t necessarily help crime prevention.