WORRY BLOWER | Qiya Huang | Rodolfo Cossovich

 

CONCEPT AND DESIGN

In our project, we aimed to address the pervasive issue of worry and negative emotions by providing users with a tangible and cathartic outlet for their concerns. The concept emerged from a synthesis of two key elements: bubbles and worries. We observed how people are constantly burdened by worries in today’s fast-paced world and sought to create a symbolic representation where worries, like bubbles, could be released and ultimately dissipated.

Central to our design was the interaction between users and the animated head displayed on a monitor, filled with bubbles representing worries and negative emotions. We integrated face and speech recognition technologies to enable users to communicate their worries verbally. A threshold volume feature allows the user to “blow away” their worries by speaking out loud, thus reducing the number of bubbles on the screen. At the same time, the bubble machine will begin to operate, blowing bubbles depending on the volume level. This interaction is intended to provide a tangible interactive experience, symbolizing the process of releasing worries and seeing them eventually disappear, like bubbles in the air.

During the user testing phase and final presentation, we received valuable feedback that had a significant impact on the design. One important aspect that stood out was the sensitivity of expressing concerns in public. Recognizing this, I realized that we were overlooking parts of the user experience and considered whether this entire project should lean towards a peaceful, soothing atmosphere or a more intense emotional release. This feedback prompted us to improve the direction of the project.

FABRICATION AND PRODUCTION

During the fabrication and production phase of our project, we encountered both successes and challenges as we worked towards realizing our vision of a bubble-based worry release system.  Our primary focus was on creating a functional bubble machine and integrating it with facial and voice recognition technologies, as well as developing the necessary Arduino circuits and code.

The fabrication process began with testing the fan to ensure it operated as needed. 

Once confirmed, we followed a tutorial to construct the automatic bubble machine. However, our initial design encountered a significant hurdle: the bubbles did not blow out easily due to the large hole size and insufficient fan power. To address this, we iterated on the design, reducing the hole size and increasing their number. This adjustment proved successful, leading to a version of the bubble machine that effectively produced and released bubbles.

In terms of electronics, our Arduino and circuit setup mainly underwent two iterations. Initially, we tested the bubble machine’s analog reactivity using a potentiometer before integrating it with the p5 environment. After confirming that the bubble machine could adjust its speed of operation based on the potentiometer-transformed values with the existing circuitry and code, we proceeded to integrate the values from p5 in place of the potentiometer-transformed ones.  This transition allowed us to establish a direct link between the user’s input and the bubble machine’s behavior.

In integrating facial and voice recognition functionalities into our project, we opted for the p5.js framework over Processing due to its compatibility with web-based applications and ease of implementation.

For facial recognition, we used a method developed by Kyle McDonald that effectively detects faces and facial features. Using this technique, the user can see his or her face on the monitor. Based on the indexed data provided by McDonald, we sketched out the shape of the eyebrows, eyes, nose, mouth and face to accurately represent a mirrored animated character on the computer.

For voice recognition, we employed the p5.speech.js library, which facilitated the transcription of user speech. By initializing a new instance of p5.SpeechRec and specifying the language (“en-US”), we could capture and process user utterances in real-time. The parseResult() function allowed us to extract and utilize the most recent word or phrase detected, enabling responsive interactions based on user input.

For bubble creation, I drew inspiration from previous creative coding projects while introducing novel interactions based on the goals of our project. One of the major innovations was to adjust the size of the bubbles on the screen according to the volume level of the user input. When the volume exceeds a pre-set threshold, the size of the bubble decreases until it disappears completely.

One of the most challenging aspects was establishing serial communication between the p5 and the Arduino. We initially tried various methods, including the p5.serialcontrol app. We ended up using a more traditional method recommended by Prof. Rudi. This involved coding the communication process directly in the p5 and Arduino to ensure reliable data transfer between the two platforms.

The code snippets below illustrate the process of successfully receiving data from Arduino in p5.js. We use the p5.SerialPort object to establish a serial connection and listen for incoming data. When data is received, the serialEvent() function is called to process it. It’s important to note that the data type used in p5.js to receive and process data from Arduino should match the data type sent by Arduino. In our case, we found that using the Number() function to convert the received data to a number was essential for successful communication. Other data types like Byte() and Int() were not suitable for our application and resulted in communication failures. 

And to send data from p5.js to Arduino, we mainly use “Serial.write().” In the Arduino code, we define the pin connected to the fan motor and set it as an output. Inside the loop() function, we continuously check for incoming data from p5.js using the Serial.available() function. If data is available, we read it using the Serial.read() function and use analogWrite() to control the fan motor speed based on the received data.

By refining our designs and overcoming technical hurdles, we were able to create a functional and impactful project that effectively fulfilled its intended purpose.

CONCLUSIONS

Our project aimed to provide users with a tangible and cathartic outlet for expressing and releasing worries through an interactive bubble-based system. We sought to create an experience where users could visually and physically engage with their emotions, ultimately experiencing a sense of relief and catharsis. In reflection, our project largely achieved its stated goals. We successfully implemented facial and voice recognition technologies, integrated them with the bubble machine, and enabled dynamic interaction between the user, the digital interface, and the real world. Users could see their mirrored faces, express their worries verbally, and witness the tangible representation of their emotions through the creation and release of bubbles.

From the setbacks and failures encountered during the project, I learned the importance of iterative design and the value of embracing challenges as opportunities for growth. Each obstacle we faced presented a chance to learn, adapt, and ultimately improve the project. These experiences underscored the iterative nature of design and the necessity of resilience in the face of challenges.

In conclusion, our project has demonstrated the power of interactive design in fostering emotional expression and connection. Through thoughtful integration of technologies and a user-centered approach, we were able to create an engaging and meaningful experience that resonated with our audience.

DISASSEMBLY

APPENDIX

Bubble Machine 

• https://www.youtube.com/watch?v=CWFRqQhGa6w(mechanism)
• https://forum.arduino.cc/t/running-two-motors-in-a-loop/967922(dc motor control)
• https://docs.google.com/presentation/d/1riWKSLQ8dvxT7iRjQZRR9aIbOMKsuxgG1sxSNWiaHiM/edit#slide=id.g2bd96f4d727_0_45(controlling motors)

P5 Facial/Audio Recognition

• https://itp.nyu.edu/physcomp/labs/labs-serial-communication/two-way-duplex-serial-communication-using-p5js/(p5 serial control we used)
• https://kylemcdonald.github.io/cv-examples/(facial recognition)
• https://github.com/kylemcdonald/AppropriatingNewTechnologies/wiki/Week-2(facial recognition 2)
• https://florianschulz.info/stt/(Speech to text library with google chrome)
• https://processing.org/reference/libraries/sound/AudioIn.html(aduio lib)
• http://learningprocessing.com/examples/chp20/example-20-09-mic-input(mic input)
• https://github.com/p5-serial/p5.serialserver(another p5 serial control)
• https://www.youtube.com/watch?v=MtO1nDoM41Y(arduino to p5)
• https://www.youtube.com/watch?v=MtO1nDoM41Y(two way communication)
• https://www.youtube.com/watch?v=EA3-k9mnLHs(Processing facial recognition)

The Fisherman
    Made by Qiya Huang
    Instructed by Professor Rudi

Context and Significance

In developing our project, “The Fisherman,” we drew upon experiences from previous group projects, particularly our exploration of servo and stepper motors. In the Wolverine Claw project, we used servo motors to create a claw retraction effect, and in the Whack-a-Mole game, we used stepper motors to realize the movement of the mole. Inspired by this, we opted to use a stepper motor to rotate a wooden stick, enabling a unique mechanism for simulating the fish movements. This decision added more fun to the dynamic interaction between users and our fishing game. Additionally, our past experiments with sensors informed our choice of detection mechanism. While we initially considered force sensors, we ultimately opted for reed switches due to their compatibility with our magnetic fishing mechanism and their reliability in detecting fish captures accurately.

Conception and Design

Drawing from the principles outlined in “The Design of Everyday Things” by Don Norman, our design philosophy centered on prioritizing user feedback and interaction clarity. We aimed to create an intuitive and engaging gaming experience from the moment users interacted with our project. Incorporating a start button provided clear feedback on game initiation, setting the stage for user engagement right from the start. To further enhance user engagement and enjoyment, we integrated positive feedback cues such as blinking green lights and buzzer sounds upon successful catches. These feedback mechanisms not only rewarded users for their actions but also reinforced their engagement with the game. Additionally, we introduced a winning condition that was signaled by all red LEDs blinking, providing a clear indication of game success. This feedback loop not only informed users of their progress but also incentivized continued gameplay.

In terms of materials used, while our initial prototype relied heavily on cardboard, practical testing revealed issues with its rough texture, particularly concerning the gears. The rough surface led to frequent snags, disrupting the smooth operation of the gear mechanism and interrupting gameplay flow. Reflecting on these challenges, we recognized the need for a smoother surface to ensure the seamless operation of the gears. In hindsight, utilizing a different, smoother material or applying packing tape to cover the surfaces could have effectively mitigated these challenges and improved the overall gameplay experience.

Fabrication and Production

 

 

 

 

 

 

 

 

My primary responsibility was to address motion instability. To achieve smooth and reliable motion for the fish, I focused on utilizing gears to implement different up and down motions using a single motor. Initially, we encountered issues with the main wooden stick twisting during rotation, which posed safety concerns and jeopardized the durability of the mechanism. To mitigate this, we replaced the main stick with a sturdier, round wooden stick recommended by one of the IMA professors.

In terms of LED circuitry, we aimed to create the effect of six red LEDs blinking simultaneously using only one pin, minimizing complexity and conserving Arduino board resources. Researching alternative connection methods, I discovered the technique of parallel connection, which allowed all LEDs to be connected in the breadboard first before being linked to a single pin. Implementing this method not only streamlined the circuitry but also achieved the desired blinking effect efficiently.

Furthermore, I integrated the game mode button to add an element of fun and interactivity to the gameplay experience. Programming the button required careful consideration of game states and timing to ensure seamless transitions between start, play, and end states. I assigned a fixed game duration and a Boolean variable “gameRunning” at the beginning of the program and tracked the game time using the millis() function. By developing two functions, “startGame()” and “endGame(),” I established a logical framework for managing game states, facilitating smooth gameplay transitions, and enhancing the overall user experience. Here is my code snippet for making the game mode system:

void loop() {
    if (digitalRead(buttonPin) == LOW && !gameRunning) {
        startGame();  
    }

    if (gameRunning) {
        elapsedTime = millis() - startTime;
        if (elapsedTime >= gameDuration) {
            endGame(); 
        }
    }
}

void startGame() {
    gameRunning = true;  
    startTime = millis();  
    Serial.println("Game started!");
    
    if(fishAmount > 2){
      Serial.println("You win!");
    } 
}

void endGame() {
  gameRunning = false;  
  fishAmount = 0;
  fishCaught = false;
  Serial.println("Game ended!");
}

Additionally, feedback from the user testing session prompted a shift in theme from an abrupt “overfishing” theme to one that emphasized the importance of marine ecosystems and diversity in fishing practices. By incorporating trash and other marine creatures into the setting, we introduced new challenges and educational opportunities for players. Distinguishing trash with wool further enhanced gameplay dynamics, fostering a deeper understanding of environmental conservation and sustainable fishing practices among players.

Conclusion

Our project, “The Fisherman,” aimed to create an engaging and interactive gaming experience centered around the act of fishing. By integrating elements such as motion mechanics, LED feedback, and thematic storytelling, our goal was to immerse players in a dynamic and educational gameplay environment. The project results align closely with our definition of interaction, as we successfully designed a system where users could engage with the game through physical actions such as pressing buttons, controlling the fishing rod, and receiving feedback through LED indicators and sound cues. However, while the physical interactive elements were well executed, we could have incorporated more dynamic storytelling elements to further increase player engagement and immersion.

Ultimately, audiences interacted positively with our program, expressing enthusiasm for the game mechanics and thematic elements. Players were actively engaged in the fishing experience, and many appreciated the educational elements incorporated into the game.

Disassembly

Appendix

• The Gear Mechanism Inspiration: https://www.youtube.com/watch?v=IpBA7emMpb8
• Midterm Project Images and Videos Documentation Folder: https://drive.google.com/drive/folders/1GJ3YDy69793Hz4ww8Q-d0nVbd0aemKSZ?usp=sharing
• Arduino Code: https://drive.google.com/file/d/10C-SJFT82JP-vV-YfXmioY7sXw5v2Zsl/view?usp=sharing