Curious Cube–TwinkleTone

Melanie Quan & Jean Zhang

Instructor: Yeseul Song

Concept and Design

The curious cube is designed in the form of a music box. The design of the concept intends to present a novelty octave box with a visual effect of traditional folding fan. At the same time, the audience can interact and control the on/off and sound level of the octave through intangible interaction of changing distance of approaching.

The concept is about the storytelling music box and focusing on giving the audience the sense of control with the box through intangible interaction. I hope that the audience can experience the beauty of the art of folding fans in the interaction, and at the same time find special sense of control from the interaction with the music box.

Fabrication and Production

Box Structure

For the fabrication, the box structure is laser cut with wooden board. With the servo motor inside the box, every time the box is activated, the servo motor will push up the lid to open it. The motor is mounted on the back plate with machine screws and the rotating handle of the motor is extended with chopsticks. The front side is laser cut to give the Time of Flight sensor a place.

For the paper fan folding, I refer to the instruction online (https://www.bilibili.com/video/BV1YS4y1g7p7?vd_source=40c899b1eceb837c10a11f2e65e4d0b3). In terms of research on materials, I tried with regular A3 paper which is too thick, rice paper too soft, and vellum too easy to break, and finally decided to use thinner version of regular paper.

Coding and Serial Communications

For the coding on the Arduino side, it includes function such as ToF sensor sensing, servo motor movement, loop interval for the structure with cases, LED blinking, and serial communication. On the TouchDesigner side, by receiving the data from serial communication, the audio file and out are referring to the chops that map the data.

Conclusion and Future Development

For future development, I am considering adding visuals to the fan part with projection or playing with the shadow.

Another development idea is about the storytelling. I often think back to my grandmother’s voice telling me stories in Shanghainese dialect when I was little. The sound input in TouchDesigner can be replaced by the recording of storytelling. The shape of the box can be transformed into a thinner structure, while the angle at which the paper folding opens can be changed from 90 degrees to 180 degrees, which would make this box more like an audio storybook. At the same time, the softening of the voice as the audience interacts with it (approaching) can also simulate the feeling of a storyteller whispering in the ear.

Video Documentation

Full version

In-class play-testing

Version without sounds/serial communication

Group: Jean & Melanie

Concept:

• Black box theatre/Music box
• Curious about what to see when opening the box up
• Beauty of Chinese folding fan arts

Interactions:

• Sensor: Time of Flight sensor–Detecting human approaching and open the box
• Response: Music box opening (in a form of jewelry box) every time human approaches, and closing after certain amount of time; Music plays every time the box opens

(working process)

Visuals:

• LED light–blinking in the box
• Paper fan showing when the box opens
• Shadows from the lights and reflecting on the paper shield
• Possible visual types:

*Paper Folding Reference: https://www.bilibili.com/video/BV1YS4y1g7p7?vd_source=40c899b1eceb837c10a11f2e65e4d0b3*

Box Mechanics: Open & Close–servo motor

(Prototype)

Project Planning

Group Members: Jean, Melanie

Sensor Description

Adafruit APDS9960 as described on its website, is a compact, feature-packed sensor breakout board that integrates proximity sensing, ambient light sensing, RGB color detection, and gesture recognition into a single module. It can detect simple gestures (left to right, right to left, up to down, down to up are currently supported), return the amount of red, blue, green, and clear light, or return how close an object is to the front of the sensor. Designed for ease of use, this sensor connects to microcontrollers via an I2C interface, making it simple to wire up and integrate into projects. 

The sensor from Avago Technologies uses an integrated IR LED and four directional photodiodes to detect reflected IR energy. These photodiodes are strategically placed to capture changes in light reflectance from different directions, allowing the sensor to interpret gestures and measure proximity. For RGB and ambient light sensing, the sensor analyzes the intensity of red, green, blue, and clear light using its built-in color detection system.

Here’s a video of the sensor in action:

Detection Attributes

Proximity Detection

Sensing angle is approximately ±30°.

Gesture Detection

The sensing angle is wider to capture the movements, but the detection is based on the position change relative to the 90° directly above.

RGB and Light Detection

The sensor has a wide sensing angle of about almost all ranges of ±90°, but the data change unstably and will change depending on the condition of the light.

RGB and Light Detection

The sensor has a wide sensing angle of about almost all ranges of ±90°, but the data change unstably and will change depending on the condition of the light.

Min/Max Values

Gesture Detection

Effective range for gesture detection is about 1-20cm, and the detection accuracy is best at the distance of 5-10 cm. Movements can be detected in areas that are closer (1-5 cm) or further away (15-20), but accuracy of detection drops dramatically. 

Here’s a video of being too far from the sensor:

Here’s a video of being too close to the sensor:

RGB and Light Detection

There is no specific distance limitation, but it appears the proximity of an object can affect the range and intensity of light that can be detected, resulting in inaccurate detection.

Proximity Detection

Max distance the sensor can detect is about 20 cm, and any distance farther than 20 will send a number 0. Minimum distance is about 2-3 cm, and the value stays at 255 even as I move closer from 2 cm.

Strength/Weakness

• The sensor is compact and integrates multiple sensing capabilities in one module (gesture, proximity, ambient light, and RGB color sensing), which make it easier if multiple combined sensing are required, and in an energy-efficient way. 
• The function of detecting hand gestures without physical contact can be an interesting and useful intangible interaction to use. In the situations of slow simple movements, the sensing is relatively accurate. However, if the movement is fast and multiple repetition of movements or complex gestures will make it difficult to detect. 
• As the gesture detection and proximity are infrared-based, the sensor works well even in low-light conditions. But for the color and light detection, the sensor is not accurate and is too sensitive to the change in the light intensity. 
• The detection range and sensing angle could be enough for small-scale projects, but it would be too narrow for bigger scale projects.

Existing Application/Inspiration 

Meta is currently developing hand gesture recognition watch to replace mouse, it is possible that it requires proximity sensors to detect gestures like swiping right, left, up or down.

Smartphones black-screening when held near the ear is also an application of such sensors. Also, some clocking-in mechanisms through actions can be used with simple gesture sensing through the sensor. 

Members: Jean Zhang, Melanie Quan, Sherry Gao                              

Intangible Interactive System:  Paper Towel Dispenser

I Observation:  

Analysis through Implicit Interaction Framework:

II Research:

For Research Question 1

A. Closer look at the system:

• Research Objects:

1. Several inductive paper towel dispensers in the women restroom in 6 MetroTech Centre/370 Jay St (from enMotion)
2. Other types of paper towel dispensers in NYU (Manual; Fully automatic paper dispenser in Stern)

• Findings:

1. The machine will react instantly when users put their hands less than about 5 centimetres away ahead of the black sensor (circled in the figure), no matter still or wave in a motion. The sensor will spark red and then dispense one sheet of paper. 
2. Staying still or waving at a different position other than the sensor part cannot be triggered.
   1. When there is already one sheet dispensed, the machine will not react no matter what the user does, to reduce waste of paper. Also, if the current sheet is torn off quickly, then the user stays more than 5 centimetres away, the sensor will only spark red for 1-2 seconds but not dispense. 
   2. Movements at different positions cannot be triggered: 
   3. Other possible situation
   • Use something other than hands to reach the sensing area: 
   • Try to pick up more pieces before the previous one being dispensed: 

    

   1. Different types of paper dispensers in NYU campus
   • Manual pickup
   • Automatic paper dispenser (Location: Stern-Kaufman Center) Video

   Finding: The machine only reacts when the currently dispensed paper is torn off, and no matter how fast the user tears off the paper, it will dispense another sheet of paper.

    

   • Most Common Errors:
   1. Sensor: The sensors are not always sensitive and finding the right sensing area is tricky 
   2. In both situations of inductive paper towel dispenser and automatic version, an extra sheet of paper dispensed which is exposed to the air before usage and can cause waste as users may tend to tear off a new sheet
   3. Some machines stop working around 5:30 p.m. (at 370 Jay St) for unknown reasons.

B. online research:

An automatic paper towel dispenser includes a dispensing mechanism configured to dispense a paper towel, a light detection and ranging (LIDAR) sensor, and a controller in communication with the LIDAR sensor. The LIDAR sensor is configured to emit light toward an object within a detection region at a first time, detect the light reflected by the object a second time, and calculate a time of flight of the light. The time of flight is an amount of time that elapses between the first time and the second time. The controller is configured to calculate a distance between the LIDAR sensor and the object based on the time of flight and actuate the dispensing mechanism based on the calculated distance.

Citation: https://patents.google.com/patent/US10791884B2/en

Existing paper towel dispensers typically use two main hand wave sensor technologies: infrared reflectance (IR) and capacitive field.

Sensor 1:

IR sensors measure the amount and characteristics of light reflected back to the sensor from the user’s hand. Infrared reflectance is a technique that uses infrared light to analyze the composition and orientation of molecules on a surface.

How it works 

• Infrared light is shone on a sample
• Some of the light is absorbed by the sample, and some is reflected off the surface
• The reflected light is detected

Sensor 2:

A capacitive field is an electrostatic field created by a capacitive sensor. Capacitive sensors detect changes in the capacitance of a material, which is the amount of charge the material can hold. 

How it works 

• The sensor has a metal plate that’s electrically connected to an oscillator circuit. 
• The target to be sensed acts as the second plate of the capacitor. 
• As the target approaches the sensor, the oscillations increase until they reach a threshold level. 
• The sensor switches states and gives an output when the threshold amount of capacitance is reached.  

Applications of capacitive sensors

• Capacitive touchscreens: Use the electrical properties of the human body to detect touch 
• Proximity sensors: Detect objects like liquids, glass, wood, and paper 
• Temperature sensors: Detect the temperature of liquids and solids, even in extreme temperatures 
• Perimeter intrusion systems: Detect when an intruder moves in a capacitive field 

Additional Sensor:
A miniature snap-action switch, also trademarked and frequently known as a micro switch or microswitch, is an electric switch that is actuated by very little physical force, through the use of a tipping-point mechanism, sometimes called an “over-center” mechanism.

There is a miniature snap action switch in the machine that restricts the dispensing of the next sheet of paper until the previous sheet has been taken. The micro switch will be clicked when the user pulls the paper down, which sends the signal to the machine that the paper is dispensed. 

How does a Micro Switch work? https://youtu.be/mcvTaJocwiI?si=8PqxT0TOxn7y7F-5
→Part III Idea 5–Hacking the machine!

III More Ideas:

1. If a machine uses IR sensor, would it be discriminating for it is harder for people with darker skin to trigger the sensor? 
2. Is it better to use a rotating towel? https://huebsch-services.com/cloth-roll-towels/
   /fully automatic dispenser? 
3. Further implication: Disney water harp (fun, Interactive, educational): https://www.youtube.com/watch?v=q6KtcLTd4G4 
4. Is there an alternative button/sensor to get the machine to work when it gets stuck or not sensing?
5. Is there a better way if someone wants to get a bunch of paper towels?(Not in the way of hacking like the video shows: https://youtube.com/shorts/Ki_nvj0I6Es?si=cCSuviJG9oAZ5-oN)                                                                                                               Probable implication: another analog sensor to let the user decide the length; Change in the use of micro switch