mAGICAL STORYTELLING

Designers

• KNS / Kruthika NS)
• Zi Chen
• Una Zhang

Weekly Updates

9/12/24 – Before Designing

Questions to Consider:

1. Could a blind person use and understand my project?
2. Could a Deaf person use and understand my project?
3. Could a Deafblind person use and understand my project?

Assignment #2: 9/19/24 – Project idea

Our project is to create a “magical” book. We want to focus on how a book may be experienced when sight is not the only dominating modality. We want to explore how to use sensory elements to tell and push forward a narrative. 

Initial proposed sketch

 

Assignment #3: 9/26/24 – project description

Before week 4, we met again as a group to discuss how multisensory a book can be “read”.

This time, we wanted to go in the direction of a rotating book, as it would also open up the possibility of adding 3D elements to facilitate audio, smell and tactile components to be fit in the final iteration. We also chose this route since the action of rotation is similar to the natural gesture people use when flipping books. 

Project Description

• A problem statement
  • A reader needs to view+hear+smell+touch the pages of the book, so that they can viscerally immerse themselves in the narrative.
• The goal of the interface
  • create a multisensory reading experience that is engaging, impactful, contextual, and sensorially appealing
• What components will be related to that goal
  • textured illustrations
  • scratch to smell / similar smell-related components
  • sound during the page turn / related to the page narrative
  • moving illustrations — paper engineering (analog) or haptic responses (electric)
  • depending on the narrative, perhaps a taste element
• Which sensory elements are present
  • smell
  • haptic
  • auditory
  • visual
  • possibly taste
• What the user does
  • flipping through the interactive story book, experience and interact with the book
• What the interfaces does in response
  • vibrates
  • temperature changes
  • dispenses smell
  • image moves
  • audio plays
  • possibly offers taste

How can we reimagine what a story would be like with multiple sensory modals? 

We thought about making a book a 3-dimensional and rotating book. We shall rethink the “cover page” by adding a specific starting point to the rotating system so the reader knows where to begin. Eg: Having a textured beginning and end. We also have pumps for smell and magnetic switch mechanisms to change audio at the turn. By taking the focus off visuals, we shall turn to the tactile to complement the text/braille with textures or fabricated sculptures on the page. We will also use tactile illustration using thermal pens.

 

Assignment #4: Week 5: 10/3/24 – Rapid prototype

We built a simple prototype to test out the concept. 

 

 

Assignment #5: 10/10/24 – user research objectives

Preparing a user testable prototype

Before we began our user testing process, we decided to make a larger prototype so the users could experience the “book” in a size that would be closer to the proposed final iteration. Before we designed this, we needed a short evocative story to help get feedback. We chose “The Flowering Tree”, a folktale passed on by word of mouth by the women of Karnataka, as we found it evocative and sensorially rich. The story also has themes of ecofeminism, which appealed to us.

For the prototype, we did not have the time to iterate on how best to tell the story as we were focused on making the prototype of the book’s rotating structure. Hence, we kept the tactile “illustrations” very abstract on each page. We stuck different textiles and objects with glue onto the foamboard pages and decided to spray a grass-like scent when the viewer was on the specific page which described grass. The story was recorded by one of our teammates orally on a phone recorder, and split into 4 different scenes corresponding to 4 different page spreads.

In our final iteration, we wanted the audio to play automatically based on which page spread faced the viewer as they rotated it. Similarly, a scent for the corresponding page would need to automatically be sprayed. However, for this round of user testing, we manually changed the audio, and sprayed the scent.

 

 

Hypothesis

If we create a 3D turning book with textures and audios, users will be able to interact with it intuitively and the combination of different sensory modalities will enhance their engagement and experience of the story. 

Research Goals

1. To how users understand and interact with a vertical rotating story book 
2. Do they know where to start and end the story? 
3. To explore how multisensory modalities (touch & smell) affect user’s experience with a story

Methodology

Number of participants: 5
Location: New York
Age range: 20-30
Experience: NA

Procedure

Data collection: Tasks, Thinking aloud, User Observation, Structured Interview 
Length: 15 minutes of interaction, allowing users to chat about their experience while they interact
Location: 370 Jay St

Testing Script

Informed Consent

Hi, my name is Una/KNS/Zi. Today we want to learn how users might understand and interact with this. 

Please let me know if you’d like a break, have any access requirements, or need any accommodations. There are no right or wrong answers and we can stop at any point. 

 

Do I have your permission to video and audio record? Quotes will be anonymized and all data will only be shared internally with my team, stored on a secure, password-protected cloud server, and will be deleted after the study is completed. 

 

Tasks

I’m going to ask you to perform a series of tasks and to think out loud as you go. Please describe your experience and what you’re thinking as you complete the task. I promise it won’t hurt my feelings because it’ll help us to improve the project.

 

Task 1: Rotation as a new way to flip through 

• Objective: Evaluate how users interact with the physical rotation of the book.
• Questions: 
  • Please play with the book to explore the different vertical pages and describe what feels natural or difficult about rotating the book.
  • What do you think the extra texture around the one page signifies? 
  • How did the rotation feel for you? 

Task 2: Interacting with Tactile Features

• Objective: Assess how users engage with the tactile elements on the vertical pages.
• Questions: 
  • Feel the surface of the pages and describe how the tactile elements contribute to your understanding of the story.
  • How does touching the page affect your experience of the story?
  • Did you find any textures more engaging or confusing?

Task 3: Multisensory Engagement (Touch and Audio)

• Objective: Understand how users respond to the combination of touch and auditory cues.
• Instruction: “Touch different parts of the pages while listening to the corresponding audio and describe how these two senses work together to shape your experience.”
  • Follow-up Questions:
    • Did the sound enhance or detract from your experience?
    • How did the combination of touch and sound affect your engagement with the content?

 

Follow-up Questions

 

1. From your perspective, what went well here?
2. From your perspective, what didn’t go well here?
3. Please describe any areas of the interface you had questions about
4. What do you like about reading stories?

 

Thank you

Thank you so much for helping me out today. This will really help me improve the interface. Is there anything else you’d like to ask questions about?

Interview response summary

Interviewee 1:

Age: 23

Gender: F

Experience: 

 

Notes: 

• intuitively turning it both ways
• like a lot of material, ask if gear is part of it, organic, natural; 
• because of the way the pieces line up, user feels like there should be a specific way to turn; 
• purple part is the beginning of the new cycle
• like cotton, like soft material, or organic material, didn’t want to touch the dead leaf

Interviewee 2: 

• Why different? Reminds her of a Puzzle, and explores the rotation. 
• She likes clouds and hair, squishy feeling dead leave remind her of dragon fly; how color informed; 
• elements to trigger smell 
• How audio is playing: help understand expectation of user, how long to stay on one page 
• Need more interaction 

 

Interviewee 3: 

• Interaction piece but not inviting to do a certain way, 
• look touchable but fragile
• Not inviting to go closer and sniff any smell 
• Unclear how things are connected together. 
• “In books, the text guides people to where to look” but this is not as much

 

Interviewee 4: 

• Where are we in the story, not sure where to look?  Where is this person in the story What point of view? 
• The form of the book indicates it’s a story for many people to experience at once 360
• Visual jigsaw, guides people to 
• Touching wise: Better to go under, weird twist of the wrist
• direction of beginning and closing the story? 
• Could respond to the touch

 

Interviewee 5: 

• the form makes it inviting for four people to experience. 
• What if to eliminate turning, able to focus more on the texture and touching. 
• Horizontal would be harder to rotate
• card flipping display
• The textures are may not look as inviting. What if to take away the element of flipping? Needs more interaction 

 

Summary: 

• Users gets how audio and the turn correspond to each other 
• Need to time audio and tactile interactions
• Touching different parts as a way to activate response? Vibration, sounds, smell?
• The first sentence of the audio, needs something specific that people can pick up. A
• Hard to predict what people do with a rotating book, may try to spin it as fast as they can. 
• Visual readers will judge the tactile based on visual information. How to mitigate that? Paint all white? 
• Hard to touch things, flip, and listen to the story at the same time
• Headphone is a very intimate act though, people need to commit to it. 

 

Assignment #6: 10/17/24: User feedback findings & analysis

Usability tests summary: 

• Users gets how audio and the turn correspond to each other 
• Need to time audio and tactile interactions
• Touching different parts as a way to activate response? Vibration, sounds, smell?
• The first sentence of the audio, needs something specific that people can pick up. A
• Hard to predict what people do with a rotating book, may try to spin it as fast as they can. 
• Visual readers will judge the tactile based on visual information. How to mitigate that? Paint all white? 
• Hard to touch things, flip, and listen to the story at the same time
• Headphone is a very intimate act though, people need to commit to it.

We met as a group and also observed that the 4 page spread was very limiting for such a rich story, and a rotating mechanism only provided us 360 degrees to play with. Moreover, the key of this class — to explore multisensory design — was being taken over by our need to create a rotating mechanism. 

We also decided that the script would need to be reworked in a way that there were shorter paragraphs for the reader to take in at a time. This would mean that the story had to be split into multiple rotating mechanisms if it got longer.

We weighed our pros and cons, and hence decided to abandon our first prototype idea of a rotating mechanism. We decided to explore a storytelling “experience” rather than a “book” with multisensory elements instead, and retail the magic through the experience that is a story, rather than the object that is a book.

5 W’s Chart

Source: Velasco, Carlos and Marianna Obrist. Multisensory Experiences. 2020.

Components	Questions and considerations
Background (Why)	Why do we want to design this interface?
Impressions (What)	What impressions do we want the user to feel when interacting with the interface?
Events (When)	What is the user’s journey?
Sensory Elements (How)	What sensory modalities should we select and why? Are there trade-offs? Who might be excluded?
Designer/User (Who/Whom)	Who is the designer? For who are we designing?

Assignment #7: 10/24/24 – iterating on the prototype

 

Assignment #8: 10/31/24 – List of material

Item	Qty
Raspberry Pi Pico RP2040	14
S8050 NPN to-92	28
Breadboard 400-Holes	14
433Mhz RX Module	14
3V Pump	8
5mm LED	15
Vibration Motor	5
3.7V Lithium battery	14
6x6mm Push Button	25

System Diagram

Code

Prototype II

Choosing background color and testing them on different boards. We ended up choosing the black board. 

More tactile elements

More tactile elements

Using Playdough to clean up rangoli. 

Hiding Speaker and making holes for sounds to go through.

 

user testing II

We tested our prototype with built in scent and audio with Georgina. We got feedback on how to guide users through our journey and decided to add an QR code for screen reader users. 

 

Final Presentation

Team at Tandon MakerSpace Prototyping Fund Showcase

Link to Canva file used for Prototyping Fund Showcase:

https://www.canva.com/design/DAGYGdzDqso/G5vYrNphuaiartFspTvaJQ/edit

Link to Final Video: 

https://drive.google.com/file/d/1UYFttzuojG_dKYBztwm0HfcUahzWzxeQ/view?usp=drive_link