Part1: For My Readers
I am Ruiqi Liu, the project “I AM A SNAIL” I made is an interactive picture book, with childlike illustrations.
The story is presented as an interactive cartoon picture book with some simple games inserted. Users can use the mouse and keyboard to control the snail to move around through different frames and scenes and experience its story from the snail’s point of view.
Abstract:
Nowadays, people pursue a fast pace of life but forget about slowing down to enjoy and appreciate everything around them. When mentioning a snail, the first thing that comes to people’s minds is their low speed, thus, this is the first reason that I chose the snail as my topic. In the project, the user will listen to a little snail telling its story, with a cute kid’s voice narrative.
To interact, the user will click on the mouse initially on the canvas to move the snails around. By following the sign or moving to a certain position, the user will be able to move the snail to the next scene. The users could view the entire story when they get through all 9 scenes. To make it more visually appealing, I made it look like a comic book style, with many frames appearing at the same time.
For the 9 scenes, I drew all the pictures that appear on the canvas, such as the snail, the scene, and all the characters. There are different kinds of interactions, including simply going to a certain position to trigger some reactions, games such as using the left and right keyboards to move the snails, avoiding being stepped on, etc. Besides, by acting AI-generated narrative and sounds to imitate the sound of nature, the users could wear the headphone to feel like immersing in nature.
Part2: For Myself
Process: Design and Composition
1. The Style of Comic Book
Initially, I wanted to make a story of a snail, and the users could go to different scenes by interacting with my project, but I could not decide exactly how to illustrate the story. Professor Marcela suggested placing them in different frames of a comic book.
Thus, I drew a rough storyboard for my project.
2. Drawings and Paintings
Initially, I decided to use sources from the internet (the first demo sketch I created), and all the pics are found on the internet.
Prof. Marcela suggests I draw the pictures in the simplest drawing style. Thus, I decided to draw all the characters and backgrounds on Procreate myself.
3. Local Storage
When the snail leaves a canvas, I wanted it to appear on another canvas. To make these sketches communicate, Professor. Moon suggests I use local storage for p5, such as storeItem() and getItem() , so that I can store an item in the local storage of the p5 editor and use them in different p5 sketches.
(For example, when the snail falls into the hole on the first canvas, I set a boolean variable to send it to the second scene, and only if that boolean variable is true, the second scene starts)
4. “function MousePressed()” OR “if (KeyIspressed)“?
At first, I decided to make the snail move by pressing the left and right arrow keys throughout the whole story. (Like the video below)
Because I wrote many p5 sketches and each scene is one, following the suggestion by Professor Moon, I used iframe to place different p5 sketches on the same HTML page. However, if I use keyIsPressed() for movement, the user always has to click on the new scene when a new scene is triggered/highlighted. I thought it was not a big deal initially, so I went to professors after I almost finished all the scenes. However, Professor. Moon told me it is a rule for highlighting web pages, which cannot be solved.
If I am using keys as my major interaction way, the user needs to click the mouse and then move the hand back to the keyboard for each scene change. My initial goal is to make the comic book demonstrate smoothly, so this would definitely make the experience feels lagging. Meanwhile, always using keyboards makes my project could only be played on mobile devices, so by having meetings with professors, I decided to use mouse interactions.
5. AI Narrator
To add the poem I wrote to the sketch, I want to add some voice narrative. However, it would be better if the sound is a child’s voice. Thus, I used AI-generated voice.
The AI sound generator website:
https://www.narakeet.com/app/text-to-audio/?projectId=e6f84990-09ed-46ac-803f-3cd6133e2422
Process: Technical
1. Mouse interactions aren’t smooth
Before the interaction day, the way I moved the snail is that whenever the mouse is pressed, the function mousePressed() is being called and the snail would start moving for 1 second. The users are supposed to be clicking the screen only if the snail stops moving. However, during the interaction day, I found that most of the users are clicking the mouse a lot of time and this would slow down the snail. Thus, I added a boolean variable to make the function mousePressed() only being called if the snail stops moving and it worked.
2. “noLoop()“ AND “filter(GRAY)“
To make my project more visually appealing, and indicate which scene is currently highlighted, professors suggest I use noLoop() and filter(GRAY)for the scene without the snail. However, filter(GRAY) every scene may be too much computation for the computer as I had a lot of images, and the sketch went slow. Thus, I tried if conditions with noLoop(), but as it stops looping, it never goes back to looping. So I had to give up the process of turning colorful when a certain scene is highlighted, but the scene will still turn gray and stop looping once the snail leaves it.
3. Flexbox Display
To place the canvases on the page, I used iframes as different classes. In the CSS style file, I used positions initially, and manually set the position with pixels by trial and error. However, the display changes when using different devices. Thus, instead of using positions, I used Flexbox.
4. Local Storage is not Stable
By storing local values in different sketches, and loading them in a main HTML page, the sequence of the sketches being loaded is random. Thus, they if(startScene == true) are sometimes not working when loading the page for the first time. Thus, Professor. Moon told me to add a p5 sketch on the main page for only loading the values, and it makes the sketch work.
Reflection and Future Development
I like the idea of the snail moving through different canvases, the storytelling part, and the pictures I drew. With the help of Professor. Marcela and Moon, the feedback from interaction day, and my observations, I worked a lot on how to make the user experience more smooth.
During the interaction day, many people told me that the transitions between mouse and key in scene 4 make them feel weird. However, this is the only way I found out that could solve the problem of highlighting the iframe without letting users click on the frame.
Things to improve:
1. During the IMA show, I received feedback that it is not that clear for some users that sometimes they need to move left and sometimes right. Thus, I think I need to work harder on improving the user experience.
2. Besides, personally, I am not that satisfied with the layout of my project, and the frame sequence on other’s computers is sometimes not the same as mine. I should work on that.
3. I would work harder on the layout and the style of my project web page.
4. When loading the source on another computer, the sound and p5 sketch (especially when getting to scene 7), slows down. Maybe I will find out a way to make it works on everyone else’s computer.
(Thanks for the enormous help from Professor Marcela and Moon and the code samples from them!!!)