A.PROJECT TITLE – YOUR NAME – YOUR INSTRUCTOR’S NAME
Re-walk the Journey of Little Prince-Jingyi Ma-Gottfried Haider
B.CONCEPTION AND DESIGN:
Our project’s concept is a “storytelling” experience by combining Little Prince’s journey in the novel with the visual and audio interaction with the users. Users will put the “Little Prince” on different planets, and the magnet sensor on Little Prince’s foot will trigger off the magnet sensor and switch the corresponding LED strip on. Meanwhile, the computer screen will play music and sound, showing what Little Prince has encountered on that planet. My definition of “interaction” is “The activity between two or more entities, with input, processing, output, feedback, and consideration.” So a single “showing” cannot be interaction, interaction needs mutual feedback. In our brainstorming period, we thought of making a Little Prince themed music box with sound visualization, but this did not provide any input from users, so we abandoned this opinion. After re-considering the concept of interaction mentioned above, we implemented the concept of “点读机” or reading machine to our final project. Reading machine is an old-style electronic common in our childhood in China. This electronic can play music or sound if you put a particular book on it and use an electronic pen to tickle different areas. We think this is a very good example of interaction in daily life, and we decided to combine it with our project. We use magnet sensors as input signals and using sound and lights as output, a very simple but effective way of interaction. During the user testing session, the biggest problem was we lacked instructions to make the users tap the magnet on the sensor. Also, during that time, we lacked corresponding audio sounds, and Testors would like to have more sound effects rather than visual words. To make up these bugs, I used an A4 size hot-shrink sheet and marker to make a chess of Little Prince, stuck paper stars on his foot with a magnet, and put other stars around the magnet sensor to guide the users. Also, we used AI-generated voices (Narakeet) to read out the conversation of individuals. The AI voices worked well in the presentations and documents, but they didn’t reach our desired effect in the IMA show, for it was too noisy and it was hard for people to focus on the sound.
C.FABRICATION AND PRODUCTION:
4/21:In the beginning, we thought of more keyboard and mouse interaction with the screen.
The professor suggested we use sequencers at first to make a “music game”, but we found that it’s not so good when implemented with other sounds as birds of planes, so we abandoned this plan. I built a prototype in Processing with various planets being shown at the same time. However, after discussing it with professor, he thought that it was more important to have more physical interaction, so we abandoned this plan. The first type of production is to build the prototype for our project. We originally wanted to make an interactive music box, and we went to a musical box shop but it’s too hard to build so we changed it to the form of a book.
(The original design was more like a music box.)
4/23: We decided to use laser-cutting on 3mm wood to make the effect of a book. We used the cuttle.XYZ webpage to create SVG, and we combined the SVG of Little Prince found on the internet and some emoji from Cuttle to make the graph of each planet. I used cardboard to test the effect of Cuttle, but the first prototype sucked, for I misunderstood the relationship of “stroke” and “filling”, so some of the parts are engraved, while other parts are not cut through. After asking Professor Andy, we figured out how to change the “engrave” into “strokes”, by “create path” function. For our final woodcut, helped by our friend, the LA Sophia, we made our wooden board successfully.
4/24:Our friend Astroyd and Melaine’s Interaction Lab project last semester, “Portion Class”, inspired us greatly. Their project does not use LED strips to show sound visualization but makes it into a light source. We decided to make the whole lighting of the project with an LED strip. The project has a complex “landscape”, so we had to cut the LED strip into smaller parts.
We used the thinnest wire we could find in the IMA studio, but they were too thin so Jiaqi had to use nails to strip them by hand. We soldered each part of the LED strip, six points per strip, and it was a dizzy job. 
After lighting up the LED strip, we found that some parts were not lighting the way we want-very frustrating. It turned out that some parts were, actually, shorted because of the wire attached to each other. So we used a glue gun to block these parts and stopped the short circuits.
The tester effect looks good. 
4/25:After the LED strip works well, we want to implement sensors in the project. We agreed that magnet sensors are the most reliable ones, for they won’t be disturbed by sound and light in the show. Our original intention of interaction is to make a “magic pen” or something else. We found that the problem of failure in adding the magnet sensor is due to 1) We forgot to connect the magnet sensor with resistors, so the digital value was not stable. 2) no variables were storing the initial value of the sensor 0 and current value 1. After fixing this part, our project can turn on the LED strip when the magnet sensor is triggered. This is okay for user testing, but only for the Arduio part.
(The Arduino part was okay at user testing.)
4/27: After the user testing, we decided to make Serial Communication on Processing and also a figure for Little Prince to make his journey more “real”. I printed a graph from the internet, drew its sketch on hot-shrink sheet and Jiaqi colored it. I used the oven to make this shrink to a size comfortably held by hand. We used two magnets on that “Little Prince Chess” as they will stick to each other automatically.
4/30: As I was stuck in the coding with serial communication, Jiaqi and I communicated with Professor Gottfried and figured out what was wrong with our Serial Communication part. It turned out that we forgot to write the arrays storing Arduino values. After fixing this, we recorded the voices of each planet from the internet open-ended sources, like the sound of coins, birds, and wine. I used the loop() function to play these sounds, but I found that it wouldn’t stop playing once the music was played. After consulting our fellow Kevin, we used the stop() function to fix this problem. We used sound, picture, and text functions in Processing to mimic the effect of storytelling. Finally, during the presentation, we showed the effect of turning the LED on and playing corresponding music and voices at the same time.
coin.loop(); globe.stop(); lamp.stop(); dove.stop(); wine.stop(); clap.stop(); plane.stop(); crown.stop();
5/2: We finished the presentation and things worked well on Arduino and Processing, the LED didn’t go wrong and music was played. My classmates and professors gave us some advice. They suggest making more direct directions on how to trigger the magnet sensors, as well as making more sound interaction and reducing the text. So we use the AI-generated text-to-speech tool Narakeet to generate the dialogue of Little Prince and other people. We found that it’s hard to make the dialogue being spoken separately(will involve millis() and more coding). We merged the audio and only played them once. This problem is solved.
textFont(story, 40); fill(255,255,255); text("-Why are you drinking?",750,200); drinkall.play(); fill(50, 205, 50); text("-So that I may forget.",750,250); fill(255,255,255); text("-Forget what?",750,300); fill(50, 205, 50); text("-Forget that I am ashamed.",750,350); fill(255,255,255); text("-Ashamed of what?",750,400); fill(50, 205, 50); text("-Ashamed of drinking!",750,450);
5/10:
During the IMA show, our project receives no less than 30 users! A lot of them give very good feedback on the visual and text effects. However, as it was too noisy in the studio, though we turned on the biggest volume, the sound effect was disturbed. But in all, people are deeply engaged in our project, and I think it’s a good ending for our semester.
D.CONCLUSIONS:
Our project achieved the goal of “interactive storytelling”. By putting the plastic “chess” of Little Prince on different planets, users can light up different planets and discover what happens between Little Prince and other people on a computer screen. Therefore, they can re-walk Little Prince’s journey and retrieve memories from the novel, reflecting on the relationship of childhood and adulthood, and learn to see the world with their heart. By implementing sound, text, and pictures, as well as the interaction of sensors, we create mutual communication between computers and humans, giving the user an immersive experience of reading a storybook.
If we have more time, I will try to refer to my friends from the communications lab, to improve the text and picture parts shown on the screen, to make a better visual effect. I also consider implementing a speaker to make a bigger voice in noisy situations.
Due to the project being very big, it’s hard to capture the LED lights and the screen clearly at the same time. In our document, we shoot two videos, one is more focused on LED and one is more focused on screen.
We encountered many bugs and troubles in making this project. For example, when designing the model, we thought of many prototypes like a music box. We considered Chapter VIII in Monument Valley, also a design of a box. Unluckily, as it’s too complex to make a box, we abandoned this plan. Also, we initially wanted to use sequencers to make audio interactions, and we spent a lot of time building the sequencer, but after testing, we found its effects were no better than just looping the video. Also, during the documentation, a NullPointerExpectation happened and the whole processing failed, which nearly drove me to tears. After consulting our friend Jingran, we found the necessity to check the setup() and found this is an error of writing an mp3 audio in WAV format. We need to be more careful about details and be more eager to consult others when we can’t figure out the problem by ourselves.
During the second half of this class, I am very glad that I learned to draw objects with code, as well as make screen-mechanism interactions by doing Serial Communications. My coding ability enhanced a lot in this semester, and I’m grateful to my professor, my teammate, and all the fellows, and LAs who helped me in this progress.
E.DISASSEMBLY:
Because we cut the LED strips in this project and glued the sensors on the 3mm wood, we cannot recycle them for other projects. I disassembled the Adruino, resistors, and other cables. We decided to keep the remaining parts of this project as decoration in my home to make full use of it.
APPENDIX
(There were no magnet sensors in Tinkercard. We use light sensors in the graph to replace magnet sensors.)
Arduino Code:
PImage[] pic = new PImage[11]; boolean[] imageVisibility = new boolean[11]; // Array to track image visibility int[] xc = new int[11]; int[] yc = new int[11]; // Array with center positions String[]lines = new String[11]; float angle; boolean click_other = false; PFont title; PFont story; import processing.serial.*; import processing.sound.*; Serial serialPort; int NUM_OF_VALUES_FROM_ARDUINO = 10; int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO]; int prev_arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO]; int AllOn = 0; SoundFile start; SoundFile beginning; SoundFile coin; SoundFile globe; SoundFile lamp; SoundFile dove; SoundFile wine; SoundFile clap; SoundFile plane; SoundFile crown; SoundFile ending; SoundFile flower; SoundFile richall; SoundFile geoall; SoundFile lampall; SoundFile pilot; SoundFile birds; SoundFile drinkall; SoundFile admireall; SoundFile kingall; SoundFile thankyou; void setup() { size(1600, 900); background(25, 25, 60); initializeImages(); printArray(Serial.list()); serialPort = new Serial(this, "/dev/cu.usbmodem21101", 9600); title = loadFont("ArimaMadurai-Bold-100.vlw"); story = loadFont("Klee-Medium-50.vlw"); textFont(title, 60); fill(255); text(" 'All grown-ups were once children...\n but only few of them remember it.'\n\nLet's re-walk the journey of the little prince.\n Our journey starts from planet B612 \nand ends at earth.", 100, 200); start = new SoundFile(this, "start.wav"); globe = new SoundFile(this, "rotatingdiqiuyi-geologist.wav"); coin = new SoundFile(this, "coin-businessman.wav"); lamp = new SoundFile(this, "switch-lightingperson.wav"); dove = new SoundFile(this, "gezi-earth.wav"); wine = new SoundFile(this, "pouralchohol-drunk.wav"); clap = new SoundFile(this, "applauding-vain.wav"); plane = new SoundFile(this, "luoxuanjing-earth.wav"); crown = new SoundFile(this, "drum.mp3"); ending = new SoundFile(this, "ending.wav"); beginning = new SoundFile(this, "beginning.wav"); flower = new SoundFile(this, "Flower.wav"); richall = new SoundFile(this, "richall.mp3"); geoall = new SoundFile(this, "geoall.mp3"); lampall = new SoundFile(this, "lampall.mp3"); drinkall = new SoundFile(this, "drinkall.mp3"); admireall= new SoundFile(this, "admireall.mp3"); kingall= new SoundFile(this, "kingall.mp3"); thankyou = new SoundFile(this, "Thankyou.mp3"); birds = new SoundFile(this, "birds.mp3"); pilot = new SoundFile(this, "pilotall.mp3"); start.loop(); beginning.play(); } int lastImageTriggered = -1; void draw() { getSerialData(); if (arduino_values[0] == 1 && prev_arduino_values[0] == 0) { background(25, 25, 60); textFont(story, 40); fill(255); text("I have a flower,\n whom I water every day.\n I cultivate her, \nand I attend to her.\n Yet I have nothing to show for it.\n It's as though I had been nurturing her for nothing.", 100, 200); flower.play(); textFont(title, 60); fill(255, 250, 205); text("Planet B612", 300, 700); lastImageTriggered = 0; AllOn +=1; } else if (arduino_values[1] == 1 && prev_arduino_values[1] == 0) { background(25, 25, 60); textFont(story, 40); fill(0, 255, 255); text("-Oh, look! Here is an explorer! Where do you come from?", 100, 100); fill(255); geoall.play(); text("-Oh, where I live,it is not very interesting.\n I have three volcanoes. I have also a flower.", 100, 200); fill(0, 255, 255); text("-We do not record flowers,because they are ephemeral.", 100, 300); fill(255); text("-My flower is ephemeral,\nand she has only four thorns to defend herself against the world. \nAnd I have left her on my planet, all alone!", 100, 400); textFont(title, 60); fill(0, 255, 255); text("Planet of Geographer", 500, 750); globe.loop(); coin.stop(); lamp.stop(); dove.stop(); wine.stop(); clap.stop(); plane.stop(); crown.stop(); lastImageTriggered = 1; AllOn +=1; } else if (arduino_values[2] == 1 && prev_arduino_values[2] == 0) { background(25, 25, 60); lastImageTriggered = 2; AllOn +=1; textFont(title, 60); fill(0, 255, 30); text("Planet of Businessman", 600, 750); textFont(story, 40); fill(255, 255, 255); text("-What good does it do you to own the stars?", 100, 450); richall.play(); fill(0, 255, 0); text("-It does me the good of making me rich.",100, 500); fill(255, 255, 255); text("-And what good does it do you to be rich?", 100, 550); fill(0, 255, 0); text("-It makes it possible for me to buy more stars, if any are discovered.",100,600); coin.loop(); globe.stop(); lamp.stop(); dove.stop(); wine.stop(); clap.stop(); plane.stop(); crown.stop(); } else if (arduino_values[3] == 1 && prev_arduino_values[3] == 0) { background(25, 25, 60); lamp.loop(); coin.stop(); globe.stop(); dove.stop(); wine.stop(); clap.stop(); plane.stop(); crown.stop(); lastImageTriggered = 3; AllOn +=1; textFont(title, 60); fill(255, 215, 0); text("Planet of Lamplighter", 300, 100); textFont(story, 40); fill(255, 250, 205); text("-From year to year the planet has turned more rapidly\n and the orders have not been changed!\nThe planet now makes a complete turn every minute,\n and I no longer have a single second for repose.\n Once every minute I have to light my lamp and put it out!", 50, 150); lampall.play(); fill(255, 255, 255); text("-Your planet is so small that three strides will take you all the way around it.\n To be always in the sunshine, you need only walk along rather slowly.\nThat man is the only one of them all whom I could have made my friend.\n But his planet is indeed too small. There is no room on it for two people. . .", 50, 600); } else if (arduino_values[4] == 1 && prev_arduino_values[4] == 0) { background(25, 25, 60); dove.loop(); coin.stop(); globe.stop(); lamp.stop(); wine.stop(); clap.stop(); plane.stop(); crown.stop(); lastImageTriggered = 4; textFont(title, 60); fill(255, 215, 255); text("Birds", 700, 100); textFont(story, 40); fill(255,255,255); text("The king claims he rules over everything he sees,\n but he doesn't control over a single bird.", 200, 300); birds.play(); AllOn +=1; } else if (arduino_values[5] == 1 && prev_arduino_values[5] == 0) { background(25, 25, 60); wine.loop(); coin.stop(); globe.stop(); lamp.stop(); dove.stop(); clap.stop(); plane.stop(); crown.stop(); textFont(title, 60); fill(50, 205, 50); text("Planet of Tippler", 50, 600); textFont(story, 40); fill(255,255,255); text("-Why are you drinking?",750,200); drinkall.play(); fill(50, 205, 50); text("-So that I may forget.",750,250); fill(255,255,255); text("-Forget what?",750,300); fill(50, 205, 50); text("-Forget that I am ashamed.",750,350); fill(255,255,255); text("-Ashamed of what?",750,400); fill(50, 205, 50); text("-Ashamed of drinking!",750,450); lastImageTriggered = 5; AllOn +=1; } else if (arduino_values[6] == 1 && prev_arduino_values[6] == 0) { background(25, 25, 60); clap.loop(); wine.stop(); coin.stop(); globe.stop(); lamp.stop(); dove.stop(); plane.stop(); crown.stop(); textFont(title, 60); fill(255, 105, 180); text("Planet of Conceited Man", 600, 100); textFont(story, 40); text("-Ah! I am about to receive a visit\n from an admirer!",500,200); fill(255,255,255); text("-What does that mean--'admire'?",500,300); admireall.play(); delay(1500); fill(255, 105, 180); text("-To admire means that you regard me as\n the handsomest, the best-dressed, the richest,\n and the most intelligent man on this planet.",500,350); fill(255,255,255); text("-But you are the only man on your planet!",500,600); lastImageTriggered = 6; AllOn +=1; } else if (arduino_values[7] == 1 && prev_arduino_values[7] == 0) { background(25, 25, 60); plane.loop(); clap.stop(); wine.stop(); coin.stop(); globe.stop(); lamp.stop(); dove.stop(); crown.stop(); textFont(title, 60); fill(238, 233, 233); text("The Pilot", 500, 100); textFont(story, 40); text("I have lived a great deal among grown-ups.\n I have seen them intimately, close at hand.\n And that hasn't much improved my opinion of them.",200,200); pilot.play(); lastImageTriggered = 7; AllOn +=1; } else if (arduino_values[8] == 1 && prev_arduino_values[8] == 0) { background(25, 25, 60); crown.loop(); plane.stop(); clap.stop(); wine.stop(); coin.stop(); globe.stop(); lamp.stop(); dove.stop(); textFont(title, 60); fill(160, 32, 240); kingall.play(); text("Planet of King", 500, 100); textFont(story, 40); text("-Ah! Here's a subjet.", 500, 200); fill(255,255,255); text("-May I sit down?", 500, 250); delay(1000); fill(160, 32, 240); text("-I order you to do so.", 500, 300); fill(255,255,255); text("But the little prince was wondering . . . \nThe planet was tiny. \nOver what could this king really rule?",500,350); kingall.play(); lastImageTriggered = 8; AllOn +=1; } else if (arduino_values[9] == 1 && prev_arduino_values[9] == 0) { background(25, 25, 60); start.stop(); crown.stop(); plane.stop(); clap.stop(); wine.stop(); coin.stop(); globe.stop(); lamp.stop(); dove.stop(); ending.play(); lastImageTriggered = 9; textFont(title, 60); fill(0, 191, 255); text("Earth", 500, 100); fill(255,255,200); text("Thank you for re-walking my journey!\nGrowing up is not the problem; forgetting is.", 50, 200); textFont(story, 40); text("And now here is my secret, a very simple secret:\n It is only with the heart that one can see rightly;\n what is essential is invisible to the eye.", 450, 400); AllOn +=1; thankyou.play(); } if (lastImageTriggered == 0) { image(pic[0], xc[0], yc[0]); } else if (lastImageTriggered == 1) { image(pic[1], xc[1], yc[1]); } else if (lastImageTriggered == 2) { image(pic[2], xc[2], yc[2]); } else if (lastImageTriggered == 3) { image(pic[3], xc[3], yc[3]); } else if (lastImageTriggered == 4) { image(pic[4], xc[4], yc[4]); } else if (lastImageTriggered == 5) { image(pic[5], xc[5], yc[5]); } else if (lastImageTriggered == 6) { image(pic[6], xc[6], yc[6]); } else if (lastImageTriggered == 7) { image(pic[7], xc[7], yc[7]); } else if (lastImageTriggered == 8) { image(pic[8], xc[8], yc[8]);//king } else if (lastImageTriggered == 9) { image(pic[9], xc[9], yc[9]); } for (int i=0; i < NUM_OF_VALUES_FROM_ARDUINO; i=i+1) { prev_arduino_values[i] = arduino_values[i]; } } void getSerialData() { //while (serialPort.available() > 0) { String in = serialPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII if (in != null) { print("From Arduino: " + in); String[] serialInArray = split(trim(in), ","); if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) { for (int i=0; i<serialInArray.length; i++) { arduino_values[i] = int(serialInArray[i]); } } } // } }//Arduion to Processing void initializeImages() { pic[0] = loadImage("b612.png"); xc[0]= 1050;//b612 yc[0]= 450;//b612 pic[0].resize(400, 400); pic[1] = loadImage("geo.png"); xc[1]= 1100; yc[1]= 50; pic[1].resize(280, 280);//geographer pic[2] = loadImage("businessman.png"); xc[2]= 740; yc[2]= 80; pic[2].resize(300, 300);//businessman pic[3] = loadImage("lamp.png"); xc[3]= 500; yc[3]= 400; pic[3].resize(150, 150);//lamplighter pic[4] = loadImage("earth.png");//dove xc[4]= 0;//dove yc[4]= 400; //dove pic[4].resize(500, 500); pic[5] = loadImage("drunkard.png"); xc[5]= 475; yc[5]= 30; pic[5].resize(250, 250);//drunkard pic[6] = loadImage("thevain.png"); xc[6]= 300; yc[6]= 150; pic[6].resize(200, 300);//thevain pic[7] = loadImage("earth.png"); xc[7]= 0;//plane yc[7]= 400; //plane pic[7].resize(500, 500); pic[8] = loadImage("king.png"); xc[8]= 30; yc[8]= 50; pic[8].resize(250, 300);//king pic[9] = loadImage("earth.png"); xc[9]= 0;//earth yc[9]= 400; //earth pic[9].resize(500, 500); }
Background music: “Little Prince” from Rolling Sky 2 Original Soundtrack
Sound of coins, wine, cash, switch, doves, globe, plane: https://freesound.org/
Pictures of the planets: retrieved from Pinterest.com and cut background with https://www.remove.bg/zh/upload
Sincere gratitude to Professor Gottfried, Jiaqi, and other fellows and LAs.