Part 1

PROJECT DESCRIPTION

SOUNDS FAMILIAR? : ENGLISH IN SHANGHAINESE

by Jiasheng Li (Jason)

2023

Interactive Storytelling & Tool

link: https://jasonlee557.github.io/CCLab/project-b/

ELEVATOR PITCH

The project “Sounds Familiar?” uses an interactive story to engage the audience in the loanwords in Shanghainese from English. By exploring the story, the audience will have a general sense of Shanghainese words, take a quiz to check their learning outcome, and learn more in the dictionary. 

Abstract:

As a Shanghainese, I found out that a lot of peers cannot speak or forget Shanghainese after going to school due to the requirement of a Mandarin-speaking environment at school. I realized that it is important to preserve Shanghainese and engage more people to learn this dialect. Being at NYU Shanghai, I want to let more people participate in this project. Given that everyone knows English in this community, it inspired me to make this website based on English loanwords in Shanghainese. In other words, it might be easier for English users to start with words that sound similar to English pronunciation. To make the journey interesting, I made up a story based on six of the words. Hope you can learn these words with the quiz and learn more in the extra dictionary. Enjoy!

Part 2

1) Process: Design and Composition

VISUAL

The first page is a statement about the website and its goals. 

From page 2 to page 4 is the interactive story.

Skimming a series of Shanghainese loanwords from English, I picked six words [taxi(charter), dashing, biscuit, nougat, baby boss(kite), and slightly(a minimum)]. The six words were separated into three pages (taxi, bakery, and baby boss). Each page is a sentence with a pair of words. The sentences are connected. 

One day, Joey is in a taxi, feeling dashing as if on a limo.

Arriving at a bakery, Joey buys some biscuits and nougats.

Joey slightly feels like being a baby boss.

The visual designs are related to the storylines. Here is an example from the bakery page.

As for the quiz part, the red box is used as a visual cue for the audience to click on the dictation. The white box is used for the audience to input their answers. Lastly, the grey box is for the audience to submit their answers and get to the next exercise.

Lastly, the dictionary is designed to show the original English word, the Shanghainese pronunciation, and its (new) meaning in Shanghainese.

INTERACTION

The interaction within the story on each page is related to the scene on that page. Users can use buttons to go back and forth from different pages.

1)Sound

Generally, users can press ENTER to get the audio of the part of the story on that page. They can also press the first letter of the English word to get the sound of the Shanghainese, except for the bakery page where the interaction of the story and the sound play are mixed together. 

2)Story Interaction

For the taxi page, the interaction is for the users to move the MOUSE to park the taxi at the bakery. It would stop when it is at the bakery. 

For the bakery page, users can use LEFT RIGHT UP DOWN to “eat” the items. LEFT and RIGHT are for eating the nougats while UP and DOWN are for eating the biscuit. The pronunciation of the eaten food would be played once it is eaten. The video below is a demonstration without sound.

For the baby boss page, users could open their cameras to be the baby boss. I specifically chose JOEY as the name, for its gender neutrality to bring more inclusion in the user engagement.

3)QUIZ

The quiz is designed for the users to click the red box to listen to the words that they just learned. They should type in the meaning of the Shanghainese. If they get the correct answer, the score will add one. The answer to the question will be shown in the next question. The video below is a demonstration without sound.

4)Dictionary
 Users can click the speaker token 📢 to get the sound of the Shanghainese.

2) Process: Technical

Website Structure

The website’s inner code structure is basically an HTML with other HTML.

I had problems with linking pages with each other because I individually created one folder for each sketch with their assets, JS, HTML, and CSS at first. The problems would be tedious and hard to call for the files, and the folder names would be like “page1/page2/page3/page4″ as if nesting dolls. It would cause errors if the folder being called is not the subfolder of the folder.

Therefore, I merged them into one folder, and directly called the HTML instead. 

</div> <button class="button1"><a href="index2.html">nougats</a> </button>

</div> <button class="button2"><a href="quiz.html">quiz</a> </button>

.text {

position: fixed;

top: 10%;
left: 50%;
background-color: #f0d7a7;
text-align: center;
padding-top: 100px;
padding-bottom: 100px;
width: 50%;

}

let carX = 0;
let check = 0;

carX = mouseX;

if (mouseX > width – 10) {
check = check + 1;
}
if (check > 0) {
carX = width – 10;
}

The smoke is made by randomness and transparency. 

//smoke
push();
translate(width-100, height / 2);
fill(255,255,255,150);
noStroke();
arc(0, -51, 25, 20, 0, PI);
triangle(-12.5, -50, 12.5, -50, 0, random(-300,-150));
pop();

The taxi is made through function for the convenience of moving as a whole.

function drawCar(xPos, yPos, name) {
push();
translate(xPos, yPos);
scale(3);

// light
c1 = "white";
fill("c1");
ellipse(-10, -40, 10, 20);

// body
fill("green");
rect(-20, -40, 40, 40);
rect(-60, 0, 120, 40);

// decoration
fill("white");
rect(-60, 20, 120, 5);
rect(-60, 25, 120, 5);

fill(255);
textSize(20);
text(name, -50, 20);

// wheels
fill(0);
drawWheel(-25, 40);
drawWheel(25, 40);
pop();

function drawWheel(xPos, yPos) {
push();
translate(xPos, yPos);

fill(50);
ellipse(0, 0, 35, 32);

pop();
}
}

The bakery scene is an example of how the sound is played. It first checks whether the key is pressed and then whether the sound is being played. By checking whether the sound is played or not, the interaction avoids the annoying echoes. 

function draw() {
if (keyIsPressed) {
// eat left
if (keyCode == LEFT_ARROW&&nougat.isPlaying() == false) {
fill("white");
noStroke();
rect(0, 0, candy.width*0.25/2, height);
nougat.play();
}
// eat right
if (keyCode == RIGHT_ARROW && biscuit.isPlaying()==false) {
fill("white");
noStroke();
rect((2 * width) / 3, 0, width / 3, height);
biscuit.play();
}
// eat mid
if (keyCode == UP_ARROW) {
fill("white");
noStroke();
rect(width / 3, 0, width / 6, height);
}
if (keyCode == DOWN_ARROW) {
fill("white");
noStroke();
rect(width / 2, 0, width / 6, height);
}
// story sound
if (keyCode == ENTER && story2.isPlaying() == false) {
story2.play();
}
}
}

For the baby boss sketch, the main issue is to add another layer for the camera to float on the baby boss cartoon. It is done by Graphics.

function setup() {

createCanvas(355, 530);
// float on the pic
pg = createGraphics(300, 300);
cam = createCapture(VIDEO);
cam.hide();

}
function draw() {

background(245);

image(img, 0, 0, img.width, img.height);
translate(width,0);
scale(-1, 1);
//zoom in the user's face
pg.image(cam, -200, 0);
//display
image(pg, 65, 30, 230, 250);

For the quiz, I used the list to keep track of the question-answer-sound. 

let questions = [
"1.What is the sound in the red?",
"2.What is the sound?(answer1:taxi)",
"3.What is the sound?(answer2:dashing)",
"4.What is the sound?(answer3:biscuit)",
"5.What is the sound?(answer4:nougat)",
"6.babyboss(answer5:slightly)",
"Click the button at the bottom",
];
let answers = ["taxi", "dashing", "biscuit", "nougat", "slightly", "babyboss"];
let s = [];
function draw() {
clear();
textSize(20);
fill(0);
text(score, 10, 20);
text(questions[counter], width / 2 - 100, height / 2 - 50);
push();
fill("red");
rect(width / 4, height / 2, 50, 50);
pop();
if (
mouseIsPressed &&
mouseX > width / 4 &&
mouseX < width / 4 + 50 &&
mouseY > height / 2 &&
mouseY < height / 2 + 50 &&
s[counter].isPlaying() == false
) {
s[counter].play();
}
}

3) Reflection and Future Development

Generally, I am proud that I recorded all the Shanghainese pronunciations myself because I didn’t find useable resources online for me to embed into the website. I completed the project with an interactive story, a quiz, and a dictionary. And during the presentation day, everyone was engaged in my project. I would like to thank them for repeating the Shanghainese with me.

I felt a sense of satisfaction when people asked me about Shanghainese during the Q&A session. Presentation makes impression; visibility emerges opportunity. I really appreciate the questions about whether Shanghainese is about to extinct and the difference between the Shanghai dialect and Mandarin raised by Ethan and Professor Moon. They recognized and sublimated the value of the project.

There is also a lot of constructive advice that I received throughout the whole process. I have appropriated them in the updated version.

At first, I only chose the topic of Shanghainese for the input of sound and then Professor Godoy told me to narrow down it to a specific part. I realized that the project should have a more nuanced angle. Also, she pointed out that taxis can just drive horizontally instead of driving from top to bottom. It is an obvious and natural thing but I didn’t notice that in my hand-written sketch at the beginning. 

After skimming a list of Shanghainese loanwords, I came up with the story. During the pitch presentation, Joey suggested that I add a camera for the user to see themselves speaking Shanghainese. I changed it into the interaction that everyone could be Joey and feel like a baby boss.

During the presentation day, IMA Fellow Carrot and Professor Godoy suggested I add a quiz for people to test what they just learned. I added them later on in the updated version.

For future development, I think the most important parts are the aesthetics and making the interactions more seamless as Professor Moon advised. Currently, the cartoons are a bit childish and the interactions are more separated within each individual page. Even though I added a “bakery” with a chimney in the updated version, there is still more that could be done for continuity. 

As for the quiz and dictionary, I believe they could be better developed in terms of interface. For now, they are just in a simple version. Images could be added to make them more colorful.

4)Credits

The topic, Shanghainese, is inspired by Mia Fan’s Dialect Protection in China in IMA Capstone Show 2023 at NYU Shanghai. The format, interactive storytelling, is introduced by XKCD. The Shanghainese words and further details are from an article called: Tracing the Heritage of Pidgin English in Mainland China: The Influences of Yangjingbang English on Contemporary Culture and Language in Shanghai by Jian Li published online by Cambridge University Press: 03 January 2017.

As the project went on, I was unclear about the loading issues. Professor Godoy said that it would be better to call assets online instead of calling things locally. I thought that would be the same case for inserting the sketches into HTML. Fortunately, with the help of the professor, I changed the website links of p5 sketches to js files.

To add the camera as another layer, Professor Godoy taught me how to use Graphics. 

I learned how to set up the input box and the button in p5 through YouTube video “How to make a Quiz in P5 JavaScript“. 

Lastly, thank the IMA community for the support and companionship. The caring and non-competitive environment generated a lot of helpful feedback and inspiration.