Some color schemes are inspired from random stuff in my dorm room when I was making them. Some are corresponds to the movie and cartoons I’ve watched. For example, the first group at the up right corner is from the Disney movie Beauty and the Beast, and the middle one on the sencond column is from the Disney cartnoon Mickey Mouse. Some are from famous paintings. For instance the two in the middle of the right column are reference from Girl with a Pearl Earring and Impression Sunrise.
Reading Reflection 1
Before reading this blog, please click on this link to Marble Springs and explore the site for a while.
What do you feel about it? Maybe confused and lost?
The reading of Story of Your Life reminds me of the structure of “hypertext”, which the ideas connected to each other like a web instead of in a linear sense. Marble Springs is a story about a town that doesn’t follow a classic storyline that we usually read(chapters followed by chapters, pargraphs followed by paragraphs, pages followed by pages). Readers of such story may feel confused because we are used to process that goes from the top to the end. When we jump to a context with no orientation or direction, it feels like a new world.
A New Friend
image from https://mutantreviewers.files.wordpress.com/2022/05/arrival.jpg?w=723&h=370
In the story Story of Your Life, an alien is introduced to the human being on earth. The main character, a mother and also a linguistician, embark on a journey to explore a completely different language system as well as the worldview of human’s.
A Totally Different Language System
The linguistician faced difficulties when tried to decode and learn the aliens’ language. The reason of it was that their spoken and written language followed totally different rules and pattern. While their spoken languge was still constructed in a linear sense, though may differ from human language, their written language were more like a drawing which a character included many elements that represents a whole sentence. And more interestingly, the alteration of curves, orientation and direction of the elements create different meanings for the character.
A Non-Linear Culture
The feature of their language corresponds to the physical structures of their bodies. Unlike human who has faces(with eyes, mouth, noses) facing towards front, the body structure of the aliens has no so-called “direction”. That is why we tend to think and convey things in a linear process, whereas the culture of the aliens don’t take directions or orientation into account. This may be a kind of experience we as a human can’t never have and relate to, provided that we are used to worldview that there is a consequence before any results, as well as the time squence of what is past, now, and future.
Final Reflection
Super Princess
Jade Wen & Robbin Wu
Instructor: Andy Garcia
Super Princess is a game following the storyline of a princess running through woods, fighting monsters, and finally saving the prince. Players use controllers with slide potentiometer and tilt sensors as swords to play the game. The game implies a message to encourage female characters in heroine story and gameplay, with modified characters from Super Mario games, a game of a male plumer saving the princess, that can resonate with the general audience (“Super Mario Bros. (Video Game 1985)”).
Before setting our idea on this project, we planned to make games (a) with ultra sensors that examine physical positions to play a modified forest version of Minesweeper, and (b) with a storyline that leads to different outcomes according to every decision the player makes. But we expected that the former would deal with technical challenges and we want a project with deeper and more interesting interaction. Therefore, we combine the elements in our proposal–a storyline that sets in the woods, and involves body movement, in which players swing the “swords” to beat the monster.
Our project is composed of two sensors, which is a slide potentiometer and a tilt sensor. The reason why we choose these two sensors is because how they are used is straightforwardly corresponds to the gameplay experience. The potentiometer sliding from left to right controls the character’s horizontal movements, and the sensor counts on the time the sword is “waved” or tilted. On top of that, we spent most of the time designing visual graphics for the characters and coding. Robbin was responsible for the graphics design, and I managed most of the coding. We first build the game structure with simple shapes that represent the characters and elements.
And then we add “switch case” to distinguish different stages of the game. We encountered a problem: we couldn’t reset case 1 when returning from case 3. After discussing with the Professor, we added code to re-setup the character in case 3 to solve the problem.
Then we replaced all the shapes with videos and pictures that Robbin designed for the characters. But the problem was during the battle. We wanted the video to play from the top to the bottom every time the sensor value became 1, but the video stopped playing every time the value became 0, which meant that the player finished swinging the sword. The solution was using
battleRight2.jump(0);
code to the video to the beginning, and didn’t stop playing until the video reached the end. The sequence and the position of the codes are the most difficult part. We tried several times with different combinations to make it work.
Our goal is to make a game that engages a fun interactive user experience with movements and visuals, as well as to imply the message that is meaningful. Instead of having a traditional joystick game controller, we made it simple and straightforward, with tilt sensors that detect people’s arm waving. In this sense, it’s not a game on fingers anymore, but involves the player’s movements that correspond to the character’s in the game, making the interaction more interesting. The main character is the modification of the princess in Super Mario. Not wearing a long dress anymore, she wears armor, and has weapons to fight against all odds to save the prince. Turning around the story of a game that is common in many people’s childhood experiences, we aimed to shed light on the female characters in gameplay and hero stories. Although we have made lots of effort on visual design, we can improve more on it if we had time. Additionally, we can also design different levels for the game, adding richness to the gaming experience.
We did face a lot of challenges in our original proposal of project and coding. But we developed and converged our ideas to a final decision through discussion within the team and with the Professor. It’s a process that takes time and inspiration, but also interesting. We originally didn’t have any implication planned, but it was suggested by the Professor to have one. Even though the game itself is interesting, having a message in it makes it more meaningful, and it carries the social responsibility to make a difference in society. Overall, despite the improvements expected, I’m satisfied with the outcome and team experience of overcoming challenges though working apart from each other.
Prcocessing Code:
import processing.serial.*; import processing.video.*; PImage tree, monster; Movie running, battleRight1, battleRight2, battleLeft; int NUM_OF_VALUES_FROM_ARDUINO = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/ int sensorValues[]; /** this array stores values from Arduino **/ String myString = null; Serial myPort; int monsteramount=1; int amount = 3; float[] x1 = new float[amount]; float[] x2 = new float[amount]; float[] x3 = new float[amount]; float[] m1 = new float[monsteramount]; float[] m2 = new float[monsteramount]; float[] my = new float[monsteramount]; float[] y = new float[amount]; float[] size = new float[amount]; float[] speed = new float[amount]; boolean battleplay=true; int swing=26; int charactersize=200; int diameter; int situation; int pMs, ms, s, m, value, arcMs, arcS; void setup() { setupSerial(); //size (1000,1000); fullScreen(); for (int j=0; j<monsteramount; j++) { m1[j]= random(650, 690); m2[j]= random(1310, 1340); my[j]= random(-height*20, -height*19); } for (int i=0; i<amount; i++) { x1[i] = random(0, 650); x2[i] = random(690, 1310); x3[i]= random(1340, width); y[i] = random(-height*3, -height); size[i] = 100; speed[i] = 5; } tree = loadImage("tree.jpg"); monster = loadImage("monster.jpg"); running= new Movie(this, "running.mp4"); battleRight1= new Movie(this, "battleRight1.mp4"); battleRight2= new Movie(this, "battleRight2.mp4"); battleLeft= new Movie(this, "battleLeft.mp4"); running.loop(); battleRight1.loop(); battleLeft.loop(); battleRight2.play(); battleRight2.pause(); } void draw() { getSerialData(); printArray(sensorValues); background(#DFBD91); //for (int i=0; i<amount; i++) { switch(situation) { default: fill(0); rectMode(CENTER); rect(width/2, height/2, width/4, height/4); fill(255); textAlign(CENTER, CENTER); textSize(100); text("RUN", width/2, height/2); if (mousePressed) { if (mouseX>=width/2-width/8 && mouseX<=width/2+width/8 && mouseY>=height/2-height/8 && mouseY<=height/2+height/8) { situation='1'; } } break; case '1': case1(); break; case '2': case2(); break; case '3': case3(); break; case '4': case4(); break; } } void case1() { if (running.available()) { running.read(); } for (int j=0; j<monsteramount; j++) { //m1[j]= random(650, 690); //m2[j]= random(1310, 1340); //my[j]= random(-height*5,-height*3); for (int i=0; i<amount; i++) { imageMode(CENTER); noStroke(); fill(255, 120, 139); image(tree, x1[i], y[i], size[i], size[i]); image(tree, x2[i], y[i], size[i], size[i]); image(tree, x3[i], y[i], size[i], size[i]); y[i] += speed[i]; image(monster, m1[j], my[j], size[i]*1.5, size[i]*1.5); image(monster, m2[j], my[j], size[i]*1.5, size[i]*1.5); my[j] += speed[i]+5; if (y[i] > height+(size[i]/2)) { x1[i] = random(0, 650); x2[i] = random(690, 1310); x3[i]= random(1340, width); y[i] = random(0, -height); size[i] = 100; speed[i] = 5; } if (my[j]>height+size[i]) { m1[j]= random(650, 690); m2[j]= random(1310, 1340); my[j]= random(-height*10, -height*9); size[i] = 100; speed[i] = 5; } diameter=int(map(sensorValues[0], 0, 1023, 0, width)); fill(255, 0, 0); image(running, diameter, height-charactersize, charactersize, charactersize*1); if (dist(diameter, height-charactersize/2, x1[i], y[i])<=size[i]/2+charactersize/2) { situation='3'; } if (dist(diameter, height-charactersize/2, x2[i], y[i])<=size[i]/2+charactersize/2) { situation='3'; } if (dist(diameter, height-charactersize/2, x3[i], y[i])<=size[i]/2+charactersize/2) { situation='3'; } if (dist(diameter, height-charactersize/2, m1[j], my[j])<=size[i]*1.5/2+charactersize/2) { situation='2'; } if (dist(diameter, height-charactersize/2, m2[j], my[j])<=size[i]*1.5/2+charactersize/2) { situation='2'; } } } } void case2() { if (battleLeft.available()) { battleLeft.read(); } image(battleLeft, width/2-500, height/2, width/3, width/3); if (sensorValues[0]>=1000) { beginShape(POINTS); for (int a = 0; a < 360; a+=6) { float angle = radians(a); float x = width/2 + cos(angle) *70; float y = 200 + sin(angle) * 70; vertex(x, y); } ms = millis(); s = (ms-pMs)/1000; m = (ms-pMs)%1000; if ((millis()>=60000+pMs)) { textSize(32); textAlign(CENTER); text("time up ", width/2, 50); situation='3'; } else { fill(2); noStroke(); textSize(32); textAlign(RIGHT); text(59-s, width/2-20, 50); textAlign(LEFT); text(999-m, width/2, 50); textAlign(CENTER); text("SWING Your Sword", width/2, 450); noFill(); stroke(2); strokeWeight(5); float arcMs = map(999-m, 0, 1000, 0, TWO_PI) - HALF_PI; arc(width/2, 200, 150, 150, -HALF_PI, arcMs); float arcS = map(59-s, 0, 60, 0, TWO_PI) - HALF_PI; strokeWeight(2); line(width/2, 200, width/2 + cos(arcS) * 60, 200 + sin(arcS) * 60); } }else { pMs= millis(); fill(2); noStroke(); textSize(32); textAlign(RIGHT); text(60, width/2-20, 50); textAlign(LEFT); text("000", width/2, 50); textSize(22); textAlign(CENTER); text("Swipe the slider to the TOP to begin.", width/2, 370); noFill(); stroke(2); strokeWeight(5); arc(width/2, 200, 150, 150, -HALF_PI, -HALF_PI); strokeWeight(2); line(width/2, 200, width/2, 200 - 60); } float mt = battleRight2.time(); float md = battleRight2.duration(); imageMode(CENTER); if (battleRight1.available()) { battleRight1.read(); } if (battleRight2.available()) { battleRight2.read(); } if (battleLeft.available()) { battleLeft.read(); } image(battleLeft, width/2-500, height/2, width/3, width/3); image(battleRight1, width/2+500, height/2, width/3, width/3); image(battleRight2, width/2+500, height/2, width/3, width/3); if (sensorValues[1]==1) { if (mt >= md*0.95) { battleplay=true; } } textSize(50); text(swing, width-200, 200); if (battleplay==true) { swing-=1; battleplay=false; battleRight2.jump(0); battleRight2.play(); } if (swing==0) { situation='4'; } } void case3() { for (int j=0; j<monsteramount; j++) { for (int i=0; i<amount; i++) { x1[i] = random(0, 650); x2[i] = random(690, 1310); x3[i]= random(1340, width); y[i] = random(-height*10, -height*9); m1[j]= random(650, 690); m2[j]= random(1310, 1340); my[j]= random(-height*5, -height*3); size[i] = 100; speed[i] = 5; background(0); fill(255); rectMode(CENTER); rect(width/2, height/2+200, width/4, height/4); text("UH-OH, YOU DIDN'T MAKE IT", width/2, height/2-100); fill(0); textAlign(CENTER, CENTER); textSize(100); text("RESTART", width/2, height/2+200); if (mousePressed) { if (mouseX>=width/2-width/8 && mouseX<=width/2+width/8 && mouseY>=height/2-height/8+200 && mouseY<=height/2+height/8+200) { situation='1'; } } } } } void case4() { for (int j=0; j<monsteramount; j++) { for (int i=0; i<amount; i++) { x1[i] = random(0, 650); x2[i] = random(690, 1310); x3[i]= random(1340, width); y[i] = random(-height*10, -height*9); m1[j]= random(650, 690); m2[j]= random(1310, 1340); my[j]= random(-height*5, -height*3); size[i] = 100; speed[i] = 5; background(0); fill(255); rectMode(CENTER); rect(width/2, height/2+200, width/4, height/4); text("You Are My Heroine, Girl", width/2, height/2-100); fill(0); textAlign(CENTER, CENTER); textSize(100); text("RESTART", width/2, height/2+200); if (mousePressed) { if (mouseX>=width/2-width/8 && mouseX<=width/2+width/8 && mouseY>=height/2-height/8+200 && mouseY<=height/2+height/8+200) { situation='1'; } } } } } void setupSerial() { printArray(Serial.list()); myPort = new Serial(this, Serial.list()[ 0 ], 9600); // WARNING! // You will definitely get an error here. // Change the PORT_INDEX to 0 and try running it again. // And then, check the list of the ports, // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" // and replace PORT_INDEX above with the index number of the port. myPort.clear(); // Throw out the first reading, // in case we started reading in the middle of a string from the sender. myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII myString = null; sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO]; } void getSerialData() { while (myPort.available() > 0) { myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII if (myString != null) { String[] serialInArray = split(trim(myString), ","); if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) { for (int i=0; i<serialInArray.length; i++) { sensorValues[i] = int(serialInArray[i]); } } } } }
Arduino Code:
const int SENSOR_PIN = 6; int tiltVal; int prevTiltVal; int curls = 0; unsigned long lastDebounceTime = 0; // the last time the output pin was toggled unsigned long debounceDelay = 50; // the debounce time; increase if the output flickers void setup() { Serial.begin(9600); pinMode(SENSOR_PIN, INPUT); // Set sensor pin as an INPUT pin } void loop() { // to send values to Processing assign the values you want to send //this is an example int sensor1 = analogRead(A0); int sensor2; //tiltsensor: int reading = digitalRead(SENSOR_PIN); // if the tilt sensor value changed, print the new value if (reading != prevTiltVal) { lastDebounceTime = millis(); //Serial.println(tiltVal); } if ( (reading != tiltVal) && (millis() - lastDebounceTime) > debounceDelay) { // whatever the reading is at, it's been there for longer than the debounce // delay, so take it as the actual current state: tiltVal = reading; if (reading == HIGH) { sensor2=1; }else{ sensor2=0; } } prevTiltVal = reading; // for Serial Plotter use //Serial.println(tiltVal); //delay(1); // send the values keeping this format Serial.print(sensor1); Serial.print(","); // put comma between sensor values Serial.print(sensor2); Serial.println(); // add linefeed after sending the last sensor value // too fast communication might cause some latency in Processing // this delay resolves the issue. delay(100); // end of example sending values }
Works Cited
The Timer during the Battle:
Recitation 10
Exercise 1
This exercise was quite easy to make. And here is the picture I got from: https://www.pexels.com/zh-cn/photo/2343170/
Processing:
import processing.serial.*; PImage photo; int NUM_OF_VALUES_FROM_ARDUINO = 1; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/ int sensorValues[]; /** this array stores values from Arduino **/ String myString = null; Serial myPort; void setup() { size(500, 500); background(255); setupSerial(); photo = loadImage("pexels-ylanite-koppens-2343170.jpg"); } void draw() { background(255); getSerialData(); printArray(sensorValues); image(photo,0,0, sensorValues[0],sensorValues[0]); // use the values like this! // sensorValues[0] // add your code // } void setupSerial() { printArray(Serial.list()); myPort = new Serial(this, Serial.list()[ 0 ], 9600); // WARNING! // You will definitely get an error here. // Change the PORT_INDEX to 0 and try running it again. // And then, check the list of the ports, // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" // and replace PORT_INDEX above with the index number of the port. myPort.clear(); // Throw out the first reading, // in case we started reading in the middle of a string from the sender. myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII myString = null; sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO]; } void getSerialData() { while (myPort.available() > 0) { myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII if (myString != null) { String[] serialInArray = split(trim(myString), ","); if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) { for (int i=0; i<serialInArray.length; i++) { sensorValues[i] = int(serialInArray[i]); } } } } }
Arduino:
void setup() { Serial.begin(9600); } void loop() { // to send values to Processing assign the values you want to send //this is an example int sensor1 = analogRead(A0); //int sensor2 = analogRead(A1); //int sensor3 = analogRead(A2); // send the values keeping this format Serial.print(sensor1); //Serial.print(","); // put comma between sensor values // Serial.print(sensor2); // Serial.print(","); // put comma between sensor values // Serial.print(sensor3); Serial.println(); // add linefeed after sending the last sensor value // too fast communication might cause some latency in Processing // this delay resolves the issue. delay(100); // end of example sending values }
Exercise 2
This one was a little bit challenging. I tried to combine different example codes we discussed in class. And it takes time to figure out the sequence of the code.
import processing.sound.*; import processing.video.*; String[] cameras = Capture.list(); Capture cam; int size=60; color c; float redValue ; float preredValue; SinOsc sine ; Env envelope; float freq ; void setup() { size(600,400); printArray(cameras); cam = new Capture(this, cameras[0]); cam.start(); sine = new SinOsc (this); envelope = new Env (this); freq = map(redValue, 0,255, 100,800); } void draw() { if (cam.available()) { cam.read(); } c = cam.get(width/2,height/2); redValue= red(c); for (int x=0; x<width; x=x+20) { for (int y=0; y<width; y=y+20) { color redchange= cam.get(x,y); // draw a circle with the color fill(redchange); noStroke(); ellipse(x, y, size,size); } } if(abs(redValue-preredValue)>10){ sine.freq(freq); envelope.play(sine, 0.005, 0.05, 0.3, 1); sine.play(); } preredValue=redValue; }
Recitation 9
Exercise 1
I had trouble when I needed to use lots of if functions because it was confusing to place different statements in the right place.
import processing.sound.*; // declare three SoundFile objects SoundFile A; SoundFile B; color c; color C; boolean a; boolean l; void setup() { size(640, 480); // create the objects and load sounds into them A = new SoundFile(this, "monsterwobbleloop.wav"); B = new SoundFile(this, "plunkysynthloop.wav"); c=255; C=255; } void draw() { background(0); fill(c); rectMode(CENTER); rect(width/2-100, height/2, 100, 100); fill(C); rect(width/2+100, height/2, 100, 100); if (keyPressed==true) { if (key=='a') { if (a==true) { A.loop(); c=100; a=false; } } } else { a=true; c=255; } if (keyPressed==true) { if (key=='l') { if (l==true) { B.loop(); C=100; l=false; } } } else { C=255; l=true; } } void keyReleased() { A.stop(); B.stop(); }
Exercise 2
The only problem I encounter was that I tried to remap the prcoessing value in the processing code, but it didn’t work. So I moved the code in the Arduino code. Additionally, I found that my vibration rythm wasn’t that corresponsive to the music’s beats.
Arduino
/* This is the array of values storing the data from Processing. */ int processing_values[NUM_OF_VALUES_FROM_PROCESSING]; void setup() { Serial.begin(9600); pinMode(ZD, OUTPUT); } void loop() { getSerialData(); if(processing_values[0]>0.6){ int vibration = map(processing_values[0],0,1,0,1023); analogWrite(ZD, vibration); }else{ analogWrite(ZD,0); } // add your code here using elements in the values array //this is an example connecting a buzzer to pin 8 /* if (processing_values[0] == 1) { //turn on an LED when the mouse is pressed digitalWrite(13, HIGH); // map values from mouseX to frequency from (0 - 500 pixels) //to the output pitch range (120 - 1500Hz) int f = map(processing_values[1], 0, 500, 120, 1500); // map values from mouseY to frequency from (0 - 500 pixels) //to the output duration range (10 - 2000 milliseconds) int d = map(processing_values[2], 0, 500, 10, 2000); // play the pitch: tone(8, processing_values[1], processing_values[2]); delay(1); // delay in between reads for stability } else { digitalWrite(13, LOW); } */ //end of example } //receive serial data from Processing void getSerialData() { while (Serial.available()) { char c = Serial.read(); //switch - case checks the value of the variable in the switch function //in this case, the char c, then runs one of the cases that fit the value of the variable //for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase switch (c) { //if the char c from Processing is a number between 0 and 9 case '0'...'9': //save the value of char c to tempValue //but simultaneously rearrange the existing values saved in tempValue //for the digits received through char c to remain coherent //if this does not make sense and would like to know more, send an email to me! tempValue = tempValue * 10 + c - '0'; break; //if the char c from Processing is a comma //indicating that the following values of char c is for the next element in the values array case ',': processing_values[valueIndex] = tempValue; //reset tempValue value tempValue = 0; //increment valuesIndex by 1 valueIndex++; break; //if the char c from Processing is character 'n' //which signals that it is the end of data case '\n': //save the tempValue //this will b the last element in the values array processing_values[valueIndex] = tempValue; //reset tempValue and valueIndex values //to clear out the values array for the next round of readings from Processing tempValue = 0; valueIndex = 0; break; } } }
Processing
import processing.serial.*; int NUM_OF_VALUES_FROM_PROCESSING = 1; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/ float processing_values[] = new float[NUM_OF_VALUES_FROM_PROCESSING]; /** this array stores values you might want to send to Arduino **/ Serial myPort; String myString; import processing.sound.*; //AudioIn microphone; Amplitude analysis; SoundFile file; void setup() { size(500, 500); background(0); setupSerial(); // create the AudioIn object and select the mic as the input stream file = new SoundFile(this, "Lost On The Freeway.mp3"); // start the mic input without routing it to the speakers file.play(); // create the Amplitude analysis object analysis = new Amplitude(this); // use the microphone as the input for the analysis analysis.input(file); } void draw() { background(0); println(analysis.analyze()); float volume = analysis.analyze(); // map the volume value to a useful scale float diameter = map(volume, 0, 1, 0, width); // draw a circle based on the microphone amplitude (volume) circle(width/2, height/2, diameter); processing_values[0] = volume; // send the values to Arduino. sendSerialData(); } void setupSerial() { printArray(Serial.list()); myPort = new Serial(this, Serial.list()[0], 9600); // WARNING! // You will definitely get an error here. // Change the PORT_INDEX to 0 and try running it again. // And then, check the list of the ports, // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" // and replace PORT_INDEX above with the index number of the port. myPort.clear(); // Throw out the first reading, // in case we started reading in the middle of a string from the sender. myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII myString = null; } void sendSerialData() { String data = ""; for (int i=0; i<processing_values.length; i++) { data += processing_values[i]; //if i is less than the index number of the last element in the values array if (i < processing_values.length-1) { data += ","; // add splitter character "," between each values element } //if it is the last element in the values array else { data += "\n"; // add the end of data character linefeed "\n" } } //write to Arduino myPort.write(data); print(data); // this prints to the console the values going to arduino }
Homework
import processing.sound.*; // declare an AudioIn object AudioIn microphone; // declare an Amplitude analysis object to detect the volume of sounds Amplitude analysis; float diameter; void setup() { size(640, 480); // create the AudioIn object and select the mic as the input stream microphone = new AudioIn(this, 0); // start the mic input without routing it to the speakers microphone.start(); // create the Amplitude analysis object analysis = new Amplitude(this); // use the microphone as the input for the analysis analysis.input(microphone); } void draw() { println(analysis.analyze()); background(255); // analyze the audio for its volume level float volume = analysis.analyze(); // map the volume value to a useful scale diameter = map(volume, 0, 1, 30, width); // draw a circle based on the microphone amplitude (volume) //circle(width/2, height/2, diameter); puppet(width/2,height/2,200); } void puppet(float x, float y, float size){ fill(0); circle(x,y,size); triangle(x-size*0.4,y-size*0.2, x, y-size*0.3,x-size*0.3,y-size*0.7); triangle(x+size*0.4,y-size*0.2, x, y-size*0.3,x+size*0.3,y-size*0.7); line(x-size*0.4,y,x-size*0.7,y-size*0.1); line(x-size*0.4,y,x-size*0.7,y+size*0.1); line(x+size*0.4,y,x+size*0.7,y-size*0.1); line(x+size*0.4,y,x+size*0.7,y+size*0.1); fill(255); circle(x-size*0.2, y-size*0.1,size*0.1); circle(x+size*0.2, y-size*0.1,size*0.1); circle(x,y,size*0.1); fill(255,0,0); circle(x,y+size*0.2,diameter); }
Final Project: essay
A Knight’s Mission
Many games nowadays engage with sophisticated design and user experience in visuals, sounds, and actions. We seek to develop a fun game focusing on this kind of interactive experience engaging mutual communications and jumping out of the boundaries of a stereotypical video game with controllers.
We aim to create a game that requires a player’s action and movement, in hope of making it more entertaining and fun. We plan to design an adventure where players take on and overcome every obstacle one encounters. With support of sound and visual aids as well as physical interaction, we hope it enhances the level of engagement for players. By the end of this week, we need to finalize on the details, and start prototyping that tests the feasibility of the codes with the mechanism of the sensors.
From the beginning of the semester, we have learned that interaction is a “process in which two actors alternately listen, think and speak”, according to Chris Crawford. And at the research phase, the interactive project “Future You” really impressed me by its deep, back-and-forth communication with the users. It’s a reflection of the user’s movement and will evolve itself over time into a more complex shape. It speaks to what a good interaction is: it reacts to what a user does, the user receives it and reacts back on top of the conversation, and the content of the conversation grows. We wish to build something based on this kind experience, and make it a fun game. A successful outcome should be intuitive to play, so the audience from kids to the adult is included, and entertaining. It engages a rich experience of a user’s whole movement, making the game responsive to the user as if they are communicating in a real life setting. Since we have designs like VR that are aimed for visual pleasure, future steps can lead to how to make the game experience even more real and exciting.
Works Cited
Crawford, Chris. “What Exactly Is Interactivity.” The Art of interactive design : a euphonious and illuminating guide to building successful software, No Starch Press, 2003, pp. 1-7.
“Universal Everything Creates Interactive Digital Installation for Barbican.” Dexigner, 24 May 2019, https://www.dexigner.com/news/32131.
Recitation 8
Processing Etch-A-Sketch
The code of the drawing line is pretty much similar to the code of the button. It remembers the previous position of the line to draw a continuous line. Originally, when I turn the potentiometer to a really big value, the line disappears in the screen. So I need to map the value of the potentiometer to the corresponded range of the screen size.
Processing Code:
import processing.serial.*; int NUM_OF_VALUES_FROM_ARDUINO = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/ int sensorValues[]; /** this array stores values from Arduino **/ int Presensorvalues[]; String myString = null; Serial myPort; void setup() { size(500, 500); background(0); setupSerial(); } void draw() { getSerialData(); printArray(sensorValues); //background(0); if (sensorValues[0] != Presensorvalues[0]) { if(sensorValues[1] != Presensorvalues[1]) { sensorValues[0]=int(map(sensorValues[0],0,1023,0,width)); sensorValues[1]=int(map(sensorValues[1],0,1023,0,height)); stroke(255); line(sensorValues[0], sensorValues[1], Presensorvalues[0], Presensorvalues[1]); } } // add your code Presensorvalues[0]=sensorValues[0]; Presensorvalues[1]=sensorValues[1]; } void setupSerial() { printArray(Serial.list()); myPort = new Serial(this, Serial.list()[ 0 ], 9600); // WARNING! // You will definitely get an error here. // Change the PORT_INDEX to 0 and try running it again. // And then, check the list of the ports, // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" // and replace PORT_INDEX above with the index number of the port. myPort.clear(); // Throw out the first reading, // in case we started reading in the middle of a string from the sender. myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII myString = null; sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO]; Presensorvalues = new int[NUM_OF_VALUES_FROM_ARDUINO]; } void getSerialData() { while (myPort.available() > 0) { myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII if (myString != null) { String[] serialInArray = split(trim(myString), ","); if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) { for (int i=0; i<serialInArray.length; i++) { sensorValues[i] = int(serialInArray[i]); } } } } }
Arduino Code:
void setup() { Serial.begin(9600); } void loop() { // to send values to Processing assign the values you want to send //this is an example int sensor1 = analogRead(A0); int sensor2 = analogRead(A1); //int sensor3 = analogRead(A2); // send the values keeping this format Serial.print(sensor1); Serial.print(","); // put comma between sensor values Serial.print(sensor2); // Serial.print(","); // put comma between sensor values //Serial.print(sensor3); Serial.println(); // add linefeed after sending the last sensor value // too fast communication might cause some latency in Processing // this delay resolves the issue. delay(100); // end of example sending values }
Bouncing Ball
Making the ball bouncing was quite easy. But I spent some time working on the “draw()” function. I originally used the processing values to be the x-position of the ball. But it didn’t work. So I created a variable “x” and added an if statement to determine the time when to trigger the arduino.
Processing Code:
import processing.serial.*; int NUM_OF_VALUES_FROM_PROCESSING = 1; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/ int processing_values[] = new int[NUM_OF_VALUES_FROM_PROCESSING]; /** this array stores values you might want to send to Arduino **/ Serial myPort; String myString; int x; int speed=20; void setup() { fullScreen(); background(0); setupSerial(); } void draw() { background(0); fill(255); circle(x, height/2,300); move(); if(x>=width-300){ processing_values[0]=1; }else{ processing_values[0]=0; } sendSerialData(); } void move(){ x+=speed; if(x>=width || x<=0){ speed=-speed; } } void setupSerial() { printArray(Serial.list()); myPort = new Serial(this, Serial.list()[0], 9600); // WARNING! // You will definitely get an error here. // Change the PORT_INDEX to 0 and try running it again. // And then, check the list of the ports, // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" // and replace PORT_INDEX above with the index number of the port. myPort.clear(); // Throw out the first reading, // in case we started reading in the middle of a string from the sender. myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII myString = null; } void sendSerialData() { String data = ""; for (int i=0; i<processing_values.length; i++) { data += processing_values[i]; //if i is less than the index number of the last element in the values array if (i < processing_values.length-1) { data += ","; // add splitter character "," between each values element } //if it is the last element in the values array else { data += "\n"; // add the end of data character linefeed "\n" } } //write to Arduino myPort.write(data); print(data); // this prints to the console the values going to arduino }
Arduino Code:
#define NUM_OF_VALUES_FROM_PROCESSING 1 /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/ #include <Servo.h> Servo servo; /** DO NOT REMOVE THESE **/ int tempValue = 0; int valueIndex = 0; /* This is the array of values storing the data from Processing. */ int processing_values[NUM_OF_VALUES_FROM_PROCESSING]; void setup() { Serial.begin(9600); pinMode(13, OUTPUT); servo.attach(9); } void loop() { getSerialData(); if(processing_values[0]==1){ servo.write(180); delay(500); servo.write(0); delay(500); } } //receive serial data from Processing void getSerialData() { while (Serial.available()) { char c = Serial.read(); //switch - case checks the value of the variable in the switch function //in this case, the char c, then runs one of the cases that fit the value of the variable //for more information, visit the reference page: https://www.arduino.cc/en/Reference/SwitchCase switch (c) { //if the char c from Processing is a number between 0 and 9 case '0'...'9': //save the value of char c to tempValue //but simultaneously rearrange the existing values saved in tempValue //for the digits received through char c to remain coherent //if this does not make sense and would like to know more, send an email to me! tempValue = tempValue * 10 + c - '0'; break; //if the char c from Processing is a comma //indicating that the following values of char c is for the next element in the values array case ',': processing_values[valueIndex] = tempValue; //reset tempValue value tempValue = 0; //increment valuesIndex by 1 valueIndex++; break; //if the char c from Processing is character 'n' //which signals that it is the end of data case '\n': //save the tempValue //this will b the last element in the values array processing_values[valueIndex] = tempValue; //reset tempValue and valueIndex values //to clear out the values array for the next round of readings from Processing tempValue = 0; valueIndex = 0; break; } } }
Homework
Processing Code:
import processing.serial.*; int NUM_OF_VALUES_FROM_ARDUINO = 2; /** YOU MUST CHANGE THIS ACCORDING TO YOUR PROJECT **/ int sensorValues[]; /** this array stores values from Arduino **/ String myString = null; Serial myPort; void setup() { size(500, 500); background(0); setupSerial(); } void draw() { background(0); getSerialData(); printArray(sensorValues); if (sensorValues[0]%2==1){ star(width*0.3, height*0.3, 30, 70, 5); } if (sensorValues[1]%2==1){ star(width*0.7, height*0.7, 80, 100, 40); } } void setupSerial() { printArray(Serial.list()); myPort = new Serial(this, Serial.list()[ 0 ], 9600); // WARNING! // You will definitely get an error here. // Change the PORT_INDEX to 0 and try running it again. // And then, check the list of the ports, // find the port "/dev/cu.usbmodem----" or "/dev/tty.usbmodem----" // and replace PORT_INDEX above with the index number of the port. myPort.clear(); // Throw out the first reading, // in case we started reading in the middle of a string from the sender. myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII myString = null; sensorValues = new int[NUM_OF_VALUES_FROM_ARDUINO]; } void getSerialData() { while (myPort.available() > 0) { myString = myPort.readStringUntil( 10 ); // 10 = '\n' Linefeed in ASCII if (myString != null) { String[] serialInArray = split(trim(myString), ","); if (serialInArray.length == NUM_OF_VALUES_FROM_ARDUINO) { for (int i=0; i<serialInArray.length; i++) { sensorValues[i] = int(serialInArray[i]); } } } } } void star(float x, float y, float radius1, float radius2, int npoints) { float angle = TWO_PI / npoints; float halfAngle = angle/2.0; beginShape(); for (float a = 0; a < TWO_PI; a += angle) { float sx = x + cos(a) * radius2; float sy = y + sin(a) * radius2; vertex(sx, sy); sx = x + cos(a+halfAngle) * radius1; sy = y + sin(a+halfAngle) * radius1; vertex(sx, sy); } endShape(CLOSE); }
Arduino Code:
int button1=0; int button2=0; int sensor1=0; int sensor2=0; void setup() { Serial.begin(9600); pinMode (2, INPUT); pinMode (4, INPUT); } void loop() { // to send values to Processing assign the values you want to send //this is an example int button1 = digitalRead(2); int button2 = digitalRead(4); //int sensor3 = analogRead(A2); if (button1 == HIGH){ delay(100); sensor1++; } if (button2 == HIGH){ delay(100); sensor2++; } // send the values keeping this format Serial.print(sensor1); Serial.print(","); // put comma between sensor values Serial.print(sensor2); //Serial.print(","); // put comma between sensor values //Serial.print(sensor3); Serial.println(); // add linefeed after sending the last sensor value // too fast communication might cause some latency in Processing // this delay resolves the issue. delay(100); // end of example sending values }
Final Project Proposal
Dodger VS Barriers
This kind of dodging game mostly contain only one player competing with the system. We think it would be cool if this game is able to engage more players playing different roles. One player be as a dodger try to avoid all the barriers and another can create barriers as well as the size of them. Instead of being a stand-alone game with only one player, it’s a game that can be enojoyed with family and friends.
The Adventure in the Woods
We got an inspiration from this video. They use two ultrasonic sensors to locate the x-coordinate and y-coordinate of one object. We’d like to combine this with the game Minesweeper. And it’s also performed by more than one player, dedicating for a group game that is engaging for family and friends. Players have their respective piece. They move to different places in the box, and the computer tells them the number of the “monsters” near them, hinting the position of the “monsters”. The one find the treasure first alive wins. It’s a game requiring players to infer the possibilities, which is fun and challenging.
Dungeon Explorer
After the board game project in midterm, this project will be our further step to explore the board game world. We will give the user the right to choose 3 equipments for them to pack into their backpack before they enter the dungeon. After inputting which 3 equipments they have chosen, they need to use on of the equipments every time they met an “event”. Their choice will lead to different endings of the story.
Recitation 7
Step 1
I created lightbulbs as my graphics. And repeated into a grid. At first I misunderstood the instructions. I made a shape instead of a function.
And I also figured out that if I want to display the shape in random sizes and colors, I need to place those settings inside the for loop.
float x; float y; float size; float r; float g; float b; color c; void setup() { background(50); size(1000, 1000); noLoop(); } void draw() { for (int i=100; i<width; i+=100) { for (int j=50; j<height; j+=100) { size=random(50, 100); r = random(255, 255); g = random(230, 255); b= random(0, 255); c=color(r, g, b); lightbulb(i, j, size, c); } } } void lightbulb(float x, float y, float size, color c) { noStroke(); fill(c); ellipse(x, y, size, size); triangle(x-size*0.4, y+size*0.3, x+size*0.4, y+size*0.3, x, y+size*1.02); rectMode(CENTER); fill(0); rect(x, y+size*0.9, size*0.4, size*0.1, 28); rect(x, y+size*0.8, size*0.4, size*0.1, 28); rect(x, y+size, size*0.4, size*0.1, 28); }
Step 2
I made the graphics move around and bounce within the canvas borders. And I added a little interaction. If I press the mouse, the graphics gather at it. But I did face a problem. I didn’t know how to develop the movement from the original grid pattern, so I set them with random(x,y). If I have a more time, I may try how to let them move around the border of the canvas as well(like once they touch the right side of the wall, they move down along the border to the down side of the wall).
int count=100; float x[]= new float[count]; float y[]= new float[count]; float size[]= new float[count]; float r[]= new float[count]; float g[]= new float[count]; float b[]= new float[count]; color c[]= new color[count]; float xspeed[]= new float[count]; float yspeed[]= new float[count]; void setup() { background(50); size(1000, 1000); for (int i=0; i<x.length; i++) { x[i] = random(100,width-100); y[i] = random(100,height-100); size[i] = random (50, 100); r[i] = random(255,255); g[i] = random(230,255); b[i] = random(0,255); c[i]= color(r[i], g[i], b[i]); xspeed[i]=random(2,10); yspeed[i] =random(2,10); } } void draw() { background(50); for(int i=0; i<x.length; i++){ lightbulb(x[i], y[i], size[i], c[i]); if (mousePressed){ x[i]+=(mouseX-x[i])*0.05; y[i]+=(mouseY-y[i])*0.05; } } move(); bounce(); } void move(){ for(int i=0; i<x.length; i++){ x[i]+=xspeed[i]; y[i]+=yspeed[i]; } } void bounce(){ for(int i=0; i<x.length; i++){ if(x[i]<=0+size[i]/2 || x[i]>= width-size[i]/2){ xspeed[i]=-xspeed[i]; } if(y[i]<=0+size[i]/2 || y[i]>= height-size[i]/2){ yspeed[i]=-yspeed[i]; } } } void mousePressed(){ for(int i=0; i<x.length; i++){ x[i]+=mouseX-x[i]; y[i]+=mouseY-y[i]; } } void lightbulb(float x, float y, float size, color c) { noStroke(); fill(c); ellipse(x, y, size, size); triangle(x-size*0.4, y+size*0.3, x+size*0.4, y+size*0.3, x, y+size*1.02); rectMode(CENTER); fill(0); rect(x, y+size*0.9, size*0.4, size*0.1, 28); rect(x, y+size*0.8, size*0.4, size*0.1, 28); rect(x, y+size, size*0.4, size*0.1, 28); }
Question 1
I feel like it was Dynamic Passive at the first part because the size and color can change everytime I reopen the program. But it’s not really interact with me. And at the second part, it was Dynamic Interactive, since the shapes changed and moved by themselves, and I can engage in it activly.
Question 2
An array save us a lot of time and energy if we want create a massive amount of copy of something. We can use it when we want to make an animated poster with complicated patterns.
Research for Final Project
Research Project 1: Future You
This project impressed me when the first time I saw it online. I think it’s successful because not only it is highly interactive, but it creates a long term engagement with the users as the reflection evolves over time with more sophisticated and sensitive movements. And it’s also straightfoward in use. Users stand in front of it, see it moves in reflect of themselves, and thet know how it works.
Research Project 2: Liguid Midi
I think this is very innovative. They focus on the sense of listening combining with a user interface on textile. People can control sound and music when interacting with the piece of fabric. They explore a different possibility of using experience.
From the beginning of the semester, we have defined what interaction is. “Interaction is a cyclic process in which two actors alternately listen, think and speak”, according to Chris Crawford in his book “The Art of interactive design : a euphonious and illuminating guide to building successful software” (5). During an interaction, there should be a communication between at least two subjects.
As we made through our way to Group Project and Midterm Project, I realize that a successful interactive design is intuitive in using. When users first meet it, they can make sense of it little by little, like infant does, when interacting with it. Ernest Edmonds explained in his article that it is our intuition to “find patterns” to make sense of the environment. We are born with the ability and habit to ask ourselves “If I do this and that, will the world do some particular other thing?”, as Edmonds continues. Thus in a spontaneous interaction, there is no need for complicated instructions, and users can start engaging with the communication naturally. A very good example is the project above “Future You”. Participants can easily figure out if they move, the character on the screen moves too. And they figure out that it is their reflection.
Another thing I found inspiring in Edmonds’s article is the long-term engagement in interaction art. Instead of a short time reaction to attract users, the interactive experience evolves as time goes by. What we did in the class project focus more on the present feeling. A successful design like “Future You”, the movement and shape develop over time that prolongs the engagement of the participants, because they can see the change in complexity of the reflection when they stay for another few minutes.
Works Cited
Crawford, Chris. “What Exactly Is Interactivity.” The Art of interactive design : a euphonious and illuminating guide to building successful software, No Starch Press, 2003, pp. 1-7.
Edmonds, Ernest. “Art, Interaction and Engagement.” The art of interaction : what HCI can learn from interactive art, Morgan & Claypool Publishers, 2018.
“Liquid MIDI: paper goes electronic to create unique controls and sounds.” Designboom, 20 July 2015, https://www.designboom.com/technology/ejtech-liquid-midi-07-20-2015/.
“Universal Everything Creates Interactive Digital Installation for Barbican.” Dexigner, 24 May 2019, https://www.dexigner.com/news/32131.