“GOD”

                                                                            — — Project Report

A.

The name of this project is “GOD.”

B.

It is an interactive video where players can interact with the machine through their choices, allowing the storyline to change according to the players’ desires. I believe interactivity is manifested in the fact that players can change the content of the video according to their own ideas.

 Although I provide them with limited types of choices, they can still make their own decisions. At the same time, when making choices in the video, the audience also receives feedback. For example, if the video content involves riding a bicycle, a fan next to the control console will be activated to simulate the cycling environment by blowing air at the players. In the video, there are pauses, and the characters in the video also hint at offline interactions through dialogue to change the video content.  However, in reality, during the user test, almost every user focused their attention on the screen and did not realize that they needed to interact to change the video. Through discussions with them, I discovered that it was a design problem. At that time, there was not a strong association between the control console and the video on the screen, and there was not enough information to direct the players’ attention to the interactive objects I had 3D printed. So, first, I placed the control console closer to the computer screen. Second, I made some improvements in the video content by placing more prominent letters as prompts. In the second round of user testing, it worked much better, and testers became aware that they needed to make choices.

C.

Regarding the construction part, I focused more on the implementation of processing rather than building Arduino. Because I believe the main form of presentation for my project is the interactive video, where the interaction comes from the audience’s choices. This part does not require a lot of Arduino knowledge; therefore, I focused more on coding in processing. The most challenging part of Arduino was how to make the system aware of the audience’s choices and control the video content changes.

I thought of the NFC chip in the campus card, which can distinguish each recognized object and make corresponding choices. Therefore, I used the RFID-RC522 recognition system and attached RFID stickers to each object that needed recognition.

By setting different programs for different chips in Arduino, I achieved the effect of changing the video. In the interaction between processing and Arduino, I used a temperature sensor. The idea was that when players blew air at the temperature sensor attached to the rear of the bicycle, the temperature would rise, causing the bicycle’s speed in the video to increase, and the fan would start working to simulate the cycling environment.

#include 
#include 
#define SS_PIN 10
#define RST_PIN 5

MFRC522 mfrc522(SS_PIN, RST_PIN);
const int motorPin = 9;  // 控制电机的引脚
void setup() {
  Serial.begin(9600);
  SPI.begin();
  mfrc522.PCD_Init();
  Serial.println("Scan an RFID card or wait for temperature reading...");
  pinMode(motorPin, OUTPUT);
}

void loop() {
  if (mfrc522.PICC_IsNewCardPresent() && mfrc522.PICC_ReadCardSerial()) {
    byte uid[10];
    byte uidLength = mfrc522.uid.size;
    for (byte i = 0; i < uidLength; i++) {
      uid[i] = mfrc522.uid.uidByte[i];
    }
    String cardID = "";
    for (byte i = 0; i < uidLength; i++) {
      if (uid[i] < 0x10) { cardID += "0"; } cardID += String(uid[i], HEX); } cardID.toUpperCase(); Serial.print("RFID Card ID: "); Serial.println(cardID); mfrc522.PICC_HaltA(); mfrc522.PCD_StopCrypto1(); } int temperatureValue = analogRead(A0); Serial.print("Temperature: "); Serial.println(temperatureValue); if (temperatureValue > 50) {
    // 温度传感器数值大于100，启动电机
    digitalWrite(motorPin, HIGH);  // 设定电机引脚为高电平
  } else {
    // 温度传感器数值小于或等于100，停止电机
    digitalWrite(motorPin, LOW);  // 设定电机引脚为低电平
  }

  delay(300);
} 

In processing, the difficulty I encountered was the continuous playback of the video. Playing different videos based on different conditions resulted in many strange issues, such as the video not starting from the beginning or playing without sound.（one of the videos:

It took me a whole 17 hours to debug. Later, with Rudi’s guidance, I used arrays to write the code, making the logic very smooth, and successfully achieved the complete playback of the video.

String state="Instruction";
import processing.video.*;
Movie movies[]=new Movie[10];
ArrayList<Movie>ms=new ArrayList();
String movieName[]=new String[10];
int mw, mh;
boolean left=true;
String id[]={"3DCC1EDE", "5DCB1EDE", "2DCC1EDE"};
//自行车 水瓶 伞
String getId="";
import processing.serial.*;

Serial port;
String data = "";
int temp=0;

void settings(){
  fullScreen(P2D);
}

void setup() {
  port = new Serial(this, Serial.list()[0], 9600);

  println(Serial.list()[0]);
  
  frameRate(60);
  mh=int(height*0.8);
  mw=int(mh*1.77);
  ms.add(new Movie(this, "Introduction.mp4"));
  movieName[0]= "Instruction";
  for (int i=1; i<movies.length; i++) {
    ms.add(new Movie(this, char(i+64)+".mp4"));
    movieName[i]=  char(i+64)+"";
  }

  imageMode(CENTER);
}
void processData(String data) {
  if (data.startsWith("RFID Card ID")) {
    //println("Received RFID Card ID: " + data.substring(14));
    getId= data.substring(14);
  } else if (data.startsWith("Temperature")) {
    println("Received Temperature: " + data.substring(12));
    temp=int(data.substring(12).trim());
  }
}
void draw() {
  while (port.available() > 0) {
    char received = (char) port.read();
    if (received == '\n') {
      processData(data);
      data = "";
    } else {
      data += received;
    }
  }
  println(state, temp);
  if (state=="A"&&getId.trim().equals(id[2])) {
    left=false;
  }
  if (state=="E"&&getId.trim().equals(id[1])) {
    left=true;
  } else if (state=="E") {
    left=false;
  }
  background(0);
  for (int i=0; i<ms.size(); i++) {
    Movie m=ms.get(i);
    if (state.equals(movieName[i])) {
      if (m.isPlaying()==false) {
        m.play();
      }
      if (state=="D"&&temp>150) {
        m.speed(3);
      } else if (state=="D") {
        m.speed(1);
      }
      image(m, width/2, height/2, mw, mh);

      if (m.time()==m.duration()) {
        m.stop();
        m.speed(1);
        videoControl();
      }
    } else if (m.isPlaying()==true) {
      m.stop();
      m.speed(1);
    }
  }
}
void mousePressed() {
  if (state=="Instruction") {
    state="A";
  }
}
void videoControl() {
  if (state=="Instruction") {
    movies[0].jump(0);
    movies[0].play();
  } 
  else if (state=="A") {
    if (left) {
      state="B";
    } else {
      state="C";
    }
  } 
  else if (state=="B"||state=="C") {
    state="D";
  }
  else if (state=="D" ) {
    state="E";
  } 
  else if (state=="E" ) {
    if (left) {
      state="F";
    } else {
      state="G";
    }
  } 
  else if (state=="F"||state=="G") {
    state="H";
  } 
  else if (state=="H") {
    state="I";
  } 
  else if (state=="I") {
    state="Z";
  }
}
void movieEvent(Movie m) {
  m.read();
}

 

D.

I initially came up with this idea based on my personal experience. During the final week, things kept coming at me, and I felt like I was being pushed forward every day. My life seemed to be controlled by others. So I thought, why not control someone else’s life? That’s how I came up with the idea of creating this interactive video to control the movements of the characters in the video.

From this perspective, I consider this project a success. However, there are also many areas for improvement. First, I should have thought more about how to incorporate more interaction. Storytelling should not have been the main focus of my design, but I spent more time on designing the script and shooting. Secondly, the overall presentation lacks immersion, and the design of the “control console” may not be so obvious. Having too many buttons may make users feel confused. The most important lesson I learned is to conduct more user tests and continuously improve my project based on the feedback. As a designer, I definitely have preconceived ideas that make it difficult to discover the real usability issues. Additionally, chatting with other users can provide more inspiration.

E. 

F. 

Thanks very much for the help from the website Staff, LME Editorial. “In-Depth: What Is RFID? How It Works? Interface RC522 with Arduino.” Last Minute Engineers, 10 May 2023, lastminuteengineers.com/how-rfid-works-rc522-arduino-tutorial/. Accessed 13 May 2024. I have learned how to write code to identify RFID cards and the basic knowledge of RFID. 

I also would like to take this opportunity to once again express my deep gratitude to Professor Rodolfo Cossovich, LAs, and fellows who have provided me with tremendous help and ideas. I would also like to give a special thanks to my friends: Jenny, Isabel, Arial, Benjamin, Kitty, and Emily. I am incredibly grateful for their assistance throughout my project, whether it was through words of encouragement, guidance with coding, or providing insightful perspectives. Thank you so much to all of them!

 

 

 

Last Tree Standing 

                                          — —  Project Report

 

A.

Our project is called “Last Tree Standing.” The instructor is our excellent teacher, Rodolfo Cossovich.

This story tells about a world engulfed in desert during the apocalypse. As the savior of humanity, you are tasked with opening a time-space tunnel to travel back 1000 years and save the last remaining plant from destruction.

 

B.

Our definition of interaction is to allow the users to understand how to engage without direct prompts. We aim for the users to have authentic and meaningful experiences through this interaction. By chance, I came across an article about environmental conservation. Inspired by it, we decided to use our midterm project as a means to convey environmental protection concepts to others. During our research, we came across an existing interactive installation：

We found the interactive form of plant interaction fascinating. Initially, we had an idea about triggering leaf movement through touch, but it was later rejected due to technical reasons (which I will explain later). Then, I came up with the second idea of the time-travel tunnel. The tunnel requires crawling, and I believe it is an implied interaction that users will understand without direct instructions, reflecting my understanding of interaction. Additionally, I believe that a “large-scale” crawling installation adds uniqueness to my project.

 

C. 

This is our initial sketch, which was later replaced due to the unstable operation of the electromagnet.

This is our second sketch.

Because we want interaction to have a subtle impact without direct prompts, our designs aim to achieve this as much as possible. For example, our lighting design. The tunnel is a dark environment, and we have covered all the areas where light could leak with black tape, making the entire tunnel pitch black without any additional light sources. We designed a light strip along the walls to direct the light into the interior of the tunnel, guiding participants to crawl forward. We believe that humans are phototactic, meaning that in a dark environment, the instinct is to move towards the light. The light strip provides a continuous source of light compared to individual LEDs, so we chose the light strip. Additionally, since this is a “high-tech” tunnel, we believe that cool-colored light, such as a light blue, can evoke a sense of technology. Therefore, we selected a pale blue light strip.

Moreover, why did we choose to use a real plant instead of constructing a paper one? We believe that a real plant is more authentic, something that a cardboard-made plant cannot replicate. Although our current form of interaction does not involve direct interaction with the real plant but rather with a button, we still believe that by illuminating the real plant from above, we can deepen its sense of sanctity and evoke a sense of saving it in the minds of the participants. We do not want the entire project to be based on a virtual environment. Even though the time-space tunnel itself may be surreal, we want to convey to the participants that this endeavor is not devoid of meaning by incorporating elements of reality.

 

 

D. 

In the first half of this project, our main focus was on coding the plant interaction and dealing with the electromagnet. For the plant interaction, we defined a resistor as a capacity sensor, where touching the plant would change the capacitance. We used this to control the if statements. The code is as follows:

//Libraries
#include //https://github.com/PaulStoffregen/CapacitiveSensor
//Parameters

CapacitiveSensor s1 = CapacitiveSensor(2, 6);
long val1;


void setup() {
  //Init Serial USB

  Serial.begin(9600);
  pinMode(4,OUTPUT);


}
void loop() {

  digitalWrite(4, HIGH);
  val1 = s1.capacitiveSensor(30);
  Serial.println(val1);



   if (val1 > 1000 ) 
   {
      digitalWrite(4, LOW); 
   }
   else
   {
      digitalWrite(4,HIGH);
   }
} 

However, the electromagnet was the most challenging part. Its operation was highly unpredictable. Sometimes it would work during experiments but fail when connected to the circuit, which was incredibly frustrating.

This is the code for the electromagnet:

//Libraries
#include //https://github.com/PaulStoffregen/CapacitiveSensor
//Parameters
bool autocal  = 0;
const int numReadings  = 10;
long readings [numReadings];
int readIndex  = 0;
long total  = 0;
const int sensitivity  = 1000;
const int thresh  = 200;
const int csStep  = 10000;
CapacitiveSensor cs  = CapacitiveSensor(2, 6);

void setup() {
  //Init Serial USB
  Serial.begin(9600);
  Serial.println(F("Initialize System"));
  //Init cs
  if (autocal == 0) {
    {
      cs.set_CS_AutocaL_Millis(0xFFFFFFFF);
    }
  }

pinMode(4,OUTPUT);


}
void loop() {

  digitalWrite(4, HIGH);
  

  Serial.println(smooth());
   if (smooth() > 9000 ) 
   {
      digitalWrite(4, LOW); 
   }
   else
   {
      digitalWrite(4,HIGH);
   }
}
long smooth() { /* function smooth */
  ////Perform average on sensor readings
  long average;
  // subtract the last reading:
  total = total - readings[readIndex];
  // read the sensor:
  readings[readIndex] = cs.capacitiveSensor(sensitivity);
  // add value to total:
  total = total + readings[readIndex];
  // handle index
  readIndex = readIndex + 1;
  if (readIndex >= numReadings) {
    readIndex = 0;
  }
  // calculate the average:
  average = total / numReadings;
  return average;



} 

 

After realizing the instability of the electromagnet, we attempted to find alternative solutions. During the user testing process, our classmates and Rudi mentioned that they wanted more feedback. They didn’t understand why they needed to touch the plant, and after the paper leaves fell, they were unsure of what to do next and what it signified. We also noticed that, as Rudi pointed out, there wasn’t a strong connection between a real plant and a tree made of cardboard. Therefore, we needed to establish a connection. In this process, the idea of the time-space tunnel to save the plant was born.

In this new project, the most challenging part was controlling the process of the flower wilting. I wanted the effect where when a person enters the tunnel, they would see the flower gradually wilting, and when successfully rescued, the flower would stand upright again. The solution I chose was to use a fine thread motor for manipulation. I got inspiration for this from the fifth recitation.

 Here is my overall code:

#include 
#include 

Servo myservo;
int buttonPin = 3;
int buttonPressCount = 0;
int prevButtonState = LOW;
bool motorRunning = true;
unsigned long motorStartTime = 0;
int greenLedPin = 4;
int redLedPin = 5;
int buzzerPin = 6;

void setup() {
  Serial.begin(9600);
  myservo.attach(9);
  pinMode(buttonPin, INPUT_PULLUP);
  pinMode(greenLedPin, OUTPUT);
  pinMode(redLedPin, OUTPUT);
  pinMode(buzzerPin, OUTPUT);
}

void loop() {
  if (motorRunning) {
    if (millis() - motorStartTime >= 2000) {
      // 舵机运动的代码
      static int motorAngle = 0;

      if (motorAngle <= 160) { myservo.write(motorAngle); motorAngle += 20; } else { motorRunning = false; // 停止舵机运动 } motorStartTime = millis(); } } int buttonState = digitalRead(buttonPin); if (buttonState != prevButtonState && buttonState == HIGH) { delay(50); // 软件去抖动延迟 buttonState = digitalRead(buttonPin); // 重新读取按钮状态 if (buttonState == HIGH) { buttonPressCount++; Serial.print("Button pressed: "); Serial.println(buttonPressCount); if (buttonPressCount >= 11) {
        motorRunning = false;  // 停止舵机运动
        digitalWrite(greenLedPin, HIGH);  // 点亮绿灯
        digitalWrite(redLedPin, LOW);  // 熄灭红灯
        playFailMusic();
      }
    }
  }

  if (!motorRunning && buttonPressCount < 11) {
    digitalWrite(redLedPin, HIGH);  // 点亮红灯
    digitalWrite(greenLedPin, LOW);  // 熄灭绿灯
    playFailMusic();
  }

  prevButtonState = buttonState;

  delay(100);
}

void playFailMusic() {
  tone(buzzerPin, 247, 200);  // 发出音调（B3），持续200毫秒
  delay(250);  // 等待250毫秒
  tone(buzzerPin, 233, 200);  // 发出音调（降B3），持续200毫秒
  delay(250);  // 等待250毫秒
  tone(buzzerPin, 220, 200);  // 发出音调（A3），持续200毫秒
  delay(250);  // 等待250毫秒
  tone(buzzerPin, 208, 200);  // 发出音调（降A3），持续200毫秒
  delay(250);  // 等待250毫秒
  tone(buzzerPin, 196, 200);  // 发出音调（G3），持续200毫秒
  delay(250);  // 等待250毫秒
  tone(buzzerPin, 185, 200);  // 发出音调（降G3），持续200毫秒
  delay(250);  // 等待250毫秒
  noTone(buzzerPin);  // 停止播放音调
} 

To align with our overall topic, humans need to have love for plants in order to save them. We designed the button in the shape of a heart to symbolize this love.

To make the entire process more seamless, I designed a door curtain that opens the “space-time tunnel door” when the button is pressed. My partner selected the material for the door curtain, and I took charge of connecting the entire circuit and programming the motor.

 

Here is the code:

#include 


const int servo2Pin = 6; 
const int servo3Pin = 5; 

const int buttonPin = 2; 


Servo servo2;  
Servo servo3;  


void setup() {
  Serial.begin(9600);

  pinMode(buttonPin, INPUT_PULLUP);  

  servo2.attach(servo2Pin);  
  servo3.attach(servo3Pin);  
  servo2.write(0);          
  servo3.write(90);         
  delay(1000);
  servo2.detach();
  servo3.detach();
}

void loop() {
  //Serial.println(digitalRead(buttonPin));
  if (digitalRead(buttonPin) == LOW) {

    servo2.attach(servo2Pin);  
    servo3.attach(servo3Pin);  
    servo2.write(90);        
    servo3.write(0);          
    delay(1000);
    servo2.detach();
    servo3.detach();
  }



  delay(100);  
} 

After we completed the overall construction, we conducted user tests with eight individuals. They provided feedback that the tunnel may be a bit too narrow, making it inconvenient for people with larger body sizes to move around. Additionally, due to the tight seals, the lack of ventilation inside the tunnel made it feel stuffy and uncomfortable. After experiencing both failed and successful rescue attempts, they provided feedback that they were expecting more feedback. The two small LED lights were not sufficient to meet their expectations.

Therefore, I slightly widened the space. However, I still wanted the tunnel to create a sense of urgency for them, as they are “saving humanity” after all.Due to time constraints, I was unable to address the feedback regarding feedback after successful and failed attempts. However, I did relocate the LED lights to a more prominent position to make them more visible.

 

Due to my partner falling ill, I took on the responsibility of handling most of the construction and coding tasks. My partner, despite being unwell, contributed significantly to the decoration aspects of the project.

 

 

E. 

Since the entire tunnel is enclosed, the following videos are all filmed from a first-person perspective by my users.

 

 

Our project’s goal is to raise awareness about the importance of environmental conservation, so that events like saving plants by traveling through time don’t have to happen in the future. We are delighted to see that without any prompting, people automatically crawl into the tunnel and understand the need to press the buttons. In this regard, it aligns perfectly with our definition of interaction. However, there are several issues that could be addressed if we had more time. We need more visual feedback; currently, the results are only indicated by small LED lights and a buzzer. Given more time, a possible improvement would be to change the entire lighting environment, turning it green for success and red for failure.

Throughout the project, I have learned a lot. For instance, during circuit construction, I struggled with the messiness, which made it difficult to differentiate between components. Therefore, in future projects, I plan to use methods like color coding to distinguish between wires. From the failure of the electromagnetic coil, I learned that if something doesn’t work, it’s important not to get too fixated on it. Instead, either find an immediate replacement or “fake it till you make it.” It’s crucial not to waste excessive time on a single issue.

 

F. 

The above parts are the ones I took apart, and I will return them to the teachers who lent them to me.There are two buttons, a string of LED lights, 10 electromagnets, 3 Arduino boards, four transistors, four diodes, four servos, and two motors. 

 

G. 

I would like to take this opportunity to once again express my deep gratitude to Professor Rodolfo Cossovich, LAs, and fellows who have provided me with tremendous help and ideas. I would also like to give a special thanks to my friends: Jenny, Isabel, Arial, Benjamin, Kitty, and Emily. I am incredibly grateful for their assistance throughout my project, whether it was through words of encouragement, guidance with coding, or providing insightful perspectives. Thank you so much to all of them!