“GOD”

                                                                            — — Project Report

A.

The name of this project is “GOD.”

B.

It is an interactive video where players can interact with the machine through their choices, allowing the storyline to change according to the players’ desires. I believe interactivity is manifested in the fact that players can change the content of the video according to their own ideas.

 Although I provide them with limited types of choices, they can still make their own decisions. At the same time, when making choices in the video, the audience also receives feedback. For example, if the video content involves riding a bicycle, a fan next to the control console will be activated to simulate the cycling environment by blowing air at the players. In the video, there are pauses, and the characters in the video also hint at offline interactions through dialogue to change the video content.  However, in reality, during the user test, almost every user focused their attention on the screen and did not realize that they needed to interact to change the video. Through discussions with them, I discovered that it was a design problem. At that time, there was not a strong association between the control console and the video on the screen, and there was not enough information to direct the players’ attention to the interactive objects I had 3D printed. So, first, I placed the control console closer to the computer screen. Second, I made some improvements in the video content by placing more prominent letters as prompts. In the second round of user testing, it worked much better, and testers became aware that they needed to make choices.

C.

Regarding the construction part, I focused more on the implementation of processing rather than building Arduino. Because I believe the main form of presentation for my project is the interactive video, where the interaction comes from the audience’s choices. This part does not require a lot of Arduino knowledge; therefore, I focused more on coding in processing. The most challenging part of Arduino was how to make the system aware of the audience’s choices and control the video content changes.

I thought of the NFC chip in the campus card, which can distinguish each recognized object and make corresponding choices. Therefore, I used the RFID-RC522 recognition system and attached RFID stickers to each object that needed recognition.

By setting different programs for different chips in Arduino, I achieved the effect of changing the video. In the interaction between processing and Arduino, I used a temperature sensor. The idea was that when players blew air at the temperature sensor attached to the rear of the bicycle, the temperature would rise, causing the bicycle’s speed in the video to increase, and the fan would start working to simulate the cycling environment.

#include 
#include 
#define SS_PIN 10
#define RST_PIN 5

MFRC522 mfrc522(SS_PIN, RST_PIN);
const int motorPin = 9;  // 控制电机的引脚
void setup() {
  Serial.begin(9600);
  SPI.begin();
  mfrc522.PCD_Init();
  Serial.println("Scan an RFID card or wait for temperature reading...");
  pinMode(motorPin, OUTPUT);
}

void loop() {
  if (mfrc522.PICC_IsNewCardPresent() && mfrc522.PICC_ReadCardSerial()) {
    byte uid[10];
    byte uidLength = mfrc522.uid.size;
    for (byte i = 0; i < uidLength; i++) {
      uid[i] = mfrc522.uid.uidByte[i];
    }
    String cardID = "";
    for (byte i = 0; i < uidLength; i++) {
      if (uid[i] < 0x10) { cardID += "0"; } cardID += String(uid[i], HEX); } cardID.toUpperCase(); Serial.print("RFID Card ID: "); Serial.println(cardID); mfrc522.PICC_HaltA(); mfrc522.PCD_StopCrypto1(); } int temperatureValue = analogRead(A0); Serial.print("Temperature: "); Serial.println(temperatureValue); if (temperatureValue > 50) {
    // 温度传感器数值大于100，启动电机
    digitalWrite(motorPin, HIGH);  // 设定电机引脚为高电平
  } else {
    // 温度传感器数值小于或等于100，停止电机
    digitalWrite(motorPin, LOW);  // 设定电机引脚为低电平
  }

  delay(300);
} 

In processing, the difficulty I encountered was the continuous playback of the video. Playing different videos based on different conditions resulted in many strange issues, such as the video not starting from the beginning or playing without sound.（one of the videos:

It took me a whole 17 hours to debug. Later, with Rudi’s guidance, I used arrays to write the code, making the logic very smooth, and successfully achieved the complete playback of the video.

String state="Instruction";
import processing.video.*;
Movie movies[]=new Movie[10];
ArrayList<Movie>ms=new ArrayList();
String movieName[]=new String[10];
int mw, mh;
boolean left=true;
String id[]={"3DCC1EDE", "5DCB1EDE", "2DCC1EDE"};
//自行车 水瓶 伞
String getId="";
import processing.serial.*;

Serial port;
String data = "";
int temp=0;

void settings(){
  fullScreen(P2D);
}

void setup() {
  port = new Serial(this, Serial.list()[0], 9600);

  println(Serial.list()[0]);
  
  frameRate(60);
  mh=int(height*0.8);
  mw=int(mh*1.77);
  ms.add(new Movie(this, "Introduction.mp4"));
  movieName[0]= "Instruction";
  for (int i=1; i<movies.length; i++) {
    ms.add(new Movie(this, char(i+64)+".mp4"));
    movieName[i]=  char(i+64)+"";
  }

  imageMode(CENTER);
}
void processData(String data) {
  if (data.startsWith("RFID Card ID")) {
    //println("Received RFID Card ID: " + data.substring(14));
    getId= data.substring(14);
  } else if (data.startsWith("Temperature")) {
    println("Received Temperature: " + data.substring(12));
    temp=int(data.substring(12).trim());
  }
}
void draw() {
  while (port.available() > 0) {
    char received = (char) port.read();
    if (received == '\n') {
      processData(data);
      data = "";
    } else {
      data += received;
    }
  }
  println(state, temp);
  if (state=="A"&&getId.trim().equals(id[2])) {
    left=false;
  }
  if (state=="E"&&getId.trim().equals(id[1])) {
    left=true;
  } else if (state=="E") {
    left=false;
  }
  background(0);
  for (int i=0; i<ms.size(); i++) {
    Movie m=ms.get(i);
    if (state.equals(movieName[i])) {
      if (m.isPlaying()==false) {
        m.play();
      }
      if (state=="D"&&temp>150) {
        m.speed(3);
      } else if (state=="D") {
        m.speed(1);
      }
      image(m, width/2, height/2, mw, mh);

      if (m.time()==m.duration()) {
        m.stop();
        m.speed(1);
        videoControl();
      }
    } else if (m.isPlaying()==true) {
      m.stop();
      m.speed(1);
    }
  }
}
void mousePressed() {
  if (state=="Instruction") {
    state="A";
  }
}
void videoControl() {
  if (state=="Instruction") {
    movies[0].jump(0);
    movies[0].play();
  } 
  else if (state=="A") {
    if (left) {
      state="B";
    } else {
      state="C";
    }
  } 
  else if (state=="B"||state=="C") {
    state="D";
  }
  else if (state=="D" ) {
    state="E";
  } 
  else if (state=="E" ) {
    if (left) {
      state="F";
    } else {
      state="G";
    }
  } 
  else if (state=="F"||state=="G") {
    state="H";
  } 
  else if (state=="H") {
    state="I";
  } 
  else if (state=="I") {
    state="Z";
  }
}
void movieEvent(Movie m) {
  m.read();
}

 

D.

I initially came up with this idea based on my personal experience. During the final week, things kept coming at me, and I felt like I was being pushed forward every day. My life seemed to be controlled by others. So I thought, why not control someone else’s life? That’s how I came up with the idea of creating this interactive video to control the movements of the characters in the video.

From this perspective, I consider this project a success. However, there are also many areas for improvement. First, I should have thought more about how to incorporate more interaction. Storytelling should not have been the main focus of my design, but I spent more time on designing the script and shooting. Secondly, the overall presentation lacks immersion, and the design of the “control console” may not be so obvious. Having too many buttons may make users feel confused. The most important lesson I learned is to conduct more user tests and continuously improve my project based on the feedback. As a designer, I definitely have preconceived ideas that make it difficult to discover the real usability issues. Additionally, chatting with other users can provide more inspiration.

E. 

F. 

Thanks very much for the help from the website Staff, LME Editorial. “In-Depth: What Is RFID? How It Works? Interface RC522 with Arduino.” Last Minute Engineers, 10 May 2023, lastminuteengineers.com/how-rfid-works-rc522-arduino-tutorial/. Accessed 13 May 2024. I have learned how to write code to identify RFID cards and the basic knowledge of RFID. 

I also would like to take this opportunity to once again express my deep gratitude to Professor Rodolfo Cossovich, LAs, and fellows who have provided me with tremendous help and ideas. I would also like to give a special thanks to my friends: Jenny, Isabel, Arial, Benjamin, Kitty, and Emily. I am incredibly grateful for their assistance throughout my project, whether it was through words of encouragement, guidance with coding, or providing insightful perspectives. Thank you so much to all of them!