A. TITLE
Rhythm Master – Siwei Chen(Partner)& Siwen Fang (Wendy) – Instructor: Gottfried Haider
B. CONCEPTION AND DESIGN:
I want to design a fun music game where players would react in various ways to the drums and lyrics of the music. In the pre-research, I learned that there are such games(Reference: https://en.wikipedia.org/wiki/Rhythm_game), and I hope people can use body language, interesting props, and sensors to make the game more interactive. In addition, we choose the prop: whistle, because it can make the user looks silly and make a special sound for the computer to catch its frequency and act accordingly. (Very, unfortunately, to protect the user, they cannot move their mask and try it). We added two types of Neopixel, the 8cmX8cm one will direct the user to press the button, and the strip will change its color with a different music rhythm, when the whistle is blown, it will turn all white.
We got a lot of suggestions during the user test, some people thought it would be very confusing to start the game straight away, so we added an instruction part. Some people thought the game was too hard to get involved in, so we set the game into three levels: easy, medium, and hard. Professor Rudy felt that exposing the Neopixel directly to the outside was very unattractive, so we taped A4 paper and fixed it with black tape to make it look better. In the beginning, the stars on the computer screen were all white, which some people thought was not good for the player to play the game, so we modified it so that they corresponded to the color of the buttons one by one. They all make our game more interesting and easier for players to play, and I think these improvements are very successful.
C. FABRICATION AND PRODUCTION:
We plan to use Arduino to control four buttons, an 8×8 LED board, and Neopixel. In order to make the effect of colored lights more obvious, I used three colored lights, at first I connected them all together and then connected them to Arduino. The consequence is that the first light is very bright, the last light is very dark, and the Arduino eventually broke because of overload. So I divided the lights into three sections and connected them to the circuit. I also had to change the Arduino coding part to make sure every Neopixel can receive the right coding. When I was rewriting the processing code, the random change of the lighted position at the beginning did not have the sound wave effect, so I changed the code. When preparing for hard mode, I set the higher the frequency, the redder the light will be. But since there would be no color change if the frequency was always high, I made the value controlling the blue color randomly. I also found that when I reset the Arduino, it reports an error if I don’t re-download the library. When I was making the laser cutting carton, I needed to determine the thickness of the boards and to avoid using a glue gun, I measured each board precisely so that they would snap right into each other. The hardest part was that we wanted to use the sound sensor at first, but it wasn’t sensitive. And when I put the sound sensor code together with my Neopixel, the light doesn’t work. I don’t know why, and I didn’t find a good solution after asking my professor. Finally, I chose to use processing and the microphone that comes with the computer to detect it. To distinguish the loudness of the music, I need to find the frequency of the whistle separately and record it, setting its maximum frequency to differentiate it precisely.
The Game part is the most important part of our project, to automatically capture rhythms and reach correct beat points, we originally wanted to compute each song’s BPM and then construct a for loop. However, the beat points are the same, which can be overly repetitive and simplistic. It is also difficult to get the computer to analyze the music, we need to record the drums of the music ourselves and tell the computer when to react. Therefore, we manually input the rhythm (using “println” to record the beat), which may result in some rhythmic inaccuracies but allows for varied beat points. To enhance the gaming experience, we set the judgment interval to one second, meaning that pressing the button within one second of the beat point will be considered successful, but pressing it outside of that time will be considered unsuccessful. We started by finishing the Processing portion, which can already be played as a small rhythm game because we can use the keyboard.
The screen part is the same as our previous idea, there is a red heart in the middle of the screen protected by a white circle, and four stars will appear from the four corners, when the stars touch the white line a moment, the player should press the button to protect the red heart.
Then came the most struggling part, we changed the usage of the keyboard into buttons(we have no choice of color because they are the only 4 works in ER), but how can we connect the Arduino with Processing? In order to remind the player before each star crosses the white line, we must first send a signal to the Arduino. If the player presses a button that faces the star’s direction, the Arduino must then send a message to Processing, which will be inserted into the judgment interval to determine whether the player succeeds. My partner created numerous array definitions to finish this section and finally it is successful(Thank God).
It would be challenging for players who are unfamiliar with how a game works or have never played one to control it by gazing at the computer screen rather than the buttons. Therefore, we sought to synchronize the LED board in the button’s center with the stars on the computer screen (only the direction of movement is opposite because the LED moves from the center to the button. During the user test, we also got some feedback because some people think it is confusing while others don’t think so, so we didn’t make changes). The hardest part of this is that we had no prior knowledge of the Neopixel 8*8 board. We measured the order of the LEDs before adding some fundamental adjustments, including blink and fade.
With the help of the professor and my friend, we listed 20 cases to divide the situation and used “case” and “break” to control the movement of the LEDs (this step took a long time). After many attempts, we basically finished the Arduino part of the programming. But we still have some problems. The first problem is in the Processing part, at first we assumed that if we miss a small ball, a big fork will appear on the screen, but for some reason, this fork appears very randomly, we asked the professor but couldn’t find the reason, so we finally removed it and added a score table in the top left corner as an alternative. Another problem is the Neopixel board, our code is built on the basis that the next LED will glow only after the previous one finishes glowing and goes off, sometimes the LED will somehow get stuck in one place and not move, causing the whole LED board to go wrong. After testing countless times (very painful), my partner Siwei came up with the final solution: reduce the beat points (not too many beat points in the Processing, otherwise the Arduino part will have problems); limit the song length.
Coding in Arduino for the Neopixel strips:
#include "SerialRecord.h"
#include
#define NUM_LEDS 60 // How many leds in your strip?
#define DATA_PINA 3
#define DATA_PINB 4
#define DATA_PINC 5
// Which pin are you connecting Arduino to Data In?
CRGB leds[NUM_LEDS];
// Change this number to the number of values you want to receive
SerialRecord reader(4);
void setup ()
{
Serial.begin(9600);
FastLED.addLeds<NEOPIXEL, DATA_PINA>(leds, NUM_LEDS); // Initialize
FastLED.addLeds<NEOPIXEL, DATA_PINB>(leds, NUM_LEDS);
FastLED.addLeds<NEOPIXEL, DATA_PINC>(leds, NUM_LEDS);
FastLED.setBrightness(10); // BEWARE: external power for full (255)
}
void loop() {
if (reader.read()) {
int n = reader[0];
int r = reader[1];
int g = reader[2];
int b = reader[3];
leds[reader[0]] = CRGB(reader[1], reader[2], reader[3]); // Prepare the color information using CRGB( Red, Green, Blue
FastLED.show(); // Pass the information of color to the LED
}
}
Coding in Arduino for the buttons and the Neopixel board:
#include "SerialRecord.h"
#include
int pre_state1 = LOW;
int pre_state2 = LOW;
int pre_state3 = LOW;
int pre_state4 = LOW;
int buttonState1=0;
int buttonState2=0;
int buttonState3=0;
int buttonState4=0;
SerialRecord reader(1);
SerialRecord writer(1);
int buttona=2;
int buttonb=3;
int buttonc=4;
int buttond=5;
int button1=6;
int button2=7;
int button3=8;
int button4=9;
int led1 = 10;
int led2 = 11;
int led3 = 12;
int led4 = 13;
int beats[88]= {
39,
109,
175,
243,
312,
377,
446,
513,
580,
645,
716,
782,
849,
915,
982,
1053,
1086,
1119,
1153,
1186,
1221,
1255,
1288,
1321,
1355,
1388,
1422,
1457,
1491,
1524,
1558,
1592,
1627,
1658,
1693,
1724,
1757,
1791,
1826,
1858,
1883,
1927,
1960,
1994,
2028,
2063,
2095,
2129,
2162,
2196,
2231,
2263,
2297,
2331,
2365,
2398,
2433,
2467,
2501,
2532,
2567,
2601,
2635,
2669,
2702,
2736,
2771,
2802,
2836,
2869,
2902,
2937,
2970,
2987,
3004,
3021,
3038,
3054,
3072,
3088,
3105,
3122,
3139,
3156,
3173,
3190,
3208,
3225,
};
void setup() {
Serial.begin(9600);
pinMode(A0,INPUT);
pinMode(button1, INPUT_PULLUP);
pinMode(button2, INPUT_PULLUP);
pinMode(button3, INPUT_PULLUP);
pinMode(button4, INPUT_PULLUP);
pinMode(buttona, OUTPUT);
pinMode(buttonb, OUTPUT);
pinMode(buttonc, OUTPUT);
pinMode(buttond, OUTPUT);
//frameRate(60);
}
void loop() {
reader.read();
buttonState1 = digitalRead(button1);
buttonState2 = digitalRead(button2);
buttonState3 = digitalRead(button3);
buttonState4 = digitalRead(button4);
if(reader[0]==1){
digitalWrite(2,HIGH);
}
if(reader[0]==2){
digitalWrite(3,HIGH);
}
if(reader[0]==3){
digitalWrite(4,HIGH);
}
if(reader[0]==4){
digitalWrite(5,HIGH);
}
if(reader[0]==5){
digitalWrite(2,LOW);
digitalWrite(3,LOW);
digitalWrite(4,LOW);
digitalWrite(5,LOW);
}
if (!buttonState1 == HIGH && buttonState1 != pre_state1) {
writer[0] = 1;
writer.send();
}
if (!buttonState2 == HIGH && buttonState2 != pre_state2) {
writer[0] = 2;
writer.send();
}
if (!buttonState3 == HIGH && buttonState3 != pre_state3) {
writer[0] = 3;
writer.send();
}
if (!buttonState4 == HIGH && buttonState4 != pre_state4 ) {
writer[0] = 4;
writer.send();
}
pre_state1 = buttonState1;
pre_state2 = buttonState2;
pre_state3 = buttonState3;
pre_state4 = buttonState4;
/*for(int i=0; i<30; i++){
leds[0] = CRGB(255, 0, 0);
FastLED.show();
}*/
}
Coding in Processing for the Neopixel Strips: ( the coding changes with the file name and different coding for the RGB)
Instruction part:
import processing.sound.*;
import processing.serial.*;
import osteele.processing.SerialRecord.*;
import processing.sound.*;
// declare an AudioIn object
AudioIn microphone;
// declare an Frequency analysis object to detect the frequencies in a sound
FFT freqAnalysis;
// declare a variable for the amount of frequencies to analyze
// should be a multiple of 64 for best results
int frequencies = 1024;
// Define the frequencies wanted for our visualization. Above that treshold frequencies are rarely atteigned and stay flat.
int freqWanted = 128;
// declare an array to store the frequency anlysis results in
float[] spectrum = new float[freqWanted];
// Declare a drawing variable for calculating the width of the
float circleWidth;
Serial serialPort;
SerialRecord serialRecord;
SoundFile sample;
Amplitude analysis;
int NUM = 60; //amount of pixels
int[] r = new int[NUM]; //red of each tile
int[] g = new int[NUM]; //red of each tile
int[] b = new int[NUM]; //red of each tile
int startTime;
void setup() {
fullScreen();
// load and play a sound file in a loop
sample = new SoundFile(this, "g.mp3");
sample.loop();
// create the Amplitude analysis object
analysis = new Amplitude(this);
// analyze the playing sound file
analysis.input(sample);
String serialPortName = SerialUtils.findArduinoPort();
serialPort = new Serial(this, serialPortName, 9600);
serialRecord = new SerialRecord(this, serialPort, 4);
serialRecord.logToCanvas(false);
rectMode(CENTER);
startTime = millis();
circleWidth = width/float(freqWanted);
// create the AudioIn object and select the mic as the input stream
microphone = new AudioIn(this, 0);
// start the mic input without routing it to the speakers
microphone.start();
// create the Frequency analysis object and tell it how many frequencies to analyze
freqAnalysis = new FFT(this, frequencies);
// use the microphone as the input for the analysis
freqAnalysis.input(microphone);
}
void draw() {
println(analysis.analyze());
long t = millis() - startTime;
if (t < 1500) {
background(125, 255, 125);
} else if ((t>=1500) && t<3000) {
background(255);
} else {
background(55);
}
noStroke();
fill(255, 0, 150);
// analyze the audio for its volume level
float volume = analysis.analyze();
// volume is a number between 0.0 and 1.0
// map the volume value to a useful scale
float diameter = map(volume, 0, 1, 0, width);
float light = map(volume, 0, 1, 0, 255);
//int n = int(random(60));
int n = frameCount % 60;
r[n] = floor(light);
g[n] = floor(255-light);
b[n] = 0;
serialRecord.values[0] = n; // which pixel we change (0-59)
serialRecord.values[1] = r[n]; // how much red (0-255)
serialRecord.values[2] = g[n]; // how much green (0-255)
serialRecord.values[3] = b[n]; // how much blue (0-255)
serialRecord.send();
// draw a circle based on the microphone amplitude (volume)
circle(width/2, height/2, diameter);
// analyze the frequency spectrum and store it in the array
freqAnalysis.analyze(spectrum);
//printArray(spectrum);
background(0);
fill(200, 0, 100, 150);
noStroke();
float loudest = 0;
int loudestIdx = 0;
for (int i=0; i<freqWanted; i++) {
if (spectrum[i] > loudest) {
loudest = spectrum[i];
loudestIdx = i;
}
}
if ((loudestIdx == 19 || loudestIdx == 20) && loudest > 0.02) {
background(255,255,255);
int n1 = frameCount % 60;
r[n1] = floor(255);
g[n1] = floor(255);
b[n1] = floor(255);
serialRecord.values[0] = n1; // which pixel we change (0-59)
serialRecord.values[1] = r[n1]; // how much red (0-255)
serialRecord.values[2] = g[n1]; // how much green (0-255)
serialRecord.values[3] = b[n1]; // how much blue (0-255)// how much blue (0-255)
serialRecord.send(); // send it!
}
if (loudest > 0.01) {
println(loudestIdx + " @ " + loudest);
}
// draw circles based on the values stored in the spectrum array
// adjust the scale variable as needed
float scale = 600;
for (int i=0; i<freqWanted; i++) {
circle(i*circleWidth, height/2, spectrum[i]*scale);
}
}
Medium model:
import processing.sound.*;
import processing.serial.*;
import osteele.processing.SerialRecord.*;
import processing.sound.*;
// declare an AudioIn object
AudioIn microphone;
// declare an Frequency analysis object to detect the frequencies in a sound
FFT freqAnalysis;
// declare a variable for the amount of frequencies to analyze
// should be a multiple of 64 for best results
int frequencies = 1024;
// Define the frequencies wanted for our visualization. Above that treshold frequencies are rarely atteigned and stay flat.
int freqWanted = 128;
// declare an array to store the frequency anlysis results in
float[] spectrum = new float[freqWanted];
// Declare a drawing variable for calculating the width of the
float circleWidth;
Serial serialPort;
SerialRecord serialRecord;
SoundFile sample;
Amplitude analysis;
int NUM = 60; //amount of pixels
int[] r = new int[NUM]; //red of each tile
int[] g = new int[NUM]; //red of each tile
int[] b = new int[NUM]; //red of each tile
int startTime;
void setup() {
fullScreen();
// load and play a sound file in a loop
sample = new SoundFile(this, "t.mp3");
sample.loop();
// create the Amplitude analysis object
analysis = new Amplitude(this);
// analyze the playing sound file
analysis.input(sample);
String serialPortName = SerialUtils.findArduinoPort();
serialPort = new Serial(this, serialPortName, 9600);
serialRecord = new SerialRecord(this, serialPort, 4);
serialRecord.logToCanvas(false);
rectMode(CENTER);
startTime = millis();
circleWidth = width/float(freqWanted);
// create the AudioIn object and select the mic as the input stream
microphone = new AudioIn(this, 0);
// start the mic input without routing it to the speakers
microphone.start();
// create the Frequency analysis object and tell it how many frequencies to analyze
freqAnalysis = new FFT(this, frequencies);
// use the microphone as the input for the analysis
freqAnalysis.input(microphone);
}
void draw() {
println(analysis.analyze());
long t = millis() - startTime;
if (t < 1500) {
background(125, 255, 125);
} else if ((t>=1500) && t<3000) {
background(255);
} else {
background(55);
}
noStroke();
fill(255, 0, 150);
// analyze the audio for its volume level
float volume = analysis.analyze();
// volume is a number between 0.0 and 1.0
// map the volume value to a useful scale
float diameter = map(volume, 0, 1, 0, width);
float light = map(volume, 0, 1, 0, 255);
//int n = int(random(60));
int n = frameCount % 60;
r[n] = floor(light);
g[n] = floor(255-light);
b[n] = 0;
serialRecord.values[0] = n; // which pixel we change (0-59)
serialRecord.values[1] = r[n]; // how much red (0-255)
serialRecord.values[2] = g[n]; // how much green (0-255)
serialRecord.values[3] = b[n]; // how much blue (0-255)
serialRecord.send();
// draw a circle based on the microphone amplitude (volume)
circle(width/2, height/2, diameter);
// analyze the frequency spectrum and store it in the array
freqAnalysis.analyze(spectrum);
//printArray(spectrum);
background(0);
fill(200, 0, 100, 150);
noStroke();
float loudest = 0;
int loudestIdx = 0;
for (int i=0; i<freqWanted; i++) {
if (spectrum[i] > loudest) {
loudest = spectrum[i];
loudestIdx = i;
}
}
if ((loudestIdx == 19 || loudestIdx == 20) && loudest > 0.02) {
background(255,255,255);
int n1 = frameCount % 60;
r[n1] = floor(255);
g[n1] = floor(255);
b[n1] = floor(255);
serialRecord.values[0] = n1; // which pixel we change (0-59)
serialRecord.values[1] = r[n1]; // how much red (0-255)
serialRecord.values[2] = g[n1]; // how much green (0-255)
serialRecord.values[3] = b[n1]; // how much blue (0-255)// how much blue (0-255)
serialRecord.send(); // send it!
}
if (loudest > 0.01) {
println(loudestIdx + " @ " + loudest);
}
// draw circles based on the values stored in the spectrum array
// adjust the scale variable as needed
float scale = 600;
for (int i=0; i<freqWanted; i++) {
circle(i*circleWidth, height/2, spectrum[i]*scale);
}
}
Hard model:
import processing.sound.*;
import processing.serial.*;
import osteele.processing.SerialRecord.*;
import processing.sound.*;
// declare an AudioIn object
AudioIn microphone;
// declare an Frequency analysis object to detect the frequencies in a sound
FFT freqAnalysis;
// declare a variable for the amount of frequencies to analyze
// should be a multiple of 64 for best results
int frequencies = 1024;
// Define the frequencies wanted for our visualization. Above that treshold frequencies are rarely atteigned and stay flat.
int freqWanted = 128;
// declare an array to store the frequency anlysis results in
float[] spectrum = new float[freqWanted];
// Declare a drawing variable for calculating the width of the
float circleWidth;
float a;
Serial serialPort;
SerialRecord serialRecord;
SoundFile sample;
Amplitude analysis;
int NUM = 60; //amount of pixels
int[] r = new int[NUM]; //red of each tile
int[] g = new int[NUM]; //red of each tile
int[] b = new int[NUM]; //red of each tile
int startTime;
void setup() {
fullScreen();
// load and play a sound file in a loop
sample = new SoundFile(this, "t.wav");
sample.loop();
// create the Amplitude analysis object
analysis = new Amplitude(this);
// analyze the playing sound file
analysis.input(sample);
String serialPortName = SerialUtils.findArduinoPort();
serialPort = new Serial(this, serialPortName, 9600);
serialRecord = new SerialRecord(this, serialPort, 4);
serialRecord.logToCanvas(false);
rectMode(CENTER);
startTime = millis();
circleWidth = width/float(freqWanted);
// create the AudioIn object and select the mic as the input stream
microphone = new AudioIn(this, 0);
// start the mic input without routing it to the speakers
microphone.start();
// create the Frequency analysis object and tell it how many frequencies to analyze
freqAnalysis = new FFT(this, frequencies);
// use the microphone as the input for the analysis
freqAnalysis.input(microphone);
}
void draw() {
println(analysis.analyze());
long t = millis() - startTime;
if (t < 1500) {
background(125, 255, 125);
} else if ((t>=1500) && t<3000) {
background(255);
} else {
background(55);
}
noStroke();
fill(255, 0, 150);
// analyze the audio for its volume level
float volume = analysis.analyze();
// volume is a number between 0.0 and 1.0
// map the volume value to a useful scale
float diameter = map(volume, 0, 1, 0, width);
float light = map(volume, 0, 1, 0, 255);
//int n = int(random(60));
a = random(0, 255);
int n = frameCount % 60;
r[n] = floor(light/2);
g[n] = floor(255-light);
b[n] = floor(a);
serialRecord.values[0] = n; // which pixel we change (0-59)
serialRecord.values[1] = r[n]; // how much red (0-255)
serialRecord.values[2] = g[n]; // how much green (0-255)
serialRecord.values[3] = b[n]; // how much blue (0-255)
serialRecord.send();
// draw a circle based on the microphone amplitude (volume)
circle(width/2, height/2, diameter);
// analyze the frequency spectrum and store it in the array
freqAnalysis.analyze(spectrum);
//printArray(spectrum);
background(0);
fill(200, 0, 100, 150);
noStroke();
float loudest = 0;
int loudestIdx = 0;
for (int i=0; i<freqWanted; i++) {
if (spectrum[i] > loudest) {
loudest = spectrum[i];
loudestIdx = i;
}
}
if ((loudestIdx == 19 || loudestIdx == 20) && loudest > 0.02) {
background(255,255,255);
int n1 = frameCount % 60;
r[n1] = floor(255);
g[n1] = floor(255);
b[n1] = floor(255);
serialRecord.values[0] = n1; // which pixel we change (0-59)
serialRecord.values[1] = r[n1]; // how much red (0-255)
serialRecord.values[2] = g[n1]; // how much green (0-255)
serialRecord.values[3] = b[n1]; // how much blue (0-255)// how much blue (0-255)
serialRecord.send(); // send it!
}
if (loudest > 0.01) {
println(loudestIdx + " @ " + loudest);
}
// draw circles based on the values stored in the spectrum array
// adjust the scale variable as needed
float scale = 600;
for (int i=0; i<freqWanted; i++) {
circle(i*circleWidth, height/2, spectrum[i]*scale);
}
}
Coding in Processing for the screen, buttons and Neopixel board:
Instruction:
import processing.sound.*;
import processing.serial.*;
import osteele.processing.SerialRecord.*;
Serial serialPort;
SerialRecord serialRecord;
SoundFile sound;
boolean[] flag=new boolean[18];
boolean[] flaga=new boolean[18];
boolean[] flagb=new boolean[18];
boolean[] flagc=new boolean[18];
boolean[] sendstate=new boolean[18];
int playStart;
boolean success=false;
int premil=0;
int value=0;
int num=0;
int[]beats= {
752,
863,
979,
1090,
1486,
1543,
1598,
1653,
2337,
2506,
2677,
3004,
3059,
3116,
3172,
3229,
3284,
3339,
};
int[] xpos=new int[18];
int[] ypos=new int[18];
int x;
int y;
int[] ball=new int[18];
int a=0;
void star(float x1, float y1, float radius1, float radius2, int npoints) { //ball=start
float angle = TWO_PI / npoints;
float halfAngle = angle/2.0;
beginShape();
for (float a = 0; a < TWO_PI; a += angle) {
float sx = x1 + cos(a) * radius2;
float sy = y1 + sin(a) * radius2;
vertex(sx, sy);
sx = x1 + cos(a+halfAngle) * radius1;
sy = y1 + sin(a+halfAngle) * radius1;
vertex(sx, sy);
}
endShape(CLOSE);
}
void keyPressed() {
println(frameCount+",");
//println(millis());
}
void drawBigHeart() { // if player press the button correctly, the heart becomes bigheart
fill(255, 0, 0);
rectMode(CENTER);
rect(height/2, width/2, 10, 10);
rect(height/2-10, width/2, 10, 10);
rect(height/2+10, width/2, 10, 10);
rect(height/2-20, width/2, 10, 10);
rect(height/2+20, width/2, 10, 10);
rect(height/2-30, width/2, 10, 10);
rect(height/2+30, width/2, 10, 10);
rect(height/2+10, width/2-10, 10, 10);
rect(height/2-10, width/2-10, 10, 10);
rect(height/2+20, width/2-10, 10, 10);
rect(height/2-20, width/2-10, 10, 10);
rect(height/2-15, width/2-20, 10, 10);
rect(height/2+15, width/2-20, 10, 10);
rect(height/2, width/2+10, 10, 10);
rect(height/2-10, width/2+10, 10, 10);
rect(height/2+10, width/2+10, 10, 10);
rect(height/2-20, width/2+10, 10, 10);
rect(height/2+20, width/2+10, 10, 10);
rect(height/2, width/2+20, 10, 10);
rect(height/2-10, width/2+20, 10, 10);
rect(height/2+10, width/2+20, 10, 10);
rect(height/2, width/2+30, 10, 10);
}
void drawHeart() { // normal heart
fill(255, 0, 0);
rect(height/2, width/2, 10, 10);
rect(height/2-10, width/2, 10, 10);
rect(height/2+10, width/2, 10, 10);
rect(height/2-20, width/2, 10, 10);
rect(height/2+20, width/2, 10, 10);
rect(height/2+10, width/2-10, 10, 10);
rect(height/2-10, width/2-10, 10, 10);
rect(height/2, width/2+10, 10, 10);
rect(height/2-10, width/2+10, 10, 10);
rect(height/2+10, width/2+10, 10, 10);
rect(height/2, width/2+20, 10, 10);
}
void setup() {
size(800, 800);
frameRate(60);
delay(1000);
sound = new SoundFile(this, "g.mp3");
playStart = millis();
sound.play();
noStroke(); // the text
textSize(30);
textAlign(CENTER, CENTER);
String serialPortName = SerialUtils.findArduinoPort();
serialPort = new Serial(this, serialPortName, 115200);
serialRecord = new SerialRecord(this, serialPort, 1);
for (int w=0; w<18; w++) {
sendstate[w]=true;
}
}
void key(int k) { //key is controlled by buttons
//println((frameCount - playStart)+",");
for (int i=0; i<18; i++) { //judgement
if (abs(frameCount-beats[i])<10&&ball[i]==k&&!flaga[i]) {
success=true; //if press button correctly, the heart become bigger
//println("success=true");
premil=millis();
a=a+1;
flaga[i]=true;
}
}
}
void draw() {
background(0);
fill(255, 100);
ellipse(400, 400, 180, 180);
fill(0);
ellipse(400, 400, 160, 160);
rectMode(CENTER);
drawHeart(); //heart
if (serialRecord.read()) {
value = serialRecord.get();
key(value);
//println(frameCount);
}
if (success==true) {
if (millis()-premil<300) {
fill(255);
ellipse(400, 400, 180, 180);
fill(0);
ellipse(400, 400, 160, 160);
drawBigHeart();
} else {
success=false;
}
} else drawHeart();
int now = millis() - playStart;
push();
if (now > 2000 && now < 5000) {
clear();
textSize(96);
fill(255, 255, 255);
text("instruction", height/2, width/2);
}
pop();
if (now > 5000 && now < 8783) {
clear();
fill(255, 255, 255);
text("Pressed Button when the star hits White Line", height/2, width/2);
}
if (now > 20066 && now < 23850) {
clear();
fill(255, 255, 255);
text("You can also look at the pixel LED in the middle of the box", height/2, width/2-32);
text("LEDs and the stars are moving in sync", height/2, width/2);
text("Press button when LED Reach the Edge", height/2, width/2+32);
}
if (now > 29400 && now < 34500) {
clear();
fill(255, 255, 255);
text("If you press correctly,", height/2, width/2-32);
text("the heart becomes bigger after each pressing", height/2, width/2);
text("Your score is displayed in the top left corner", height/2, width/2+32);
}
if (now > 32500) {
fill(255);
text("Your score is:", 125, 65);
text(a, 100, 100);
}
if (now >34500 && now < 37800) {
clear();
fill(255, 255, 255);
text("If the star is", height/2, width/2-32);
fill(#FFF534);
text("YELLOW", height/2, width/2);
fill(255);
text("Use Whistle and Press Button at the same time! ", height/2, width/2+32);
}
push();
if (now>46533&&now<48350) {
clear();
fill(255);
textSize(96);
text(" ENJOY PLAYING !", height/2, width/2);
}
pop();
if(frameCount>3500){
clear();
fill(255);
text("This is the end of the tutorial section",height/2, width/2);
}
//ball
int cfc=frameCount;
for (int i=max(0, num-2); i<18; i++) {
if (cfc-beats[i]>-31&&!flag[i]) {
fill(255);
if (abs(cfc-(beats[i]-31))<2) {
xpos[i]=800*floor(random(0, 2));
ypos[i]=800*floor(random(0, 2));
if (xpos[i]==0&&ypos[i]==0) {
ball[i]=1;
if (sendstate[i]==true) {
serialRecord.values[0] = 1;
serialRecord.send();
//println("1");
sendstate[i]=false;
num++;
}
}
if (xpos[i]==800&&ypos[i]==0) {
ball[i]=2;
if (sendstate[i]==true) {
serialRecord.values[0] = 2;
serialRecord.send();
//println("2");
sendstate[i]=false;
num++;
}
}
if (xpos[i]==0&&ypos[i]==800) {
ball[i]=3;
if (sendstate[i]==true) {
serialRecord.values[0] = 3;
serialRecord.send();
//println("3");
sendstate[i]=false;
num++;
}
}
if (xpos[i]==800&&ypos[i]==800) {
ball[i]=4;
if (sendstate[i]==true) {
serialRecord.values[0] = 4;
serialRecord.send();
//println("4");
sendstate[i]=false;
num++;
}
}
}
if (xpos[i]<400 && ypos[i]<400) {
xpos[i] = xpos[i] + 10;
ypos[i]= ypos[i] + 10;
}
if (xpos[i]>400&&ypos[i]<400) {
xpos[i] = xpos[i]-10;
ypos[i]=ypos[i]+10;
}
if (xpos[i]>400&&ypos[i]>400) {
xpos[i] = xpos[i]-10;
ypos[i]=ypos[i]-10;
}
if (xpos[i]<400&&ypos[i]>400) {
xpos[i] = xpos[i]+10;
ypos[i]=ypos[i]-10;
}
noStroke();
if (ypos[i]<=400) {
fill(255, 0, 0);
}
if (ypos[i]>=400) {
fill(#30CEFC);
}
if (2300<cfc&&cfc<2700) {
fill(#FFF534);
//serialRecord.values[0]=6;
// serialRecord.send();
//flagb[i]=true;
}
star(xpos[i], ypos[i], 40, 20, 5);
if (xpos[i]==400&&ypos[i]==400) {
flag[i]=true;
serialRecord.values[0] = 5;
serialRecord.send();
}
}
}
}
Medium mode:
import processing.sound.*;
import processing.serial.*;
import osteele.processing.SerialRecord.*;
Serial serialPort;
SerialRecord serialRecord;
SoundFile sound;
boolean[] flag=new boolean[83];
boolean[] flaga=new boolean[83];
boolean[] sendstate=new boolean[83];
int playStart;
boolean success=false;
int premil=0;
int value=0;
int num=0;
int[]beats= {
199,
265,
330,
397,
463,
530,
599,
667,
731,
801,
869,
935,
1004,
1071,
1106,
1141,
1175,
1210,
1244,
1277,
1310,
1344,
1379,
1414,
1445,
1479,
1513,
1548,
1580,
1615,
1647,
1680,
1714,
1750,
1783,
1816,
1851,
1884,
1918,
1951,
2018,
2052,
2085,
2119,
2152,
2187,
2221,
2254,
2286,
2319,
2353,
2388,
2422,
2456,
2489,
2523,
2557,
2591,
2623,
2657,
2691,
2724,
2758,
2791,
2825,
2859,
2894,
2927,
2959,
2977,
2995,
3013,
3029,
3046,
3063,
3080,
3097,
3113,
3129,
3146,
3163,
3181,
3196,
};
int[] xpos=new int[83];
int[] ypos=new int[83];
int x;
int y;
int[] ball=new int[83];
int a=0;
void star(float x1, float y1, float radius1, float radius2, int npoints) { //ball=start
float angle = TWO_PI / npoints;
float halfAngle = angle/2.0;
beginShape();
for (float a = 0; a < TWO_PI; a += angle) {
float sx = x1 + cos(a) * radius2;
float sy = y1 + sin(a) * radius2;
vertex(sx, sy);
sx = x1 + cos(a+halfAngle) * radius1;
sy = y1 + sin(a+halfAngle) * radius1;
vertex(sx, sy);
}
endShape(CLOSE);
}
void keyPressed() {
//println(frameCount+",");
//println(millis());
}
void drawBigHeart() { // if player press the button correctly, the heart becomes bigheart
fill(255, 0, 0);
rectMode(CENTER);
rect(height/2, width/2, 10, 10);
rect(height/2-10, width/2, 10, 10);
rect(height/2+10, width/2, 10, 10);
rect(height/2-20, width/2, 10, 10);
rect(height/2+20, width/2, 10, 10);
rect(height/2-30, width/2, 10, 10);
rect(height/2+30, width/2, 10, 10);
rect(height/2+10, width/2-10, 10, 10);
rect(height/2-10, width/2-10, 10, 10);
rect(height/2+20, width/2-10, 10, 10);
rect(height/2-20, width/2-10, 10, 10);
rect(height/2-15, width/2-20, 10, 10);
rect(height/2+15, width/2-20, 10, 10);
rect(height/2, width/2+10, 10, 10);
rect(height/2-10, width/2+10, 10, 10);
rect(height/2+10, width/2+10, 10, 10);
rect(height/2-20, width/2+10, 10, 10);
rect(height/2+20, width/2+10, 10, 10);
rect(height/2, width/2+20, 10, 10);
rect(height/2-10, width/2+20, 10, 10);
rect(height/2+10, width/2+20, 10, 10);
rect(height/2, width/2+30, 10, 10);
}
void drawHeart() { // normal heart
fill(255, 0, 0);
rect(height/2, width/2, 10, 10);
rect(height/2-10, width/2, 10, 10);
rect(height/2+10, width/2, 10, 10);
rect(height/2-20, width/2, 10, 10);
rect(height/2+20, width/2, 10, 10);
rect(height/2+10, width/2-10, 10, 10);
rect(height/2-10, width/2-10, 10, 10);
rect(height/2, width/2+10, 10, 10);
rect(height/2-10, width/2+10, 10, 10);
rect(height/2+10, width/2+10, 10, 10);
rect(height/2, width/2+20, 10, 10);
}
void setup() {
size(800, 800);
frameRate(60);
delay(1000);
sound = new SoundFile(this, "t.mp3");
playStart = millis();
sound.play();
noStroke(); // the text
textSize(90);
textAlign(CENTER, CENTER);
String serialPortName = SerialUtils.findArduinoPort();
serialPort = new Serial(this, serialPortName, 115200);
serialRecord = new SerialRecord(this, serialPort, 1);
for (int w=0; w<83; w++) {
sendstate[w]=true;
}
}
void key(int k) { //key is controlled by buttons
//println((frameCount - playStart)+",");
for (int i=0; i<83; i++) { //judgement
if (abs(frameCount-beats[i])<10&&ball[i]==k&&!flaga[i]) {
success=true; //if press button correctly, the heart become bigger
//println("success=true");
premil=millis();
a=a+1;
flaga[i]=true;
}
/*if (success==false) { //draw cross
fill(255, 0, 0);
push();
translate(400, 400);
rotate(PI/4);
rect(0, 0, 400, 100);
pop();
push();
translate(400, 400);
rotate(-PI/4);
rect(0, 0, 400, 100);
pop();
}*/
}
}
void draw() {
background(0);
fill(255, 100);
ellipse(400, 400, 180, 180);
fill(0);
ellipse(400, 400, 160, 160);
rectMode(CENTER);
drawHeart(); //heart
// we add a text
int now = millis() - playStart;
if (now > 1000 && now < 1500) {
fill(255);
text("3", height/2, width/4);
}
if (now > 1500 && now < 2000) {
fill(255);
text("2", height/2, width/4);
}
if (now > 2000 && now < 2500) {
fill(255);
text("1", height/2, width/4);
}
if (now > 2500 && now < 3000) {
fill(255);
text("Start!", height/2, width/4);
}
if (now > 17000 && now < 18500) {
fill(255, 255, 255);
text("It will be faster", height/2, width/4);
}
if (now > 48500 && now < 49500) {
fill(255, 255, 255);
text("Even more faster", height/2, width/4);
}
push();
textSize(32);
fill(255);
text("Your score is:", 125, 65);
text(a, 100, 100);
text("/83", 135, 100);
pop();
if (serialRecord.read()) {
value = serialRecord.get();
key(value);
//println(frameCount);
}
if (success==true) {
if (millis()-premil<300) {
fill(255);
ellipse(400, 400, 180, 180);
fill(0);
ellipse(400, 400, 160, 160);
drawBigHeart();
} else {
success=false;
}
} else drawHeart();
//ball
int cfc=frameCount;
for (int i=max(0, num-2); i<83; i++) {
if (cfc-beats[i]>-31&&!flag[i]) {
fill(255);
if (abs(cfc-(beats[i]-31))<2) {
xpos[i]=800*floor(random(0, 2));
ypos[i]=800*floor(random(0, 2));
if (xpos[i]==0&&ypos[i]==0) {
ball[i]=1;
if (sendstate[i]==true) {
serialRecord.values[0] = 1;
serialRecord.send();
//println("1");
sendstate[i]=false;
num++;
}
}
if (xpos[i]==800&&ypos[i]==0) {
ball[i]=2;
if (sendstate[i]==true) {
serialRecord.values[0] = 2;
serialRecord.send();
//println("2");
sendstate[i]=false;
num++;
}
}
if (xpos[i]==0&&ypos[i]==800) {
ball[i]=3;
if (sendstate[i]==true) {
serialRecord.values[0] = 3;
serialRecord.send();
//println("3");
sendstate[i]=false;
num++;
}
}
if (xpos[i]==800&&ypos[i]==800) {
ball[i]=4;
if (sendstate[i]==true) {
serialRecord.values[0] = 4;
serialRecord.send();
//println("4");
sendstate[i]=false;
num++;
}
}
}
if (xpos[i]<400 && ypos[i]<400) {
xpos[i] = xpos[i] + 10;
ypos[i]= ypos[i] + 10;
}
if (xpos[i]>400&&ypos[i]<400) {
xpos[i] = xpos[i]-10;
ypos[i]=ypos[i]+10;
}
if (xpos[i]>400&&ypos[i]>400) {
xpos[i] = xpos[i]-10;
ypos[i]=ypos[i]-10;
}
if (xpos[i]<400&&ypos[i]>400) {
xpos[i] = xpos[i]+10;
ypos[i]=ypos[i]-10;
}
noStroke();
if (ypos[i]<=400) {
fill(255, 0, 0);
}
if (ypos[i]>=400) {
fill(#30CEFC);
}
if (1024<=cfc&&cfc<=1072||1526<=cfc&&cfc<=1578||2100<=cfc&&cfc<=2148||2600<=cfc&&cfc<=2648) {
fill(#FFF534);
}
star(xpos[i], ypos[i], 40, 20, 5);
if (xpos[i]==400&&ypos[i]==400) {
flag[i]=true;
serialRecord.values[0] = 5;
serialRecord.send();
}
}
}
if(millis()>58500){
clear();
fill(255);
text("Your score is:", height/2,width/2-50);
text(a, height/2-50,width/2+50);
text("/83", height/2+50,width/2+50);
}
}
Hard mode:
import processing.sound.*;
import processing.serial.*;
import osteele.processing.SerialRecord.*;
Serial serialPort;
SerialRecord serialRecord;
SoundFile sound;
boolean[] flag=new boolean[74];
boolean[] flaga=new boolean[74];
boolean[] sendstate=new boolean[74];
int playStart;
boolean success=false;
int premil=0;
int value=0;
int num=0;
int[]beats= {
186,
210,
232,
255,
279,
302,
324,
348,
373,
388,
419,
435,
464,
487,
509,
532,
557,
579,
602,
626,
650,
672,
695,
720,
744,
761,
789,
804,
833,
857,
880,
904,
926,
950,
973,
997,
1022,
1045,
1068,
1090,
1113,
1136,
1159,
1185,
1208,
1231,
1255,
1274,
1300,
1323,
1346,
1368,
1392,
1417,
1439,
1463,
1484,
1502,
1531,
1548,
1576,
1601,
1627,
1650,
1672,
1690,
1719,
1735,
1762,
1786,
1811,
1835,
1858,
1951,
};
int[] xpos=new int[74];
int[] ypos=new int[74];
int x;
int y;
int[] ball=new int[74];
int a=0;
void star(float x1, float y1, float radius1, float radius2, int npoints) { //ball=start
float angle = TWO_PI / npoints;
float halfAngle = angle/2.0;
beginShape();
for (float a = 0; a < TWO_PI; a += angle) {
float sx = x1 + cos(a) * radius2;
float sy = y1 + sin(a) * radius2;
vertex(sx, sy);
sx = x1 + cos(a+halfAngle) * radius1;
sy = y1 + sin(a+halfAngle) * radius1;
vertex(sx, sy);
}
endShape(CLOSE);
}
void keyPressed() {
println(frameCount+",");
//println(millis());
}
void drawBigHeart() { // if player press the button correctly, the heart becomes bigheart
fill(255, 0, 0);
rectMode(CENTER);
rect(height/2, width/2, 10, 10);
rect(height/2-10, width/2, 10, 10);
rect(height/2+10, width/2, 10, 10);
rect(height/2-20, width/2, 10, 10);
rect(height/2+20, width/2, 10, 10);
rect(height/2-30, width/2, 10, 10);
rect(height/2+30, width/2, 10, 10);
rect(height/2+10, width/2-10, 10, 10);
rect(height/2-10, width/2-10, 10, 10);
rect(height/2+20, width/2-10, 10, 10);
rect(height/2-20, width/2-10, 10, 10);
rect(height/2-15, width/2-20, 10, 10);
rect(height/2+15, width/2-20, 10, 10);
rect(height/2, width/2+10, 10, 10);
rect(height/2-10, width/2+10, 10, 10);
rect(height/2+10, width/2+10, 10, 10);
rect(height/2-20, width/2+10, 10, 10);
rect(height/2+20, width/2+10, 10, 10);
rect(height/2, width/2+20, 10, 10);
rect(height/2-10, width/2+20, 10, 10);
rect(height/2+10, width/2+20, 10, 10);
rect(height/2, width/2+30, 10, 10);
}
void drawHeart() { // normal heart
fill(255, 0, 0);
rect(height/2, width/2, 10, 10);
rect(height/2-10, width/2, 10, 10);
rect(height/2+10, width/2, 10, 10);
rect(height/2-20, width/2, 10, 10);
rect(height/2+20, width/2, 10, 10);
rect(height/2+10, width/2-10, 10, 10);
rect(height/2-10, width/2-10, 10, 10);
rect(height/2, width/2+10, 10, 10);
rect(height/2-10, width/2+10, 10, 10);
rect(height/2+10, width/2+10, 10, 10);
rect(height/2, width/2+20, 10, 10);
}
void setup() {
size(800, 800);
frameRate(60);
delay(1000);
sound = new SoundFile(this, "t.wav");
playStart = millis();
sound.play();
noStroke(); // the text
textSize(90);
textAlign(CENTER, CENTER);
String serialPortName = SerialUtils.findArduinoPort();
serialPort = new Serial(this, serialPortName, 115200);
serialRecord = new SerialRecord(this, serialPort, 1);
for (int w=0; w<74; w++) {
sendstate[w]=true;
}
}
void key(int k) { //key is controlled by buttons
//println((frameCount - playStart)+",");
for (int i=0; i<74; i++) { //judgement
if (abs(frameCount-beats[i])<10&&ball[i]==k&&!flaga[i]) {
success=true; //if press button correctly, the heart become bigger
//println("success=true");
premil=millis();
a=a+1;
flaga[i]=true;
}
}
}
void draw() {
background(0);
fill(255, 100);
ellipse(400, 400, 180, 180);
fill(0);
ellipse(400, 400, 160, 160);
rectMode(CENTER);
drawHeart(); //heart
// we add a text
int now = millis() - playStart;
if (now > 1000 && now < 1500) {
fill(255);
text("3", height/2, width/4);
}
if (now > 1500 && now < 2000) {
fill(255);
text("2", height/2, width/4);
}
if (now > 2000 && now < 2500) {
fill(255);
text("1", height/2, width/4);
}
if (now > 2500 && now < 3000) {
fill(255);
text("Start!", height/2, width/4);
}
push();
textSize(32);
fill(255);
text("Your score is:", 125, 65);
text(a, 100, 100);
text("/74", 135, 100);
pop();
if (serialRecord.read()) {
value = serialRecord.get();
key(value);
//println(frameCount);
}
if (success==true) {
if (millis()-premil<300) {
fill(255);
ellipse(400, 400, 180, 180);
fill(0);
ellipse(400, 400, 160, 160);
drawBigHeart();
} else {
success=false;
}
} else drawHeart();
//ball
int cfc=frameCount;
for (int i=max(0, num-2); i<74; i++) {
if (cfc-beats[i]>-31&&!flag[i]) {
fill(255);
if (abs(cfc-(beats[i]-31))<2) {
xpos[i]=800*floor(random(0, 2));
ypos[i]=800*floor(random(0, 2));
if (xpos[i]==0&&ypos[i]==0) {
ball[i]=1;
if (sendstate[i]==true) {
serialRecord.values[0] = 1;
serialRecord.send();
//println("1");
sendstate[i]=false;
num++;
}
}
if (xpos[i]==800&&ypos[i]==0) {
ball[i]=2;
if (sendstate[i]==true) {
serialRecord.values[0] = 2;
serialRecord.send();
//println("2");
sendstate[i]=false;
num++;
}
}
if (xpos[i]==0&&ypos[i]==800) {
ball[i]=3;
if (sendstate[i]==true) {
serialRecord.values[0] = 3;
serialRecord.send();
//println("3");
sendstate[i]=false;
num++;
}
}
if (xpos[i]==800&&ypos[i]==800) {
ball[i]=4;
if (sendstate[i]==true) {
serialRecord.values[0] = 4;
serialRecord.send();
//println("4");
sendstate[i]=false;
num++;
}
}
}
if (xpos[i]<400 && ypos[i]<400) {
xpos[i] = xpos[i] + 10;
ypos[i]= ypos[i] + 10;
}
if (xpos[i]>400&&ypos[i]<400) {
xpos[i] = xpos[i]-10;
ypos[i]=ypos[i]+10;
}
if (xpos[i]>400&&ypos[i]>400) {
xpos[i] = xpos[i]-10;
ypos[i]=ypos[i]-10;
}
if (xpos[i]<400&&ypos[i]>400) {
xpos[i] = xpos[i]+10;
ypos[i]=ypos[i]-10;
}
noStroke();
if (ypos[i]<=400) {
fill(255, 0, 0);
}
if (ypos[i]>=400) {
fill(#30CEFC);
}
star(xpos[i], ypos[i], 40, 20, 5);
if (xpos[i]==400&&ypos[i]==400) {
flag[i]=true;
serialRecord.values[0] = 5;
serialRecord.send();
}
}
}
if(frameCount>2040){
clear();
fill(255);
text("Your score is:", height/2,width/2-50);
text(a, height/2-50,width/2+50);
text("/74", height/2+50,width/2+50);
}
}
Our circuit:
Laser cutting：
Final Project Outlook：
Video：
Music we used：
I don’t know how to include the wav. version music
The title of the hard mode song is: Malicious Mischance
D. CONCLUSIONS:
Our project’s objective is to create an interactive rhythm game. I’m glad that at the user test and the final lecture, people liked our game and wanted to try it, I think it was a great success. At the very beginning, my definition of interactive is, human to human, human to object, object to object. Arduino and Processing realize the interaction of object to object, the series of reactions made by the player and the artifact realize the interaction of human to object, the audience will also be very involved by looking at the screen and will look forward to the player’s game process to realize the interaction of human to human. So I think our game is very successful!
If I had more time I would still like to add the code and possibilities for the 8×8 Neopixel board. In the final presentation, someone will see the yellow star and be confused about what needs to be done. We initially tried to make the board all white, but since the next rhythm point comes too quickly it becomes stuck, if there is a chance I would like to improve this part of the code to tell the player that they need to whistle. And I would like to be able to use a larger display so that players can focus more on the box in front of them, but the audience can see their game process through the large screen. I also hope to add a ranking, so that players know their level haha.
The programming part of this project was difficult, I have never imagined I can finish such a complex project(at least to me). Many thanks to my partner Siwei, she helped me a lot and I learned more programming languages, such as boolean, using defining a flag to make a true/false judgment for start/stop a loop. Especially the part about getting the Arduino and processing connected, and a big thanks to LA for always teaching me how I should write the program. When we finally succeeded in making this game, it was very, very fulfilling!! If you want to add another music to play, you can simply alter the beat point and do so in the code that we finished. Throughout the course, I had new concepts and ideas about interaction, and I made work that fit this concept through my efforts. It was a painful process, but I learned a lot!
By the way, I saw a lot of cute artifacts!
Really enjoy all the classes!
Thank you all!!!