Nature Feels

A major part of Interaction Lab is understanding what effective interaction is, particularly in the context of today. Roughly put, successful interaction reveals itself when someone is lost in the present moment and place. For this to happen, one’s attention must be locked by the project: this is done mainly through captivating sensory stimuli and intuitive and familiar decision flow throughout the experience. As such, in the project design phase, it is natural for most to go in the direction of utilising strong stimulus for engagement. Considering today’s age where attention is highly demanded and shortening, we wanted to go in the opposite direction: a meditative experience that gives you to option of affording and practicing your attention in a neutral, natural environment away from daily distractions. Nature Feels is an ambient soundscape where users navigate a range of natural environments by interacting with saplings in a plant pot, guided only by a symbolic transforming tree. A primary theme of this project was encouraging users to let go and practice connection between body and environment, to walk and explore a new setting without knowing where you are but allowing the space to absorb you. The User Testing Session made it quite apparent that letting go was a problem, as the feeling of not being fully in control was too unfamiliar for most, and hence I felt that the project was all the more successful here in its purpose of overcoming that. However, the transforming tree as an indication of the environment being too vague was feedback that we did sort to ameliorate. The three variables that dictated the environment were originally represented by the tree’s growth, the level of rainfall, and the rising fire. We replaced these variables and focused on aspects of the tree: the tree growing, leaves falling and growing, and flowers growing with appropriate colours to effectively provide feedback to the user when interacting with the plant pot. This visual feedback was effective in grounding the user in the larger picture of the experience, and for immediate feedback, we added a walking sound effect that also established the sense of journey.

The most crucial portion of the production process was the first step: idea brainstorming. We wanted to ensure a collective passion and pride in the project, and we easily decided to deal with the sensor-side using Arduino and to use Processing to take charge of feedback to the user. We also decided on a theme of nature, and for the feedback to take the form of ambient sound, though originally as a composition of various frequencies and rhythms that sounded passable especially when controlled by the user, and this was very challenging. After the mid-term project, we knew that the simplification of sensors was vital to ensuring intuitive user input, so we settled for three saplings, each made of coloured wire and two flex sensors wrapped in cotton string. The flex sensors mimicked the feeling of leaves on a sapling, and being wrapped in cotton string attracted user interaction as opposed to the wire trunk and ensured that the sensitive sensors remained reliably functional. With three sensors, each controlling a variable from range 0-100, we realised that this was easily illustratable with a three-dimensional graph forming a cube and that this cube could be sectioned into environments, just as in real life, and as such, having the sensors control movement within an environment would parallel real-life and hence be intuitive to the user and promising in immersive effect. As such, the ambient sound originally as a composition would evolve into natural ambient sound and random chance events dictated by location in an environment and distances from certain landmarks. Ultimately, the challenge of a vague output was unravelled by developing the hardware, and input side of the project. We also wanted to provide users with visual feedback on the monitor that was vaguely indicative of the environment, similar to a submarine’s radar screen. Essentially, we needed to express three variables, and after the User Testing Session, we used aspects of the tree to express the variables for continuity as the user interacted with saplings. However, other feedback involved replacing the tree with the three-dimensional mapping of the variables as our logic followed. This was a challenging call to make, but we rejected the change as we wanted to preserve the feeling of letting go of control and encouraging the journey and immersion rather than making calculated decisions in a simulation too reminiscent of a real-life conquering attitude. We were very happy with this choice as in hindsight it preserved the project’s mysterious and curious attraction. The code was divided into two sections: one for the ambient sound feedback, and the other for the visual tree feedback. The tree was composed of fractals in the branches, leaves and flowers, each controlled by a mapping of x, y, and z variables. The ambient sound involved the x, y, and z variables forming a cube that was sectioned into multiple cuboids that represent environment spaces: desolate, desert, grasslands, ocean, forest, beach, and city. Each environment has a primary ambient sound and various sound samples that can randomly occur with a percentage every 10th of a second. The amplitude, frequency, cutoff frequencies and per cent chance of sounds are adjusted depending on the location in the cube from particular points and lines.

The goal of Nature Feels was to provide users with a meditative experience that practised mind-body-environment connection, detachment from particular behavioural patterns, and immersion within a nurturing environment that may be lost to most today. I believe our project was absolutely successful in its goals. At the NYUSH Spring 24 IMA show, Nature Feels contrasted the rush and exchange of activity in the room as our users sat down, folded their legs, nestled their ears into a headset, and escaped into the desert or forest. Actually, the success of our project came to our surprise, as we anticipated the experience being too much of a challenge for most people’s focus, especially considering the variety of awesome projects by our peers in the room. Instead, the contrast seemed to benefit our project, and this supported our goal as the distractions in the room served as an exaggerated reflection of the real world. What came most to our surprise was that younger children were more engaged throughout the experience: more patient and delicate with the sensors of the plant pot, and immersed within the soundscape experience. My original hypothesis was that the younger generation would be too easily distracted by other externals, but Nature Feels seemed to convincingly capture their wandering minds. One girl stayed for 15 minutes, which I thought was incredible (and incredibly selfish). Instead, the university student demographic seemed to be the audience most able to appreciate the experience, but least able to remain for longer than 3 minutes. I define Interaction as the collection of inputs, outputs, and logical relations taking place between two agents, and most importantly serving the purpose of world-building. Nature Feels gives users an escape back to nature, where we came from and I believe can learn everything we need to know. As such, users have a new reference to reflect on their selves in relation to the outside world, creating new world perspectives and understandings from simulation. One improvement for the project could be expanding on the environments and sound samples, though this is natural given the fact that we aimed to make the project scalable. A primary aspect of this project that made it successful was the delegation of tasks, where specialisation into different aspects of code allowed us to further develop both sides of the project. This served as a lesson learned from the prior midterm project and a reinforced takeaway for group work. One of the greatest struggles was trying to achieve something theoretically sound, but seemingly not facilitated by the tools. As such, it is important to make decisions based on what is realistically possible as an end product, rather than what is ideally possible.

Nature Feels is a stepping stone for users to reconnect with nature. I believe that just as everything is derived from nature, so too does nature have all the answers we need in life, possibly requiring translation, faith and tenacity. We are not born into this world. We are born from it, just as an apple grows and drops from the tree and enters a foreign soil that once nourished it.

Appendix – Arduino and Processing Code for Nature Feels

Arduino Code:

void setup() {
  Serial.begin(9600);
}

void loop() {
  // to send values to Processing assign the values you want to send
  // this is an example:
  int sensor0 = analogRead(A0);
  int sensor1 = analogRead(A1);
  int sensor2 = analogRead(A2);
  int sensor3 = analogRead(A3);
  int sensor4 = analogRead(A4);
  int sensor5 = analogRead(A5);


  // send the values keeping this format
  Serial.print(sensor0);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor1);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor2);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor3);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor4);
  Serial.print(",");  // put comma between sensor values
  Serial.print(sensor5);
  Serial.println();  // add linefeed after sending the last sensor value

  // too fast communication might cause some latency in Processing
  // this delay resolves the issue
  delay(20);

  // end of example sending values
}

Processing Code:
 

import processing.serial.*;

Serial serialPort0;


int NUM_OF_VALUES_FROM_ARDUINO_0 = 6;  /* CHANGE THIS ACCORDING TO YOUR PROJECT */

/* This array stores values from Arduino */
int arduino_values[] = new int[NUM_OF_VALUES_FROM_ARDUINO_0];


//                              HYPER PARAMETERS  --- SET TO CALIBRATE FLEX SENSORS
float grass_up_start = 145;
float grass_up_top = 195;
float grass_up_bot = 75;

float grass_down_start = 155;
float grass_down_top = 260;
float grass_down_bot = 70;

float fire_up_start = 125;
float fire_up_top = 190;
float fire_up_bot = 60;

float fire_down_start = 145;
float fire_down_top = 260;
float fire_down_bot = 80;

float water_up_start = 110;
float water_up_top = 200;
float water_up_bot = 50;

float water_down_start = 150;
float water_down_top = 280;
float water_down_bot = 50;

//                               VALUES ZERO OR ONE, SHOWING SENSOR ACTIVATION
int grass_up = 0;
int grass_down = 0;
int fire_up = 0;
int fire_down = 0;
int water_up = 0;
int water_down = 0;



//                              xyz values for the three variables
float water = 0;
float fire = 0;
float grass = 0;

String state;

//                              for triggering chance events
int prev_time = -1;
int time = 0;
int active = 0;

//                              initialising starting area
int game_on = 1;
int desolate_counter = 0;
int of_six;
int desolate_isPlaying = 0;

int isWalking;




//                                  SOUND
import processing.sound.*;

//                                  Sound Files Importing
LowPass lowPass;

//start
SoundFile start;
SoundFile footsteps;
//desolate

SoundFile desolate_1;
SoundFile desolate_2;
SoundFile desolate_3;
SoundFile desolate_4;
SoundFile desolate_5;
SoundFile desolate_6;

//desert
SoundFile desert;
SoundFile coyotes;
SoundFile plane;
SoundFile thumper;
SoundFile thunder;
//ocean
SoundFile ocean;
SoundFile storm;
SoundFile dolphin;
SoundFile whale;
//grassland
SoundFile grassland;
SoundFile bees;
SoundFile birds;
SoundFile grasshopper;
SoundFile horses;
SoundFile bison;
SoundFile stream_grasslands;
//beach
SoundFile beach;
SoundFile ship_horn_1;
SoundFile ship_horn_2;
SoundFile beach_bars;
SoundFile seagull;
//forest
SoundFile forest;
SoundFile stick_break;
SoundFile tree_falling;
SoundFile chainsaw;
SoundFile gunshot;
//SoundFile forest_singer;
SoundFile frog;
//city
SoundFile nyc;
SoundFile street_performance;
SoundFile dog;


//                              
float dist_desolate;
float dist_desolate_map;

float dist_desert;
float dist_desert_map;

float dist_ocean;
float dist_ocean_map;
int storm_on = 0;

float dist_grassland;
float dist_grassland_map;
int stream_grasslands_on = 0;

float dist_beach;
float dist_beach_map;
int beach_bars_on = 0;

float dist_forest;
float dist_forest_map;
//int forest_singer_on = 0;

float dist_city;
float dist_city_map;
int street_performance_on = 0;

//                                                    TREE

ArrayList<Branch> tree;
ArrayList<Leave> leaves;
ArrayList<Flower> flowers;
int pressCount = 0;
int count = 8;
boolean pressed = false;
float speed = 0.5;
float RedRatio = 0;
float FlowerRatio = 0;
int regrow_count = 0;
int finishedBranchCount = 0;
boolean restart = false;
int flowerGrow_count = 0;
float flower_size = random(0.3, 0.5);

color colorA = color(255, 0, 0, 125); 
color colorB = color(0, 255, 0, 125); 
float lerpAmount = 0.05; 







void setup() {
  size(1200, 800);
  
  printArray(Serial.list());
  serialPort0 = new Serial(this, "/dev/cu.usbmodem1101", 9600); // NAME OF PORT 0
  
  //                              IMPOORTING SOUNDFILES
  lowPass = new LowPass(this);
  footsteps = new SoundFile(this, "footsteps.wav");
  start = new SoundFile(this, "start.wav");
  desolate_1 = new SoundFile(this, "desolate_1.wav");;
  desolate_2 = new SoundFile(this, "desolate_2.wav");;
  desolate_3 = new SoundFile(this, "desolate_3.wav");;
  desolate_4 = new SoundFile(this, "desolate_4.wav");;
  desolate_5 = new SoundFile(this, "desolate_5.wav");;
  desolate_6 = new SoundFile(this, "desolate_6.wav");;
    
  beach_bars = new SoundFile(this, "beach_bars.wav");
  beach = new SoundFile(this, "beach.wav");
  bees = new SoundFile(this, "bees.wav");
  birds = new SoundFile(this, "birds.wav");
  bison = new SoundFile(this, "bison.wav");
  chainsaw = new SoundFile(this, "chainsaw.wav");
  coyotes = new SoundFile(this, "coyotes.wav");
  desert = new SoundFile(this, "desert.wav");
  dog = new SoundFile(this, "dog.wav");
  dolphin = new SoundFile(this, "dolphin.wav");
  forest = new SoundFile(this, "forest.wav");
  frog = new SoundFile(this, "frog.wav");
  grasshopper = new SoundFile(this, "grasshopper.wav");
  grassland = new SoundFile(this, "grassland.wav");
  gunshot = new SoundFile(this, "gunshot.wav");
  horses = new SoundFile(this, "horses.wav");
  nyc = new SoundFile(this, "nyc.wav");
  ocean = new SoundFile(this, "ocean.wav");
  plane = new SoundFile(this, "plane.wav");
  seagull = new SoundFile(this, "seagull.wav");
  ship_horn_1 = new SoundFile(this, "ship_horn_1.wav");
  ship_horn_2 = new SoundFile(this, "ship_horn_2.wav");
  stick_break = new SoundFile(this, "stick_break.wav");
  storm = new SoundFile(this, "storm.wav");
  stream_grasslands = new SoundFile(this, "stream_grasslands.wav");
  street_performance = new SoundFile(this, "street_performance.wav");
  thumper = new SoundFile(this, "thumper.wav");
  thunder = new SoundFile(this, "thunder.wav");
  tree_falling = new SoundFile(this, "tree_falling.wav");
  whale = new SoundFile(this, "whale.wav");
  
  
  //                          TREE

  
  noCursor();
  tree = new ArrayList<Branch>();
  leaves = new ArrayList<Leave>();
  flowers = new ArrayList<Flower>();
  PVector a = new PVector(width / 2, height);
  PVector b = new PVector(width / 2, height - 300);
  Branch root = new Branch(a, b, 10);
  root.count = -1;
  tree.add(root);
  
}

void draw() {
  background(0);
  stroke(255);
  fill(255);

  //                        receive the values from Arduino
  getSerialData0();
  
  float grass_increase = arduino_values[0];
  float grass_decrease = arduino_values[1];
  float fire_increase = arduino_values[2];
  float fire_decrease = arduino_values[3];
  float water_increase = arduino_values[4];
  float water_decrease = arduino_values[5];

  //                         SENSORS - testing values within calibration
  if (game_on == 1) {
    if (((grass_increase > grass_up_start + (grass_up_top - grass_up_start)/2) || (grass_increase < grass_up_start - (grass_up_start-grass_up_bot)/2)) && (grass < 100))   {
      grass = grass + 0.5;
      grass_up = 1;
    }
    else {
      grass_up = 0;
    }
    if (((grass_decrease > grass_down_start + (grass_down_top - grass_down_start)/2) || (grass_decrease < grass_down_start - (grass_down_start - grass_down_bot)/2)) && (grass > 0))  {
      grass = grass - 0.5;
      grass_down = 1;
    }
    else {
      grass_down = 0;
    }
    if (((fire_increase > fire_up_start + (fire_up_top - fire_up_start)/2) || (fire_increase < fire_up_start - (fire_up_start-fire_up_bot)/2)) && (fire < 100))   {
      fire = fire + 0.5;
      fire_up = 1;
    }
    else {
      fire_up = 0;
    }
    if (((fire_decrease > fire_down_start + (fire_down_top - fire_down_start)/2) || (fire_decrease < fire_down_start - (fire_down_start - fire_down_bot)/2)) && (fire > 0))  {
      fire = fire - 0.5;
      fire_down = 1;
    }
    else {
      fire_down = 0;
    }
    if (((water_increase > water_up_start + (water_up_top - water_up_start)/2) || (water_increase < water_up_start - (water_up_start-water_up_bot)/2)) && (water < 100))   {
      water = water + 0.5;
      water_up = 1;
    }
    else {
      water_up = 0;
    }
    if (((water_decrease > water_down_start + (water_down_top - water_down_start)/2) || (water_decrease < water_down_start - (water_down_start - water_down_bot)/2)) && (water > 0))  {
      water = water - 0.5;
      water_down = 1;
    }
    else {
      water_down = 0;
    }
  }
  
  if (keyPressed) {
    if (key == 's') {
      game_on = 1;
    }
  }
 

  
  
  // SETTING STATES - defining areas within 100x100x100 cube
  if (game_on == 1) {
    if ((grass >= 60) && (fire >= 60) && (water >= 60)) {
      state = "city";
    }
    else if ((grass < 30) && (fire < 30) && (water < 30)) {
      state = "desolate";
    }
    else if ((grass <= 100) && (fire < 30) && (water < 30)) {
      state = "grassland";
    }
    else if ((grass < 30) && (fire <= 100) && (water < 30)) {
      state = "desert";
    }
    else if ((grass < 30) && (fire < 30) && (water <= 100)) {
      state = "ocean";
    }
    else if ((fire > grass) && (water > grass)) {
      state = "beach";
    }
    else {
      state = "forest";
    }
  }
  else {
    state = "start";
  }
  
  // now we have parameters: state, grass, fire, water
  
  //           TIMER FOR RANDOM CHANCE EVENTS PER 10th SECOND
  time = ceil(millis()/100);
  if (prev_time != time) {
    active = 1;
    prev_time = time;
  }
  //                         WALKING
  if ((grass_up == 1 && grass_down == 0) || (grass_up == 0 && grass_down == 1) || (fire_up == 1 && fire_down == 0) || (fire_up == 0 && fire_down == 1) || (water_up == 1 && water_down == 0) || (water_up == 0 && water_down == 1)) {
    isWalking = 1;
  }
  else {
    isWalking = 0;
  }
  if (isWalking == 1) {
    if (footsteps.isPlaying()) {
      footsteps.pause();
    }
    else {
      footsteps.cue(0);
    }
    footsteps.play(1.3);
  }
  else {
    if (footsteps.isPlaying()) {
      footsteps.pause();
    }
  }



  //                         ALL STATES

  if (state == "start") {
    lowPass.process(start);
    lowPass.freq(16000);
    start.play();
    lowPass.stop();
  }
  else {
    if (start.isPlaying()) {
      start.pause();
      println("START PAUSED");
    }
  }
    
  // DESOLATE  
  if (state == "desolate") {
    //dist_desolate = sqrt(sq(grass - 0) + sq(fire - 0) + sq(water - 0));     // ----ADJUST
    //dist_desolate_map = map(dist_desolate, 0, sqrt(2700), 12800, 1600);            // ----ADJUST
    
    //if (desolate.isPlaying()) {
    //  desolate.pause();
    //}
    //else {
    //  desolate.cue(0);
    //  desolate.play();
    //}
    if (desolate_1.isPlaying() || desolate_2.isPlaying() || desolate_3.isPlaying() || desolate_4.isPlaying() || desolate_5.isPlaying() || desolate_6.isPlaying()) {
      desolate_isPlaying = 1;
    }
    else {
      desolate_isPlaying = 0;
    }
    if (desolate_isPlaying == 0) {
      of_six = int(random(1,7));
      if (of_six == 1) {
        desolate_1.play();
      }
      if (of_six == 2) {
        desolate_2.play();
      }
      if (of_six == 3) {
        desolate_3.play();
      }
      if (of_six == 4) {
        desolate_4.play();
      }
      if (of_six == 5) {
        desolate_5.play();
      }
      if (of_six == 6) {
        desolate_6.play();
      }
    }
  else {
    desolate_isPlaying = 0;
  }


    
    //lowPass.process(desolate);
    //lowPass.freq(dist_desolate_map);
    //desolate.play();
    //lowPass.stop();
    
    
  }

  
  

  
  // OCEAN
  if (state == "ocean") {
    dist_ocean = sqrt(sq(grass - 0) + sq(fire - 0));     // ----ADJUST
    dist_ocean_map = map(dist_ocean, 0, sqrt(1800), 12800, 1600);            // ----ADJUST

    if (ocean.isPlaying()) {
      ocean.pause();
    }
    else {
      ocean.cue(0);
    }
    //lowPass.stop();
    lowPass.process(ocean);
    lowPass.freq(dist_ocean_map);
    //lowPass.res(1);
    ocean.play();
    lowPass.stop();
    
    if (active == 1) {
      if (random(0,100)< map(dist_ocean,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(dolphin);
        lowPass.freq(map(dist_ocean,0,sqrt(1800),32000,1600));
        dolphin.play();
      }
      if (random(0,100)< map(dist_ocean,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(whale);
        lowPass.freq(map(dist_ocean,0,sqrt(1800),32000,1600));
        whale.play();
      }
      if (random(0,100)< map(dist_ocean,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(ship_horn_1);
        lowPass.freq(map(dist_ocean,0,sqrt(1800),32000,1600));
        ship_horn_1.play();
      }
      if (random(0,100)< map(dist_ocean,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(ship_horn_2);
        lowPass.freq(map(dist_ocean,0,sqrt(1800),32000,1600));
        ship_horn_2.play();
      }
      if ((random(0,100)< map(dist_ocean,0,sqrt(1800),1,.3)) && (storm_on == 0))  { //-------adjust 0-sqrt(3)
        storm.play(1, 0.5);
        storm_on = 1;
      }
    }
  }
  else {
    if (ocean.isPlaying()){
      ocean.pause();
    }
    if (storm.isPlaying()){
      storm.pause();
      storm_on = 0;
    }
  }

  
  
  // DESERT
  if (state == "desert") {
    dist_desert = sqrt(sq(grass - 0) + sq(water - 0));     // ----ADJUST
    dist_desert_map = map(dist_desert, 0, sqrt(1800), 12800, 1600);            // ----ADJUST

    if (desert.isPlaying()) {
      desert.pause();
    }
    else {
      desert.cue(0);
    }
    //lowPass.stop();
    lowPass.process(desert);
    lowPass.freq(dist_desert_map);
    //lowPass.res(1);
    desert.play();
    lowPass.stop();
    
    if (active == 1) {
      if (random(0,100)< map(dist_desert,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(coyotes);
        lowPass.freq(map(dist_desert,0,sqrt(1800),32000,1600));
        coyotes.play();
      }
      if (random(0,100)< map(dist_desert,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(plane);
        lowPass.freq(map(dist_desert,0,sqrt(1800),32000,1600));
        plane.play();
      }
      if (random(0,100)< map(dist_desert,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(thumper);
        lowPass.freq(map(dist_desert,0,sqrt(1800),32000,1600));
        thumper.play();
      }
      if (random(0,100)< map(dist_desert,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(thunder);
        lowPass.freq(map(dist_desert,0,sqrt(1800),32000,1600));
        thunder.play();
      }

    }
  }
  else {
    if (desert.isPlaying()){
      desert.pause();
    }
  }
  
  
  
  // GRASSLAND
  if (state == "grassland") {
    dist_grassland = sqrt(sq(fire - 0) + sq(water - 0));     // ----ADJUST
    dist_grassland_map = map(dist_grassland, 0, sqrt(1800), 12800, 1600);            // ----ADJUST

    if (grassland.isPlaying()) {
      grassland.pause();
    }
    else {
      grassland.cue(0);
    }
    //lowPass.stop();
    lowPass.process(grassland);
    lowPass.freq(dist_grassland_map);
    //lowPass.res(1);
    grassland.play(1,1);
    lowPass.stop();
    
    if (active == 1) {
      if (random(0,100)< map(dist_grassland,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(bees);
        lowPass.freq(map(dist_grassland,0,sqrt(1800),32000,1600));
        bees.play();
      }
      if (random(0,100)< map(dist_grassland,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(birds);
        lowPass.freq(map(dist_grassland,0,sqrt(1800),32000,1600));
        birds.play();
      }
      if (random(0,100)< map(dist_grassland,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(grasshopper);
        lowPass.freq(map(dist_grassland,0,sqrt(1800),32000,1600));
        grasshopper.play();
      }
      if (random(0,100)< map(dist_grassland,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(horses);
        lowPass.freq(map(dist_grassland,0,sqrt(1800),32000,1600));
        horses.play();
      }
      if (random(0,100)< map(dist_grassland,0,sqrt(1800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(bison);
        lowPass.freq(map(dist_grassland,0,sqrt(1800),32000,1600));
        bison.play();
      }
      if ((random(0,100)< map(dist_grassland,0,sqrt(1800),1,.3)) && (stream_grasslands_on == 0))  { //-------adjust 0-sqrt(3)
        stream_grasslands.play(1, 0.5);
        stream_grasslands_on = 1;
      }
    }
  }
  else {
    if (grassland.isPlaying()){
      grassland.pause();
    }
    if (stream_grasslands.isPlaying()){
      stream_grasslands.pause();
      stream_grasslands_on = 0;
    }
  }
  
  
  
  // BEACH
  if (state == "beach") {
    dist_beach = sqrt(sq(grass - 0) + sq(fire - 100) + sq(water - 100));     // ----ADJUST
    dist_beach_map = map(dist_beach, 0, sqrt(6800), 12800, 1600);            // ----ADJUST
    
    if (beach.isPlaying()) {
      beach.pause();
    }
    else {
      beach.cue(0);
    }
    //lowPass.stop();
    lowPass.process(beach);
    lowPass.freq(dist_beach_map);
    //lowPass.res(1);
    beach.play();
    lowPass.stop();
    
    if (active == 1) {
      if (random(0,100)< map(dist_beach,0,sqrt(6800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(ship_horn_1);
        lowPass.freq(map(dist_beach,0,sqrt(6800),32000,1600));
        ship_horn_1.play();
      }
      if (random(0,100)< map(dist_beach,0,sqrt(6800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(seagull);
        lowPass.freq(map(dist_beach,0,sqrt(6800),32000,1600));
        seagull.play();
      }
      if ((random(0,100)< map(dist_beach,0,sqrt(6800),1,.3)) && (beach_bars_on == 0))  { //-------adjust 0-sqrt(3)
        beach_bars.play(1, 0.5);
        beach_bars_on = 1;
      } 
    }
  }
  else {
    if (beach.isPlaying()){
      beach.pause();
    }
    if (beach_bars.isPlaying()){
      beach_bars.pause();
      beach_bars_on = 0;
    }
  }
  
  
  
  // FOREST
  if (state == "forest") {
    dist_forest = sqrt(sq(grass - 100));       // ----ADJUST
    dist_forest_map = map(dist_forest, 0, sqrt(4900), 12800, 1600);                  // ----ADJUST
    
    if (forest.isPlaying()) {
      forest.pause();
    }
    else {
      forest.cue(0);
    }
    //lowPass.stop();
    lowPass.process(forest);
    lowPass.freq(dist_forest_map);
    //lowPass.res(1);
    forest.play();
    lowPass.stop();
    
    if (active == 1) {
      if (random(0,100)< map(dist_forest,0,sqrt(4900),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(stick_break);
        lowPass.freq(map(dist_forest,0,sqrt(4900),32000,1600));
        stick_break.play();
      }
      if (random(0,100)< map(dist_forest,0,sqrt(4900),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(tree_falling);
        lowPass.freq(map(dist_forest,0,sqrt(4900),32000,1600));
        tree_falling.play();
      }
      if (random(0,100)< map(dist_forest,0,sqrt(4900),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(chainsaw);
        lowPass.freq(map(dist_forest,0,sqrt(4900),32000,1600));
        chainsaw.play();
      }
      if (random(0,100)< map(dist_forest,0,sqrt(4900),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(gunshot);
        lowPass.freq(map(dist_forest,0,sqrt(4900),32000,1600));
        gunshot.play();
      }
      //if ((random(0,100)< map(dist_forest,0,sqrt(4900),1,.3)) && (forest_singer_on == 0))  { //-------adjust 0-sqrt(3)
      //  forest_singer.play(1, 0.5);
      //  forest_singer_on = 1;
      //} 
    }
  }
  else {
    if (forest.isPlaying()){
      forest.pause();
    }
    //if (forest_singer.isPlaying()){
    //  forest_singer_.pause();
    //  forest_singer_on = 0;
    //}
  }
  
  
  
  // CITY
  if (state == "city") {
    dist_city = sqrt(sq(grass - 100) + sq(fire - 100) + sq(water - 100));     // ----ADJUST
    dist_city_map = map(dist_city, 0, sqrt(4800), 12800, 1600);            // ----ADJUST
    
    if (nyc.isPlaying()) {
      nyc.pause();
    }
    else {
      nyc.cue(0);
    }
    //lowPass.stop();
    lowPass.process(nyc);
    lowPass.freq(dist_city_map);
    //lowPass.res(1);
    nyc.play();
    lowPass.stop();
    
    if (active == 1) {
      if (random(0,100)< map(dist_city,0,sqrt(4800),1,.3)) { //-------adjust 0-sqrt(3)
        lowPass.process(dog);
        lowPass.freq(map(dist_city,0,sqrt(4800),32000,1600));
        dog.play();
      }
      
      if ((random(0,100)< map(dist_city,0,sqrt(4800),1,.3)) && (street_performance_on == 0))  { //-------adjust 0-sqrt(3)
        street_performance.play(1, 0.5);
        street_performance_on = 1;
      } 
    }
  }
  else {
    if (nyc.isPlaying()){
      nyc.pause();
    }
    if (street_performance.isPlaying()){
      street_performance.pause();
      street_performance_on = 0;
    }
  }
  

  
  // TESTING
  println(grass, fire, water, state, of_six, desolate_isPlaying, isWalking, grass_up, grass_down);
  //println(grass_up_start, grass_down_start, fire_up_start, fire_down_start, water_up_start, water_down_start);
  
  
  active = 0;
 
 
 
 // TREE
 
 background(map(fire,0,100,0,254/2),map(grass,0,100,0,254/2),map(water,0,100,0,254/2));
  //background(0);
  
  
  if (regrow_count >= finishedBranchCount && finishedBranchCount !=0) {
    println("yes");
    for (int i = 0; i < tree.size(); i++) {
      tree.get(i).regrow = false;
    }
    regrow_count = 0;
  }

  if (flowerGrow_count >= finishedBranchCount && finishedBranchCount !=0) {
    for (int i = 0; i < tree.size(); i++) {
      tree.get(i).flowerGrow = false;
    }
    flowerGrow_count = 0;
  }

  for (int i = 0; i < tree.size(); i++) {
    tree.get(i).show();
  }

  for (int i = 0; i < leaves.size(); i++) {
    leaves.get(i).display();

    if (leaves.get(i).turn) {
      leaves.get(i).update();
      leaves.get(i).speed_update();

      if (leaves.get(i).vanished) {
        leaves.remove(i);
      }
    }
  }

  for (int i = 0; i < flowers.size(); i++) {
    flowers.get(i).display();
    flowers.get(i).fly();

    if (flowers.get(i).vanished) {
      flowers.remove(i);
    }
  }

  int targetPressCount = int(map(water, 0, 100, 2, 7));

  if (targetPressCount < pressCount) {
    removeLatestBranches();
    pressCount--;

    finishedBranchCount = 0;
    for (int i = 0; i < tree.size(); i++) {
      if (!tree.get(i).finished) {
        finishedBranchCount++;
      }
    }
  } else if (targetPressCount > pressCount) {
    generateBranches();
    pressCount++;

    finishedBranchCount = 0;
    for (int i = 0; i < tree.size(); i++) {
      if (!tree.get(i).finished) {
        finishedBranchCount++;
      }
    }
  }

  if (fire_up == 1 && fire_down == 0) {
    if(random(1)<0.5){
    pressed = true;

    if (RedRatio <= 1) {
      RedRatio += 0.01;
    }
    }
  }

  if (fire_down == 1 && fire_up == 0) {
    if(random(1)<0.5){
    if (RedRatio >= 0) {
      RedRatio -= 0.01;
    }
    
    int red_leaves = 0;
    for (int i = 0; i < leaves.size(); i++) {
      if (leaves.get(i).turn) {
        red_leaves ++;
      }
    }

    // 统计已生成的叶子数量
    int numLeavesGenerated = 0;
    int leavesRemoved = 0;

    // 遍历树的末端
    for (int i = 0; i < tree.size(); i++) {
      Branch branch = tree.get(i);
      if (branch.count == pressCount && !branch.regrow) {
        PVector leafPos = branch.end.copy();
        leaves.add(new Leave(leafPos.x, leafPos.y, speed));
        branch.regrow = true;

        numLeavesGenerated++;
        regrow_count++;
        
        println(finishedBranchCount);
        if (finishedBranchCount >= 100){
          if (numLeavesGenerated >= finishedBranchCount/50 ) {
          break;
        }
        }
          else if (numLeavesGenerated >= 2) {
          break;
        }
        
      }
    }

    for (int i = 0; i < leaves.size(); i++) {
      if (leaves.get(i).turn && leaves.get(i).position.y>=height-100) {
        leaves.remove(i);

        leavesRemoved ++;

        if (red_leaves>= 100){
          if (leavesRemoved>= red_leaves / 50 && leavesRemoved>= finishedBranchCount/35) {
          break;
        }
        }
        else if (leavesRemoved>= 3) {
          break;
        }
      }
    }
  }
  }


  if (grass_up == 1 && grass_down == 0) {

    if (FlowerRatio >= 0) {
      FlowerRatio -= 0.1;
    }

    if (flower_size<=50) {
      flower_size += 0.2;
    }

    for (int i = 0; i < flowers.size(); i++) {
      flowers.get(i).update();
    }
  }


  if (grass_down == 1 && grass_up == 0) {
    if (flower_size>=0.3) {
      flower_size -= 0.2;
    }

    for (int i = 0; i < flowers.size(); i++) {
      flowers.get(i).decrease();
    }
  }
}


  
  




void removeLatestBranches() {
  for (int i = tree.size() - 1; i >= 0; i--) {
    if (tree.get(i).count == pressCount) {
      tree.remove(i);
    }
  }

  for (int i = 0; i < tree.size(); i++) {
    tree.get(i).finished = false;
  }

  leaves.clear();
  flowers.clear();

  for (int i = 0; i < tree.size(); i++) {
    Branch branch = tree.get(i);
    if (branch.count == pressCount-1) {
      PVector leafPos = branch.end.copy();
      leaves.add(new Leave(leafPos.x, leafPos.y, speed));
      flowers.add(new Flower(leafPos.x, leafPos.y, flower_size));
    }
  }
}

void generateBranches() {
  for (int i = tree.size() - 1; i >= 0; i--) {
    if (!tree.get(i).finished) {
      Branch[] branches = tree.get(i).branch();
      for (Branch branch : branches) {
        branch.count = pressCount+1;
        tree.add(branch);
      }
    }
    tree.get(i).finished = true;
  }


  leaves.clear();
  flowers.clear();

  for (int i = 0; i < tree.size(); i++) {
    Branch branch = tree.get(i);
    if (!branch.finished) {
      PVector leafPos = branch.end.copy();
      leaves.add(new Leave(leafPos.x, leafPos.y, speed));
      flowers.add(new Flower(leafPos.x, leafPos.y, flower_size));
    }
  }
}


class Branch {
  PVector begin;
  PVector end;
  boolean finished = false;
  float angle;
  float theta;
  float ratio;
  float strokeWeight;
  int count;
  PVector dir;
  float branch_length;
  float fire_color;
  boolean regrow;
  boolean flowerGrow;

  Branch(PVector begin, PVector end, float strokeWeight) {
    this.begin = begin;
    this.end = end;
    this.angle = random(-0.5, 0.5);
    this.theta = PI / random(4, 6);
    this.strokeWeight = strokeWeight;
    this.count = pressCount+1;
    this.dir = PVector.sub(end, begin);
    this.branch_length = this.dir.mag();
    this.fire_color = random(1);
    this.regrow = false;
    this.flowerGrow = false;

    while (PVector.dist(begin, end) < 1) {
      end = new PVector(end.x + random(-5, 5), end.y + random(-5, 5));
    }
  }

  void show() {
    stroke(255);
    strokeWeight(this.strokeWeight);
    line(begin.x, begin.y, end.x, end.y);
  }

  Branch[] branch() {
    int num_branches = floor(random(2.8, 5));
    Branch[] branches = new Branch[num_branches];
    PVector dir = PVector.sub(end, begin);
    float currentRatio = map(water, 0, 100, 0.6, 0.7);

    for (int i = 0; i < num_branches; i++) {
      float branchTheta = this.theta * (random(1) > 0.5 ? 1 : -1);

      PVector branchDir = dir.copy();
      branchDir.rotate(branchTheta + this.angle);
      branchDir.mult(currentRatio);
      PVector newEnd = PVector.add(end, branchDir);
      branches[i] = new Branch(end, newEnd, this.strokeWeight * map(dir.mag(), 0, 300, 0.5, 0.9));
    }

    return branches;
  }
}

class Leave {
  PVector position;
  float speed;
  float red;
  float acc;
  boolean turn;
  boolean vanished;
  boolean regrow;
  float pile;
  float x_speed;
  float rotate;
  float random_x;
  float random_y;
  color current_color;

  Leave(float x, float y, float speed) {
    position = new PVector(x, y);
    this.speed = random(5, 10);
    this.red = random(1);
    this.acc = random(0.3, 0.5);
    this.turn = false;
    this.vanished = false;
    this.regrow = false;
    this.pile = random(30, 100);
    this.x_speed = random(-3,3);
    this.rotate = random(-PI/4,PI/4);
    this.random_x = random(-5,5);
    this.random_y = random(-10,10);
    this.current_color = colorB;
  }

  void update() {

    if (position.y < height-this.pile) {
      position.y += this.speed;
      position.x += this.x_speed;
    }
  }

  void speed_update() {
    this.speed += this.acc;
  }

  void display() {
    if (this.red<RedRatio) {
      this.turn = true;
    }
    if (this.turn == true) {
      fill(this.current_color, 125);
      if (this.current_color != colorA) {
          this.current_color = lerpColor(this.current_color, colorA, lerpAmount);
        }
    } else if (!this.turn) {
      fill(0, 255, 0, 125);
      this.turn = false;
    }
    noStroke();
    push();
    translate(position.x, position.y);
    rotate(this.rotate);
    ellipse(0+this.random_x, this.random_y, 12, 20);
    pop();
  }
}

class Flower {
  PVector position;
  float speed;
  float random_x;
  float random_y;
  boolean turn;
  boolean vanished;
  boolean regrow;
  float pile;
  float size;
  float flower_fly;

  Flower(float x, float y, float size) {
    position = new PVector(x, y);
    this.speed = random(0.3, 0.5);
    this.random_x = random(-10, 10);
    this.random_y = random(-10, 10);
    this.turn = false;
    this.vanished = false;
    this.regrow = false;
    this.pile = random(5, 30);
    this.size = size;
    this.flower_fly = random(1);
  }

  void update() {
    if (this.size<=50) {
      this.size += 0.2;
    }
  }

  void decrease() {
    if (this.size >=0.3) {
      this.size -= 0.2;
    }
  }

  void fly() {
    if (this.flower_fly<FlowerRatio) {
      position.x += this.speed*1.5*noise(3);
      position.y += this.speed;
    }

    if (position.x >= width || position.y >= height ) {
      this.vanished = true;
    }
  }

  void display() {
    fill(245, 200, 240, 50);
    noStroke();
    ellipse(position.x+this.random_x, position.y+this.random_y, this.size, this.size);
  }
}




// the helper function below receives the values from Arduino
// in the "arduino_values" array from a connected Arduino
// running the "serial_AtoP_arduino" sketch
// (You won't need to change this code.)

void getSerialData0() {
  while (serialPort0.available() > 0) {
    String in = serialPort0.readStringUntil( 10 );  // 10 = '\n'  Linefeed in ASCII
    if (in != null) {
      //print("From Arduino0: " + in);
      String[] serialInArray0 = split(trim(in), ",");
      if (serialInArray0.length == NUM_OF_VALUES_FROM_ARDUINO_0) {
        for (int i=0; i<serialInArray0.length; i++) {
          arduino_values[i] = int(serialInArray0[i]);
        }
      }
    }
  }
}