The design:

For lower structure: it was rather easy.

the Upper structure: I can’t eliminate the small spaces close to the center and I have tried multiple initial shapes, but still it didn’t work out.

The video recording of the process:

Fabrication:

Product:

Task #1:

My processing and Arduino framework provided a hyper-sensitive interaction, as could be seen in the two video recorded as following IMG_2934 IMG_2935. A minor mistake in the code that had generated this issue, the pinModes for the potentiometers were adjusted as “OUTPUT”. Thus, instead of recording the changes in resistance, it recorded as outputs the potentiometers. 

Code for Processing:

import processing.serial.*;
import osteele.processing.SerialRecord.*;

int x1 = 0;
int x2 = 1450;

Serial serialPort;
SerialRecord serialRecord;

void setup() {
  fullScreen();
  frameRate(30);


  String serialPortName = SerialUtils.findArduinoPort();
  
  serialPort = new Serial(this, serialPortName, 9600);
  
  serialRecord = new SerialRecord(this,serialPort,2);
}

void draw() {
  background(0);
  noStroke();
  if (x1 == 1450 && x2 == 0){
   x1 = 0;
   x2 = 1450;
   serialRecord.values[0] = 0;
   serialRecord.values[1] = 1;
   serialRecord.send();
  }
  
  if (x1 == 0 && x2 == 1450){
    
   serialRecord.values[0] = 1;
   serialRecord.values[1] = 0;
   serialRecord.send();
  }
  
  
  
  if (x1<1450) {
    circle(x1, 550, 50);
    x1=x1+50;
    x2 = 1450;
    serialRecord.values[0] = 0;
    serialRecord.values[1] = 0;
    serialRecord.send();
  } else {
    circle(x2, 550, 50);
    x2=x2-50;
    serialRecord.values[0] = 0;
    serialRecord.values[1] = 0;
    serialRecord.send();
  }
}

Code for Arduino:

#include "SerialRecord.h"
SerialRecord writer(2);

int pinX = A0;
int pinY = A1;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600);
  // pinMode(pinX,OUTPUT);
  // pinMode(pinY,OUTPUT);
}

void loop() {
  // put your main code here, to run repeatedly:
  int X = analogRead(pinX);
  int Y = analogRead(pinY);
  writer[0] = X;
  writer[1] = Y;
  
  writer.send();
  delay(10);
}

Task2:

Teammate: Tawan

Our code was absolutely fine throughout the process. The inefficiency was caused by some matters in Arduino and Processing system. Before we embedded the code that sent messages to Arduino, the circle bounced back and forth just fine after we simply managed the coordinates of borders. But once the code was embedded while  there was no wrongs in grammar or mechanism of the Arduino code, Processing gave us all blank output when Arduino had sent the code of Arduino to Arduino Uno. It always said “port busy” in Processing. So, we shut it but the servo once received the message started moving forever non-stop. Thus,  I have to forcefully made it stop by changing the Processing-sent value. After thousands of tries, I added two led lights to the servo to see if it was the problem with the servo. Surprisingly, the servos started to work fine. The interaction of the device was very much low-level, machine-to-machine, pale single-direction signal. However, it did stroke me that for Arduino-Processing artifacts, codes should be sent to the Arduino Uno first and I should shut the Arduino thoroughly before proceeding to Processing. 

Recordings:

Code for Processing:

import processing.serial.*;
import osteele.processing.SerialRecord.*;

int x1 = 0;
int x2 = 1000;
Serial serialPort;
SerialRecord serialRecord;

void setup() {
  fullScreen();
  frameRate(30);


  String serialPortName = SerialUtils.findArduinoPort();
  
  serialPort = new Serial(this, serialPortName, 9600);
  
  serialRecord = new SerialRecord(this,serialPort,2);
}

void draw() {
  background(0);
  noStroke();
  if (x1 == 1450 && x2 == 0){
    x1 = 0;
    x2 = 1450;
   serialRecord.values[0] = 0;
   serialRecord.values[1] = 1;
   serialRecord.send();
  }
  
  if (x1 == 0 && x2 == 1450){
   serialRecord.values[0] = 1;
   serialRecord.values[1] = 0;
   serialRecord.send();
  }
  
  
  
  if (x1<1450) {
    circle(x1, 550, 50);
    x1=x1+50;
    x2 = 1450;
    serialRecord.values[0] = 0;
    serialRecord.values[1] = 0;
    serialRecord.send();
  } else {
    circle(x2, 550, 50);
    x2=x2-50;
    serialRecord.values[0] = 0;
    serialRecord.values[1] = 0;
    serialRecord.send();
  }
}

Code for Arduino:

#include "SerialRecord.h"
#include <Servo.h>
int led1 = 12;
int led2 = 13;
Servo servo1;
Servo servo2;
SerialRecord reader(2);
int val = 180;

void setup() {
  Serial.begin(9600);
  servo1.attach(7);
  servo2.attach(4);
  pinMode(led1,OUTPUT);
  pinMode(led2,OUTPUT);
}

void loop() {
 reader.read();

 if (reader[0] == 1){           
  servo1.write(val);
  delay(100);  
  servo1.write(0);
  reader[0] = 0;
  digitalWrite(led1, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(100);                       // wait for a second
  digitalWrite(led1, LOW);    // turn the LED off by making the voltage LOW
  delay(100);                 

 }

 if (reader[1] == 1){           
  servo2.write(val);   
  delay(100);          
  servo2.write(0);
  reader[1] = 0;
  digitalWrite(led2, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(100);                       // wait for a second
  digitalWrite(led2, LOW);    // turn the LED off by making the voltage LOW
  delay(100);                 
 }
}

Sketching Arts from the Hand-disabled: Sketch Your Dreams with Sketch-it

Project Statement of Purpose

The interactive designs I explored triumphed not over the high-tech they included but the interaction they provided for the audience and it was often the case that success of the projects was the graphitization  of people’s mindset or mentality. It stroke me that sketching was also a way to express one’s mindset graphically. Though a simple work of hand, it was almost impossible for many people with hand disabilities and even if they can manage so, it would take great efforts. Thus, the project aims to help people with hand disabilities to accomplish their dream of sketching art, to help them express themselves of sketching in an easier way. 

Project Proposal Plan

The project was basically a robotic sketcher, consisting of two stepper motors hanging on a drawing mat vertical to the ground, which were attached with two strings connected to one servo whose fans were combined with a pencil. On key board, people can control the servo to have the pencil sketching in half circles by clicking the mouse and relocate the pencil on the drawing mat by moving the mouse. The setting up of such device, including laser cuttings and 3D printing, would take about three days to finish without concerning other urgent missions from other courses that I need to accomplish while the code of it would take about one day or three depending on whether I will have time to finish the harder but more interactive version of the code. Starting from the easy version, when the code in Processing is run, the interface would be split into 4 areas correspondingly to four directions of the movements of the servo with pencil, up-left, up-right, down-left, and down-right. For example, when the mouse is placed in up-left area, the stepper motor at the up-left corner of the drawing mat would start rolling to shorten the string attached to the left side of the servo so that it moves in the up-left direction. The harder version would take algorithms. There would not be specific indications of where to place the mouse to have the servo moving towards specific directions. Instead, by tracing the trail of the movement of the mouse, the servo would move to the location on the mat corresponding the location of the mouse in the interface of Processing. Moreover, I wanted to add a variable to adjust the rotation of the servo so that the pencil doesn’t always draw half circles. The variable could be changed according to the strength applied to a force sensor. 

Context and Significance

 I think in this project, it was fulfilled. There was a successful work, Supersynthesis by Amay Kataria which visualized the music produced by people in the periodical glowing of lights according to notes and melodies.( n.d.) Amay Kataria described the purpose of the art is to visualize people’s minds (mediations) in the form of lights per se.( n.d.) I think sketching was also a direct way of speaking from one’s mind in the form of visuals. Though it was easy for us to sketch with a few techniques of drawing, it was hard enough for people with hand disabilities to draw. There is a Polish artist called Mariusz Kedzierski, who was born without hands, but succeeded in making a career though sketching with his arms hold pencils. His works of art were still on sale in his personal website, https://www.mariuszkedzierski.net, and captured multiple global prizes. He was such an inspiring power for people that he became a motivational speaker influencing the world. (Bern, n.d.) 

Interactions, in the perspective of Interactive Media Art, from my perspective would be the comprehensive indications and responses from/for human users that are received and generated from/for electronic artifacts. Processing understands the movements and clicking of mouse while the users, on seeing what have been drawn on the mat could decide her next move to make. The project aims its target audience specifically at the hand-disabled, but doesn’t limit its users or audience within such. The terminal ideology of the project was intended to help people with hand disabilities produce their graphic work of art in sketches. By the works of arts produced, the recordings of production of arts by these people and the mechanisms of the device, more social attention could be drawn to hand-disabled people, to whom more help should be offered, instead of marginalizing. 

References

Dockery, Regan. “Light & Sound Synthesis: In Conversation with Amay Kataria.”interviewed  by Regan Dockery. In Environment Sound. Can-V5. Creative Applications. 2022. Accessed Nov. 15th, 2022. https://www.creativeapplications.net/environment/light-sound-synthesis-in-conversation-with-amay-kataria/.

Bern, Marija. “We Can’t Believe What These 37 Artists Can Do Despite Their Disabilities.” In Art, People. BoredPanda. 2018. Accessed Nov. 20th, 2022. https://www.boredpanda.com/inspiring-disabled-artists/?utm_source=google&utm_medium=organic&utm_campaign=organic.

1. YourBeat: Musics played by the official music apps might not cater to one’s taste. Despite the matters of the lyrics, the beats and the melodies could also be a problem. Meanwhile it was sometimes the case that by adding certain beats of certain comparative volume to the music can promote the music and incline the engagement of audience to a whole new level. YourBeat, a simple device could manage such. When music played, visualized music as varying columns generated by  Processing, would show in the 8*32 Neo-pixel. By motions above the distance sensor, the volume of the music could be adjusted while by one motion above the motion sensor, the beat could be played once. Thus, by simply shaking hands above the motion sensor, beats of your own could be added to the music, and visualized on the Neo-pixel. 
2. Sketcher: The device was intended for people with disabilities in hands, to say missing fingers or hands, that disabled them from sketching. Sketching needed high intensity of vibrations and oscillations of pencils, so would be rather hard for people with such disability to manage. Thus, I designed this device, which is made up of servo, pencil and a machine that could move the servo around according to the movement of mouse by contacting with the touchpad. By pressing the touchpad, the servo attached with a pencil would run by once. Meanwhile, by contacting the touch pad, the pencil could be moved to the corresponding location to the attachment of the touchpad of the frame that holds it. 
3. Move-up: There’s a kind of games called MUG (Music Game) that originated in Japan and around gained its prevalence around the world. The game was also labeled as a specific kind of game for “otaku”, homebody or indoorsman. Thus, to switch this label for the game and shad light on the health of this group of people for lack of physical exercise, I decided add on more movements to the game. To physicalize it from the screens to physical movements. Four motion sensors would be able to capture users’ movements to encourage them to win the game by moving certain body parts to the corresponding motion sensor following the beats or nodes of the music played. 

The following is the sketch for three devices. 

Task 1:

I managed to connect the cable, which took me about 1 min. FastLed has already been downloaded before the recitation. From the material, I learnt to give supply for the larger pixel boards to glow the same way.

Task 2:

I lit up accordingly, the second light in purple. To be seen in the link. 

IMG_2916

On copying the code, I produced the scene below. 

IMG_2884

Task 3:
 I tried to let all the pixels’ colors change with the sound but I can’t. And found out that it was the problem with the “constrain” function. On changing “constrain” to “map” . “map” was very useful a function that by ratio, it could turn a variable from one range to another correspondingly while “constrain” only limited a variable within the maximum and minimum. This is the situation before the change. 

IMG_2885

This is after. 

IMG_2887

It is such an ambiguity that I wanted it to be all purple and only bring changes on the saturability of purple. 

/* This is a code example for Processing, to be used on Recitation 7
  You need to have installed the SerialRecord library.
  
  Interaction Lab
  IMA NYU Shanghai
  2022 Fall
*/
import processing.sound.*;
import processing.serial.*;
import osteele.processing.SerialRecord.*;
SoundFile sample;
FFT fft;
int bands =256;
float smoothingFactor = 0.2;

Serial serialPort;
SerialRecord serialRecord;

float[] sum = new float[bands];
int scale = 8;
float barWidth;


int W;         //width of the tiles
int NUM = 255;  //amount of pixels
int[] r = new int[NUM]; //red of each tile
int[] g = new int[0]; //green of each tile
int[] b = new int[NUM]; //blue of each tile

void setup() {
  size(1024, 768);
  background(255);
  barWidth = 10*width/float(bands);
  sample = new SoundFile(this, "Post Malone - A Thousand Bad Times.mp3");
  
  // You can use this syntax and change COM3 for your serial port
  // printArray(Serial.list());
  // serialPort = new Serial(this, "COM3", 9600);
  // in MacOS it looks like "/dev/cu.usbmodem1101"
  //or you can try to use this instead:
  fft = new FFT(this, bands);
  fft.input(sample);
  String serialPortName = SerialUtils.findArduinoPort();
  serialPort = new Serial(this, serialPortName, 9600);
  
  serialRecord = new SerialRecord(this, serialPort, 4);
  serialRecord.logToCanvas(false);
  
  rectMode(CENTER);
}

void draw() {
  background(125, 255, 125);
  fill(255, 0, 150);
  noStroke();
  
  fft.analyze();
  
  //int n = floor(constrain(mouseX/W , 0, NUM-1));
  //r[n] = floor(255);
  //g[n] = floor(random(255));
  //b[n] = floor(255);
  
  for (int i = 0; i < 32; i++) {
    // Smooth the FFT spectrum data by smoothing factor
    sum[i] += (fft.spectrum[i] - sum[i]) * smoothingFactor;

    // Draw the rectangles, adjust their height using the scale factor
    rect(i*barWidth, height, barWidth, -sum[i]*height*scale);
  
 
    
      r[i] = floor(255);
      g[i] = floor(random(255));
      b[i] = floor(255);
      
      serialRecord.values[0] = i;     // which pixel we change (0-59)
      serialRecord.values[1] = r[i];  // how much red (0-255)
      serialRecord.values[2] = g[i];  // how much green (0-255)
      serialRecord.values[3] = b[i];  // how much blue (0-255)
      serialRecord.send();            // send it!
    
  }
}

void mousePressed(){//点击鼠标
 if(sample.isPlaying()){//如果声音在播放
   sample.pause();//暂停
 }else{//否则
   sample.loop();//播放声音
   //play()函数只播放声音一次
 }
}

But errors on behalf of the drive appeared. It said so:

SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
RuntimeException: Error opening serial port /dev/cu.usbmodem11301: Port busy

Interactions, in the perspective of Interactive Media Art, from my perspective would be the comprehensive indications and responses from/for human users that are received and generated from/for electronic artifacts. These interactions are limited in the very standard that they should be comprehendible bi-directionally. That is to say, machine can understand and proceed the indications from human users and give back corresponding reposes, and vice versa. The two spheres of interaction acquire the equivalent level and amount of knowledge in the aspect of the specific interaction(s), right enough for proceeding inputs and outputs. 

On the development of science and technology, interactive art emphasized more and more  on the compromise of machines and the naturality of human actions. Instead of giving indications to the users that limit users actions within certain disciplines, machines are more and more able to sense natural movements of human users. As the third kind of Interaction engagement and experience defined by Ernest Edmonds (n.d.),  non-invasive devices captured audience movements with sensors and generate certain responses. I think this category of devices is what I think of as the final project that I am going to produce. Thus, I did researches on projects concerning such:

First, Supersynthesis. According to the standards raised by Ernest Edmonds, the time spent by the processing of inputs and the generating of outputs result in the interactivity of the project. The project gave immediate responses as the signals of flashing light according to the sound given by the users. Basically, it was a visualization of sounds, but according to the creator of it, Amay Kataria, the lights of it, “were repeatedly brought up to express guidance, clarity, and direction in their own inner journeys of spiritual practice”.  ( n.d.)(Dockery, n.d.)

The lights were complied into waves to express waves of sounds. When sounds came, the lights cooperated to glow with the beats and generated waves of shines. What’s even special about it is that the glow left by former users would be recorded and promoted to the device again. The success of this artifact was that it itself was only a half product, and it left huge space for the audience to finish this art, which caters to the interaction art raised by Ernest Edmonds, to let the audience finish the rest of the piece.(n.d.) 

Second, the MIDI Experiments. Mark Wheeler generated two animations that could vary with the changing of the sound input.(n.d.) As could be seen below, the key pressed, the animation started producing pictures accordingly. 

(Eats, n.d.)

Then, Russ Chimes and Clay Weishaar, abstract the two experiment from the screen to reality, by a gargantuan amount  of projectors. They created the tunnels of lights in suburbia. (Visnjic, n.d.) The artifact, like the former gave immediate reactions from the input of users and that the art of it was made varying by the users. It created a visual reality for the users, bringing them into a suburbia that was renewed by the lights. 

Conclusion:

The most significant essence of success of these interactive arts was the remaining blank of the art to be completed by the audience. They, in my understanding, are half-products before the participation of the audience. But as long as the actions of audience detected, they generated complete arts. No strong indications needed. No invasive devices used. No specific or instructed actions needed to be done on the audience part. The actions of audience were just captured and abstracted to the outputs. Machines and audience acted as co-artist to produce a great piece of visual reality on their comprehensive understanding of each others’ reactions and indications. The advanced usage of science and technology, by no doubt, contributed to the success of the two artifacts. But more importantly, they invited the audience to join the creation of art, the visualization of audience’s mentality, while keeping the naturality of human movements, which could be described as “Dynamic-Interactive (Communicating) art systems”(Edmonds, n.d.). The lasting outputs, the combinations of lights are “Attracting”; the  varyings, the oscillation and vibration of lights are “Sustaining”; On the short time that were needed for the changes activated by the audience to show, they are very much “Relating”.(Edmonds, n.d.) The triumph of interactive arts depends not on the many sensors, detectors or high-techs used, but that how much it could interact with the audience, how much naturality can be kept for the audience and how the outputs vary and perceived by the audience. 

Works Cited

Dockery, Regan. “Light & Sound Synthesis: In Conversation with Amay Kataria.”interviewed  by Regan Dockery. In Environment Sound. Can-V5. Creative Applications. 2022. Accessed Nov. 15th, 2022. https://www.creativeapplications.net/environment/light-sound-synthesis-in-conversation-with-amay-kataria/

Edmonds, Ernest. “Introduction.” In Art, Interaction and Engagement. N.d. https://drive.google.com/file/d/1YYyhVTtNTBXQP2dlK8vE_QRJnpmCGvTE/view

Eats, Mark. “MIDI Visualizer Experiment 2.” Vimeo. https://vimeo.com/90654648?embedded=true&source=vimeo_logo&owner=443954

Visnjic, Filip. “Visual Sound Experiments – Transforming Suburbia with Light and Sound.” In Members openFrameworks Sound. Can-V5. Creative Applications. 2022. Accessed Nov. 15th, 2022. https://www.creativeapplications.net/sound/visual-sound-experiments-transforming-suburbia-with-light-and-sound/

Initially, I want to draw a face of a man. But on the piece of paper given, I realized that no matter how I draw, it would be too hard to find the exact coordinates corresponding to each dot that I used. Thus, I decided to make something that went more randomly, to say, grass. 

This was the graph that I wanna draw, but then I realized that it was kind of hard to draw because of all those lines and shapes of grass. So I wanted it automatically done. Thus, I made the following. 

The code is here. 

void setup() {
size(800,800);
strokeWeight(10);
background(255);
}
void draw() {

fill(0,200,100);
beginShape();
vertex(0,700);
vertex(100,80);
vertex(200,400);
vertex(300,80);
vertex(400,500);
vertex(500,80);
vertex(600,400);
vertex(700,80);
vertex(800,700);
endShape();
color(100,100,100,100);
fill(0,200,150);
for (int i = 0; i < 40; i+=10) {
bezier(
width/2, height,
width/2, random(height),
random(width), random(height),
random(width), random(height)
);
}
}