Week 7 Connecting Arduino to p5.js :)

Iteration 2 (Oct.31st) 

I successfully made things work!! It turned out that I shouldn’t overburden my p5.js. It can work with simple codes. 

This is me trying to test out the button and the potentiometer. The console.log function is working.

Now I am trying to use the potentiometer to control the volume — it can’t be reflected in the video because I was wearing headphones, but I could tell the difference, and the background was changing. 

the codes: edit link | fullscreen link

lessons learnt 

“serial.write(“x”); ” is very essential to trigger and maintain the communication between p5 and arduino

Before, my serial.write(“x”); only happened once (when I started to run the script), because it was only in the initiateSerial() function. It should also be in the serialEvent() function as well ;’) 

The reason why it worked (but slowly) was because I am printing stuff outside of the if statement. 

Also to maintain realtime communication, I should close the serial monitor in arduino. 

—————————————————-

Pervious iterations 

This week I explored two sets of codes. I tried to play with music & tried to manipulate the images with p5.js. 

Playing with music — visualizing the music

I got my inspiration from a video I saw recently. In Shanghai there is a club that held an event for deaf people — although deaf people can’t hear the music, they can use their body to feel the music beats and use their eyes to perceive the changes of the lightings. Initially I was thinking maybe I can use a sound input to detect the music, however, I got banned from the shop again?? So I gave up on this idea, and this project is a bit underdeveloped ; ; 

editing link | fullscreen link

One problem I encountered was that when I changed the project into fullscreen, the threshold of color changing might not work well. It was a bit difficult to do the math calculation. 

reference: youtube video by colorful coding link 

Image Manipulation

I was trying to play with vapor wave aesthetics in the very beginning because I thought it is fun :). But I ended up referencing this youtube video (link). What I have in mind is that when I hit the button connected to arduino, the effect will be triggered. However, I think there might be too many codes going on inside my computer that it never succeeded. (But when I tried with other codes, it worked.

with arduino – fullscreen link — editing link

without arduino – fullscreen link — editing link

notes ive taken based on the YouTube video: 

#version 300 es 
// GLES (OpenGL Shading Language) – this MUST be the first line of the code

// this is a fragment shader, which computes the color of each pixel in a rendered image

precision mediump float;
// sets the precision of floaating-point variables to meduim, can be lowp or highp as well
// but changing it will affect how floating-point numbers are handled

in vec2 pos;
out vec4 colour;
// int – integer value
// vec2, vec3, vec4 – xD vector;
// vec2 – consists of 2 floating-point nums, to present positions or directions
// vec3 – colors, 3D positions or directions
// vec4 – RGBA colors, 3D positions with an additional component for transformations

uniform sampler2D image;
uniform vec2 aspect;
uniform float t; //time
uniform vec2 centre; 
// uniform is used to declare global variables that remain constant during execution of a shader program for all pixels or vertices. often used for data that need to remain cosistent

const float maxRadius = 0.5;

// SDF from https://iquilezles.org/articles/distfunctions2d/
float sdBox(vec2 p, vec2 b) {
vec2 d = abs(p)-b;
return length(max(d,0.0)) + min(max(d.x,d.y),0.0);
}

float getOffsetStrength(float t, vec2 dir) {
// float d = length(dir/aspect) – t * maxRadius; // SDF of circle
float d = sdBox(dir/aspect, vec2(t * maxRadius));//SDF of a square

d *= 1. – smoothstep(0., 0.05, abs(d)); // Mask the ripple

d *= smoothstep(0., 0.05, t); // Smooth intro
d *= 1. – smoothstep(0.5, 1., t); // Smooth outro
return d;
}

void main() {
// 4. Chromatic aberation – color effect
vec2 dir = centre – pos;
float tOffset = 0.05 * sin(t * 3.14);
float rD = getOffsetStrength(t + tOffset, dir);
float gD = getOffsetStrength(t, dir);
float bD = getOffsetStrength(t – tOffset, dir);

dir = normalize(dir);
// normalize returns a vector with the same direction as its parameter, v, but with length 1.

float r = texture(image, pos + dir * rD).r;
float g = texture(image, pos + dir * gD).g;
float b = texture(image, pos + dir * bD).b;

float shading = gD * 8.;
colour = vec4(r, g, b, 1.);
colour.rgb += shading;
}

references:

Inigo Quilez link

youtube tutorial link