Case Study I. Generative Arts — Music Visualization Inspiration

Audio Visualization Project — Inspirations and Methods

Inspired by:

“Creative Audio Visualizers” on tympanus.net: https://tympanus.net/Development/AudioVisualizers/

Coding Reference: https://github.com/codrops/AudioVisualizers/

Coding explanation: https://tympanus.net/codrops/2018/03/06/creative-audio-visualizers/

Audio Visualization with P5js 

Since my aim is to visualize classical music through 2D/3D animation created in (or imported into) p5js, I researched online for visualization projects that use p5js as the scripting language. After researching online, I decided to start with 2D animation for visualization. I got inspiration from “Creative Audio Visualizers” about the visual elements and sketched the method for achieving my goal.

Aesthetics of “Creative Audio Visualizers”

In the 5 demos, visualization is mainly completed through the transformation of 2D geometries and lines. 4 Demos also add some mouse interactions in visual transformations. All the transformations are centered around the middle point of the canvas and have spinning effects.

The above two demos mainly utilize lines as visual elements, which caters to the shrill feeling of the music.

With a combination of squares and lines, Demo4 creates the feeling of a black hole, into which things are being swallowed. The darkness and the mystery of music are well depicted through the color and the graphics. The music in demo5 is more colorful and soothing compared to the previous demos. And the color of the graphic as well as the transformation is also more gentle. The lines are tangling up instead of pointing towards the audience as if showing a net weaved by dreaming. The triangles, on the other hand, gives a mysterious feeling. Demo3 is the most special one, as there is no mouse interaction and the dots become the dominant visual elements. The music in Demo3 is the most peaceful and intangible (according to my feelings). The simpleness of the visual effect represents the music well.

Methods

The artist first uses the loadSound function to load audio and then extracts the amplitude as the variable that will control the graphic transformation.

To add more variations, the artist utilizes the FFT object in this project, which enables him/her to extract the amplitude of different instruments played. Detailed coding is specified below (cited from Codrops). 

// Initiate the FFT object
  var fft = new p5.FFT();

  // Run the analysis, while the audio is playing
  fft.analyze();

  // Get different values for different frequency ranges

  // p5.sound comes with predefined keywords, 
  // but giving getEnergy() 2 numbers instead of a keyword 
  // you could use your custom range if needed
  var bass    = fft.getEnergy( "bass" );
  var treble  = fft.getEnergy( "treble" );
  var mid     = fft.getEnergy( "mid" );     
  var custom  = fft.getEnergy( 100, 200 );

The extracted amplitude values are stored as variables and used in the creation of visual elements such as lines and squares. In this way, the positions, as well as the shape of the visual elements, are constantly changing according to the amplitude of the music.