IMA Documentation

Project Link (Temporary): http://thomastai.com:3001/
Project Code: https://github.com/thomastai1666/Open-Orchestra

Video Demo

Introduction

For this week’s project, we were asked to design and implement an interactive exploration of melody. I was inspired by the projects we saw in class, and other projects I have seen on the internet. Most notably, I like the strings example from Chrome’s music lab and the “In C” web performance of Terry Riley’s piece In C. Since we are all separated right now, I thought it would be a great time to make a networked application to make music together.

Chrome Music Lab: Strings

Design

When I think of music, it is often represented on a score like the one shown below. The higher the line, the higher the frequency of the note. The lower the line, the lower the frequency of the note is. This was my first choice for the interface. 

Source: https://blog.flat.io/music-theory-ryhthm-measure/

I made mockups of my interface in Sketch, which later evolved and changed as I used predefined bootstrap styles and added buttons and features as I was designing the application. I drew inspiration from stringed instruments and decided on using an Orchestra (cello, bass, violin, viola) as my instruments.

Code

Due to time constraints, many of the components that are used were from other projects. Most of my work was to bring together to components to make the interface I desired. I defined an Instrument object which also inherits the Magenta Player and Tone Player. For networking, I used the socket.IO library which makes use of the shiny new web sockets introduced a few years ago. I send the note value and instrument name of each note being played to all the other users online. It is somewhat hacked together like some of the code, but it works just fine. When a player plucks the string, the data is sent to the Piano Genie model to predict a suitable note to play. It makes use of LSTM (Long short term memory) model to find a suitable sequence.

Future Work

There are some minor changes I would like to make if given the time. I find when too many notes are played at once, the chords tend to be in disharmony. This is a limitation of the Piano Genie model I used in the project. It wasn’t intended for use with string instruments, so it certainly isn’t optimized for this purpose. In fact, there are probably better ways to choose notes that don’t use a machine learning model. I found using a simple scale made more pleasing sounds.

Sources and Attribution

Chrome Experiments Strings: https://musiclab.chromeexperiments.com/Strings/

String Code:
https://experiments.withgoogle.com/jam-with-chrome

Clef Images: 
https://en.wikipedia.org/wiki/Clef

Magenta Library and Demo Code: 
https://magenta.tensorflow.org/pianogenie

Socket.IO
https://socket.io/

Introduction

For this week’s task, we were asked to create a Melody Sequencer. We were to use the Tone.js Sampler and set each track to play a different pitch. I added a different number of steps but kept the same pitches, just several octaves higher than the one from class. There are exactly 8 different frequencies or one octave of notes from the C major scale.

Demo

Link: https://editor.p5js.org/thomastai/sketches/9rjFm1j9A

Process

I started with the code from the previous drum machine. I added 4 more tracks for a total of 8 frequencies on this sequencer. If I had more time, I would add a button to control the number of octaves. The colors were chosen from a rainbow I found online that I found aesthetically pleasing and added to an array. Using the included css stylesheet, I added custom css to customize the look and feel of the slider and buttons. I also added a much needed play and pause button since Tone.js can randomly stop and pause without notice. I found the harp soundfile from freewavesamples.com and used the Tone.js sampler to create the other pitches on the scale. Overall, I’m pleased with the sound but I think the chords could be improved as there is some distortion when multiple notes are played. I would like to also experiment with different types of input and interfaces as this melody sequencer is quite simple.