Design for Real Life – QWERTYBeats research

Writing assignment for Design For The Real World with Claire Kearney-Volpe and Diana Castro – research about a new rhythm interface for blind and low-vision novice musicians

Definition

I propose a new web-based accessible rhythm instrument called QWERTYBeats.Traditional instruments are highly accessible to blind and low-vision musicians. Electronic music production tools are not. I look at the history of accessible instruments and software interfaces, give an overview of current electronic music hardware and software, and discuss the design considerations underlying my project.

QWERTYBeats logo

Historical overview

Acoustic instruments give rich auditory and haptic feedback, and pose little obstacle to blind musicians. We need look no further for proof than the long history of iconic blind musicians like Ray Charles and Stevie Wonder. Even sighted instrumentalists rarely look at their instruments once they have attained a sufficient level of proficiency. Music notation is not accessible, but Braille notation has existed since the language’s inception. Also, a great many musicians both blind and sighted play entirely by ear anyway.

Most of the academic literature around accessibility issues in music education focuses on wider adoption of and support for Braille notation. See, for example, Rush, T. W. (2015). Incorporating Assistive Technology for Students with Visual Impairments into the Music Classroom. Music Educators Journal, 102(2), 78–83. For electronic music, notation is rarely if ever a factor.

Electronic instruments pose some new accessibility challenges. They may use graphical interfaces with nested menus, complex banks of knobs and patch cables, and other visual control surfaces. Feedback may be given entirely with LED lights and small text labels. Nevertheless, blind users can master these devices with sufficient practice, memorization and assistance. For example, Stevie Wonder has incorporated synthesizers and drum machines in most of his best-known recordings.

Most electronic music creation is currently done not with instruments, but rather using specialized software applications called digital audio workstations (DAWs). Keyboards and other controllers are mostly used to access features of the software, rather than as standalone instruments. The most commonly-used DAWs include Avid Pro Tools, Apple Logic, Ableton Live, and Steinberg Cubase. Mobile DAWs are more limited than their desktop counterparts, but are nevertheless becoming robust music creation tools in their own right. Examples include Apple GarageBand and Steinberg Cubasis. Notated music is commonly composed using score editing software like Sibelius and Finale, whose functionality increasingly overlaps with DAWs, especially in regard to MIDI sequencing.

DAWs and notation editors pose steep accessibility challenges due to their graphical and spatial interfaces, not to mention their sheer complexity. In class, we were given a presentation by Leona Godin, a blind musician who records and edits audio using Pro Tools by means of VoiceOver. While it must have taken a heroic effort on her part to learn the program, Leona demonstrates that it is possible. However, some DAWs pose insurmountable problems even to very determined blind users because they do not use standard operating system elements, making them inaccessible via screen readers.

Technological interventions

There are no mass-market electronic interfaces specifically geared toward blind or low-vision users. In this section, I discuss one product frequently hailed for its “accessibility” in the colloquial rather than blindess-specific sense, along with some more experimental and academic designs.

Ableton Push

Push layout for IMPACT Faculty Showcase

Ableton Live has become the DAW of choice for electronic music producers. Low-vision users can zoom in to the interface and modify the color scheme. However, Live is inaccessible via screen readers.

In recent years, Ableton has introduced a hardware controller, the Push, which is designed to make the software experience more tactile and instrument-like. The Push combines an eight by eight grid of LED-lit touch pads with banks of knobs, buttons and touch strips. It makes it possible to create, perform and record a piece of music from scratch without looking at the computer screen. In addition to drum programming and sampler performance, the Push also has an innovative melodic mode which maps scales onto the grid in such a way that users can not play a wrong note. Other comparable products exist; see, for example, the Native Instruments Maschine.

There are many pad-based drum machines and samplers. Live’s main differentiator is its Session view, where the pads launch clips: segments of audio or MIDI that can vary in length from a single drum hit to the length of an entire song. Clip launching is tempo-synced, so when you trigger a clip, playback is delayed until the start of the next measure (or whatever the quantization interval is.) Clip launching is a forgiving and beginner-friendly performance method, because it removes the possibility of playing something out of rhythm. Like other DAWs, Live also gives rhythmic scaffolding in its software instruments by means of arpeggiators, delay and other tempo-synced features.

The Push is a remarkable interface, but it has some shortcomings for blind users. First of all, it is expensive, $800 for the entry-level version and $1400 for the full-featured software suite. Much of its feedback is visual, in the form of LED screens and color-coded lighting on the pads. It switches between multiple modes which can be challenging to distinguish even for sighted users. And, like the software it accompanies, the Push is highly complex, with a steep learning curve unsuited to novice users, blind or sighted.

The aQWERTYon

Most DAWs enable users to perform MIDI instruments on the QWERTY keyboard. The most familiar example is the Musical Typing feature in Apple GarageBand.

GarageBand musical typing

Musical Typing makes it possible to play software instruments without an external MIDI controller, which is convenient and useful. However, its layout counterintuively follows the piano keyboard, which is an awkward fit for the computer keyboard. There is no easy way to distinguish the black and white keys, and even expert users find themselves inadvertantly hitting the keyboard shortcut for recording while hunting for F-sharp.

The aQWERTYon is a web interface developed by the NYU Music Experience Design Lab specifically intended to address the shortcomings of Musical Typing.

aQWERTYon screencap

Rather than emulating the piano keyboard, the aQWERTYon draws its inspiration from the chord buttons of an accordion. It fills the entire keyboard with harmonically related notes in a way that supports discovery by naive users. Specifically, it maps scales across the rows of keys, staggered by intervals such that each column forms a chord within the scale. Root notes and scales can be set from pulldown menus within the interface, or preset using URL parameters. It can be played as a standalone instrument, or as a MIDI controller in conjunction with a DAW. Here is a playlist of music I created using the aQWERTYon and GarageBand or Ableton Live:

The aQWERTYon is a completely tactile experience. Sighted users can carefully match keys to note names using the screen, but more typically approach the instrument by feel, seeking out patterns on the keyboard by ear. A blind user would need assistance loading the aQWERTYon initially and setting the scale and root note parameters, but otherwise, it is perfectly accessible. The present project was motivated in large part by a desire to make exploration of rhythm as playful and intuitive as the aQWERTYon makes exploring chords and scales.

Soundplant

The QWERTY keyboard can be turned into a simple drum machine quite easily using a free program called Soundplant. The user simply drags audio files onto a graphical key to have it triggered by that physical key. I was able to create a TR-808 kit in a matter of minutes:

Soundplant with 808 samples

After it is set up and configured, Soundplant can be as effortlessly accessible as the aQWERTYon. However, it does not give the user any rhythmic assistance. Drumming in perfect time is an advanced musical skill, and playing drum machine samples out of time is not much more satisfying than banging on a metal bowl with a spoon out of time. An ideal drum interface would offer beginners some of the rhythmic scaffolding and support that Ableton provides via Session view, arpeggiators, and the like.

The Groove Pizza

Drum machines and their software counterparts offer an alternative form of rhythmic scaffolding. The user sequences patterns in a time-unit box system or piano roll, and the computer performs those patterns flawlessly. The MusEDLab‘s Groove Pizza app is a web-based drum sequencer that wraps the time-unit box system into a circle.

Groove Pizza - Bembe

The Groove Pizza was designed to make drum programming more intuitive by visualizing the symmetries and patterns inherent in musical-sounding rhythms. However, it is totally unsuitable for blind or low-vision users. Interaction is only possible through the mouse pointer or touch, and there are no standard user interface elements that can be parsed by screen readers.

Before ever considering designing for the blind, the MusEDLab had already considered the Groove Pizza’s limitations for younger children and users with special needs: there is no “live performance” mode, and there is always some delay in feedback between making a change in the drum pattern and hearing the result. We have been considering ways to make a rhythm interface that is more immediate, performance-oriented and tactile. One possible direction would be to create a hardware version of the Groove Pizza; indeed, one of the earliest prototypes was a hardware version built by Adam November out of a pizza box. However, hardware design is vastly more complex and difficult than software, so for the time being, software promises more immediate results.

Haenselmann-Lemelson-Effelsberg MIDI sequencer

This experimental interface is described in Haenselmann, T., Lemelson, H., & Effelsberg, W. (2011). A zero-vision music recording paradigm for visually impaired people. Multimedia Tools and Applications, 5, 1–19.

Haenselmann-Lemelson-Effelsberg MIDI sequencer

The authors create a new mode for a standard MIDI keyboard that maps piano keys to DAW functions like playback, quantization, track selection, and so on. They also add “earcons” (auditory icons) to give sonic feedback when particular functions have been activated that normally only give graphical feedback. For example, one earcon sounds when recording is enabled; another sounds for regular playback. This interface sounds promising, but there are significant obstacles to its adoption. While the authors have released the source code as a free download, that requires a would-be user to be able to compile and run it. This is presuming that they could access the code in the first place; the download link given in the paper is inactive. It is an all-too-common fate of academic projects to never get widespread usage. By posting our projects on the web, the MusEDLab hopes to avoid this outcome.

Statement

Music education philosophy

My project is animated by a constructivist philosophy of music education, which operates by the following axiomatic assumptions:

  • Learning by doing is better than learning by being told.
  • Learning is not something done to you, but rather something done by you.
  • You do not get ideas; you make ideas. You are not a container that gets filled with knowledge and new ideas by the world around you; rather, you actively construct knowledge and ideas out of the materials at hand, building on top of your existing mental structures and models.
  • The most effective learning experiences grow out of the active construction of all types of things, particularly things that are personally or socially meaningful, that you develop through interactions with others, and that support thinking about your own thinking.

If an activity’s challenge level is beyond than your ability, you experience anxiety. If your ability at the activity far exceeds the challenge, the result is boredom. Flow happens when challenge and ability are well-balanced, as seen in this diagram adapted from Csikszentmihalyi.

Flow

Music students face significant obstacles to flow at the left side of the Ability axis. Most instruments require extensive practice before it is possible to make anything that resembles “real” music. Electronic music presents an opportunity here, because even a complete novice can produce music with a high degree of polish quickly. It is empowering to use technologies that make it impossible to do anything wrong; it frees you to begin exploring what you find to sound right. Beginners can be scaffolded in their pitch explorations with MIDI scale filters, Auto-Tune, and the configurable software keyboards in apps like Thumbjam and Animoog. Rhythmic scaffolding is more rare, but it can be had via Ableton’s quantized clip launcher, by MIDI arpeggiators, and using the Note Repeat feature on many drum machines.

QWERTYBeats proposal

My project takes drum machine Note Repeat as its jumping off point. When Note Repeat is activated, holding down a drum pad triggers the corresponding sound at a particular rhythmic interval: quarter notes, eighth notes, and so on. On the Ableton Push, Note Repeat automatically syncs to the global tempo, making it effortless to produce musically satisfying rhythms. However, this mode has a major shortcoming: it applies globally to all of the drum pads. To my knowledge, no drum machine makes it possible to simultaneously have, say, the snare drum playing every dotted eighth note while the hi-hat plays every sixteenth note.

I propose a web application called QWERTYBeats that maps drums to the computer keyboard as follows:

  • Each row of the keyboard triggers a different drum/beatbox sound (e.g. kick, snare, closed hi-hat, open hi-hat).
  • Each column retriggers the sample at a different rhythmic interval (e.g. quarter note, dotted eighth note).
  • Circles dynamically divide into “pie slices” to show rhythmic values.

The rhythm values are shown below by column, with descriptions followed by the time interval as shown as a fraction of the tempo in beats per minute.

  1. quarter note (1)
  2. dotted eighth note (3/4)
  3. quarter note triplet (2/3)
  4. eighth note (1/2)
  5. dotted sixteenth note (3/8)
  6. eighth note triplet (1/3)
  7. sixteenth note (1/4)
  8. dotted thirty-second note (3/16)
  9. sixteenth note triplet (1/6)
  10. thirty-second note (1/8)

By simply holding down different combinations of keys, users can attain complex syncopations and polyrhythms. If the app is synced to the tempo of a DAW or music playback, the user can perform good-sounding rhythms over any song that is personally meaningful to them.

The column layout leaves some unused keys in the upper right corner of the keyboard: “-“, “=”, “[“, “]”, “”, etc. These can be reserved for setting the tempo and other UI elements.

The app defaults to Perform Mode, but clicking Make New Kit opens Sampler mode, where users can import or record their own drum sounds:

  • Keyboard shortcuts enable the user to select a sound, audition it, record, set start and end point, and set its volume level.
  • A login/password system enables users to save kits to the cloud where they can be accessed from any computer. Kits get unique URL identifiers, so users can also share them via email or social media.

It is my goal to make the app accessible to users with the widest possible diversity of abilities.

  • The entire layout will use plain text, CSS and JavaScript to support screen readers.
  • All user interface elements can be accessed via the keyboard: tab to change the keyboard focus, menu selections and parameter changes via the up and down arrows, and so on.

Perform Mode:

QWERTYBeats concept images - Perform mode

Sampler Mode:

sampler-mode

Mobile version

The present thought is to divide up the screen into a grid mirroring the layout of the QWERTY keyboard. User testing will determine whether this will produce a satisfying experience.

QWERTYDrum - mobile

Prototype

I created a prototype of the app using Ableton Live’s Session View.

QWERTYBeats - Ableton prototype

Here is a sample performance:

There is not much literature examining the impact of drum programming and other electronic rhythm sequencing on students’ subsequent ability to play acoustic drums, or to keep time more accurately in general. I can report anecdotally that my own time spent sequencing and programming drums improved my drumming and timekeeping enormously (and mostly inadvertently.) I will continue to seek further support for the hypothesis that electronically assisted rhythm creation builds unassisted rhythmic ability. In the meantime, I am eager to prototype and test QWERTYBeats.

Visualizing trap beats with the Groove Pizza

In a previous post, I used the Groove Pizza to visualize some classic hip-hop beats. But the kids are all about trap beats right now, which work differently from the funk-based boom-bap of my era.

IT'S A TRAP

From the dawn of jazz until about 1960, African-American popular music was based on an eighth note pulse. The advent of funk brought with it a shift to the sixteenth note pulse. Now we’re undergoing another shift, as Southern hip-hop is moving the rest of popular music over to a 32nd note pulse. The tempos have been slowing down as the beat subdivisions get finer. This may all seem like meaningless abstraction, but the consequences become real if you want to program beats of your own.

Back in the 90s, the template for a hip-hop beat looked like a planet of 16th notes orbited by kicks and snares. Click the image below to hear a simple “planet funk” pattern in the Groove Pizza. Each slice of the pizza is a sixteenth note, and the whole pizza is one bar long.

Planet Funk - 16th notes

(Music readers can also view it in Noteflight.)

You can hear the sixteenth note hi-hat pulse clearly in “So Fresh So Clean” by OutKast.

So Fresh So Clean

View in Noteflight

Trap beats have the same basic skeleton as older hip-hop styles: a kick on beat one, snares on beats two and four, and hi-hats on some or all of the beats making up the underlying pulse. However, in trap, that pulse is twice as fast as in 90s hip-hop, 32nd notes rather than sixteenths. This poses an immediate practical problem: a lot of drum machines don’t support such a fine grid resolution. For example, the interface of the ubiquitous TR-808 is sixteen buttons, one for each sixteenth note. On the computer, it’s less of an issue because you can set the grid resolution to be whatever you want, but even so, 32nd notes are a hassle. So what do you do?

The trap producer’s workaround is to double the song tempo, thereby turning sixteenths into effective 32nds. To get a trap beat at 70 beats per minute, you set the tempo to 140. Your 808 grid becomes half a bar of 32nd notes, rather than a full bar of sixteenths. And instead of putting your snares on beats two and four, you put them on beat three.

Here’s a generic trap beat I made. Each pizza slice is a 32nd note, and the whole pizza is half a bar.

View in Noteflight

Trap beats don’t use swing. Instead, they create rhythmic interest through syncopation, accenting unexpected weak beats. On the Groove Pizza, the weak beats are the ones in between the north, south, east and west. Afro-Cuban music is a good source of syncopated patterns. The snare pattern in the last quarter of my beat is a rotation of son clave, and the kick pattern is somewhat clave-like as well.

It's A Trap - last bar

Now let’s take a look at two real-life trap beats. First, there’s the inescapable “Trap Queen” by Fetty Wap.

Here’s a simplified version of the beat. (“Trap Queen” uses a few 64th notes on the hi-hat, which you can’t yet do on the Groove Pizza.)

Trap Queen simplified

View in Noteflight

The beat has an appealing symmetry. In each half bar, both the kick and snare each play a strong beat and a weak beat. The hi-hat pattern is mostly sixteenth notes, with just a few thirty-second notes as embellishments. The location of those embellishments changes from one half-bar to the next. It’s a simple technique, and it’s effective.

My other real-world example is “Panda” by Desiigner.

Here’s the beat on the GP, once again simplified a bit.

View in Noteflight

Unlike my generic trap beat, “Panda” doesn’t have any hi-hats on the 32nd notes at all. It feels more like an old-school sixteenth note pulse at a very slow tempo. The really “trappy” part comes at the very end, with a quick pair of kick drums on the last two 32nd notes. While the lawn-sprinkler effect of doubletime hi-hats has become a cliche, doubletime kick rolls are still startlingly fresh (at least to my ears.)

To make authentic trap beats, you’ll need a more full-featured tool than the Groove Pizza. For one thing, you need 64th notes and triplets. Also, trap isn’t just about the placement of the drum hits, it’s about specific sounds. In addition to closed hi-hats, you  need open hi-hats and crash cymbals. You want more than one snare or handclap, and maybe multiple kicks too. And you’d want to be able to alter the pitch of your drums too. The best resource to learn more, as always, is the music itself.

The evolution of the Groove Pizza

The Groove Pizza is a playful tool for creating grooves using math concepts like shapes, angles, and patterns. Here’s a beat I made just nowTry it yourself!

 
This post explains how and why we designed Groove Pizza.

What it does

The Groove Pizza represents beats as concentric rhythm necklaces. The circle represents one measure. Each slice of the pizza is a sixteenth note. The outermost ring controls the kick drum; the middle one controls the snare; and the innermost one plays cymbals.

Connecting the dots on a given ring creates shapes, like the square formed by the snare drum in the pattern below.

Groove Pizza - jazz swing

The pizza can play time signatures other than 4/4 by changing the number of slices. Here’s a twelve-slice pizza playing an African bell pattern.

Groove Pizza - Bembe

You can explore the geometry of musical rhythm by dragging shapes onto the circular grid. Patterns that are visually appealing tend to sound good, and patterns that sound good tend to look cool.

Groove Pizza - shapes

Herbie Hancock did some user testing for us, and he suggested that we make it possible to show the interior angles of the shapes.

Groove Pizza - angles

Groove Pizza History

The ideas behind the Groove Pizza began in my masters thesis work in 2013 at NYU. For his NYU senior thesis, Adam November built web and physical prototypes. In late summer 2015, Adam wrote what would become the Groove Pizza 1.0 (GP1), with a library of drum patterns that he and I curated. The MusEDLab has been user testing this version for the past year, both with kids and with music and math educators in New York City.

In January 2016, the Music Experience Design Lab began developing the Groove Pizza 2.0 (GP2) as part of the MathScienceMusic initiative.

MathScienceMusic Groove Pizza Credits:

  • Original Ideas: Ethan Hein, Adam November & Alex Ruthmann
  • Design: Diana Castro
  • Software Architect: Kevin Irlen
  • Creative Code Guru: Matthew Kaney
  • Backend Code Guru: Seth Hillinger
  • Play Testing: Marijke Jorritsma, Angela Lau, Harshini Karunaratne, Matt McLean
  • Odds & Ends: Asyrique Thevendran, Jamie Ehrenfeld, Jason Sigal

The learning opportunity

The goals of the Groove Pizza are to help novice drummers and drum programmers get started; to create a gentler introduction to beatmaking with more complex tools like Logic or Ableton Live; and to use music to open windows into math and geometry. The Groove Pizza is intended to be simple enough to be learned easily without prior experience or formal training, but it must also have sufficient depth to teach substantial and transferable skills and concepts, including:

  • Familiarity with the component instruments in a drum beat and the ability to pick them individually out of the sound mass.
  • A repertoire of standard patterns and rhythmic motifs. Understanding of where to place the kick, snare, hi-hats and so on to produce satisfying beats.
  • Awareness of different genres and styles and how they are distinguished by their different degrees of syncopation, customary kick drum patterns and claves, tempo ranges and so on.
  • An intuitive understanding of the difference between strong and weak beats and the emotional effect of syncopation.
  • Acquaintance with the concept of hemiola and other more complex rhythmic devices.

Marshall (2010) recommends “folding musical analysis into musical experience.” Programming drums in pop and dance idioms makes the rhythmic abstractions concrete.

Visualizing rhythm

Western music notation is fairly intuitive on the pitch axis, where height on the staff corresponds clearly to pitch height. On the time axis, however, Western notation is less easily parsed—horizontal space need not have any bearing at all on time values. A popular alternative is the “time-unit box system,” a kind of rhythm tablature used by ethnomusicologists. In a time-unit box system, each pulse is represented by a square. Rhythmic onsets are shown as filled boxes.

Clave patterns in TUBS

Nearly all electronic music production interfaces use the time-unit box system scheme, including grid sequencers and the MIDI piano roll.

Ableton TUBS

A row of time-unit boxes can also be wrapped in a circle to form a rhythm necklace. The Groove Pizza is simply a set of rhythm necklaces arranged concentrically.

Circular rhythm visualization offers a significant advantage over linear notation: it more clearly shows metrical function. We can define meter as “the grouping of perceived beats or pulses into equivalence classes” (Forth, Wiggin & McLean, 2010, 521). Linear musical concepts like small-scale melodies depend mostly on relationships between adjacent events, or at least closely spaced events. But periodicity and meter depend on relationships between nonadjacent events. Linear representations of music do not show meter directly. Simply by looking at the page, there is no indication that the first and third beats of a measure of 4/4 time are functionally related, as are the second and fourth beats.

However, when we wrap the musical timeline into a circle, meter becomes much easier to parse. Pairs of metrically related beats are directly opposite one another on the circle. Rotational and reflectional symmetries give strong clues to metrical function generally. For example, this illustration of 2-3 son clave adapted from Barth (2011) shows an axis of reflective symmetry between the fourth and twelfth beats of the pattern. This symmetry is considerably less obvious when viewed in more conventional notation.

Son clave symmetry

The Groove Pizza adds a layer of dynamic interaction to circular representation. Users can change time signatures during playback by adding or removing slices. In this way, very complex metrical shifts can be performed by complete novices. Furthermore, each rhythm necklace can be rotated during playback, enabling a rhythmic modularity characteristic of the most sophisticated Afro-Latin and jazz rhythms. Exploring rotational rhythmic transformation typically requires very sophisticated music-reading and performance skills to understand and execute, but doing so is effortlessly accessible to Groove Pizza users.

Visualizing swing

We traditionally associate swing with jazz, but it is omnipresent in American vernacular music: in rock, country, funk, reggae, hip-hop, EDM, and so on. For that reason, swing is a standard feature of notation software, MIDI sequencers, and drum machines. However, while swing is crucial to rhythmic expressiveness, it is rarely visualized in any explicit way, in notation or in software interfaces. Sequencers will sometimes show swing by displacing events on the MIDI piano roll, but the user must place those events first. The grid itself generally does not show swing.

The Groove Pizza uses a novel (and to our knowledge unprecedented) graphical representation of swing on the background grid, not just on the musical events. The slices alternately expand and contract in width according to the amount of swing specified. At 0% swing, the wedges are all of uniform width. At 50% swing, the odd-numbered slice in each pair is twice as long as the following even-numbered slice. As the user adjusts the swing slider, the slices dynamically change their width accordingly.

Straight 16ths vs swing 16ths

Our swing visualization system also addresses the issue of whether swing should be applied to eighth notes or sixteenths. In the jazz era, swing was understood to apply to eighth notes. However, since the 1960s, swing is more commonly applied to sixteenth notes, reflecting a broader shift from eighth note to sixteenth note pulse in American vernacular music. To hear the difference, compare the swung eighth note pulse of “Rockin’ Robin” by Bobby Day (1958) with the sixteenth note pulse of “I Want You Back” by the Jackson Five (1969). Electronic music production tools like Ableton Live and Logic default to sixteenth-note swing. However, notation programs like Sibelius, Finale and Noteflight can only apply swing to eighth notes.

The Groove Pizza supports both eighth and sixteenth swing simply by changing the slice labeling. The default labeling scheme is agnostic, simply numbering the slices sequentially from one. In GP1, users can choose to label a sixteen-slice pizza either as one measure of sixteenth notes or two measures of eighth notes. The grid looks the same either way; only the labels change.

Drum kits

With one drum sound per ring, the number of sounds available to the user is limited by the number of rings that can reasonably fit on the screen. In my thesis prototype, we were able to accommodate six sounds per “drum kit.” GP1 was reduced to five rings, and GP2 has only three rings, prioritizing simplicity over musical versatility.

GP1 offers three drum kits: Acoustic, Hip-Hop, and Techno. The Acoustic kit uses samples of a real drum kit; the Hip-Hop kit uses samples of the Roland TR-808 drum machine; and the Techno kit uses samples of the Roland TR-909. GP2 adds two additional kits: Jazz (an acoustic drum kit played with brushes), and Afro-Latin (congas, bell, and shaker.) Preset patterns automatically load with specific kits selected, but the user is free to change kits after loading.

In GP1, sounds can be mixed and matched at wiell, so the user can, for example, combine the acoustic kick with the hip-hop snare. In GP2, kits cannot be customized. A wider variety of sounds would present a wider variety of sonic choices. However, placing strict limits on the sounds available has its own creative advantage: it eliminates option paralysis and forces users to concentrate on creating interesting patterns, rather than struggling to choose from a long list of sounds.

It became clear in the course of testing that open and closed hi-hats need not operate separate rings, since it is not desirable to ever have them sound at the same time. (While drum machines are not bound by the physical limitations of human drummers, our rhythmic traditions are.) In future versions of the GP, we plan to place closed and open hi-hats together on the same ring. Clicking a beat in the hi-hat ring will place a closed hi-hat; clicking it again will replace it with an open hi-hat; and a third click will return the beat to silence. We will use the same mechanic to toggle between high and low cowbells or congas.

Preset patterns

In keeping with the constructivist value of working with authentic cultural materials, the exercises in the Groove Pizza are based on rhythms drawn from actual music. Most of the patterns are breakbeats—drums and percussion sampled from funk, rock and soul recordings that have been widely repurposed in electronic dance and hip-hop music. There are also generic rock, pop and dance rhythms, as well as an assortment of traditional Afro-Cuban patterns.

The GP1 offers a broad selection of preset patterns. The GP2 uses a smaller subset of these presets.

Breakbeats

  • The Winstons, ”Amen, Brother” (1969)
  • James Brown, ”Cold Sweat” (1967)”
  • James Brown, “The Funky Drummer” (1970)
  • Bobby Byrd, “I Know You Got Soul” (1971)
  • The Honeydrippers, “Impeach The President” (1973)
  • Skull Snaps, “It’s A New Day” (1973)
  • Joe Tex, ”Papa Was Too” (1966)
  • Stevie Wonder, “Superstition” (1972)
  • Melvin Bliss, “Synthetic Substitution”(1973)

Afro-Cuban

  • Bembé—also known as the “standard bell pattern”
  • Rumba clave
  • Son clave (3-2)
  • Son clave (2-3)

Pop

  • Michael Jackson, ”Billie Jean” (1982)
  • Boots-n-cats—a prototypical disco pattern, e.g. “Funkytown” by Lipps Inc (1979)
  • INXS, “Need You Tonight” (1987)
  • Uhnntsss—the standard “four on the floor” pattern common to disco and electronic dance music

Hip-hop

  • Lil Mama, “Lip Gloss” (2008)
  • Nas, “Nas Is Like” (1999)
  • Digable Planets, “Rebirth Of Slick (Cool Like Dat)” (1993)
  • OutKast, “So Fresh, So Clean” (2000)
  • Audio Two, “Top Billin’” (1987)

Rock

  • Pink Floyd, ”Money” (1973)
  • Peter Gabriel, “Solisbury Hill” (1977)
  • Billy Squier, “The Big Beat” (1980)
  • Aerosmith, “Walk This Way” (1975)
  • Queen, “We Will Rock You” (1977)
  • Led Zeppelin, “When The Levee Breaks” (1971)

Jazz

  • Bossa nova, e.g. “The Girl From Ipanima” by Antônio Carlos Jobim (1964)
  • Herbie Hancock, ”Chameleon” (1973)
  • Miles Davis, ”It’s About That Time” (1969)
  • Jazz spang-a-lang—the standard swing ride cymbal pattern
  • Jazz waltz—e.g. “My Favorite Things” as performed by John Coltrane (1961)
  • Dizzy Gillespie, ”Manteca” (1947)
  • Horace Silver, ”Song For My Father” (1965)
  • Paul Desmond, ”Take Five” (1959)
  • Herbie Hancock, “Watermelon Man” (1973)

Mathematical applications

The most substantial new feature of GP2 is “shapes mode.” The user can drag shapes onto the grid and rotate them to create geometric drum patterns: triangle, square, pentagon, hexagon, and octagon. Placing shapes in this way creates maximally even rhythms that are nearly always musically satisfying (Toussaint 2011). For example, on a sixteen-slice pizza, the pentagon forms rumba or bossa nova clave, while the hexagon creates a tresillo rhythm. As a general matter, the way that a rhythm “looks” gives insight into the way it sounds, and vice versa.

Because of the way it uses circle geometry, the Groove Pizza can be used to teach or reinforce the following subjects:

  • Fractions
  • Ratios and proportional relationships
  • Angles
  • Polar vs Cartesian coordinates
  • Symmetry: rotations, reflections
  • Frequency vs duration
  • Modular arithmetic
  • The unit circle in the complex plane

Specific kinds of music can help to introduce specific mathematical concepts. For example, Afro-Cuban patterns and other grooves built on hemiola are useful for graphically illustrating the concept of least common multiples. When presented with a kick playing every four slices and a snare playing every three slices, a student can both see and hear how they will line up every twelve slices. Bamberger and diSessa (2003) describe the “aha” moment that students have when they grasp this concept in a music context. One student in their study is quoted as describing the twelve-beat cycle “pulling” the other two beats together. Once students grasp least common multiples in a musical context, they have a valuable new inroad into a variety of scientific and mathematical concepts: harmonics in sound analysis, gears, pendulums, tiling patterns, and much else.

In addition to eighth and sixteenth notes, GP1 users can also label the pizza slices as fractions or angles, both Cartesian and polar. Users can thereby describe musical concepts in mathematical terms, and vice versa. It is an intriguing coincidence that the polar angle π/16 represents a sixteenth note. One could go even further with polar mode and use it as the unit circle on the complex plane. From there, lessons could move into powers of e, the relationship between sine and cosine waves, and other more advanced topics. The Groove Pizza could thereby be used to lay the ground work for concepts in electrical engineering, signal processing, and anything else involving wave mechanics.

Future work

The Groove Pizza does not offer any tone controls like duration, pitch, EQ and the like. This choice was due to a combination of expediency and the push to reduce option paralysis. However, velocity (loudness) control is a high-priority future feature. While nuanced velocity control is not necessary for the artificial aesthetic of electronic dance music, a basic loud/medium/soft toggle would make the Groove Pizza a more versatile tool.

The next step beyond preset patterns is to offer drum programming exercises or challenges. In exercises, users are presented with a pattern. They may alter this pattern as they see fit by adding and removing drum hits, and by rotating instrument parts within their respective rings. There are restraints of various kinds, to ensure that the results are appealing and musical-sounding. The restraints are tighter for more basic exercises, and looser for more advanced ones. For example, we might present users with a locked four-on-the-floor kick pattern, and ask them to create a satisfying techno beat using the snares and hi-hats. We also plan to create game-like challenges, where users are given the sound of a beat and must figure out how to represent it on the circular grid.

The Groove Pizza would be more useful for the purposes of trigonometry and circle geometry if it were presented slightly differently. Presently, the first beat of each pattern is at twelve o’clock, with playback running clockwise. However, angles are usually representing as originating at three o’clock and increasing in a counterclockwise direction. To create “math mode,” the radial grid would need to be reflected left-to-right and rotated ninety degrees.

References

Ankney, K.L. (2012). Alternative representations for musical composition. Visions of Research in Music Education, 20.

Bamberger, J., & DiSessa, A. (2003). Music As Embodied Mathematics: A Study Of A Mutually Informing Affinity. International Journal of Computers for Mathematical Learning, 8(2), 123–160.

Bamberger, J. (1996). Turning Music Theory On Its Ear. International Journal of Computers for Mathematical Learning, 1: 33-55.

Bamberger, J. (1994). Developing Musical Structures: Going Beyond the Simples. In R. Atlas & M. Cherlin (Eds.), Musical Transformation and Musical Intuition. Ovenbird Press.

Barth, E. (2011). Geometry of Music. In Greenwald, S. and Thomley, J., eds., Essays in Encyclopedia of Mathematics and Society. Ipswich, MA: Salem Press.

Bell, A. (2013). Oblivious Trailblazers: Case Studies of the Role of Recording Technology in the Music-Making Processes of Amateur Home Studio Users. Doctoral dissertation, New York University.

Benadon, F. (2007). A Circular Plot for Rhythm Visualization and Analysis. Music Theory Online, Volume 13, Issue 3.

Demaine, E.; Gomez-Martin, F.; Meijer, H.; Rappaport, D.; Taslakian, P.; Toussaint, G.; Winograd, T.; & Wood, D. (2009). The Distance Geometry of Music. Computational Geometry 42, 429–454.

Forth, J.; Wiggin, G.; & McLean, A. (2010). Unifying Conceptual Spaces: Concept Formation in Musical Creative Systems. Minds & Machines, 20:503–532.

Magnusson, T. (2010). Designing Constraints: Composing and Performing with Digital Musical Systems. Computer Music Journal, Volume 34, Number 4, pp. 62 – 73.

Marrington, M. (2011). Experiencing Musical Composition In The DAW: The Software Interface As Mediator Of The Musical Idea. The Journal on the Art of Record Production, (5).

Marshall, W. (2010). Mashup Poetics as Pedagogical Practice. In Biamonte, N., ed. Pop-Culture Pedagogy in the Music Classroom: Teaching Tools from American Idol to YouTube. Lanham, MD: Scarecrow Press.

McClary, S. (2004). Rap, Minimalism and Structures of Time in Late Twentieth-Century Culture. In Warner, D. ed., Audio Culture. London: Continuum International Publishing Group.

Monson, I. (1999). Riffs, Repetition, and Theories of Globalization. Ethnomusicology, Vol. 43, No. 1, 31-65.

New York State Learning Standards and Core Curriculum — Mathematics

Ruthmann, A. (2012). Engaging Adolescents with Music and Technology. In Burton, S. (Ed.). Engaging Musical Practices: A Sourcebook for Middle School General Music. Lanham, MD: R&L Education.

Thibeault, M. (2011). Wisdom for Music Education from the Recording Studio. General Music Today, 20 October 2011.

Thompson, P. (2012). An Empirical Study Into the Learning Practices and Enculturation of DJs, Turntablists, Hip-Hop and Dance Music Producers.” Journal of Music, Technology & Education, Volume 5, Number 1, 43 – 58.

Toussaint, G. (2013). The Geometry of Musical Rhythm. Cleveland: Chapman and Hall/CRC.

____ (2005). The Euclidean algorithm generates traditional musical rhythms. Proceedings of BRIDGES: Mathematical Connections in Art, Music, and Science, Banff, Alberta, Canada, July 31 to August 3, 2005, pp. 47-56.

____ (2004). A comparison of rhythmic similarity measures. Proceedings of ISMIR 2004: 5th International Conference on Music Information Retrieval, Universitat Pompeu Fabra, Barcelona, Spain, October 10-14, 2004, pp. 242-245.

____ (2003). Classification and phylogenetic analysis of African ternary rhythm timelines. Proceedings of BRIDGES: Mathematical Connections in Art, Music, and Science, University of Granada, Granada, Spain July 23-27, 2003, pp. 25-36.

____ (2002). A mathematical analysis of African, Brazilian, and Cuban clave rhythms. Proceedings of BRIDGES: Mathematical Connections in Art, Music and Science, Townson University, Towson, MD, July 27-29, 2002, pp. 157-168.

Whosampled.com. “The 10 Most Sampled Breakbeats of All Time.”

Wiggins, J. (2001). Teaching for musical understanding. Rochester, Michigan: Center for Applied Research in Musical Understanding, Oakland University.

Wilkie, K.; Holland, S.; & Mulholland, P. (2010). What Can the Language of Musicians Tell Us about Music Interaction Design?” Computer Music Journal, Vol. 34, No. 4, 34-48.

Musical simples – Teenage Dream

I’m working with Soundfly on the next installment of Theory For Producers, our ultra-futuristic online music theory course. The first unit covered the black keys of the piano and the pentatonic scales. The next one will talk about the white keys  and the diatonic modes. We were gathering examples, and we needed to find a well-known pop song that uses Lydian mode. My usual go-to example for Lydian is “Possibly Maybe” by Björk. But the course already uses a Björk tune for different example, and the Soundfly guys quite reasonably wanted something a little more millennial-friendly anyway. We decided to use Katy Perry’s “Teenage Dream” instead.

A couple of years ago, Slate ran an analysis of this tune by Owen Pallett. It’s an okay explanation, but it doesn’t delve too deep. We thought we could do better.

Here’s my transcription of the chorus:

When you look at the melody, this would seem to be a straightforward use of the B-flat major scale. However, the chord changes tell a different story. The tune doesn’t ever use a B-flat major chord. Instead, it oscillates back and forth between E-flat and F. In this harmonic context, the melody doesn’t belong to the plain vanilla B-flat major scale at all, but rather the dreamy and modernist E-flat Lydian mode. The graphic below shows the difference.

Teenage Dream Eb Lydian circles

Both scales use the same seven pitches: B-flat, C, D, E-flat, F, G, and A. The only difference between the two is which note you consider to be “home base.” Let’s consider B-flat major first.

To make chords from a scale, you pick any note, and then go clockwise around the scale, skipping every other degree. The chords are named for the note you start on. If you start on the fourth note, E-flat, you get the IV chord (the other two notes are G and B-flat.) If you start on the fifth note, F, you get the V chord (the other two notes are A and C.) In a major key, IV and V are very important chords. They’re called the subdominant and dominant chords, respectively, and they both create a feeling of suspense. You can resolve the suspense by following either one with the I chord. The weird thing about “Teenage Dream” is that if you think about it as being in B-flat, then it never lands on the I chord at all. It just oscillates back and fourth between IV and V. The suspense never gets resolved.

If we think of “Teenage Dream” as being in E-flat Lydian, then the E-flat chord is I, which makes more sense. The function of the F chord in this context isn’t clearly defined by music theory, but it does sound good. Lydian is very similar to the major scale, with only one difference: while the fourth note of E-flat major is A-flat, the fourth note of E-flat Lydian is A natural. That raised fourth gives Lydian mode its otherworldly sound. The F chord gets its airborne quality from that raised fourth.

Click here to play over “Teenage Dream” using the aQWERTYon. The two chords can be played on the letters Z-A-Q and X-S-W. For comparison, try playing it with B-flat major. Read more about the aQWERTYon here.

“Teenage Dream” is not the only well-known song to use the Lydian I-II progression. Other high profile examples include “Dreams” by Fleetwood Mac and “Jane Says” by Jane’s Addiction. over the same chords. Try singing any of these songs over any of the others; they all fit seamlessly.

The chorus of “Teenage Dream” uses a striking rhythm on the phrases “you make me”, “teenage dream”, and “I can’t sleep”. The song is in 4/4 time, like nearly all contemporary pop tracks, but that chorus rhythm has a feeling of three about it. It’s no illusion. The words “you” and “make” in the first line are each three eighth notes long. It sounds like an attempt to divide the eight eighth notes into groups of three. This rhythm is called Tresillo, and it’s one of the building blocks of Afro-Cuban drumming.

tresillo

Tresillo is the front half of son clave. It’s extraordinarily common in American vernacular music, especially in accompaniment patterns. You hear Tresillo in the bassline to “Hound Dog” and countless other fifties rock songs; in the generic acoustic guitar strumming pattern used by singer-songwriters everywhere; and in the kick and snare pattern characteristic of reggaetón. Tresillo is ubiquitous in jazz, and in the dance music of India and the Middle East.

“Teenage Dream” alternates the Tresillo with a funky syncopated rhythm pattern that skips the first beat of the measure. When you listen to the line “feel like I’m livin’ a”, there’s a hole right before the word “feel”. That hole is the downbeat, which is the usual place to start a phrase. When you avoid the obvious beat, you surprise the listener, which grabs their attention. The drums underneath this melody hammer relentlessly away on the strong beats, so it’s easy to parse out the rhythmic sophistication. Katy Perry songs have a lot of empty calories, but they taste as good as they do for a reason.

Ultralight Beam

The first song on Kanye West’s Life Of Pablo album, and my favorite so far, is the beautiful, gospel-saturated “Ultralight Beam.” See Kanye and company perform it live on SNL.

Ultralight Beam

The song uses only four chords, but they’re an interesting four: C minor, E-flat major, A-flat major, and G7. To find out why they sound so good together, let’s do a little music theory.

“Ultralight Beam” is in the key of C minor, and three of the four chords come from the C natural minor scale, shown below. Click the image to play the scale in the aQWERTYon (requires Chrome).

Ultralight Beam C natural minor

To make a chord, start on any scale degree, then skip two degrees clockwise, and then skip another two, and so on. To make C minor, you start on C, then jump to E-flat, and then to G. To make E-flat major, you start on E-flat, then jump to G, and then to B-flat. And to make A-flat major, you start on A-flat, then jump to C, and then to E-flat. Simple enough so far.

The C natural minor scale shares its seven notes with the E-flat major scale:

Ultralight Beam Eb major circles

All we’ve really done here is rotate the circle three slots counterclockwise. All the relationships stay the same, and you can form the same chords in the same way. The two scales are so closely related that if noodle around on C natural minor long enough, it starts just sounding like E-flat major. Try it!

The last of the four chords in “Ultralight Beam” is G7, and to make it, we need a note that isn’t in C natural minor (or E-flat major): the leading tone, B natural. If you take C natural minor and replace B-flat with B natural, you get a new scale: C harmonic minor.

Ultralight Beam C harmonic minor

If you make a chord starting on G from C natural minor, you get G minor (G, B-flat, D). The chord sounds fine, and you could use it with the other three above without offending anyone. But if you make the same chord using C harmonic minor, you get G major (G, B, D). This is a much more dramatic and exciting sound. If you add one more chord degree, you get G7 (G, B, D, F), known as the dominant chord in C minor. In the diagram below, the G7 chord is in blue, and C minor is in green.

Ultralight Beam C harmonic minor with V7 chord

Feel how much more intensely that B natural pulls to C than B-flat did? That’s what gives the song its drama, and what puts it unambivalently in C minor rather than E-flat major.

“Ultralight Beam” has a nice chord progression, but that isn’t its most distinctive feature. The thing that jumps out most immediately is the unusual beat. Nearly all hip-hop is in 4/4 time, where each measure is subdivided into four beats, and each of those four beats is subdivided into four sixteenth notes. “Ultralight Beam” uses 12/8 time, which was prevalent in the first half of the twentieth century, but is a rarity now. Each measure still has four beats in it, but these beats are subdivided into three beats rather than four.

four-four vs twelve-eight

The track states this rhythm very obliquely. The drum track is comprised almost entirely of silence. The vocals and other instruments skip lightly around the beat. Chance The Rapper’s verse in particular pulls against the meter in all kinds of complex ways.

The song’s structure is unusual too, a wide departure from the standard “verse-hook-verse-hook”.

Ultralight Beam song structure

The intro is six bars long, two bars of ambient voices, four bars over the chord progression. The song proper begins with just the first half of the chorus (known in hip-hop circles as the hook.) Kanye has an eight bar verse, followed by the first full chorus. Kelly Price gets the next eight bar verse. So far, so typical. But then, where you expect the next chorus, The-Dream gets his four-bar verse, followed by Chance The Rapper’s ecstatic sixteen-bar verse. Next is what feels like the last chorus, but that’s followed by Kirk Franklin’s four bar verse, and then a four-bar outtro with just the choir singing haunting single words. It’s strange, but it works. Say what you want about Kanye as a public figure, but as a musician, he is in complete control of his craft.