The aQWERTYon pitch wheels and the future of music theory visualization

The MusEDLab will soon be launching a revamped version of the aQWERTYon with some enhancements to its visual design, including a new scale picker. Beyond our desire to make our stuff look cooler, the scale picker represents a challenge that we’ve struggled with since the earliest days of aQW development. On the one hand, we want to offer users a wide variety of intriguing and exotic scales to play with. On the other hand, our audience of beginner and intermediate musicians is likely to be horrified by a list of terms like “Lydian dominant mode.” I recently had the idea to represent all the scales as colorful icons, like so:

Read more about the rationale and process behind this change here. In this post, I’ll explain what the icons mean, and how they can someday become the basis for a set of new interactive music theory visualizations.

Musical pitches rise and fall linearly, but pitch class is circular. When you go up or down the chromatic scale, the note names “wrap around” every twelve notes. This naming convention reflects the fact that we hear notes an octave apart as being “the same”, probably because they share so many overtones. (Non-human primates hear octaves as being equivalent too.)

chromatic circle

The note names and numbers are all based on the C major scale, which is Western music’s “default setting.” The scale notes C, D, E, F, G, A and B (the white keys on the piano) are the “normal” notes. (Why do they start on C and not A? I have no idea.) You get D-flat, E-flat, G-flat, A-flat and B-flat (the black keys on the piano) by lowering (flatting) their corresponding white key notes. Alternately, you can get the black key notes by raising or sharping the white key notes, in which case they’ll be called C-sharp, D-sharp, F-sharp, G-sharp, and A-sharp. (Let’s just briefly acknowledge that the imagery of the “normal” white and “deviant” black keys is just one of many ways that Western musical culture is super racist, and move on.)

You can represent any scale on the chromatic circle just by “switching” notes on and off. For example, if you activate the notes C, D, E-flat, F, G, A-flat and B, you get C harmonic minor. (Alternatively, you could just deactivate D-flat, E, G-flat, A, and B-flat.) Here’s how the scale looks when you write it this way:

C harmonic minor - monochrome

This is how I conceive scales in my head, as a pattern of activated and deactivated chromatic scale notes. As a guitarist, it’s the most intuitive way to think about them, because each box on the circular grid corresponds to a fret, so you can read the fingering pattern right off the circle. When I think “harmonic minor,” I don’t think of note names, I think “pattern of notes and gaps with one unusually wide gap.”

Another beauty of the circle view is that you can get the other eleven harmonic minor scales just by rotating the note names while keeping the pattern of activated/deactivated notes the same. If I want E-flat harmonic minor, I just have to grab the outer ring and rotate it counterclockwise a few notches:

E-flat harmonic minor

My next thought was to color-code the scale tones to give an indication of their sound and function:

C harmonic minor scale necklace

Here’s how the color scheme works:

  • Green – major, natural, sharp, augmented
  • Blue – minor, flat, diminished
  • Purple – perfect (neither major nor minor)
  • Grey – not in the scale

Scales with more green in them sound “happier” or brighter. Scales with more blue sound “sadder” or darker. Scales with a mixture of blue and green (like harmonic minor) will have a more complex and ambiguous feeling.

My ambition with the pitch wheels is not just to make the aQWERTYon’s scale menu more visually appealing. I’d eventually like to have it be an interactive way to visualize chords too. Followers of this blog will notice a strong similarity between the circular scale and the rhythm necklaces that inspired the Groove Pizza. Just like symmetries and patterns on the rhythm necklace can tell you a lot about how beats work, so too can symmetries and patterns on the scale necklace can tell you how harmony works. So here’s my dream for the aQWERTYon’s future theory visualization interface. If you load the app and set it to C harmonic minor, here’s how it would look. To the right is a staff notation view with the appropriate key signature.

When you play a note, it would change color on the keyboard and the wheel, and appear on the staff. The app would also tell you which scale degree it is (in this case, seven.)

If you play two notes simultaneously, in this case the third and seventh notes in C Mixolydian mode, the app would draw a line between the two notes on the circle:

If you play three notes at a time, like the first, fourth and fifth notes in C Lydian, you’d get a triangle.

If your three notes spell out a chord, like the second, fourth and sixth notes in C Phrygian mode, the app would recognize it and shows the chord symbol on the staff.

The pattern continues if you play four notes at a time:

Or five notes at a time:

By rotating the outer ring of the pitch wheel, you could change the root of the scale, like I showed above with C harmonic minor. And if you rotated the inner ring, showing the scale degrees, you could get different modes of the scale. Modes are one of the most difficult concepts in music theory. That is, they’re difficult until you learn to imagine them as rotations of the scale necklace, at which point they become nothing harder than a memorization exercise.

I’m designing this system to be used with the aQWERTYon, but there’s no reason it couldn’t take ordinary MIDI input as well. Wouldn’t it be nice to have this in a window in your DAW or notation program?

Music theory is hard. There’s a whole Twitter account devoted to retweeting students’ complaints about it. Some of this difficulty is due to the intrinsic complexity of modern harmony. But a lot of it is due to terminology and notation. Our naming system for notes and chords is a set of historically contingent kludges. No rational person would design it this way from the ground up. Thanks to path dependency, we’re stuck with it, much like we’re stuck with English grammar and the QWERTY keyboard layout. Fortunately, technology gives us a lot of new ways to make all the arcana more accessible, by showing multiple representations simultaneously and by making those representations discoverable through playful tinkering.

Do you find this idea exciting? Would you like it to be functioning software, and not just a bunch of flat images I laboriously made by hand? Help the MusEDLab find a partner to fund the developer and designer time. A grant or gift would work, and we’d also be open to exploring a commercial partnership. The aQW has been a labor of volunteer love for the lab so far, and it’s already one of the best music theory pedagogy tools on the internet. But development would go a lot faster if we could fund it properly. If you have ideas, please be in touch!

Update: Will Kuhn’s response to this post.

Deconstructing the bassline in Herbie Hancock’s “Chameleon”

If you have even a passing interest in funk, you will want to familiarize yourself with Herbie Hancock’s “Chameleon.” And if you are preoccupied and dedicated to the preservation of the movement of the hips, then the bassline needs to be a cornerstone of your practice.

Chameleon - circular bass

Here’s a transcription I did in Noteflight – huge props to them for recently introducing sixteenth note swing.

And here’s how it looks in the MIDI piano roll:

The “Chameleon” bassline packs an incredible amount of music into just two bars. To understand how it’s put together, it’s helpful to take a look at the scale that Herbie built the tune around, the B-flat Dorian mode. Click the image below to play it on the aQWERTYon. I recommend doing some jamming with it over the song before you move on.

B-flat Dorian

Fun fact: this scale contains the same pitches as A-flat major. If you find that fact confusing, then feel free to ignore it. You can learn more about scales and modes in my Soundfly course.

The chord progression

The opening section of “Chameleon” is an endless loop of two chords, B♭-7 and E♭7. You build both of them using the notes in B-flat Dorian. To make B♭-7, start on the root of the scale, B-flat. Skip over the second scale degree to land on the third, D-flat. Skip over the fourth scale degree to land on the fifth, F. Then skip over the sixth to land on the seventh, A-flat. If you want to add extensions to the chord, just keep skipping scale degrees, like so:

B-flat Dorian mode chords

To make E♭7, you’re going to use the same seven pitches in the same order, but you’re going to treat E-flat as home base rather than B-flat. You could think of this new scale as being E-flat Mixolydian, or B-flat Dorian starting on E-flat; they’re perfectly interchangeable. Click to play E-flat Mixolydian on the aQWERTYon. You build your E♭7 chord like so:

B-flat Dorian mode chords on E-flat

Once you’ve got the sound of B♭-7 and E♭7 in your head, let’s try an extremely simplified version of the bassline.

Chord roots only

At the most basic level, the “Chameleon” bassline exists to spell out the chord progression in a rhythmically interesting way. (This is what all basslines do.) Here’s a version of the bassline that removes all of the notes except the ones on the first beat of each bar. They play the roots of the chords, B-flat and E-flat.

That’s boring, but effective. You can never go wrong playing chord roots on the downbeat.

Simple arpeggios

Next, we’ll hear a bassline that plays all of the notes in B♭-7 and E♭7 one at a time. When you play chords in this way, they’re called arpeggios.

The actual arpeggios

The real “Chameleon” bassline plays partial arpeggios–they don’t have all of the notes from each chord. Also, the rhythm is a complicated and interesting one.

Below, you can explore the rhythm in the Groove Pizza. The orange triangle shows the rhythm of the arpeggio notes, played on the snare. The yellow quadrilateral shows the rhythm of the walkups, played on the kick–we’ll get to those below.

The snare rhythm has a hit every three sixteenth notes. It’s a figure known in Afro-Latin music as tresillo, which you hear absolutely everywhere in all styles of American popular and vernacular music. Tresillo also forms the front half of the equally ubiquitous son clave. (By the way, you can also use the Groove Pizza to experiment with the “Chameleon” drum pattern.)

As for the pitches: Instead of going root-third-fifth-seventh, the bassline plays partial arpeggios. The figure over B♭-7 is just the root, seventh and root again, while the one over E♭7 is the root, fifth and seventh.

Adding the walkups

Now let’s forget about the arpeggios for a minute and go back to just playing the chord roots on the downbeats. The bassline walks up to each of these notes via the chromatic scale, that is, every pitch on the piano keyboard.

Chromatic walkups are a great way to introduce some hip dissonance into your basslines, because they can include notes that aren’t in the underlying scale. In “Chameleon” the walkups include A natural and D natural. Both of these notes sound really weird if you sustain them over B-flat Dorian, but in the context of the walkup they sound perfectly fine.

Putting it all together

The full bassline consists of the broken arpeggios anticipated by the walkups.

If you’re a guitarist or bassist, you can play this without even shifting position. Use your index on the third fret, your middle on the fourth fret, your ring on the fifth fret, and your pinkie on the sixth fret.

              .          . .

If you’ve got this under your fingers, maybe you’d like to figure out the various keyboard and horn parts. They aren’t difficult, but you’ll need one more scale, the B-flat blues scale. Click the image to jam with it over the song and experience how great it sounds.

B-flat blues

There you have it, one of the cornerstones of funk. Good luck getting it out of your head!

Affordances and Constraints

Note-taking for User Experience Design with June Ahn

Don Norman discusses affordances and constraints in The Design of Everyday Things, Chapter Four: Knowing What To Do.

Don Norman - The Design of Everyday Things

User experience design is easy in situations where there’s only one thing that the user can possibly do. But as the possibilities multiply, so do the challenges. We can deal with new things using information from our prior experiences, or by being instructed. The best-designed things include the instructions for their own use, like video games whose first level act as tutorials, or doors with handles that communicate how you should operate them by their shape and placement.

We use affordances and constraints to learn how things work. Affordances suggest the range of possibilities, and constraints limit the alternatives. Constraints include:

  • Physical limitations. Door keys can only be inserted into keyholes vertically, but you can still insert the key upside down. Car keys work in both orientations.
  • Semantic constraints. We know that red lights mean stop and green lights mean go, so we infer that a red light means a device is off or inoperative, and a green light means it’s on or ready to function. We have a slow cooker that uses lights in the opposite way and it screws me up every time.
  • Cultural constraints. Otherwise known as conventions. (Not sure how these are different from semantic constraints.) Somehow we all know without being told that we’re supposed to face forward in the elevator. Google Glass was an epic failure because its early adopters ran into the cultural constraint of people not liking to be photographed without consent.
  • Logical constraints. The arrangement of knobs controlling your stove burners should match the arrangement of the burners themselves.

The absence of constraints makes things confusing. Norman gives examples of how much designers love rows of identical switches which give no clues as to their function. Distinguishing the switches by shape, size, or grouping might not look as elegant, but would make it easier to remember which one does what thing.

Helpful designs use visibility (making the relevant parts visible) and feedback (giving actions an immediate and obvious effect.) Everyone hates the power buttons on iMacs because they’re hidden on the back, flush with the case. Feedback is an important way to help us distinguish the functional parts from the decorative ones. Propellerheads Reason is an annoying program because its skeuomorphic design puts as many decorative elements on the screen as functional ones. Ableton Live is easier to use because everything on the screen is functional.

When you can’t make things visible, you can give feedback via sound. Pressing a Mac’s power button doesn’t immediately cause the screen to light up, but that’s okay, because it plays the famous startup sound. Norman’s examples of low-tech sound feedback include the “zzz” sound of a functioning zipper, a tea kettle’s whistle, and the various sounds that machines make when they have mechanical problems. The problem with sound as feedback is that it can be intrusive and annoying.

The term “affordance” is the source for a lot of confusion. Norman tries to clarify it in his article “Affordance, Conventions and Design.” He makes a distinction between real and perceived affordances. Anything that appears on a computer screen is a perceived affordance. The real affordances of a computer are its physical components: the screen itself, the keyboard, the trackpad. The MusEDLab was motivated to create the aQWERTYon by considering the computer’s real affordances for music making. Most software design ignores the real affordances and only considers the perceived ones.

Designers of graphical user interfaces rely entirely on conceptual models and cultural conventions. (Consider how many programs use a graphic of a floppy disk as a Save icon, and now compare to the last time you saw an actual floppy disk.) For Norman, graphics are perceived affordances by definition.

Joanna McGrenere and Wayne Ho try to nail the concept down harder in “Affordances: Clarifying and Evolving a Concept.” The term was coined by the perceptual psychologist James J. Gibson in his book The Ecological Approach to Visual Perception. For Gibson, affordances exist independent of the actor’s ability to perceive them, and don’t depend on the actor’s experiences and culture. For Norman, affordances can include both perceived and actual properties, which to me makes more sense. If you can’t figure out that an affordance exists, then what does it matter if it exists or not?

Norman collapses two distinct aspects of design: an object’s utility of an object and the way that users learn or discover that utility. But are designing affordances and designing the information about the affordances the same thing? McGrenere and Ho say no, that it’s the difference between usefulness versus usability. They complain that the HCI community has focused on usability at the expense of usefulness. Norman says that a scrollbar is a learned convention, not a real affordance. McGrenere and Ho disagree, because the scrollbar affords scrolling in a way that’s built into the software, making it every bit as much a real affordance as if it were a physical thing. The learned convention is the visual representation of the scrollbar, not the basic fact of it.

The best reason to distinguish affordances from their communication or representation is that sometimes the communication gets in the way of the affordance itself. For example, novice software users need graphical user interfaces, while advanced users prefer text commands and keyboard shortcuts. A beginner needs to see all the available commands, while a pro prefers to keep the screen free of unnecessary clutter. Ableton Live is a notoriously beginner-unfriendly program because it prioritizes visual economy and minimalism over user handholding. A number of basic functions are either invisible or so tiny as to be effectively invisible. Apple’s GarageBand welcomes beginners with photorealistic depictions of everything, but its lack of keyboard shortcuts makes it feel like wearing oven mitts for expert users. For McGrenere and Ho, the same feature of one of these programs can be an affordance or anti-affordance depending on the user.

Careless Whisper

 The infamous saxophone riff in “Careless Whisper” is one of the most infectious earworms in musical history. Love it or hate it, there is no getting it out of your head. In honor of the late George Michael, let’s take a look at what makes it work.

Play the riff yourself using your computer keyboard!

Press these keys to get the riff:Careless Whisper aQW score
So why is the riff so impossible to forget? Its melodic structure certainly jumps right out at you. The first three phrases are descending lines spelling out chords using similar rhythms. The fourth phrase is an ascending line running up a scale, using a very different rhythm.

First let’s take a closer look at those rhythms. The first three phrases are heavily syncopated. After the downbeats, every single note in each pattern falls on a weak beat. The fourth phrase is less syncopated; it’s a predictable pattern of eighth notes. But because your ear has become used to the pattern of the first three phrases, the straighter rhythm in the fourth one feels more “syncopated” because it defies your expectation.

Now let’s consider the harmonic content. The left diagram below shows the D natural minor scale on the chromatic circle. The right diagram shows it on the circle of fifths. Scale tones have a white background, while non-scale tones are greyed out.

Three of the four phrases in the “Careless Whisper” riff are arpeggios, the notes from a chord played one at a time. Here’s how you make the chords.

  • Take the D natural minor scale. Start on the root (D). Skip the second (E) and land on the third (F). Skip the fourth (G) and land on the fifth (A). Skip the sixth (B-flat) and land on the seventh (C). Finally, skip the root (D) and land on the ninth (E). These pitches – D, F, A, C, and E – make a D minor 9 (Dm9) chord. Now look at the first bar of the sax riff. All the pitches in D minor 9 are there except for C.
  • If you do the same process, but starting on G, you get the pitches G, B-flat, D, F, A, C, which make up a G minor 11 chord. The second phrase has most of those pitches.
  • Do the same process starting on B-flat, and you get B-flat, D, F, and A, making a B-flat major 7 (B♭maj7) chord. The third phrase has all of these pitches.

Careless Whisper D natural minor scale chords

The fourth phrase is different from the others. Rather than outlining an arpeggio, it runs up the D natural minor scale from A to A. This sequence of pitches (A, B-flat, C, D, E, F, G, A) is also known as the A Phrygian mode. The half-step interval between A and B-flat gives Phrygian its exotic quality.

This riff certainly is catchy. It’s also notoriously corny, and to many people’s ears, quite annoying. Why? Some of it is the timbre. The use of unrestrainedly passionate alto sax through heavy reverb was briefly in vogue in the 1980s, and then fell permanently out of style. To my ears, though, the real problem is the chord progression. In D minor, both Gm11 and B♭maj7 are subdominants, and functionally they’re interchangeable. Jazz musicians like me hear them as being essentially the same chord. It would be hipper to replace the Gm with G7, or the B♭maj7 with B♭7. The A minor in the last bar is weak too; it would be more satisfying to replace the C with C-sharp, to make D harmonic minor. But your mileage may vary.

Enjoy my mashup of this track with “Calabria 2007” by Enur featuring Natasja.

Designing a more welcoming aQWERTYon experience

This post documents my final project for User Experience Design with June Ahn

The best aQWERTYon screencap

Overview of the problem

The aQWERTYon is a web-based music performance and theory learning interface designed by the NYU Music Experience Design Lab. The name is a play on “QWERTY accordion.” The aQWERTYon invites novices to improvise and compose using a variety of scales and chords normally available only to advanced musicians. Notes map onto the computer keyboard such that the rows play scales and the columns play chords. The user can not play any wrong notes, which encourages free and playful exploration. The aQWERTYon has a variety of instrument sounds to choose from, and it can also act as a standard MIDI controller for digital audio workstations (DAWs) like GarageBand, Logic, and Ableton Live. As of this writing, there have been aQWERTYon 32,000 sessions.

One of our core design principles is to work within our users’ real-world technological limitations. We build tools in the browser so they will be platform-independent and accessible anywhere where there is internet access. Our aim with the aQWERTYon was to find the musical possibilities in a typical computer with no additional software or hardware. That question led us to investigate ways of turning the standard QWERTY keyboard into a beginner-friendly instrument.

While the aQWERTYon has been an effective tool in classrooms and online, it has some design deficiencies as well. It is difficult for unassisted users to figure out what the app is for. While its functionality is easily discovered through trial and error, its musical applications are less self-explanatory. Some of this is due to the intrinsic complexity of music theory and all the daunting terminology that comes with it. But some of it is the lack of context and guidance we provide to new users.

The conjecture

This assignment coincided with discussions already taking place in the lab around redesigning the aQW. Many of those focused on a particular element of the user interface, the scale picker.

aQWERTYon scale picker

The user has a variety of scales to choose from, ranging from the familiar to the exotic. However, these scales all have impenetrable names. How are music theory novices supposed to make sense of names like harmonic minor or Lydian mode? How would they know to choose one scale or another? We debated the least off-putting way of presenting these choices: should we represent them graphically? Associate each one with a well-known piece of music? Or just list them alphabetically? I proposed a system of graphical icons showing the notes comprising each scale. While novices will find them no more intelligible than the names, the hope is that they would be sufficiently visually appealing to invite users to explore them by ear.

aQW scale picker interactive wireframe

Conversations with June helped me understand that there are some broader and more profound user experience problems to solve before users ever get to the scale picker. What is the experience of simply landing on the app for the first time? How do people know what to do? From this conversation came the germ of a new idea, a landing page offering a tutorial or introduction. We want users to have a feeling of discovery, a musical “aha moment”, the chance to be a musical insider. The best way to do that seemed to be to give users a playlist of preset songs to jam with.

User characteristics and personas

There are three major user groups for the aQWERTYon, who I will describe as students, teachers, and explorers.

Students and teachers

Students use the aQW in a guided and structured setting: a classroom, a private lesson, or an online tutorial. There are several distinct user personas: elementary, middle and high school students, both mainstream and with special needs; college students; and online learners, mostly adults. Each student persona has its corresponding teacher persona. For example, I use the aQW with my music technology students at Montclair State University and NYU, and with some private students.

The aQW’s biggest fan is MusEDLab partner Matt McLean, who teaches at the Little Red Schoolhouse and runs a nonprofit organization called the Young Composers and Improvisors Workshop. Matt uses the aQW to teach composition in both settings, in person and online. He has documented his students’ use of the aQW extensively. Some examples:


I use the term explorers to describe people who use the aQW without any outside guidance. Explorers do not fit into specific demographic groups, but they center around two broad, overlapping personas: bedroom producers and music theory autodidacts. Explorers may find the aQW via a link, a social media posting, or a Google search. We know little about these users beyond what is captured by Google Analytics. However, we can make some assumptions based on our known referral sources. For example, this blog is a significant driver of traffic to the aQW. I have numerous posts on music theory and composition that link to the aQW so that readers can explore the concepts for themselves. My blog readership includes other music educators and some professional musicians, but the majority are amateur musicians and very enthusiastic listeners. These are exactly the users we are trying to serve: people who want to learn about music independently, either for creative purposes or to simply satisfy curiosity.

While I am a music educator, I have spent most of my life as a self-taught bedroom producer, so I identify naturally with the explorers. I have created several original pieces of music with the aQW, both for user testing purposes and to show its creative potential. While I have an extensive music theory background, I am a rudimentary keyboard player at best. This has limited my electronic music creation to drawing in the MIDI piano roll with the mouse pointer, since I can not perform my ideas on a piano-style controller. The aQW suits my needs perfectly, since I can set it to any scale I want and shred fearlessly. Here is an unedited improvisation I performed using a synthesizer instrument I created in Ableton Live:

My hope is that more would-be explorers feel invited to use the aQW for similar creative purposes in their own performance and composition.

Tasks and Scenarios

It is possible to configure the aQWERTYon via URL parameters to set the key and scale, and to hide components of the user interface. When teachers create exercises or assignments, they can link or embed the aQW with its settings locked to keep students from getting lost or confused. However, this does not necessarily invite the user to explore or experiment. Here is an example of an aQW preset to accompany a Beyoncé song. This preset might be used for a variety of pedagogical tasks, including learning some or all of the melody, creating a new countermelody, or improvising a solo. The harmonic major scale is not one that is usually taught, but it a useful way to blend major and minor tonalities. Students might try using more standard scales like major or harmonic minor, and listen for ways that they clash with Beyoncé’s song.

Tasks and scenarios for explorers might include creating a melody, bassline or chords for an original piece of music. For example, a self-taught dance music producer might feel limited by the scales that are easiest to play on a piano-style keyboard (major, natural minor, pentatonics) and be in search of richer and more exotic sounds. This producer might play their track in progress and improvise on top using different scale settings.

One of the users I tested with suggested an alternative explorer use case. He is an enthusiastic amateur composer and arranger, who is trying to arrange choral versions of pop and rock songs. He is a guitarist who has little formal music theory knowledge. He might use the aQW to try out harmonic ideas by ear, write down note names that form pleasing combinations, and then transfer them to the guitar or piano-based MIDI controller.

Understanding the problem

In the age of the computer and the internet, many aspects of music performance, composition and production are easy to self-teach. However, music theory remains an obstacle for many bedroom producers and pop musicians (not to mention schooled musicians!) There are so many chords and scales and rules and technical vocabulary, all of which have to be applied in all twelve keys. To make matters worse, terminology hangs around long after its historical context has disappeared. We no longer know what the Greek modes sound like, but we use their names to describe modern scales. C-sharp and D-flat were different pitches in historical tuning systems, but now both names describe the same pitch. The harmonic and melodic minor scales are named after a stylistic rule for writing melodies that was abandoned hundreds of years ago. And so on.

Most existing theory resources draw on the Western classical tradition, using examples and conventions from a repertoire most contemporary musicians and listeners find unfamiliar. Furthermore, these resources presume the ability to read standard music notation. Web resources that do address popular music are usually confusing and riddled with errors. I have worked with Soundfly to fill this vacuum by creating high-quality online courses aimed at popular musicians. Even with the best teaching resources, though, theory remains daunting. Exploring different chords and scales on an instrument requires significant technical mastery, and many musicians give up before ever reaching that point.

The aQW is intended to ease music theory learning by making scales and chords easy to discover even by complete novices. Our expectation is that after explorers are able to try theory ideas out in a low-pressure and creative setting, they will be motivated to put them to work playing instruments, composing or producing. Alternatively, users can simply perform and compose directly with the aQW itself.

Social and technical context

Most computer-based melody input systems are modeled on the piano. This is most obvious for hardware, since nearly all MIDI controllers take the form of literal piano keyboards. It is also true for software, which takes the piano keyboard as the primary visualization scheme for pitch. For example, the MIDI editor in every DAW displays pitches on a “piano roll”.

Some DAWs include a “musical typing” feature that maps the piano layout to the QWERTY keyboard, as an expediency for users who either lack MIDI hardware controllers, or who do not have them on hand. Apple’s GarageBand uses the ASDFG row of the keyboard for the white keys and the QWERTY row for the black keys. They use the other rows for such useful controls as pitch bend, modulation, sustain, octave shifting and simple velocity control.

GarageBand musical typing

Useful and expedient though it is, Musical Typing has some grave shortcomings as a user interface. It presumes familiarity with the piano keyboard, but is not very playable for users do who possess that familiarity. The piano layout makes a poor fit for the grid of computer keys. For example, there is no black key on the piano between the notes E and F, but the QWERTY keyboard gives no visual reminder of that fact, so it is necessary to just remember it. Unfortunately, the “missing” black key between E and F happens to be the letter R, which is GarageBand’s keyboard shortcut for recording. While hunting for E-flat or F-sharp, users are prone to accidentally start recording over their work. I have been using GarageBand for seven years and still do this routinely.

Ableton’s Push controller represents an interesting break with MIDI controller orthodoxy. It is a grid of 64 touch pads surrounded by various buttons, knobs and sliders.

Ableton Push

The pads were designed to trigger samples and loops like a typical drum machine, but Ableton also includes a melody mode for the Push. By default, it maps notes to the grid in rows staggered by fourths, which makes the layout identical to the bottom four strings of the guitar. This is quite a gift for guitarists like me, since I can use my familiar chord and scale fingerings, rather than hunting and pecking for them on the piano. Furthermore, the Push can be set so that the pads play only the notes within a particular scale, giving a “no wrong notes” experience similar to the aQWERTYon. Delightful though this mode is, however, it is imperfect. Root notes of the scale are colored blue, and other notes are colored white. While this makes the roots easy to distinguish, it is not so easy to visually differentiate the other pitches.

Touchscreen devices like the iPhone and iPad open up additional new possibilities for melodic interfaces. Many mobile apps continue to use the piano keyboard for note input, but some take advantage of the touchscreen’s unique affordances. One such is Thumbjam, which enables the user to divide the screen into slices of arbitrary thickness that can map to any arbitrary combination of notes.


The app offers hundreds of preset scales to choose from. The user may have a small range of notes, each of which is large and easy to distinguish, or a huge range of notes, each of which occupies a narrow strip of screen area. Furthermore, the screen can be split to hold four different scales, played from four different instruments. While all of this configurability is liberating, it is also overwhelming. Also, the scales are one-dimensional lines; there is no easy way to play chords and arpeggios.

Evaluation criteria

Is the aQW’s potential obvious enough to draw in explorers and educators? Will it be adopted as a tool for self-teaching? Does it invite playful exploration and experimentation? Is it satisfying for real-world musical usage? Is the UI self-explanatory, or at least discoverable? Is the music theory content discoverable? Have we identified the right user persona(s)? Is the aQW really a tool for beginners? Or is it an intermediate music theory learning tool? Or an advanced composition tool? Is the approach of a “playlist” of example songs the right one? Which songs, artists and genres should we include on the landing page? How many presets should we include? Should we limit it to a few, or should we offer a large, searchable database? And how do we deal with the fact that many songs require multiple scales to play?

Proposed solution

I tested several interactive wireframes of this landing page concept. Click the image to try it yourself:

aQWERTYon landing page interactive wireframe

The first wireframe had nine preset songs. I wanted to offer reasonable musical diversity without overwhelming the user. The tenth slot linked to the “classic” aQW, where users are free to select their own video, scale, root, and so on. I chose songs that appealed to me (and presumably other adult explorers), along with some current pop songs familiar to younger users. I wanted to balance the choices by race, gender, era, and genre. I was also bound by a musical constraint: all songs need to be playable using a single scale in a single key. The initial preset list was:

  • Adele – “Send My Love (To Your New Lover)”
  • Mary J Blige – “Family Affair”
  • Miles Davis – “Sssh/Peaceful”
  • Missy Elliott – “Get Ur Freak On”
  • Björk – “All Is Full Of Love”
  • Michael Jackson – “Don’t Stop ’Til You Get Enough”
  • Katy Perry – “Teenage Dream”
  • AC/DC – “Back In Black”
  • Daft Punk – “Get Lucky”

After a few test sessions, it became apparent that no one was clicking Mary J Blige. Also, the list did not include any current hip-hop. I therefore replaced her with Chance The Rapper. I initially offered a few sentences of instruction, but feedback from my MusEDLab colleagues encouraged me to reduce the prompt down to just a few words: “Pick a song, type, jam.”

Further testing showed that while adults are willing to try out any song, familiar or not, children and teens are much choosier. Therefore, I added two more presets, “Hotline Bling” by Drake and “Formation” by Beyoncé. The latter song proved problematic, however, because its instrumental backing is so sparse and minimal that it is difficult to hear how other notes might fit into it. I ultimately swapped it for “Single Ladies.” I had rejected this song initially, because it uses the idiosyncratic harmonic major scale. However, I came to see this quirk as a positive bonus–since one of our goals is to encourage users to explore new sounds and concepts, a well-known and well-loved song using an unusual scale is a rare gift.

User testing protocol

I used a think-aloud protocol, asking testers to narrate their thought processes as they explored the app. I recorded the one-on-one sessions using Screenflow. When testing with groups of kids, this was impractical, so instead I took notes during and after each session. For each user, I opened the interactive wireframe, and told them, “This is a web based application for playing music with your computer keyboard. I’m going to ask you to tell me what you see on the screen, what you think it does, and what you think will happen when you click things.” I did not offer any other explanation or context, because I wanted to see whether the landing page was self-explanatory and discoverable. I conducted informal interviews with users during and after the sessions as well.

User testing results

I tested with ten adults and around forty kids. The adults ranged in age from early twenties to fifties. All were musicians, at varying levels of ability and training, mostly enthusiastic amateurs. Sessions lasted for twenty or thirty minutes. There were two groups of kids: a small group of eighth graders at the Little Red Schoolhouse, and a large group of fourth graders from PS 3 who were visiting NYU. These testing sessions were shorter, ten to fifteen minutes each.

User testing the aQWERTYon with fourth graders

Discovering melodies

It is possible to play the aQW by clicking the notes onscreen using the mouse, though this method is slow and difficult. Nevertheless, a number of the younger testers did this, even after I suggested that it would be easier on the keyboard.

An adult tester with some keyboard and guitar experience told me, “This is great, it’s making me play patterns that I normally don’t play.” He was playing on top of the Miles Davis track, and he was quickly able to figure out a few riffs from Miles’ trumpet solo.

Discovering chords

Several testers systematically identified chords by playing alternating notes within a row, while others discovered them by holding down random groups of keys. None of the testers discovered that they could easily play chords using columns of keys until I prompted them to do so. One even asked, “Is there a relationship between keys if I play them vertically? I don’t know enough about music to know that.” After I suggested he try the columns, he said, “If I didn’t know [by ear] how chords worked, I’d miss the beauty of this.” He compared the aQW to GarageBand’s musical typing: “This is not that. This is a whole new thing. This is chord oriented. As a guitarist, I appreciate that.” The message is clear: we need to make the chords more obvious, or more actively assist users in finding them.

Other theory issues

For the most part, testers were content to play the scales they were given, though some of the more expert musicians changed the scales before even listening to the presets. However, not everyone realized that the presets were set to match the song. A few asked me: “How do I know what key this song is in?” We could probably state explicitly that the presets line up automatically.

In general, adult testers found the value of the aQW as a theory learning tool to be immediately apparent. One told me: “If I had this when I was a kid, I would have studied music a lot. I used to hate music theory. I learned a lot of stuff, but the learning process was awful… Your kids’ generation will learn music like this (snaps fingers).”


The aQW comes with a large collection of SoundFonts, and users of all ages enjoyed auditioning them, sometimes for long periods of time. Sometimes they apologized for how fascinating they found the sounds to be. But it is remarkable to have access to so many instrument timbres so effortlessly. Computers turn us all into potential orchestrators, arrangers, and sound designers.

Screen layout

The more design-oriented testers appreciated the sparseness and minimalism of the graphics, finding them calming and easy to understand.

Several testers complained that the video window takes up too much screen real estate, and is placed too prominently. Two commented that videos showing live performers, like “Back In Back,” were valuable because that helped with timekeeping and inspiration. Otherwise, however, testers found the videos to either be of little value or actively distracting. One suggested having the videos hidden or minimized by default, with the option to click to expand them. Others requested that the video be below the keyboard and other crucial controls. Also, the eighth graders reported that some of the video content was distracting because of its content, for example the partying shown in “Teenage Dream.” Unsuitable content will be an ongoing issue using many of the pop songs that kids like.

Technical browser issues

Having the aQWERTYon run in the browser has significant benefits, but a few limitations as well. Because the URL updates every time the parameters change, clicking the browser’s Back button does not produce the expected behavior–it might take ten or fifteen clicks to actually return to a previous page. I changed the links in later versions so each one opens the aQW in a new tab so the landing page would always be available. However, web audio is very memory-intensive, and the aQW will function slowly or not at all if it is open in more than one tab simultaneously.

Song choices

The best mix of presets is always going to depend on the specific demographics of any given group of users. However, the assortment I arrived at was satisfying enough for the groups I tested with. Miles Davis and Björk do not have the wide appeal of Daft Punk or Michael Jackson, but their presence was very gratifying for the more hipster-ish testers. I was extremely impressed that an eighth grader selected the Miles song, though this kid turns out to be the son of a Very Famous Musician and is not typical.

Recording functionality

Testers repeatedly requested the ability to record their playing. The aQW did start out with a very primitive recording feature, but it will require some development to make it usable. The question is always, how much functionality is enough? Should users be able to overdub? If so, how many tracks? Is simple recording enough, or would users need to able to mix, edit, and select takes?

One reason that recording has been a low development priority is that users can easily record their performances via MIDI into any DAW or notation program. The aQW behaves as if it were a standard MIDI controller plugged into the computer. With so many excellent DAWs in the world, it seems less urgent for us to replicate their functionality. However, there is one major limitation of recording this way: it captures the notes being played, but not the sounds. Instead, the DAW plays back the MIDI using whatever software instruments it has available. Users who are attached to a specific SoundFont cannot record them unless they use a workaround like Soundflower. This issue will require more discussion and design work.

New conjectures and future work

One of my most significant user testers for the landing page wireframe was Kevin Irlen, the MusEDLab’s chief software architect and main developer of the aQW itself. He found the landing page concept sufficiently inspiring that he created a more sophisticated version of it, the app sequencer:

aQWERTYon app sequencer v1

We can add presets to the app sequencer using a simple web form, which is a significant improvement over the tedious process of creating my wireframes by hand. The sequencer pulls images automatically from YouTube, another major labor-saver. Kevin also added a comment field, which gives additional opportunity to give prompts and instructions. Each sequencer preset generates a unique URL, making it possible to generate any number of different landing pages. We will be able to create custom landing pages focusing on different artists, genres or themes.

Songs beyond the presets

Testing with the fourth graders showed that we will need to design a better system for users who want to play over songs that we do not include among the presets. That tutorial needs to instruct users how to locate YouTube URLs, and more dauntingly, how to identify keys and scales. I propose an overlay or popup:


Testing with fourth graders also showed that helping novice users with keyfinding may not be as challenging as I had feared. The aQW defaults to the D minor pentatonic scale, and that scale turns out to fit fairly well over most current pop songs. If it doesn’t, some other minor pentatonic scale is very likely to work. This is due to a music-theoretical quirk of the pentatonic scale: it happens to share pitches with many other commonly-used scales and chords. As long as the root is somewhere within the key, the minor pentatonic will sound fine. For example, in C major:

  • C minor pentatonic sounds like C blues
  • D minor pentatonic sounds like Csus4
  • E minor pentatonic sounds like Cmaj7
  • F minor pentatonic sounds like C natural minor
  • G minor pentatonic sounds like C7sus4
  • A minor pentatonic is the same as C major pentatonic
  • B minor pentatonic sounds like C Lydian mode


We are planning to revamp the root picker to show both a larger piano keyboard and a pitch wheel. We also plan to add more dynamic visualization options for notes as they are played, including a staff notation view, the chromatic circle, and the circle of fifths. The aQW leaves several keys on the keyboard unused, and we could use them for additional controls. For example, we might use the Control key to make note velocities louder, and Option to make them quieter. The arrow keys might be used to cycle through the scale menu and to shift the root.

Built-in theory pedagogy

There is a great deal of opportunity to build more theory pedagogy on top of the aQW, and to include more of it within the app itself. We might encourage chord playing by automatically showing chord labels at the top of each column. We might include popups or links next to each scale giving some explanation of why they sound the way they do, and to give some suggested musical uses. One user proposes a game mode for more advanced users, where the scale is set to chromatic and players must identify the “wrong” or outside notes. Another proposes a mode similar to Hooktheory, where users could sequence chord progressions to play on top of.

Rhythmic assistance

A few testers requested some kind of help or guidance with timekeeping. One suggested a graphical score in the style of Guitar Hero, or a “follow the bouncing ball” rhythm visualization. Another pointed out that an obvious solution would be to incorporate the Groove Pizza, perhaps in miniature form in a corner of the screen. Synchronizing all of this to YouTube videos would need to be done by hand, so far as I know, but perhaps an automated solution exists. Beat detection is certainly an easier MIR challenge than key or chord detection. If we were able to automatically sync to the tempo of a song, we could add the DJ functionality requested by one tester, letting users add cue points, loop certain sections, and slow them down.

Odds and ends

One eighth grader suggested that we make aQW accounts with “musical passwords.”

An adult tester referred to the landing page as the “Choose Your Own Adventure screen.” The idea of musical adventure is exactly the feeling I was hoping for.

In addition to notes on the staff, one tester requested a spectrum visualizer. This is perhaps an esoteric request, but real-time spectrograms are quite intuitive and might be useful.

Finally, one tester made a comment that was striking in its broader implications for music education: “I’m not very musical, I don’t really play an instrument, so these kinds of tricks are helpful for me. It didn’t take me long to figure out how the notes are arranged.” This person is a highly expert producer, beatmaker and live performer using Ableton Live. I asked how he came to this expertise, and he said he felt compelled to learn it to compensate for his lack of “musicianship”. It makes me sad that such a sophisticated musician does not realize that his skills “count”. In empowering music learners with the aQW, I also hope we are able to help computer musicians value themselves.

Design for Real Life – QWERTYBeats research

Writing assignment for Design For The Real World with Claire Kearney-Volpe and Diana Castro – research about a new rhythm interface for blind and low-vision novice musicians


I propose a new web-based accessible rhythm instrument called QWERTYBeats.Traditional instruments are highly accessible to blind and low-vision musicians. Electronic music production tools are not. I look at the history of accessible instruments and software interfaces, give an overview of current electronic music hardware and software, and discuss the design considerations underlying my project.

QWERTYBeats logo

Historical overview

Acoustic instruments give rich auditory and haptic feedback, and pose little obstacle to blind musicians. We need look no further for proof than the long history of iconic blind musicians like Ray Charles and Stevie Wonder. Even sighted instrumentalists rarely look at their instruments once they have attained a sufficient level of proficiency. Music notation is not accessible, but Braille notation has existed since the language’s inception. Also, a great many musicians both blind and sighted play entirely by ear anyway.

Most of the academic literature around accessibility issues in music education focuses on wider adoption of and support for Braille notation. See, for example, Rush, T. W. (2015). Incorporating Assistive Technology for Students with Visual Impairments into the Music Classroom. Music Educators Journal, 102(2), 78–83. For electronic music, notation is rarely if ever a factor.

Electronic instruments pose some new accessibility challenges. They may use graphical interfaces with nested menus, complex banks of knobs and patch cables, and other visual control surfaces. Feedback may be given entirely with LED lights and small text labels. Nevertheless, blind users can master these devices with sufficient practice, memorization and assistance. For example, Stevie Wonder has incorporated synthesizers and drum machines in most of his best-known recordings.

Most electronic music creation is currently done not with instruments, but rather using specialized software applications called digital audio workstations (DAWs). Keyboards and other controllers are mostly used to access features of the software, rather than as standalone instruments. The most commonly-used DAWs include Avid Pro Tools, Apple Logic, Ableton Live, and Steinberg Cubase. Mobile DAWs are more limited than their desktop counterparts, but are nevertheless becoming robust music creation tools in their own right. Examples include Apple GarageBand and Steinberg Cubasis. Notated music is commonly composed using score editing software like Sibelius and Finale, whose functionality increasingly overlaps with DAWs, especially in regard to MIDI sequencing.

DAWs and notation editors pose steep accessibility challenges due to their graphical and spatial interfaces, not to mention their sheer complexity. In class, we were given a presentation by Leona Godin, a blind musician who records and edits audio using Pro Tools by means of VoiceOver. While it must have taken a heroic effort on her part to learn the program, Leona demonstrates that it is possible. However, some DAWs pose insurmountable problems even to very determined blind users because they do not use standard operating system elements, making them inaccessible via screen readers.

Technological interventions

There are no mass-market electronic interfaces specifically geared toward blind or low-vision users. In this section, I discuss one product frequently hailed for its “accessibility” in the colloquial rather than blindess-specific sense, along with some more experimental and academic designs.

Ableton Push

Push layout for IMPACT Faculty Showcase

Ableton Live has become the DAW of choice for electronic music producers. Low-vision users can zoom in to the interface and modify the color scheme. However, Live is inaccessible via screen readers.

In recent years, Ableton has introduced a hardware controller, the Push, which is designed to make the software experience more tactile and instrument-like. The Push combines an eight by eight grid of LED-lit touch pads with banks of knobs, buttons and touch strips. It makes it possible to create, perform and record a piece of music from scratch without looking at the computer screen. In addition to drum programming and sampler performance, the Push also has an innovative melodic mode which maps scales onto the grid in such a way that users can not play a wrong note. Other comparable products exist; see, for example, the Native Instruments Maschine.

There are many pad-based drum machines and samplers. Live’s main differentiator is its Session view, where the pads launch clips: segments of audio or MIDI that can vary in length from a single drum hit to the length of an entire song. Clip launching is tempo-synced, so when you trigger a clip, playback is delayed until the start of the next measure (or whatever the quantization interval is.) Clip launching is a forgiving and beginner-friendly performance method, because it removes the possibility of playing something out of rhythm. Like other DAWs, Live also gives rhythmic scaffolding in its software instruments by means of arpeggiators, delay and other tempo-synced features.

The Push is a remarkable interface, but it has some shortcomings for blind users. First of all, it is expensive, $800 for the entry-level version and $1400 for the full-featured software suite. Much of its feedback is visual, in the form of LED screens and color-coded lighting on the pads. It switches between multiple modes which can be challenging to distinguish even for sighted users. And, like the software it accompanies, the Push is highly complex, with a steep learning curve unsuited to novice users, blind or sighted.


Most DAWs enable users to perform MIDI instruments on the QWERTY keyboard. The most familiar example is the Musical Typing feature in Apple GarageBand.

GarageBand musical typing

Musical Typing makes it possible to play software instruments without an external MIDI controller, which is convenient and useful. However, its layout counterintuively follows the piano keyboard, which is an awkward fit for the computer keyboard. There is no easy way to distinguish the black and white keys, and even expert users find themselves inadvertantly hitting the keyboard shortcut for recording while hunting for F-sharp.

The aQWERTYon is a web interface developed by the NYU Music Experience Design Lab specifically intended to address the shortcomings of Musical Typing.

aQWERTYon screencap

Rather than emulating the piano keyboard, the aQWERTYon draws its inspiration from the chord buttons of an accordion. It fills the entire keyboard with harmonically related notes in a way that supports discovery by naive users. Specifically, it maps scales across the rows of keys, staggered by intervals such that each column forms a chord within the scale. Root notes and scales can be set from pulldown menus within the interface, or preset using URL parameters. It can be played as a standalone instrument, or as a MIDI controller in conjunction with a DAW. Here is a playlist of music I created using the aQWERTYon and GarageBand or Ableton Live:

The aQWERTYon is a completely tactile experience. Sighted users can carefully match keys to note names using the screen, but more typically approach the instrument by feel, seeking out patterns on the keyboard by ear. A blind user would need assistance loading the aQWERTYon initially and setting the scale and root note parameters, but otherwise, it is perfectly accessible. The present project was motivated in large part by a desire to make exploration of rhythm as playful and intuitive as the aQWERTYon makes exploring chords and scales.


The QWERTY keyboard can be turned into a simple drum machine quite easily using a free program called Soundplant. The user simply drags audio files onto a graphical key to have it triggered by that physical key. I was able to create a TR-808 kit in a matter of minutes:

Soundplant with 808 samples

After it is set up and configured, Soundplant can be as effortlessly accessible as the aQWERTYon. However, it does not give the user any rhythmic assistance. Drumming in perfect time is an advanced musical skill, and playing drum machine samples out of time is not much more satisfying than banging on a metal bowl with a spoon out of time. An ideal drum interface would offer beginners some of the rhythmic scaffolding and support that Ableton provides via Session view, arpeggiators, and the like.

The Groove Pizza

Drum machines and their software counterparts offer an alternative form of rhythmic scaffolding. The user sequences patterns in a time-unit box system or piano roll, and the computer performs those patterns flawlessly. The MusEDLab‘s Groove Pizza app is a web-based drum sequencer that wraps the time-unit box system into a circle.

Groove Pizza - Bembe

The Groove Pizza was designed to make drum programming more intuitive by visualizing the symmetries and patterns inherent in musical-sounding rhythms. However, it is totally unsuitable for blind or low-vision users. Interaction is only possible through the mouse pointer or touch, and there are no standard user interface elements that can be parsed by screen readers.

Before ever considering designing for the blind, the MusEDLab had already considered the Groove Pizza’s limitations for younger children and users with special needs: there is no “live performance” mode, and there is always some delay in feedback between making a change in the drum pattern and hearing the result. We have been considering ways to make a rhythm interface that is more immediate, performance-oriented and tactile. One possible direction would be to create a hardware version of the Groove Pizza; indeed, one of the earliest prototypes was a hardware version built by Adam November out of a pizza box. However, hardware design is vastly more complex and difficult than software, so for the time being, software promises more immediate results.

Haenselmann-Lemelson-Effelsberg MIDI sequencer

This experimental interface is described in Haenselmann, T., Lemelson, H., & Effelsberg, W. (2011). A zero-vision music recording paradigm for visually impaired people. Multimedia Tools and Applications, 5, 1–19.

Haenselmann-Lemelson-Effelsberg MIDI sequencer

The authors create a new mode for a standard MIDI keyboard that maps piano keys to DAW functions like playback, quantization, track selection, and so on. They also add “earcons” (auditory icons) to give sonic feedback when particular functions have been activated that normally only give graphical feedback. For example, one earcon sounds when recording is enabled; another sounds for regular playback. This interface sounds promising, but there are significant obstacles to its adoption. While the authors have released the source code as a free download, that requires a would-be user to be able to compile and run it. This is presuming that they could access the code in the first place; the download link given in the paper is inactive. It is an all-too-common fate of academic projects to never get widespread usage. By posting our projects on the web, the MusEDLab hopes to avoid this outcome.


Music education philosophy

My project is animated by a constructivist philosophy of music education, which operates by the following axiomatic assumptions:

  • Learning by doing is better than learning by being told.
  • Learning is not something done to you, but rather something done by you.
  • You do not get ideas; you make ideas. You are not a container that gets filled with knowledge and new ideas by the world around you; rather, you actively construct knowledge and ideas out of the materials at hand, building on top of your existing mental structures and models.
  • The most effective learning experiences grow out of the active construction of all types of things, particularly things that are personally or socially meaningful, that you develop through interactions with others, and that support thinking about your own thinking.

If an activity’s challenge level is beyond than your ability, you experience anxiety. If your ability at the activity far exceeds the challenge, the result is boredom. Flow happens when challenge and ability are well-balanced, as seen in this diagram adapted from Csikszentmihalyi.


Music students face significant obstacles to flow at the left side of the Ability axis. Most instruments require extensive practice before it is possible to make anything that resembles “real” music. Electronic music presents an opportunity here, because even a complete novice can produce music with a high degree of polish quickly. It is empowering to use technologies that make it impossible to do anything wrong; it frees you to begin exploring what you find to sound right. Beginners can be scaffolded in their pitch explorations with MIDI scale filters, Auto-Tune, and the configurable software keyboards in apps like Thumbjam and Animoog. Rhythmic scaffolding is more rare, but it can be had via Ableton’s quantized clip launcher, by MIDI arpeggiators, and using the Note Repeat feature on many drum machines.

QWERTYBeats proposal

My project takes drum machine Note Repeat as its jumping off point. When Note Repeat is activated, holding down a drum pad triggers the corresponding sound at a particular rhythmic interval: quarter notes, eighth notes, and so on. On the Ableton Push, Note Repeat automatically syncs to the global tempo, making it effortless to produce musically satisfying rhythms. However, this mode has a major shortcoming: it applies globally to all of the drum pads. To my knowledge, no drum machine makes it possible to simultaneously have, say, the snare drum playing every dotted eighth note while the hi-hat plays every sixteenth note.

I propose a web application called QWERTYBeats that maps drums to the computer keyboard as follows:

  • Each row of the keyboard triggers a different drum/beatbox sound (e.g. kick, snare, closed hi-hat, open hi-hat).
  • Each column retriggers the sample at a different rhythmic interval (e.g. quarter note, dotted eighth note).
  • Circles dynamically divide into “pie slices” to show rhythmic values.

The rhythm values are shown below by column, with descriptions followed by the time interval as shown as a fraction of the tempo in beats per minute.

  1. quarter note (1)
  2. dotted eighth note (3/4)
  3. quarter note triplet (2/3)
  4. eighth note (1/2)
  5. dotted sixteenth note (3/8)
  6. eighth note triplet (1/3)
  7. sixteenth note (1/4)
  8. dotted thirty-second note (3/16)
  9. sixteenth note triplet (1/6)
  10. thirty-second note (1/8)

By simply holding down different combinations of keys, users can attain complex syncopations and polyrhythms. If the app is synced to the tempo of a DAW or music playback, the user can perform good-sounding rhythms over any song that is personally meaningful to them.

The column layout leaves some unused keys in the upper right corner of the keyboard: “-“, “=”, “[“, “]”, “”, etc. These can be reserved for setting the tempo and other UI elements.

The app defaults to Perform Mode, but clicking Make New Kit opens Sampler mode, where users can import or record their own drum sounds:

  • Keyboard shortcuts enable the user to select a sound, audition it, record, set start and end point, and set its volume level.
  • A login/password system enables users to save kits to the cloud where they can be accessed from any computer. Kits get unique URL identifiers, so users can also share them via email or social media.

It is my goal to make the app accessible to users with the widest possible diversity of abilities.

  • The entire layout will use plain text, CSS and JavaScript to support screen readers.
  • All user interface elements can be accessed via the keyboard: tab to change the keyboard focus, menu selections and parameter changes via the up and down arrows, and so on.

Perform Mode:

QWERTYBeats concept images - Perform mode

Sampler Mode:


Mobile version

The present thought is to divide up the screen into a grid mirroring the layout of the QWERTY keyboard. User testing will determine whether this will produce a satisfying experience.

QWERTYDrum - mobile


I created a prototype of the app using Ableton Live’s Session View.

QWERTYBeats - Ableton prototype

Here is a sample performance:

There is not much literature examining the impact of drum programming and other electronic rhythm sequencing on students’ subsequent ability to play acoustic drums, or to keep time more accurately in general. I can report anecdotally that my own time spent sequencing and programming drums improved my drumming and timekeeping enormously (and mostly inadvertently.) I will continue to seek further support for the hypothesis that electronically assisted rhythm creation builds unassisted rhythmic ability. In the meantime, I am eager to prototype and test QWERTYBeats.

Composing in the classroom

The hippest music teachers help their students create original music. But what exactly does that mean? What even is composition? In this post, I take a look at two innovators in music education and try to arrive at an answer.

Matt McLean is the founder of the amazing Young Composers and Improvisers Workshop. He teaches his students composition using a combination of Noteflight, an online notation editor, and the MusEDLab‘s own aQWERTYon, a web app that turns your regular computer keyboard into an intuitive musical interface.

Matt explains:

Participating students in YCIW as well as my own students at LREI have been using Noteflight for over 6 years to compose music for chamber orchestras, symphony orchestras, jazz ensembles, movie soundtracks, video game music, school band and more – hundreds of compositions.

Before the advent of the aQWERTYon, students needed to enter music into Noteflight either by clicking with the mouse or by playing notes in with a MIDI keyboard. The former method is accessible but slow; the latter method is fast but requires some keyboard technique. The aQWERTYon combines the accessibility of the mouse with the immediacy of the piano keyboard.

For the first time there is a viable way for every student to generate and notate her ideas in a tactile manner with an instrument that can be played by all. We founded Young Composers & Improvisors Workshop so that every student can have the experience of composing original music. Much of my time has been spent exploring ways to emphasize the “experiencing” part of this endeavor. Students had previously learned parts of their composition on instruments after their piece was completed. Also, students with piano or guitar skills could work out their ideas prior to notating them. But efforts to incorporate MIDI keyboards or other interfaces with Noteflight in order to give students a way to perform their ideas into notation always fell short.

The aQWERTYon lets novices try out ideas the way that more experienced musicians do: by improvising with an instrument and reacting to the sounds intuitively. It’s possible to compose without using an instrument at all, using a kind of sudoku-solving method, but it’s not likely to yield good results. Your analytical consciousness, the part of your mind that can write notation, is also its slowest and dumbest part. You really need your emotions, your ear, and your motor cortex involved. Before computers, you needed considerable technical expertise to be able to improvise musical ideas, and remember them long enough to write them down. The advent of recording and MIDI removed a lot of the friction from the notation step, because you could preserve your ideas just by playing them. With the aQWERTYon and interfaces like it, you can do your improvisation before learning any instrumental technique at all.

Student feedback suggests that kids like being able to play along to previously notated parts as a way to find new parts to add to their composition. As a teacher I am curious to measure the effect of students being able to practice their ideas at home using aQWERTYon and then sharing their performances before using their idea in their composition. It is likely that this will create a stronger connection between the composer and her musical idea than if she had only notated it first.

Those of us who have been making original music in DAWs are familiar with the pleasures of creating ideas through playful jamming. It feels like a major advance to put that experience in the hands of elementary school students.

Matt uses progressive methods to teach a traditional kind of musical expression: writing notated scores that will then be performed live by instrumentalists. Matt’s kids are using futuristic tools, but the model for their compositional technique is the one established in the era of Beethoven.


(I just now noticed that the manuscript Beethoven is holding in this painting is in the key of D-sharp. That’s a tough key to read!)

Other models of composition exist. There’s the Lennon and McCartney method, which doesn’t involve any music notation. Like most untrained rock musicians, the Beatles worked from lyric sheets with chords written on them as a mnemonic. The “lyrics plus chords” method continues to be the standard for rock, folk and country musicians. It’s a notation system that’s only really useful if you already have a good idea of how the song is supposed to sound.

Lennon and McCartney writing

Lennon and McCartney originally wrote their songs to be performed live for an audience. They played in clubs for several years before ever entering a recording studio. As their career progressed, however, the Beatles stopped performing live, and began writing with the specific goal of creating studio recordings. Some of those later Beatles tunes would be difficult or impossible to perform live. Contemporary artists like Missy Elliott and Pharrell Williams have pushed the Beatles’ idea to its logical extreme: songs existing entirely within the computer as sequences of samples and software synths, with improvised vocals arranged into shape after being recorded. For Missy and Pharrell, creating the score and the finished recording are one and the same act.

Pharrell and Missy Elliott in the studio

Is it possible to teach the Missy and Pharrell method in the classroom? Alex Ruthmann, MusEDLab founder and my soon-to-be PhD advisor, documented his method for doing so in 2007.

As a middle school general music teacher, I’ve often wrestled with how to engage my students in meaningful composing experiences. Many of the approaches I’d read about seemed disconnected from the real-world musicality I saw daily in the music my students created at home and what they did in my classes. This disconnect prompted me to look for ways of bridging the gap’ between the students’ musical world outside music class and their in-class composing experiences.

It’s an axiom of constructivist music education that students will be most motivated to learn music that’s personally meaningful to them. There are kids out there for whom notated music performed on instruments is personally meaningful. But the musical world outside music class usually follows the Missy and Pharrell method.

[T]he majority of approaches to teaching music with technology center around notating musical ideas and are often rooted in European classical notions of composing (for example, creating ABA pieces, or restricting composing tasks to predetermined rhythmic values). These approaches require students to have a fairly sophisticated knowledge of standard music notation and a fluency working with rhythms and pitches before being able to explore and express their musical ideas through broader musical dimensions like form, texture, mood, and style.

Noteflight imposes some limitations on these musical dimensions. Some forms, textures, moods and styles are difficult to capture in standard notation. Some are impossible. If you want to specify a particular drum machine sound combined with a sampled breakbeat, or an ambient synth pad, or a particular stereo image, standard notation is not the right tool for the job.

Common approaches to organizing composing experiences with synthesizers and software often focus on simplified classical forms without regard to whether these forms are authentic to the genre or to technologies chosen as a medium for creation.

There is nothing wrong with teaching classical forms. But when making music with computers, the best results come from making the music that’s idiomatic to computers. Matt McLean goes to extraordinary lengths to have student compositions performed by professional musicians, but most kids will be confined to the sounds made by the computer itself. Classical forms and idioms sound awkward at best when played by the computer, but electronic music sounds terrific.

The middle school students enrolled in these classes came without much interest in performing, working with notation, or studying the classical music canon. Many saw themselves as “failed” musicians, placed in a general music class because they had not succeeded in or desired to continue with traditional performance-based music classes. Though they no longer had the desire to perform in traditional school ensembles, they were excited about having the opportunity to create music that might be personally meaningful to them.

Here it is, the story of my life as a music student. Too bad I didn’t go to Alex’s school.

How could I teach so that composing for personal expression could be a transformative experience for students? How could I let the voices and needs of the students guide lessons for the composition process? How could I draw on the deep, complex musical understandings that these students brought to class to help them develop as musicians and composers? What tools could I use to quickly engage them in organizing sound in musical and meaningful ways?

Alex draws parallels between writing music and writing English. Both are usually done alone at a computer, and both pose a combination of technical and creative challenges.

Musical thinking (thinking in sound) and linguistic thinking (thinking using language phrases and ideas) are personal creative processes, yet both occur within social and cultural contexts. Noting these parallels, I began to think about connections between the whole-language approach to writing used by language arts teachers in my school and approaches I might take in my music classroom.

In the whole-language approach to writing, students work individually as they learn to write, yet are supported through collaborative scaffolding-support from their peers and the teacher. At the earliest stages, students tell their stories and attempt to write them down using pictures, drawings, and invented notation. Students write about topics that are personally meaningful to them, learning from their own writing and from the writing of their peers, their teacher, and their families. They also study literature of published authors. Classes that take this approach to teaching writing are often referred to as “writers’ workshops”… The teacher facilitates [students’] growth as writers through minilessons, share sessions, and conferring sessions tailored to meet the needs that emerge as the writers progress in their work. Students’ original ideas and writings often become an important component of the curriculum. However, students in these settings do not spend their entire class time “freewriting.” There are also opportunities for students to share writing in progress and get feedback and support from teacher and peers. Revision and extension of students’ writing occur throughout the process. Lessons are not organized by uniform, prescriptive assignments, but rather are tailored to the students’ interests and needs. In this way, the direction of the curriculum and successive projects are informed by the students’ needs as developing writers.

Alex set about creating an equivalent “composers’ workshop,” combining composition, improvisation, and performing with analytical listening and genre studies.

The broad curricular goal of the composers’ workshop is to engage students collaboratively in:

  • Organizing and expressing musical ideas and feelings through sound with real-world, authentic reasons for and means of composing
  • Listening to and analyzing musical works appropriate to students’ interests and experiences, drawn from a broad spectrum of sources
  • Studying processes of experienced music creators through listening to, performing, and analyzing their music, as well as being informed by accounts of the composition process written by these creators.

Alex recommends production software with strong loop libraries so students can make high-level musical decisions with “real” sounds immediately.

While students do not initially work directly with rhythms and pitch, working with loops enables students to begin composing through working with several broad musical dimensions, including texture, form, mood, and affect. As our semester progresses, students begin to add their own original melodies and musical ideas to their loop-based compositions through work with synthesizers and voices.

As they listen to musical exemplars, I try to have students listen for the musical decisions and understand the processes that artists, sound engineers, and producers make when crafting their pieces. These listening experiences often open the door to further dialogue on and study of the multiplicity of musical roles’ that are a part of creating today’s popular music. Having students read accounts of the steps that audio engineers, producers, songwriters, film-score composers, and studio musicians go through when creating music has proven to be informative and has helped students learn the skills for more accurately expressing the musical ideas they have in their heads.

Alex shares my belief in project-based music technology teaching. Rather than walking through the software feature-by-feature, he plunges students directly into a creative challenge, trusting them to pick up the necessary software functionality as they go. Rather than tightly prescribe creative approaches, Alex observes the students’ explorations and uses them as opportunities to ask questions.

I often ask students about their composing and their musical intentions to better understand how they create and what meanings they’re constructing and expressing through their compositions. Insights drawn from these initial dialogues help me identify strategies I can use to guide their future composing and also help me identify listening experiences that might support their work or techniques they might use to achieve their musical ideas.

Some musical challenges are more structured–Alex does “genre studies” where students have to pick out the qualities that define techno or rock or film scores, and then create using those idioms. This is especially useful for younger students who may not have a lot of experience listening closely to a wide range of music.

Rather than devoting entire classes to demonstrations or lectures, Alex prefers to devote the bulk of classroom time to working on the projects, offering “minilessons” to smaller groups or individuals as the need arises.

Teaching through minilessons targeted to individuals or small groups of students has helped to maintain the musical flow of students’ compositional work. As a result, I can provide more individual feedback and support to students as they compose. The students themselves also offer their own minilessons to peers when they have designed to teach more about advanced features of the software, such as how to record a vocal track, add a fade-in or fade-out, or copy their musical material. These technology skills are taught directly to a few students, who then become the experts in that skill, responsible for teaching other students in the class who need the skill.

Not only does the peer-to-peer learning help with cultural authenticity, but it also gives students invaluable experience with the role of teacher.

One of my first questions is usually, “Is there anything that you would like me to listen for or know about before I listen?” This provides an opportunity for students to seek my help with particular aspects of their composing process. After listening to their compositions, I share my impressions of what I hear and offer my perspective on how to solve their musical problems. If students choose not to accept my ideas, that’s fine; after all, it’s their composition and personal expression… Use of conferring by both teacher and students fosters a culture of collaboration and helps students develop skills in peer scaffolding.

Alex recommends creating an online gallery of class compositions. This has become easier to implement since 2007 with the explosion of blog platforms like Tumblr, audio hosting tools like SoundCloud, and video hosts like YouTube. There are always going to be privacy considerations with such platforms, but there is no shortage of options to choose from.

Once a work is online, students can listen to and comment on these compositions at home outside of class time. Sometimes students post pieces in progress, but for the most part, works are posted when deemed “finished” by the composer. The online gallery can also be set up so students can hear works written by participants in other classes. Students are encouraged to listen to pieces published online for ideas to further their own work, to make comments, and to share these works with their friends and family. The realworld publishing of students’ music on the Internet seems to contribute to their motivation.

Assessing creative work is always going to be a challenge, since there’s no objective basis to assess it on. Alex looks at how well a student composer has met the goal of the assignment, and how well they have achieved their own compositional intent.

The word “composition” is problematic in the context of contemporary computer-based production. It carries the cultural baggage of Western Europe, the idea of music as having a sole identifiable author (or authors.) The sampling and remixing ethos of hip-hop and electronica are closer to the traditions of non-European cultures where music may be owned by everyone and no one. I’ve had good results bringing remixing into the classroom, having students rework each others’ tracks, or beginning with a shared pool of audio samples, or doing more complex collaborative activities like musical shares. Remixes are a way of talking about music via the medium of music, and remixes of remixes can make for some rich and deep conversation. The word “composition” makes less sense in this context. I prefer the broader term “production”, which includes both the creation of new musical ideas and the realization of those ideas in sound.

So far in this post, I’ve presented notation-based composition and loop-based production as if they’re diametrical opposites. In reality, the two overlap, and can be easily combined. A student can create a part as a MIDI sequence and then convert it to notation, or vice versa. The school band or choir can perform alongside recorded or sequenced tracks. Instrumental or vocal performances can be recorded, sampled, and turned into new works. Electronic productions can be arranged for live instruments, and acoustic pieces can be reconceived as electronica. If a hip-hop track can incorporate a sample of Duke Ellington, there’s no reason that sample couldn’t be performed by a high school jazz band. The possibilities are endless.

Theory for Producers: the White Keys

I’m pleased to announce the second installment of Theory For Producers, jointly produced by Soundfly and the MusEDLab. The first part discussed the scales you can play on the black keys of the piano. This one talks about three of the scales you get from the white keys. The next segment will deal with four additional white-key scales. Go try it!

Theory for Producers: the White Keys

If you’re a music educator or theory nerd, and would like to read more about the motivation behind the course design, read on.

Some of my colleagues in the music teaching world are puzzled by the order in which we’re presenting concepts. Theory resources almost always start with the C major scale, but we start with E-flat minor pentatonic. While it’s harder to represent in notation, E-flat minor pentatonic is easier to play and learn by ear, and for our target audience, that’s the most important consideration.

Okay, fine, the pentatonics are simple, it makes sense to start with them. But surely we would begin the white key part on the major scale, right? Nope! We start with Mixolydian mode. In electronica, hip-hop, rock, and pop, Mixolydian is more “basic” than major is. The sound of the flat seventh is more native to this music than the leading tone, and V-I cadences are rare or absent. I once had a student complain that the major scale makes everything sound like “Happy Birthday.” Our Mixolydian example, a Michael Jackson tune, was chosen to make our audience of producers feel culturally at home, to make them feel like we value the dance music of the African diaspora over the folk and classical of Western Europe.

After Mixolydian, we discuss Lydian mode. While it’s a pretty exotic scale, we chose to address it before major because it’s more forgiving to improvise with–Lydian doesn’t have any “wrong” notes. In major, you have to be careful about the fourth, because it has strong functional connotations, and because it conflicts hard with the third. In Lydian, you can play notes in any order and any combination without fear of hitting a clunker. Also, exotic though it may be, Lydian does pop up in a few well-known songs, like in a recent Katy Perry hit.

Finally, we do get to major, using David Bowie, and Queen. Even here, though, we downplay functional harmony, treating major as just another mode. Our song example uses a I-IV-V chord progression, but it runs over a static riff bassline, which makes it float rather than resolve.

This class only deals with the three major diatonic modes. We’ll get to the minor ones (natural minor, Dorian, Phrygian and Locrian) in the third class. We debated doing minor first, but there are more of the minor modes, and they’re more complicated.

We also debated whether or not to talk about chords. The chord changes in our examples are minimal, but they’re present. We ultimately decided to stick to horizontal scales only for the time being, and to treat chords separately. We plan to go back through all of the modes and talk about the chord progressions characteristic of each one. For example, with Mixolydian, we’ll talk about I-bVII-IV; with Lydian we’ll do I-II; and with major we’ll do all the permutations of I, IV, V and vi.

Once again, we know it’s unconventional to deal with modes so thoroughly before even touching any chords, but for our audience, we think this approach will make more sense. Electronic music is not big on complex harmony, but it is big on modes.

Musical simples – Teenage Dream

I’m working with Soundfly on the next installment of Theory For Producers, our ultra-futuristic online music theory course. The first unit covered the black keys of the piano and the pentatonic scales. The next one will talk about the white keys  and the diatonic modes. We were gathering examples, and we needed to find a well-known pop song that uses Lydian mode. My usual go-to example for Lydian is “Possibly Maybe” by Björk. But the course already uses a Björk tune for different example, and the Soundfly guys quite reasonably wanted something a little more millennial-friendly anyway. We decided to use Katy Perry’s “Teenage Dream” instead.

A couple of years ago, Slate ran an analysis of this tune by Owen Pallett. It’s an okay explanation, but it doesn’t delve too deep. We thought we could do better.

Here’s my transcription of the chorus:

When you look at the melody, this would seem to be a straightforward use of the B-flat major scale. However, the chord changes tell a different story. The tune doesn’t ever use a B-flat major chord. Instead, it oscillates back and forth between E-flat and F. In this harmonic context, the melody doesn’t belong to the plain vanilla B-flat major scale at all, but rather the dreamy and modernist E-flat Lydian mode. The graphic below shows the difference.

Teenage Dream Eb Lydian circles

Both scales use the same seven pitches: B-flat, C, D, E-flat, F, G, and A. The only difference between the two is which note you consider to be “home base.” Let’s consider B-flat major first.

To make chords from a scale, you pick any note, and then go clockwise around the scale, skipping every other degree. The chords are named for the note you start on. If you start on the fourth note, E-flat, you get the IV chord (the other two notes are G and B-flat.) If you start on the fifth note, F, you get the V chord (the other two notes are A and C.) In a major key, IV and V are very important chords. They’re called the subdominant and dominant chords, respectively, and they both create a feeling of suspense. You can resolve the suspense by following either one with the I chord. The weird thing about “Teenage Dream” is that if you think about it as being in B-flat, then it never lands on the I chord at all. It just oscillates back and fourth between IV and V. The suspense never gets resolved.

If we think of “Teenage Dream” as being in E-flat Lydian, then the E-flat chord is I, which makes more sense. The function of the F chord in this context isn’t clearly defined by music theory, but it does sound good. Lydian is very similar to the major scale, with only one difference: while the fourth note of E-flat major is A-flat, the fourth note of E-flat Lydian is A natural. That raised fourth gives Lydian mode its otherworldly sound. The F chord gets its airborne quality from that raised fourth.

Click here to play over “Teenage Dream” using the aQWERTYon. The two chords can be played on the letters Z-A-Q and X-S-W. For comparison, try playing it with B-flat major. Read more about the aQWERTYon here.

“Teenage Dream” is not the only well-known song to use the Lydian I-II progression. Other high profile examples include “Dreams” by Fleetwood Mac and “Jane Says” by Jane’s Addiction. over the same chords. Try singing any of these songs over any of the others; they all fit seamlessly.

The chorus of “Teenage Dream” uses a striking rhythm on the phrases “you make me”, “teenage dream”, and “I can’t sleep”. The song is in 4/4 time, like nearly all contemporary pop tracks, but that chorus rhythm has a feeling of three about it. It’s no illusion. The words “you” and “make” in the first line are each three eighth notes long. It sounds like an attempt to divide the eight eighth notes into groups of three. This rhythm is called Tresillo, and it’s one of the building blocks of Afro-Cuban drumming.


Tresillo is the front half of son clave. It’s extraordinarily common in American vernacular music, especially in accompaniment patterns. You hear Tresillo in the bassline to “Hound Dog” and countless other fifties rock songs; in the generic acoustic guitar strumming pattern used by singer-songwriters everywhere; and in the kick and snare pattern characteristic of reggaetón. Tresillo is ubiquitous in jazz, and in the dance music of India and the Middle East.

“Teenage Dream” alternates the Tresillo with a funky syncopated rhythm pattern that skips the first beat of the measure. When you listen to the line “feel like I’m livin’ a”, there’s a hole right before the word “feel”. That hole is the downbeat, which is the usual place to start a phrase. When you avoid the obvious beat, you surprise the listener, which grabs their attention. The drums underneath this melody hammer relentlessly away on the strong beats, so it’s easy to parse out the rhythmic sophistication. Katy Perry songs have a lot of empty calories, but they taste as good as they do for a reason.

Milo meets Beethoven

For his birthday, Milo got a book called Welcome to the Symphony by Carolyn Sloan. We finally got around to showing it to him recently, and now he’s totally obsessed.

Welcome To The Symphony by Carolyn Sloan

The book has buttons along the side which you can press to hear little audio samples. They include each orchestra instrument playing a short Beethoven riff. All of the string instruments play the same “bum-bum-bum-BUMMM” so you can compare the sounds easily. All the winds play a different little phrase, and the brass another. The book itself is fine and all, but the thing that really hooked Milo is triggering the riffs one after another, Ableton-style, and singing merrily along.

Milo got primed to enjoy this book by two coincidental things. One is that in his preschool, they’ve been listening to Peter and the Wolf a lot, dancing to it, acting it out, etc. They use a YouTube video that shows both the story and the instruments side by side, so Milo has very clear ideas of what the oboe, clarinet, etc all look like and sound like. When he saw them in the orchestra book, he recognized them all immediately.

The other thing is this weird computer animated cartoon called Taratabong, which is about anthropomorphic musical instruments. Milo has been watching it on YouTube a bunch, to the point of wanting me to pretend to be different characters and “talk” to him (which is an entertaining challenge for me–how do you have a conversation as a snare drum?) So Milo also recognizes different instruments in the orchestra book as Taratabong characters.

Milo has now voluntarily watched a YouTube video of the entire first movement of Beethoven’s Fifth conducted by Leonard Bernstein, several times. That’s like nine minutes of classical music, which for a three-year-old is equivalent to nine hours. He sings along to all the riffs he recognized, announces each instrument as he sees it, and tells me about how Leonard Bernstein is Grandfather from Peter and the Wolf. I want to emphasize that we haven’t pushed him into any of this. If you read this blog, you know that I’m an outspoken anti-fan of Beethoven. We just put this stuff under Milo’s nose, and if he hadn’t been interested, we wouldn’t have pushed it.

The classical music tribe expresses continual anguish about how hard it is to draw people into the music. Having inadvertently created a budding Beethoven lover, I have a few insights to offer. Milo got connected to the music through multiple media simultaneously, in multiple settings. He was exposed initially in the context of stories about animals and cartoon characters. That exposure happened in the context of acting and dancing, not passive sitting or being lectured to. And when he did start listening, it was via playback devices that he controls completely: YouTube Kids on the iPad, and the buttons on the book.

Of all these different music experiences, the Ableton-like sample triggering is the one that has most seized Milo’s enthusiasm. Sometimes he wants to read the book and play the sounds when the text indicates. Sometimes he wants to systematically listen through each sound, singing along and acting out the instruments. Sometimes he just jams out, playing the excerpts in different orders and in different rhythms. I suspect he’d be even happier if he could get the sounds to loop. He wants to sing along, but the little phrases are half over before he can even get oriented. If the phrases looped in a musical-sounding way, I bet he would dig in much deeper.

This is not Milo’s first experience triggering sample playback. Before he even turned two, we spent a lot of time playing around with an APC 40.


Milo adores the lights and colors, and instantly grasped how the volume faders work. In general, though, the APC experience was too complicated for him. It was too easy to make it stop working, to lose the connection between button pushes and the music changing, and to generally get lost in the interface. (I have some of those same problems!) The orchestra book has the advantage of being vastly simpler and more predictable.

There’s a page in the book that shows Beethoven with quill pen, writing the music. (Milo is continually disappointed not to see Beethoven himself in any of the performance videos.) Interestingly, Milo has started using the phrase “writing music” as a synonym for “playing music”, either from an instrument or from iTunes. He seems not to know or care about the distinction between playing back pre-recorded music and creating new music. This conflation of writing and playing music was likely helped by the time Milo has spent with the aQWERTYon, an interface developed by the NYU MusEDLab for performing music on the computer keyboard.

aQWERTYon screencap

Milo isn’t extremely interested in the musical aspect of the aQWERTYon. He calls it “ABCs” and is mostly interested in using it to type his favorite letters. He also enjoys singing the alphabet song while playing semi-randomly along.

The MusEDLab’s work is motivated by the fact that computers make it enormously easier for total novices to participate actively in music. If Beethoven symphonies can be played with as toys, participated in as games, and connected to meaningful stories and activities, then it’s inevitable that kids are going to want to get involved. If I had experienced Beethoven as raw material for my own expression, I’d probably feel quite differently about him.