Freedom ’90

Since George Michael died, I’ve been enjoying all of his hits, but none of them more than this one. Listening to it now, it’s painfully obvious how much it’s about George Michael’s struggles with his sexual orientation. I wonder whether he was being deliberately coy in the lyrics, or if he just wasn’t yet fully in touch with his identity. Being gay in the eighties must have been a nightmare.

This is the funkiest song that George Michael ever wrote, which is saying something. Was he the funkiest white British guy in history? Quite possibly. 

The beat

There are five layers to the drum pattern: a simple closed hi-hat from a drum machine, some programmed bongos and congas, a sampled tambourine playing lightly swung sixteenth notes, and finally, once the full groove kicks in, the good old Funky Drummer break. I include a Noteflight transcription of all that stuff below, but don’t listen to it, it sounds comically awful.

George Michael uses the Funky Drummer break on at least two of the songs on Listen Without Prejudice Vol 1. Hear him discuss the break and how it informed his writing process in this must-watch 1990 documentary.

The intro and choruses

Harmonically, this is a boilerplate C Mixolydian progression: the chords built on the first, seventh and fourth degrees of the scale. You can hear the same progression in uncountably many classic rock songs.

C Mixolydian chords

For a more detailed explanation of this scale and others like it, check out Theory For Producers.

The rhythm is what makes this groove so fresh. It’s an Afro-Cuban pattern full of syncopation and hemiola. Here’s an abstraction of it on the Groove Pizza. If you know the correct name of this rhythm, please tell me in the comments!

The verses

There’s a switch to plain vanilla C major, the chords built on the fifth, fourth and root of the scale.

C major chords

Like the chorus, this is standard issue pop/rock harmonically speaking, but it also gets its life from a funky Latin rhythm. It’s a kind of clave pattern, five hits spread more or less evenly across the sixteen sixteenth notes in the bar. Here it is on the Groove Pizza.

The prechorus and bridge

This section unexpectedly jumps over to C minor, and now things get harmonically interesting. The chords are built around a descending chromatic bassline: C, B, B-flat, A. It’s a simple idea but with complicated implications, because it implies four chords built on three different scales between them. First, we have the tonic triad in C natural minor, no big deal there. Next comes the V chord in C harmonic minor. Then we’re back to C natural minor, but with the seventh in the bass. Finally, we go to the IV chord in C Dorian mode. Really, all that we’re doing is stretching C natural minor to accommodate a couple of new notes, B natural in the second chord, and A natural in the fourth one.

C minor - descending chromatic bassline

The rhythm here is similar but not identical to the clave-like pattern in the verse–the final chord stab is a sixteenth note earlier. See and hear it on the Groove Pizza.

I don’t have the time to transcribe the whole bassline, but it’s absurdly tight and soulful. The album credits list bass played both by Deon Estus and by George Michael himself. Whichever one of them laid this down, they nailed it.

Song structure

“Freedom ’90” has an exceedingly peculiar structure for a mainstream pop song. The first chorus doesn’t hit until almost two minutes in, which is an eternity–most pop songs are practically over that that point. The graphic below shows the song segments as I marked them in Ableton.

Freedom '90 structure

The song begins with a four bar instrumental intro, nothing remarkable about that. But then it immediately moves into an eight bar section that I have trouble classifying. It’s the spot that would normally be occupied by verse one, but this part uses the chorus harmony and is different from the other verses. I labeled it “intro verse” for lack of a better term. (Update: upon listening again, I realized that this section is the backing vocals from the back half of the chorus. Clever, George Michael!) Then there’s an eight bar instrumental break, before the song has really even started. George Michael brings you on board with this unconventional sequence because it’s all so catchy, but it’s definitely strange.

Finally, twenty bars in, the song settles into a more traditional verse-prechorus-chorus loop. The verses are long, sixteen bars. The prechorus is eight bars, and the chorus is sixteen. You could think of the chorus as being two eight bar sections, the part that goes “All we have to do…” and the part that goes “Freedom…” but I hear it as all one big section.

After two verse-prechorus-chorus units, there’s a four bar breakdown on the prechorus chord progression. This leads into sixteen bar bridge, still following the prechorus form. Finally, the song ends with a climactic third chorus, which repeats and fades out as an outtro. All told, the song is over six minutes. That’s enough time (and musical information) for two songs by a lesser artist.

A word about dynamics: just from looking at the audio waveform, you can see that “Freedom ’90” has very little contrast in loudness and fullness over its duration. It starts sparse, but once the Funky Drummer loop kicks in at measure 13, the sound stays constantly big and full until the breakdown and bridge. These sections are a little emptier without the busy piano part. The final chorus is a little bigger than the rest of the song because there are more vocals layered in, but that still isn’t a lot of contrast. I guess George Michael decided that the groove was so hot, why mess with it by introducing contrast for the sake of contrast? He was right to feel that way.

Affordances and Constraints

Note-taking for User Experience Design with June Ahn

Don Norman discusses affordances and constraints in The Design of Everyday Things, Chapter Four: Knowing What To Do.

Don Norman - The Design of Everyday Things

User experience design is easy in situations where there’s only one thing that the user can possibly do. But as the possibilities multiply, so do the challenges. We can deal with new things using information from our prior experiences, or by being instructed. The best-designed things include the instructions for their own use, like video games whose first level act as tutorials, or doors with handles that communicate how you should operate them by their shape and placement.

We use affordances and constraints to learn how things work. Affordances suggest the range of possibilities, and constraints limit the alternatives. Constraints include:

  • Physical limitations. Door keys can only be inserted into keyholes vertically, but you can still insert the key upside down. Car keys work in both orientations.
  • Semantic constraints. We know that red lights mean stop and green lights mean go, so we infer that a red light means a device is off or inoperative, and a green light means it’s on or ready to function. We have a slow cooker that uses lights in the opposite way and it screws me up every time.
  • Cultural constraints. Otherwise known as conventions. (Not sure how these are different from semantic constraints.) Somehow we all know without being told that we’re supposed to face forward in the elevator. Google Glass was an epic failure because its early adopters ran into the cultural constraint of people not liking to be photographed without consent.
  • Logical constraints. The arrangement of knobs controlling your stove burners should match the arrangement of the burners themselves.

The absence of constraints makes things confusing. Norman gives examples of how much designers love rows of identical switches which give no clues as to their function. Distinguishing the switches by shape, size, or grouping might not look as elegant, but would make it easier to remember which one does what thing.

Helpful designs use visibility (making the relevant parts visible) and feedback (giving actions an immediate and obvious effect.) Everyone hates the power buttons on iMacs because they’re hidden on the back, flush with the case. Feedback is an important way to help us distinguish the functional parts from the decorative ones. Propellerheads Reason is an annoying program because its skeuomorphic design puts as many decorative elements on the screen as functional ones. Ableton Live is easier to use because everything on the screen is functional.

When you can’t make things visible, you can give feedback via sound. Pressing a Mac’s power button doesn’t immediately cause the screen to light up, but that’s okay, because it plays the famous startup sound. Norman’s examples of low-tech sound feedback include the “zzz” sound of a functioning zipper, a tea kettle’s whistle, and the various sounds that machines make when they have mechanical problems. The problem with sound as feedback is that it can be intrusive and annoying.

The term “affordance” is the source for a lot of confusion. Norman tries to clarify it in his article “Affordance, Conventions and Design.” He makes a distinction between real and perceived affordances. Anything that appears on a computer screen is a perceived affordance. The real affordances of a computer are its physical components: the screen itself, the keyboard, the trackpad. The MusEDLab was motivated to create the aQWERTYon by considering the computer’s real affordances for music making. Most software design ignores the real affordances and only considers the perceived ones.

Designers of graphical user interfaces rely entirely on conceptual models and cultural conventions. (Consider how many programs use a graphic of a floppy disk as a Save icon, and now compare to the last time you saw an actual floppy disk.) For Norman, graphics are perceived affordances by definition.

Joanna McGrenere and Wayne Ho try to nail the concept down harder in “Affordances: Clarifying and Evolving a Concept.” The term was coined by the perceptual psychologist James J. Gibson in his book The Ecological Approach to Visual Perception. For Gibson, affordances exist independent of the actor’s ability to perceive them, and don’t depend on the actor’s experiences and culture. For Norman, affordances can include both perceived and actual properties, which to me makes more sense. If you can’t figure out that an affordance exists, then what does it matter if it exists or not?

Norman collapses two distinct aspects of design: an object’s utility of an object and the way that users learn or discover that utility. But are designing affordances and designing the information about the affordances the same thing? McGrenere and Ho say no, that it’s the difference between usefulness versus usability. They complain that the HCI community has focused on usability at the expense of usefulness. Norman says that a scrollbar is a learned convention, not a real affordance. McGrenere and Ho disagree, because the scrollbar affords scrolling in a way that’s built into the software, making it every bit as much a real affordance as if it were a physical thing. The learned convention is the visual representation of the scrollbar, not the basic fact of it.

The best reason to distinguish affordances from their communication or representation is that sometimes the communication gets in the way of the affordance itself. For example, novice software users need graphical user interfaces, while advanced users prefer text commands and keyboard shortcuts. A beginner needs to see all the available commands, while a pro prefers to keep the screen free of unnecessary clutter. Ableton Live is a notoriously beginner-unfriendly program because it prioritizes visual economy and minimalism over user handholding. A number of basic functions are either invisible or so tiny as to be effectively invisible. Apple’s GarageBand welcomes beginners with photorealistic depictions of everything, but its lack of keyboard shortcuts makes it feel like wearing oven mitts for expert users. For McGrenere and Ho, the same feature of one of these programs can be an affordance or anti-affordance depending on the user.

Deconstructing the bassline in Herbie Hancock’s “Chameleon”

If you have even a passing interest in funk, you will want to familiarize yourself with Herbie Hancock’s “Chameleon.” And if you are preoccupied and dedicated to the preservation of the movement of the hips, then the bassline needs to be a cornerstone of your practice.

Chameleon - circular bass

Here’s a transcription I did in Noteflight – huge props to them for recently introducing sixteenth note swing.

And here’s how it looks in the MIDI piano roll:

The “Chameleon” bassline packs an incredible amount of music into just two bars. To understand how it’s put together, it’s helpful to take a look at the scale that Herbie built the tune around, the B-flat Dorian mode. Click the image below to play it on the aQWERTYon. I recommend doing some jamming with it over the song before you move on.

B-flat Dorian

Fun fact: this scale contains the same pitches as A-flat major. If you find that fact confusing, then feel free to ignore it. You can learn more about scales and modes in my Soundfly course.

The chord progression

The opening section of “Chameleon” is an endless loop of two chords, B♭-7 and E♭7. You build both of them using the notes in B-flat Dorian. To make B♭-7, start on the root of the scale, B-flat. Skip over the second scale degree to land on the third, D-flat. Skip over the fourth scale degree to land on the fifth, F. Then skip over the sixth to land on the seventh, A-flat. If you want to add extensions to the chord, just keep skipping scale degrees, like so:

B-flat Dorian mode chords

To make E♭7, you’re going to use the same seven pitches in the same order, but you’re going to treat E-flat as home base rather than B-flat. You could think of this new scale as being E-flat Mixolydian, or B-flat Dorian starting on E-flat; they’re perfectly interchangeable. Click to play E-flat Mixolydian on the aQWERTYon. You build your E♭7 chord like so:

B-flat Dorian mode chords on E-flat

Once you’ve got the sound of B♭-7 and E♭7 in your head, let’s try an extremely simplified version of the bassline.

Chord roots only

At the most basic level, the “Chameleon” bassline exists to spell out the chord progression in a rhythmically interesting way. (This is what all basslines do.) Here’s a version of the bassline that removes all of the notes except the ones on the first beat of each bar. They play the roots of the chords, B-flat and E-flat.

That’s boring, but effective. You can never go wrong playing chord roots on the downbeat.

Simple arpeggios

Next, we’ll hear a bassline that plays all of the notes in B♭-7 and E♭7 one at a time. When you play chords in this way, they’re called arpeggios.

The actual arpeggios

The real “Chameleon” bassline plays partial arpeggios–they don’t have all of the notes from each chord. Also, the rhythm is a complicated and interesting one.

Below, you can explore the rhythm in the Groove Pizza. The orange triangle shows the rhythm of the arpeggio notes, played on the snare. The yellow quadrilateral shows the rhythm of the walkups, played on the kick–we’ll get to those below.

The snare rhythm has a hit every three sixteenth notes. It’s a figure known in Afro-Latin music as tresillo, which you hear absolutely everywhere in all styles of American popular and vernacular music. Tresillo also forms the front half of the equally ubiquitous son clave. (By the way, you can also use the Groove Pizza to experiment with the “Chameleon” drum pattern.)

As for the pitches: Instead of going root-third-fifth-seventh, the bassline plays partial arpeggios. The figure over B♭-7 is just the root, seventh and root again, while the one over E♭7 is the root, fifth and seventh.

Adding the walkups

Now let’s forget about the arpeggios for a minute and go back to just playing the chord roots on the downbeats. The bassline walks up to each of these notes via the chromatic scale, that is, every pitch on the piano keyboard.

Chromatic walkups are a great way to introduce some hip dissonance into your basslines, because they can include notes that aren’t in the underlying scale. In “Chameleon” the walkups include A natural and D natural. Both of these notes sound really weird if you sustain them over B-flat Dorian, but in the context of the walkup they sound perfectly fine.

Putting it all together

The full bassline consists of the broken arpeggios anticipated by the walkups.

If you’re a guitarist or bassist, you can play this without even shifting position. Use your index on the third fret, your middle on the fourth fret, your ring on the fifth fret, and your pinkie on the sixth fret.

              .          . .
G|----------.-3----------3-6--|
D|----------6-----------------|
A|---------------3-4-5-6------|
E|--3-4-5-6-------------------|

If you’ve got this under your fingers, maybe you’d like to figure out the various keyboard and horn parts. They aren’t difficult, but you’ll need one more scale, the B-flat blues scale. Click the image to jam with it over the song and experience how great it sounds.

B-flat blues

There you have it, one of the cornerstones of funk. Good luck getting it out of your head!

The Groove Pizza now exports MIDI

Since its launch, you’ve been able to export your Groove Pizza beats as WAV files, or continue working on them in Soundtrap. But now, thanks to MusEDLab developer Jordana Bombi, you can also save your beats as MIDI files as well.

Groove Pizza MIDI export

You can bring these MIDI files into your music production software tool of choice: Ableton Live, Logic, Pro Tools, whatever. How cool is that?

There are a few limitations at the moment: your beats will be rendered in 4/4 time, regardless of how many slices your pizza has. You can always set the right time signature after you bring the MIDI into your production software. Also, your grooves will export with no swing–you’ll need to reinstate that in your software as well.

We have some more enhancements in the pipeline, aside from fixing the limitations just mentioned. We’re working on a “continue in Noteflight” feature, real-time MIDI input and output, and live performance using the QWERTY keyboard. I’ll keep you posted.

Design-Based Research

Note-taking for Research on Games and Simulations with Jan Plass

Barab, S. A. (2014). Design-based research: a methodological toolkit for engineering change. In K. Sawyer (ed.) Handbook of the Learning Sciences, Vol 2, (pp. 233-270), Cambridge, MA: Cambridge University Press.

Design-based research

Design-based research (DBR) is a subject close to my heart, because it was the basis of my masters thesis, and informs the work of the NYU Music Experience Design Lab. All of our tools are designed and tested in the context of messy and complex natural learning and creating environments: classrooms, bedrooms, studios, and public events. We evaluate our tools continuously, but the only purely empirical and “experimental” methods we use involve Google analytics. We sometimes conduct user research in formal settings, but mostly observe practice “in the wild” between regular iterations.

DBR follows Dewey’s model of praxis, not so much a traditional positivist science or ethnographic tradition of inquiry, but rather a pragmatic form of inquiry “where theories are judged not by their claims to truth, but by their ability to do work in the world.” It entails the study of learning innovations inspired by theory, conducted in naturalistic contexts. The goal is to produce new tools and practices that can be generalized to other learning environments (typically schools, but not necessarily.) DBR also aims to produce new theories, viewing them as intertwined with practice rather than occurring prior to it.

DBR would appear to fail to meet tests of rigor, freedom from bias, and reproducibility. However, advocates respond that for a learning intervention to be studied with full empirical rigor, variables must be isolated to the point where they no longer resemble the reality we are supposed to be studying. We can draw an analogy from music psychology. A teacher of mine studies musical tension and release. She isolates the various musical parameters that might create a sense of tension and release – pitch, rhythm, timbre and so on – as synthesizer tones varying only along each parameter, and measures listener responses. While her results are rigorous, her test material bears almost no resemblance to actual music. It may well be that musical tension and release are emergent properties of parameters interacting in social/cultural context. Research in a more naturalistic context might hopelessly confound the variables, but might be the only way to access a valid explanation. As in music, the context of learning experiences is not simply a backdrop against which things occur, but “an integral part of the complex causal mechanisms that give rise to the phenomenon under study.”

DBR is to traditional empirical research what agile methodologies are to waterfall-style software development. Rather than thinking of a study as a hypothesis which is then tested and written up, DBR is an ongoing process of iteration, testing, and iterating further. This makes it difficult to point to a clear, unambiguous result. DBR findings are more likely to take the form of narratives, descriptions of the various iterations and tests. The challenge then becomes to present these narratives in ways that readers can generalize from them, rather than limiting their relevance to the specific situations being described. While DBR is not ethnography, its results may well take the form of ethnographic “thick description” in Clifford Geertz’s sense. As in qualitative social science research, rigor “comes from principled accounts that provide logical chains of reasoning and prove useful to others.”

In considering DBR, we need to draw a distinction between measuring outputs and outcomes. Outputs are directly measurable results of an intervention, while outcomes are larger-scale and longer-term consequences. For example, standardized test scores are an output of schooling, while learning is a consequence. Because outputs are so much more readily measured than outcomes, there is a danger that we will optimize around outputs rather than outcomes (as schools have around standardized tests.) To combat this danger, DBR must provide explanatory accounts of outcomes with the same persuasive power as quantitative measures of outputs. Research must combine “experience-near meanings” (the local story) with “experience-distance significance” (the more general implications and applications.) Good DBR will produce grounded “petite generalizations” that can be built up into broader generalizations. Researchers will also need to be explicit about the assumptions and theoretical bases that underlie the work, since these can not be controlled for.

A participant ethnography of the Ed Sullivan Fellows program

Note: I refer to mentors by their real names, and to participants by pseudonyms

Ed Sullivan Fellows (ESF) is a mentorship and artist development program run by the NYU Steinhardt Music Experience Design Lab. It came about by a combination of happenstances. I had a private music production student named Rob Precht, who had found my blog via a Google search. He and I usually held our lessons in the lab’s office space. Over the course of a few months, Rob met people from the lab and heard about our projects. He found us sufficiently inspiring that he approached us with an idea. He wanted to give us a grant to start a program that would help young people from under-resourced communities get a start in the music industry. He asked us to name it after his grandfather, Ed Sullivan, whose show had been crucial to launching the careers of Elvis, the Beatles, and the Jackson 5. While Rob’s initial idea had been to work with refugees who had relocated to New York, we agreed to shift the focus to native New York City residents, since our connections and competencies were stronger there.

Ed Sullivan Fellows

The Ed Sullivan Fellows program is run by Jamie Ehrenfeld, a graduate of NYU’s music education program, a founding member of the Music Experience Design Lab, and a music teacher at Eagle Academy in Brownsville. Like many members of the lab, she straddles musical worlds, bringing her training in classical voice to her work mentoring rappers and R&B singers. Participants are young men and women between the ages of 15 and 20, mostly low-SES people of color. They meet on Saturday afternoons at NYU Steinhardt to write and record songs; to get mentorship on the music business, marketing and branding; and to socialize. We had originally conceived of ESF as a series of formally organized classes, but it became immediately obvious that such a structure was going to be impractical. While there is a regular core of attendees, their lives are complicated, and there is no way to predict who will show up week to week or when they will arrive and leave. Instead, sessions have taken on a clubhouse feel, a series of ad-hoc jam sessions, cyphers, informal talks, and open-ended creativity. Conversations are as likely to focus on participants’ emotions, politics, social life and identity as they are on anything pertaining to music.

There is a “core squad” of nineteen regular ESF participants, and an additional thirty occasional attendees. Many are students at Eagle Academy and members of their social networks. This group is mostly black and Latino. Another smaller group attends City-As-School. Only three Fellows total are white. The Fellows are mostly male, partially because many of them attend an all-male school, and partially because of hip-hop’s skewed gender dynamics generally. There are six core mentors (including myself) and another sixteen peripheral mentors. Some are young black men and women from the Fellows’ social networks, and the rest are NYU people, or are socially connected to the lab. All of the mentors are musicians, but otherwise come from a variety of backgrounds: education, business, software development, design.

The ethnomusicologist Thomas Turino draws a distinction between “participatory” and “presentational” cultures of music performance. Presentational performances include all of the music you would hear at a professional concert: classical, jazz, rock, and so on. Participatory performances include campfire singalongs, jam sessions, and drum circles, where there is little to no distinction between performers and audience. We tend to regard presentational performance as “real” music. Turino argues that we undervalue participatory music subcultures, because they are some of the few cultural spaces in America where monetary profit is not a primary value, and where our jobs and economic status are not our major identifying characteristics. While the ostensible goal of ESF is developing young artists professionally, the actual music-making that takes place is highly participatory in nature.

Hip-hop is not the only style of music that the Fellows create. Some are singer-songwriters in a pop, R&B or gospel style. Still, hip-hop is the default, the unifying thread, and the common vocabulary. Among the forty-six Fellows, there are twenty-three emcees, nineteen singers, eighteen producers, thirteen live instrumentalists, and twenty-nine improvisors who are comfortable participating in a live jam. (These categories are not mutually exclusive.) Among the nine more mentors, there are three emcees, four singers, four producers, six live instrumentalists, and eight improvisors. Of the twenty-three Fellows who self-identify as rappers, sixteen of them can freestyle, improvising lyrics on the spot, a formidably challenging musical practice. Participation in cyphers and jams is a core part of the ESF ethos.

The Fellows are familiar with the drug-influenced mumble rappers who currently dominate the charts, but their sensibilities are more clear-eyed, narrative, and direct. For example, Rashad cites Chance The Rapper as his major musical inspiration. Chance has a densely intellectual flow with an irrepressible sunniness, and raps about his life, his community, politics, and his relationship with God. Other Fellows express outspoken admiration for Kendrick Lamar, who is less cheerful and optimistic, but who also has a strong social and political conscience. Like most current hip-hop artists, ESF participants favor beats that are extremely slow and sparse, with electronic drums playing stuttering subdivisions of the beat accompanied by disjointed samples or soft textural ambience on top. I try to keep current with hip-hop, as much as a 41-year-old white dad can, but this music continues to surprise me with how futuristic it sounds. It has a science-fictional dystopian quality, but for all its iciness, the funk heartbeat remains.

ESF meets and works in spaces belonging to NYU Steinhardt’s music technology and music education departments: primarily a conference room and recording studio, spilling over into various labs and classrooms as needed. During the week, these spaces host classes and presentations, and are otherwise occupied by NYU students, who socialize in low-key ways or work on their laptops. While NYU’s culture is informal, it is still an academic institution, and the predominant feeling in the common space is quiet and productive. On the rare occasion when music is played on the conference area PA system during the week, it is part of a class or lecture. During ESF sessions, by contrast, the PA plays hip-hop beats, sometimes looped endlessly for long periods of time, and usually at party volumes. The Fellows have an unreserved social style, and the feeling when they occupy the space is more one of play than of work.

ESF periodically records in the James Dolan studio, named for the owner of Madison Square Garden, himself an enthusiastic amateur musician and at one time the parent of two NYU music technology students. He noticed that the school’s recording facilities were old and run-down, so he essentially gave Steinhardt a blank check to build a state-of-the-art studio. Ten million dollars later, NYU boasts one of the best studios in New York, with top-of-the-line equipment and immaculate acoustics. The monitor speakers alone cost twenty thousand dollars; the mixing desk costs on the order of a hundred thousand. When the Fellows record, they are assisted by well-trained and highly competent student engineers. I always feel like more of a “real musician” whenever I work in there, and clearly it has a similar effect on the Fellows. For all its luxuriousness, though, the Dolan studio was designed to capture live performances using Pro Tools, not for creative hip-hop production with Logic or FL Studio or Ableton Live. The Fellows would be better served by a group of smaller, less grandiose spaces equipped with the software and hardware designed specifically for their methods.

For all its progressiveness, New York is one of the most racially segregated cities in America. NYU students and ESF participants live within a few miles of each other, but occupy very different social worlds. Nearly all of the Fellows are black or Latino, and all are of low socioeconomic status. NYU students are ethnically diverse, but this is because of the prevalence of international students; the students of color are predominantly Asian. NYU is an extremely expensive private institution, and unlike Ivy League schools, it does not have a large endowment that it can use for financial aid and scholarships. While not all NYU students are wealthy, a substantial percentage certainly are, and an air of casual privilege pervades. NYU music technology students know hip-hop, and some are aficionados, but their tastes center more on indie rock, electronica, and experimental music. Music education students mostly inhabit the self-contained classical world, or the similarly insular subculture of musical theater.

The ESF working style, in or out of the studio, is low-key, social, casual, and, at times, indistinguishable from simply hanging out. This is well in keeping with the broader norms of hip-hop. For all its apparent lack of focus, this ad-hoc working style is richly generative of original music. After extended socializing, the Fellows tend to make their creative choices quickly and decisively, and for the most part are confident and relaxed performers. The same is broadly true of other hip-hop musicians I have worked with.

While the music emerges seamlessly out of playful fraternizing, this is not to say that it is always effortless. The Fellows are not all expert musicians, and they sometimes show dissatisfaction or frustration with their music. Also, they vary in their willingness to share their ideas, especially the unfinished or insufficiently polished ones. That said, I can not recall seeing anyone in ESF display anxiety. This is a conspicuous difference from NYU’s music students, for whom anxiety is a dominant emotion in their creative spaces, especially the recording studio. During one session I led for some of my NYU undergraduate students, one woman came close to a panic attack from simply sitting in the control room listening to her peers recording. Classical music students face continual and strict scrutiny, and the studio represents the harshest scrutiny of all—an error that might go unnoticed in a live performance is painfully obvious on a recording.

Due to family obligations, I am not able to be a regular participant in ESF. When I can attend sessions, I teach audio engineering, work with the Fellows on mixing and editing their tracks, give creative feedback, or most commonly, make myself available and see what happens. Today it will be the latter. I arrive at 2 pm, the session’s scheduled start time. Jamie is there, as is another mentor, Amber, an NYU music education student. There are only two Fellows present, Juan and Marcus, and no one is making any music yet. Most of the Fellows will arrive late, and while the session is supposed to end at 6, Jamie tells me that “they’ll still be kicking it at 6:30 or 7:00… You can’t fight their body clock.”

Juan and Marcus join me at the table where I am sitting with my computer. They talk about the new Kendrick Lamar album and other recent developments in the rap world. Then Juan mentions that he is presently homeless due to a fight with his mother. (He is not the only homeless ESF participant.) There was apparently some police involvement, and a restraining order was issued. As a result, Juan missed a performance, so now on top of everything else, he will not be able to get booked at the venue again. He tells us this with the same wry detachment he used to talk about the new Kendrick. Either this happens to him routinely, or he is putting a brave face on a very bad situation, or both. The subject changes to whether a mutual friend is gay. Then Juan sings something, and Marcus asks, “You know the guy who sings that song?” Juan replies, “Who, Chris Brown?” Marcus says, “Yeah, you should let him sing it.” This is just friendly trash talk; Juan sings beautifully.

Three more Fellows drift in at 3:00 and gather in a far corner of the room. They plug a laptop into the PA and play a beat they’re working on. It is a four bar loop, endlessly repeating, with jazzy major seventh chords on piano over a drum machine. The three guys let it run while they shoot the breeze. As other Fellows arrive, they make a point of greeting me, shaking my hand firmly or fist bumping me, whether they have met me before or not. They look at their phones, noodle on the piano, and talk. It appears that nothing whatsoever is happening here, but I know from experience that it is all part of the process. After spending 45 minutes just letting their loop run, the group in the corner begins scrolling through different drum sounds. Then they quickly lay down a synth bassline on the MIDI controller. A notebook is produced, and songwriting begins in earnest.

Jamie and I are the only white people present. She and Amber continue to hang out, since the Fellows presently do not need any guidance. Amber complains about NYU’s music curriculum, that she is forced to study serialist composition. “I take all these music classes and only one involves me writing songs.” Jamie responds, “I got a whole music degree here and have never written a song.” She is committed to making expression the center of ESF; she wants everyone to write songs, to manifest themselves as creative and empowered beings. Kigan, another mentor, listens to us critique the Eurocentrism of the music academy, and is appalled to learn that universities did not begin to consider jazz an acceptable subject of study until decades after its peak cultural relevance. Kigan says that trap music now is what jazz was in the 1930s, that it’s where all the creativity is happening. He is not even referring to rap when he says this; he means the instrumental component of the music. He recommends a producer named Flosstradamus to me, and I make a note to look him up on SoundCloud later.

At 4:30, there is another beat looping on the speaker system. This one is in a minor key, with a mysterious vocal sample that sounds like aliens chanting. The beat is trap style, an extremely slow tempo with hi-hats stuttering in doubletime. Juan begins freestyling effortlessly over it. Another Fellow plays a line on the upright piano. Amber begins writing out a song structure on the whiteboard. Kigan and Jamie eat pizza and continue chatting. The energy in the room has picked up undeniably, even if it still seems unfocused.

Jamie and I talk about a grant proposal she is working on. She tries to articulate the value of what is happening here. “Saturdays are not the program. The space is not the program. The interactions are the program.” She wants to give ESF a sense of “accountability,” though she knows that this goal will run up against the chaotic reality of the Fellows’ lives. Rather than imposing some kind of discipline, she wants to foster intrinsic motivation from the sense of community: “Oh man, I saw on Facebook Live that you guys had a great session.” She ponders doing a “reboot” after Labor Day. Until then, the periodic recording sessions in the Dolan studio will continue to be natural anchor points. Jamie has also been bringing the Fellows to hackathons at Spotify and Splice–she wants them to imagine themselves someday working at those kinds of companies.

Alex Ruthmann, the director of the Music Experience Design Lab, is on the Steinhardt music education faculty, and has already started thinking of ways to integrate ESF with the official curriculum. The worlds of ESF and NYU have much to offer each other. NYU has its facilities and equipment, its expert faculty, its glamorous central location, and the accumulated expertise of all those well-trained musicians and composers and engineers. ESF has none of the material wealth or the privilege. But the Fellows are part of hip-hop, the single most important driver of America’s musical culture. A recent study conducted by Spotify concluded that hip-hop is the most-listened to genre of music on their service, not just in the United States, but everywhere in the world. It is astonishing to me that our country’s most marginalized young people are producing its most valued music. I hope that the academy learns to value their ideas as much as mass culture does.

Designing a more welcoming aQWERTYon experience

This post documents my final project for User Experience Design with June Ahn

The best aQWERTYon screencap

Overview of the problem

The aQWERTYon is a web-based music performance and theory learning interface designed by the NYU Music Experience Design Lab. The name is a play on “QWERTY accordion.” The aQWERTYon invites novices to improvise and compose using a variety of scales and chords normally available only to advanced musicians. Notes map onto the computer keyboard such that the rows play scales and the columns play chords. The user can not play any wrong notes, which encourages free and playful exploration. The aQWERTYon has a variety of instrument sounds to choose from, and it can also act as a standard MIDI controller for digital audio workstations (DAWs) like GarageBand, Logic, and Ableton Live. As of this writing, there have been aQWERTYon 32,000 sessions.

One of our core design principles is to work within our users’ real-world technological limitations. We build tools in the browser so they will be platform-independent and accessible anywhere where there is internet access. Our aim with the aQWERTYon was to find the musical possibilities in a typical computer with no additional software or hardware. That question led us to investigate ways of turning the standard QWERTY keyboard into a beginner-friendly instrument.

While the aQWERTYon has been an effective tool in classrooms and online, it has some design deficiencies as well. It is difficult for unassisted users to figure out what the app is for. While its functionality is easily discovered through trial and error, its musical applications are less self-explanatory. Some of this is due to the intrinsic complexity of music theory and all the daunting terminology that comes with it. But some of it is the lack of context and guidance we provide to new users.

The conjecture

This assignment coincided with discussions already taking place in the lab around redesigning the aQW. Many of those focused on a particular element of the user interface, the scale picker.

aQWERTYon scale picker

The user has a variety of scales to choose from, ranging from the familiar to the exotic. However, these scales all have impenetrable names. How are music theory novices supposed to make sense of names like harmonic minor or Lydian mode? How would they know to choose one scale or another? We debated the least off-putting way of presenting these choices: should we represent them graphically? Associate each one with a well-known piece of music? Or just list them alphabetically? I proposed a system of graphical icons showing the notes comprising each scale. While novices will find them no more intelligible than the names, the hope is that they would be sufficiently visually appealing to invite users to explore them by ear.

aQW scale picker interactive wireframe

Conversations with June helped me understand that there are some broader and more profound user experience problems to solve before users ever get to the scale picker. What is the experience of simply landing on the app for the first time? How do people know what to do? From this conversation came the germ of a new idea, a landing page offering a tutorial or introduction. We want users to have a feeling of discovery, a musical “aha moment”, the chance to be a musical insider. The best way to do that seemed to be to give users a playlist of preset songs to jam with.

User characteristics and personas

There are three major user groups for the aQWERTYon, who I will describe as students, teachers, and explorers.

Students and teachers

Students use the aQW in a guided and structured setting: a classroom, a private lesson, or an online tutorial. There are several distinct user personas: elementary, middle and high school students, both mainstream and with special needs; college students; and online learners, mostly adults. Each student persona has its corresponding teacher persona. For example, I use the aQW with my music technology students at Montclair State University and NYU, and with some private students.

The aQW’s biggest fan is MusEDLab partner Matt McLean, who teaches at the Little Red Schoolhouse and runs a nonprofit organization called the Young Composers and Improvisors Workshop. Matt uses the aQW to teach composition in both settings, in person and online. He has documented his students’ use of the aQW extensively. Some examples:

Explorers

I use the term explorers to describe people who use the aQW without any outside guidance. Explorers do not fit into specific demographic groups, but they center around two broad, overlapping personas: bedroom producers and music theory autodidacts. Explorers may find the aQW via a link, a social media posting, or a Google search. We know little about these users beyond what is captured by Google Analytics. However, we can make some assumptions based on our known referral sources. For example, this blog is a significant driver of traffic to the aQW. I have numerous posts on music theory and composition that link to the aQW so that readers can explore the concepts for themselves. My blog readership includes other music educators and some professional musicians, but the majority are amateur musicians and very enthusiastic listeners. These are exactly the users we are trying to serve: people who want to learn about music independently, either for creative purposes or to simply satisfy curiosity.

While I am a music educator, I have spent most of my life as a self-taught bedroom producer, so I identify naturally with the explorers. I have created several original pieces of music with the aQW, both for user testing purposes and to show its creative potential. While I have an extensive music theory background, I am a rudimentary keyboard player at best. This has limited my electronic music creation to drawing in the MIDI piano roll with the mouse pointer, since I can not perform my ideas on a piano-style controller. The aQW suits my needs perfectly, since I can set it to any scale I want and shred fearlessly. Here is an unedited improvisation I performed using a synthesizer instrument I created in Ableton Live:

My hope is that more would-be explorers feel invited to use the aQW for similar creative purposes in their own performance and composition.

Tasks and Scenarios

It is possible to configure the aQWERTYon via URL parameters to set the key and scale, and to hide components of the user interface. When teachers create exercises or assignments, they can link or embed the aQW with its settings locked to keep students from getting lost or confused. However, this does not necessarily invite the user to explore or experiment. Here is an example of an aQW preset to accompany a Beyoncé song. This preset might be used for a variety of pedagogical tasks, including learning some or all of the melody, creating a new countermelody, or improvising a solo. The harmonic major scale is not one that is usually taught, but it a useful way to blend major and minor tonalities. Students might try using more standard scales like major or harmonic minor, and listen for ways that they clash with Beyoncé’s song.

Tasks and scenarios for explorers might include creating a melody, bassline or chords for an original piece of music. For example, a self-taught dance music producer might feel limited by the scales that are easiest to play on a piano-style keyboard (major, natural minor, pentatonics) and be in search of richer and more exotic sounds. This producer might play their track in progress and improvise on top using different scale settings.

One of the users I tested with suggested an alternative explorer use case. He is an enthusiastic amateur composer and arranger, who is trying to arrange choral versions of pop and rock songs. He is a guitarist who has little formal music theory knowledge. He might use the aQW to try out harmonic ideas by ear, write down note names that form pleasing combinations, and then transfer them to the guitar or piano-based MIDI controller.

Understanding the problem

In the age of the computer and the internet, many aspects of music performance, composition and production are easy to self-teach. However, music theory remains an obstacle for many bedroom producers and pop musicians (not to mention schooled musicians!) There are so many chords and scales and rules and technical vocabulary, all of which have to be applied in all twelve keys. To make matters worse, terminology hangs around long after its historical context has disappeared. We no longer know what the Greek modes sound like, but we use their names to describe modern scales. C-sharp and D-flat were different pitches in historical tuning systems, but now both names describe the same pitch. The harmonic and melodic minor scales are named after a stylistic rule for writing melodies that was abandoned hundreds of years ago. And so on.

Most existing theory resources draw on the Western classical tradition, using examples and conventions from a repertoire most contemporary musicians and listeners find unfamiliar. Furthermore, these resources presume the ability to read standard music notation. Web resources that do address popular music are usually confusing and riddled with errors. I have worked with Soundfly to fill this vacuum by creating high-quality online courses aimed at popular musicians. Even with the best teaching resources, though, theory remains daunting. Exploring different chords and scales on an instrument requires significant technical mastery, and many musicians give up before ever reaching that point.

The aQW is intended to ease music theory learning by making scales and chords easy to discover even by complete novices. Our expectation is that after explorers are able to try theory ideas out in a low-pressure and creative setting, they will be motivated to put them to work playing instruments, composing or producing. Alternatively, users can simply perform and compose directly with the aQW itself.

Social and technical context

Most computer-based melody input systems are modeled on the piano. This is most obvious for hardware, since nearly all MIDI controllers take the form of literal piano keyboards. It is also true for software, which takes the piano keyboard as the primary visualization scheme for pitch. For example, the MIDI editor in every DAW displays pitches on a “piano roll”.

Some DAWs include a “musical typing” feature that maps the piano layout to the QWERTY keyboard, as an expediency for users who either lack MIDI hardware controllers, or who do not have them on hand. Apple’s GarageBand uses the ASDFG row of the keyboard for the white keys and the QWERTY row for the black keys. They use the other rows for such useful controls as pitch bend, modulation, sustain, octave shifting and simple velocity control.

GarageBand musical typing

Useful and expedient though it is, Musical Typing has some grave shortcomings as a user interface. It presumes familiarity with the piano keyboard, but is not very playable for users do who possess that familiarity. The piano layout makes a poor fit for the grid of computer keys. For example, there is no black key on the piano between the notes E and F, but the QWERTY keyboard gives no visual reminder of that fact, so it is necessary to just remember it. Unfortunately, the “missing” black key between E and F happens to be the letter R, which is GarageBand’s keyboard shortcut for recording. While hunting for E-flat or F-sharp, users are prone to accidentally start recording over their work. I have been using GarageBand for seven years and still do this routinely.

Ableton’s Push controller represents an interesting break with MIDI controller orthodoxy. It is a grid of 64 touch pads surrounded by various buttons, knobs and sliders.

Ableton Push

The pads were designed to trigger samples and loops like a typical drum machine, but Ableton also includes a melody mode for the Push. By default, it maps notes to the grid in rows staggered by fourths, which makes the layout identical to the bottom four strings of the guitar. This is quite a gift for guitarists like me, since I can use my familiar chord and scale fingerings, rather than hunting and pecking for them on the piano. Furthermore, the Push can be set so that the pads play only the notes within a particular scale, giving a “no wrong notes” experience similar to the aQWERTYon. Delightful though this mode is, however, it is imperfect. Root notes of the scale are colored blue, and other notes are colored white. While this makes the roots easy to distinguish, it is not so easy to visually differentiate the other pitches.

Touchscreen devices like the iPhone and iPad open up additional new possibilities for melodic interfaces. Many mobile apps continue to use the piano keyboard for note input, but some take advantage of the touchscreen’s unique affordances. One such is Thumbjam, which enables the user to divide the screen into slices of arbitrary thickness that can map to any arbitrary combination of notes.

Thumbjam

The app offers hundreds of preset scales to choose from. The user may have a small range of notes, each of which is large and easy to distinguish, or a huge range of notes, each of which occupies a narrow strip of screen area. Furthermore, the screen can be split to hold four different scales, played from four different instruments. While all of this configurability is liberating, it is also overwhelming. Also, the scales are one-dimensional lines; there is no easy way to play chords and arpeggios.

Evaluation criteria

Is the aQW’s potential obvious enough to draw in explorers and educators? Will it be adopted as a tool for self-teaching? Does it invite playful exploration and experimentation? Is it satisfying for real-world musical usage? Is the UI self-explanatory, or at least discoverable? Is the music theory content discoverable? Have we identified the right user persona(s)? Is the aQW really a tool for beginners? Or is it an intermediate music theory learning tool? Or an advanced composition tool? Is the approach of a “playlist” of example songs the right one? Which songs, artists and genres should we include on the landing page? How many presets should we include? Should we limit it to a few, or should we offer a large, searchable database? And how do we deal with the fact that many songs require multiple scales to play?

Proposed solution

I tested several interactive wireframes of this landing page concept. Click the image to try it yourself:

aQWERTYon landing page interactive wireframe

The first wireframe had nine preset songs. I wanted to offer reasonable musical diversity without overwhelming the user. The tenth slot linked to the “classic” aQW, where users are free to select their own video, scale, root, and so on. I chose songs that appealed to me (and presumably other adult explorers), along with some current pop songs familiar to younger users. I wanted to balance the choices by race, gender, era, and genre. I was also bound by a musical constraint: all songs need to be playable using a single scale in a single key. The initial preset list was:

  • Adele – “Send My Love (To Your New Lover)”
  • Mary J Blige – “Family Affair”
  • Miles Davis – “Sssh/Peaceful”
  • Missy Elliott – “Get Ur Freak On”
  • Björk – “All Is Full Of Love”
  • Michael Jackson – “Don’t Stop ’Til You Get Enough”
  • Katy Perry – “Teenage Dream”
  • AC/DC – “Back In Black”
  • Daft Punk – “Get Lucky”

After a few test sessions, it became apparent that no one was clicking Mary J Blige. Also, the list did not include any current hip-hop. I therefore replaced her with Chance The Rapper. I initially offered a few sentences of instruction, but feedback from my MusEDLab colleagues encouraged me to reduce the prompt down to just a few words: “Pick a song, type, jam.”

Further testing showed that while adults are willing to try out any song, familiar or not, children and teens are much choosier. Therefore, I added two more presets, “Hotline Bling” by Drake and “Formation” by Beyoncé. The latter song proved problematic, however, because its instrumental backing is so sparse and minimal that it is difficult to hear how other notes might fit into it. I ultimately swapped it for “Single Ladies.” I had rejected this song initially, because it uses the idiosyncratic harmonic major scale. However, I came to see this quirk as a positive bonus–since one of our goals is to encourage users to explore new sounds and concepts, a well-known and well-loved song using an unusual scale is a rare gift.

User testing protocol

I used a think-aloud protocol, asking testers to narrate their thought processes as they explored the app. I recorded the one-on-one sessions using Screenflow. When testing with groups of kids, this was impractical, so instead I took notes during and after each session. For each user, I opened the interactive wireframe, and told them, “This is a web based application for playing music with your computer keyboard. I’m going to ask you to tell me what you see on the screen, what you think it does, and what you think will happen when you click things.” I did not offer any other explanation or context, because I wanted to see whether the landing page was self-explanatory and discoverable. I conducted informal interviews with users during and after the sessions as well.

User testing results

I tested with ten adults and around forty kids. The adults ranged in age from early twenties to fifties. All were musicians, at varying levels of ability and training, mostly enthusiastic amateurs. Sessions lasted for twenty or thirty minutes. There were two groups of kids: a small group of eighth graders at the Little Red Schoolhouse, and a large group of fourth graders from PS 3 who were visiting NYU. These testing sessions were shorter, ten to fifteen minutes each.

User testing the aQWERTYon with fourth graders

Discovering melodies

It is possible to play the aQW by clicking the notes onscreen using the mouse, though this method is slow and difficult. Nevertheless, a number of the younger testers did this, even after I suggested that it would be easier on the keyboard.

An adult tester with some keyboard and guitar experience told me, “This is great, it’s making me play patterns that I normally don’t play.” He was playing on top of the Miles Davis track, and he was quickly able to figure out a few riffs from Miles’ trumpet solo.

Discovering chords

Several testers systematically identified chords by playing alternating notes within a row, while others discovered them by holding down random groups of keys. None of the testers discovered that they could easily play chords using columns of keys until I prompted them to do so. One even asked, “Is there a relationship between keys if I play them vertically? I don’t know enough about music to know that.” After I suggested he try the columns, he said, “If I didn’t know [by ear] how chords worked, I’d miss the beauty of this.” He compared the aQW to GarageBand’s musical typing: “This is not that. This is a whole new thing. This is chord oriented. As a guitarist, I appreciate that.” The message is clear: we need to make the chords more obvious, or more actively assist users in finding them.

Other theory issues

For the most part, testers were content to play the scales they were given, though some of the more expert musicians changed the scales before even listening to the presets. However, not everyone realized that the presets were set to match the song. A few asked me: “How do I know what key this song is in?” We could probably state explicitly that the presets line up automatically.

In general, adult testers found the value of the aQW as a theory learning tool to be immediately apparent. One told me: “If I had this when I was a kid, I would have studied music a lot. I used to hate music theory. I learned a lot of stuff, but the learning process was awful… Your kids’ generation will learn music like this (snaps fingers).”

Sounds

The aQW comes with a large collection of SoundFonts, and users of all ages enjoyed auditioning them, sometimes for long periods of time. Sometimes they apologized for how fascinating they found the sounds to be. But it is remarkable to have access to so many instrument timbres so effortlessly. Computers turn us all into potential orchestrators, arrangers, and sound designers.

Screen layout

The more design-oriented testers appreciated the sparseness and minimalism of the graphics, finding them calming and easy to understand.

Several testers complained that the video window takes up too much screen real estate, and is placed too prominently. Two commented that videos showing live performers, like “Back In Back,” were valuable because that helped with timekeeping and inspiration. Otherwise, however, testers found the videos to either be of little value or actively distracting. One suggested having the videos hidden or minimized by default, with the option to click to expand them. Others requested that the video be below the keyboard and other crucial controls. Also, the eighth graders reported that some of the video content was distracting because of its content, for example the partying shown in “Teenage Dream.” Unsuitable content will be an ongoing issue using many of the pop songs that kids like.

Technical browser issues

Having the aQWERTYon run in the browser has significant benefits, but a few limitations as well. Because the URL updates every time the parameters change, clicking the browser’s Back button does not produce the expected behavior–it might take ten or fifteen clicks to actually return to a previous page. I changed the links in later versions so each one opens the aQW in a new tab so the landing page would always be available. However, web audio is very memory-intensive, and the aQW will function slowly or not at all if it is open in more than one tab simultaneously.

Song choices

The best mix of presets is always going to depend on the specific demographics of any given group of users. However, the assortment I arrived at was satisfying enough for the groups I tested with. Miles Davis and Björk do not have the wide appeal of Daft Punk or Michael Jackson, but their presence was very gratifying for the more hipster-ish testers. I was extremely impressed that an eighth grader selected the Miles song, though this kid turns out to be the son of a Very Famous Musician and is not typical.

Recording functionality

Testers repeatedly requested the ability to record their playing. The aQW did start out with a very primitive recording feature, but it will require some development to make it usable. The question is always, how much functionality is enough? Should users be able to overdub? If so, how many tracks? Is simple recording enough, or would users need to able to mix, edit, and select takes?

One reason that recording has been a low development priority is that users can easily record their performances via MIDI into any DAW or notation program. The aQW behaves as if it were a standard MIDI controller plugged into the computer. With so many excellent DAWs in the world, it seems less urgent for us to replicate their functionality. However, there is one major limitation of recording this way: it captures the notes being played, but not the sounds. Instead, the DAW plays back the MIDI using whatever software instruments it has available. Users who are attached to a specific SoundFont cannot record them unless they use a workaround like Soundflower. This issue will require more discussion and design work.

New conjectures and future work

One of my most significant user testers for the landing page wireframe was Kevin Irlen, the MusEDLab’s chief software architect and main developer of the aQW itself. He found the landing page concept sufficiently inspiring that he created a more sophisticated version of it, the app sequencer:

aQWERTYon app sequencer v1

We can add presets to the app sequencer using a simple web form, which is a significant improvement over the tedious process of creating my wireframes by hand. The sequencer pulls images automatically from YouTube, another major labor-saver. Kevin also added a comment field, which gives additional opportunity to give prompts and instructions. Each sequencer preset generates a unique URL, making it possible to generate any number of different landing pages. We will be able to create custom landing pages focusing on different artists, genres or themes.

Songs beyond the presets

Testing with the fourth graders showed that we will need to design a better system for users who want to play over songs that we do not include among the presets. That tutorial needs to instruct users how to locate YouTube URLs, and more dauntingly, how to identify keys and scales. I propose an overlay or popup:

Keyfinding

Testing with fourth graders also showed that helping novice users with keyfinding may not be as challenging as I had feared. The aQW defaults to the D minor pentatonic scale, and that scale turns out to fit fairly well over most current pop songs. If it doesn’t, some other minor pentatonic scale is very likely to work. This is due to a music-theoretical quirk of the pentatonic scale: it happens to share pitches with many other commonly-used scales and chords. As long as the root is somewhere within the key, the minor pentatonic will sound fine. For example, in C major:

  • C minor pentatonic sounds like C blues
  • D minor pentatonic sounds like Csus4
  • E minor pentatonic sounds like Cmaj7
  • F minor pentatonic sounds like C natural minor
  • G minor pentatonic sounds like C7sus4
  • A minor pentatonic is the same as C major pentatonic
  • B minor pentatonic sounds like C Lydian mode

 

We are planning to revamp the root picker to show both a larger piano keyboard and a pitch wheel. We also plan to add more dynamic visualization options for notes as they are played, including a staff notation view, the chromatic circle, and the circle of fifths. The aQW leaves several keys on the keyboard unused, and we could use them for additional controls. For example, we might use the Control key to make note velocities louder, and Option to make them quieter. The arrow keys might be used to cycle through the scale menu and to shift the root.

Built-in theory pedagogy

There is a great deal of opportunity to build more theory pedagogy on top of the aQW, and to include more of it within the app itself. We might encourage chord playing by automatically showing chord labels at the top of each column. We might include popups or links next to each scale giving some explanation of why they sound the way they do, and to give some suggested musical uses. One user proposes a game mode for more advanced users, where the scale is set to chromatic and players must identify the “wrong” or outside notes. Another proposes a mode similar to Hooktheory, where users could sequence chord progressions to play on top of.

Rhythmic assistance

A few testers requested some kind of help or guidance with timekeeping. One suggested a graphical score in the style of Guitar Hero, or a “follow the bouncing ball” rhythm visualization. Another pointed out that an obvious solution would be to incorporate the Groove Pizza, perhaps in miniature form in a corner of the screen. Synchronizing all of this to YouTube videos would need to be done by hand, so far as I know, but perhaps an automated solution exists. Beat detection is certainly an easier MIR challenge than key or chord detection. If we were able to automatically sync to the tempo of a song, we could add the DJ functionality requested by one tester, letting users add cue points, loop certain sections, and slow them down.

Odds and ends

One eighth grader suggested that we make aQW accounts with “musical passwords.”

An adult tester referred to the landing page as the “Choose Your Own Adventure screen.” The idea of musical adventure is exactly the feeling I was hoping for.

In addition to notes on the staff, one tester requested a spectrum visualizer. This is perhaps an esoteric request, but real-time spectrograms are quite intuitive and might be useful.

Finally, one tester made a comment that was striking in its broader implications for music education: “I’m not very musical, I don’t really play an instrument, so these kinds of tricks are helpful for me. It didn’t take me long to figure out how the notes are arranged.” This person is a highly expert producer, beatmaker and live performer using Ableton Live. I asked how he came to this expertise, and he said he felt compelled to learn it to compensate for his lack of “musicianship”. It makes me sad that such a sophisticated musician does not realize that his skills “count”. In empowering music learners with the aQW, I also hope we are able to help computer musicians value themselves.

Research proposal – Hip-Hop Pedagogy

Final paper for Principles of Empirical Research with Catherine Voulgarides

Research questions

Jamie Ehrenfeld is a colleague of mine in the NYU Music Experience Design Lab. She graduated from NYU’s music education program, and now teaches music at Eagle Academy in Brownsville. Like many members of the lab, she straddles musical worlds, bringing her training in classical voice to her work mentoring rappers and R&B singers. We often talk about our own music learning experiences. In one such discussion, Jamie remarked: “I got a music degree without ever writing a song” (personal communication, April 29 2017). Across her secondary and undergraduate training, she had no opportunity to engage with the creative processes behind popular music. Her experience is hardly unusual. There is a wide and growing divide behind the culture of school music and the culture of music generally. Music educators are steeped in the habitus of classical music, at a time when our culture is increasingly defined by the music of the African diaspora: hip-hop, R&B, electronic dance music, and rock. 

The music academy’s near-exclusive focus on Western classical tradition places it strikingly at odds with the world that our students inhabit. In this paper, I examine the ideological basis for this divide. Why does the music academy generally and the training of music educators in particular hold so closely to the traditions of Western European classical music? Why has the music academy been slow to embrace African diasporic vernacular musics? Why does it outspokenly reject hip-hop? What racial and class forces drive the divide between music educators and the culture of their students? How might we make music education more culturally responsive? How can music educators support students in developing their own musical creativity via songwriting and beatmaking? What assumptions about musical and educational values must we challenge in order to do so?

Framing of research topic

Music education scholars commonly use “non-Western” as a shorthand for music outside the European classical tradition. This might lead one to naively believe that hip-hop is non-Western music. But it arose in the United States, so how can that be? Are our racial and ethnic minorities part of our civilization, or are they not? While the American cultural mainstream has increasingly embraced black musical styles, the music education field has not followed suit. As an example, consider a meme posted to a group for music teachers on Facebook. The meme’s original author is unknown. The caption was something like, “Typical middle school/high school student.” I will leave the person who posted it to Facebook anonymous, because they no doubt meant well.

You kids like the wrong music

The meme-maker is dismayed that young people do not care how little their music adheres to the stylistic norms of the Western European classical tradition. The author dismisses contemporary popular music and can not imagine why anyone else might enjoy it. The condescending presumption is that young people do not “really” enjoy pop, that they are being tricked into it by marketing and image, and that they are too lazy and ignorant to make critical choices. The choice of the word “molester” is a remarkable one, with its connotation of sexual violence. Classically trained educators feel their culture to be under attack, with their own students leading the charge.

Eurocentrism in American music education

In examining educational practice, we must look for the “hidden curriculum” (Anyon, 1980), the ideological content that comes along with the ostensible curricular goals. For example, The Complete Musician by Steven Laitz (2015) is a widely used college-level theory text. (I used a similar book of Laitz’s to fulfill my own graduate music theory requirement.) The title asserts an all-encompassing scope, but the text only discusses Western classical harmony and counterpoint. Other elements of music, like rhythm or timbre, receive cursory treatment at most. African diasporic and non-Western musics are not mentioned. The hidden curriculum here is barely even hidden. Mcclary (2000) asks why the particular musical conventions that emerged in Europe during the eighteenth and nineteenth centuries appealed so much to musicians and audiences, what needs they satisfied, and what cultural functions they performed. We might ask, since those conventions no longer appeal to most musicians or audiences, whose needs are being satisfied by school music? What cultural functions is it performing?

America has embraced every black musical form from ragtime through trap. But while our laws and culture have become less overtly racist over time, the oppression of people of color continues, African-Americans especially. For example, while they are no more likely to use drugs than white people, black people are many times more likely to be incarcerated for it. A white applicant with a felony drug conviction is more likely to get a callback for an entry-level job than a black applicant with no criminal record at all (Pager, 2007). Our large cities are extraordinarily segregated, with black neighborhoods isolated and concentrated (Denton & Massey, 1993). Perhaps this isolation has contributed to the evolution of hip-hop and its radical break with European-descended musical practices. Perry (2004) argues that, while hip-hop is a hybrid music, it is nevertheless a fundamentally black one due to four central characteristics:

(1) its primary language is African American Vernacular English (AAVE); (2) it has a political location in society distinctly ascribed to black people, music, and cultural forms; (3) it is derived from black American oral culture; and (4) it is derived from black American musical traditions (Perry 2004, 10).

The white mainstream adores the music while showering the people who created it with contempt (Perry 2004, 27).

Black music versus white educators

If the popular mainstream is dominated by innovations in black music, the field of musical education is unified by its extraordinary whiteness, both demographically and musically. Prospective teachers tend to be white, and come from suburban, low-poverty areas (Doyle, 2014). There is corresponding disproportionality among participants in formal music classes and ensembles—privileged groups are overrepresented, while less-privileged groups are underrepresented. This is true for white students versus students of color, high-SES students versus low-SES students, native English speakers versus English language learners, students whose parents have more versus less education, and so on (Elpus & Abril, 2011). Some of the disparity is due to the fact that schools in less privileged communities are less likely to offer music in the first place. But the disparities hold true among schools that do offer music, and persist even when schools supply free instruments. Lack of access alone can not explain the overwhelming whiteness and privilege of most participants in school music.

A great deal of research shows enrollment in school music declining precipitously for the past few decades. Budget cuts alone can not explain this decline, since enrollment in other arts courses has not declined as much (Kratus, 2007). As America’s student population becomes less white, its Eurocentric music education culture is evidently becoming steadily less appealing. Finney (2007) attributes the gap between music educators and their students to differing musical codes. “Teachers tend to use elaborated codes derived from Western European ‘elite’ culture, whereas students use vernacular codes… Students and teachers are therefore in danger of standing on opposite sides of a musical and linguistic chasm with few holding the key to unlock the other’s code” (18). Williams (2011) points to large ensemble model of school music that was imported to the United States from the European conservatory tradition in the early twentieth century, and which has barely changed since. Music educators teach what they learned, and what they learned is likely to have been the conservatory-style large ensemble.

Is the solution to expand the canon of “acceptable” music to include more artists of color? A typical undergraduate music history curriculum now tacks Duke Ellington or Charlie Parker onto the end of the succession of white European composers. But the canon is a political entity, not just an aesthetic one. If we try to expand the canon to include a greater diversity of musics, we will fail to challenge the basic fact of its existence and its role in academic culture. “[T]he canon is an epistemology; it is a way of understanding the world that privileges certain aesthetic criteria and that organizes a narrative about the history and development of music around such criteria and based on that understanding of the world. In other worlds, the canon is an ideology more than a specific repertory” (Madrid 2017, 125). Diversity is of no help if we simply use it to perpetuate privilege and power inequalities. “What does it mean when the tools of a racist patriarchy are used to examine the fruits of that same patriarchy? It means that only the most narrow parameters of change are possible and allowable” (Lorde 1984, 110). Rather than making incremental changes to the canon, we must ask how we can re-orient the basic assumptions of music education, its mission, its values, and its goals.

Literature review

In this section, I examine the present state of music education scholarship addressing the racial and class dynamics of music education, as well as the rise of culturally responsive pedagogies, particularly surrounding hip-hop.

Who is school music for?

By excluding entire categories of music and musicianship from the official curriculum, music educators send powerful and lasting messages to students (and everyone else) about what our society values and what it does not (Bledsoe, 2015). I am living proof; my own experiences with school music left me bored and alienated, and I came to the conclusion that I was not a musician at all. It took me years of self-guided practice to disabuse myself of that notion. I have had endless conversations with non-classical musicians at every level about how they do not regard themselves as “real” or “legitimate” musicians, no matter how professionally or creatively accomplished they may be. Fortunately, school music is not the only vector for music education. Most popular musicians learn informally from peers or on their own, a method that has become easier thanks to the internet. Still, the stigma of “failure” is a heavy psychological burden to overcome.

School music is usually competitive. There is a competitive process to become part of an ensemble, and those ensembles compete intramurally in much the same way that sports teams do. Conservatories that produce professional musicians need to be competitive. But should we continue to model all school music on the conservatory? The similarity between school ensembles and sports teams should trouble us. Schools are not obligated to let everyone play varsity football, regardless of ability. However, we do believe that schools should teach everyone reading and math. Our efforts to support struggling readers and math learners may be inadequate or even counterproductive, but at least we try to meet all students’ needs, and we certainly do not exclude low performers from studying these subjects entirely.

Some music teachers appear to exhibit the attitude of a physician who complains that all the patients in the waiting room are sick! In other words, they prefer to work only with the talented, ‘musically healthy’ few, when it is those who are in the most need of intervention who deserve at least equal attention (Regelski 2009, 32).

What if we held music teachers to the standards of math teachers rather than football coaches? We might follow the model of physical education classes and public health initiatives, prioritizing lifetime wellness over the identification and training of elite athletes only (Dillon, 2007).

Music and identity

In traditional aesthetic approaches to the Eurocentric canon, the locus of musical expressivity and meaning of the music is embedded entirely within the music itself. Listeners’ subjective experiences are not considered to be significant; our job is to decipher the formal relationships that the composer has encoded into the score. By contrast, Elliott and Silverman (2015) argue that we should take an embodied approach to musical understanding, seeing music as an enactive process emerging from the performance and listeners’ experience of it in social/emotional context. In the embodied approach, we see music as a tool for listeners to make their own meaning, to build their identity, and to communicate and modulate their emotions, all by means of bodily and social lived experience (van der Schyff, Schiavio & Elliott, 2016). Music is “a device for ordering the self” (DeNora 2000, 73). The role of music in building individual and group identity and a sense of belonging is especially critical in adolescence, when its ability to release or control difficult emotions may be literally lifesaving (Campbell, Connell & Beegle, 2007).

Music can also be the organizing principle behind new cultures and subcultures, a locus for tribal self-identification. Turino (2016) proposes that participatory music cultures offer an alternative form of citizenship, with the potential to be fundamental to our sense of self and a cornerstone of our happiness.

Fostering creative expression

Ruthmann (2007) suggests that we teach music the way that English teachers teach writing: use creative prompts that encourage students to develop individual authentic voices capable of expressing their own ideas and thoughts. Like writing generally, songwriting is hardly an elite or specialized practice. All young children spontaneously make up songs, which can sometimes be strangely catchy. My son wrote his first song at age four without any prompting or assistance, inspired by an episode of Thomas The Tank Engine (Pomykala-Hein, 2017). For many young people, music is entirely comprised of songs (Kratus, 2016). But after elementary school, school music is more about “pieces” than songs, symptomatic of the broader gap between in-school and out-of-school music cultures.

While music therapists have long taught songwriting, it is a rare practice in school music curricula. Kratus advocates songwriting for its therapeutic benefits, and for its lifelong learning benefits as well. Few adults have the opportunity to play oboe in an orchestra, but anyone with a guitar or keyboard or smartphone can write and perform songs. Historically, the technology for writing English has been dramatically more accessible than the technology for writing music, but that is changing rapidly. The software and hardware for recording, producing and composing music becomes cheaper and more user-friendly with each passing year. The instrumental backing track for “Pride” by Kendrick Lamar (2017) was produced by the eighteen-year-old Steve Lacy entirely on his iPhone. What are the other creative possibilities inherent in the devices students carry in their pockets and backpacks?

The psychological benefits of songwriting extend beyond musical learning. Like other art media, songwriting is an opportunity to practice what Sennett (2008) calls “craftsmanship,” defined as “the desire to do a job well for its own sake.” Craftsmanship is a habit of mind that “serves the computer programmer, the doctor, and the artist; parenting improves when it is practiced as a skilled craft, as does citizenship” (Sennett 2008, 9). Musical performers exercise craftsmanship as well, but not along as many different dimensions as songwriters and producers do.

Music creation is also a potential site of ethical development. We treat our favorite songs as imaginary people who we feel loving toward and protective of. This kind of idealization is akin to what we do “when we constitute others as persons, or when we invest others with personhood” (Elliott & Silverman 2015, 190). We imagine a personhood for the music, and we try to make that personhood real. In so doing, we learn how to create personhood for each other, and for ourselves. The point of musical education should not just be training in music, but developing ethical people through music (Bowman 2007, 2016). We can consider musical sensitivity to be a particular form of emotional sensitivity, and musical intelligence to be a particular application of emotional intelligence. Musical problem solving is an excellent simulator for social problem solving generally. Both in music and in life, the challenges are ambiguous, contingent, and loaded with irreconcilable contradiction. Performance and interpretation entail some musical problem-solving, but in the classical ensemble model that is typically the purview of the conductor. Songwriting poses musical problem-solving challenges to all who attempt it.

Hip-hop pedagogies

Brian Eno (2004) observes that the recording studio is a creative medium unto itself, one with different requirements for musicality from composition or performance. Indeed, no “composing” or “performing” need ever take place in modern studio practice. Eno is a case in point—while he has produced a string of famous and revered recordings, he does not consider himself to be adept at any instrument, and can not read or write notation. The digital studio has collapsed the distinction between musicians, composers, and engineers (Bell, 2014). The word “producer” is a useful descriptor for creators working across such role boundaries. In the analog recording era, producers were figures like Quincy Jones, executive managers of a commercial process. However, the term “producer” has come to describe anyone creating recorded music in any capacity, including songwriting, beatmaking, MIDI sequencing, and audio manipulation. We might expand the word further to include anyone who actively creates music, be it recorded, notated or live. To be a producer is a category of behavior, not a category of person.

Contemporary popular music is produced more than it is performed. This is nowhere more true than in the case of hip-hop, which in its instrumental aspect is almost entirely “postperformance” (Thibeault, 2010). The processes of producers like J Dilla and Kanye West resemble those of Brian Eno far more than those of Quincy Jones. This dramatic break with traditional musical practice poses major challenges for educators trained in the classical idiom, but it also presents new opportunities for culturally relevant and critically engaged pedagogy. Hip-hop-based education is mostly discussed in the urban classroom context, aimed toward “at-risk” youth (Irby & Hall, 2011). However, as hip-hop has expanded from its black urban origins to define the rest of mainstream musical culture, so too can it move into the educational mainstream as well.

There are several ways to incorporate hip-hop into education. Pedagogies with hip-hop connect hip-hop cultures and school experiences, using hip-hop as a bridge. Pedagogies about hip-hop engage teachers and students with critical perspectives on issues within the music and its culture, using hip-hop as a lens. Pedagogies of hip-hop apply hip-hop worldviews and practices within education settings (Kruse, 2016). Music educators can use hip-hop to enhance cultural relevance and connect to the large and growing percentage of students who identify as part of hip-hop culture. However, it is the use of hip-hop practices that most interests me as a research direction.

We should avoid using hip-hop as bait to get kids interested in “legitimate” music. Instead, we can apply the hip-hop ethos of authentic, culturally engaged expression to music education generally. Kratus (2007) points out that large ensembles are some of the last remaining school settings where the teaching model maintains a top-down autocratic structure, untouched by the cognitive revolution. This method does not create independently functioning musicians. How might we find ways for students to engage in music on their own cultural and technological terms? One method might be to do sampling and remixing of familiar music as an entry point into creation. This is the approach taken by Will Kuhn (personal communication, 2017), who teaches high school students to build songs entirely out of pieces of existing songs. Students can then replace those appropriated samples with material of their own.

Hip-hop has many controversial aspects, but none provokes the ire of legacy musicians more than the practice of sampling. There is a widespread perception that sampling is nothing more than a way to avoid learning instruments or hiring musicians. This may be true in some instances, but it is easy to identify examples of artists who went to considerable expense and trouble to license samples when they did not need to do so. For example, while Ahmir “Questlove” Thompson of the Roots is a highly regarded drummer, he still uses sampled breakbeats in his productions. Why would he prefer a sample to his own playing? In hip-hop, “[e]xisting recordings are not randomly or instrumentally incorporated so much as they become the simultaneous subject and object of a creative work” (Culter 2004, 154). Samples have specific timbral qualities that evoke specific memories and associations, situating the music in webs of intertextual reference.

Rice (2003) encourages non-music educators to draw on the practice of sampling. Students might approach cultural artifacts and texts the way that producers approach recorded music, looking for fragments that might be appropriated and repurposed to form the basis of new works.

The pedagogical sampler, with a computer or without a computer, allows cultural criticism to save isolated moments and then juxtapose them as a final product. The student writer looks at the various distinct moments she has collected and figures out how these moments together produce knowledge. Just as DJs often search for breaks and cuts in the music that reveal patterns, so, too, does the student writer look for a pattern as a way to unite these moments into a new alternative argument and critique (465).

Rice advocates what he calls the “whatever” principle of sampling. In the hip-hop context, “whatever” can have two meanings. First, there is the conventional sense of the word, that everything is on the table, that anything goes. There is also the slang sense of “whatever” as a statement of defiance, indifference, and dismissal. In a pedagogical context, the “whatever” principle encourages us to be accepting of what is new and unexpected, and be dismissive of what is fake or irrelevant. As Missy Elliott (2002) puts it: “Whatever, let’s just have fun. It’s hip-hop, man, this is hip-hop.”

I asked Jamie Ehrenfeld, if she had written songs while getting her music degree, what kind of material might she have written? She responded:

I would think of bits of music in my head and then associate them with some other song I’d already heard and felt like nothing I could think of was really original, and I didn’t get that it’s okay that in writing a song having some elements of other songs can come together to make something new, and that actually being original is more of what existing pieces you weave together in addition to ‘original’ thought (personal communication, April 28 2017).

In other words, the sampling ethos might have validated the intuitive creative processes she was already spontaneously carrying out, whether she had realized those impulses in the form of digitally produced recordings or pencil-and-paper scores.

Can a work based on samples be wholly original? Perhaps not. But hip-hop slang offers a different standard of quality that may be more apposite: the idea of freshness. There are several different definitions of “fresh.” It can mean new or different; well-rested, energetic, and healthy-looking; or appealing food, water, or air. “Fresh” is also a dated slang term for impudence or impertinence. In hip-hop culture, “fresh” is one among many synonyms for “cool,” but it could be referencing any of the various original senses of the word: new, refreshing, appetizing, attractive, or sassy. Rather than evaluating music in terms of its originality, we might judge music by its freshness (Hein, 2015). A track that includes samples can not be wholly original by definition, but it can be fresh. It is this sense of making new meaning out of existing resources that animates the Fresh Ed curriculum (Miles et al, 2015), a culturally responsive teaching resource created by the Urban Arts Partnership. Rather than treating students as receptacles for information, Fresh Ed places new knowledge in familiar contexts, for example in the form of rap songs. When students are able to draw on their prior knowledge and cultural competencies, they are better equipped to engage and think critically.

Proposed methods

Luker (2008) describes the case that chooses you, or that you sample yourself into (131). My own trajectory as a musician and educator has made me an exemplar of the shortcomings of Eurocentric music pedagogy and the benefits of personal creativity through producing and songwriting; certainly it feels like this case chose me. Since my own motivations are borne out of subjective experience, and since my research questions were provoked by the experiences of others like me, my research into those questions must necessarily follow an interpretivist paradigm. In choosing methods aligning to that paradigm, I want to identify one that supports the use of music creation itself as a tool for inquiry into music pedagogy. One such method is Eisner’s (1997) model of educational inquiry by means of connoisseurship and criticism. Connoisseurship is the “ability to make fine-grained discriminations among complex and subtle qualities” (Eisner 1997, 63). Criticism is judgment that illuminates and interprets the qualities of a practice in order to transform it. As a subjective researcher, I am obliged to systematically identify my subjectivity  (Peshkin, 1988), and I view my role as connoisseur and critic in music as a source of clarity rather than bias.

Ethnography

An interpretivist paradigm is well supported by methods of ethnography, since participant observation and unstructured interviews dovetail exactly with a subjectivist epistemology. Ethnographers typically allow their methods to evolve over the course of the study, and can only define their procedures in retrospect, in the form of a narrative of what actually happened, rather than a detailed plan ahead of time. This form of research is iterative, like agile software development. Data comes in the form of interpretations of interpretations of interpretations, and in that sense is a “fiction”—not in the sense that it is counterfactual (we hope), but in the original sense of the word, a thing that is constructed. We must involve our imagination in constructing our interpretive fictions (Geertz, 1973).

Institutional ethnographers examine work settings and processes, combining observation with discourse analysis of texts, documents and procedures. The goal is to show how people in the workplace align their activities with structures that may originate elsewhere (Devault, 2006). This method asks us to seek out “ruling relations” (Smith 2005, 11), textually mediated connections and organizations shaping everyday life, especially those that are the most taken for granted. In so doing, we examine the ways that texts bind small social groups into institutions, and bind those together into larger power structures. This method is well suited to a profession like music teaching.

Taber (2010) combines autoethnography with institutional ethnography to tell the story of her own experience in the military, as an entry point into understanding the experience of other women. She questions whether researching the lives of others was a way to hide from her own problematic experience, and chooses instead to foreground her internal conflicts, using a “reflexivity of discomfort” (19). This is emblematic of the institutional ethnographic practice of examining aspects of organizations that their inhabitants find problematic, troubling or contradictory. Since the story of my own music education is one of internal conflict and discomfort, I expect a similar method to Taber’s to yield rich results.

Naturally, an inquiry into music education will involve some ethnomusicology. Given how technologically mediated hip-hop and other contemporary forms are, it will be useful to take on the lens of “technomusicology” (Marshall, 2017). Music educators who feel pressured to use computers in their practice quickly run up against the fact that digital audio tools are a poor fit for classical music. However, these tools are the most natural medium for hip-hop and other electronic dance musics. The technological and cultural issues are inseparable.

Hip-hop grows out orality and African-American Vernacular English. Therefore, it is prone to being dismissed by scholars working in a literate value system. Similarly, it is all too common to view AAVE through the lens of deprivationism, as a failure to learn “correct” English. To overcome this spurious attitude, we can employ an ethnopoetic approach. Speakers of AAVE are only linguistically “impoverished” because we institutionally deem them to be so, not because they have any difficulty communicating or expressing themselves (McDermott & Varenne, 1995). By the same token, classical music culture sees the lack of complex harmony and melody in African diasporic music like hip-hop as a shortcoming, a poverty of musical means. But the hip-hop aesthetic puts a premium on rhythm and timbre, and harmony functions mostly as a way to signpost locations within the cyclical metrical structure. In learning to value hip-hop on its own terms, we broaden our ability to understand other musical and cultural value systems as well.

Participatory research

Participatory research methods like cooperative inquiry and participatory action research treat research participants as collaborators, rather than as objects of study. The related method of constructivist instructional design puts these principles into action in the form of new technologies, experiences and curricula, the educational equivalent of critical theorists’ activism. When teachers and designers act as researchers, they function as participant observers. While I am an avid hip-hop fan and a dedicated student of it, I am ultimately a tourist. My research will therefore necessarily be incomplete unless it is a collaborative effort with members of hip-hop culture.

Instructional design as participatory research follows a Reflective and Recursive Design and Development (R2D2) model, based on the principles of recursion, nonlinearity and reflection (Willis, 2007). Designers test and prototype continually alongside users, and feed the results back into the next design iteration. This process for developing instructional material enables end users and experts to work jointly toward the end product. This loop of feedback and iteration is an example of reflective practice, made up of the “arts” of problem framing, implementation, and improvisation (Schön, 1987). These same arts are the ones used in musical problem-solving, both as a practitioner and educator. The Music Experience Design Lab follows a participatory design methodology in developing our technologies for music learning and expression, and the idea of using the same techniques to examine the broader social context of our work is quite appealing to me.

Narrative inquiry

There may be universal physical truths, but mental, emotional and social truths are contextual and particular. To examine these truths, then, we need verstehen, understanding of context, both historical and contemporary (Willis, 2007). To that end, we can draw on phenomenology, asking how humans view themselves and the world around them. This perspective attends to experience “from the neck down,” not just to cognition. We need to understand the bodily sensations of numbness, anxiety or anger that too many students feel in the music classroom, knowing that something is wrong but not knowing how to name it. For example, I spent my music graduate theory seminar in a continual low boil of rage, and it was only years later that I was able to point to the white supremacist ideology animating the curriculum as the source of this intense emotion. A number of my fellow musicians aligned with black music have described the same feelings. It is a primary research goal of mine to give those feelings a name and a clear target, so they can be put to work in the service of systemic change.

Bruner (1991) cites Vygotsky’s dictum that cultural products like language mediate our thought and shape our representations of reality. (This is certainly true of music.) Constructionists assume that we produce reality through the social exchange of meanings. We use language not as isolated individuals, but within social groups, organizations, institutions and cultures. Within our contexts, we speak as we understand it to be appropriate to speak (Galasinski & Ziólkowska, 2013). As narratives accrue into traditions, they take on a life of their own that can outlive their original context—this is a likely explanation for the persistence of classical music habitus far beyond the conservatory.

Close readers of narrative must study not only the syntactic content of the words themselves, but also their literary qualities, their tone (Riessman, 2008). There is a close parallel here with musicology. When we compare Julie Andrews’ performance of “My Favorite Things” in The Sound Of Music (1965) with the one recorded by John Coltrane (1961), it is like comparing the same text spoken by two very different speakers. We can perform a neat inverse of this process by examining the same musical performance across contexts; for example, comparing Tom Scott’s recording of Balin and Kantner’s “Today” (1967) with the sample of that recording that forms the centerpiece of Pete Rock & C.L. Smooth’s “They Reminisce Over You (T.R.O.Y.)” (1992). Here, the same performance gives rise to different musical meanings in different settings. We should be similarly attentive to the performative and contextual aspects of narrative.

Validity and reliability

If we are examining attitudes and interpretations rather than more easily observable “facts,” how do we ensure validity and reliability? In place of a search for straightforward logical explanations, we can instead build a case on Lyotardian paralogy, and “let contradictions remain in tension” (Lather 1993, 679), like the unresolved tritones enriching the blues and jazz. We should not expect to find tree-shaped hierarchies of explanation, but instead hold ourselves to a “rhizomatic” standard of validity. “Rather than a linear progress, rhizomatics is a journey among intersections, nodes, and regionalizations through a multi-centered complexity” (Lather 1993, 680). We can understand the complexities of music and schooling and race to have the topology of a network, not a tree. We should expect that when we pull on any part of the network, we will encounter a tangle.

In my research thus far, I have instinctively used reciprocity to treat my interviews more as two-way conversations. Such judicious use of self-disclosure can give rise to richer data. We can attain further reciprocity by showing participants field notes and drafts, building in “member checks” early on to ensure trustworthiness throughout the process. As feminist researchers, Harrison, MacGibbon and Morton (2001) hold attention to emotional aspects of the research and the relationships it entails as a key criterion of trustworthiness. This kind of emotionally aware collaborative/shared authorship aligns naturally with participatory research, and with hip-hop pedagogy. Larson (1997) argues that narrative inquiry gains greater validity by having the story-giver reflect on the transcript and analysis so they can revise or go deeper into their story. If a lived experience is an iceberg, then its initial retelling may just describe the tip. It takes reflection to bring more of the iceberg to the surface. We may therefore do better to examine a few icebergs thoroughly than to survey many tips.

Sample data and future research

Ed Sullivan Fellows (ESF) is a mentorship and artist development program run by the NYU Steinhardt Music Experience Design Lab. Participants are young men and women between the ages of 15 and 20, mostly low-SES people of color. They meet on Saturday afternoons at NYU to write and record songs; to get mentorship on the music business, marketing and branding; and to socialize. Sessions have a clubhouse feel, a series of ad-hoc jam sessions, cyphers, informal talks, and open-ended creativity. Conversations are as likely to focus on participants’ emotions, politics, social life and identity as they are on anything pertaining to music. I intend to conduct my research among hip-hop educators like Jamie and the other ESF mentors. They teach music concepts like song structure and harmony, but their project is much larger: to provide emotional support, to build resilience and confidence, to foster social connections across class and racial lines. Hein (2017) is a set of preliminary observations on ESF, showing the close connection between its musical and social values.

Conclusion

If music education is failing to address the needs of the substantial majority of students, it should be no wonder that enrollment and societal support are declining.

Every ‘failure’ to succeed in competition, every drop-out, and every student who is relieved to have compulsory music study behind them (including lessons enforced by parental fiat) represents not just a lack of ‘conversion’ to musical ‘virtue’ but gives such future members of the public compelling reason to doubt whether their music education has served any lasting purpose or value (Regelski 2009, 12).

Music educators’ advocacy efforts are mostly devoted to preserving existing methods and policies. However, these same methods and practices are driving music education’s irrelevance. At some point, advocacy starts to look less like a high-minded push for society’s interest, and more like an effort on behalf of music teachers’ self-interest.

Most (if not all) people have an inborn capacity and intrinsic motivation for engaging in music. However, that capacity and motivation need to be activated and nurtured by “musically and educationally excellent teachers and… inspiring models of musicing in contexts of welcoming, sustaining, and educative musical settings, including home and community contexts” (Elliott & Silverman 2015, 240). To restrict this opportunity to “talented” students is anti-democratic in Dewey’s sense. Good music serves particular human needs. One of those needs is aesthetic contemplation and appreciation of the Eurocentric canon. But there are many other legitimate ends that music education can pursue. In order to meet more students’ musical needs, we must embrace the musical culture of the present, and confront all the challenges of race and class that entails.

References

Anyon, J. (1980). Social Class and the Hidden Curriculum of Work. The Journal of Education, 162(1), 67–92.

Bell, A. P. (2014). Trial-by-fire : A case study of the musician – engineer hybrid role in the home studio. Journal of Music, Technology & Education, 7(3), 295–312.

Bledsoe, R. (2015). Music Education for All? General Music Today, 28(2), 18–22.

Bowman, W. (2007). Who is the “We”? Rethinking Professionalism in Music Education. Action, Criticism, and Theory for Music Education, 6(4), 109–131.

Bowman, W. (2016). Artistry, Ethics, and Citizenship. In D. Elliott, M. Silverman, & W. Bowman (Eds.), Artistic Citizenship: Artistry, Social Responsibility, and Ethical Praxis. New York: Oxford University Press.

Campbell, P. S., Connell, C., & Beegle, A. (2007). Adolescents’ expressed meanings of music in and out of school. Journal of Research in Music Education, 55(3), 220–236.

Cutler, C. (2004). Plunderphonia. In C. Cox & D. Warner (Eds.), Audio culture: Readings in modern music (pp. 138–156). London: Continuum International Publishing Group.

DeNora, T. (2000). Music in everyday life. New York: Cambridge University Press.

Devault, M. L. (2006). Introduction: What is Institutional Ethnography? Social Problems, 53(3), 294–298.

Dillon, S. (2007). Music, Meaning and Transformation: Meaningful Music Making for Life. Cambridge Scholars Publishing.

Doyle, J. L. (2014). Cultural relevance in urban music education: a synthesis of the literature. Applications of Research in Music Education, 32(2), 44–51.

Eisner, E. (1991). The enlightened eye: Qualitative inquiry and the enhancement of educational practice. Toronto: Macmillan.

Elliott, D. J., & Silverman, M. (2014). Music Matters: A Philosophy of Music Education (2nd ed.). Oxford: Oxford University Press.

Elpus, K., & Abril, C. R. (2011). High School Music Ensemble Students in the United States: A Demographic Profile. Journal of Research in Music Education, 59(2), 128–145.

Eno, B. (2004). The Studio As Compositional Tool. In C. Cox & D. Warner (Eds.), Audio culture: Readings in modern music (pp. 127–130). London: Continuum International Publishing Group.

Ester, D. P., & Turner, K. (2009). The impact of a school loaner-instrument program on the attitudes and achievement of low-income music students. Contributions to Music Education, 36(1), 53–71.

Finney, J. (2007). Music Education as Identity Project in a World of Electronic Desires. In J. Finney & P. Burnard (Eds.), Music education with digital technology. London: Bloomsbury Academic.

Harrison, J., MacGibbon, L., & Morton, M. (2001). Regimes of Trustworthiness in Qualitative Research: The Rigors of Reciprocity. Qualitative Inquiry, 7(3), 323–345.

Hein, E. (2015). Mad Fresh. NewMusicBox. Retrieved March 24, 2015, from http://www.newmusicbox.org/articles/mad-fresh/

Hein, E. (2017). A participant ethnography of the Ed Sullivan Fellows program. Retrieved May 9, 2017, from http://www.ethanhein.com/wp/2017/a-participant-ethnography-of-the-ed-sullivan-fellows-program/

Irby, D. J., & Hall, H. B. (2011). Fresh Faces, New Places: Moving Beyond Teacher-Researcher Perspectives in Hip-Hop-Based Education Research. Urban Education, 46(2), 216–240.

Kratus, J. (2016). Songwriting: A new direction for secondary music education. Music Educators Journal, 102(3), 60–65.

Kratus, J. (2007). Music Education at the Tipping Point. Music Educators Journal, 94(2), 42–48.

Kruse, A. J. (2016). Toward hip-hop pedagogies for music education. International Journal of Music Education, 34(2), 247–260.

Laitz, S. G. (2015). The complete musician: An integrated approach to tonal theory, analysis, and listening (4th ed.). Oxford University Press.

Lather, P. (1993). Fertile Obsession : Validity after Poststructuralism. The Sociological Quarterly, 34(4), 673–693.

Lorde, A. (1984). The master’s tools will never dismantle the master’s house. Sister Outsider: Essays and Speeches by Audre Lorde, 110–113.

Luker, K. (2008). Salsa Dancing into the Social Sciences: Research in an Age of Info-glut. Cambridge: Harvard University Press.

Madrid, A. L. (2017). Diversity, Tokenism, Non-Canonical Musics, and the Crisis of the Humanities in U.S. Academia, 7(2), 124–129.

Marshall, W. (2017). Technomusicology | Harvard Extension School. Retrieved May 8, 2017, from http://www.extension.harvard.edu/academics/courses/technomusicology/24318

Massey, D. S., & Denton, N. A. (1993). American apartheid: segregation and the making of the underclass. Cambridge: Harvard University Press.

Mcclary, S. (2000). Conventional Wisdom: The Content of Musical Form. University of California Press.

McDermott, R., & Varenne, H. (1995). Culture as Disability. Anthropology and Education Quarterly, 26(3), 324–348.

Miles, J., Hogan, E., Boland, B., Ehrenfeld, J., & Berry, L. (2015). Fresh Ed: A Field Guide to Culturally Responsive Pedagogy. New York: Urban Arts Partnership. Retrieved from http://freshed.urbanarts.org/fresh-field-guide/

Perry, I. (2004). Prophets of the Hood. Duke University Press.

Peshkin, A. (1988). In Search of Subjectivity–One’ s Own. Educational Researcher, 17–21.

Pomykala-Hein, M. (2017). Searching [Online musical score]. Retrieved May 5, 2017, from https://www.noteflight.com/scores/view/180d4db69af3646e6e70fae8002648d7f2048a7d

Regelski, T. A. (2009). The Ethics of Music Teaching as Profession and Praxis. Visions of Research in Music Education, 13(2009), 1–34.

Rice, J. (2016). The 1963 hip-hop machine: Hip-hop pedagogy as composition. College Composition and Communication, 54(3), 453–471.

Ruthmann, A. (2007). The Composers’ Workshop: An Approach to Composing in the Classroom. Music Educators Journal, 93(4), 38.

Schön, D. (1987). Teaching artistry through reflection in action. In Educating the reflective practitioner: Educating the reflective practitioner for teaching and learning in the professions (pp. 22–40). San Francisco: Jossey-Bass.

Sennett, R. (2008). The Craftsman (New Haven). Yale University Press.

Smith, D. (2005). Institutional Ethnography: A Sociology for People. Walnut Creek, CA: AltaMira Press.

Taber, N. (2010). Institutional ethnography, autoethnography, and narrative: an argument for incorporating multiple methodologies. Qualitative Research, 10(1), 5–25. http://doi.org/10.1177/1468794109348680

Thibeault, M. (2010). Hip-Hop, Digital Media, and the Changing Face of Music Education. General Music Today, 24(1), 46–49. http://doi.org/10.1177/1048371310379097

Turino, T. (2016). Music, Social Change, and Alternative Forms of Citizenship. In D. Elliott, M. Silverman, & W. Bowman (Eds.), Artistic Citizenship: Artistry, Social Responsibility, and Ethical Praxis (p. 616). New York: Oxford University Press.

van der Schyff, D., Schiavio, A., & Elliott, D. J. (2016). Critical ontology for an enactive music pedagogy. Action, Criticism, and Theory for Music Education, 15(5), 81–121.

Williams, D. A. (2011). The Elephant in the Room. Music Education: Navigating the Future, 98(1), 51–57.

Willis, J. W. (2007). Foundations of Qualitative Research: Interpretive and Critical Approaches. Thousand Oaks, CA: Sage.

Wise, R. (1965). The Sound Of Music. United States: 20th Century Fox.

Discography

Balin, M. and Kantner, P. (1967). Today [recorded by Tom Scott and The California Dreamers]. On The Honeysuckle Breeze [LP]. Santa Monica: Impulse! (1967)

Elliott, Missy (2002). Work It. On Under Construction [CD]. New York: Goldmind/Elektra. (November 12, 2002)

Lamar, Kendrick (2017). Pride. On DAMN. [CD/streaming]. Santa Monica, CA: Top Dawg/Aftermath/Interscope. (April 14, 2017)

Pete Rock & C.L. Smooth (1992). They Reminisce Over You (T.R.O.Y.). On Mecca and the Soul Brother [LP]. New York: Untouchables/Elektra. (April 2, 1992)

Rodgers, Richard and Hammerstein, Oscar (1959). My Favorite Things [recorded by John Coltrane]. On My Favorite Things [LP]. New York: Atlantic. (March, 1961)

Learning music from Ableton

Ableton recently launched a delightful web site that teaches the basics of beatmaking, production and music theory using elegant interactives. If you’re interested in music education, creation, or user experience design, you owe it to yourself to try it out.

Ableton - Learning Music site

One of the site’s co-creators is Dennis DeSantis, who wrote Live’s unusually lucid documentation, and also their first book, a highly-recommended collection of strategies for music creation (not just in the electronic idiom.)

Dennis DeSantis - Making Music

The other co-creator is Jack Schaedler, who also created this totally gorgeous interactive digital signal theory primer.

If you’ve been following the work of the NYU Music Experience Design Lab, you might notice some strong similarities between Ableton’s site and our tools. That’s no coincidence. Dennis and I have been having an informal back and forth on the role of technology in music education for a few years now. It’s a relationship that’s going to get a step more formal this fall at the 2017 Loop Conference – more details on that as it develops.

Meanwhile, Peter Kirn’s review of the Learning Music site raises some probing questions about why Ableton might be getting involved in education in the first place. But first, he makes some broad statements about the state of the musical world that are worth repeating in full.

I think there’s a common myth that music production tools somehow take away from the need to understand music theory. I’d say exactly the opposite: they’re more demanding.

Every musician is now in the position of composer. You have an opportunity to arrange new sounds in new ways without any clear frame from the past. You’re now part of a community of listeners who have more access to traditions across geography and essentially from the dawn of time. In other words, there’s almost no choice too obvious.

The music education world has been slow to react to these new realities. We still think of composition as an elite and esoteric skill, one reserved only for small class of highly trained specialists. Before computers, this was a reasonable enough attitude to have, because it was mostly true. Not many of us can learn an instrument well enough to compose with it, then learn to notate our ideas. Even fewer of us will be able to find musicians to perform those compositions. But anyone with an iPhone and twenty dollars worth of apps can make original music using an infinite variety of sounds, and share that music online to anyone willing to listen. My kids started playing with iOS music apps when they were one year old. With the technical barriers to musical creativity falling away, the remaining challenge is gaining an understanding of music itself, how it works, why some things sound good and others don’t. This is the challenge that we as music educators are suddenly free to take up.

There’s an important question to ask here, though: why Ableton?

To me, the answer to this is self-evident. Ableton has been in the music education business since its founding. Like Adam Bell says, every piece of music creation software is a de facto education experience. Designers of DAWs might even be the most culturally impactful music educators of our time. Most popular music is made by self-taught producers, and a lot of that self-teaching consists of exploring DAWs like Ableton Live. The presets, factory sounds and affordances of your DAW powerfully inform your understanding of musical possibility. If DAW makers are going to be teaching the world’s producers, I’d prefer if they do it intentionally.

So far, there has been a divide between “serious” music making tools like Ableton Live and the toy-like iOS and web apps that my kids use. If you’re sufficiently motivated, you can integrate them all together, but it takes some skill. One of the most interesting features of Ableton’s web site, then, is that each interactive tool includes a link that will open up your little creation in a Live session. Peter Kirn shares my excitement about this feature.

There are plenty of interactive learning examples online, but I think that “export” feature – the ability to integrate with serious desktop features – represents a kind of breakthrough.

Ableton Live is a superb creation tool, but I’ve been hesitant to recommend it to beginner producers. The web site could change my mind about that.

So, this is all wonderful. But Kirn points out a dark side.

The richness of music knowledge is something we’ve received because of healthy music communities and music institutions, because of a network of overlapping ecosystems. And it’s important that many of these are independent. I think it’s great that software companies are getting into the action, and I hope they continue to do so. In fact, I think that’s one healthy part of the present ecosystem.

It’s the rest of the ecosystem that’s worrying – the one outside individual brands and what they support. Public music education is getting squeezed in different ways all around the world. Independent content production is, too, even in advertising-supported publications like this one, but more so in other spheres. Worse, I think education around music technology hasn’t even begun to be reconciled with traditional music education – in the sense that people with specialties in one field tend not to have any understanding of the other. And right now, we need both – and both are getting their resources squeezed.

This might feel like I’m going on a tangent, but if your DAW has to teach you how harmony works, it’s worth asking the question – did some other part of the system break down?

Yes it did! Sure, you can learn the fundamentals of rhythm, harmony, and form from any of a thousand schools, courses, or books. But there aren’t many places you can go to learn about it in the context of Beyoncé, Daft Punk, or A Tribe Called Quest. Not many educators are hip enough to include the Sleng Teng riddim as one of the fundamentals. I’m doing my best to rectify this imbalance–that’s what my courses with Soundfly classes are for. But I join Peter Kirn in wondering why it’s left to private companies to do this work. Why isn’t school music more culturally relevant? Why do so many educators insist that you kids like the wrong music? Why is it so common to get a music degree without ever writing a song? Why is the chasm between the culture of school music and music generally so wide?

Like Kirn, I’m distressed that school music programs are getting their budgets cut. But there’s a reason that’s happening, and it isn’t that politicians and school boards are philistines. Enrollment in school music is declining in places where the budgets aren’t being cut, and even where schools are offering free instruments. We need to look at the content of school music itself to see why it’s driving kids away. Both the content of school music programs and the people teaching them are whiter than the student population. Even white kids are likely to be alienated from a Eurocentric curriculum that doesn’t reflect America’s increasingly Afrocentric musical culture. The large ensemble model that we imported from European conservatories is incompatible with the riot of polyglot individualism in the kids’ earbuds.

While music therapists have been teaching songwriting for years, it’s rare to find it in school music curricula. Production and beatmaking are even more rare. Not many adults can play oboe in an orchestra, but anyone with a guitar or keyboard or smartphone can write and perform songs. Music performance is a wonderful experience, one I wish were available to everyone, but music creation is on another level of emotional meaning entirely. It’s like the difference between watching basketball on TV and playing it yourself. It’s a way to understand your own innermost experiences and the innermost experiences of others. It changes the way you listen to music, and the way you approach any kind of art for that matter. It’s a tool that anyone should be able to have in their kit. Ableton is doing the music education world an invaluable service; I hope more of us follow their example.

Design for Real Life – QWERTYBeats research

Writing assignment for Design For The Real World with Claire Kearney-Volpe and Diana Castro – research about a new rhythm interface for blind and low-vision novice musicians

Definition

I propose a new web-based accessible rhythm instrument called QWERTYBeats.Traditional instruments are highly accessible to blind and low-vision musicians. Electronic music production tools are not. I look at the history of accessible instruments and software interfaces, give an overview of current electronic music hardware and software, and discuss the design considerations underlying my project.

QWERTYBeats logo

Historical overview

Acoustic instruments give rich auditory and haptic feedback, and pose little obstacle to blind musicians. We need look no further for proof than the long history of iconic blind musicians like Ray Charles and Stevie Wonder. Even sighted instrumentalists rarely look at their instruments once they have attained a sufficient level of proficiency. Music notation is not accessible, but Braille notation has existed since the language’s inception. Also, a great many musicians both blind and sighted play entirely by ear anyway.

Most of the academic literature around accessibility issues in music education focuses on wider adoption of and support for Braille notation. See, for example, Rush, T. W. (2015). Incorporating Assistive Technology for Students with Visual Impairments into the Music Classroom. Music Educators Journal, 102(2), 78–83. For electronic music, notation is rarely if ever a factor.

Electronic instruments pose some new accessibility challenges. They may use graphical interfaces with nested menus, complex banks of knobs and patch cables, and other visual control surfaces. Feedback may be given entirely with LED lights and small text labels. Nevertheless, blind users can master these devices with sufficient practice, memorization and assistance. For example, Stevie Wonder has incorporated synthesizers and drum machines in most of his best-known recordings.

Most electronic music creation is currently done not with instruments, but rather using specialized software applications called digital audio workstations (DAWs). Keyboards and other controllers are mostly used to access features of the software, rather than as standalone instruments. The most commonly-used DAWs include Avid Pro Tools, Apple Logic, Ableton Live, and Steinberg Cubase. Mobile DAWs are more limited than their desktop counterparts, but are nevertheless becoming robust music creation tools in their own right. Examples include Apple GarageBand and Steinberg Cubasis. Notated music is commonly composed using score editing software like Sibelius and Finale, whose functionality increasingly overlaps with DAWs, especially in regard to MIDI sequencing.

DAWs and notation editors pose steep accessibility challenges due to their graphical and spatial interfaces, not to mention their sheer complexity. In class, we were given a presentation by Leona Godin, a blind musician who records and edits audio using Pro Tools by means of VoiceOver. While it must have taken a heroic effort on her part to learn the program, Leona demonstrates that it is possible. However, some DAWs pose insurmountable problems even to very determined blind users because they do not use standard operating system elements, making them inaccessible via screen readers.

Technological interventions

There are no mass-market electronic interfaces specifically geared toward blind or low-vision users. In this section, I discuss one product frequently hailed for its “accessibility” in the colloquial rather than blindess-specific sense, along with some more experimental and academic designs.

Ableton Push

Push layout for IMPACT Faculty Showcase

Ableton Live has become the DAW of choice for electronic music producers. Low-vision users can zoom in to the interface and modify the color scheme. However, Live is inaccessible via screen readers.

In recent years, Ableton has introduced a hardware controller, the Push, which is designed to make the software experience more tactile and instrument-like. The Push combines an eight by eight grid of LED-lit touch pads with banks of knobs, buttons and touch strips. It makes it possible to create, perform and record a piece of music from scratch without looking at the computer screen. In addition to drum programming and sampler performance, the Push also has an innovative melodic mode which maps scales onto the grid in such a way that users can not play a wrong note. Other comparable products exist; see, for example, the Native Instruments Maschine.

There are many pad-based drum machines and samplers. Live’s main differentiator is its Session view, where the pads launch clips: segments of audio or MIDI that can vary in length from a single drum hit to the length of an entire song. Clip launching is tempo-synced, so when you trigger a clip, playback is delayed until the start of the next measure (or whatever the quantization interval is.) Clip launching is a forgiving and beginner-friendly performance method, because it removes the possibility of playing something out of rhythm. Like other DAWs, Live also gives rhythmic scaffolding in its software instruments by means of arpeggiators, delay and other tempo-synced features.

The Push is a remarkable interface, but it has some shortcomings for blind users. First of all, it is expensive, $800 for the entry-level version and $1400 for the full-featured software suite. Much of its feedback is visual, in the form of LED screens and color-coded lighting on the pads. It switches between multiple modes which can be challenging to distinguish even for sighted users. And, like the software it accompanies, the Push is highly complex, with a steep learning curve unsuited to novice users, blind or sighted.

The aQWERTYon

Most DAWs enable users to perform MIDI instruments on the QWERTY keyboard. The most familiar example is the Musical Typing feature in Apple GarageBand.

GarageBand musical typing

Musical Typing makes it possible to play software instruments without an external MIDI controller, which is convenient and useful. However, its layout counterintuively follows the piano keyboard, which is an awkward fit for the computer keyboard. There is no easy way to distinguish the black and white keys, and even expert users find themselves inadvertantly hitting the keyboard shortcut for recording while hunting for F-sharp.

The aQWERTYon is a web interface developed by the NYU Music Experience Design Lab specifically intended to address the shortcomings of Musical Typing.

aQWERTYon screencap

Rather than emulating the piano keyboard, the aQWERTYon draws its inspiration from the chord buttons of an accordion. It fills the entire keyboard with harmonically related notes in a way that supports discovery by naive users. Specifically, it maps scales across the rows of keys, staggered by intervals such that each column forms a chord within the scale. Root notes and scales can be set from pulldown menus within the interface, or preset using URL parameters. It can be played as a standalone instrument, or as a MIDI controller in conjunction with a DAW. Here is a playlist of music I created using the aQWERTYon and GarageBand or Ableton Live:

The aQWERTYon is a completely tactile experience. Sighted users can carefully match keys to note names using the screen, but more typically approach the instrument by feel, seeking out patterns on the keyboard by ear. A blind user would need assistance loading the aQWERTYon initially and setting the scale and root note parameters, but otherwise, it is perfectly accessible. The present project was motivated in large part by a desire to make exploration of rhythm as playful and intuitive as the aQWERTYon makes exploring chords and scales.

Soundplant

The QWERTY keyboard can be turned into a simple drum machine quite easily using a free program called Soundplant. The user simply drags audio files onto a graphical key to have it triggered by that physical key. I was able to create a TR-808 kit in a matter of minutes:

Soundplant with 808 samples

After it is set up and configured, Soundplant can be as effortlessly accessible as the aQWERTYon. However, it does not give the user any rhythmic assistance. Drumming in perfect time is an advanced musical skill, and playing drum machine samples out of time is not much more satisfying than banging on a metal bowl with a spoon out of time. An ideal drum interface would offer beginners some of the rhythmic scaffolding and support that Ableton provides via Session view, arpeggiators, and the like.

The Groove Pizza

Drum machines and their software counterparts offer an alternative form of rhythmic scaffolding. The user sequences patterns in a time-unit box system or piano roll, and the computer performs those patterns flawlessly. The MusEDLab‘s Groove Pizza app is a web-based drum sequencer that wraps the time-unit box system into a circle.

Groove Pizza - Bembe

The Groove Pizza was designed to make drum programming more intuitive by visualizing the symmetries and patterns inherent in musical-sounding rhythms. However, it is totally unsuitable for blind or low-vision users. Interaction is only possible through the mouse pointer or touch, and there are no standard user interface elements that can be parsed by screen readers.

Before ever considering designing for the blind, the MusEDLab had already considered the Groove Pizza’s limitations for younger children and users with special needs: there is no “live performance” mode, and there is always some delay in feedback between making a change in the drum pattern and hearing the result. We have been considering ways to make a rhythm interface that is more immediate, performance-oriented and tactile. One possible direction would be to create a hardware version of the Groove Pizza; indeed, one of the earliest prototypes was a hardware version built by Adam November out of a pizza box. However, hardware design is vastly more complex and difficult than software, so for the time being, software promises more immediate results.

Haenselmann-Lemelson-Effelsberg MIDI sequencer

This experimental interface is described in Haenselmann, T., Lemelson, H., & Effelsberg, W. (2011). A zero-vision music recording paradigm for visually impaired people. Multimedia Tools and Applications, 5, 1–19.

Haenselmann-Lemelson-Effelsberg MIDI sequencer

The authors create a new mode for a standard MIDI keyboard that maps piano keys to DAW functions like playback, quantization, track selection, and so on. They also add “earcons” (auditory icons) to give sonic feedback when particular functions have been activated that normally only give graphical feedback. For example, one earcon sounds when recording is enabled; another sounds for regular playback. This interface sounds promising, but there are significant obstacles to its adoption. While the authors have released the source code as a free download, that requires a would-be user to be able to compile and run it. This is presuming that they could access the code in the first place; the download link given in the paper is inactive. It is an all-too-common fate of academic projects to never get widespread usage. By posting our projects on the web, the MusEDLab hopes to avoid this outcome.

Statement

Music education philosophy

My project is animated by a constructivist philosophy of music education, which operates by the following axiomatic assumptions:

  • Learning by doing is better than learning by being told.
  • Learning is not something done to you, but rather something done by you.
  • You do not get ideas; you make ideas. You are not a container that gets filled with knowledge and new ideas by the world around you; rather, you actively construct knowledge and ideas out of the materials at hand, building on top of your existing mental structures and models.
  • The most effective learning experiences grow out of the active construction of all types of things, particularly things that are personally or socially meaningful, that you develop through interactions with others, and that support thinking about your own thinking.

If an activity’s challenge level is beyond than your ability, you experience anxiety. If your ability at the activity far exceeds the challenge, the result is boredom. Flow happens when challenge and ability are well-balanced, as seen in this diagram adapted from Csikszentmihalyi.

Flow

Music students face significant obstacles to flow at the left side of the Ability axis. Most instruments require extensive practice before it is possible to make anything that resembles “real” music. Electronic music presents an opportunity here, because even a complete novice can produce music with a high degree of polish quickly. It is empowering to use technologies that make it impossible to do anything wrong; it frees you to begin exploring what you find to sound right. Beginners can be scaffolded in their pitch explorations with MIDI scale filters, Auto-Tune, and the configurable software keyboards in apps like Thumbjam and Animoog. Rhythmic scaffolding is more rare, but it can be had via Ableton’s quantized clip launcher, by MIDI arpeggiators, and using the Note Repeat feature on many drum machines.

QWERTYBeats proposal

My project takes drum machine Note Repeat as its jumping off point. When Note Repeat is activated, holding down a drum pad triggers the corresponding sound at a particular rhythmic interval: quarter notes, eighth notes, and so on. On the Ableton Push, Note Repeat automatically syncs to the global tempo, making it effortless to produce musically satisfying rhythms. However, this mode has a major shortcoming: it applies globally to all of the drum pads. To my knowledge, no drum machine makes it possible to simultaneously have, say, the snare drum playing every dotted eighth note while the hi-hat plays every sixteenth note.

I propose a web application called QWERTYBeats that maps drums to the computer keyboard as follows:

  • Each row of the keyboard triggers a different drum/beatbox sound (e.g. kick, snare, closed hi-hat, open hi-hat).
  • Each column retriggers the sample at a different rhythmic interval (e.g. quarter note, dotted eighth note).
  • Circles dynamically divide into “pie slices” to show rhythmic values.

The rhythm values are shown below by column, with descriptions followed by the time interval as shown as a fraction of the tempo in beats per minute.

  1. quarter note (1)
  2. dotted eighth note (3/4)
  3. quarter note triplet (2/3)
  4. eighth note (1/2)
  5. dotted sixteenth note (3/8)
  6. eighth note triplet (1/3)
  7. sixteenth note (1/4)
  8. dotted thirty-second note (3/16)
  9. sixteenth note triplet (1/6)
  10. thirty-second note (1/8)

By simply holding down different combinations of keys, users can attain complex syncopations and polyrhythms. If the app is synced to the tempo of a DAW or music playback, the user can perform good-sounding rhythms over any song that is personally meaningful to them.

The column layout leaves some unused keys in the upper right corner of the keyboard: “-“, “=”, “[“, “]”, “”, etc. These can be reserved for setting the tempo and other UI elements.

The app defaults to Perform Mode, but clicking Make New Kit opens Sampler mode, where users can import or record their own drum sounds:

  • Keyboard shortcuts enable the user to select a sound, audition it, record, set start and end point, and set its volume level.
  • A login/password system enables users to save kits to the cloud where they can be accessed from any computer. Kits get unique URL identifiers, so users can also share them via email or social media.

It is my goal to make the app accessible to users with the widest possible diversity of abilities.

  • The entire layout will use plain text, CSS and JavaScript to support screen readers.
  • All user interface elements can be accessed via the keyboard: tab to change the keyboard focus, menu selections and parameter changes via the up and down arrows, and so on.

Perform Mode:

QWERTYBeats concept images - Perform mode

Sampler Mode:

sampler-mode

Mobile version

The present thought is to divide up the screen into a grid mirroring the layout of the QWERTY keyboard. User testing will determine whether this will produce a satisfying experience.

QWERTYDrum - mobile

Prototype

I created a prototype of the app using Ableton Live’s Session View.

QWERTYBeats - Ableton prototype

Here is a sample performance:

There is not much literature examining the impact of drum programming and other electronic rhythm sequencing on students’ subsequent ability to play acoustic drums, or to keep time more accurately in general. I can report anecdotally that my own time spent sequencing and programming drums improved my drumming and timekeeping enormously (and mostly inadvertently.) I will continue to seek further support for the hypothesis that electronically assisted rhythm creation builds unassisted rhythmic ability. In the meantime, I am eager to prototype and test QWERTYBeats.