Affordances and Constraints

Note-taking for User Experience Design with June Ahn

Don Norman discusses affordances and constraints in The Design of Everyday Things, Chapter Four: Knowing What To Do.

Don Norman - The Design of Everyday Things

User experience design is easy in situations where there’s only one thing that the user can possibly do. But as the possibilities multiply, so do the challenges. We can deal with new things using information from our prior experiences, or by being instructed. The best-designed things include the instructions for their own use, like video games whose first level act as tutorials, or doors with handles that communicate how you should operate them by their shape and placement.

We use affordances and constraints to learn how things work. Affordances suggest the range of possibilities, and constraints limit the alternatives. Constraints include:

  • Physical limitations. Door keys can only be inserted into keyholes vertically, but you can still insert the key upside down. Car keys work in both orientations.
  • Semantic constraints. We know that red lights mean stop and green lights mean go, so we infer that a red light means a device is off or inoperative, and a green light means it’s on or ready to function. We have a slow cooker that uses lights in the opposite way and it screws me up every time.
  • Cultural constraints. Otherwise known as conventions. (Not sure how these are different from semantic constraints.) Somehow we all know without being told that we’re supposed to face forward in the elevator. Google Glass was an epic failure because its early adopters ran into the cultural constraint of people not liking to be photographed without consent.
  • Logical constraints. The arrangement of knobs controlling your stove burners should match the arrangement of the burners themselves.

The absence of constraints makes things confusing. Norman gives examples of how much designers love rows of identical switches which give no clues as to their function. Distinguishing the switches by shape, size, or grouping might not look as elegant, but would make it easier to remember which one does what thing.

Helpful designs use visibility (making the relevant parts visible) and feedback (giving actions an immediate and obvious effect.) Everyone hates the power buttons on iMacs because they’re hidden on the back, flush with the case. Feedback is an important way to help us distinguish the functional parts from the decorative ones. Propellerheads Reason is an annoying program because its skeuomorphic design puts as many decorative elements on the screen as functional ones. Ableton Live is easier to use because everything on the screen is functional.

When you can’t make things visible, you can give feedback via sound. Pressing a Mac’s power button doesn’t immediately cause the screen to light up, but that’s okay, because it plays the famous startup sound. Norman’s examples of low-tech sound feedback include the “zzz” sound of a functioning zipper, a tea kettle’s whistle, and the various sounds that machines make when they have mechanical problems. The problem with sound as feedback is that it can be intrusive and annoying.

The term “affordance” is the source for a lot of confusion. Norman tries to clarify it in his article “Affordance, Conventions and Design.” He makes a distinction between real and perceived affordances. Anything that appears on a computer screen is a perceived affordance. The real affordances of a computer are its physical components: the screen itself, the keyboard, the trackpad. The MusEDLab was motivated to create the aQWERTYon by considering the computer’s real affordances for music making. Most software design ignores the real affordances and only considers the perceived ones.

Designers of graphical user interfaces rely entirely on conceptual models and cultural conventions. (Consider how many programs use a graphic of a floppy disk as a Save icon, and now compare to the last time you saw an actual floppy disk.) For Norman, graphics are perceived affordances by definition.

Joanna McGrenere and Wayne Ho try to nail the concept down harder in “Affordances: Clarifying and Evolving a Concept.” The term was coined by the perceptual psychologist James J. Gibson in his book The Ecological Approach to Visual Perception. For Gibson, affordances exist independent of the actor’s ability to perceive them, and don’t depend on the actor’s experiences and culture. For Norman, affordances can include both perceived and actual properties, which to me makes more sense. If you can’t figure out that an affordance exists, then what does it matter if it exists or not?

Norman collapses two distinct aspects of design: an object’s utility of an object and the way that users learn or discover that utility. But are designing affordances and designing the information about the affordances the same thing? McGrenere and Ho say no, that it’s the difference between usefulness versus usability. They complain that the HCI community has focused on usability at the expense of usefulness. Norman says that a scrollbar is a learned convention, not a real affordance. McGrenere and Ho disagree, because the scrollbar affords scrolling in a way that’s built into the software, making it every bit as much a real affordance as if it were a physical thing. The learned convention is the visual representation of the scrollbar, not the basic fact of it.

The best reason to distinguish affordances from their communication or representation is that sometimes the communication gets in the way of the affordance itself. For example, novice software users need graphical user interfaces, while advanced users prefer text commands and keyboard shortcuts. A beginner needs to see all the available commands, while a pro prefers to keep the screen free of unnecessary clutter. Ableton Live is a notoriously beginner-unfriendly program because it prioritizes visual economy and minimalism over user handholding. A number of basic functions are either invisible or so tiny as to be effectively invisible. Apple’s GarageBand welcomes beginners with photorealistic depictions of everything, but its lack of keyboard shortcuts makes it feel like wearing oven mitts for expert users. For McGrenere and Ho, the same feature of one of these programs can be an affordance or anti-affordance depending on the user.

Designing a more welcoming aQWERTYon experience

This post documents my final project for User Experience Design with June Ahn

The best aQWERTYon screencap

Overview of the problem

The aQWERTYon is a web-based music performance and theory learning interface designed by the NYU Music Experience Design Lab. The name is a play on “QWERTY accordion.” The aQWERTYon invites novices to improvise and compose using a variety of scales and chords normally available only to advanced musicians. Notes map onto the computer keyboard such that the rows play scales and the columns play chords. The user can not play any wrong notes, which encourages free and playful exploration. The aQWERTYon has a variety of instrument sounds to choose from, and it can also act as a standard MIDI controller for digital audio workstations (DAWs) like GarageBand, Logic, and Ableton Live. As of this writing, there have been aQWERTYon 32,000 sessions.

One of our core design principles is to work within our users’ real-world technological limitations. We build tools in the browser so they will be platform-independent and accessible anywhere where there is internet access. Our aim with the aQWERTYon was to find the musical possibilities in a typical computer with no additional software or hardware. That question led us to investigate ways of turning the standard QWERTY keyboard into a beginner-friendly instrument.

While the aQWERTYon has been an effective tool in classrooms and online, it has some design deficiencies as well. It is difficult for unassisted users to figure out what the app is for. While its functionality is easily discovered through trial and error, its musical applications are less self-explanatory. Some of this is due to the intrinsic complexity of music theory and all the daunting terminology that comes with it. But some of it is the lack of context and guidance we provide to new users.

The conjecture

This assignment coincided with discussions already taking place in the lab around redesigning the aQW. Many of those focused on a particular element of the user interface, the scale picker.

aQWERTYon scale picker

The user has a variety of scales to choose from, ranging from the familiar to the exotic. However, these scales all have impenetrable names. How are music theory novices supposed to make sense of names like harmonic minor or Lydian mode? How would they know to choose one scale or another? We debated the least off-putting way of presenting these choices: should we represent them graphically? Associate each one with a well-known piece of music? Or just list them alphabetically? I proposed a system of graphical icons showing the notes comprising each scale. While novices will find them no more intelligible than the names, the hope is that they would be sufficiently visually appealing to invite users to explore them by ear.

aQW scale picker interactive wireframe

Conversations with June helped me understand that there are some broader and more profound user experience problems to solve before users ever get to the scale picker. What is the experience of simply landing on the app for the first time? How do people know what to do? From this conversation came the germ of a new idea, a landing page offering a tutorial or introduction. We want users to have a feeling of discovery, a musical “aha moment”, the chance to be a musical insider. The best way to do that seemed to be to give users a playlist of preset songs to jam with.

User characteristics and personas

There are three major user groups for the aQWERTYon, who I will describe as students, teachers, and explorers.

Students and teachers

Students use the aQW in a guided and structured setting: a classroom, a private lesson, or an online tutorial. There are several distinct user personas: elementary, middle and high school students, both mainstream and with special needs; college students; and online learners, mostly adults. Each student persona has its corresponding teacher persona. For example, I use the aQW with my music technology students at Montclair State University and NYU, and with some private students.

The aQW’s biggest fan is MusEDLab partner Matt McLean, who teaches at the Little Red Schoolhouse and runs a nonprofit organization called the Young Composers and Improvisors Workshop. Matt uses the aQW to teach composition in both settings, in person and online. He has documented his students’ use of the aQW extensively. Some examples:

Explorers

I use the term explorers to describe people who use the aQW without any outside guidance. Explorers do not fit into specific demographic groups, but they center around two broad, overlapping personas: bedroom producers and music theory autodidacts. Explorers may find the aQW via a link, a social media posting, or a Google search. We know little about these users beyond what is captured by Google Analytics. However, we can make some assumptions based on our known referral sources. For example, this blog is a significant driver of traffic to the aQW. I have numerous posts on music theory and composition that link to the aQW so that readers can explore the concepts for themselves. My blog readership includes other music educators and some professional musicians, but the majority are amateur musicians and very enthusiastic listeners. These are exactly the users we are trying to serve: people who want to learn about music independently, either for creative purposes or to simply satisfy curiosity.

While I am a music educator, I have spent most of my life as a self-taught bedroom producer, so I identify naturally with the explorers. I have created several original pieces of music with the aQW, both for user testing purposes and to show its creative potential. While I have an extensive music theory background, I am a rudimentary keyboard player at best. This has limited my electronic music creation to drawing in the MIDI piano roll with the mouse pointer, since I can not perform my ideas on a piano-style controller. The aQW suits my needs perfectly, since I can set it to any scale I want and shred fearlessly. Here is an unedited improvisation I performed using a synthesizer instrument I created in Ableton Live:

My hope is that more would-be explorers feel invited to use the aQW for similar creative purposes in their own performance and composition.

Tasks and Scenarios

It is possible to configure the aQWERTYon via URL parameters to set the key and scale, and to hide components of the user interface. When teachers create exercises or assignments, they can link or embed the aQW with its settings locked to keep students from getting lost or confused. However, this does not necessarily invite the user to explore or experiment. Here is an example of an aQW preset to accompany a Beyoncé song. This preset might be used for a variety of pedagogical tasks, including learning some or all of the melody, creating a new countermelody, or improvising a solo. The harmonic major scale is not one that is usually taught, but it a useful way to blend major and minor tonalities. Students might try using more standard scales like major or harmonic minor, and listen for ways that they clash with Beyoncé’s song.

Tasks and scenarios for explorers might include creating a melody, bassline or chords for an original piece of music. For example, a self-taught dance music producer might feel limited by the scales that are easiest to play on a piano-style keyboard (major, natural minor, pentatonics) and be in search of richer and more exotic sounds. This producer might play their track in progress and improvise on top using different scale settings.

One of the users I tested with suggested an alternative explorer use case. He is an enthusiastic amateur composer and arranger, who is trying to arrange choral versions of pop and rock songs. He is a guitarist who has little formal music theory knowledge. He might use the aQW to try out harmonic ideas by ear, write down note names that form pleasing combinations, and then transfer them to the guitar or piano-based MIDI controller.

Understanding the problem

In the age of the computer and the internet, many aspects of music performance, composition and production are easy to self-teach. However, music theory remains an obstacle for many bedroom producers and pop musicians (not to mention schooled musicians!) There are so many chords and scales and rules and technical vocabulary, all of which have to be applied in all twelve keys. To make matters worse, terminology hangs around long after its historical context has disappeared. We no longer know what the Greek modes sound like, but we use their names to describe modern scales. C-sharp and D-flat were different pitches in historical tuning systems, but now both names describe the same pitch. The harmonic and melodic minor scales are named after a stylistic rule for writing melodies that was abandoned hundreds of years ago. And so on.

Most existing theory resources draw on the Western classical tradition, using examples and conventions from a repertoire most contemporary musicians and listeners find unfamiliar. Furthermore, these resources presume the ability to read standard music notation. Web resources that do address popular music are usually confusing and riddled with errors. I have worked with Soundfly to fill this vacuum by creating high-quality online courses aimed at popular musicians. Even with the best teaching resources, though, theory remains daunting. Exploring different chords and scales on an instrument requires significant technical mastery, and many musicians give up before ever reaching that point.

The aQW is intended to ease music theory learning by making scales and chords easy to discover even by complete novices. Our expectation is that after explorers are able to try theory ideas out in a low-pressure and creative setting, they will be motivated to put them to work playing instruments, composing or producing. Alternatively, users can simply perform and compose directly with the aQW itself.

Social and technical context

Most computer-based melody input systems are modeled on the piano. This is most obvious for hardware, since nearly all MIDI controllers take the form of literal piano keyboards. It is also true for software, which takes the piano keyboard as the primary visualization scheme for pitch. For example, the MIDI editor in every DAW displays pitches on a “piano roll”.

Some DAWs include a “musical typing” feature that maps the piano layout to the QWERTY keyboard, as an expediency for users who either lack MIDI hardware controllers, or who do not have them on hand. Apple’s GarageBand uses the ASDFG row of the keyboard for the white keys and the QWERTY row for the black keys. They use the other rows for such useful controls as pitch bend, modulation, sustain, octave shifting and simple velocity control.

GarageBand musical typing

Useful and expedient though it is, Musical Typing has some grave shortcomings as a user interface. It presumes familiarity with the piano keyboard, but is not very playable for users do who possess that familiarity. The piano layout makes a poor fit for the grid of computer keys. For example, there is no black key on the piano between the notes E and F, but the QWERTY keyboard gives no visual reminder of that fact, so it is necessary to just remember it. Unfortunately, the “missing” black key between E and F happens to be the letter R, which is GarageBand’s keyboard shortcut for recording. While hunting for E-flat or F-sharp, users are prone to accidentally start recording over their work. I have been using GarageBand for seven years and still do this routinely.

Ableton’s Push controller represents an interesting break with MIDI controller orthodoxy. It is a grid of 64 touch pads surrounded by various buttons, knobs and sliders.

Ableton Push

The pads were designed to trigger samples and loops like a typical drum machine, but Ableton also includes a melody mode for the Push. By default, it maps notes to the grid in rows staggered by fourths, which makes the layout identical to the bottom four strings of the guitar. This is quite a gift for guitarists like me, since I can use my familiar chord and scale fingerings, rather than hunting and pecking for them on the piano. Furthermore, the Push can be set so that the pads play only the notes within a particular scale, giving a “no wrong notes” experience similar to the aQWERTYon. Delightful though this mode is, however, it is imperfect. Root notes of the scale are colored blue, and other notes are colored white. While this makes the roots easy to distinguish, it is not so easy to visually differentiate the other pitches.

Touchscreen devices like the iPhone and iPad open up additional new possibilities for melodic interfaces. Many mobile apps continue to use the piano keyboard for note input, but some take advantage of the touchscreen’s unique affordances. One such is Thumbjam, which enables the user to divide the screen into slices of arbitrary thickness that can map to any arbitrary combination of notes.

Thumbjam

The app offers hundreds of preset scales to choose from. The user may have a small range of notes, each of which is large and easy to distinguish, or a huge range of notes, each of which occupies a narrow strip of screen area. Furthermore, the screen can be split to hold four different scales, played from four different instruments. While all of this configurability is liberating, it is also overwhelming. Also, the scales are one-dimensional lines; there is no easy way to play chords and arpeggios.

Evaluation criteria

Is the aQW’s potential obvious enough to draw in explorers and educators? Will it be adopted as a tool for self-teaching? Does it invite playful exploration and experimentation? Is it satisfying for real-world musical usage? Is the UI self-explanatory, or at least discoverable? Is the music theory content discoverable? Have we identified the right user persona(s)? Is the aQW really a tool for beginners? Or is it an intermediate music theory learning tool? Or an advanced composition tool? Is the approach of a “playlist” of example songs the right one? Which songs, artists and genres should we include on the landing page? How many presets should we include? Should we limit it to a few, or should we offer a large, searchable database? And how do we deal with the fact that many songs require multiple scales to play?

Proposed solution

I tested several interactive wireframes of this landing page concept. Click the image to try it yourself:

aQWERTYon landing page interactive wireframe

The first wireframe had nine preset songs. I wanted to offer reasonable musical diversity without overwhelming the user. The tenth slot linked to the “classic” aQW, where users are free to select their own video, scale, root, and so on. I chose songs that appealed to me (and presumably other adult explorers), along with some current pop songs familiar to younger users. I wanted to balance the choices by race, gender, era, and genre. I was also bound by a musical constraint: all songs need to be playable using a single scale in a single key. The initial preset list was:

  • Adele – “Send My Love (To Your New Lover)”
  • Mary J Blige – “Family Affair”
  • Miles Davis – “Sssh/Peaceful”
  • Missy Elliott – “Get Ur Freak On”
  • Björk – “All Is Full Of Love”
  • Michael Jackson – “Don’t Stop ’Til You Get Enough”
  • Katy Perry – “Teenage Dream”
  • AC/DC – “Back In Black”
  • Daft Punk – “Get Lucky”

After a few test sessions, it became apparent that no one was clicking Mary J Blige. Also, the list did not include any current hip-hop. I therefore replaced her with Chance The Rapper. I initially offered a few sentences of instruction, but feedback from my MusEDLab colleagues encouraged me to reduce the prompt down to just a few words: “Pick a song, type, jam.”

Further testing showed that while adults are willing to try out any song, familiar or not, children and teens are much choosier. Therefore, I added two more presets, “Hotline Bling” by Drake and “Formation” by Beyoncé. The latter song proved problematic, however, because its instrumental backing is so sparse and minimal that it is difficult to hear how other notes might fit into it. I ultimately swapped it for “Single Ladies.” I had rejected this song initially, because it uses the idiosyncratic harmonic major scale. However, I came to see this quirk as a positive bonus–since one of our goals is to encourage users to explore new sounds and concepts, a well-known and well-loved song using an unusual scale is a rare gift.

User testing protocol

I used a think-aloud protocol, asking testers to narrate their thought processes as they explored the app. I recorded the one-on-one sessions using Screenflow. When testing with groups of kids, this was impractical, so instead I took notes during and after each session. For each user, I opened the interactive wireframe, and told them, “This is a web based application for playing music with your computer keyboard. I’m going to ask you to tell me what you see on the screen, what you think it does, and what you think will happen when you click things.” I did not offer any other explanation or context, because I wanted to see whether the landing page was self-explanatory and discoverable. I conducted informal interviews with users during and after the sessions as well.

User testing results

I tested with ten adults and around forty kids. The adults ranged in age from early twenties to fifties. All were musicians, at varying levels of ability and training, mostly enthusiastic amateurs. Sessions lasted for twenty or thirty minutes. There were two groups of kids: a small group of eighth graders at the Little Red Schoolhouse, and a large group of fourth graders from PS 3 who were visiting NYU. These testing sessions were shorter, ten to fifteen minutes each.

User testing the aQWERTYon with fourth graders

Discovering melodies

It is possible to play the aQW by clicking the notes onscreen using the mouse, though this method is slow and difficult. Nevertheless, a number of the younger testers did this, even after I suggested that it would be easier on the keyboard.

An adult tester with some keyboard and guitar experience told me, “This is great, it’s making me play patterns that I normally don’t play.” He was playing on top of the Miles Davis track, and he was quickly able to figure out a few riffs from Miles’ trumpet solo.

Discovering chords

Several testers systematically identified chords by playing alternating notes within a row, while others discovered them by holding down random groups of keys. None of the testers discovered that they could easily play chords using columns of keys until I prompted them to do so. One even asked, “Is there a relationship between keys if I play them vertically? I don’t know enough about music to know that.” After I suggested he try the columns, he said, “If I didn’t know [by ear] how chords worked, I’d miss the beauty of this.” He compared the aQW to GarageBand’s musical typing: “This is not that. This is a whole new thing. This is chord oriented. As a guitarist, I appreciate that.” The message is clear: we need to make the chords more obvious, or more actively assist users in finding them.

Other theory issues

For the most part, testers were content to play the scales they were given, though some of the more expert musicians changed the scales before even listening to the presets. However, not everyone realized that the presets were set to match the song. A few asked me: “How do I know what key this song is in?” We could probably state explicitly that the presets line up automatically.

In general, adult testers found the value of the aQW as a theory learning tool to be immediately apparent. One told me: “If I had this when I was a kid, I would have studied music a lot. I used to hate music theory. I learned a lot of stuff, but the learning process was awful… Your kids’ generation will learn music like this (snaps fingers).”

Sounds

The aQW comes with a large collection of SoundFonts, and users of all ages enjoyed auditioning them, sometimes for long periods of time. Sometimes they apologized for how fascinating they found the sounds to be. But it is remarkable to have access to so many instrument timbres so effortlessly. Computers turn us all into potential orchestrators, arrangers, and sound designers.

Screen layout

The more design-oriented testers appreciated the sparseness and minimalism of the graphics, finding them calming and easy to understand.

Several testers complained that the video window takes up too much screen real estate, and is placed too prominently. Two commented that videos showing live performers, like “Back In Back,” were valuable because that helped with timekeeping and inspiration. Otherwise, however, testers found the videos to either be of little value or actively distracting. One suggested having the videos hidden or minimized by default, with the option to click to expand them. Others requested that the video be below the keyboard and other crucial controls. Also, the eighth graders reported that some of the video content was distracting because of its content, for example the partying shown in “Teenage Dream.” Unsuitable content will be an ongoing issue using many of the pop songs that kids like.

Technical browser issues

Having the aQWERTYon run in the browser has significant benefits, but a few limitations as well. Because the URL updates every time the parameters change, clicking the browser’s Back button does not produce the expected behavior–it might take ten or fifteen clicks to actually return to a previous page. I changed the links in later versions so each one opens the aQW in a new tab so the landing page would always be available. However, web audio is very memory-intensive, and the aQW will function slowly or not at all if it is open in more than one tab simultaneously.

Song choices

The best mix of presets is always going to depend on the specific demographics of any given group of users. However, the assortment I arrived at was satisfying enough for the groups I tested with. Miles Davis and Björk do not have the wide appeal of Daft Punk or Michael Jackson, but their presence was very gratifying for the more hipster-ish testers. I was extremely impressed that an eighth grader selected the Miles song, though this kid turns out to be the son of a Very Famous Musician and is not typical.

Recording functionality

Testers repeatedly requested the ability to record their playing. The aQW did start out with a very primitive recording feature, but it will require some development to make it usable. The question is always, how much functionality is enough? Should users be able to overdub? If so, how many tracks? Is simple recording enough, or would users need to able to mix, edit, and select takes?

One reason that recording has been a low development priority is that users can easily record their performances via MIDI into any DAW or notation program. The aQW behaves as if it were a standard MIDI controller plugged into the computer. With so many excellent DAWs in the world, it seems less urgent for us to replicate their functionality. However, there is one major limitation of recording this way: it captures the notes being played, but not the sounds. Instead, the DAW plays back the MIDI using whatever software instruments it has available. Users who are attached to a specific SoundFont cannot record them unless they use a workaround like Soundflower. This issue will require more discussion and design work.

New conjectures and future work

One of my most significant user testers for the landing page wireframe was Kevin Irlen, the MusEDLab’s chief software architect and main developer of the aQW itself. He found the landing page concept sufficiently inspiring that he created a more sophisticated version of it, the app sequencer:

aQWERTYon app sequencer v1

We can add presets to the app sequencer using a simple web form, which is a significant improvement over the tedious process of creating my wireframes by hand. The sequencer pulls images automatically from YouTube, another major labor-saver. Kevin also added a comment field, which gives additional opportunity to give prompts and instructions. Each sequencer preset generates a unique URL, making it possible to generate any number of different landing pages. We will be able to create custom landing pages focusing on different artists, genres or themes.

Songs beyond the presets

Testing with the fourth graders showed that we will need to design a better system for users who want to play over songs that we do not include among the presets. That tutorial needs to instruct users how to locate YouTube URLs, and more dauntingly, how to identify keys and scales. I propose an overlay or popup:

Keyfinding

Testing with fourth graders also showed that helping novice users with keyfinding may not be as challenging as I had feared. The aQW defaults to the D minor pentatonic scale, and that scale turns out to fit fairly well over most current pop songs. If it doesn’t, some other minor pentatonic scale is very likely to work. This is due to a music-theoretical quirk of the pentatonic scale: it happens to share pitches with many other commonly-used scales and chords. As long as the root is somewhere within the key, the minor pentatonic will sound fine. For example, in C major:

  • C minor pentatonic sounds like C blues
  • D minor pentatonic sounds like Csus4
  • E minor pentatonic sounds like Cmaj7
  • F minor pentatonic sounds like C natural minor
  • G minor pentatonic sounds like C7sus4
  • A minor pentatonic is the same as C major pentatonic
  • B minor pentatonic sounds like C Lydian mode

 

We are planning to revamp the root picker to show both a larger piano keyboard and a pitch wheel. We also plan to add more dynamic visualization options for notes as they are played, including a staff notation view, the chromatic circle, and the circle of fifths. The aQW leaves several keys on the keyboard unused, and we could use them for additional controls. For example, we might use the Control key to make note velocities louder, and Option to make them quieter. The arrow keys might be used to cycle through the scale menu and to shift the root.

Built-in theory pedagogy

There is a great deal of opportunity to build more theory pedagogy on top of the aQW, and to include more of it within the app itself. We might encourage chord playing by automatically showing chord labels at the top of each column. We might include popups or links next to each scale giving some explanation of why they sound the way they do, and to give some suggested musical uses. One user proposes a game mode for more advanced users, where the scale is set to chromatic and players must identify the “wrong” or outside notes. Another proposes a mode similar to Hooktheory, where users could sequence chord progressions to play on top of.

Rhythmic assistance

A few testers requested some kind of help or guidance with timekeeping. One suggested a graphical score in the style of Guitar Hero, or a “follow the bouncing ball” rhythm visualization. Another pointed out that an obvious solution would be to incorporate the Groove Pizza, perhaps in miniature form in a corner of the screen. Synchronizing all of this to YouTube videos would need to be done by hand, so far as I know, but perhaps an automated solution exists. Beat detection is certainly an easier MIR challenge than key or chord detection. If we were able to automatically sync to the tempo of a song, we could add the DJ functionality requested by one tester, letting users add cue points, loop certain sections, and slow them down.

Odds and ends

One eighth grader suggested that we make aQW accounts with “musical passwords.”

An adult tester referred to the landing page as the “Choose Your Own Adventure screen.” The idea of musical adventure is exactly the feeling I was hoping for.

In addition to notes on the staff, one tester requested a spectrum visualizer. This is perhaps an esoteric request, but real-time spectrograms are quite intuitive and might be useful.

Finally, one tester made a comment that was striking in its broader implications for music education: “I’m not very musical, I don’t really play an instrument, so these kinds of tricks are helpful for me. It didn’t take me long to figure out how the notes are arranged.” This person is a highly expert producer, beatmaker and live performer using Ableton Live. I asked how he came to this expertise, and he said he felt compelled to learn it to compensate for his lack of “musicianship”. It makes me sad that such a sophisticated musician does not realize that his skills “count”. In empowering music learners with the aQW, I also hope we are able to help computer musicians value themselves.

Design-Based Research

Note-taking for Research on Games and Simulations with Jan Plass

Barab, S. A. (2014). Design-based research: a methodological toolkit for engineering change. In K. Sawyer (ed.) Handbook of the Learning Sciences, Vol 2, (pp. 233-270), Cambridge, MA: Cambridge University Press.

Design-based research

Design-based research (DBR) is a subject close to my heart, because it was the basis of my masters thesis, and informs the work of the NYU Music Experience Design Lab. All of our tools are designed and tested in the context of messy and complex natural learning and creating environments: classrooms, bedrooms, studios, and public events. We evaluate our tools continuously, but the only purely empirical and “experimental” methods we use involve Google analytics. We sometimes conduct user research in formal settings, but mostly observe practice “in the wild” between regular iterations.

DBR follows Dewey’s model of praxis, not so much a traditional positivist science or ethnographic tradition of inquiry, but rather a pragmatic form of inquiry “where theories are judged not by their claims to truth, but by their ability to do work in the world.” It entails the study of learning innovations inspired by theory, conducted in naturalistic contexts. The goal is to produce new tools and practices that can be generalized to other learning environments (typically schools, but not necessarily.) DBR also aims to produce new theories, viewing them as intertwined with practice rather than occurring prior to it.

DBR would appear to fail to meet tests of rigor, freedom from bias, and reproducibility. However, advocates respond that for a learning intervention to be studied with full empirical rigor, variables must be isolated to the point where they no longer resemble the reality we are supposed to be studying. We can draw an analogy from music psychology. A teacher of mine studies musical tension and release. She isolates the various musical parameters that might create a sense of tension and release – pitch, rhythm, timbre and so on – as synthesizer tones varying only along each parameter, and measures listener responses. While her results are rigorous, her test material bears almost no resemblance to actual music. It may well be that musical tension and release are emergent properties of parameters interacting in social/cultural context. Research in a more naturalistic context might hopelessly confound the variables, but might be the only way to access a valid explanation. As in music, the context of learning experiences is not simply a backdrop against which things occur, but “an integral part of the complex causal mechanisms that give rise to the phenomenon under study.”

DBR is to traditional empirical research what agile methodologies are to waterfall-style software development. Rather than thinking of a study as a hypothesis which is then tested and written up, DBR is an ongoing process of iteration, testing, and iterating further. This makes it difficult to point to a clear, unambiguous result. DBR findings are more likely to take the form of narratives, descriptions of the various iterations and tests. The challenge then becomes to present these narratives in ways that readers can generalize from them, rather than limiting their relevance to the specific situations being described. While DBR is not ethnography, its results may well take the form of ethnographic “thick description” in Clifford Geertz’s sense. As in qualitative social science research, rigor “comes from principled accounts that provide logical chains of reasoning and prove useful to others.”

In considering DBR, we need to draw a distinction between measuring outputs and outcomes. Outputs are directly measurable results of an intervention, while outcomes are larger-scale and longer-term consequences. For example, standardized test scores are an output of schooling, while learning is a consequence. Because outputs are so much more readily measured than outcomes, there is a danger that we will optimize around outputs rather than outcomes (as schools have around standardized tests.) To combat this danger, DBR must provide explanatory accounts of outcomes with the same persuasive power as quantitative measures of outputs. Research must combine “experience-near meanings” (the local story) with “experience-distance significance” (the more general implications and applications.) Good DBR will produce grounded “petite generalizations” that can be built up into broader generalizations. Researchers will also need to be explicit about the assumptions and theoretical bases that underlie the work, since these can not be controlled for.

Research proposal – Hip-Hop Pedagogy

Final paper for Principles of Empirical Research with Catherine Voulgarides

Research questions

Jamie Ehrenfeld is a colleague of mine in the NYU Music Experience Design Lab. She graduated from NYU’s music education program, and now teaches music at Eagle Academy in Brownsville. Like many members of the lab, she straddles musical worlds, bringing her training in classical voice to her work mentoring rappers and R&B singers. We often talk about our own music learning experiences. In one such discussion, Jamie remarked: “I got a music degree without ever writing a song” (personal communication, April 29 2017). Across her secondary and undergraduate training, she had no opportunity to engage with the creative processes behind popular music. Her experience is hardly unusual. There is a wide and growing divide behind the culture of school music and the culture of music generally. Music educators are steeped in the habitus of classical music, at a time when our culture is increasingly defined by the music of the African diaspora: hip-hop, R&B, electronic dance music, and rock. 

The music academy’s near-exclusive focus on Western classical tradition places it strikingly at odds with the world that our students inhabit. In this paper, I examine the ideological basis for this divide. Why does the music academy generally and the training of music educators in particular hold so closely to the traditions of Western European classical music? Why has the music academy been slow to embrace African diasporic vernacular musics? Why does it outspokenly reject hip-hop? What racial and class forces drive the divide between music educators and the culture of their students? How might we make music education more culturally responsive? How can music educators support students in developing their own musical creativity via songwriting and beatmaking? What assumptions about musical and educational values must we challenge in order to do so?

Framing of research topic

Music education scholars commonly use “non-Western” as a shorthand for music outside the European classical tradition. This might lead one to naively believe that hip-hop is non-Western music. But it arose in the United States, so how can that be? Are our racial and ethnic minorities part of our civilization, or are they not? While the American cultural mainstream has increasingly embraced black musical styles, the music education field has not followed suit. As an example, consider a meme posted to a group for music teachers on Facebook. The meme’s original author is unknown. The caption was something like, “Typical middle school/high school student.” I will leave the person who posted it to Facebook anonymous, because they no doubt meant well.

You kids like the wrong music

The meme-maker is dismayed that young people do not care how little their music adheres to the stylistic norms of the Western European classical tradition. The author dismisses contemporary popular music and can not imagine why anyone else might enjoy it. The condescending presumption is that young people do not “really” enjoy pop, that they are being tricked into it by marketing and image, and that they are too lazy and ignorant to make critical choices. The choice of the word “molester” is a remarkable one, with its connotation of sexual violence. Classically trained educators feel their culture to be under attack, with their own students leading the charge.

Eurocentrism in American music education

In examining educational practice, we must look for the “hidden curriculum” (Anyon, 1980), the ideological content that comes along with the ostensible curricular goals. For example, The Complete Musician by Steven Laitz (2015) is a widely used college-level theory text. (I used a similar book of Laitz’s to fulfill my own graduate music theory requirement.) The title asserts an all-encompassing scope, but the text only discusses Western classical harmony and counterpoint. Other elements of music, like rhythm or timbre, receive cursory treatment at most. African diasporic and non-Western musics are not mentioned. The hidden curriculum here is barely even hidden. Mcclary (2000) asks why the particular musical conventions that emerged in Europe during the eighteenth and nineteenth centuries appealed so much to musicians and audiences, what needs they satisfied, and what cultural functions they performed. We might ask, since those conventions no longer appeal to most musicians or audiences, whose needs are being satisfied by school music? What cultural functions is it performing?

America has embraced every black musical form from ragtime through trap. But while our laws and culture have become less overtly racist over time, the oppression of people of color continues, African-Americans especially. For example, while they are no more likely to use drugs than white people, black people are many times more likely to be incarcerated for it. A white applicant with a felony drug conviction is more likely to get a callback for an entry-level job than a black applicant with no criminal record at all (Pager, 2007). Our large cities are extraordinarily segregated, with black neighborhoods isolated and concentrated (Denton & Massey, 1993). Perhaps this isolation has contributed to the evolution of hip-hop and its radical break with European-descended musical practices. Perry (2004) argues that, while hip-hop is a hybrid music, it is nevertheless a fundamentally black one due to four central characteristics:

(1) its primary language is African American Vernacular English (AAVE); (2) it has a political location in society distinctly ascribed to black people, music, and cultural forms; (3) it is derived from black American oral culture; and (4) it is derived from black American musical traditions (Perry 2004, 10).

The white mainstream adores the music while showering the people who created it with contempt (Perry 2004, 27).

Black music versus white educators

If the popular mainstream is dominated by innovations in black music, the field of musical education is unified by its extraordinary whiteness, both demographically and musically. Prospective teachers tend to be white, and come from suburban, low-poverty areas (Doyle, 2014). There is corresponding disproportionality among participants in formal music classes and ensembles—privileged groups are overrepresented, while less-privileged groups are underrepresented. This is true for white students versus students of color, high-SES students versus low-SES students, native English speakers versus English language learners, students whose parents have more versus less education, and so on (Elpus & Abril, 2011). Some of the disparity is due to the fact that schools in less privileged communities are less likely to offer music in the first place. But the disparities hold true among schools that do offer music, and persist even when schools supply free instruments. Lack of access alone can not explain the overwhelming whiteness and privilege of most participants in school music.

A great deal of research shows enrollment in school music declining precipitously for the past few decades. Budget cuts alone can not explain this decline, since enrollment in other arts courses has not declined as much (Kratus, 2007). As America’s student population becomes less white, its Eurocentric music education culture is evidently becoming steadily less appealing. Finney (2007) attributes the gap between music educators and their students to differing musical codes. “Teachers tend to use elaborated codes derived from Western European ‘elite’ culture, whereas students use vernacular codes… Students and teachers are therefore in danger of standing on opposite sides of a musical and linguistic chasm with few holding the key to unlock the other’s code” (18). Williams (2011) points to large ensemble model of school music that was imported to the United States from the European conservatory tradition in the early twentieth century, and which has barely changed since. Music educators teach what they learned, and what they learned is likely to have been the conservatory-style large ensemble.

Is the solution to expand the canon of “acceptable” music to include more artists of color? A typical undergraduate music history curriculum now tacks Duke Ellington or Charlie Parker onto the end of the succession of white European composers. But the canon is a political entity, not just an aesthetic one. If we try to expand the canon to include a greater diversity of musics, we will fail to challenge the basic fact of its existence and its role in academic culture. “[T]he canon is an epistemology; it is a way of understanding the world that privileges certain aesthetic criteria and that organizes a narrative about the history and development of music around such criteria and based on that understanding of the world. In other worlds, the canon is an ideology more than a specific repertory” (Madrid 2017, 125). Diversity is of no help if we simply use it to perpetuate privilege and power inequalities. “What does it mean when the tools of a racist patriarchy are used to examine the fruits of that same patriarchy? It means that only the most narrow parameters of change are possible and allowable” (Lorde 1984, 110). Rather than making incremental changes to the canon, we must ask how we can re-orient the basic assumptions of music education, its mission, its values, and its goals.

Literature review

In this section, I examine the present state of music education scholarship addressing the racial and class dynamics of music education, as well as the rise of culturally responsive pedagogies, particularly surrounding hip-hop.

Who is school music for?

By excluding entire categories of music and musicianship from the official curriculum, music educators send powerful and lasting messages to students (and everyone else) about what our society values and what it does not (Bledsoe, 2015). I am living proof; my own experiences with school music left me bored and alienated, and I came to the conclusion that I was not a musician at all. It took me years of self-guided practice to disabuse myself of that notion. I have had endless conversations with non-classical musicians at every level about how they do not regard themselves as “real” or “legitimate” musicians, no matter how professionally or creatively accomplished they may be. Fortunately, school music is not the only vector for music education. Most popular musicians learn informally from peers or on their own, a method that has become easier thanks to the internet. Still, the stigma of “failure” is a heavy psychological burden to overcome.

School music is usually competitive. There is a competitive process to become part of an ensemble, and those ensembles compete intramurally in much the same way that sports teams do. Conservatories that produce professional musicians need to be competitive. But should we continue to model all school music on the conservatory? The similarity between school ensembles and sports teams should trouble us. Schools are not obligated to let everyone play varsity football, regardless of ability. However, we do believe that schools should teach everyone reading and math. Our efforts to support struggling readers and math learners may be inadequate or even counterproductive, but at least we try to meet all students’ needs, and we certainly do not exclude low performers from studying these subjects entirely.

Some music teachers appear to exhibit the attitude of a physician who complains that all the patients in the waiting room are sick! In other words, they prefer to work only with the talented, ‘musically healthy’ few, when it is those who are in the most need of intervention who deserve at least equal attention (Regelski 2009, 32).

What if we held music teachers to the standards of math teachers rather than football coaches? We might follow the model of physical education classes and public health initiatives, prioritizing lifetime wellness over the identification and training of elite athletes only (Dillon, 2007).

Music and identity

In traditional aesthetic approaches to the Eurocentric canon, the locus of musical expressivity and meaning of the music is embedded entirely within the music itself. Listeners’ subjective experiences are not considered to be significant; our job is to decipher the formal relationships that the composer has encoded into the score. By contrast, Elliott and Silverman (2015) argue that we should take an embodied approach to musical understanding, seeing music as an enactive process emerging from the performance and listeners’ experience of it in social/emotional context. In the embodied approach, we see music as a tool for listeners to make their own meaning, to build their identity, and to communicate and modulate their emotions, all by means of bodily and social lived experience (van der Schyff, Schiavio & Elliott, 2016). Music is “a device for ordering the self” (DeNora 2000, 73). The role of music in building individual and group identity and a sense of belonging is especially critical in adolescence, when its ability to release or control difficult emotions may be literally lifesaving (Campbell, Connell & Beegle, 2007).

Music can also be the organizing principle behind new cultures and subcultures, a locus for tribal self-identification. Turino (2016) proposes that participatory music cultures offer an alternative form of citizenship, with the potential to be fundamental to our sense of self and a cornerstone of our happiness.

Fostering creative expression

Ruthmann (2007) suggests that we teach music the way that English teachers teach writing: use creative prompts that encourage students to develop individual authentic voices capable of expressing their own ideas and thoughts. Like writing generally, songwriting is hardly an elite or specialized practice. All young children spontaneously make up songs, which can sometimes be strangely catchy. My son wrote his first song at age four without any prompting or assistance, inspired by an episode of Thomas The Tank Engine (Pomykala-Hein, 2017). For many young people, music is entirely comprised of songs (Kratus, 2016). But after elementary school, school music is more about “pieces” than songs, symptomatic of the broader gap between in-school and out-of-school music cultures.

While music therapists have long taught songwriting, it is a rare practice in school music curricula. Kratus advocates songwriting for its therapeutic benefits, and for its lifelong learning benefits as well. Few adults have the opportunity to play oboe in an orchestra, but anyone with a guitar or keyboard or smartphone can write and perform songs. Historically, the technology for writing English has been dramatically more accessible than the technology for writing music, but that is changing rapidly. The software and hardware for recording, producing and composing music becomes cheaper and more user-friendly with each passing year. The instrumental backing track for “Pride” by Kendrick Lamar (2017) was produced by the eighteen-year-old Steve Lacy entirely on his iPhone. What are the other creative possibilities inherent in the devices students carry in their pockets and backpacks?

The psychological benefits of songwriting extend beyond musical learning. Like other art media, songwriting is an opportunity to practice what Sennett (2008) calls “craftsmanship,” defined as “the desire to do a job well for its own sake.” Craftsmanship is a habit of mind that “serves the computer programmer, the doctor, and the artist; parenting improves when it is practiced as a skilled craft, as does citizenship” (Sennett 2008, 9). Musical performers exercise craftsmanship as well, but not along as many different dimensions as songwriters and producers do.

Music creation is also a potential site of ethical development. We treat our favorite songs as imaginary people who we feel loving toward and protective of. This kind of idealization is akin to what we do “when we constitute others as persons, or when we invest others with personhood” (Elliott & Silverman 2015, 190). We imagine a personhood for the music, and we try to make that personhood real. In so doing, we learn how to create personhood for each other, and for ourselves. The point of musical education should not just be training in music, but developing ethical people through music (Bowman 2007, 2016). We can consider musical sensitivity to be a particular form of emotional sensitivity, and musical intelligence to be a particular application of emotional intelligence. Musical problem solving is an excellent simulator for social problem solving generally. Both in music and in life, the challenges are ambiguous, contingent, and loaded with irreconcilable contradiction. Performance and interpretation entail some musical problem-solving, but in the classical ensemble model that is typically the purview of the conductor. Songwriting poses musical problem-solving challenges to all who attempt it.

Hip-hop pedagogies

Brian Eno (2004) observes that the recording studio is a creative medium unto itself, one with different requirements for musicality from composition or performance. Indeed, no “composing” or “performing” need ever take place in modern studio practice. Eno is a case in point—while he has produced a string of famous and revered recordings, he does not consider himself to be adept at any instrument, and can not read or write notation. The digital studio has collapsed the distinction between musicians, composers, and engineers (Bell, 2014). The word “producer” is a useful descriptor for creators working across such role boundaries. In the analog recording era, producers were figures like Quincy Jones, executive managers of a commercial process. However, the term “producer” has come to describe anyone creating recorded music in any capacity, including songwriting, beatmaking, MIDI sequencing, and audio manipulation. We might expand the word further to include anyone who actively creates music, be it recorded, notated or live. To be a producer is a category of behavior, not a category of person.

Contemporary popular music is produced more than it is performed. This is nowhere more true than in the case of hip-hop, which in its instrumental aspect is almost entirely “postperformance” (Thibeault, 2010). The processes of producers like J Dilla and Kanye West resemble those of Brian Eno far more than those of Quincy Jones. This dramatic break with traditional musical practice poses major challenges for educators trained in the classical idiom, but it also presents new opportunities for culturally relevant and critically engaged pedagogy. Hip-hop-based education is mostly discussed in the urban classroom context, aimed toward “at-risk” youth (Irby & Hall, 2011). However, as hip-hop has expanded from its black urban origins to define the rest of mainstream musical culture, so too can it move into the educational mainstream as well.

There are several ways to incorporate hip-hop into education. Pedagogies with hip-hop connect hip-hop cultures and school experiences, using hip-hop as a bridge. Pedagogies about hip-hop engage teachers and students with critical perspectives on issues within the music and its culture, using hip-hop as a lens. Pedagogies of hip-hop apply hip-hop worldviews and practices within education settings (Kruse, 2016). Music educators can use hip-hop to enhance cultural relevance and connect to the large and growing percentage of students who identify as part of hip-hop culture. However, it is the use of hip-hop practices that most interests me as a research direction.

We should avoid using hip-hop as bait to get kids interested in “legitimate” music. Instead, we can apply the hip-hop ethos of authentic, culturally engaged expression to music education generally. Kratus (2007) points out that large ensembles are some of the last remaining school settings where the teaching model maintains a top-down autocratic structure, untouched by the cognitive revolution. This method does not create independently functioning musicians. How might we find ways for students to engage in music on their own cultural and technological terms? One method might be to do sampling and remixing of familiar music as an entry point into creation. This is the approach taken by Will Kuhn (personal communication, 2017), who teaches high school students to build songs entirely out of pieces of existing songs. Students can then replace those appropriated samples with material of their own.

Hip-hop has many controversial aspects, but none provokes the ire of legacy musicians more than the practice of sampling. There is a widespread perception that sampling is nothing more than a way to avoid learning instruments or hiring musicians. This may be true in some instances, but it is easy to identify examples of artists who went to considerable expense and trouble to license samples when they did not need to do so. For example, while Ahmir “Questlove” Thompson of the Roots is a highly regarded drummer, he still uses sampled breakbeats in his productions. Why would he prefer a sample to his own playing? In hip-hop, “[e]xisting recordings are not randomly or instrumentally incorporated so much as they become the simultaneous subject and object of a creative work” (Culter 2004, 154). Samples have specific timbral qualities that evoke specific memories and associations, situating the music in webs of intertextual reference.

Rice (2003) encourages non-music educators to draw on the practice of sampling. Students might approach cultural artifacts and texts the way that producers approach recorded music, looking for fragments that might be appropriated and repurposed to form the basis of new works.

The pedagogical sampler, with a computer or without a computer, allows cultural criticism to save isolated moments and then juxtapose them as a final product. The student writer looks at the various distinct moments she has collected and figures out how these moments together produce knowledge. Just as DJs often search for breaks and cuts in the music that reveal patterns, so, too, does the student writer look for a pattern as a way to unite these moments into a new alternative argument and critique (465).

Rice advocates what he calls the “whatever” principle of sampling. In the hip-hop context, “whatever” can have two meanings. First, there is the conventional sense of the word, that everything is on the table, that anything goes. There is also the slang sense of “whatever” as a statement of defiance, indifference, and dismissal. In a pedagogical context, the “whatever” principle encourages us to be accepting of what is new and unexpected, and be dismissive of what is fake or irrelevant. As Missy Elliott (2002) puts it: “Whatever, let’s just have fun. It’s hip-hop, man, this is hip-hop.”

I asked Jamie Ehrenfeld, if she had written songs while getting her music degree, what kind of material might she have written? She responded:

I would think of bits of music in my head and then associate them with some other song I’d already heard and felt like nothing I could think of was really original, and I didn’t get that it’s okay that in writing a song having some elements of other songs can come together to make something new, and that actually being original is more of what existing pieces you weave together in addition to ‘original’ thought (personal communication, April 28 2017).

In other words, the sampling ethos might have validated the intuitive creative processes she was already spontaneously carrying out, whether she had realized those impulses in the form of digitally produced recordings or pencil-and-paper scores.

Can a work based on samples be wholly original? Perhaps not. But hip-hop slang offers a different standard of quality that may be more apposite: the idea of freshness. There are several different definitions of “fresh.” It can mean new or different; well-rested, energetic, and healthy-looking; or appealing food, water, or air. “Fresh” is also a dated slang term for impudence or impertinence. In hip-hop culture, “fresh” is one among many synonyms for “cool,” but it could be referencing any of the various original senses of the word: new, refreshing, appetizing, attractive, or sassy. Rather than evaluating music in terms of its originality, we might judge music by its freshness (Hein, 2015). A track that includes samples can not be wholly original by definition, but it can be fresh. It is this sense of making new meaning out of existing resources that animates the Fresh Ed curriculum (Miles et al, 2015), a culturally responsive teaching resource created by the Urban Arts Partnership. Rather than treating students as receptacles for information, Fresh Ed places new knowledge in familiar contexts, for example in the form of rap songs. When students are able to draw on their prior knowledge and cultural competencies, they are better equipped to engage and think critically.

Proposed methods

Luker (2008) describes the case that chooses you, or that you sample yourself into (131). My own trajectory as a musician and educator has made me an exemplar of the shortcomings of Eurocentric music pedagogy and the benefits of personal creativity through producing and songwriting; certainly it feels like this case chose me. Since my own motivations are borne out of subjective experience, and since my research questions were provoked by the experiences of others like me, my research into those questions must necessarily follow an interpretivist paradigm. In choosing methods aligning to that paradigm, I want to identify one that supports the use of music creation itself as a tool for inquiry into music pedagogy. One such method is Eisner’s (1997) model of educational inquiry by means of connoisseurship and criticism. Connoisseurship is the “ability to make fine-grained discriminations among complex and subtle qualities” (Eisner 1997, 63). Criticism is judgment that illuminates and interprets the qualities of a practice in order to transform it. As a subjective researcher, I am obliged to systematically identify my subjectivity  (Peshkin, 1988), and I view my role as connoisseur and critic in music as a source of clarity rather than bias.

Ethnography

An interpretivist paradigm is well supported by methods of ethnography, since participant observation and unstructured interviews dovetail exactly with a subjectivist epistemology. Ethnographers typically allow their methods to evolve over the course of the study, and can only define their procedures in retrospect, in the form of a narrative of what actually happened, rather than a detailed plan ahead of time. This form of research is iterative, like agile software development. Data comes in the form of interpretations of interpretations of interpretations, and in that sense is a “fiction”—not in the sense that it is counterfactual (we hope), but in the original sense of the word, a thing that is constructed. We must involve our imagination in constructing our interpretive fictions (Geertz, 1973).

Institutional ethnographers examine work settings and processes, combining observation with discourse analysis of texts, documents and procedures. The goal is to show how people in the workplace align their activities with structures that may originate elsewhere (Devault, 2006). This method asks us to seek out “ruling relations” (Smith 2005, 11), textually mediated connections and organizations shaping everyday life, especially those that are the most taken for granted. In so doing, we examine the ways that texts bind small social groups into institutions, and bind those together into larger power structures. This method is well suited to a profession like music teaching.

Taber (2010) combines autoethnography with institutional ethnography to tell the story of her own experience in the military, as an entry point into understanding the experience of other women. She questions whether researching the lives of others was a way to hide from her own problematic experience, and chooses instead to foreground her internal conflicts, using a “reflexivity of discomfort” (19). This is emblematic of the institutional ethnographic practice of examining aspects of organizations that their inhabitants find problematic, troubling or contradictory. Since the story of my own music education is one of internal conflict and discomfort, I expect a similar method to Taber’s to yield rich results.

Naturally, an inquiry into music education will involve some ethnomusicology. Given how technologically mediated hip-hop and other contemporary forms are, it will be useful to take on the lens of “technomusicology” (Marshall, 2017). Music educators who feel pressured to use computers in their practice quickly run up against the fact that digital audio tools are a poor fit for classical music. However, these tools are the most natural medium for hip-hop and other electronic dance musics. The technological and cultural issues are inseparable.

Hip-hop grows out orality and African-American Vernacular English. Therefore, it is prone to being dismissed by scholars working in a literate value system. Similarly, it is all too common to view AAVE through the lens of deprivationism, as a failure to learn “correct” English. To overcome this spurious attitude, we can employ an ethnopoetic approach. Speakers of AAVE are only linguistically “impoverished” because we institutionally deem them to be so, not because they have any difficulty communicating or expressing themselves (McDermott & Varenne, 1995). By the same token, classical music culture sees the lack of complex harmony and melody in African diasporic music like hip-hop as a shortcoming, a poverty of musical means. But the hip-hop aesthetic puts a premium on rhythm and timbre, and harmony functions mostly as a way to signpost locations within the cyclical metrical structure. In learning to value hip-hop on its own terms, we broaden our ability to understand other musical and cultural value systems as well.

Participatory research

Participatory research methods like cooperative inquiry and participatory action research treat research participants as collaborators, rather than as objects of study. The related method of constructivist instructional design puts these principles into action in the form of new technologies, experiences and curricula, the educational equivalent of critical theorists’ activism. When teachers and designers act as researchers, they function as participant observers. While I am an avid hip-hop fan and a dedicated student of it, I am ultimately a tourist. My research will therefore necessarily be incomplete unless it is a collaborative effort with members of hip-hop culture.

Instructional design as participatory research follows a Reflective and Recursive Design and Development (R2D2) model, based on the principles of recursion, nonlinearity and reflection (Willis, 2007). Designers test and prototype continually alongside users, and feed the results back into the next design iteration. This process for developing instructional material enables end users and experts to work jointly toward the end product. This loop of feedback and iteration is an example of reflective practice, made up of the “arts” of problem framing, implementation, and improvisation (Schön, 1987). These same arts are the ones used in musical problem-solving, both as a practitioner and educator. The Music Experience Design Lab follows a participatory design methodology in developing our technologies for music learning and expression, and the idea of using the same techniques to examine the broader social context of our work is quite appealing to me.

Narrative inquiry

There may be universal physical truths, but mental, emotional and social truths are contextual and particular. To examine these truths, then, we need verstehen, understanding of context, both historical and contemporary (Willis, 2007). To that end, we can draw on phenomenology, asking how humans view themselves and the world around them. This perspective attends to experience “from the neck down,” not just to cognition. We need to understand the bodily sensations of numbness, anxiety or anger that too many students feel in the music classroom, knowing that something is wrong but not knowing how to name it. For example, I spent my music graduate theory seminar in a continual low boil of rage, and it was only years later that I was able to point to the white supremacist ideology animating the curriculum as the source of this intense emotion. A number of my fellow musicians aligned with black music have described the same feelings. It is a primary research goal of mine to give those feelings a name and a clear target, so they can be put to work in the service of systemic change.

Bruner (1991) cites Vygotsky’s dictum that cultural products like language mediate our thought and shape our representations of reality. (This is certainly true of music.) Constructionists assume that we produce reality through the social exchange of meanings. We use language not as isolated individuals, but within social groups, organizations, institutions and cultures. Within our contexts, we speak as we understand it to be appropriate to speak (Galasinski & Ziólkowska, 2013). As narratives accrue into traditions, they take on a life of their own that can outlive their original context—this is a likely explanation for the persistence of classical music habitus far beyond the conservatory.

Close readers of narrative must study not only the syntactic content of the words themselves, but also their literary qualities, their tone (Riessman, 2008). There is a close parallel here with musicology. When we compare Julie Andrews’ performance of “My Favorite Things” in The Sound Of Music (1965) with the one recorded by John Coltrane (1961), it is like comparing the same text spoken by two very different speakers. We can perform a neat inverse of this process by examining the same musical performance across contexts; for example, comparing Tom Scott’s recording of Balin and Kantner’s “Today” (1967) with the sample of that recording that forms the centerpiece of Pete Rock & C.L. Smooth’s “They Reminisce Over You (T.R.O.Y.)” (1992). Here, the same performance gives rise to different musical meanings in different settings. We should be similarly attentive to the performative and contextual aspects of narrative.

Validity and reliability

If we are examining attitudes and interpretations rather than more easily observable “facts,” how do we ensure validity and reliability? In place of a search for straightforward logical explanations, we can instead build a case on Lyotardian paralogy, and “let contradictions remain in tension” (Lather 1993, 679), like the unresolved tritones enriching the blues and jazz. We should not expect to find tree-shaped hierarchies of explanation, but instead hold ourselves to a “rhizomatic” standard of validity. “Rather than a linear progress, rhizomatics is a journey among intersections, nodes, and regionalizations through a multi-centered complexity” (Lather 1993, 680). We can understand the complexities of music and schooling and race to have the topology of a network, not a tree. We should expect that when we pull on any part of the network, we will encounter a tangle.

In my research thus far, I have instinctively used reciprocity to treat my interviews more as two-way conversations. Such judicious use of self-disclosure can give rise to richer data. We can attain further reciprocity by showing participants field notes and drafts, building in “member checks” early on to ensure trustworthiness throughout the process. As feminist researchers, Harrison, MacGibbon and Morton (2001) hold attention to emotional aspects of the research and the relationships it entails as a key criterion of trustworthiness. This kind of emotionally aware collaborative/shared authorship aligns naturally with participatory research, and with hip-hop pedagogy. Larson (1997) argues that narrative inquiry gains greater validity by having the story-giver reflect on the transcript and analysis so they can revise or go deeper into their story. If a lived experience is an iceberg, then its initial retelling may just describe the tip. It takes reflection to bring more of the iceberg to the surface. We may therefore do better to examine a few icebergs thoroughly than to survey many tips.

Sample data and future research

Ed Sullivan Fellows (ESF) is a mentorship and artist development program run by the NYU Steinhardt Music Experience Design Lab. Participants are young men and women between the ages of 15 and 20, mostly low-SES people of color. They meet on Saturday afternoons at NYU to write and record songs; to get mentorship on the music business, marketing and branding; and to socialize. Sessions have a clubhouse feel, a series of ad-hoc jam sessions, cyphers, informal talks, and open-ended creativity. Conversations are as likely to focus on participants’ emotions, politics, social life and identity as they are on anything pertaining to music. I intend to conduct my research among hip-hop educators like Jamie and the other ESF mentors. They teach music concepts like song structure and harmony, but their project is much larger: to provide emotional support, to build resilience and confidence, to foster social connections across class and racial lines. Hein (2017) is a set of preliminary observations on ESF, showing the close connection between its musical and social values.

Conclusion

If music education is failing to address the needs of the substantial majority of students, it should be no wonder that enrollment and societal support are declining.

Every ‘failure’ to succeed in competition, every drop-out, and every student who is relieved to have compulsory music study behind them (including lessons enforced by parental fiat) represents not just a lack of ‘conversion’ to musical ‘virtue’ but gives such future members of the public compelling reason to doubt whether their music education has served any lasting purpose or value (Regelski 2009, 12).

Music educators’ advocacy efforts are mostly devoted to preserving existing methods and policies. However, these same methods and practices are driving music education’s irrelevance. At some point, advocacy starts to look less like a high-minded push for society’s interest, and more like an effort on behalf of music teachers’ self-interest.

Most (if not all) people have an inborn capacity and intrinsic motivation for engaging in music. However, that capacity and motivation need to be activated and nurtured by “musically and educationally excellent teachers and… inspiring models of musicing in contexts of welcoming, sustaining, and educative musical settings, including home and community contexts” (Elliott & Silverman 2015, 240). To restrict this opportunity to “talented” students is anti-democratic in Dewey’s sense. Good music serves particular human needs. One of those needs is aesthetic contemplation and appreciation of the Eurocentric canon. But there are many other legitimate ends that music education can pursue. In order to meet more students’ musical needs, we must embrace the musical culture of the present, and confront all the challenges of race and class that entails.

References

Anyon, J. (1980). Social Class and the Hidden Curriculum of Work. The Journal of Education, 162(1), 67–92.

Bell, A. P. (2014). Trial-by-fire : A case study of the musician – engineer hybrid role in the home studio. Journal of Music, Technology & Education, 7(3), 295–312.

Bledsoe, R. (2015). Music Education for All? General Music Today, 28(2), 18–22.

Bowman, W. (2007). Who is the “We”? Rethinking Professionalism in Music Education. Action, Criticism, and Theory for Music Education, 6(4), 109–131.

Bowman, W. (2016). Artistry, Ethics, and Citizenship. In D. Elliott, M. Silverman, & W. Bowman (Eds.), Artistic Citizenship: Artistry, Social Responsibility, and Ethical Praxis. New York: Oxford University Press.

Campbell, P. S., Connell, C., & Beegle, A. (2007). Adolescents’ expressed meanings of music in and out of school. Journal of Research in Music Education, 55(3), 220–236.

Cutler, C. (2004). Plunderphonia. In C. Cox & D. Warner (Eds.), Audio culture: Readings in modern music (pp. 138–156). London: Continuum International Publishing Group.

DeNora, T. (2000). Music in everyday life. New York: Cambridge University Press.

Devault, M. L. (2006). Introduction: What is Institutional Ethnography? Social Problems, 53(3), 294–298.

Dillon, S. (2007). Music, Meaning and Transformation: Meaningful Music Making for Life. Cambridge Scholars Publishing.

Doyle, J. L. (2014). Cultural relevance in urban music education: a synthesis of the literature. Applications of Research in Music Education, 32(2), 44–51.

Eisner, E. (1991). The enlightened eye: Qualitative inquiry and the enhancement of educational practice. Toronto: Macmillan.

Elliott, D. J., & Silverman, M. (2014). Music Matters: A Philosophy of Music Education (2nd ed.). Oxford: Oxford University Press.

Elpus, K., & Abril, C. R. (2011). High School Music Ensemble Students in the United States: A Demographic Profile. Journal of Research in Music Education, 59(2), 128–145.

Eno, B. (2004). The Studio As Compositional Tool. In C. Cox & D. Warner (Eds.), Audio culture: Readings in modern music (pp. 127–130). London: Continuum International Publishing Group.

Ester, D. P., & Turner, K. (2009). The impact of a school loaner-instrument program on the attitudes and achievement of low-income music students. Contributions to Music Education, 36(1), 53–71.

Finney, J. (2007). Music Education as Identity Project in a World of Electronic Desires. In J. Finney & P. Burnard (Eds.), Music education with digital technology. London: Bloomsbury Academic.

Harrison, J., MacGibbon, L., & Morton, M. (2001). Regimes of Trustworthiness in Qualitative Research: The Rigors of Reciprocity. Qualitative Inquiry, 7(3), 323–345.

Hein, E. (2015). Mad Fresh. NewMusicBox. Retrieved March 24, 2015, from http://www.newmusicbox.org/articles/mad-fresh/

Hein, E. (2017). A participant ethnography of the Ed Sullivan Fellows program. Retrieved May 9, 2017, from http://www.ethanhein.com/wp/2017/a-participant-ethnography-of-the-ed-sullivan-fellows-program/

Irby, D. J., & Hall, H. B. (2011). Fresh Faces, New Places: Moving Beyond Teacher-Researcher Perspectives in Hip-Hop-Based Education Research. Urban Education, 46(2), 216–240.

Kratus, J. (2016). Songwriting: A new direction for secondary music education. Music Educators Journal, 102(3), 60–65.

Kratus, J. (2007). Music Education at the Tipping Point. Music Educators Journal, 94(2), 42–48.

Kruse, A. J. (2016). Toward hip-hop pedagogies for music education. International Journal of Music Education, 34(2), 247–260.

Laitz, S. G. (2015). The complete musician: An integrated approach to tonal theory, analysis, and listening (4th ed.). Oxford University Press.

Lather, P. (1993). Fertile Obsession : Validity after Poststructuralism. The Sociological Quarterly, 34(4), 673–693.

Lorde, A. (1984). The master’s tools will never dismantle the master’s house. Sister Outsider: Essays and Speeches by Audre Lorde, 110–113.

Luker, K. (2008). Salsa Dancing into the Social Sciences: Research in an Age of Info-glut. Cambridge: Harvard University Press.

Madrid, A. L. (2017). Diversity, Tokenism, Non-Canonical Musics, and the Crisis of the Humanities in U.S. Academia, 7(2), 124–129.

Marshall, W. (2017). Technomusicology | Harvard Extension School. Retrieved May 8, 2017, from http://www.extension.harvard.edu/academics/courses/technomusicology/24318

Massey, D. S., & Denton, N. A. (1993). American apartheid: segregation and the making of the underclass. Cambridge: Harvard University Press.

Mcclary, S. (2000). Conventional Wisdom: The Content of Musical Form. University of California Press.

McDermott, R., & Varenne, H. (1995). Culture as Disability. Anthropology and Education Quarterly, 26(3), 324–348.

Miles, J., Hogan, E., Boland, B., Ehrenfeld, J., & Berry, L. (2015). Fresh Ed: A Field Guide to Culturally Responsive Pedagogy. New York: Urban Arts Partnership. Retrieved from http://freshed.urbanarts.org/fresh-field-guide/

Perry, I. (2004). Prophets of the Hood. Duke University Press.

Peshkin, A. (1988). In Search of Subjectivity–One’ s Own. Educational Researcher, 17–21.

Pomykala-Hein, M. (2017). Searching [Online musical score]. Retrieved May 5, 2017, from https://www.noteflight.com/scores/view/180d4db69af3646e6e70fae8002648d7f2048a7d

Regelski, T. A. (2009). The Ethics of Music Teaching as Profession and Praxis. Visions of Research in Music Education, 13(2009), 1–34.

Rice, J. (2016). The 1963 hip-hop machine: Hip-hop pedagogy as composition. College Composition and Communication, 54(3), 453–471.

Ruthmann, A. (2007). The Composers’ Workshop: An Approach to Composing in the Classroom. Music Educators Journal, 93(4), 38.

Schön, D. (1987). Teaching artistry through reflection in action. In Educating the reflective practitioner: Educating the reflective practitioner for teaching and learning in the professions (pp. 22–40). San Francisco: Jossey-Bass.

Sennett, R. (2008). The Craftsman (New Haven). Yale University Press.

Smith, D. (2005). Institutional Ethnography: A Sociology for People. Walnut Creek, CA: AltaMira Press.

Taber, N. (2010). Institutional ethnography, autoethnography, and narrative: an argument for incorporating multiple methodologies. Qualitative Research, 10(1), 5–25. http://doi.org/10.1177/1468794109348680

Thibeault, M. (2010). Hip-Hop, Digital Media, and the Changing Face of Music Education. General Music Today, 24(1), 46–49. http://doi.org/10.1177/1048371310379097

Turino, T. (2016). Music, Social Change, and Alternative Forms of Citizenship. In D. Elliott, M. Silverman, & W. Bowman (Eds.), Artistic Citizenship: Artistry, Social Responsibility, and Ethical Praxis (p. 616). New York: Oxford University Press.

van der Schyff, D., Schiavio, A., & Elliott, D. J. (2016). Critical ontology for an enactive music pedagogy. Action, Criticism, and Theory for Music Education, 15(5), 81–121.

Williams, D. A. (2011). The Elephant in the Room. Music Education: Navigating the Future, 98(1), 51–57.

Willis, J. W. (2007). Foundations of Qualitative Research: Interpretive and Critical Approaches. Thousand Oaks, CA: Sage.

Wise, R. (1965). The Sound Of Music. United States: 20th Century Fox.

Discography

Balin, M. and Kantner, P. (1967). Today [recorded by Tom Scott and The California Dreamers]. On The Honeysuckle Breeze [LP]. Santa Monica: Impulse! (1967)

Elliott, Missy (2002). Work It. On Under Construction [CD]. New York: Goldmind/Elektra. (November 12, 2002)

Lamar, Kendrick (2017). Pride. On DAMN. [CD/streaming]. Santa Monica, CA: Top Dawg/Aftermath/Interscope. (April 14, 2017)

Pete Rock & C.L. Smooth (1992). They Reminisce Over You (T.R.O.Y.). On Mecca and the Soul Brother [LP]. New York: Untouchables/Elektra. (April 2, 1992)

Rodgers, Richard and Hammerstein, Oscar (1959). My Favorite Things [recorded by John Coltrane]. On My Favorite Things [LP]. New York: Atlantic. (March, 1961)

Design for Real Life – QWERTYBeats research

Writing assignment for Design For The Real World with Claire Kearney-Volpe and Diana Castro – research about a new rhythm interface for blind and low-vision novice musicians

Definition

I propose a new web-based accessible rhythm instrument called QWERTYBeats.Traditional instruments are highly accessible to blind and low-vision musicians. Electronic music production tools are not. I look at the history of accessible instruments and software interfaces, give an overview of current electronic music hardware and software, and discuss the design considerations underlying my project.

QWERTYBeats logo

Historical overview

Acoustic instruments give rich auditory and haptic feedback, and pose little obstacle to blind musicians. We need look no further for proof than the long history of iconic blind musicians like Ray Charles and Stevie Wonder. Even sighted instrumentalists rarely look at their instruments once they have attained a sufficient level of proficiency. Music notation is not accessible, but Braille notation has existed since the language’s inception. Also, a great many musicians both blind and sighted play entirely by ear anyway.

Most of the academic literature around accessibility issues in music education focuses on wider adoption of and support for Braille notation. See, for example, Rush, T. W. (2015). Incorporating Assistive Technology for Students with Visual Impairments into the Music Classroom. Music Educators Journal, 102(2), 78–83. For electronic music, notation is rarely if ever a factor.

Electronic instruments pose some new accessibility challenges. They may use graphical interfaces with nested menus, complex banks of knobs and patch cables, and other visual control surfaces. Feedback may be given entirely with LED lights and small text labels. Nevertheless, blind users can master these devices with sufficient practice, memorization and assistance. For example, Stevie Wonder has incorporated synthesizers and drum machines in most of his best-known recordings.

Most electronic music creation is currently done not with instruments, but rather using specialized software applications called digital audio workstations (DAWs). Keyboards and other controllers are mostly used to access features of the software, rather than as standalone instruments. The most commonly-used DAWs include Avid Pro Tools, Apple Logic, Ableton Live, and Steinberg Cubase. Mobile DAWs are more limited than their desktop counterparts, but are nevertheless becoming robust music creation tools in their own right. Examples include Apple GarageBand and Steinberg Cubasis. Notated music is commonly composed using score editing software like Sibelius and Finale, whose functionality increasingly overlaps with DAWs, especially in regard to MIDI sequencing.

DAWs and notation editors pose steep accessibility challenges due to their graphical and spatial interfaces, not to mention their sheer complexity. In class, we were given a presentation by Leona Godin, a blind musician who records and edits audio using Pro Tools by means of VoiceOver. While it must have taken a heroic effort on her part to learn the program, Leona demonstrates that it is possible. However, some DAWs pose insurmountable problems even to very determined blind users because they do not use standard operating system elements, making them inaccessible via screen readers.

Technological interventions

There are no mass-market electronic interfaces specifically geared toward blind or low-vision users. In this section, I discuss one product frequently hailed for its “accessibility” in the colloquial rather than blindess-specific sense, along with some more experimental and academic designs.

Ableton Push

Push layout for IMPACT Faculty Showcase

Ableton Live has become the DAW of choice for electronic music producers. Low-vision users can zoom in to the interface and modify the color scheme. However, Live is inaccessible via screen readers.

In recent years, Ableton has introduced a hardware controller, the Push, which is designed to make the software experience more tactile and instrument-like. The Push combines an eight by eight grid of LED-lit touch pads with banks of knobs, buttons and touch strips. It makes it possible to create, perform and record a piece of music from scratch without looking at the computer screen. In addition to drum programming and sampler performance, the Push also has an innovative melodic mode which maps scales onto the grid in such a way that users can not play a wrong note. Other comparable products exist; see, for example, the Native Instruments Maschine.

There are many pad-based drum machines and samplers. Live’s main differentiator is its Session view, where the pads launch clips: segments of audio or MIDI that can vary in length from a single drum hit to the length of an entire song. Clip launching is tempo-synced, so when you trigger a clip, playback is delayed until the start of the next measure (or whatever the quantization interval is.) Clip launching is a forgiving and beginner-friendly performance method, because it removes the possibility of playing something out of rhythm. Like other DAWs, Live also gives rhythmic scaffolding in its software instruments by means of arpeggiators, delay and other tempo-synced features.

The Push is a remarkable interface, but it has some shortcomings for blind users. First of all, it is expensive, $800 for the entry-level version and $1400 for the full-featured software suite. Much of its feedback is visual, in the form of LED screens and color-coded lighting on the pads. It switches between multiple modes which can be challenging to distinguish even for sighted users. And, like the software it accompanies, the Push is highly complex, with a steep learning curve unsuited to novice users, blind or sighted.

The aQWERTYon

Most DAWs enable users to perform MIDI instruments on the QWERTY keyboard. The most familiar example is the Musical Typing feature in Apple GarageBand.

GarageBand musical typing

Musical Typing makes it possible to play software instruments without an external MIDI controller, which is convenient and useful. However, its layout counterintuively follows the piano keyboard, which is an awkward fit for the computer keyboard. There is no easy way to distinguish the black and white keys, and even expert users find themselves inadvertantly hitting the keyboard shortcut for recording while hunting for F-sharp.

The aQWERTYon is a web interface developed by the NYU Music Experience Design Lab specifically intended to address the shortcomings of Musical Typing.

aQWERTYon screencap

Rather than emulating the piano keyboard, the aQWERTYon draws its inspiration from the chord buttons of an accordion. It fills the entire keyboard with harmonically related notes in a way that supports discovery by naive users. Specifically, it maps scales across the rows of keys, staggered by intervals such that each column forms a chord within the scale. Root notes and scales can be set from pulldown menus within the interface, or preset using URL parameters. It can be played as a standalone instrument, or as a MIDI controller in conjunction with a DAW. Here is a playlist of music I created using the aQWERTYon and GarageBand or Ableton Live:

The aQWERTYon is a completely tactile experience. Sighted users can carefully match keys to note names using the screen, but more typically approach the instrument by feel, seeking out patterns on the keyboard by ear. A blind user would need assistance loading the aQWERTYon initially and setting the scale and root note parameters, but otherwise, it is perfectly accessible. The present project was motivated in large part by a desire to make exploration of rhythm as playful and intuitive as the aQWERTYon makes exploring chords and scales.

Soundplant

The QWERTY keyboard can be turned into a simple drum machine quite easily using a free program called Soundplant. The user simply drags audio files onto a graphical key to have it triggered by that physical key. I was able to create a TR-808 kit in a matter of minutes:

Soundplant with 808 samples

After it is set up and configured, Soundplant can be as effortlessly accessible as the aQWERTYon. However, it does not give the user any rhythmic assistance. Drumming in perfect time is an advanced musical skill, and playing drum machine samples out of time is not much more satisfying than banging on a metal bowl with a spoon out of time. An ideal drum interface would offer beginners some of the rhythmic scaffolding and support that Ableton provides via Session view, arpeggiators, and the like.

The Groove Pizza

Drum machines and their software counterparts offer an alternative form of rhythmic scaffolding. The user sequences patterns in a time-unit box system or piano roll, and the computer performs those patterns flawlessly. The MusEDLab‘s Groove Pizza app is a web-based drum sequencer that wraps the time-unit box system into a circle.

Groove Pizza - Bembe

The Groove Pizza was designed to make drum programming more intuitive by visualizing the symmetries and patterns inherent in musical-sounding rhythms. However, it is totally unsuitable for blind or low-vision users. Interaction is only possible through the mouse pointer or touch, and there are no standard user interface elements that can be parsed by screen readers.

Before ever considering designing for the blind, the MusEDLab had already considered the Groove Pizza’s limitations for younger children and users with special needs: there is no “live performance” mode, and there is always some delay in feedback between making a change in the drum pattern and hearing the result. We have been considering ways to make a rhythm interface that is more immediate, performance-oriented and tactile. One possible direction would be to create a hardware version of the Groove Pizza; indeed, one of the earliest prototypes was a hardware version built by Adam November out of a pizza box. However, hardware design is vastly more complex and difficult than software, so for the time being, software promises more immediate results.

Haenselmann-Lemelson-Effelsberg MIDI sequencer

This experimental interface is described in Haenselmann, T., Lemelson, H., & Effelsberg, W. (2011). A zero-vision music recording paradigm for visually impaired people. Multimedia Tools and Applications, 5, 1–19.

Haenselmann-Lemelson-Effelsberg MIDI sequencer

The authors create a new mode for a standard MIDI keyboard that maps piano keys to DAW functions like playback, quantization, track selection, and so on. They also add “earcons” (auditory icons) to give sonic feedback when particular functions have been activated that normally only give graphical feedback. For example, one earcon sounds when recording is enabled; another sounds for regular playback. This interface sounds promising, but there are significant obstacles to its adoption. While the authors have released the source code as a free download, that requires a would-be user to be able to compile and run it. This is presuming that they could access the code in the first place; the download link given in the paper is inactive. It is an all-too-common fate of academic projects to never get widespread usage. By posting our projects on the web, the MusEDLab hopes to avoid this outcome.

Statement

Music education philosophy

My project is animated by a constructivist philosophy of music education, which operates by the following axiomatic assumptions:

  • Learning by doing is better than learning by being told.
  • Learning is not something done to you, but rather something done by you.
  • You do not get ideas; you make ideas. You are not a container that gets filled with knowledge and new ideas by the world around you; rather, you actively construct knowledge and ideas out of the materials at hand, building on top of your existing mental structures and models.
  • The most effective learning experiences grow out of the active construction of all types of things, particularly things that are personally or socially meaningful, that you develop through interactions with others, and that support thinking about your own thinking.

If an activity’s challenge level is beyond than your ability, you experience anxiety. If your ability at the activity far exceeds the challenge, the result is boredom. Flow happens when challenge and ability are well-balanced, as seen in this diagram adapted from Csikszentmihalyi.

Flow

Music students face significant obstacles to flow at the left side of the Ability axis. Most instruments require extensive practice before it is possible to make anything that resembles “real” music. Electronic music presents an opportunity here, because even a complete novice can produce music with a high degree of polish quickly. It is empowering to use technologies that make it impossible to do anything wrong; it frees you to begin exploring what you find to sound right. Beginners can be scaffolded in their pitch explorations with MIDI scale filters, Auto-Tune, and the configurable software keyboards in apps like Thumbjam and Animoog. Rhythmic scaffolding is more rare, but it can be had via Ableton’s quantized clip launcher, by MIDI arpeggiators, and using the Note Repeat feature on many drum machines.

QWERTYBeats proposal

My project takes drum machine Note Repeat as its jumping off point. When Note Repeat is activated, holding down a drum pad triggers the corresponding sound at a particular rhythmic interval: quarter notes, eighth notes, and so on. On the Ableton Push, Note Repeat automatically syncs to the global tempo, making it effortless to produce musically satisfying rhythms. However, this mode has a major shortcoming: it applies globally to all of the drum pads. To my knowledge, no drum machine makes it possible to simultaneously have, say, the snare drum playing every dotted eighth note while the hi-hat plays every sixteenth note.

I propose a web application called QWERTYBeats that maps drums to the computer keyboard as follows:

  • Each row of the keyboard triggers a different drum/beatbox sound (e.g. kick, snare, closed hi-hat, open hi-hat).
  • Each column retriggers the sample at a different rhythmic interval (e.g. quarter note, dotted eighth note).
  • Circles dynamically divide into “pie slices” to show rhythmic values.

The rhythm values are shown below by column, with descriptions followed by the time interval as shown as a fraction of the tempo in beats per minute.

  1. quarter note (1)
  2. dotted eighth note (3/4)
  3. quarter note triplet (2/3)
  4. eighth note (1/2)
  5. dotted sixteenth note (3/8)
  6. eighth note triplet (1/3)
  7. sixteenth note (1/4)
  8. dotted thirty-second note (3/16)
  9. sixteenth note triplet (1/6)
  10. thirty-second note (1/8)

By simply holding down different combinations of keys, users can attain complex syncopations and polyrhythms. If the app is synced to the tempo of a DAW or music playback, the user can perform good-sounding rhythms over any song that is personally meaningful to them.

The column layout leaves some unused keys in the upper right corner of the keyboard: “-“, “=”, “[“, “]”, “”, etc. These can be reserved for setting the tempo and other UI elements.

The app defaults to Perform Mode, but clicking Make New Kit opens Sampler mode, where users can import or record their own drum sounds:

  • Keyboard shortcuts enable the user to select a sound, audition it, record, set start and end point, and set its volume level.
  • A login/password system enables users to save kits to the cloud where they can be accessed from any computer. Kits get unique URL identifiers, so users can also share them via email or social media.

It is my goal to make the app accessible to users with the widest possible diversity of abilities.

  • The entire layout will use plain text, CSS and JavaScript to support screen readers.
  • All user interface elements can be accessed via the keyboard: tab to change the keyboard focus, menu selections and parameter changes via the up and down arrows, and so on.

Perform Mode:

QWERTYBeats concept images - Perform mode

Sampler Mode:

sampler-mode

Mobile version

The present thought is to divide up the screen into a grid mirroring the layout of the QWERTY keyboard. User testing will determine whether this will produce a satisfying experience.

QWERTYDrum - mobile

Prototype

I created a prototype of the app using Ableton Live’s Session View.

QWERTYBeats - Ableton prototype

Here is a sample performance:

There is not much literature examining the impact of drum programming and other electronic rhythm sequencing on students’ subsequent ability to play acoustic drums, or to keep time more accurately in general. I can report anecdotally that my own time spent sequencing and programming drums improved my drumming and timekeeping enormously (and mostly inadvertently.) I will continue to seek further support for the hypothesis that electronically assisted rhythm creation builds unassisted rhythmic ability. In the meantime, I am eager to prototype and test QWERTYBeats.