Designing Technology & Experiences for Music Making, Learning, & Engagement

This Fall I will be teaching a graduate course at NYU called Designing Technologies and Experiences for Music Making, Learning, and Engagement. This course is heavily inspired by the Hack Day process, but applied over the span of a semester-long course. Students from across the many programs within the NYU Department of Music and Performing Arts Professions will work together individually and in teams to develop a technology and/or experience that that they will iterate at least twice over the course of the semester with a specified audience/group of stakeholders. Students will read articles about and case studies of best practices in music education, meaningful engagement, experience design, technology development and entrepreneurialism, and meet regularly with guest presenters from industry and education. At the end of the course, students will present their projects to a panel of music educators and industry representatives for feedback. Selected students will have the opportunity to compete for scholarships to work within my research group and some of the industry sponsors during the Spring 2014 semester to potentially license and commercialize their ideas and projects.

In this course, we will be implementing a research & development process designed by Andrew R. Brown called Software Development as (Music Education) Research (SoDaR). This process was piloted and used throughout the development of the Jam2Jam networked media jamming software project led by the late Steve Dillon. This process actively involves the end users of a particular piece of software in the design process at all stages. The field of music education technology is just now starting to move toward this end, where in the past educators were often marketed music technologies designed for professional musicians (e.g., professional keyboard synthesizer, Finale, Sibelius, Reason, ProTools, Ableton, etc.). It’s notable that relatively new technologies NoteflightMusicFirst, and MusicDelta have engaged educators in the design and refinement of their tools, and see music educators and students as their primary user audience.

Exploring New Media Musically and Creatively

images

I was very excited to receive the hard copy of the new book Teaching Music Creatively by Pamela Burnard and Regina Murphy in the mail today. I contributed a chapter to this book entitled Exploring new media musically and creatively where I put forward two creative projects for the primary classroom that get pupils engaged in making music with new media.

Here’s an excerpt from the chapter abstract:

This chapter provides an introduction to projects and tools for exploring the creative dimensions of new media with primary pupils. I begin with an introduction to creative musicianship with new media, followed by an overview of tools for creating and being creative with new media. These tools are discussed in the context of practical projects and creative strategies for teacher and pupil exploration within primary classrooms. The first project I share focuses on a mobile app for Apple’s iDevices – the application Singing Fingers, which enables children to “finger paint in sound” connecting physical gesture, drawing, and sound. The second project focuses on creative performing, improvising and composing experiences with the Scratch multimedia programming environment created by the Lifelong Kindergarten Group at the MIT Media Lab.

Though specifically written as a resource for primary generalist teachers in the UK and Ireland, this book has a number of international contributors including Rena Upitis (Canada), Deborah Blair (USA), Kathryn Marsh (Australia), Jenny Boyack (New Zealand), Gillian Howell (Australia), Emily Okuno (Kenya), Marcelo Giglio (Switzerland), James Biddulph (England), and Jane Wheeler (England). Give it a browse and hopefully add it to your collection!

To Cite the Chapter: Ruthmann, S. A. (2013). Exploring new media musically and creatively. In P. Burnard & R. Murphy, Teaching Music Creatively (pp. 85-97). London: Routledge. ISBN: 978041565606.

A look at the human-centered design process behind Music Education Hack 2013

In advance of the first Music Education Hack event, the leaders at iZone/InnovateNYC and Spotify engaged a group of NYC music teachers in IDEO’s human-centered design process. This process explored the question: Can technology transform music education in NYC schools?

In a school system as large as New York City’s, a lot of money is spent on technologies in classrooms. In the past, the decisions around which technologies were deployed to classrooms were often controlled by a few very upper administrators who purchased the technologies in bulk and deployed them to schools. Once deployed, that was that. We all can probably point to points in our own experiences when we have witnessed this. Also, we probably know of times when we or our colleagues were not engaged in a discussion around the selection of those technologies. As a result, unused copies of expensive software or hardware often went unused, and money wasted.

In direct challenge to this model, Music Education Hack emerged as a test case to engage teachers in directly providing input into the development of new technologies that may in the future make their way into NYC classrooms. As a result of IDEO’s human-centered design process, the following guidelines emerged and were presented to the group of hackers present at Music Education Hack:

  1. Teachers need to convey musical concepts and skills to students who have different styles of learning and levels of ability. What if there were an app that helps them explain volume and pitch?
  2. Students need help with short-term motivation to manage and maintain a long-term commitment to the practice and study of music? What if there were an app that made music homework and practice more fun?
  3. Classmates need to collaborate with one another to learn and play music. What if there were an app that helps us teach music to each other?
  4. Parents and guardians want to support their kids’ music education outside the classroom. What if there were an app that helps them learn music with their kids?
  5. Administrators need to record and report student accomplishments in music education. What if there were an app that helps us track their students’ progress?

As you can see, the framing of the challenge to Music Education Hack participants focused on solutions for teachers, students, classmates, parents and guardians, and administrators. The majority of hacks addressed at least one of the above items. Here are the judging criteria used to evaluate the hacks that won the 1st, 2nd, and 3rd prizes:

  • Integration of music and education: Does this advance the fields of music and education? Is the application useful to a student, teacher, or musician?
  • Level of Innovation: Is the application unique and innovative? Does it bring new functionality to the music and education space?
  • Depth of integration with partner APIs: Does the application take advantage of the functionality of the partner APIs?
  • User Experience: Is the application easy to use? Is the application aesthetically pleasing?

It is still unknown as to whether or not any of the hacks created at Music Education Hack will result in technologies that will be widely deployed in the NYC schools. However, this process is an intriguing and disruptive innovation that seeks to inspire the design and deployment of music technologies that music educators, administrators, students, and parents/guardians will actually use and benefit. This push is in part driven from Mayor Bloomberg as part of a push to spend NYC Department of Education resources more efficiently.

It is my understanding that the developers of some of the hacks (most all had ongoing feedback during the hack event from music educators) will be invited to explore further development and possibly adoption within NYC schools or beyond. I’ll keep you posted as I hear more about this.

Music, Creativity and Technology: An Example from Actual Practice

Step 1: Find music notation scribbled on a bathroom wall.

Step 2: Take a picture of it with your cell phone camera and post it online to ImageShack:

http://img120.imageshack.us/my.php?image=picture1ix9.jpg

Step 3: Post a link to that photo on Reddit for others to see and discuss:

http://www.reddit.com/r/pics/comments/7bqjc/music_found_in_the_toilet/

Step 4: Transcribe melody into Noteflight and create a custom arrangement of “Toilet Melody”:

click on Noteflight logo to launch score at Noteflight.com

Step 5: Repost Noteflight arrangement back on Reddit for others to discuss:

http://www.reddit.com/r/pics/comments/7bqjc/music_found_in_the_toilet/

Credits:
Photo = TrippingChilly
Music Arrangement = Skynare

If this is how young people are using technology in their lives, how can we draw on this in the classes we teach?

Bonus points for identifying the origin of the tune in “Toilet Melody.” :)

Using Noteflight in and outside of the music classroom

Earlier this month Evan Tobias posted about Noteflight, a new online flash-based notation application available at http://www.noteflight.com/. Over the past few weeks I have been exploring this software with college students in my Technology in Music Education course and with high school students enrolled in a beginning piano class at Lowell High School (LHS). Those of you who know me know how apprehensive I am when it comes to using notation software with students in general music or other technology classes in K-12 schools. Most of my concern centers around the common conflation of “notation software” with “composing software.” All too often I see teachers using notation software as a technological endpoint rather than as a means to the musical end of live performance. However, Noteflight is not your ordinary notation software.

What interests me about Noteflight is not the notation component. Instead, it is in the social tools that surround the notation engine. When you sign up at Noteflight.com (currently free) you create personal profile, just like you would at a social networking site like Facebook, MySpace or custom sites created at Ning.com. Once signed in, you can create a new score, view existing scores, or scores created by other users.

Picture 2Built in to the web application is the ability to share your scores with other users. These scores can be easily embedded just like a YouTube video in a class website. The embedded score can be played back by clicking on the play button and additional interactive functions are being planned which could be helpful in guided listening activities. Coming from a constructivist perspective, this functionality enables teachers to give students the opportunity to share their musical understanding in interactive ways within and beyond class time. For example, a band director could post a Noteflight score without added articulation. Students could then be assigned to add their own articulations to the score. During the next class, the students and director could choose a few scores to play through. This approach gives students the opportunity to make creative articulation decisions as composers, rather than traditionally learning it through listening and performing.

A variation on this assignment could be to post an audio file of a musical line performed with different articulations. Below the audio file, a director could post the notation for that performance, but again without articulation added. As an assessment, students could then open the score and add articulations that in their mind matched the recorded performance.

Right now, there are some limitations to accomplishing this, but I’ve been assured by Joe Berkovitz, CEO of Noteflight, that these functions are currently in development.

Picture 1
This screenshot shows the “version” function for Noteflight. As you work on a score in Noteflight, it periodically saves a snapshot of your piece and gives you access to it as a different “Version.” If you open your score up to be added to by others, their versions show up in this box as well. At any point you can go back (revert) to a prior version. This is a cool function, not only because you can go back, but as a window into your students’ compositional processes. Though not a full account of their process, these snapshots can provide an opportunity to have discussions with your students about the changes they made in their composition and are great starting points for assessment.

Here’s a short piece I notated in Noteflight:

Right now, the interactivity is limited to simple whole piece playback and playback within measures (click above the measure). Soon, functions will be added that will enable the composer to add additional interactivity through scripting. Very cool. :)

My students and Noteflight

My college students have been using Noteflight with beginning piano students at Lowell High School (LHS) for the past few weeks. Students in my class created incomplete duets to be co-composed and performed with their partner students at LHS. The music teacher at LHS has for the most part have been using Alfred’s Adult Beginner Piano book to structure the curriculum. My college students wanted to add a composing/creativity aspect to the lessons. To do this, they created simple piano scores with either a chord progression in the bass clef or a melody in the treble clef (or some combination of the two) as a compositional frame to help scaffold the LHS students. Because the scores are online and viewable by the LHS students and my college students, both can practice alone and make edits to their duet scores. Tomorrow, they will meet again in person for a final run through and performance for the class. I’ll post some of the pieces and performances here soon.

Because Noteflight is an online application, the potential for collaborative work and learning with other students is high. I’m in the middle of planning a distance composing project with another school later in the term through Noteflight. Facilitated by a custom Ning.com social network, students at LHS will notate compositions in Noteflight and share them with other students at a distance site. Ning will enable them to post their files and provide peer comment and critique. This use is inspired in part by the work at the Vermont MIDI Project, but instead centers on the students as providers of compositional critique and feedback, rather than professional composers.

I’m very excited to see how this technology develops. If you are interested in collaborative projects using Noteflight with your students, drop me an email.

Copyright: Ben Stein vs. Yoko Ono – Implications for “fair use” in music education?

Caveat # 1: I am not a lawyer and do not pretend to be one.

Today, I read an article posted on Ars Technica written by Timothy Lee detailing a recent “fair use” Copyright decision by Judge Sidney Stein of the U.S. District Court – Southern District of New York.

From the article:

Judge Stein’s task wasn’t to critique the dubious logic of this segment, but to evaluate the narrower question of whether the film’s use of “Imagine” is fair under copyright law. He noted that the film was focused on a subject of public interest, and that the film was commenting on Lennon’s anti-religious message. The excerpting of copyrighted works for purpose of “comment and criticism” is explicitly protected by the Copyright Act, and Judge Stein ruled that this provision applied in this case.

The decision quotes extensively from Bill Graham Archives v. Dorling Kindersley, a 2006 decision that allowed the reprinting of reduced-size versions of several historical posters used in a coffee-table book about the Grateful Dead. In that case, as in this one, the alleged infringers had used the works in a commercial product, but the US Court of Appeals for the Second Circuit found that “courts are more willing to find a secondary use fair when it produces a value that benefits the broader public interest.” Whatever the merits of its argument, Expelled is clearly commentary on an issue of public concern, and the use of “Imagine” was central to its argument. Those facts weighed heavily in favor of a finding of fair use.

Stein and company were defended by lawyers from Stanford’s Fair Use Project. In a blog post announcing their decision to take the case, executive director Anthony Falzone wrote that “The right to quote from copyrighted works in order to criticize them and discuss the views they represent lies at the heart of the fair use doctrine,” and argued that Ono’s actions threaten free speech.

This decision and the 2006 decision referenced above cause me to ask a few questions regarding the implications for music education:

In the 2006 decision, the use of reduced sized Grateful Dead posters was upheld as “fair use” within a commercial product because “courts are more willing to find a secondary use fair when it produces a value that benefits the broader public interest.”

In the Sidney Stein decision, the use of an excerpt from John Lennon’s Imagine used in a commercial film for the purpose of criticizing and commenting on issues that “benefit the broader public interest.”

So, what are the implications of using copyrighted samples or excerpts of commercial music or videos as part of our students’ educational pursuits? Is careful musical and educational use of commercial music and video in school projects of “benefit to the broader public interest?” If our students are utilizing these materials (including YouTube videos) for the purpose of artistic, musical “comment and criticism,” would that not also be considered “fair use” in light of these decisions?

What is particularly interesting to me is that both of the approved uses described above – using a copyrighted image in reduced resolution and using an excerpt of a copyrighted and performance-righted musical recording – were found to be “fair use” in two commerical settings. Also, both uses of copyrighted material seem to have been interpreted b the Judges as a “transformative” use (see Wikipedia entry on Fair Use). It would seem to me (again I am NO lawyer) that similar uses and creation of original multimedia using music and popular commercial and non-commercial video for “comment and criticism” of “benefit to the broader public interest” where the work has been “transformed” and not wholly-duplicated within an non-profit educational setting of a school would now be permissible as documented by the above case law.

Let’s take a look at the “fair use” section of the Copyright Act of 1976, 17 U.S.C. § 107:

Notwithstanding the provisions of sections § 106 and § 106A, the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright. In determining whether the use made of a work in any particular case is a fair use the factors to be considered shall include:

  1. the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;
  2. the nature of the copyrighted work;
  3. the amount and substantiality of the portion used in relation to the copyrighted work as a whole;
  4. and the effect of the use upon the potential market for or value of the copyrighted work.

The fact that a work is unpublished shall not itself bar a finding of fair use if such finding is made upon consideration of all the above factors.

Since the above uses were found to be “fair use” within commercial settings, factor #1 in the Copyright Act of 1976 would seem to provide students and teachers working in an educational context even more protection under “fair use.” I find the Sidney Stein ruling of particular importance to music educators because it provides case law that extends the “fair use” of images to copyrighted and performance-righted musical recordings.

In light of the cases described here, I feel more comfortable letting my students use copyrighted images and musical excerpts in the creative and educational work they do in my K-College music and music ed courses, with the following caveats:

  1. The use of the works is in part, and not in whole (e.g., reduced resolution or size)
  2. The use of the works for the purpose of “criticism and commentary”
  3. The use and creation of the works results in a “value that benefits the public interest”
  4. The use of the works is “transformative” such as in a parody or for “criticism and commentary”
  5. The use of the works do not devalue or negatively impact the market of the original copyrighted works

And, I might even be inclined to allow them to put together a compilation CD or DVD and sell them as a fundraiser….

What do you think?

Spring Cleaning and New Beginnings

Now that the semester is finished, papers marked, and grades assigned, I am starting to clean out my office at Indiana State. However, instead of sorting, reorganizing and planning for next year, I am boxing everything up in preparation for my upcoming move to beautiful Lowell, Massachusetts.

Beginning September 1, I will join the faculty at the University of Massachusetts Lowell. I am so excited to join my good friend and colleague, Dr. Gena Greher, at UMass Lowell. Over the past 6 years, she has helped to build a phenomenal music teacher preparation program. One of the most impressive aspects of the program is the depth and breadth of the partnerships among the music department and local schools. In each music education course, students have extensive experiences in the local schools applying what they learn on campus to real-life situations with real students. During one of my campus visits, I was impressed by the number of local K-12 students on campus after school. Almost every night of the week there are local students participating in after school honor ensemblescreative sound play classes and the UML String Project. Many schools have similar programs, but none that I know of are infused so extensively in the local community. And, very few music education programs truly integrate hands-on field experiences as extensively as UML.

Also of note are the innovative general education courses that bring together the arts, sciences and local community. Students can take interdisciplinary courses like PerformamaticsArtBotics, and Radical Design. Each of these courses have major service learning components interfacing UML students with the local community. One project I hope to become involved with is their Assistive Technology Program. It would be great to work with the engineering students on developing music specific assistive technologies for special needs students. I wish I had these kinds of courses in my undergrad!

UML GUI programming and Music Education students working on creating graphic notation software
UML Music Education and GUI Programming students working together to create
original graphic notation software in the Spring 2008 Performamatics course.
This Fall I will be teaching courses in music education research and technology in music education. I get a little bit of relief in my teaching load so that I can spend some time in the local schools getting to know the program and area. My Technology in Music Education students will be working closely with music technology classes at Lowell High School exploring innovative ways of using technology in their own teaching through working with real students and teachers in real classrooms. I’ll be updating this blog much more regularly as a part of that course in particular.

Dr. Greher and I are already starting to brainstorm new professional development and Masters-level courses in composing and technology for Summer 2009. Dr. Greher brings extensive experience in technology and creative musicianship as a former music producer for the advertising industry and from her work with the Teachers College Creative Arts Lab in New York City. Stay tuned for more information in the coming months!

With the change in job comes a change in contact information. My new email address is Alex_Ruthmann @ uml.edu. I will continue to check my ISU email through the end of the summer, however.