I seek to understand how listeners perceive musical structures in real time and to develop computational models of music perception and cognition. My research primarily focuses on modeling high-level aspects of music perception such as tension and large-scale structure. I am also interested in exploring how computational models of music perception can be incorporated into computer applications that facilitate and assist in musical creativity.

Music Perception and Cognition

Modeling musical tension and narrative structure
Musical tension is a high-level concept that is difficult to formalize due to its subjective and multi-dimensional nature. This work attempts to describe tension globally, rather than focusing on a subset of features. The goal is to develop a new theory based on the interaction of disparate musical features in real time, which can then can be implemented as a computational model. These musical parameters can include any type of feature, including harmony, pitch height, melodic expectation, dynamics, onset frequency, tempo, rhythmic regularity, and timbre. Tension is also an important factor in conveying narrative structure in music. Current work extends to the investigation of crossmodal (audiovisual) tension. [Code available on github]

Comparing music and speech
Tonal processing provides a means to better understand the similarities between speech and music perception. Unlike most work on music and language, which entails comparisons on a syntactic level, our work has focused on temporal aspects of the two domains in question. In particular, we have examined the timescales at which key identification is possible and used key-finding as a musical analog to word identification in order to investigate the similarities and differences between music and speech processing. Other related studies include the exploration of listener perception of hierarchical structure in music in relation to the cortical overlap between music and language processing at multiple timescales.

Perception of harmony and tonal structures
This research addresses a number of questions regarding the perception of tonal structures:
Perception of modulation. When any modulation occurs, what are the structural conditions necessary to strengthen the perception of the new key and how long does it take before a listener is acclimated to the new key? How long does a modulation to a foreign key have to last to negate the effect of tonal closure caused by an eventual return to the original key?
Pitch memory. How is pitch memory affected by a tonal context?
More generally, how can we use tonal structures to design experiments that illuminate our understanding of memory?
All of these questions pertain to question of the strength of tonal hierarchies and the mind’s ability to retain prolongational structures over significant spans of time. A goal of this project is to examine this issue in more detail and offer a perspective that incorporates the effect of time and cognitive constraints such as memory.

Computer Music Applications

Computer-assisted and algorithmic composition systems
This is a general research area that encompasses many of the ongoing projects in the lab. The goal is to incorporate cognitive models into software applications that help composers in the creative process. The focus is on designing intelligent systems that help composers generate musical material, on both small and large scales, as well as automate any aspects of the compositional process once material has been generated.

Graphical Score Interface
Graphical Score Interface (GSI) is a software environment that facilitates music composition by providing users with graphical tools for visualizing, manipulating, and generating high-level musical structures. The core design elements are based on Hyperscore, an earlier software application I designed to help users with little or no musical training to compose music. A tension model within GSI serves as a way for users to visualize and shape the dynamic flow of their compositions from a large-scale perspective. Color is mapped to motivic material, which can be designated by the user or automatically determined by the system using an algorithm that extracts recurring melodic patterns from both audio and symbolic representations of music. The ultimate goal of GSI is to enable users of all musical backgrounds to create, edit, and analyze musical material through the assistance of an intelligent musical system based on models of music cognition.

Automatic analysis of musical patterns
This purpose of this project is to develop algorithms that automatically extracts musical patterns (motives) from music. The approach combines perceptual grouping principles with data mining techniques using audio or score-based representation of music as input. The methods are evaluated by comparing their output to human-annotated data.