Juan Pablo Bello

Music Information Retrieval, Machine Listening, Audio Signal Processing

MIR

MPATE-GE 2623 Music Information Retrieval

Meetings (Fall 2018): Mondays 4.55 – 6.35pm, Studio F, 8th floor Ed. building.

Office hours: Tuesdays 2-5pm, Room 626, 6th floor, 35 W 4th street

__________________________________________________________________________________________

Overview: This course gives a comprehensive overview of research on the multi-disciplinary field of Music Information Retrieval (MIR). MIR uses knowledge from areas as diverse as signal processing, machine learning, information and music theory. The course will explore how this knowledge can be used for the development of novel methodologies for browsing and retrieval on large music collections, a hot topic given recent advances in online music distribution and searching. Emphasis would be given to audio signal processing techniques.

Goals: Students will undergo advanced training in techniques for automatic music analysis and retrieval. The knowledge that they will acquire, will be relevant for future careers in the music generation, processing, recording, reproduction and distribution industries, with special emphasis on music-oriented on-line services. They will read and understand the literature describing state-of-the-art methods for MIR and gain hands-on experience on the implementation and application of standard and advanced methods by means of MATLAB assignments and a final project.

Pre-requisites: Basic background in digital signal processing and programming is desirable.

__________________________________________________________________________________________

Calendar and Lecture notes:

Lecture notes will be added and/or updated (as pdf files) as the course progresses, sometimes just before the corresponding lecture. Dates are tentative and subject to change

The Instructor will provide individual guidance during office hours and by email.

Additionally, a tutor will be available to assist with class content and other practical issues:

Yu Wang (wangyu@nyu.edu), Thursdays 10am-12pm, 2nd floor MARL room, Ed. building

____________________________________________________________________________________________

Assessment:

  • Assignments: 40% (10% each) — see details below.
  • Project: 40% — see details below
    • Project Proposal: 10%
    • Final project report + presentation: 30%
  • Mid-term Exam: 20% (best 2 out of 3 questions)
  • Class Participation: extra points (discussions, questions, attendance, interest and enthusiasm)

____________________________________________________________________________________________

Assignments:

There are 4 assignments to be distributed during the semester. Please read carefully and follow these instructions:

  1. All assignments consist of two parts: (A) Implementation and (B) Analysis.

  2. All submissions consist of a single zip file of the folder named YourLastNameAssignment# (e.g. Bello1.zip or Bello3.zip).

  3. The folder should include the Matlab code implementation, following the exact naming and I/O conventions of Part A of the assignment, and a PDF addressing the questions in Part B.

  4. Submissions that fail to follow these conventions will not be accepted as valid.

  5. Clearly indicate your name in the body of every file (at the top of your pdf document, and as a comment on your m-file).

  6. All submissions must be uploaded to the course’s NYU Classes page before 11.55pm on the due date. A penalty of 0.5 pts will be applied for every hour of delay until assignments are no longer accepted at 7.55pm of the following day.

Keep checking this space regularly for assignments and due dates.

____________________________________________________________________________________________

Projects and presentations:

  1. Projects are done in groups of 2 students each.

  2. The project consists of proposing and implementing a solution to a selected MIR problem (preferably in MATLAB or Python); writing a report discussing the specifics of the problem, the approach taken, and its results; and demonstrating your implementation and presenting the highlights of your work to the class.

  3. Projects should go beyond the  materials covered on the assignments, and include a combination of signal processing and machine learning challenges. They do not need to be new or original, i.e. can be a reimplementation of existing work, but all code and results most be your own.

  4. Standard topics include automatic chord recognition, segmentation, audio-score alignment, melody extraction, instrument identification, auto-tagging and cover-song identification. Please discuss with the instructor if you wish to explore a different topic.

  5. Students should select a group and topic early in the semester, and attend office hours to seek guidance and advise about topic selection, and the execution of the project.

Important dates: 

11.16  Project Proposals (10%, 4 pages or less): this document should include a project title and clearly explain the proposal by introducing: context, problem, proposed algorithm(s), evaluation method,  brief workplan and at least 5 bibliographical references that are directly relevant. The document should also name the group members and briefly discuss how the work will be divided amongst them.

12.17  Project demonstration and final report (30%, both a written report of no more than 10 pages and source code should be submitted).

Instructions for Final Project Submission and Presentation (PLEASE READ CAREFULLY): 

  1. Presentations slots are strictly 10 minutes long: 7 minutes for the presentation and 3 minutes for questions. This time includes changeover time (the time it takes you to set up everything). You should bring the presentation on a laptop and try the projector and sound on the Studio beforehand. This is specially important for those of you who want to use any additional piece of equipment for the demonstration.

  2. Attendance and participation on all presentations is mandatory (ask questions, make comments).

  3. The final project report should not exceed a max. number of 10 pages, and should be written like a conference paper. Structure should be more or less as follows: introduction (including motivation), theoretical background, your approach, implementation, experimental part (including a detailed explanation of the evaluation method), discussion, conclusions and future work.

  4. Students are encouraged to develop their project using Matlab or Python, but are welcomed to use other languages as they see fit (C++, Java, Max/MSP, SuperCollider, etc).

  5. During the project due date you will be expected to demonstrate your software application.

____________________________________________________________________________________________

Recommended Books:

  1. Müller, M. “Fundamentals of Music Processing: Audio, Analysis, Algorithms and Applications”. Springer (2015)

  2. Lerch, A. “An Introduction to Audio Content Analysis”. John Wiley & Sons (2012)

  3. Li, T., Ogihara, M. and Tzanetakis, G. “Music Data Mining”. CRC Press (2012)

  4. Klapuri, A. and Davy, M. (Eds.) “Signal Processing Methods for Music Transcription”. Springer (2006)

  5. Müller, M. “Information Retrieval for Music and Motion”. Springer (2007)

  6. Smith, J.O. “Mathematics of the Discrete Fourier Transform (DFT)”. 2nd Edition, W3K Publishing (2007)

  7. Witten, I. and Frank, E. “Data Mining: Practical Machine Learning Tools and Techniques”. Morgan Kaufmann (2005)

  8. Further reading will be recommended as the course progresses (see last slide of every lecture).

____________________________________________________________________________________________

Tools:

  1. Matlab documentation, tutorials, examples: http://www.mathworks.com/access/helpdesk/help/techdoc/matlab.html

  2. Signal Processing Toolbox documentation, tutorials, examples: http://www.mathworks.com/access/helpdesk/help/toolbox/signal/

  3. MATLAB array manipulation tips and tricks by Peter J. Acklam

  4. Data Mining Software: http://www.cs.waikato.ac.nz/ml/weka/

  5. Sonic Visualizer: http://www.sonicvisualiser.org/

Research Resources:

  1. ISMIR home: http://www.ismir.net

  2. All ISMIR papers: http://dblp.uni-trier.de/db/conf/ismir/index.html

  3. MIR-related PhD theses (outdated): http://www.pampalk.at/mir-phds/

  4. MIR Evaluation eXchange (MIREX): http://www.music-ir.org/mirex/wiki/MIREX_HOME