RhythMus · A generic framework for analyzing temporal structure in music

A generic framework for analyzing temporal structure in music

Most musics in the world are characterized by a certain amount of repetition that shapes musical time and makes music a unique experience. This repetitive structure has a hierarchical form that consists of sound event onsets, a more or less regular pulse, rhythmic patterns, and higher units of musical form. While many computational analysis methods concentrate on one level of the rhythm hierarchy in isolation, this project proposed dynamic Bayesian models for the rhythm analysis that take into account the relations between the hierarchy levels. Within this project, one important goal was to increase the flexibility of the state of the art in analysis of rhythmic structure. Until the beginning of this project, most approaches were tailored towards certain properties of Eurogenetic music. One example of such properties is the assumption of strict isochrony of the beat pulse, which is the pulse a human listener will be most likely tap her foot to the music. This isochrony is not common in many cultures, as, for instance, Turkish, Greek, and Indian musics. The developed models unify the tracking of note onsets and the most prominent beat with the observation of similarity on a higher metrical level into one unified Bayesian analysis framework. The advantage of this framework is the flexibility to changing signal characteristics and to non-isochrony of the beat. The main achievement of this project was the successful development of computational tools that are capable to determine the type of meter and to track the alignment of the metric cycle to an unknown music signal. This works with high accuracy for various forms of meter, and for a large variety of musical style, with the only precondition being the availability of a small amount of annotated representative music examples that the model can be trained on. The project reached the main objectives stated in the proposal to full extent, and did some steps to explore the applicability of the proposed model to different analysis problems. State-of-the-art methods for onset detection were integrated as observation models that couple the proposed Bayesian networks to the music signal. Most importantly, a dynamic Bayesian network was implemented that integrates the tracking process on several metrical levels. These levels can be flexibly defined, and the observation models can be adapted to new styles easily. The existing model is capable to identify the metrical structure of a piece, and to track it on several levels on Greek, Indian, and Turkish musics with an accuracy that was far beyond reach before this project. On the other hand, performance on Eurogenetic styles is equal to the best available approaches presented previously. Several inference methods were developed that decrease the computational demands of the developed analysis method, creating a critical mass of Bayesian inference related publications in MIR throughout the last years. The proposed model was adapted to melodic analysis as well, in an approach that targets the alignment of written notation to the sound of a performance. This way, the work in this project went beyond the targeted rhythmic aspects, and demonstrated the value of the developed models on a more common ground. The used Bayesian models attracted the attention from several other European research groups, in specific the Johannes Kepler University (Linz, Austria), and the University Pompeu Fabra (Barcelona, Spain). The fellow was able to establish a network of collaboration between these institutes, which lead to the development of shared code repositories that will be made available for research purposes. The publication of these repositories is scheduled for the conference of the International Society for Music Information Retrival (ISMIR) in 2015, and will represent an important landmark of a larger international collaboration that was made feasible by the RhythMus project. The outcome of this project contributes to the state of the art in Music Information Retrieval (MIR). The proposed methods are an important contribution to music distribution and recommendation platforms, because the models can be adapted to previously unseen styles with minimal human expert interaction. This way, they can provide valuable information about tempo, meter type, and timing characteristics of musical styles that were so far out of reach for automatic analysis methods. These capabilities make the proposed models interesting for research in ethnomusicology as well, since the proposed analysis methods in combination with appropriate visualizations can provide a novel perspective on various aspects related to rhythm in various musics of the world. The outcome of this project and the related publications can be accessed at http://www.rhythmos.org/rhythmus.html.

Data: CORDIS, © European Union