Visualizing Musical Expression

Welcome to this interactive visualization of two experiments in perceived musical expression that were performed in the context for the Con Espressione project !

In the first experiment listeners were asked to describe, in free text (preferable adjectives), the perceived expressive character of different performances of several Baroque, Classical, and Romantic solo piano pieces. In the second experiment, two groups of music experts were asked to sort the 150 most common descriptors of the first experiment into "piles" of similar terms.

This visualization reveals some of the results of these experiments. In the right column you see the performances used in the first experiment. In the center you see a two-dimensional representation of the 150 most important terms, created using Multidimensional Scaling. In the left column you see two lists of piles as created and named by the two groups of experts, representing two alternative ways of structuring this space of musical characterizations. There are three ways to interact with this visualization:

(1) Click on a performance to see the terms used by listeners to describe this performance. Every term is in turn connected to the two piles it was sorted into. By clicking on a performance title you can also listen to a short excerpt of the performance.
(2) Click on a pile to see only the terms that were sorted into it. Every term is in turn connected to all the performance it was used for.
(3) Click on a term to see both piles it was sorted into as well as all performances it was used for. All terms that were were sorted into the same two piles are connected too.

For more details on the experiments and the creation and analysis of this data, please see the publications below. The datasets (including many modalities not used in this visualization) are available for further research too.

Publications

Cancino-Chacón, C., Peter, S., Chowdhury, S., Aljanaki, A., and Widmer, G.:
Sorting Musical Expression: Characterization of Descriptions of Expressive Piano Performances
In Proceedings of the 16th International Conference on Music Perception and Cognition (ICMPC), 2021

Cancino-Chacón, C., Peter, S., Chowdhury, S., Aljanaki, A., and Widmer, G.:
On the Characterization of Expressive Performance in Classical Music: First Results of the Con Espressione Game
In Proceedings of the 21st International Society for Music Information Retrieval Conference (ISMIR), 2020

Data and Repositories

The Con Espressione Game Dataset can be obtained here:
Con Espressione Dataset

Acknowledgments

This research has received support from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 670035 (project "Con Espressione") and by the Research Council of Norway through its Centers of Excellence scheme, project number 262762 and the MIRAGE project, grant number 287152.

We gratefully acknowledge the effort invested by our music expert, Hans Georg Nicklaus (Anton Bruckner Private University of Music, Linz) for helping with the selection of the different performances in the dataset. We thank Olivier Lartillot for sharing the Matlab code to compute the loudness features.