A system for embodied social active listening to sound and music content

Abstract

This article introduces a model and a system for embodied social active listening to sound and music content. The model is based on the simultaneous navigation/exploration of multiple maps, starting from a low-level physical map, up to a high-level emotional, affective map. The paper discusses the concepts underlying such a model, the system implementing it, and two concrete examples: The Orchestra Explorer designed for the Museo degli Strumenti Musicali at Accademia Nazionale di Santa Cecilia, Roma, Italy, and the interactive dance and music performance The Bow Is bent and Drawn (composer Nicola Ferrari), presented at Casa Paganini, Genova, Italy, in occasion of the opening concert of the 8th Intl. Conference on New Interfaces for Musical Expression (NIME08), June 4, 2008, and in museum interactive experiences at Festival della Scienza, Genova. This work is part of current research at Casa Paganini—InfoMus on embodied social active listening to sound and music content through the analysis and processing of expressiveness in human full-body movement and gesture. A user-centric interactive multimedia system architecture is proposed, operating on prerecorded music. From the perspective of valorization of cultural heritage, this research provides engaging paradigms of interaction with prerecorded music content, enabling a large number of nonexpert users to rediscover the musical heritage (e.g., classical and contemporary music) they may not be familiar with. Research was carried out in the framework of the EU-ICT Project SAME (Sound and Music for Everyone, Everyday, Everywhere, Every Way, www.sameproject.eu, 2008–2010) and has been recently extended to active experience of audiovisual content, and in particular to cultural heritage and museum scenarios: the novel interactive permanent museum exhibition Viaggiatori di Sguardo has been designed and developed at Palazzo Ducale, Genova, enabling visitors to explore virtually the UNESCO Heritage Palazzi dei Rolli in Genova.