Musical Gestures Research Articles

Towards the Design of a Natural User Interface for Performing and Learning Musical Gestures

A large variety of musical instruments, either acoustical or digital, are based on a keyboard scheme. Keyboard instruments can produce sounds through acoustic means but they are increasingly used to control digital sound synthesis processes with nowadays music. Interestingly, with all the different possibilities of sonic outcomes, the input remains a musical gesture. In this paper we present the conceptualization of a Natural User Interface (NUI), named the Intangible Musical Instrument (IMI), aiming to support both learning of expert musical gestures and performing music as a unified user experience. The IMI is designed to recognize metaphors of pianistic gestures, focusing on subtle uses of fingers and upper-body. Based on a typology of musical gestures, a gesture vocabulary has been created, hierarchized from basic to complex. These piano-like gestures are finally recognized and transformed into sounds.

Music Gestural Skills Development Engaging Teachers, Learners and Expert Performers

This article presents a platform for learning theoretical knowledge and practical motor skills of musical gestures by combining functionalities of Learning Management Systems (LMS) and Serious Gaming (SG). The teacher designs his/her educational scenario that can be articulated by both theoretical and practical activities. The learner accesses online multimedia courses by using his/her LMS client which can be a computer, tablet or smartphone and the serious game by using his/her computer and the motion capture sensors. During practicing, his/her gestures are compared in real-time with the expert gestures and s/he is evaluated both in terms of correct fingerings and kinematics. Finally, the platform offers a single profile for the learner for theoretical and practical activities.

Finger musical gesture recognition in 3D space without any tangible instrument for performing arts

This paper proposes a marker–less computer vision methodology for the simultaneous recognition of complex finger musical gestures performed in space without any tangible musical instrument. Image analysis techniques are applied in order to detect and identify the fingertips on a video. Scale and rotation invariance techniques are also applied. The finger gesture recognition and prediction is based on the stochastic modelling of the extracted high–level features with the help of hidden Markov models and Gaussian mixture models. The applications concern directly the finger gesture control of sound in performing arts. The proposed system can be considered in the long term as an intangible musical instrument that implies 'everyday gestures' and reduces the gap between non–musicians or blind people and music.

God Bless America: The Surprising History of an Iconic Song by Sheryl Kaskowitz (review)

Bless America: The Surprising History of an Iconic Song. By Sheryl Kaskowitz. New York: Oxford Uni- versity Press, 2013. [xiii, 210 p. ISBN 9780199919772. $29.95.] Music exam- ples, illustrations, companion Web site, bibliography, discography, index.Irving Berlin's Bless is an ideal popular song: memorable, easy to perform, and capable of supporting a vari- ety of interpretations. In Bless America: The Surprising History of an Iconic Song, Sheryl Kaskowitz presents a thorough inves- tigation of the song's history and the con- tested meanings ascribed to the work by different audiences throughout the last century. Far more than a study of a single song, the book addresses everything from ideas about the role of the com- poser and performer in popular music to intervention and war, assimilation and ac- ceptance of outsiders, the rise of the ideo- logical Right, rifts between generations, and definitions of Americanness itself (p. 4).Kaskowitz begins by contextualizing the song's history in the popular traditions of the Victorian parlor and the Tin Pan Alley ballad, as well as the repertoire of unofficial national anthems (pp. 5-6). She suggests that the song's effectiveness and stems from Berlin's use of famil- iar expressive schemas that portray a com- munal American identity. From there, the author unpacks the mythology surrounding the song's creation. Originally written in 1918 for a revue with an all-soldier cast, Bless was cut from the show and remained in Berlin's trunk until he re- vised it for Kate Smith's 1938 premiere. The song's subsequent led to ongoing conflicts between Smith (repre- sented by her manager Ted Collins) and Berlin that exemplify problems of owner- ship, creation, and the underlying ten- sions between composer and performer in American popular music (p. 7).The next chapter thoroughly investigates the song's compositional history, shedding light on the intersection of Berlin's musical craftsmanship and business acumen. Berlin was acutely aware of how changing geopo- litical tensions would influence the song's reception: the circumstances surrounding the 1918 version's associations with inter- ventionism had changed by 1938, and Berlin consequently tempered lyrics and musical gestures appropriate for a country at war (p. 36) with alterations that in- cluded the addition of a first verse. This verse casts the as a hymn and reflects the anti-interventionism prevalent in pre- World War II American cultural life. In the final revision (one week before Smith's pre- miere), Berlin changed lyrics, added a new C section (from the mountains. . . .), and alter[ed] the climactic center of the song by shifting the final line's emphasis from America to God (p. 40). These changes, coupled with the deletion of the first verse, reveal the drastic swing in American public opinion from staunchly anti-interventionist to interventionist, while paradoxically re- moving Bless from any spe- cific historical context. Kaskowitz suggests that this adaptability made the timeless, no doubt contributing to its on- going popularity (p. 42). Kaskowitz em- phasizes the influence of Smith's perfor- mances and the arrangement on the song's initial popularity, which both continued the shift away from pop tune sensibilities that Berlin's revisions had begun (p. 43) and positioned the as a hymn or an- them, solidif[ying] the song's associations with group singing (p. 48).The remainder of the book concentrates on the song's reception and popularity. After its premiere, Bless functioned like an unofficial national an- them, which, Kaskowitz argues in chapter 3, was possible because of the song's style (hymn- or anthem-like and not popular), singability (becoming a vehicle for public expressions of patriotism and community), and timing (appropriately expressing senti- ments widely held both before and during World War II). …

Audio-visuomotor processing in the Musician's brain: an ERP study on professional violinists and clarinetists

The temporal dynamics of brain activation during visual and auditory perception of congruent vs. incongruent musical video clips was investigated in 12 musicians from the Milan Conservatory of music and 12 controls. 368 videos of a clarinetist and a violinist playing the same score with their instruments were presented. The sounds were similar in pitch, intensity, rhythm and duration. To produce an audiovisual discrepancy, in half of the trials, the visual information was incongruent with the soundtrack in pitch. ERPs were recorded from 128 sites. Only in musicians for their own instruments was a N400-like negative deflection elicited due to the incongruent audiovisual information. SwLORETA applied to the N400 response identified the areas mediating multimodal motor processing: the prefrontal cortex, the right superior and middle temporal gyrus, the premotor cortex, the inferior frontal and inferior parietal areas, the EBA, somatosensory cortex, cerebellum and SMA. The data indicate the existence of audiomotor mirror neurons responding to incongruent visual and auditory information, thus suggesting that they may encode multimodal representations of musical gestures and sounds. These systems may underlie the ability to learn how to play a musical instrument.

Wenn in Time: Linguistic Moments in Robert Schumann’s Dichterliebe

AbstractThat song is both a concrete musical entity and an abstract metaphor for romantic subjectivity is the central paradox of musical lyricism. This paradox lends the lyric mode in music a self-consciousness that is, I believe, linguistic in character. It invites the listener to hear musical gestures both as signs that participate in the teleology of tonal discourse, and as sounds; and such sounds, in turn, are experienced viscerally at the same time that they serve as an abstract ideal of pure, natural expression. The present article examines, with reference to Schumann’sDichterliebe, the aesthetic and discursive bases for lyricism’s engagement with temporality. Two “linguistic moments” from Schumann’sDichterliebeillustrate how the dialectic between sign and sound plays with the various parameters of musical discourse, reflects or comments upon its own artifice, and breaks the illusion that music is a transparent medium of expression. Finally, the semiotic implications of lyricism’s engagement with temporality are explored with reference to Nietzsche’s conception of theAugenblick.

Natural human–robot musical interaction: understanding the music conductor gestures by using the WB-4 inertial measurement system

This paper presents an inertial measurement unit-based human gesture recognition system for a robot instrument player to understand the instructions dictated by an orchestra conductor and accordingly adapt its musical performance. It is an extension of our previous publications on natural human–robot musical interaction. With this system, the robot can understand the real-time variations in musical parameters dictated by the conductor’s movements, adding expression to its performance while being synchronized with all the other human partner musicians. The enhanced interaction ability would obviously lead to an improvement of the overall live performance, but also allow the partner musicians, as well as the conductor, to better appreciate a joint musical performance, thanks to the complete naturalness of the interaction.

Making Musicians: A Personal History of the Britten-Pears School by Moira Bennett (review)

Making Musicians: A Personal History of the Britten-Pears School. By Moira Bennett. Orford, Suffolk, Eng.: Bittern Press, 2012. [232 p. ISBN 9780957167209. $24.95.] Illustrations, index.Making Musicians, by Moira Bennett, chronicles the struggles and successes of the Britten-Pears School for Advanced Musical Studies, a program for young musi- cians founded by composer Benjamin Britten and tenor Peter Pears in Suffolk, U.K. content of the book's 232 pages is split fairly evenly between historical narra- tive on the one hand and images of various kinds on the other. combination makes for a pleasant read with most pages featuring engaging pictures of the school's participants as well as block quotations of their remembrances. book's land- scape-style format with squat, wide pages al- lows for two columns of text and images per page, facilitating a plethora of pictures that show well on thick, glossy paper.Thumbing through the book, a reader cannot help but be taken in by the lively black-and-white photos that capture the ex- pressive faces and bodies of musicians in action: perusing scores, conducting ensem- bles, demonstrating musical gestures, play- ing instruments, dancing, talking, singing, and thinking, in rehearsal, classes, or even in fully costumed operatic productions. Beyond the artists themselves, photos also record the transformation of the old brew- ery buildings into rooms dedicated to mak- ing music, with before pictures of the rus- tic old beams of the granary and pictures featuring the rehearsal space, en- trance and recital halls, and library shelves full of books. In addition to substantial quotes from participants set off by rectan- gle borders, other insets recreate a variety of course listings and donor records as well as concert programs. Although they are not facsimiles of actual documents, these items complement the story of the school.Bennett's personal history-part mem- oir, part carefully researched record- bubbles with an excitement reflecting the school's evolution from merely an idea in the 1950s, to a new venture in the 1970s and 1980s, to a well-established with an international reputation by the end of the century. Bennett herself joined the staff of the Britten-Pears School in 1979, the year that it officially opened, and stayed un- til 1982, when she became Head of Development for the Aldeburgh Founda- tion (the umbrella organization for the school, the Aldeburgh Festival, and the Snape Maltings Concert Hall). Graham Johnson, an accompanist for early master classes, describes Bennett as a young mother of the school who enchanted Peter Pears and made the a happy place for students and teachers alike (p. 11).The historical narrative proceeds in more or less chronological order, flashing backward and forward as needed to build the backstory, as well as give a sense of what the young musicians went on to accomplish after attending the school. In an account that features so many participants, this back-and-forth allows Bennett to follow the thread of each individual's story, although at times it can become somewhat confus- ing. book is divided into five chapters with the two outer ones serving as introduc- tion and conclusion. three interior chapters cover successive time spans. first focuses on the early years (1953-79) in which ideas took shape, initial classes were held, and the building renovation was planned and executed. second covers the years following the official opening of the (1979-87) when routines were established. third addresses the years directly after Pears's death (1987-91), dur- ing which the went through impor- tant changes in an effort to stay current, eventually becoming the Britten-Pears Young Artist Program. narrative leaves off in 1991; some readers may feel that it cuts off too early, but the final chapter pro- vides closure and points toward the future.The introduction, subtitled The Setting, surveys the sometimes benign, sometimes desolate landscape of the east- ern Suffolk coast and particularly the nearby towns of Snape, where the was located on the site of an old maltings, and Aldeburgh, the sea-battered coastal town where Britten and Pears lived. …

Reconstructing Samuel Barber’s Christmas Cantata Christmas Eve: A Trio with Solos, Op. XIII

Samuel Barber’s early cantata for voices and organ, Christmas Eve , is a previously unstudied area of Barber’s childhood output. Utilizing the composer’s manuscripts from his time as a student of Sidney Homer, the score has been reconstructed. The reconstruction has revealed Barber’s considerable ability to intuit and assimilate the styles of contemporary American sacred choral works of the late nineteenth and early twentieth centuries, while capturing the essence of dramatic musical gesture. It is a work that demonstrates a creative sophistication far beyond the early years of a young composer. By examining this work we are able to observe the artistic energies of a future master composer, celebrated for his works in the vocal canon, as well as consider afresh the composer-performer-audience relationship of the late nineteenth and early twentieth centuries.

Pastorals, Passepieds, and Pendants: Interpreting Characterization Through Aria Pairs in Handel’sRodelinda

Like pendant portraits in Baroque visual art, which were meant to be viewed and understood as pairs, contemporaneous opera seria arias may be interpreted with respect to one another. In this article, I illustrate the usefulness of regarding arias as pendants by examining two pairs from G.F. Handel’s operaRodelinda(1725). Structural and semiotic investigations comprise my musical analyses with a focus on musical topics, voice leading, and musical gestures. The analytical methodology of Matthew Shaftel (2009) coupled with the foundational interpretive frameworks of Robert Hatten (1994 and 2004), Lawrence Kramer (1990), and Wye Jamison Allanbrook (1983) provide a consistent set of strategies with which to negotiate the disparate domains of musical structure and dramatic content. Examples of aria pairs sung by two different characters and by the same character are considered. Viewing two arias as pendants aids in developing specific interpretations and has broader ramifications for understanding characterization throughout the work.

The Evolution of Neumes into Square Notation in Chant Manuscripts

The earliest form of musical notation, neumes, uses discrete or connected symbols to represent one or several pitches in a musical gesture. The next major stage in notation’s development, square notes, presents the reader with a notation made up of variously shaped note-heads and stems or ligatures. The transition between these two forms of notation is the focal point of this study, which compares the virga, the clivis, and various alterations to represent liquescence, in three distinct geographical/scribal regions across Europe. Samples of notation were only taken from manuscripts that can be securely dated within a quarter-century, from 900 to 1500. Results of the comparison show that the transition from neumes to square notes begins first in northern France, at the beginning of the twelfth century. The Germanic and central European areas begin this transition slightly later. In all regions studied, the transition was complete by the end of the thirteenth century. Some reasons for this transition, including the role of the musical manuscript in liturgical culture and the function of notation for the singer, are discussed.

Performing Ritual: Physical and Musical Gesture in Benjamin Britten’s Curlew River

Benjamin Britten’s Curlew River (1964) defies traditional genre labels, exhibiting characteristics of opera, Japanese Noh drama, and religious ritual. Set in medieval England and given a Christian theme, Curlew River is based on the Noh play Sumidagawa and uses a unique language of physical gesture inspired by Noh traditions. The integration of these physical gestures with the music is one of the ways in which Curlew River projects an atmosphere of ritual. In this paper I examine two passages from Curlew River, each of which demonstrates a close connection between the development of individual musical gestures and the progression of physical actions performed at the same time. In the arietta “Near the Black Mountains,” sung by the character of the Madwoman, the subtle development of a single musical figure is linked to the gradual transformation of the actor’s posture. A similar relationship is present in the Ferryman’s introductory scene, in which physical movements act as punctuation for a sequence of musical statements. In both instances, musical and physical gesture are integrated into a unified form of expression, the intensity and focus of which lend Curlew River its ritual quality.

Survey on automatic transcription of music

Abstract An automatic music transcriber is a device that detects, without human interference, the musical gestures required to play a particular piece. Many techniques have been proposed to solve the problem of automatic music transcription. This paper presents an overview on the theme, discussing digital signal processing techniques, pattern classification techniques and heuristic assumptions derived from music knowledge that were used to build some of the main systems found in the literature. The paper is focused on the motivations behind each technique, aiming to serve both as an introduction to the theme and as resource for the development of new solutions for automatic transcription.

Measuring lips control on flute-like instruments using active vision

Flute-like instruments operate under a feedback system between an air jet and a resonator. The characteristics of the air jet and its interaction with the resonator are defined by construction in some flute-like instruments such as the organ pipes, or completely determined by the musician’s control like in the Shakuhachi, panpipe, or transverse flute. In this article, a 3D lips detection system based on active vision using laser grid triangulation has been designed and implemented. Simple musical gestures from transverse flute players have been measured and analyzed, outlying the strategies used to coordinate the parameters available, such as lips to labium distance, jet speed, and jet shape, to orchestrate an adequate sound. The analysis is presented in a non-dimensional representation capable of generalizing to other members of the flute family. The measurement system developed is compared with previous strategies and reveals a promising tool to further understand the complexity of the human control in this popular family of instruments.

Introduction to Pierrot at the Cinema Part III

Carlo Piccardi has demonstrated the connection between the music for pantomime and both silent and sound film. In the first two parts published in MaMI Vol. 1, no. 2 and MaMI Vol.2, no.2 “Pierrot at the Cinema” cites a number of critics whose comments about the relationship between music, mood and physical gesture in the silent film era are still relevant today to scholars and film composers. In this part he discusses the music for pantomimes that were made into films and what happened to the music in the transformation from one genre to the other. Sometimes the necessity to follow the screen action caused the music to become exceedingly linear, a slave to the image. It could even lose its internal logic. Often, however, talented composers came up with musical solutions that caught the physical gestures and moods on screen, and, while writing good “action” music, they developed an overarching musical logic as well. Piccardi discusses Andre Wormser’s score for both the pantomime and film versions of L’Enfant Prodigue, Pasquale Mario Costa’s score for the pantomime and film versions of L’histoire d’un Pierrot, Fernand le Borne’s film pantomime L’empreinte, Engelbert Humperdinck’s music for Das Mirakel, Arnold Schoenberg’s Die gluckliche Hand, Richard Strauss’ Josephs Legende and Ruggero Leoncavallo’s Pierrot au cinema. The variety of musical solutions used in the silent film era are traced into the sound film era right up to Nino Rota’s collaborations with Federico Fellini. Initially, in publishing this translation of “Pierrot at the Cinema,” our aim was to call attention to the very sophisticated school of Italian film music studies and the contributions they have been making to film music history. Piccardi was the first candidate for the obvious reason that his research and discussion of sources was simply in a league of its own. Many of the sources dwelt on the way music interacted with pantomime and film action, an element that we as scholar/ practitioners find particularly fascinating. However, it would be well to note that there is no chapter on pantomime in Rick Altman’s equally well-researched Silent Film Sound (NY, Columbia University Press, 2004). In a quick review of pantomime practices in the US to 1922 via Chronicling America, a Library of Congress/National Endowment for the Humanities digital newspaper archive, I found many references to an active dance, acrobatic and mime pantomime tradition (many accompanied by elaborate music) that included both American and European individuals and troupes. Carlo Piccardi has established the connection for Europe; the connection between pantomime and cinema music in the US still remains to be established. Gillian Anderson

Performing the Ineffable: Poem, Gesture and Performance in Debussy’s ‘Éventail’

Drawing on Peter Kivy’s recent work on reading as performance and Vladimir Jankélévitch’s analysis of meaningful silence in Debussy, this essay seeks to demonstrate that the composer’s concentration on musical gesture merges with Mallarmé’s poetic exploration of directionless movement in ‘Autre Éventail’ in order to get beyond the idea of progression in favour of alternative models of artistic expression that, via direct address, invite readers or listeners to participate in the act of establishing coherent experience from the sounds and words of a song that relies on gesture, silence and suggestion rather than traditional guideposts such as strophic form or harmonic prolongation and tonal resolution to guide the listener.

Embodying Failure: Music Performance, Risk and Authenticity

Certain trends in the recital of Western art music composition and performance have embraced indeterminacy through an emerging sonic aesthetic that seeks to redefine the notion of failure. From Charles Ives' adoption of bi-tonality in the early twentieth century to the ‘glitch’ movement in contemporary computer music, this article traces ways in which musicians have sought to embrace the risk for failure in performance with special attention to virtuosic instrumental music ( Cascone 2006 ; Rodgers 2003 ; Godlovitch 1998 ; Rosen 2002 ). Drawing upon the author's recent interdisciplinary practice-led research ‘Woman=Music=Desire’ (2010) and adopting a choreographic approach to the re-appropriation of musical gesture, the author explores how the risk for failure contributes to live musical experience. This discussion is then extended to the process of corporeal acquisition necessary in rehearsing and performing a piece of music which, the author suggests, results in a degree of gestural self-simulation. In this way, the performer's personal authenticity is discussed as a potential locus of failure in which the physical manifestation of emotional expression helps to determine empathetic identification between performer, spectator and instrument ( Kivy 1995 ). Drawing upon Steven Baker's notion of ‘botched taxidermy’ (2000), the author suggests that this empathetic identification creates a space in which the potential risk for failure might be considered intrinsic to conceptions of corporeality in music performance. In this way, live musical experience is posited a site of risk in which the performer, as a desiring subject, emerges as the embodiment of failure. A short excerpt of the case study ‘Woman=Music=Desire’ may be viewed at: http://www.imogene-newland.co.uk/perf_women_md.php

The Magic Wand of the Wagnerians: Musik als Ausdruck

As is well-known, throughout most of the nineteenth century the defence of Wagner's music was not undertaken on the same grounds on which it was attacked. Critics such as Ludwig Bischoff and Eduard Hanslick attacked Wagner's music for its alleged "formlessness" and harmonic illogicalities, while Wagner's partisans countered with appeals to vaguer criteria of beauty and truthfulness, couched generally in leitmotivic terms, often focusing on the so-called "symphonic web" of Wagner's late works. One particular strategy is encapsulated by the phrase "Musik als Ausdruck," which forms the basis of a number of late nineteenth- and early twentieth-century monographs and articles about Wagner's works. Musical gestures were held to encode a particular emotional state and to reawaken that state in the listener, who would intuit the "meaning" of the gesture instinctively; within such an aesthetic, music was held to represent the essence of phenomenon, the "thing-in-itself." This article attempts to provide an account of the dual foundations of this aesthetic paradigm in philosophy and science, as manifested by Arthur Schopenhauer's Die Welt als Wille und Vorstellung and by Friedrich von Hausegger's Die Musik als Ausdruck respectively, and suggests that the resulting position – as articulated by Hans von Wolzogen and Curt Mey – is not only incompatible with but also incomparable to that of the formalists. The reconciliation of these two positions would not take place until the 1920s and 1930s with the work of Alfred Lorenz.

Walking the Line: Deconstructing Identity, Suicide and Betrayal in Peter Maxwell Davies' Mr Emmet Takes a Walk

AbstractThis article explores how Peter Maxwell Davies uses thematic and structural devices to chart the means by which the title character in Mr Emmet Takes a Walk is led towards suicide. Discussion centres on how Davies integrates use of quotation and trademark musical gestures into the development of characterisation in order to explore different states of ‘reality, dream and waking fantasy’ by which he seeks to explain the reasons for Emmet's suicide.

Characterising musical gestures

Despite the widespread use of the term gesture in writings about music, the term is not defined in most musical dictionaries. Moreover, as this paper shows, the term is employed by different writers in a wide variety of ways. One common use of the term refers to sonic instances that are close analogies of physical gestures. These could be termed expressive unit gestures which, like physical gestures, are perceived as a short, unified, expressive events. To enable a more detailed study of these, this paper outlines a systematic approach to their description in the hope that these will open possibilities for a more detailed study of this aspect of musical gestures. The HamNoSys notation was developed for the systematic description of sign language. Gestures of sign language are notated through a systematic profiling of the actions involved (hand shapes, movement types, etc.). By analogy a musical gesture can be described through its auditory properties such as accent patterns, pitch contour, register, and so forth. This paper suggests that studying these expressive unit gestures offers a way of linking the dynamics of the music with expressive potential and can, therefore, contribute to an experiential account of music; could lead to new methods of investigating listeners engagement with music; and could potentially offer new ideas in the field of music information retrieval.