Sonic Events Research Articles

Understanding ‘understanding music’ in Proust

ABSTRACT Sonic events of various kinds (musical and otherwise) are often complex semiotic occasions in Proust’s Recherche. One of the ways we attest to the meaningfulness of sounds we hear, to our understanding of them, is by applying language to them. When we talk or write about sounds or react to them, we also produce social effects. This – by inducing us to react to them and then produce language about them – is one of the ways sounds make social meanings happen in Proust (and in general). By examining language produced about sounds from a couple different volumes of the Recherche, I investigate what Bourdieu calls the differential social effect of works of art. I consider how scenes of language production around sound and music in the novel are constructed to reveal the social effects of apprehending sound and reacting to it. Finally, I speculate on how all of this work the novel does is part of a larger project of understanding: understanding what it means to understand musical sounds.

The acoustemology of Hong Kong in protest: a sonic understanding of the Anti-Extradition Law Amendment Bill Movement

ABSTRACT This paper attempts to construct a sonic understanding of the recent past of Hong Kong in protest. By utilizing and extending Steven Feld’s framework of acoustemology, I examine the sonic events during protests and identify sounds in the cultural system understood among the inhabitants. I conducted a digital archival search of newspaper articles and created a dataset of the events sounding ‘Glory to Hong Kong’ during the Anti-Extradition Law Amendment Bill Movement. I explore three aspects: the participants, the space where the protest activities took place, and the sound generated and/or sounded in the activities. By demonstrating the interrelationality of them, I argue that sounds are able to reveal and (re-)claim spaces, establish a relationship between humans and space, bring an understanding of Hong Kong, and construct ‘Hong Kong,’ via collective sounding in protests and daily activities across time. This is the acoustemology of Hong Kong in protest.

Posthumous Sound and the General Imagination

Abstract The 1997 discovery of a fifty-thousand-year-old flute made from the femur of a cave bear, with its intimation of reanimating nonhumans, and the 1977 launch of the Voyager spacecraft carrying an eclectic set of sound recordings intended to be heard in the distant future by nonhuman others: two sonic events that frame the possible meanings of posthumous. Together these examples and others question whether everything audible is already over—the bear's lost life, electronic recording procedures—or indefinitely deferred until an act of listening that may never occur. An ecological address to the problems of making sonic culture at a historical turning point at or beyond terminal risk prompts a politics of the commons grounded in a general imagination (modeled on Marx's general intellect). Against earlier modernist claims for both rationality and its failure, sound cultures enact a drama of melancholy and hope in the ecological continuity of body and world at the moment of their end.

Experiencing Sounds in Ancient Spaces: An Overview

This paper aims to analyse recent studies which have raised new hypotheses concerning archaeomusicology, archaeoacoustics, and aural architecture as an emerging trend in humanities research, with a particular focus on the intersection of musical performances, theatrical spaces, and sound experience in the past. These studies have highlighted how theatrical buildings not only defined a place for performances, but also established the setting for multisensorial events in which music, dance, and other sonic events played an important role. This overview also takes into consideration how digital technologies can help shape our understanding of the architecture-sound nexus.

Listening in Sacred Spaces: The Sanctuary of Poseidonia and Selinunte’s Main Urban Sanctuary

Abstract The aim of this work is to analyse recent studies which have raised new hypotheses concerning aural architecture as an emerging trend in humanities research, with a particular focus on the intersection of sacred space, rituals, and sound in the past. These studies have highlighted how sacred buildings not only defined a sacred place as a physical and symbolic expression of a specific form of worship but also established the setting for performative and multisensorial ceremonies in which music, dance, and other sonic events played an important role. In this contribution, we investigate studies on aural architecture to explore if the location of sacred spaces indicates whether ancient people reacted to ritual and musical developments by modifying sanctuaries or by designing and constructing new buildings and spaces for performances. In addition, this article explores studies on aural architecture to obtain an overview of how specific sonic features could have influenced the soundscape of sacred spaces, which consisted not only of songs, music, prayers, recitations and religious sonic and vocal utterances but also of natural elements, such as animals, water, and wind. This overview also takes into consideration how digital technologies and virtual acoustics can help shape our understanding of the architecture-sound nexus.

More than an Afterimage: Music as Holocaust Spatial Representation and Legacy

Music occupies a unique and multi-faceted role in spatial representation of the Holocaust, both in terms of documenting its horrors and in cultivating legacy. This uniqueness derives from music’s dual temporal and physical essence as it is represented by written scores that serve as a blueprint, as sonic events that fill both time and space, and as musical instruments that operate as conduits for both. String instruments, in particular, have occupied a vital place in Jewish culture and, consequently, during the Holocaust. In the most tragic sense, some of these instruments even became actual containers of genocidal evidence as with violins played outside concentration camp crematoria that filled with the human ash that fell. This article will demonstrate that, when played, these instruments transform into living artifacts and musical witnesses, with voices that can speak for those who have been silenced, and that the resulting music that resonates from the printed page fills a sonic space that serves as a powerful medium for memory and representation. The phrase “bearing witness” often refers to representing the stories of people, places, and experiences through words, either written or spoken. But material culture also has a role to play in representation. While objects, art, and architecture certainly support language-based witness, they also provide their own unique lens and conduit for testimony. This seems especially true for music, which has the ability to exist as and cross between both words and objects. Nevertheless, music as material witness remains a complex and often understudied aspect of historical testimony. As a result, this paper will explore through an interdisciplinary approach the divergent nature of music as an aural form, as a creative art, and as a cultural artifact and will offer examples of how music can enhance, elucidate, and complicate Holocaust representation.

Composing what happened

ABSTRACT Time and memory in Gertrude Stein are explored using on the one hand a fragment of her play, What Happened; and on the other hand, the conceptual underpinnings of Fourier transforms as applied in musical composition to relate the ‘time domain’ to the ‘frequency domain’ in the description of sonic events. The paper sets these concepts to work in a reconstruction of the operation of memory in interaction with real-time events – both the event being remembered and the act of remembering as an event in time itself, with memory being understood as a harmony-like resonance connecting those moments. The concept of ‘harmony’ here is given a wider understanding, extending from relations between sounds to relations between people (musicians, collaborators). The paper looks in detail at the collaborative diachronic resonances that lead from Stein’s attending a birthday party in 1913, through her writing a play, then a collective reading of the play and the collaborative composition of an opera, to the reconstitution in-time of this act of memory in performance, in collaboration with musicians. Finally, the paper provides an answer to an important question raised by Stein on birthday cakes – is a cut a slice?

Imagining the name of the rose with Deleuze: organizational and self world-making on the screen

ABSTRACT In this research, the process of world-making at stake in movie direction and movie images is explored. Following Deleuze and Bergson, I investigate how the process of designing and experimenting with images contributes to a prelinguistic world-making process from which multiple selves can happen. With and from an autoethnography of my experience of the movie The Name of the Rose, I develop a cinematographic language that can describe organizing actions in coherence with an ontology of becoming. By watching and rewatching the movie and exploring my memory of its images, I identify three modalities of visual and sound organizing at the heart of processual world-making. As visual and sonic events lived and relived, these modalities contribute to the happening of multiple pasts, presents and futures selves interwoven with organizational world-making.

SIIIGNAL: An Electroacoustic Composition/Instrument in Virtual Reality

SIIIGNAL is an electroacoustic composition/instrument in Virtual Reality (VR). The piece explores modes of interaction afforded by the VR environment; in com- bination with hand-tracking, gestural analysis and the physics engine of Unity3D, it provides a framework for musical expression using the VR medium. The player can instantiate sound particles in the space, perform sample triggering through collision detection, arrange sound objects in space, populate the environment with processing modules, spawn pulses that emanate spherically from the cen- tral sound object throughout the space triggering further sonic events, among others. The piece is divided into multiple sections with changing parameters over time and different rules. It starts requiring simple sonic gestures by the player, and as it progresses, emergent behaviours lead to more sonic complexity and possibilities for musical expression.

Plugin Cultures

Plugin Cultures

Characterization of Sonic Events Present in Natural-Urban Hybrid Habitats Using UMAP and SEDnet: The Case of the Urban Wetlands

We investigated whether the use of technological tools can effectively help in manipulating the increasing volume of audio data available through the use of long field recordings. We also explored whether we can address, by using these recordings and tools, audio data analysis, feature extraction and determine predominant patterns in the data. Similarly, we explored whether we can visualize feature clusters in the data and automatically detect sonic events. Our focus was primarily on enhancing the importance of natural-urban hybrid habitats within cities, which benefit communities in various ways, specifically through the natural soundscapes of these habitats that evoke memories and reinforce a sense of belonging for inhabitants. The loss of sonic heritage can be a precursor to the extinction of biodiversity within these habitats. By quantifying changes in the soundscape of these habitats over long periods of time, we can collect relevant information linked to this eventual loss. In this respect, we developed two approaches. The first was the comparison among habitats that progressively changed from natural to urban. The second was the optimization of the field recordings’ labeling process. This was performed with labels corresponding to the annotations of classes of sonic events and their respective start and end times, including events temporarily superimposed on one another. We compared three habitats over time by using their sonic characteristics collected in field conditions. Comparisons of sonic similarity or dissimilarity among patches were made based on the Jaccard coefficient and uniform manifold approximation and projection (UMAP). Our SEDnet model achieves a F1-score of 0.79 with error rate 0.377 and with the area under PSD-ROC curve of 71.0. In terms of computational efficiency, the model is able to detect sound events from an audio file in a time of 14.49 s. With these results, we confirm the usefulness of the methods used in this work for the process of labeling field recordings.

Moving Cage: Vibration, Sonification and the Quanta of Time

Dear John is an experimental choreomusical work that reinterprets Cage's works while advancing his ideas of sound as sonic events and embodied choreography. In this episodic work, improvised movement unfolds to a soundscape of defamiliarized instruments, sound devices and sonicities of macro- and micro-movements. The correspondence and (in)congruence between dance movements and music's kinetic energy become the means to examine a politics of the body and sound, of music on movement. Additionally, in this ‘auditory architecture’ the quanta of time, its relations and (lack of) unity are exposed. This article then examines the intersubjective interplay of movement and music, body and sonicity; it considers the resonance of the performing body as intermaterial vibration and how this invites a sonic politics of relational possibility. The article will then also investigate the ways in which the interaction of motion and music, movement and stillness engenders experiences of time's indeterminacy and elasticity.

W.B. Yeats and the Delegated Voice:The Words upon the Window-PaneandA Full Moon in March

This article uses the concept of phonography, defined as the representation of sonic events, to examine two of W.B. Yeats’s later plays, The Words upon the Window-Pane and A Full Moon in March. Each of these plays allegorizes its own genesis as a phonographic artwork, a sonic inscription designed to transmit the author’s voice. By asking whether text can serve as a conduit, they provide ample evidence of Yeats’s attachment to phonocentric thought. The Words anticipates Roland Barthes’s pronouncements on “the death of the author” by pessimistically deconstructing Yeats’s own phonocentric position. Its centrepiece is a corrupted séance that functions as a travesty of the play itself, enacting a mechanical medium through a spiritualist medium who seems to relay a recording of the voice of Jonathan Swift. While The Words registers a host of anxieties about mediation, social degeneration, and nullified authority, A Full Moon reconciles voice and inscription in the symbol of the singing severed head. This reconciliation is allegorized as a sacred marriage, a scripted ritual that involves an execution by beheading. The head’s song, which can be read as Yeats’s figure for the phonographic inscription, represents the magically immediate transmission of the poet’s voice to a posthumous audience.

Agostino Di Scipio and Dario Sanfilippo: Machine Milieu

Agostino Di Scipio and Dario Sanfilippo: Machine Milieu

Long-term deep learning-facilitated environmental acoustic monitoring in the Capital Region of New York State

The effect of anthropogenic activity on animal communication is of increasing ecological concern. Passive acoustic recording offers a robust, minimally disruptive, long-term approach to monitoring species interactions, particularly because many indicator species of environmental health factors such as biodiversity, habitat quality, and pollution produce distinct vocalizations. Machine learning algorithms have been used in recent decades to automatically analyze the large quantities of audio data that result. In this study, a microphone array was used to collect continuous audio data at a site in the Capital Region of New York State for twelve months, resulting in over 8000 h of recordings. A 19-class database containing a variety of bio- and anthrophony was used to train a convolutional neural network in order to generate a reliable record of species-specific calling activity for the entire study period. These results were used to calculate an acoustics-based pseudo-species richness and abundance distribution. Additionally, heatmap plots were used to visualize (i) the time of day (x), sound category (y), and predicted number of sonic events for an average 30-day period and (ii) the day of the year (x), time of day (y), and predicted number of sonic events for each sound category. The correlations between these sonic events and various abiotic factors such as number of daylight hours, temperature, and weather activity were also examined.

Douce Mélancholie: Sonic Negotiations of Absence in the Works of Susan Philipsz and Félicia Atkinson

On July 5, 2019, French composer, poet, and publisher Félicia Atkinson released an experimental album titled The Flower and the Vessel. On September 16, 2019, the Pulitzer Arts Foundation in St. Louis, Missouri, opened the exhibition Susan Philipsz: Seven Tears. By reflecting on these two contemporary sonic events through the lens of affect theory, this essay aims to explore the embodied experience of absence.

Using a Sonic Time-Lapse Method as a Compositional Tool

This article describes an innovative compositional method based on the use of a sonic time-lapse algorithm to create soundscape audio montages. The method is based on the superposition of short audio samples of 24-hour continuous field recordings carried out in various kinds of wildlife sonic environments. The optimisation of the algorithm focused on the enhancement of gradual crossfade transitions between recorded samples and the use of variable sample durations as a way of recreating a natural sense of evolution of sonic events in time. Future developments of the project will integrate virtual reality interfaces and environmental education projects as part of the time-lapse algorithm machine-learning techniques and also creative tools suitable for multimedia installations.

Computational Designing of Sonic Morphologies

Much electroacoustic music composition and sound art, and the commentary that surrounds them, is locked into a materialist sound-object mindset in which the hierarchical organisation of sonic events, especially those developed through abstraction, are considered antithetical to sounds ‘being themselves’. This article argues that musical sounds are not just material objects, and that musical notations, on paper or in computer code, are not just symbolic abstractions, but instructions for embodied actions. When notation is employed computationally to control resonance and gestural actuators at multiple acoustic, psychoacoustic and conceptual levels of music form, vibrant sonic morphologies may emerge from the quantum-like boundaries between them. In order to achieve that result, it is necessary to replace our primary focus of compositional attention from the Digital Audio Workstation sound transformation tools currently in vogue, with those that support algorithmic thinking at all levels of compositional design.

Binaural models that can assess and understand architectural environments

Traditional binaural models, such as those based on cross-correlation and equalization/cancellation algorithms, use a bottom-up approach. They simulate the functions of the ascending auditory pathway from signals at the ears up through higher-level structures, typically the midbrain. In contrast, top-down methods are frequently used in practical robotics applications, for instance, when applying simultaneous-localization-and-mapping (SLAM) techniques. Such top-down approaches, however, are generally absent from models that foster understanding of our auditory system. This talk focuses on binaural-localization models that integrate traditional signal-driven models with psychoacoustically motivated top-down structures, for example, by means of feedback loops. The aim is to design advanced models that operate in architectural environments in order to understand both the environment and activities within the environment. These models are presented in the context of the authors' current work on a contributed book entitled, “The Technology of Binaural Understanding,” which is in press at Springer. The discussed approaches include: models to estimate room-impulse responses and binaural-activity maps from running signals; algorithms to utilize head movements for strategically reading the environment; and algorithms for the automatic labeling of sonic events using deep-learning techniques. [Work supported by the EU-FET project “TwoEars,” Contract No. 618075, NSF BCS-1539276, and CISL (RPI).]Traditional binaural models, such as those based on cross-correlation and equalization/cancellation algorithms, use a bottom-up approach. They simulate the functions of the ascending auditory pathway from signals at the ears up through higher-level structures, typically the midbrain. In contrast, top-down methods are frequently used in practical robotics applications, for instance, when applying simultaneous-localization-and-mapping (SLAM) techniques. Such top-down approaches, however, are generally absent from models that foster understanding of our auditory system. This talk focuses on binaural-localization models that integrate traditional signal-driven models with psychoacoustically motivated top-down structures, for example, by means of feedback loops. The aim is to design advanced models that operate in architectural environments in order to understand both the environment and activities within the environment. These models are presented in the context of the authors' current work on a contributed b...

Data driven filtering of bowel sounds using multivariate empirical mode decomposition

BackgroundThe analysis of abdominal sounds can help to diagnose gastro-intestinal diseases. Sounds originating from the stomach and the intestine, the so-called bowel sounds, occur in various forms. They are described as loose successions or clusters of rather sudden bursts. Realistic recordings of abdominal sounds are contaminated with noise and artifacts from which the bowel sounds must be differentiated.MethodsThe proposed intrinsic mode function-fractal dimension (IMF-FD) filtering utilizes the property of the multivariate empirical mode decomposition (MEMD) to behave as a series of band pass filters. The MEMD decomposes the abdominal signal into its different frequency components. The resulting intrinsic mode functions (IMFs) are modulated in amplitude and frequency where transient sonic events occur. Based on the complexity of the IMFs, measured by their fractal dimension (FD) in sliding windows, the information-carrying IMFs are selected. The filtered signal is formed as the superposition of all selected IMFs. The IMF-FD filter not only enhances the non-linear components of the original signal but also segments them from the rest. Another important aspect of this work is that typical artifacts that occur in the same frequency range as bowel sounds can be subsequently eliminated by heuristic rules.ConclusionsThe method is tested on a realistic, contaminated data set with promising performance: close to 100% of the manually labeled bowel sounds are identified.