Multichannel Playback Research Articles

Creating with timbre

Unfolding processes of timbre and memory in improvisational piano performance This exposition is an introduction to my research and practice as a pianist, in which I unfold processes of timbre and memory in improvised music from a performer’s perspective. Timbre is often understood as a purely sonic perceptual phenomenon. However, this is not in accordance with a site-specific improvisational practice with changing spatial circumstances impacting the listening experience, nor does it take into account the agency of the instrument and objects used or the performer’s movements and gestures. In my practice, I have found a concept as part of the creating process in improvised music which has compelling potential: Timbre orchestration. My research takes the many and complex aspects of a performance environment into account and offers an extended understanding of timbre, which embraces spatial, material and bodily aspects of sound in improvised music performance. The investigative projects described in this exposition offer a methodology to explore timbral improvisational processes integrated into my practice, which is further extended through collaborations with sound engineers, an instrument builder and a choreographer: -experiments in amplification and recording, resulting in Memory piece, a series of works for amplified piano and multichannel playback - Piano mapping, a performance approach, with a custom-built device for live spatialization as means to expand and deepen spatio-timbral relationships; - Accretion, a project with choreographer Toby Kassell for three grand pianos and a pianist, where gestural approaches are used to activate and compose timbre in space. Together, the projects explore memory as a structural, reflective and performative tool and the creation of performing and listening modes as integrated parts of timbre orchestration. Orchestration and choreography of timbre turn into an open and hybrid compositional approach, which can be applied to various contexts, engaging with dynamic relationships and re-configuring them.

The fictional environment meets acoustic reality—Simulating and creating acoustical conditions in for laboratory testing

In recent years, we have installed electro-acoustic enhancement systems in specialized acoustic laboratories. These labs incorporate sound isolating environments with loudspeakers that enable multi-channel playback of soundscape recordings that reproduce acoustical conditions of a variety of spaces. In addition, the systems use electronic architecture to carefully match the acoustics of the soundscape recordings to enable live sources in the lab to interact with these environments in real time.

Automatic Sound Scene Control Using Image Sensor Network

We proposed the automatic sound scene control system using the image sensor network for preserving the constant sound scene without respect to the users' movement. In the proposed system, the image sensor network detects the human location in the multichannel playback environment and the SSC (sound scene control) module automatically controls the sound scene of the multichannel audio signals according to the estimated human location which is the angle information. To estimate the direction of the human face, we used the normalized RGB (red, green, and blue) and the HSV (hue, saturation, and value) calculated from the images obtained by the image sensor network. The direction of the human face can be easily decided as the image sensor to capture the image with the highest number of pixels to satisfy the thresholds of the normalized RGB and the HSV. The estimated direction of the human face is directly fed to the SSC module, and the controlled sound scene can be simply generated. Experimental results show that the image sensor network successfully detected the human location with the accuracy of about 98% and the controlled sound scene by the SSC according to the detected human location was perceived as the original sound scene with the accuracy of 95%.

Designing an auditory lab including active acoustics

LIVELab, a new facility at McMaster University, has been developed in order to help support research in areas such as developing intelligent hearing aids, measuring the physiological impact of media presentations, understanding interactions between musicians, and determining effective sounds for medical and warning systems. This room combines a quiet performance space featuring a low nominal reverberation time with technology including active acoustics, multichannel playback, and integral measurement and instrumentation to create a sonically flexible facility for auditory research. The design of the room and its acoustic variability for supporting these research endeavors will be discussed.

Applications of sampling musical instruments to simulations and virtual environments

Sampling musical instruments are a powerful resource for virtual environment sound generators. Samplers provide real-time multi-channel playback of digitized audio signals such as individual musical instrument notes or sound effects, under control of real-time user input devices, such as a music keyboard or computer. Samplers also transform the sound through pitch shifting, time-varying amplitude modification (enveloping), filtering, and looping or repeating a section of the sound to create an indefinite sustain from a short representative segment. Interactive simulators can exploit these transformations; pitch shifting can simulate Doppler shift for moving sound sources, and simple models of spatialization can be implemented using amplitude and time delay panning and low-pass filtering. Sampling is analogous to image processing in computer graphics; digitizing and processing of real images (or sounds) may be faster than developing a synthetic model, if the model is even feasible. Sampling and its extensions may thus always offer an advantage over pure model-based approaches.