Immersive Audio Systems Research Articles

Sound environment control using an immersive audio system—Advancing sound experience for creators and listeners using Active Field Control

Spatial and immersive systems have gained popularity in recent years and are broadly used in various types of applications to realize natural or augmented sound environment. To create a variety of sound environment freely and flexibly, Yamaha immersive audio system, known as Active Filed Control (AFC), has two functions. One is “sound source control” based on object-based audio rendering technique and another is “sound field control” based on acoustic enhancement technology. This paper presents several unique solutions applying immersive audio systems that use either or both sound source control and sound field control. We conducted post-event acoustic measurements or survey of questionnaire supporting that intended acoustic controls had been successfully implemented.

First Live Production and Transmission of Immersive Audio on Commercial OTA Digital Television in Latin America

Since 2017, Globo Television has been testing and evaluating immersive audio systems to use on over-the-air (OTA), pay TV, and streaming services to increase the user experience of our viewers and listeners, giving a sense of “being there” and exploring off-screen sound information, especially on live events. From that point until the present experimental content has been produced, some experimental transmissions made via streaming services, over-the-top (OTT), and OTA. This was done in collaboration with representatives from industrial and academic sectors in a regulatory committee. In 2020, a reviewed standard NBR 15602–2—the audio coding part of Brazil’s Digital TV standards—was published, which included immersive audio systems as optional. In September 2020, immersive audio was delivered on an OTA commercial channel in São Paulo, reaching more than 17 million people. Dolby Atmos audio service was used in the transmission with no negative impact on legacy receivers. This innovation presents live immersive audio transmissions across an entire workflow, from venue microphone audio capture, through an audio production chain, and extending from the exhibition center to TV transmission.

Real rooms vs. artificial reverberation: An evaluation of actual source audio vs. artificial ambience in the rear height channels of immersive audio systems

Many spatial audio researchers and content producers agree that the best source material for height channels in immersive audio is provided by the capture of actual elevated channels in the room. Particularly for music recording, this technique is preferred as opposed to signal processing, providing a more natural and realistic impression of immersion. While previous work has proven this to be the case in the front channels of various 3D playback systems such as 22.2, the content of the rear height channels has not been specifically evaluated in this respect. Multichannel audio recording, specifically 3D recording, can be a cumbersome task as the channel counts expand—and so the question arises—is it really necessary to capture discrete rear height information? This research compares four height channel capture points are compared to two capture points applied to the front height channels in conjunction with artificial reverberation in the rear channels. A two-part study is employed—the first is a simple ABX test to determine discriminability between the real rooms and the artificially generated version. Part two is a preference test, based on several standard acoustic/perceptual descriptors, revealing the subtle differences between real and artificial rear height channel information.

On Building Immersive Audio Applications Using Robust Adaptive Beamforming and Joint Audio-Video Source Localization

This paper deals with some of the different problems, strategies, and solutions of building true immersive audio systems oriented to future communication applications. The aim is to build a system where the acoustic field of a chamber is recorded using a microphone array and then is reconstructed or rendered again, in a different chamber using loudspeaker array-based techniques. Our proposal explores the possibility of using recent robust adaptive beamforming techniques for effectively estimating the original sources of the emitting room. A joint audio-video localization method needed in the estimation process as well as in the rendering engine is also presented. The estimated source signal and the source localization information drive a wave field synthesis engine that renders the acoustic field again at the receiving chamber. The system performance is tested using MUSHRA-based subjective tests.

Inverse filter design for immersive audio rendering over loudspeakers

Immersive audio systems can be used to render virtual sound sources in three-dimensional (3-D) space around a listener. This is achieved by simulating the head-related transfer function (HRTF) amplitude and phase characteristics using digital filters. In this paper, we examine certain key signal processing considerations in spatial sound rendering over headphones and loudspeakers. We address the problem of crosstalk inherent in loudspeaker rendering and examine two methods for implementing crosstalk cancellation and loudspeaker frequency response inversion in real time. We demonstrate that it is possible to achieve crosstalk cancellation of 30 dB using both methods, but one of the two (the Fast RLS Transversal Filter Method) offers a significant advantage in terms of computational efficiency. Our analysis is easily extendable to nonsymmetric listening positions and moving listeners.

Surrounded by sound

The authors discuss immersive audio systems and the signal processing issues that pertain to the acquisition and subsequent rendering of 3D sound fields over loudspeakers. On the acquisition side, recent advances in statistical methods for achieving acoustical arrays in audio applications are reviewed. Classical array signal processing addresses two major aspects of spatial filtering, namely localization of a signal of interest, and adaptation of the spatial response of an array of sensors to achieve steering in a given direction. The achieved spatial focusing in the direction of interest makes array signal processing a necessary component in immersive sound acquisition systems. On the rendering side, 3D audio signal processing methods are described that allow rendering of virtual sources around the listener using only two loudspeakers. Finally, the authors discuss the commercial implications of audio DSP.

Fundamental and technological limitations of immersive audio systems

Numerous applications are currently envisioned for immersive audio systems. The principal function of such systems is to synthesize, manipulate, and render sound fields in real time. We examine several fundamental and technological limitations that impede the development of seamless immersive audio systems. Such limitations stem from signal processing requirements, acoustical considerations, human listening characteristics, and listener movement. We present a brief historical overview to outline the development of immersive audio technologies and discuss the performance and future research directions of immersive audio systems with respect to such limits. Last, we present a novel desktop audio system with integrated listener-tracking capability that circumvents several of the technological limitations faced by today's digital audio workstations.