Audio Games Research Articles

Echoes of Player Experience: A Literature Review on Audio Assessment and Player Experience in Games

Player Experience (PX) in video games is a multifaceted construct encompassing affective, perceived, and subjective dimensions. While extensively explored in Human-Computer Interaction (HCI) and games user research studies, the practical evaluation of PX, especially concerning audio elements, remains challenging. This paper presents a Literature Review (LR) (n=26), which addresses the gap in the literature related to audio in PX evaluation, unveiling trends and issues. We used Creswell's methodology for LR, exploring three major research questions that investigate the approaches, instruments, and methodologies used to evaluate audio in digital games, the constructs of player experience related to audio assessment, and the challenges faced by researchers in this context. Our findings underscores challenges in the generalization of study outcomes, audio personalization and variation and audio-games evaluation. This work strives to bridge the gap between research and practice, fostering a deeper understanding of the intricate relationship between audio and PX in games.

Virtual Reality Audio Game for Entertainment & Sound Localization Training

Within the gaming and electronics industry, there is a continuous evolution of alternative applications. Nevertheless, accessibility to video games remains a persistent hurdle for individuals with disabilities, especially those with visual impairments due to the inherent visual-oriented design of games. Audio games (AGs) are electronic games that rely primarily on auditory cues instead of visual interfaces. This study focuses on the creation of a virtual reality AG for cell phones that integrates natural head and torso movements involved in spatial hearing. Its assessment encompasses user experience, interface usability, and sound localization performance. The study engaged eighteen sighted participants in a pre-post test with a control group. The experimental group underwent 7 training sessions with the AG. Via interviews, facets of the gaming experience were explored, while horizontal plane sound source localization was also tested before and after the training. The results enabled the characterization of sensations related to the use of the game and the interaction with the interfaces. Sound localization tests demonstrated distinct enhancements in performance among trained participants, varying with assessed stimuli. These promising results show advances for future virtual AGs, presenting prospects for auditory training. These innovations hold potential for skill development, entertainment, and the integration of visually impaired individuals.

Speech Puzzles (Spuzzles): Engaging the Reduced, Causal, and Semantic Listening Modes for Puzzle Design in Audio Games

This paper proposes a novel approach to audio game design by introducing the concept of speech puzzles (spuzzles) to describe the utilisation of recorded voice for the creation of audio puzzles in ways that challenge players’ different listening modes. In the fields of audio games and audio-interactive applications, speech serves instructive, descriptive, narrative, and in some cases—in the form of hints or quizzes—gameplay purposes by addressing users through language. The suggested approach of spuzzles extends this potential by including, besides encoded meaning, the acoustic properties of sound, thus engaging the user’s causal and reduced listening modes in parallel with the semantic listening mode. An audio game consisting of four inherently different spuzzles was designed as proof of concept and tested by seven third-year students of Audiovisual Arts, who elaborated on their experience through a focus group semi-structured discussion. Despite their difficulty, the spuzzles were well accepted by most of the participants (5/7), whereas all participants agreed on their acoustic richness, need for concentration, and independence from pre-existing musical knowledge. Therefore, the authors suggest that the proposed design approach could serve as a paradigm for future research in the design of complex audio-based game mechanics.

BatSight: A Navigation Game to Map Environmental Information into Audio Cues

By using sound to visualize the real-world environment, audio games can offer a completely new game experience. In this paper, we study how audio cues can be visualized in computer games to enhance the navigation skills of players. To this end, we propose an audio game in which blindfolded players move through a physical maze with the help of audio cues. To realize this game, a sonar headset is designed and built based on ultrasonic sensors, which maps the external environment features into musical sounds. The game is played in a real-world environment and uses an augmented reality approach as it adds virtual sounds to the real world. The gaming environment of this game is a physical maze, where the blindfolded players enter the maze while wearing the sonar headset. A user study was conducted to evaluate the effect of using different sound mapping techniques on navigation performance and playing experience in the game proposed in this paper. The results show that producing musical sound can lead to better navigation performance, game experience, and immersion in players.

Can the Pikmin Speak?

Abstract In this article, we take a player-centered approach to the music and sound of Pikmin 3 (2013). We investigate how the game's audio suppresses the Pikmin through a weaponized colonial aurality that hears the Pikmin as the subaltern and numbs the player into ignoring their colonialist enactments.

Sound propagation in realistic interactive 3D scenes with parameterized sources using deep neural operators.

We address the challenge of acoustic simulations in three-dimensional (3D) virtual rooms with parametric source positions, which have applications in virtual/augmented reality, game audio, and spatial computing. The wave equation can fully describe wave phenomena such as diffraction and interference. However, conventional numerical discretization methods are computationally expensive when simulating hundreds of source and receiver positions, making simulations with parametric source positions impractical. To overcome this limitation, we propose using deep operator networks to approximate linear wave-equation operators. This enables the rapid prediction of sound propagation in realistic 3D acoustic scenes with parametric source positions, achieving millisecond-scale computations. By learning a compact surrogate model, we avoid the offline calculation and storage of impulse responses for all relevant source/listener pairs. Our experiments, including various complex scene geometries, show good agreement with reference solutions, with root mean squared errors ranging from 0.02 to 0.10 Pa. Notably, our method signifies a paradigm shift as-to our knowledge-no prior machine learning approach has achieved precise predictions of complete wave fields within realistic domains.

Understanding Auditory Space in Digital Games for Visually Impaired People

The article seeks to establish a foundational framework for comprehending crucial definitions and concepts related to auditory space in digital gaming, particularly focusing on audio games designed for visually impaired individuals (VI), often devoid of graphical interfaces. While existing studies often emphasize players’ interactions with interfaces and the acquisition of real-life skills, this text explores audio games through the lens of entertainment and immersion. Numerous studies indicate a demand among visually impaired players for more intricate and challenging games, incorporating elements usually used in standard digital gaming experiences. By combining definitions from both audio and traditional digital games, this article broadens the scope of sound design considerations, encompassing various aspects and classifications. It presents several auditory dimensions, consolidating them into a comprehensive dimension called attenuation, putting their roles within a game’s context. The author is a sound designer currently working on an audio game; therefore, many of the presented definitions are also a guide that will represent some of his own considerations while working on this game. These include not only the use of binaural or ambisonic sound but also understanding the ways and means of how to work with (3D) space in the context of audio and its possible functions.

Scott Joplin in the Overworld: Super Mario's Rag-Inflected Score

Abstract A rolling banjo, walking bass, steel drum, and syncopated piano. What strange music to accompany your trials and tribulations throughout the Mushroom Kingdom. Eclecticism pervades the Mario franchise and manifests in its digital places, the ways players interact within them, and, indeed, the sounds emanating from them. Mario and his jumping adventures have long been in the hearts of players, the minds of scholars, and the ears of ludomusicologists. By engaging with Koji Kondo's scores for early Mario games, this article highlights an overlooked component in the iconic scores of early Super Mario games: ragtime music. Furthermore, this research contributes examples of African American and Latino musical creation and innovation to ludomusicological discourse. I argue that, as a means to connect, contextualize, and enhance Mario's movement with music, composer Koji Kondo draws from ragtime and early jazz styles because of their distinctive rhythmic qualities. His rag-inflected score establishes ragtime as fundamental to early Super Mario soundscapes. While a substantial ludomusicological body of literature focuses on the Western classical tradition in games, jazz-centered research is scarce. This cross-disciplinary research draws on a wide range of scholarly perspectives from jazz studies and ludomusicology, including work by Andrew Schartmann, Edward Berlin, Ingrid Monson, Neil Lerner, David Butler, and Charles Hiroshi Garrett. As game audio is indispensable in facilitating a player's sense of verisimilitude of Mario's imaginative and, at times, nonsensical worlds, an inquiry into how jazz participates in this unique audiovisual relationship is long overdue.

Editors’ introduction

This issue brings together exciting new work that reflects the expanding scholarly remit of ‘radio studies’. The six research articles are broad in scope and geographical reach, ranging from an exploration of audio in video games, to podcasting apps, industry funding models, and emerging and historical formats. Also included are two reviews of key texts published in the past year. The editors’ introduction concludes by highlighting some of the most remarkable conferences and symposia that took place in 2023.

Video Game Audio in the Metaverse and Beyond

No abstract available.

Audio for extended realities: A case study informed exposition

An area of immersive storytelling in rapid evolution is that of extended reality. This emergent mode of experience employs spatial mapping, and both plane and object detection to superimpose computer-generated images in the volumetric context of a physical space via a head-mounted display. This in turn produces a unique set of challenges and opportunities for associated audio implementation and aesthetics. Creative development of this audio is often a function of evolving toolsets, and the associated workflow is far from standardized. This paper forms a context for such audio workflow, one that draws from precursor technologies such as audio for games and virtual reality and develops this into an outline taxonomy that is both representative of the state of the art, and forward facing towards evolution of the technology stack. The context is framed through a series of case studies. Between 2019 and 2021, BBC and Oculus TV commissioned Alchemy Immersive and Atlantic Productions to produce virtual reality and mixed reality experiences of several classic documentary series by Sir David Attenborough: Museum Alive, Micro Monsters, First Life VR, Museum Alive AR and Kingdom of Plants. This portfolio received numerous award nominations and prizes, including from the Raindance Festival and a double Emmy. The sound design, audio postproduction and spatial audio for these experiences were implemented by the company 1.618 Digital, and drawing from first-hand-creator involvement, the workflows are deconstructed and explored with reference to tools, technologies, techniques and perception. Such an exposition forms the basis for an analysis of both this and broader creative practice in the field of audio for extended reality, and this is subsequently used to present a speculative vision of audio in the future of immersive storytelling.

Quartet: Designing for an Experimental Audio Game in Mixed Reality Environments

Applying sonic interaction to game design has gained momentum in recent year. This paper proposes a novel approach to design an experimental audio game Quartet that is immersive and highly interactive in the mixed reality environment. The chief aims of designing this game are to explore the music improvisation techniques by using the software Touchdesigner and to create a new spectatorship of sonic interaction in immersive experiences. To achieve the goals, several methods were presented: interactive storytelling, audio visualization, ceding control, sonic interaction, etc. The research has also recorded valuable data about players experiences of the game including their overall level of immersion, perception of the story, and overall feeling of the sound design. The results suggested that an interactive sonic design can boost players engagement and convey emotions and information more efficiently and accurately. For further research, this audio based experiment can be contributed to the field of inclusive design especially for visually impaired people and children with multiple disabilities.

Becoming Official

This article addresses technological and aesthetic intersections between DJing and modern video game audio modding, with the aim of elucidating a technological context that allows game scorers without programming backgrounds to practice audio design and programming for video games. Game audio modding is a form of music remix that depends on access to game audio programming and design technologies that are embedded in game code. Though modern audio modders still require access to this code to create advanced audio mods, game audio programming and design technologies have grown in accessibility, such as the audio middleware program FMOD Studio, the design and functionality of which resembles technologies used by DJs. In fact, DJs are uniquely suited to modern game audio modding, and hence game audio design and programming, as these activities necessitate looping and transitioning between musical themes, as well as designing and programming such techniques in relation to gameplay, resulting in a kind of gameplay-automated DJ for modded scores. To elucidate this point, this article will use video game music composed by Lena Raine, and its accompanying audio programming project file designed and programmed by Kevin Regamey and Power-Up Audio, for the 2018 multiplatform two-dimensional platformer game Celeste, as an instructive example.

3D Audio: The Next Step in Realism for Gaming: Benefits and pitfalls of 3D audio in games from a creative content and technical implementation point of view

When using 3D audio in games, the goal should be to create an immersive, believable auditory experience. To achieve this on the technical side of things, game developers must combine the appropriate hardware with sophisticated software algorithms and sound propagation techniques. Regarding content, creators need to distinguish between 3D sounds inside the scene and sounds with a narrative function that are non-diegetic. This article looks behind the curtains of the spatial audio hype and lets you know what you're in for and what is coming in the future.

Using A. I. for the generation of visual references in digital audio games from the perspective of sound design

The aim of this article is to examine the application of artificial intelligence in producing visual references during the development of a digital game. These references are a standard inclusion in Game Design Documents or Art Design Documents and are typically displayed as mood boards. In this article, we will discuss sound design and how artificial intelligence, mainly Bing utilizing Dall-E and Midjourney, can produce images that cater to the requirements of a particular game. One of the primary issues we will explore is the efficiency of using artificial intelligence in this creative process in terms of speed, accuracy, efficacy, creativity and how the resulting content compares with human-generated content. One objective is to examine how artificial intelligence can produce visual references that exemplify more detailed sound groups, such as ambiences or interactive sounds linked with specific in-game activities. These visual references will undergo testing within a digital game presently under development, thus achieving validation through real-world application. The game’s genre is an audio game with no standard graphics or interface, tailored to the needs of players with visual impairments. For that reason, generating visual references remains the sole approach to comprehending the actual game environment.

Rave Racing

In the origin story of futuristic racing game WipEout (Psygnosis, 1995), co-creator Nick Burcombe talks about turning down the game audio in Super Mario Kart (Nintendo, 1992) and substituting it for his own electronic dance music. Burcombe, who was himself a keen participant in the Liverpool rave scene, argues that people who went clubbing in the 1990s were always looking for new forms of interactive entertainment. WipEout, where players take control of anti-gravity ships and race them to the electronic dance music tracks of artists like Cold Storage, The Chemical Brothers, The Prodigy, and Orbital, was that next step in interactive entertainment.For its marketing campaign, WipEout’s packaging took its inspiration from dance music records and PlayStation installed consoles in nightclubs across the UK. In the follow-up to the original, WipEout 2097 or WipEout XL in North America (Psygnosis, 1996), players could put the game disk into a CD player and hear the soundtrack play out in its entirety, separate from the gameplay. Through a close textual analysis of the first and second versions of the game released on PlayStation, created as they were at the height of electronic dance music culture in the UK in the 1990s, and grounded in popular music studies and ludomusicology, this article inquires into the construction of the video game soundtrack, arguing that WipEout’s audiovisual relationship creates a space where players can become immersed in a rave-related experience of techno in their homes.The research finds that Burcombe and the team at Psygnosis set out not to replicate the rave experience in their video game. Rather, in WipEout, dance music is used to immerse players in a mediated extension of the contemporary UK rave venue or club. As players move their racers onscreen, they engage in actions that edge them closer to the game’s soundtrack itself, in its use of repetition and pulsating beats, as an embodied or corporeal performance. Rather than dancing to rave, WipEout’s players are gaming to techno.

Scoring Eldest Souls (2021)

In this interview conducted by Hannah Capstick, composer and sound designer Sergio Ronchetti explores the process of creating music for the game Eldest Souls (Fallen Flag Studio, 2021), which was nominated for Best Audio for an Indie Game at the 20th Game Audio Network Guild (GANG) Awards. The interview covers topics including creating the game’s audio during the COVID-19 pandemic, the influence of metal on the score, first experiences of producing audio for a game, initial experiences in the industry, self-directed education, and Ronchetti’s design and career philosophies.

City Symphony

City Symphony

The Soundlapse Project: Exploring Spatiotemporal Features of Wetland Soundscapes

Abstract This article discusses an interdisciplinary project aimed at highlighting the acoustical heritage of wetlands by means of field recordings and a novel time-lapse montage method. The authors discuss a site-specific sound installation that was designed using original wetlands field recordings, live processing and spatial audio multichannel reproduction. The discussion focuses on spatial and temporal features of different types of recorded wetlands soundscapes. Future developments of this project will consider the implementation of a standalone spatiotemporal application, to be used in the context of virtual reality applications, game audio and interactive dance performance.

Review of The Game Music Handbook

Review of <i>The Game Music Handbook</i>