Audio Display Research Articles

Sonic Visualiser: Visualisation, Analysis, and Annotation of Music Audio Recordings

Sonic Visualiser: Visualisation, Analysis, and Annotation of Music Audio Recordings

Spatialized audio improves call sign recognition during multi-aircraft control

We investigated the impact of a spatialized audio display on response time, workload, and accuracy while monitoring auditory information for relevance. The human ability to differentiate sound direction implies that spatial audio may be used to encode information. Therefore, it is hypothesized that spatial audio cues can be applied to aid differentiation of critical versus noncritical verbal auditory information. We used a human performance model and a laboratory study involving 24 participants to examine the effect of applying a notional, automated parser to present audio in a particular ear depending on information relevance. Operator workload and performance were assessed while subjects listened for and responded to relevant audio cues associated with critical information among additional noncritical information. Encoding relevance through spatial location in a spatial audio display system--as opposed to monophonic, binaural presentation--significantly reduced response time and workload, particularly for noncritical information. Future auditory displays employing spatial cues to indicate relevance have the potential to reduce workload and improve operator performance in similar task domains. Furthermore, these displays have the potential to reduce the dependence of workload and performance on the number of audio cues.

Composing [De]Composition: Data Sonification for Sound Art and Music Composition

Author(s): Parker, Jennifer Andrea (No.E) | Advisor(s): Dicke, Ian J | Abstract: Composing [De]Composition is a large-scale BioArt/sonification research installation that was presented at the University of California Riverside’s Sweeney Art Gallery from June-October 2015. This paper introduces compost—the main material of Composing [De]Composition—as a rich site for creative exploration and expression via the medium of data sonification. Here, the author non-reductively describes the multi-agential and poly-temporal nature of compost and data sonification through detailing the evolution of a process-based, techno-ecological artistic praxis involving: the observation, audification, and sonification of compost temperatures; the development of new sensing methods for data collection; parameter mapping; audio display/spatial sound design; dataset-based digital music composition and musification for acoustic instrumentation. The main observable driving the project is incalescence—the heat generated by the composting process. During the exhibition/research period, audification of this biological process brought a perceivably silent activity into the tangible reach of human hearing. The collection and real-time audification of temperature data using a custom interface to route sensor data to an eight-point audio display enabled listeners to better understand the complex ecology of a heterogeneous mass that was simultaneously decomposing, supporting a myriad of life forms while also enabling the bioavailability of macronutrients to the soil. The recontexualization of compost temperature data into sound also creates fertile ground for exploration in the realm of music composition—while the collection of data over time depicts inherent patterns occurring in the system analyzed, the basis of music also builds upon the use of patterns—pitched, rhythmic, and dynamic—through time. Sonification of compost temperature patterns not only capacitates human auditory observation of what is normally a perceivably silent physical biological process but also enables the composer/sound artist to create compositions in partnership with her phenomenon of study.

Free-Field Localization Performance With a Head-Tracked Virtual Auditory Display

Virtual auditory displays are systems that use signal processing techniques to manipulate the apparent spatial locations of sounds when they are presented to listeners over headphones. When the virtual audio display is limited to the presentation of stationary sounds at a finite number of source locations, it is possible to produce virtual sounds that are essentially indistinguishable from sounds presented by real loudspeakers in the free field. However, when the display is required to reproduce sound sources at arbitrary locations and respond in real-time to the head motions of the listener, it becomes much more difficult to maintain localization performance that is equivalent to the free field. The purpose of this paper is to present the results of a study that used a virtual synthesis technique to produce head-tracked virtual sounds that were comparable in terms of localization performance with real sound sources. The technique made use of an in-situ measurement and reproduction technique that made it possible to switch between the head-related transfer function measurement and the psychoacoustic validation without removing the headset from the listener. The results demonstrate the feasibility of using head-tracked virtual auditory displays to generate both short and long virtual sounds with localization performance comparable to what can be achieved in the free field.

Sound localization with head movement: implications for 3-d audio displays.

Previous studies have shown that the accuracy of sound localization is improved if listeners are allowed to move their heads during signal presentation. This study describes the function relating localization accuracy to the extent of head movement in azimuth. Sounds that are difficult to localize were presented in the free field from sources at a wide range of azimuths and elevations. Sounds remained active until the participants' heads had rotated through windows ranging in width of 2, 4, 8, 16, 32, or 64° of azimuth. Error in determining sound-source elevation and the rate of front/back confusion were found to decrease with increases in azimuth window width. Error in determining sound-source lateral angle was not found to vary with azimuth window width. Implications for 3-d audio displays: the utility of a 3-d audio display for imparting spatial information is likely to be improved if operators are able to move their heads during signal presentation. Head movement may compensate in part for a paucity of spectral cues to sound-source location resulting from limitations in either the audio signals presented or the directional filters (i.e., head-related transfer functions) used to generate a display. However, head movements of a moderate size (i.e., through around 32° of azimuth) may be required to ensure that spatial information is conveyed with high accuracy.

Effect of Velocity and Time-Step on the Continuity of a Discrete Moving Sound Image

As a basic study into 3-D audio display systems, this paper reports the conditions of moving sound image velocity and time-step where a discrete moving sound image is perceived as continuous motion. In this study, the discrete moving sound image was presented through headphones and ran along the ear-axis. The experiments tested the continuity of a discrete moving sound image using various conditions of velocity (0.25, 0.5, 0.75, 1, 2, 3, and 4 m/s) and time-step (0, 0.02, 0.04, 0.06, 0.08, 0.10, 0.12, and 0.14 s). As a result, the following were required in order to present the discrete moving sound image as continuous movement. (1) The 3-D audio display system was required to complete the sound image presentation process, including head tracking and HRTF simulation, in a time shorter than 0.02 s, in order to present sound image movement at all velocities. (2) A processing time longer than 0.1 s was not acceptable. (3) If the 3-D audio display system only presented very slow movement (less than about 0.5 m/s), processing times ranging from 0.04 s to 0.06 s were still acceptable.

The End Is Where We Start From: Communicating the Impact of a Family Music Project to Wider Audiences

There has been increasing pressure for anthropologists to communicate their ideas and thinking to new publics and so actively engage in national and international debates relating to their field. However, this is not an unproblematic practice and the politics of representation requires anthropologists to consider the sometimes conflicting dimensions of the moral, ethical, political, social, personal and academic. My fieldwork with families linked to In Harmony Liverpool, a children's music project in England, involved inviting participants variously to take part in interviews, draw maps of musical sites in their homes, construct playlists of favourite songs and take photographs of sites in their homes where music 'happens'. As my aim is to produce a visual and audio display to communicate with wider audiences, I consider the issues of representation, authenticity, potential damage and 'othering' in the planning of the research and how this shaped data collection and the plans for dissemination.

Aurally Aided Visual Threat Acquisition in a Virtual Urban Battlespace

Many of today’s military operations take place in large urban environments, which present unique challenges due to limited line of sight and increased concealment for enemy forces. This experiment evaluated the potential of a 3D audio display to aid threat detection and localization in a complex visual environment. Subjects rode as part of a convoy through a simulated city where they encountered snipers surrounded by distracting personnel. We compared 3D audio cues to verbal descriptions of the sniper’s location and to simple audio warnings of the presence of a sniper. Consistent with past research, subjects located the sniper more quickly in the 3D audio condition compared to both the semantic description and simple warning conditions. The 3D audio display was faster than the other displays across all azimuths but, in contrast to previous findings, the advantage did not increase consistently with increasing azimuth.

P‐96: Development of Fully Integrated Vacuum Fluorescent Display (FIVFD)

VFD (Vacuum Fluorescent Display) is used for various kind of applications such as automotive, audio, video, amusement, household appliance and hot water supply. The futures of VFD are 1 High luminance 2 Wide operating temperature (The response time is less then 10usec.) 3 Wide viewing angle due to self luminance display. Therefore, VFD is installed in the vehicle as audio, meter, clock odo/trip meter, HVAC display. In recent display trend, display content has moved to reconfigurable type form fixed segment type. Especially, multi information display and audio display require reconfigurable type display. On this back ground, high luminance and high resolution reconfigurable VFD was developed. The thin film multilayer wiring technology has allowed high multiplex driving. It was not available at conventional VFD. As a result, 256×64 dots 0.25mm dot pitch reconfiguration, 2000 cd/m2 in luminance display wad developed. Besides, a multi color (Green and Red) VFD was developed also. Now, these displays are on the production.

Head‐related transfer function enhancement for improved vertical‐polar localization.

Under ideal laboratory conditions, individualized head‐related transfer functions (HRTFs) can produce virtual sound localization performance approaching the level achieved with real sound sources in the free field. However, in real‐world applications of virtual audio, practical issues such as fit‐refit variability in the headphone response and nonindividualized HRTFs generally lead to much worse localization performance, particularly in the up‐down and front‐back dimensions. Here we present a new technique that “enhances” the localizability of a virtual sound source by increasing the spectral contrast of the acoustic features that are relevant for spatial perception within a set of locations with nearly identical binaural cues (i.e., a “cone‐of‐confusion”). Validation experiments show that this enhancement technique can improve localization accuracy across a broad range of conditions, with as much as a 33% reduction in vertical‐polar localization error for nonindividualized HRTFs measured on a KEMAR manikin; a 25% reduction in vertical‐polar error for nonindividualized HRTFs measured on other human listeners; and a 33% reduction in vertical‐polar error for individualized HRTFs presented under nonideal laboratory conditions (i.e., with headphone fit‐refit variability). These results suggest that the proposed technique could provide benefits across a wide range of real‐world virtual audio display applications. [Work sponsored by AFOSR.]

Virtual Audio Systems

To be immersed in a virtual environment, the user must be presented with plausible sensory input including auditory cues. A virtual (three-dimensional) audio display aims to allow the user to perceive the position of a sound source at an arbitrary position in three-dimensional space despite the fact that the generated sound may be emanating from a fixed number of loudspeakers at fixed positions in space or a pair of headphones. The foundation of virtual audio rests on the development of technology to present auditory signals to the listener's ears so that these signals are perceptually equivalent to those the listener would receive in the environment being simulated. This paper reviews the human perceptual and technical literature relevant to the modeling and generation of accurate audio displays for virtual environments. Approaches to acoustical environment simulation are summarized and the advantages and disadvantages of the various approaches are presented.

On Visual, Vibrotactile, and 3D Audio Directional Cues for Dismounted Soldier Waypoint Navigation

We compared a visual GPS, a tactile torso belt, and a 3D audio display in a waypoint navigation task for dismounted soldiers. Using these displays one at a time, the soldier's main tasks were to walk as fast and straight as possible towards the waypoints while visually scanning the terrain for detection of target flags. The results showed that all three types of displays gave similar navigation precision and target detection performance. The visual display entailed a somewhat higher navigation speed than the tactile and 3D audio displays. Both the visual and 3D audio displays, however, were rated as directing attention away from the terrain more than the tactile display. Compared to the tactile display, a higher mental workload was reported for the 3D audio, which was also rated least suitable for operational use in navigation tasks. Yet, the soldiers' ability to localize directions to waypoints with the 3D audio display may point to its potential use for radio communication and cueing of directions to threats. In an overall evaluation, eight soldiers preferred the tactile display and four the visual, while none preferred the 3D audio. We discuss the results mainly regarding shortcomings of the study and the possibility of developing and combining the three display types for the dismounted soldier.

An “Audio Annotation” Technique for Intuitive Communication in Spatial Tasks

Navigating in any environment can be a very tedious task, and it becomes considerably more difficult when the environment is unfamiliar or when it contains threats that could possibly jeopardize the success of the mission. Often in difficult navigation environments there are external sensors present in the area that could provide critical information to the ground operator. The challenge is to find a way to transmit this information to the ground operator in an intuitive, timely, and unambiguous manner. In this study, we explore a technique called “audio annotation” where the sensor information is transmitted to a remote observer who processes it and relays it verbally to an operator on the ground. Spatial information can be conveyed intuitively by using a spatial audio display to project the apparent location of the remote observer's voice to an arbitrary location relative to the ground operator. The current study compared the “audio annotation” technique to standard monaural communications in a task that required a remote observer with a high-level view of the environment to assist a ground operator to avoid threats while locating a downed pilot in an urban environment. The overall performance in the audio annotation condition was found to be superior to the standard monaural condition.

Localization of Sound Presented Via a Spatial Audio Display during Visually Induced Vection in Pitch, Roll, and Yaw

There is growing interest in the potential application of 3D audio displays to impart spatial information to aircraft pilots. As the aviation environment is dynamic, the effect on sound localization of the visual and vestibular stimulation resulting from motion during flight needs to be assessed. In this study, we investigated whether sound localization is affected by visual cues to self-motion. Participants localized bursts of broadband acoustic noise presented from the frontal hemifield under conditions of yaw, pitch, and roll vection (around the z-, y-, and x-axis, respectively) induced by wide-field visual motion of a projected sinusoidal grating. Lateral and elevation components of localization errors were measured. Average signed lateral and elevation errors were less than 1.6 degrees for all vection conditions. The effects of vection condition on lateral and elevation errors were not statistically significant. Also, the interactions between vection condition and sound-source location for these errors were not significant. The magnitudes of the 95% confidence intervals for the differences in lateral and elevation errors between the no-vection condition and each of the other vection conditions did not exceed 3 degrees. In the presence of visual cues to yawing, pitching, or rolling self-motion, the perception of auditory space is generally consistent with the available head-centered auditory cues to sound-source location. These results support the application of 3D audio displays in dynamic visual environments such as that experienced during flight.

TimbreFields: 3D Interactive Sound Models for Real-Time Audio

We describe a methodology for virtual reality designers to capture and resynthesize the variations in sound made by objects when we interact with them through contact such as touch. The timbre of contact sounds can vary greatly, depending on both the listener’s location relative to the object, and the interaction point on the object itself. We believe that an accurate rendering of this variation greatly enhances the feeling of immersion in a simulation. To do this, we model the variation with an efficient algorithm based on modal synthesis. This model contains a vector field that is defined on the product space of contact locations and listening positions around the object. The modal data are sampled on this high dimensional space using an automated measuring platform. A parameter-fitting algorithm is presented that recovers the parameters from a large set of sound recordings around objects and creates a continuous timbre field by interpolation. The model is subsequently rendered in a real-time simulation with integrated haptic, graphic, and audio display. We describe our experience with an implementation of this system and an informal evaluation of the results.

Spatial Audio Displays Improve the Detection of Target Messages in a Continuous Monitoring Task

The detection of target messages in a background of competing speech and the identification of the color/number combinations in those messages were examined in a continuous monitoring task. Previous research has shown that if listeners know when and where to listen, speech-on-speech intelligibility is improved when signals are presented via a 3-D audio display as compared with a diotic display. However, the effect of display type on detection of infrequent target messages in a continuous monitoring task has not been examined. Participants were required to monitor five communications channels conveying messages at random intervals under each of three audio display conditions: diotic, all channels in front, and channels separated in azimuth (3-D). Message detection sensitivity was significantly higher for the 3-D condition than for the in-front condition but did not differ significantly between the in-front and the diotic conditions. There were no differences in response criteria across conditions. Color/number identification sensitivity also was significantly higher for the 3-D condition than for the in-front condition but did not differ significantly between the in-front and the diotic conditions. A 3-D audio display enhances both message detection and message identification in a continuous monitoring task. Three-dimensional audio displays would be particularly beneficial in environments such as aviation, in which the information conveyed to operators via the auditory modality can be crucial to the safe and effective performance of their work.

Localization in a Head-Related Transfer Function-based virtual audio synthesis using additional high-pass and low-pass filtering of sound sources

Listening tests were carried out for investigating the localization judgments of untrained subjects through equalized headphones and with HRTF synthesis. The investigation was made on the basis of the former ‘Graphical User Interface for Blind Persons’ project in order to determine the possibilities of a 2D virtual sound screen and headphone playback. 50 untrained subjects evaluated a virtual audio display in front of the listener using different horizontal and vertical resolutions on a 2D surface. A listening test using white and filtered noise signals was followed by a special investigation using simple high-pass and low-pass filtering of the original sound in order to increase correct vertical localization judgments. The simulation uses high-pass filtering for higher elevations and low-pass filtering for lower elevations in a 5×2 and a 3×3 spatial resolution. Results of the listening test will be presented and the efficiency of the filtering in correct localization judgments will be discussed.

1A2-F04 人間の感覚面から見た力感覚通信システムの基本特性(感覚・運動・計測)

Recently, advancement of virtual reality technologies can provide various force sensations to users. In addition to current visual and audio display technologies, it is expected that a force display would further expand application fields of multimedia services. This study develops two haptic interfaces with 3'DOF for force telecommunication that have an orthodox parallel linkage using two kinds of actuators, this paper details the fundamental property of these devices. Further, this paper tries to reveal the relationships between human sensations and system architecture such as servomechanism in order to realize force telecommunication with reality.

Situated play in a tangible interface and adaptive audio museum guide

This paper explores the design issues of situated play within a museum through the study of a museum guide prototype that integrates a tangible interface, audio display, and adaptive modeling. We discuss our use of design ethnography in order to situate our interaction and to investigate the liminal and engagement qualities of a museum visit. The paper provides an overview of our case study and analysis of our user evaluation. We discuss the implications including degrees of balance in the experience design of play in interaction; the challenge in developing a discovery-based information model, and the need for a better understanding of the contextual aspects of tangible user interfaces (TUIs). We conclude that learning effectiveness and functionality can be balanced productively with playful interaction through an adaptive audio and TUI if designers balance the engagement between play and the environment, and the space between imagination and interpretation that links the audio content to the artifacts.

Spatial audio target acquisition in the frontal horizontal plane: Effects of feedback, mobility, and index of difficulty

The results of an empirical study investigating the effect of feedback, mobility, and index of difficulty on a deictic spatial audio target acquisition task in the horizontal plane in front of a user are presented. Audio feedback marked display elements are found to enable usable deictic interaction that is performed with high accuracy and can be described using Fitts’ law. Feedback does not affect perceived workload or preferred walking speed compared to interaction without feedback. Without audio feedback, spatial audio is not found to support adequately the corrective submovements necessary for high selection accuracies. Interaction with the display becomes therefore highly inefficient. Mobility is found to degrade interaction speed and accuracy by 20%. Participants were able to perform deictic spatial audio target acquisition when mobile while walking at 73% of their preferred walking speed. The proposed feedback design is examined in detail and the effects of variable target widths are quantified. Deictic interaction with a spatial audio display is found to be a feasible solution for future interface designs.