Virtual Sound Sources Research Articles

Stereo sound expander

The invention is a stereo sound expander which reproduces a realistic sound image in three dimensions. Prior art systems suffered from the disadvantage that the creation of virtual sound sources or loudspeakers gave rise to a tonal change. Attempts to correct the tonal change impaired the accuracy of the virtual sound source. The invention overcomes the problems experienced by prior art systems, by coupling modified and unmodified stereo signals. Thus by not modifying a Head Related Transfer Function (HRTF) for one signal and equalising the HRTF for the other signal, a flattened frequency response is produced with no tonal changes, but with a high degree of spatial accuracy. Resultant output signals may be used to generate binaural signals or can be fed into crosstalk cancellers.

Application of multisource discrimination to the HELS method

In most engineering applications, the acoustic fields are usually generated by multiple incoherent sources. When such acoustic fields are sampled and taken as input data to the Helmholtz equation least-squares (HELS) formulations directly, the reconstruction might not be accurate. Hence there is a need to discriminate the contributions from individual sources and separate the composite sound field into a set of spatially coherent subfields that are also mutually incoherent. In this paper, we apply the principal component decomposition technique to the HELS method to reconstruct acoustic radiation from multiple incoherent sources in a free field. The key ingredients of this technique include a diagonalization by singular value decomposition (SVD) of the cross-spectral matrices generated by a number of reference microphones. Each diagonal term corresponds to a subfield that results from a so-called virtual sound source. Even though such virtual sources are not always representative of the actual sources, they are nonetheless incoherent and thus can be used as input to HELS. Experimental validations of this technique on reconstructing acoustic pressure and time-averaged normal acoustic intensity on the surfaces of multiple incoherent sources are presented. [Work supported by NSF.]

Reduced order modeling of head related transfer functions for virtual acoustic displays

The purpose of this work is to improve the computational efficiency in acoustic virtual applications by creating and testing reduced order models of the head related transfer functions used in localizing sound sources. State space models of varying order were generated from zero-elevation Head Related Impulse Responses (HRIRs) using Kungs Single Value Decomposition (SVD) technique. The inputs to the models are the desired azimuths of the virtual sound sources (from minus 90 deg to plus 90 deg, in 10 deg increments) and the outputs are the left and right ear impulse responses. Trials were conducted in an anechoic chamber in which subjects were exposed to real sounds that were emitted by individual speakers across a numbered speaker array, phantom sources generated from the original HRIRs, and phantom sound sources generated with the different reduced order state space models. The error in the perceived direction of the phantom sources generated from the reduced order models was compared to errors in localization using the original HRIRs.

Auditory and visual distance perception: The proximity-image effect revisited

The proximity-image effect [M. B. Gardner, J. Acoust. Soc. Am. 43, 163 (1968)] describes a phenomenon in which the apparent distance of an auditory target is determined by the distance of the nearest plausible visual target rather than by acoustic distance cues. Here this effect is examined using a single visual target (an un-energized loudspeaker) and invisible virtual sound sources. These sources were synthesized from binaural impulse-response measurements at distances ranging from 1 to 5 m (0.25-m steps) in the semi-reverberant room (7.7 m×4.2 m×2.7 m) in which the experiment was conducted. Listeners (n=11) were asked whether or not the auditory target appeared to be at the same distance as the visual target. Within a block of trials, the visual target was placed at a fixed distance of 1.5, 3, or 4.5 m, and the auditory target varied randomly from trial-to-trial over the sample of measurement distances. The resulting psychometric functions are consistent with the proximity-image effect, and can be predicted using a simple model of sensory integration and decision in which perceived auditory space is both compressed in distance and has lower resolution than perceived visual space. [Work supported by NIH-NEI.]

Method of synthesizing an audio signal

A method of synthesizing an audio signal having left and right channels corresponding to an extended virtual sound source at a given apparent location in space relative to a preferred position of a listener in use is described. The information in the channels includes cues for perception of the direction of said virtual sound source from the preferred position. The extended source comprises a plurality of point virtual sources, the sound from each point source being spatially related to the sound from the other point sources, such that sound appears to be emnitted from an extended region of space. If the signal from two sound sources is the same, they are modified to be sufficiently different from one another to be separately distinguishable by a listener when they are disposed symmetrically on either side of the listener. This modification can be accomplished by filtering the two point sources using different comb filters.

Ultra-directional speaker system and speaker system drive method

An ultra-directional speaker system has a speech generation means, an ultrasonic generation means, an amplitude modulation means to which the speech generation means and ultrasonic generation means are connected, an electro-acoustic transducer means which converts an ultrasonic modulated signal that is output from the amplitude modulation means to an acoustic signal, and a virtual sound source positioning means that establishes the position of a virtual sound source by detecting the position of a listener by detecting the ultrasonic waves that are output from the electro-acoustic transducer means and reflected from the listener.

Method of positioning sound image with distance adjustment

A sound apparatus is constructed for directing a sound image of a virtual sound source at a designated source point to a listener in a virtual sound field. In the sound apparatus, a database provisionally memorizes acoustic transfer characteristics of the virtual sound field in correspondence to reference source points distributed radially around a center point of the listener. Left and right filters respectively filter audio signals of left and right channels according to the acoustic transfer characteristics loaded from the database. A processor computes a leftward acoustic direction from the designated source point to a left ear of the listener, and computes a rightward acoustic direction from the designated source point to a right ear of the listener. A controller specifies a leftward reference source point coincident with the leftward acoustic direction to load an effective acoustic transfer characteristic corresponding to the leftward reference source point from the database into the left filter, and specifies a rightward reference source point coincident with the rightward acoustic direction to load another effective acoustic transfer characteristic corresponding to the rightward reference source point from the database into the right filter. A feeder feeds an audio signal of the left channel to the left filter and feeds another audio signal of the right channel to the right filter to thereby direct the sound image of the virtual sound source located at the source point to the listener positioned at the center point.

Direct-to-reverberant energy ratio sensitivity.

Although the ratio of direct-to-reverberant sound energy is known to be an important acoustic cue to sound source distance, human sensitivity to changes in this cue is largely unknown. Here, direct-to-reverberant energy discrimination thresholds were measured for six listeners using virtual sound source techniques that allow for convenient and precise control of this stimulus parameter. Four different types of source stimuli were tested: a 50 ms noise burst with abrupt onset/offset, a 300 ms duration noise burst with gradual onset/offset, a speech syllable, and an impulse. Over a range of direct-to-reverberant ratios from 0 to 20 dB, an adaptive 2AFC procedure (3-down, 1-up) was used to measure discrimination thresholds. For all stimuli, these thresholds ranged from 5 to 6 dB. A post hoc fitting procedure confirmed that slopes of the psychometric functions were homogeneous across stimulus conditions and listeners. These threshold results suggest that direct-to-reverberant energy ratio by itself provides only a course coding of sound source distance, because threshold values correspond to greater than 2-fold changes in physical distance for the acoustic environment under examination.

The Helical Keyboard: Perspectives for Spatial Auditory Displays and Visual Music

Auditory displays with the ability to dynamically spatialize virtual sound sources under real-time conditions enable advanced applications for art and music. A listener can be deeply immersed while interacting and participating in the experience. We review some of those applications while focusing on the Helical Keyboard project and discussing the required technology. Inspired by the cyclical nature of octaves and helical structure of a scale, a model of a piano-style keyboard was prepared, which was then geometrically warped into a helicoidal configuration, one octave/revolution, pitch mapped to height and chroma. It can be driven by MIDI events, real-time or sequenced, which stream is both synthesized and spatialized by a spatial sound display. The sound of the respective notes is spatialized with respect to sinks, avatars of the human user, by default in the tube of the helix. Alternative coloring schemes can be applied, including a color map compatible with chromastereoptic eyewear. The graphical display animates polygons, interpolating between the notes of a chord across the tube of the helix. Recognition of simple chords allows directionalization of all the notes of a major triad from the position of its musical root. The system is designed to allow, for instance, separate audition of harmony and melody, commonly played by the left and right hands, respectively, on a normal keyboard. Perhaps the most exotic feature of the interface is the ability to fork one’s presence, replicating subject instead of object by installing multiple sinks at arbitrary places around a virtual scene so that, for example, harmony and melody can be separately spatialized, using two heads to normalize the octave; such a technique effectively doubles the helix from the perspective of a single listener. Rather than a symmetric arrangement of the individual helices, they are perceptually superimposed in-phase, co-extensively, so that corresponding notes in different registers are at the same azimuth.

Trajectory perception in the free field

Using virtual sound sources in an echo-attenuated chamber, the sensitivity of subjects to the direction of linear auditory motion was evaluated. Four sighted and four blind subjects participated. Subjects reported the perceived trajectory by orienting a freely rotating pointer. Subjects fell into two distinct categories: those who performed well at the task, and those who did not, with blind subjects tending to be in the first category and sighted subjects the second. Those subjects performing well at the task also showed improved performance with increasing linear velocity. Because the only stimulus parameter associated with increased linear velocity was frequency envelope, it can be concluded that frequency changes resulting from the Doppler effect can improve the accuracy of auditory trajectory perception. The same subjects performed discrimination tasks measuring sensitivity to changes in azimuthal acceleration, as well as to changes in amplitude and frequency envelopes. Two models of trajectory perception are considered: A Psychoacoustical Parameters Model uses a weighted combination of specific perceptual abilities such as sensitivity to azimuthal acceleration, and to changes in amplitude and frequency envelopes. An Informational Masking Model is considered as a possible explanation for those subjects demonstrating poor performance in the trajectory perception task.

A Method for Changing Sound-Image Position Using the Linear Loudspeaker Array and Two-Dimensional FIR Digital Filter

In this paper, the problem of sound-image control is considered as that of wave front shape of the acoustic beam radiated from a speaker array. A method is proposed for realizing the sound-image control in an acoustic beam by providing the necessary delay characteristics to a two-dimensional FIR filter as the group delay. The filter design is carried out by means of a two-dimensional Fourier series approximation in which the fan-shaped passband amplitude is specified for formation of the radiation pattern in addition to the required group delay. It is found that the degree of approximation in the filter design can be improved substantially by making the radius of the circle set for modification infinite. Further, the center of the circle forms a focal point as in an optical system and this point becomes a virtual sound source and hence the position of the sound image. The wave is expanding in a manner of concentric circles from this point. The width of the acoustic beam generated from this sound image is determined by the spread of the speaker array and the position of the sound image. © 2001 Scripta Technica, Electron Comm Jpn Pt 3, 85(4): 20–31, 2002

A Method for Changing Sound‐Image Position Using the Linear Loudspeaker Array and Two‐Dimensional FIR Digital Filter

AbstractIn this paper, the problem of sound‐image control is considered as that of wave front shape of the acoustic beam radiated from a speaker array. A method is proposed for realizing the sound‐image control in an acoustic beam by providing the necessary delay characteristics to a two‐dimensional FIR filter as the group delay. The filter design is carried out by means of a two‐dimensional Fourier series approximation in which the fan‐shaped passband amplitude is specified for formation of the radiation pattern in addition to the required group delay. It is found that the degree of approximation in the filter design can be improved substantially by making the radius of the circle set for modification infinite. Further, the center of the circle forms a focal point as in an optical system and this point becomes a virtual sound source and hence the position of the sound image. The wave is expanding in a manner of concentric circles from this point. The width of the acoustic beam generated from this sound image is determined by the spread of the speaker array and the position of the sound image. © 2001 Scripta Technica, Electron Comm Jpn Pt 3, 85(4): 20–31, 2002

From Spectrum to Space: The Contribution of Level Difference Cues to Spatial Receptive Fields in the Barn Owl Inferior Colliculus

Space-specific neurons in the owl's inferior colliculus have spatial receptive fields (RFs) computed from interaural time (ITD) and level (ILD) differences. Because of the shape of the owl's head, these cues vary with frequency in a manner specific for each location. We sought to determine the contribution of ILD to spatial selectivity. We measured the normal spatial receptive fields of space-specific neurons using virtual sound sources (i.e., noises filtered to simulate external sound sources, presented using headphones). The virtual-source filters were then altered so that ITD was fixed while frequency-specific ILDs varied according to location in the usual manner. The resulting "ILD-alone" RF typically revealed a horizontal band of excitation that included the normal RF. Above and below, the neurons were inhibited. Interestingly, the maxima of ILD-alone RFs were generally outside the normal RF, suggesting that space-specific neurons are not optimally tuned to the ILD spectrum occurring at the normal RF location. Congruously, frequency-specific ILD tuning, assessed with tones, better matched the ILDs at the peak of the ILD-alone RF than those at the peak of the normal RF. The firing evoked from the normal RF may thus reflect the balance of excitatory and inhibitory inputs needed to appropriately restrict the receptive field. Frequency-specific ILD tuning curves were combined with measured head-filtering characteristics to predict responses to the frequency-specific ILDs at each location. The predicted ILD-alone RFs, which are based on a simple sum of frequency-specific inputs, accounted for 56% of the variance in our measured ILD-alone RFs.

Listening through different ears alters spatial response fields in ferret primary auditory cortex.

The localization of sounds in space is based on spatial cues that arise from the acoustical properties of the head and external ears. Individual differences in localization cue values result from variability in the shape and dimensions of these structures. We have mapped spatial response fields of high-frequency neurons in ferret primary auditory cortex using virtual sound sources based either on the animal's own ears or on the ears of other subjects. For 73% of units, the response fields measured using the animals' own ears differed significantly in shape and/or position from those obtained using spatial cues from another ferret. The observed changes correlated with individual differences in the acoustics. These data are consistent with previous reports showing that humans localize less accurately when listening to virtual sounds from other individuals. Together these findings support the notion that neural mechanisms underlying auditory space perception are calibrated by experience to the properties of the individual.

Effects of visual-feedback training in 3-D sound displays

Current low-cost 3-D sound displays do not use individualized head-related transfer functions (HRTFs) to render acoustic space. As a result, sound source localization accuracy is often degraded when compared to the accuracy using real sources, or to higher quality displays using individualized HRTFs. Here, a way to improve accuracy was examined in which listeners were provided with paired auditory and visual feedback as to the correct sound source location. Sound localization accuracy was assessed for six listeners, using a large number of virtual sound sources sampled from a spherical grid surrounding the listener, before, during, and after feedback training. Feedback training markedly improved localization accuracy compared to a control group of five listeners that did not receive training. The largest improvements in accuracy resulted from listeners’ enhanced abilities to distinguish sources in front from sources behind. Further, these improvements were not transient short-term effects, but lasted at least 4 days between training and testing sessions. These results suggest that simple and relatively short periods of feedback training (two 45-min sessions) can effectively facilitate perceptual re-mapping to modified spatial cues, and therefore mitigate technical deficiencies in 3-D sound systems due to nonindividualized HRTFs. [Work supported by ARL-FedLab.]

How Plastic Is Spatial Hearing?

The location of a sound source is derived by the auditory system from spatial cues present in the signals at the two ears. These cues include interaural timing and level differences, as well as monaural spectral cues generated by the external ear. The values of these cues vary with individual differences in the shape and dimensions of the head and external ears. We have examined the neurophysiological consequences of these intersubject variations by recording the responses of neurons in ferret primary auditory cortex to virtual sound sources mimicking the animal’s own ears or those of other ferrets. For most neurons, the structure of the spatial response fields changed significantly when acoustic cues measured from another animal were presented. This is consistent with the finding that humans localize less accurately when listening to virtual sounds from other subjects. To examine the role of experience in shaping the ability to localize sound, we have studied the behavioural consequences of altering binaural cues by chronically plugging one ear. Ferrets raised and tested with one ear plugged learned to localize as accurately as control animals, which is consistent with previous findings that the representation of auditory space in the midbrain can accommodate abnormal sensory cues during development. Adaptive changes in behaviour were also observed in adults, particularly if they were provided with regular practice in the localization task. Together, these findings suggest that the neural circuits responsible for sound localization can be recalibrated throughout life.

Distance localization using nonindividualized head-related transfer functions

Past research has suggested that virtual auditory displays are most accurate when implemented using individualized head-related transfer functions (HRTFs). Directional localization accuracy, for example, is often substantially degraded when nonindividualized HRTFs are used, due to the resulting distortion of spectral cues to sound source direction. Although equally important for accurate spatial representation, little is known about distance localization accuracy in virtual auditory displays, and specifically, whether individualized HRTFs are necessary for accurate distance localization. Here, distance localization accuracy was assessed for six listeners, using both individualized and nonindividualized HRTFs in separate blocks of trials. The virtual sound sources were produced from special HRTF measurements (source distance ranging from 0.3 to 13.8 m) that also contained the acoustic transfer characteristics of a semireverberant room environment. This provided listeners with additional acoustic cues to source distance, including the ratio of direct-to-reverberant sound energy. Since no significant differences were observed in listener’s judgments of apparent sound source distance in the two stimulus conditions, it is concluded that distance localization accuracy, and degree of sound source externalization, do not depend on the use of individualized HRTFs when natural reverberation is available to the listener. [Work supported by NIH-NIDCD, NASA, and NIH-NEI.]

Inverse filter design for immersive audio rendering over loudspeakers

Immersive audio systems can be used to render virtual sound sources in three-dimensional (3-D) space around a listener. This is achieved by simulating the head-related transfer function (HRTF) amplitude and phase characteristics using digital filters. In this paper, we examine certain key signal processing considerations in spatial sound rendering over headphones and loudspeakers. We address the problem of crosstalk inherent in loudspeaker rendering and examine two methods for implementing crosstalk cancellation and loudspeaker frequency response inversion in real time. We demonstrate that it is possible to achieve crosstalk cancellation of 30 dB using both methods, but one of the two (the Fast RLS Transversal Filter Method) offers a significant advantage in terms of computational efficiency. Our analysis is easily extendable to nonsymmetric listening positions and moving listeners.

Head-related transfer functions of the Rhesus monkey

Head-related transfer functions (HRTFs) are direction-specific acoustic filters formed by the head, the pinnae and the ear canals. They can be used to assess acoustical cues available for sound localization and to construct virtual auditory environments. We measured the HRTFs of three anesthetized Rhesus monkeys ( Macaca mulatta) from 591 locations in the frontal hemisphere ranging from −90° (left) to 90° (right) in azimuth and −60° (down) to 90° (up) in elevation for frequencies between 0.5 and 15 kHz. Acoustic validation of the HRTFs shows good agreement between free field and virtual sound sources. Monaural spectra exhibit deep notches at frequencies above 9 kHz, providing putative cues for elevation discrimination. Interaural level differences (ILDs) and interaural time differences (ITDs) generally vary monotonically with azimuth between 0.5 and 8 kHz, suggesting that these two cues can be used to discriminate azimuthal position. Comparison with published subsets of HRTFs from squirrel monkeys ( Saimiri sciureus) shows good agreement. Comparison with published human HRTFs from the frontal hemisphere demonstrates overall similarity in the patterns of ILD and ITD, suggesting that the Rhesus monkey is a good acoustic model for these two sound localization cues in humans. Finally, the measured ITDs in the horizontal plane agree well between −40° and 40° in azimuth with those calculated from a spherical head model with a radius of 52 mm, one-half the interaural distance of the monkey.

Sound localization with eccentric head position

This study investigates the influence of head-to-trunk position on auditory localization in humans. Various methods of head pointing, of two-alternative forced choice, and hand pointing were employed. Head-pointing toward actual sound sources in darkness, by using only the subjective median plane of the head as a reference, resulted in systematic underestimations of target eccentricity. The deviations of the terminal head position from the target shifted with a mean slope of ∼0.1° per degree change in head position. A corresponding shift in the localization of virtual sound sources (presented via headphones during eccentric head positions) was demonstrated by requiring forced-choice (left or right) responses with respect to the subjective median plane of the head. Head pointing toward remembered auditory targets in darkness resulted in undershoots similar to those found with actual targets. However, when a visual marker of the actual median plane of the head was additionally presented to the subject during these tasks (by a laser attached to the head that projected a spot onto a screen), sound localization was fairly accurate. Localization of eccentric auditory targets by using a swivel hand pointer also showed systematic errors similar to those found with head pointing in darkness when the head was simultaneously oriented toward the sound. When the head remained in alignment with the trunk, hand pointing resulted in overshooting responses. These results may be related to neural processes, presumably in the posterior parietal cortex, that transform auditory and visual spatial coordinates into a common, trunk-centered, frame of reference.