Auditory Theory Research Articles

Effects of mobile phone use on driving performance in a multiresource workload scenario

Objective: This study explores the influence of mobile phone secondary tasks on driving from the perspective of visual, auditory, cognitive, and psychomotor (VACP) multiple resource theory, and it is anticipated to benefit the human-centered design of mobile phone use while driving.Methods: The present study investigated 6 typical phone use scenarios while driving and analyzed the effects of phone use distractions on driving performance. Thirty-six participants were recruited to participate in this experiment. We abandoned traditional secondary tasks such as conversations or dialing, in which cognitive resources can become interference. Instead, we adopted an arrow secondary task and an n-back delayed digit recall task.Results: The results show that all mobile phone use scenarios have a significant influence on driving performance, especially on lateral vehicle control. The visual plus psychomotor resource occupation scenario demonstrated the greatest deterioration of driving performance, and there was a significant deterioration of driving speed and steering wheel angle once the psychomotor resource was occupied.Conclusions: Phone use distraction leads to visual, cognitive, and/or motor resource functional limitations and thus causes lane violations and traffic accidents.

The contributions of Nathaniel I. Durlach to the study of informational masking

Beginning in the year 2000, Nat Durlach collaborated with our research group on the study of auditory masking and, in particular, the study of informational masking. During this collaboration of more than one and a half decades, Nat's energy, enthusiasm, creativity, and insight provided the stimulus for many research projects and ultimately a variety of publications. These many contributions to our understanding of auditory masking are embodied in journal articles, letters to the editor, grant proposals and papers at scientific conferences. This presentation will review the diverse body of empirical work that Nat drew on to formulate his views on informational masking and the implications of his work on this topic for contemporary auditory theory.

The auditory temporal processing deficit theory in children with developmental dyslexia

Developmental dyslexia (DD) or reading disability is characterized by unexpected problems in reading acquisition, despite average intelligence. It is considered the most common neurodevelopmental disorder, and many researchers aimed to identify the cognitive, neurobiological and genetic etiology of DD. At the cognitive level, the hypothesis of auditory temporal processing deficit has been widely investigated. The aim of this review is to present some of the most exciting researches on DD in the domains of this theory that will help future studies to follow.

The relationship of phonological ability, speech perception, and auditory perception in adults with dyslexia.

This study investigated whether auditory, speech perception, and phonological skills are tightly interrelated or independently contributing to reading. We assessed each of these three skills in 36 adults with a past diagnosis of dyslexia and 54 matched normal reading adults. Phonological skills were tested by the typical threefold tasks, i.e., rapid automatic naming, verbal short-term memory and phonological awareness. Dynamic auditory processing skills were assessed by means of a frequency modulation (FM) and an amplitude rise time (RT); an intensity discrimination task (ID) was included as a non-dynamic control task. Speech perception was assessed by means of sentences and words-in-noise tasks. Group analyses revealed significant group differences in auditory tasks (i.e., RT and ID) and in phonological processing measures, yet no differences were found for speech perception. In addition, performance on RT discrimination correlated with reading but this relation was mediated by phonological processing and not by speech-in-noise. Finally, inspection of the individual scores revealed that the dyslexic readers showed an increased proportion of deviant subjects on the slow-dynamic auditory and phonological tasks, yet each individual dyslexic reader does not display a clear pattern of deficiencies across the processing skills. Although our results support phonological and slow-rate dynamic auditory deficits which relate to literacy, they suggest that at the individual level, problems in reading and writing cannot be explained by the cascading auditory theory. Instead, dyslexic adults seem to vary considerably in the extent to which each of the auditory and phonological factors are expressed and interact with environmental and higher-order cognitive influences.

Musical Source Clustering and Identification in Polyphonic Audio

For music transcription or musical source separation, apart from knowing the multi-F0 contours, it is also important to know which F0 has been played by which instrument. This paper focuses on this aspect, i.e. given the polyphonic audio along with its multiple F0 contours, the proposed system clusters them so as to decide ‘which instrument played when.’ For the task of identifying the instrument or singers in the polyphonic audio, there are many supervised methods available. But many times individual source audio is not available for training. To address this problem, this paper proposes novel schemes using semi-supervised as well as unsupervised approach to source clustering. The proposed theoretical framework is based on auditory perception theory and is implemented using various tools like probabilistic latent component analysis and graph clustering, while taking into account various perceptual cues for characterizing a source. Experiments have been carried out over a wide variety of datasets - ranging from vocal to instrumental as well as from synthetic to real world music. The proposed scheme significantly outperforms a state of the art unsupervised scheme, which does not make use of the given F0 contours. The proposed semi-supervised approach also performs better than another semi-supervised scheme, which makes use of the given F0 information, in terms of computations as well as accuracy.

Robust audio fingerprinting based on GammaChirp frequency cepstral coefficients and chroma

A novel auditory feature that combines an auditory model and music theory is proposed for audio fingerprinting. First, the input audio is filtered by a GammaChirp (GC) filterbank to model the cochlear frequency selectivity. Then, the output of the filterbank is down-sampled and decorrelated by a discrete cosine transform to obtain the GammaChirp frequency cepstral coefficients (GCFCCs). Next, some lowest order GCFCCs are projected onto the chroma to characterise both melodic and harmonic information of the input. Finally, non-negative matrix factorisation is applied to the chroma matrix to reduce its dimension while maintaining its discriminative power. The experimental results illustrate that the proposed scheme achieves a stabler identification rate and lower computational complexity than the schemes based on the Mel-frequency cepstral coefficients.

Bertram Scharf and his critical contributions to the field

Jerry Tobias introduced me to Bert Scharf via his book Foundations of Modern Auditory Theory. Our research in the 1970s was on the prediction of speech intelligibility from acoustic measurements; the similarity between Fletcher's 20 frequency bands of equal contributions to intelligibility and psychoacoustic measurements of critical bands was of great interest. Bert's chapter on critical bands provided an insightful, comprehensive, concise, and critical review of the state of the art in critical-band research. Our mutual research interests brought us together when he asked me to review a draft of a paper he was preparing. My review was highly critical and after submitting it to him I felt I had been too unforgiving in my review (reviewers can be wrong). I called him to explain that I had been overly critical of a fine paper. His response was the opposite of what I expected. He said it was the best review he had received and that he was extremely grateful for my input. I realized then that he—not only set high standards for others—but for himself as well. By adhering consistently to his high standards, his many contributions to the field have been long-lasting and critical.

Social cognitive deficits and their neural correlates in progressive supranuclear palsy

Although progressive supranuclear palsy is defined by its akinetic rigidity, vertical supranuclear gaze palsy and falls, cognitive impairments are an important determinant of patients’ and carers’ quality of life. Here, we investigate whether there is a broad deficit of modality-independent social cognition in progressive supranuclear palsy and explore the neural correlates for these. We recruited 23 patients with progressive supranuclear palsy (using clinical diagnostic criteria, nine with subsequent pathological confirmation) and 22 age- and education-matched controls. Participants performed an auditory (voice) emotion recognition test, and a visual and auditory theory of mind test. Twenty-two patients and 20 controls underwent structural magnetic resonance imaging to analyse neural correlates of social cognition deficits using voxel-based morphometry. Patients were impaired on the voice emotion recognition and theory of mind tests but not auditory and visual control conditions. Grey matter atrophy in patients correlated with both voice emotion recognition and theory of mind deficits in the right inferior frontal gyrus, a region associated with prosodic auditory emotion recognition. Theory of mind deficits also correlated with atrophy of the anterior rostral medial frontal cortex, a region associated with theory of mind in health. We conclude that patients with progressive supranuclear palsy have a multimodal deficit in social cognition. This deficit is due, in part, to progressive atrophy in a network of frontal cortical regions linked to the integration of socially relevant stimuli and interpretation of their social meaning. This impairment of social cognition is important to consider for those managing and caring for patients with progressive supranuclear palsy.

Masking effects of speech and music: Does the masker's hierarchical structure matter?

Speech and music are time-varying signals organized by parallel hierarchical rules. Through a series of four experiments, this study compared the masking effects of single-talker speech and instrumental music on speech perception while manipulating the complexity of hierarchical and temporal structures of the maskers. Listeners’ word recognition was found to be similar between hierarchically intact and disrupted speech or classical music maskers (Experiment 1). When sentences served as the signal, significantly greater masking effects were observed with disrupted than intact speech or classical music maskers (Experiment 2), although not with jazz or serial music maskers, which differed from the classical music masker in their hierarchical structures (Experiment 3). Removing the classical music masker's temporal dynamics or partially restoring it affected listeners’ sentence recognition; yet, differences in performance between intact and disrupted maskers remained robust (Experiment 4). Hence, the effect of structural expectancy was largely present across maskers when comparing them before and after their hierarchical structure was purposefully disrupted. This effect seemed to lend support to the auditory stream segregation theory.SumarioEl lenguaje y la música son señales que varían en tiempo y que están organizadas por reglas jerárquicas paralelas. A través de una serie de cuatro experimentos, este estudio compara los efectos enmascarantes de un solo hablante y de música instrumental en la percepción del lenguaje mientras se manipulaba la complejidad de las estructuras jerárquicas y temporales de los enmascaradores. Se encontró que el reconocimiento de las palabras por parte de los oyentes fue similar utilizando lenguaje jerárquicamente intacto o alterado o enmascaramiento con música clásica (experimento 1). Cuando se utilizaron frases como señal, se observó un efecto enmascarante significativamente mayor con lenguaje alterado que con lenguaje intacto o con enmascarador de música clásica (experimento 2), aunque no con jazz o enmascaramiento de música serial, lo que difrrió del enmascarador de música clásica en su estructura jerárquica (experimento 3). Al remover la dinámica temporal del enmascarador de música clásica o al restaurarla parcialmente, se afectó el reconocimiento de frases.; sin embargo, las diferencias en el desempeño entre los enmascaradores intacto y perturbado siguieron siendo amplias (experimento 4). Por lo tanto, el efecto de la expectación estructural estuvo muy presente en todos y cada uno de los enmascaradores cuando fueron comparados antes y después de que su estructura jerárquica fuera profundamente perturbada. Este efecto parece apoyar la teoría de la segregación del flujo auditivo.

Correlations between auditory structures and hearing sensitivity in non‐human primates

Primates show distinctions in hearing sensitivity and auditory morphology that generally follow phylogenetic patterns. However, few previous studies have attempted to investigate how differences in primate hearing are directly related to differences in ear morphology. This research helps fill this void by exploring the form-to-function relationships of the auditory system in a phylogenetically broad sample of non-human primates. Numerous structures from the outer, middle, and inner ears were measured in taxa with known hearing capabilities. The structures investigated include the overall size and shape of the pinna, the areas of the tympanic membrane and stapedial footplate, the masses and lever arm lengths of the ossicles, the volumes of the middle ear cavities, and the length of the cochlea. The results demonstrate that a variety of auditory structures show significant correlations with certain aspects of hearing (particularly low-frequency sensitivity). Although the majority of these relationships agree with expectations from auditory theory, some traditional (and possibly outdated) ideas were not supported. For example, the common misconception that higher middle ear transformer ratios (e.g., impedance transformer ratio) result in increased hearing sensitivity was not supported. Although simple correlations between form and function do not necessarily imply causality, the relationships defined in this study not only increase our understanding of auditory patterns in extant taxa but also lay the foundation to begin investigating the hearing in fossil primates.

Cochlear outer hair cell system is a logarithmic compressor

In general, sensors of the sensory organs are ultrasensitive, so that a gain control function would be essential to protect the sensors from high intensity stimuli. This holds for the cochlea, an acoustic sensor of high sensitivity. Above all, the cochlea has to be equipped with the organ mechanically protecting the vulnerable sensor from high amplitude sounds. In addition, to expand the apparent dynamic range of the cochlear sensor, the stimulus intensity i.e. amplitude, should be considerably compressed in the cochlea. Of course, the amplitude should be compressed logarithmically so as not to miss the low level signals as much as possible. Thus, it is very convenient if a logarithmic compressor (LC) exists in the cochlea. Then, we examined literatures on this subject, and finally we reached the conclusion that the outer hair cell (OHC) system is none other than the LC. Here we propose ‘the LC theory’ that the OHC system is the LC properly compressing the signal intensity together with the tectorial membrane. However, this theory is quite contradictory to the generally accepted theory that OHCs are selective amplifiers of the basilar membrane vibrations (CA theory). In our opinion, this contradiction is due to that the logic of the CA theory is not sound. If you enumerate all possible explanations of experimental data and examine them logically, you will never conclude to the CA theory. In a word, the CA theory confuses the effect of the LC with that of the selective amplifier. Certainly, they are very mistakable. On the other hand, because the LC theory is a logical result of the well-known experimental data, it is consistent with any of them. Moreover, the LC theory can easily explain the causes of such phenomena as otoacoustic emissions, two-tone suppression and loudness recruitment that have been difficult to be reasonably explained up to now. Needless to say, if the OHC system is the LC, you will have to re-create the auditory theory fundamentally. This means that every aspect of otology such as the clinical examination, the design of cochlear implant and the etiology of hearing impairments should be re-evaluated. The conclusion is that the CA theory should be corrected as quickly as possible.

Zum Zusammenhang zwischen basaler auditorischer und phonologischer Verarbeitung

It is generally accepted that dyslexia arises due to deficits in phonological processing. However, whether these deficits are based on impaired auditory processing is still subject to debate. Here we tested whether low level auditory processing is correlated with a simple phonological processing task, thus indicating a close relationship between higher sensory and lower cognitive processing. We carried out a prospective study with 200 school children attending the third and fourth grades. The dependent variables used were just noticeable differences in intensity and frequency (JNDI, JNDF), gap detection (GD), monaural and binaural temporal order judgment (TOJ(b) and TOJ(m)), minimal pair perception and reproduction skills for real words and nonsense words. No relevant correlation was found between any auditory low level processing variable and minimal pair representation and reproduction skills. These data do not support the auditory processing deficit theory.

The across frequency independence of equalization of interaural time delay in the equalization-cancellation model of binaural unmasking.

The equalization stage in the equalization-cancellation model of binaural unmasking compensates for the interaural time delay (ITD) of a masking noise by introducing an opposite, internal delay [N. I. Durlach, in Foundations of Modern Auditory Theory, Vol. II., edited by J. V. Tobias (Academic, New York, 1972)]. Culling and Summerfield [J. Acoust. Soc. Am. 98, 785-797 (1995)] developed a multi-channel version of this model in which equalization was "free" to use the optimal delay in each channel. Two experiments were conducted to test if equalization was indeed free or if it was "restricted" to the same delay in all channels. One experiment measured binaural detection thresholds, using an adaptive procedure, for 1-, 5-, or 17-component tones against a broadband masking noise, in three binaural configurations (N0S180, N180S0, and N90S270). The thresholds for the 1-component stimuli were used to normalize the levels of each of the 5- and 17-component stimuli so that they were equally detectable. If equalization was restricted, then, for the 5- and 17-component stimuli, the N90S270 and N180S0 configurations would yield a greater threshold than the N0S180 configurations. No such difference was found. A subsequent experiment measured binaural detection thresholds, via psychometric functions, for a 2-component complex tone in the same three binaural configurations. Again, no differential effect of configuration was observed. An analytic model of the detection of a complex tone showed that the results were more consistent with free equalization than restricted equalization, although the size of the differences was found to depend on the shape of the psychometric function for detection.

Auditory target detection in reverberation.

Measurements and theoretical predictions of auditory target detection in simulated reverberant conditions are reported. The target signals were pulsed 1/3-octave bands of noise and the masker signal was a continuous wideband noise. Target and masker signals were passed through a software simulation of a reverberant room with a rigid sphere modeling a listener's head. The location of the target was fixed while the location of the masker was varied in the simulated room. Degree of reverberation was controlled by varying the uniform acoustic absorption of the simulated room's surfaces. The resulting target and masker signals were presented to the listeners over headphones in monaural-left, monaural-right, or binaural listening modes. Changes in detection performance in the monaural listening modes were largely predictable from the changes in target-to-masker ratio in the target band, but with a few dB of extra masking in reverberation. Binaural detection performance was generally well predicted by applying Durlach's [in Foundations of Modern Auditory Theory (Academic, New York, 1972)] equalization-cancellation theory to the direct-plus-reverberant ear signals. Predictions in all cases were based on a statistical description of room acoustics and on acoustic diffraction by a sphere. The success of these detection models in the present well-controlled reverberant conditions suggests that they can be used to incorporate listening mode and source location as factors in speech-intelligibility predictions.

Jozef Zwislocki: Impact on models of coding in the auditory nerve

The auditory nerve has long been considered a window on the biophysical mechanisms of cochlear transduction and the most carefully characterized aspect of the responses of single auditory‐nerve fibers has been the tuning curve. Perhaps the most intensively studied question in auditory theory is: What is the relationship between the shapes of these tuning curves and basilar membrane displacements? The basilar membrane measurements of Georg von Bekesy stimulated a generation of basilar‐membrane modelers, none more notable than Joe Zwislocki, who was awarded the first von Bekesy Medal by the Acoustical Society in 1985. The impact of Zwislocki’s basilar membrane models on our understanding of auditory nerve tuning will be reviewed. The properties of auditory‐nerve discharge patterns are also shaped by the filtering properties of the hair cell/synapse complex. The major contributions of Joe and his students to our understanding of this filtering through their elegant experimental and modeling studies of adaptation in the auditory nerve will be presented. Throughout his career, Joe Zwislocki has maintained an active interest in loudness summation and his work in relating the input/output characteristics of auditory‐nerve fibers to loudness will be highlighted.

Jozef Zwislocki’s integrated approach to psychoacoustics

Early in the psychophysics phase of his illustrious and varied scientific career, Joe Zwislocki recognized that loudness is of key importance for understanding signal processing by the auditory system. To better comprehend just how loudness can be incorporated into basic auditory theory, Joe sought to determine the loudness‐intensity relation down to near threshold levels. Together with Hellman, this aim motivated the development of absolute magnitude‐scaling procedures. Later, Joe demonstrated that absolute scaling yields results compatible with nonmetric measures of loudness additivity. His search for a comprehensive theory of auditory function led him to deduce that both the observed proportionality between loudness and sound intensity near threshold and the compressive nonlinearity in the loudness function at moderate‐to‐high levels are generated by the peripheral auditory system. These innovative concepts were incorporated into his theoretical analyses of temporal summation and central masking. They also provided the basis of a loudness model formulated to describe loudness growth in quiet and in noise. Not only are Joe Zwislocki’s extraordinary insights compatible with recent loudness measures, his loudness model can be extended successfully to predict the growth of loudness characteristic of cochlear‐impaired hearing. [Work supported by NEDO, Japan.]

A wave traveling over a Hopf instability shapes the cochlear tuning curve.

The tuning curve of the cochlea measures how intense an input is required to elicit a given output level as a function of the frequency. It is a fundamental object of auditory theory, for it summarizes how to identify sounds on the basis of the cochlear output. A simple model is presented showing that only two elements are sufficient for establishing the cochlear tuning curve: a broadly tuned traveling wave, moving unidirectionally from high to low frequencies, and a set of mechanosensors poised at the threshold of an oscillatory (Hopf) instability. These two components generate the various frequency-response regimes needed for a cochlear tuning curve with a high slope.

Modeling the relation between the intensity just-noticeable difference and loudness for pure tones and wideband noise

A classical problem in auditory theory is the relation between the loudness L(I) and the intensity just-noticeable difference (JND) ΔI(I). The intensity JND is frequently expressed in terms of the Weber fraction defined by J(I)≡ΔI/I because it is anticipated that this ratio should be a constant (i.e., Weber’s law). Unfortunately, J(I) is not a constant for the most elementary case of the pure tone JND. Furthermore it remains unexplained why Weber’s law holds for wide-band stimuli. We explore this problem and related issues. The loudness and the intensity JND are defined in terms of the first and second moments of a proposed random decision variable called the single-trial loudness L̃(I), namely the loudness is L(I)≡EL̃(I), while the variance of the single trial loudness is σL2≡E(L̃−L)2. The JND is given by the signal detection assumption ΔL=d′σL, where we define the loudness JND ΔL(I) as the change in loudness corresponding to ΔI(I). Inspired by Hellman and Hellman’s recent theory [J. Acoust. Soc. Am. 87, 1255–1271 (1990)], we compare the Riesz [Phys. Rev. 31, 867–875 (1928)] ΔI(I) data to the Fletcher and Munson [J. Acoust. Soc. Am. 5, 82–108 (1933)] loudness growth data. We then make the same comparison for Miller’s [J. Acoust. Soc. Am. 19, 609–619 (1947)] wideband noise JND and loudness match data. Based on this comparison, we show empirically that ΔL(L)∝L1/p, where p=2 below ≈5 sones and is 1 above. Since ΔL(I) is proportional to σL, when p=2 the statistics of the single-trial loudness L̃ are Poisson-like, namely σL2∝L. This is consistent with the idea that the pure tone loudness code is based a neural discharge rate (not the auditory nerve). Furthermore, when p=1 (above about 5 sones), the internal loudness signal-to-noise ratio is constant. It is concluded that Ekman’s law (ΔL/L is constant) is true, rather than Weber’s law, in this loudness range. One of the main contributions of this paper is its attempt to integrate Fletcher’s neural excitation pattern model of loudness and signal detection theory.

An auditory cortical theory of primitive auditory grouping

Auditory stream segregation is considered from the point of view of a theory of the central auditory system in which temporal information is spatially coded on three dimensions roughly corresponding to the level of the cochlea, the inferior colliculus and the auditory cortex, respectively. This theory is embodied in the form of a computational model which simulates peripheral and central processing including a cortical cross-channel correlation mechanism.

Estimating the azimuth of a sound source from the binaural spectral amplitude

A computational model of auditory localization based on processing spectral amplitude cues for localizing broadband high frequency sound sources is presented. The cues extracted are binaural spectral level di erence patterns of the Head Related Transfer Functions (HRTF) corresponding to the direction of the sound source. Four di erent pattern classi ers are used to evaluate the feasibility of localization using such spectral cues. The four classi ers investigated are normalized correlation, maximum likelihood estimation, a backpropagation based fuzzy neural network model and a novel linear ltering method which optimizes a new Discriminative Matching Measure (DMM). The DMM is de ned to quantify the relative ability of each of these classi ers to identify such patterns in the presence of noise or with missing data. An independent estimate of the localization acuity using the RMS error is also made. The results indicate that both optimal DMM ltering and the fuzzy neural network model are successful in extracting azimuthal location of broadband high frequency sound sources. The proposed model attempts to complement current auditory localization theory by exploring the role of HRTF based binaural spectral level di erence cues in localizing high frequency broadband auditory stimuli.