Auditory Model Research Articles

Auditory discrimination of natural soundscapes.

A previous modelling study reported that spectro-temporal cues perceptually relevant to humans provide enough information to accurately classify "natural soundscapes" recorded in four distinct temperate habitats of a biosphere reserve [Thoret, Varnet, Boubenec, Ferriere, Le Tourneau, Krause, and Lorenzi (2020). J. Acoust. Soc. Am. 147, 3260]. The goal of the present study was to assess this prediction for humans using 2 s samples taken from the same soundscape recordings. Thirty-one listeners were asked to discriminate these recordings based on differences in habitat, season, or period of the day using an oddity task. Listeners' performance was well above chance, demonstrating effective processing of these differences and suggesting a general high sensitivity for natural soundscape discrimination. This performance did not improve with training up to 10 h. Additional results obtained for habitat discrimination indicate that temporal cues play only a minor role; instead, listeners appear to base their decisions primarily on gross spectral cues related to biological sound sources and habitat acoustics. Convolutional neural networks were trained to perform a similar task using spectro-temporal cues extracted by an auditory model as input. The results are consistent with the idea that humans exclude the available temporal information when discriminating short samples of habitats, implying a form of a sub-optimality.

Universal global compact representation of head-related transfer functions with hemi-spherical harmonics and conformal mapping

The head-related transfer function(HRTF) is an essential part of spatial auditory display systems. In recent years, numerous HRTF databases are established to include human subjects' measurements, which enabled data-driven research projects such as HRTF prediction and personalization. However, in most cases, each HRTF database has its own unique measuring standard and source direction grid set. It's difficult to merge and efficiently represent the global HRTF information across multiple HRTF databases. Current presentation methods, such as principle component analysis (PCA) and spherical harmonics transform (SHT), are constrained by the source grid layout and regularization errors, which lays burdens for common feature extraction from different sets of HRTF measurements. In this work, we propose a novel approach for the global compact representation of HRTFs across different measuring standards, using hemispherical harmonics (HSH) and conformal mapping. The method takes into account the spatial domain covered by the typical HRTF measurements and proves to be less contained by the source direction grids. Both numerical and auditory model experiments are performed, to examine the representation error and consistency when including multiple HRTF databases with different measuring standards and grid layouts. In continuing work, we are using this method as a pre-processing approach for HRTF personalization utilizing multiple HRTF databases.

Two stages of bandwidth scaling drives efficient neural coding of natural sounds.

Theories of efficient coding propose that the auditory system is optimized for the statistical structure of natural sounds, yet the transformations underlying optimal acoustic representations are not well understood. Using a database of natural sounds including human speech and a physiologically-inspired auditory model, we explore the consequences of peripheral (cochlear) and mid-level (auditory midbrain) filter tuning transformations on the representation of natural sound spectra and modulation statistics. Whereas Fourier-based sound decompositions have constant time-frequency resolution at all frequencies, cochlear and auditory midbrain filters bandwidths increase proportional to the filter center frequency. This form of bandwidth scaling produces a systematic decrease in spectral resolution and increase in temporal resolution with increasing frequency. Here we demonstrate that cochlear bandwidth scaling produces a frequency-dependent gain that counteracts the tendency of natural sound power to decrease with frequency, resulting in a whitened output representation. Similarly, bandwidth scaling in mid-level auditory filters further enhances the representation of natural sounds by producing a whitened modulation power spectrum (MPS) with higher modulation entropy than both the cochlear outputs and the conventional Fourier MPS. These findings suggest that the tuning characteristics of the peripheral and mid-level auditory system together produce a whitened output representation in three dimensions (frequency, temporal and spectral modulation) that reduces redundancies and allows for a more efficient use of neural resources. This hierarchical multi-stage tuning strategy is thus likely optimized to extract available information and may underlies perceptual sensitivity to natural sounds.

Bispectral feature speech intelligibility assessment metric based on auditory model

A bispectral feature based predictive speech intelligibility metric (GMBSIM) using a more refined functional auditory model of ear is proposed. In the auditory model of ear, Gammatone filter banks and Meddis inner hair cell auditory model is combined to simulate the ear function. With input speech signal divided into 32 auditory subbands, and each subband signal passed through the inner hair cell model, the bispectrum of each subband signal in time domain is estimated by frames. And then bispectral features are extracted and chosen to calculate the speech intelligibility. The proposed GMBSIM has relative low computational complexity by omitting the spectrogram or neurogram image transformation. Considering the ear's perception and processing of speech signals makes the metric is advantageous to the classical metrics. And the last but not the least, the proposed GMBSIM metric is verified favorably across a range of conditions spanning reverberation, additive noise, and distortion such as jitter, which means it can be applied in most kinds of complex background noise environment.

Envelope following responses for hearing diagnosis: Robustness and methodological considerations

Recent studies have found that envelope following responses (EFRs) are a marker of age-related and noise- or ototoxic-induced cochlear synaptopathy (CS) in research animals. Whereas the cochlear injury can be well controlled in animal research studies, humans may have an unknown mixture of sensorineural hearing loss [SNHL; e.g., inner- or outer-hair-cell (OHC) damage or CS] that cannot be teased apart in a standard hearing evaluation. Hence, a direct translation of EFR markers of CS to a differential CS diagnosis in humans might be compromised by the influence of SNHL subtypes and differences in recording modalities between research animals and humans. To quantify the robustness of EFR markers for use in human studies, this study investigates the impact of methodological considerations related to electrode montage, stimulus characteristics, and presentation, as well as analysis method on human-recorded EFR markers. The main focus is on rectangularly modulated pure-tone stimuli to evoke the EFR based on a recent auditory modelling study that showed that the EFR was least affected by OHC damage and most sensitive to CS in this stimulus configuration. The outcomes of this study can help guide future clinical implementations of electroencephalography-based SNHL diagnostic tests.

Development of auditory models for measurement non-invasive intracranial pressure based on middle ear dynamics

Development of auditory models for measurement non-invasive intracranial pressure based on middle ear dynamics

A model framework for simulating spatial hearing of bilateral cochlear implant users

Bilateral cochlear implants (CIs) greatly improve spatial hearing acuity for CI users, but substantial gaps still exist compared to normal-hearing listeners. For example, CI users have poorer localization skills, little or no binaural unmasking, and reduced spatial release from masking. Multiple factors have been identified that limit binaural hearing with CIs. These include degradation of cues due to the various sound processing stages, the viability of the electrode-neuron interface, impaired brainstem neurons, and deterioration in connectivity between different cortical layers. To help quantify the relative importance and inter-relationship between these factors, computer models can and arguably should be employed. While models exploring single stages are often in good agreement with selected experimental data, their combination often does not yield a comprehensive and accurate simulation of perception. Here, we combine information from CI sound processing with computational auditory model stages in a modular and open-source framework, resembling an artificial bilateral CI user. The main stages are (a) binaural signal generation with optional head-related impulse response filtering, (b) generic CI sound processing not restricted to a specific manufacturer, (c) electrode-to-neuron transmission, (d) binaural interaction, and (e) a decision model. The function and the outputs of different model stages are demonstrated with examples of localization experiments. However, the model framework is not tailored to a specific dataset. It offers a selection of sound coding strategies and allows for third-party model extensions or substitutions; thus, it is possible to employ the model for a wide range of binaural applications and even for educational purposes.

The effect of information processing based on sensory modeling in learning to shoot from jumping high in handball

The development of the students' ability to process information through understanding, remembering, and cognition, as well as their reliance on many forms, such as classification of this information, analysis, storage, and retrieval when necessary, according to the student's sensory modeling systems (visual, auditory, sensory), which constitutes a unique way for her to receive knowledge and information. and experience, arranging and organizing this information, recording, encoding, and retaining it, and then recalling it when needed. Because of the significance of this, as well as the fact that the educational curricula used in teaching handball do not include those important information-processing strategies, as well as a lack of attention to sensory modeling (visual, auditory, sensory) that the student enjoys, the researcher resorted to better educational alternatives. By developing an educational curriculum for information processing strategy, sensory modeling systems were used to learn and retain some basic handball skills. The study sought to develop a learning curriculum that includes information processing based on sensory modeling (audio-visual-sensory) in order to teach the skill of shooting from a high jump in handball. In order to identify the difference between those with sensory modeling (visual - auditory - sensory) in the amount of information processing and in learning some shooting skill from jumping high with the handball, as well as the effect of the prepared curriculum. The researcher used a dual-design experimental method. This study was conducted on 86 female second-grade students at the College of Physical Education for Girls / University of Baghdad for the academic year 2021-2022. A sensory modeling test (visual, auditory, sensory), and a handball shooting skill test were also conducted. The researcher concluded the following: The suitability of the information processing strategy curriculum for modeling women and their mental characteristics has resulted in positive results in increasing information processing ability and learning the fundamental skill of shooting from high jumping in handball. The ability of female students with sensory models to process information is superior to that of female learners with visual and auditory models.

Multiple time-instances features based approach for reference-free speech quality measurement

This paper investigates the problem of measuring speech quality of received speech signal without employing the original speech signal. The problem of deterioration of the speech quality arises due to noise present in the surroundings. To this line, we propose a multiple time-instances (MTI) features-based approach for reference-free speech quality measurement model. A voice activity detector (VAD) is exploited primarily for calculating the number of active speech chunks of a speech signal. For these chunks and their successive combinations called here batches, multi-resolution auditory model (MRAM), mel-frequency cepstral coefficients (MFCC) and line spectral frequencies (LSF) features are extracted and called as MTI features. It is hypothesized that the MTI features are capable in capturing the distortions caused by time-localized effects of short-time transients, impulsive noise, and its differences from the plosive sounds. The MTI metric estimates (MTI-ME) are calculated corresponding to these MTI features employing the Gaussian mixture model (GMM) probabilistic technique. The overall objective speech quality of a speech signal is then determined as a linear combination of optimally weighted MTI-ME corresponding to distinct active speech chunks and their successive combinations, that is, batches of that speech signal. Minimum mean square error criterion or Pearson’s correlation maximization criterion is employed for computing optimal weights. In addition, a deep neural network (DNN)-based speech quality model is also developed for calculating a single objective speech quality while considering all active speech chunks together. Pearson’s correlation coefficient and weighted average correlation are exploited for evaluating the performance. Results demonstrate that the proposed model achieves promising improvement over the standard speech quality model (P.563) and improves correlation values by around 37%.

Frequency dependence of sensitivity to interaural phase differences in pure tones.

It is well established that in normal-hearing humans, the threshold of interaural time differences for pure tones increases dramatically above about 1300 Hz, only to become unmeasurable above 1400 Hz. However, physiological data and auditory models suggest that the actual decline in sensitivity is more gradual and only appears to be abrupt because the maximum of the psychometric function dips below the threshold proportion correct, e.g., 0.794. Published data only report thresholds at certain proportions correct but not the decline of proportions correct or of the sensitivity index d' with increasing frequencies. Here, we present pure-tone behavioral data obtained with a constant stimulus procedure. Seven of nine subjects showed proportions correct above 0.9 at 1300 Hz and virtually no sensitivity at 1500 Hz (proportion correct within 0.07 of chance level). This corresponds to a sensitivity decline of 46-78 dB/oct, much steeper than predicted by existing models or by the decline of phase locking of the auditory nerve fibers in animal data.

Binaural detection thresholds and audio quality of speech and music signals in complex acoustic environments.

Every-day acoustical environments are often complex, typically comprising one attended target sound in the presence of interfering sounds (e.g., disturbing conversations) and reverberation. Here we assessed binaural detection thresholds and (supra-threshold) binaural audio quality ratings of four distortions types: spectral ripples, non-linear saturation, intensity and spatial modifications applied to speech, guitar, and noise targets in such complex acoustic environments (CAEs). The target and (up to) two masker sounds were either co-located as if contained in a common audio stream, or were spatially separated as if originating from different sound sources. The amount of reverberation was systematically varied. Masker and reverberation had a significant effect on the distortion-detection thresholds of speech signals. Quality ratings were affected by reverberation, whereas the effect of maskers depended on the distortion. The results suggest that detection thresholds and quality ratings for distorted speech in anechoic conditions are also valid for rooms with mild reverberation, but not for moderate reverberation. Furthermore, for spectral ripples, a significant relationship between the listeners' individual detection thresholds and quality ratings was found. The current results provide baseline data for detection thresholds and audio quality ratings of different distortions of a target sound in CAEs, supporting the future development of binaural auditory models.

A novel hybrid feature method based on Caelen auditory model and gammatone filterbank for robust speaker recognition under noisy environment and speech coding distortion

A novel hybrid feature method based on Caelen auditory model and gammatone filterbank for robust speaker recognition under noisy environment and speech coding distortion

Acoustically driven orientation and navigation in enclosed spaces.

Awareness of space, and subsequent orientation and navigation in rooms, is dominated by the visual system. However, humans are able to extract auditory information about their surroundings from early reflections and reverberation in enclosed spaces. To better understand orientation and navigation based on acoustic cues only, three virtual corridor layouts (I-, U-, and Z-shaped) were presented using real-time virtual acoustics in a three-dimensional 86-channel loudspeaker array. Participants were seated on a rotating chair in the center of the loudspeaker array and navigated using real rotation and virtual locomotion by "teleporting" in steps on a grid in the invisible environment. A head mounted display showed control elements and the environment in a visual reference condition. Acoustical information about the environment originated from a virtual sound source at the collision point of a virtual ray with the boundaries. In different control modes, the ray was cast either in view or hand direction or in a rotating, "radar"-like fashion in 90° steps to all sides. Time to complete, number of collisions, and movement patterns were evaluated. Navigation and orientation were possible based on the direct sound with little effect of room acoustics and control mode. Underlying acoustic cues were analyzed using an auditory model.

Strong energy component is more important than spectral selectivity in modeling responses of midbrain auditory neurons to wide-band environmental sounds

Modeling central auditory neurons in response to complex sounds not only helps understanding neural processing of speech signals but can also provide insights for biomimetics in neuro-engineering. While modeling responses of midbrain auditory neurons to synthetic tones is rather good, modeling those to environmental sounds is less satisfactory. Environmental sounds typically contain a wide range of frequency components, often with strong and transient energy. These stimulus features have not been examined in the conventional approach of auditory modeling centered on spectral selectivity. To this end, we firstly compared responses to an environmental sound of auditory midbrain neurons across 3 subpopulations of neurons with frequency selectivity in the low, middle and high ranges; secondly, we manipulated the sound energy, both in power and in spectrum, and compared across these subpopulations how their modeled responses were affected. The environmental sound was recorded when a rat was drinking from a feeding bottle (called the ‘drinking sound’). The sound spectrum was divided into 20 non-overlapping frequency bands (from 0 to 20 kHz, at 1 kHz width) and presented to an artificial neural model built on a committee machine with parallel spectral inputs to simulate the known tonotopic organization of the auditory system. The model was trained to predict empirical response probability profiles of neurons to the repeated sounds. Results showed that model performance depended more on the strong energy components than on the spectral selectivity. Findings were interpreted to reflect general sensitivity to rapidly changing sound intensities at the auditory midbrain and in the cortex.

The Influence of Male- and Female-Spoken Vowel Acoustics on Envelope-Following Responses.

The influence of male and female vowel characteristics on the envelope-following responses (EFRs) is not well understood. This study explored the role of vowel characteristics on the EFR at the fundamental frequency (f0) in response to the vowel /ε/ (as in "head"). Vowel tokens were spoken by five males and five females and EFRs were measured in 25 young adults (21 females). An auditory model was used to estimate changes in auditory processing that might account for talker effects on EFR amplitude. There were several differences between male and female vowels in relation to the EFR. For male talkers, EFR amplitudes were correlated with the bandwidth and harmonic count of the first formant, and the amplitude of the trough below the second formant. For female talkers, EFR amplitudes were correlated with the range of f0 frequencies and the amplitude of the trough above the second formant. The model suggested that the f0 EFR reflects a wide distribution of energy in speech, with primary contributions from high-frequency harmonics mediated from cochlear regions basal to the peaks of the first and second formants, not from low-frequency harmonics with energy near f0. Vowels produced by female talkers tend to produce lower-amplitude EFR, likely because they depend on higher-frequency harmonics where speech sound levels tend to be lower. This work advances auditory electrophysiology by showing how the EFR evoked by speech relates to the acoustics of speech, for both male and female voices.

Predicting Perceived Vocal Roughness Using a Bio-Inspired Computational Model of Auditory Temporal Envelope Processing.

Vocal roughness is often present in many voice disorders but the assessment of roughness mainly depends on the subjective auditory-perceptual evaluation and lacks acoustic correlates. This study aimed to apply the concept of roughness in general sound quality perception to vocal roughness assessment and to characterize the relationship between vocal roughness and temporal envelop fluctuation measures obtained from an auditory model. Ten /ɑ/ recordings with a wide range of roughness were selected from an existing database. Ten listeners rated the roughness of the recordings in a single-variable matching task. Temporal envelope fluctuations of the recordings were analyzed with an auditory processing model of amplitude modulation that utilizes a modulation filterbank of different modulation frequencies. Pitch strength and the smoothed cepstral peak prominence were also obtained for comparison. Individual simple regression models yielded envelope standard deviation from a modulation filter with a low center frequency (64.3 Hz) as a statistically significant predictor of vocal roughness with a strong coefficient of determination (r 2 = .80). Pitch strength and CPPS were not significant predictors of roughness. This result supports the possible utility of envelope fluctuation measures from an auditory model as objective correlates of vocal roughness.

Predicting speech intelligibility in hearing-impaired listeners using a physiologically inspired auditory model

This study presents a major update and full evaluation of a speech intelligibility (SI) prediction model previously introduced by Scheidiger, Carney, Dau, and Zaar [(2018), Acta Acust. United Ac. 104, 914-917]. The model predicts SI in speech-in-noise conditions via comparison of the noisy speech and the noise-alone reference. The two signals are processed through a physiologically inspired nonlinear model of the auditory periphery, for a range of characteristic frequencies (CFs), followed by a modulation analysis in the range of the fundamental frequency of speech. The decision metric of the model is the mean of a series of short-term, across-CF correlations between population responses to noisy speech and noise alone, with a sensitivity-limitation process imposed. The decision metric is assumed to be inversely related to SI and is converted to a percent-correct score using a single data-based fitting function. The model performance was evaluated in conditions of stationary, fluctuating, and speech-like interferers using sentence-based speech-reception thresholds (SRTs) previously obtained in 5 normal-hearing (NH) and 13 hearing-impaired (HI) listeners. For the NH listener group, the model accurately predicted SRTs across the different acoustic conditions (apart from a slight overestimation of the masking release observed for fluctuating maskers), as well as plausible effects in response to changes in presentation level. For HI listeners, the model was adjusted to account for the individual audiograms using standard assumptions concerning the amount of HI attributed to inner-hair-cell (IHC) and outer-hair-cell (OHC) impairment. HI model results accounted remarkably well for elevated individual SRTs and reduced masking release. Furthermore, plausible predictions of worsened SI were obtained when the relative contribution of IHC impairment to HI was increased. Overall, the present model provides a useful tool to accurately predict speech-in-noise outcomes in NH and HI listeners, and may yield important insights into auditory processes that are crucial for speech understanding.

Substitution voicing index: towards improved speech assessment in patients who have undergone laryngeal oncosurgery

ABSTRACT This study aimed to develop a multidimensional model for the evaluation of substitution voicing (SV) after laryngeal oncosurgery. The study group consisted of 121 adult male individuals: 59 patients with SV after laryngeal oncosurgery (endolaryngeal cordectomy, partial laryngectomy, total laryngectomy with tracheoesophageal prosthesis) and 62 healthy controls. A multidimensional protocol for the assessment of SV included, 1) self-reported speech evaluation with a short version of the Speech Handicap Index, 2) auditory-perceptual assessment, and 3) acoustic speech analysis using AMPEX® (Auditory Model Based Pitch Extractor) software. Moderate correlations were observed between parameters from self-reported auditory-perceptual and acoustic speech analysis domains. The multidimensional Substitution Voicing Index (SVI), including markers from these domains, was elaborated by using linear stepwise regression to determine the optimal set of parameters for categorising SV patients. The lowest mean SVI score was revealed in the control subgroup corresponding to the normal speech, followed by cordectomy subgroup and partial laryngectomy subgroup. The highest mean SVI score was revealed in the total laryngectomy subgroup, reflecting the most severely deteriorated quality of SV. One-way analysis of variance identified statistically significant differences between the mean SVI scores in separate subgroups. The results demonstrated the potential benefits of the SVI for a multidimensional evaluation of SV in patients after laryngeal oncosurgery.

Effective connectivity inference in the whole-brain network by using rDCM method for investigating the distinction between emotional states in fMRI data

ABSTRACT In recent years, the regression dynamic causal modelling (rDCM) method was introduced as a new version of dynamic causal modelling (DCM) to derive effective connectivity in whole-brain networks for functional magnetic resonance imaging (fMRI) data. In this research, we used data obtained while applying the stimulation of audio movie comprised different emotional states. We applied this method to two networks consisting of ten auditory and forty-four regions, respectively. This method was used to study effective connections between emotional states and represent the distinction between emotions. Finally, significant effective connections were found in emotional processing and auditory regions, and between visual and memory-related regions. We also observed the distinctive connections between the pair of emotions in both models. The greatest number of significant distinctions in the coupling between regions was represented in happiness-anger and happiness-fear for the whole-brain model and happiness-sadness, sadness-love, and anger-love for the auditory model.

The Mystical Political: Reclaiming the Spectacle of the Vision in Vicente Aranda’s Libertarias (1996) and Azucena Rodríguez’s Entre rojas (1995)

This article examines the spectacle of the mystical vision in two Spanish films from the mid-1990s, Vicente Aranda’s Libertarias (1996) and Azucena Rodríguez’s Entre rojas (1995). I explore how the films invoke the visual and vocal codes of the mystical vision at key moments in the narrative to positively structure communities of women. Drawing on classic feminist film theory, Michel de Certeau’s interdisciplinary analysis of early modern mysticism, and Jean Luc Nancy’s concept of the inoperable community, this study demonstrates how the mystic’s performance of extrasensory communication, so familiar to Spanish audiences through centuries of religious practice and cultural production, can challenge conventional visual and auditory regimes and model new ways for diverse groups to coexist.