Model Of Auditory Perception Research Articles

Infiltration of Traditional Music Elements in Piano Teaching Models Combining Wavelet Packet Domain Auditory Perception

Abstract In Chinese piano teaching practice, the importance of auditory perception of music has been neglected, so that in piano performance and teaching, many people play music that lacks personality and emotional experience. This paper proposes an auditory perception model based on wavelet packet domain based on the acoustic properties of the human ear. The model analyzes the acoustical perception characteristics of piano audio signals through wavelet transform, and introduces adaptive wavelet packet decomposition into the calculation of the auditory perception model to simplify the calculation of the model, so that the obtained auditory masking threshold is closer to the actual value. Finally, the acoustic perception model is practically applied to piano teaching to verify its effectiveness. And the traditional Chinese music elements of Gong, Zheng and Shang were infiltrated in piano teaching. The results showed that the model detected the 3rd harmonic g1 with root tone C, whose frequency peak was most prominent at 397.25 Hz, and the sound pressure value reached 39.50 dB, indicating that the auditory perception ability directly affects the musical expression and creativity. This study cultivates students’ musical auditory perception ability, which significantly improves piano playing ability.

Simulated transient hearing loss improves auditory sensitivity

Recently, it was proposed that a processing principle called adaptive stochastic resonance plays a major role in the auditory system, and serves to maintain optimal sensitivity even to highly variable sound pressure levels. As a side effect, in case of reduced auditory input, such as permanent hearing loss or frequency specific deprivation, this mechanism may eventually lead to the perception of phantom sounds like tinnitus or the Zwicker tone illusion. Using computational modeling, the biological plausibility of this processing principle was already demonstrated. Here, we provide experimental results that further support the stochastic resonance model of auditory perception. In particular, Mongolian gerbils were exposed to moderate intensity, non-damaging long-term notched noise, which mimics hearing loss for frequencies within the notch. Remarkably, the animals developed significantly increased sensitivity, i.e. improved hearing thresholds, for the frequency centered within the notch, but not for frequencies outside the notch. In addition, most animals treated with the new paradigm showed identical behavioral signs of phantom sound perception (tinnitus) as animals with acoustic trauma induced tinnitus. In contrast, animals treated with broadband noise as a control condition did not show any significant threshold change, nor behavioral signs of phantom sound perception.

Multi-resolution modulation-filtered cochleagram feature for LSTM-based dimensional emotion recognition from speech

Continuous dimensional emotion recognition from speech helps robots or virtual agents capture the temporal dynamics of a speaker’s emotional state in natural human–robot interactions. Temporal modulation cues obtained directly from the time-domain model of auditory perception can better reflect temporal dynamics than the acoustic features usually processed in the frequency domain. Feature extraction, which can reflect temporal dynamics of emotion from temporal modulation cues, is challenging because of the complexity and diversity of the auditory perception model. A recent neuroscientific study suggests that human brains derive multi-resolution representations through temporal modulation analysis. This study investigates multi-resolution representations of an auditory perception model and proposes a novel feature called multi-resolution modulation-filtered cochleagram (MMCG) for predicting valence and arousal values of emotional primitives. The MMCG is constructed by combining four modulation-filtered cochleagrams at different resolutions to capture various temporal and contextual modulation information. In addition, to model the multi-temporal dependencies of the MMCG, we designed a parallel long short-term memory (LSTM) architecture. The results of extensive experiments on the RECOLA and SEWA datasets demonstrate that MMCG provides the best recognition performance in both datasets among all evaluated features. The results also show that the parallel LSTM can build multi-temporal dependencies from the MMCG features, and the performance on valence and arousal prediction is better than that of a plain LSTM method.

The stochastic resonance model of auditory perception: A unified explanation of tinnitus development, Zwicker tone illusion, and residual inhibition.

Stochastic resonance (SR) has been proposed to play a major role in auditory perception, and to maintain optimal information transmission from the cochlea to the auditory system. By this, the auditory system could adapt to changes of the auditory input at second or even sub-second timescales. In case of reduced auditory input, somatosensory projections to the dorsal cochlear nucleus would be disinhibited in order to improve hearing thresholds by means of SR. As a side effect, the increased somatosensory input corresponding to the observed tinnitus-associated neuronal hyperactivity is then perceived as tinnitus. In addition, the model can also explain transient phantom tone perceptions occurring after ear plugging, or the Zwicker tone illusion. Vice versa, the model predicts that via stimulation with acoustic noise, SR would not be needed to optimize information transmission, and hence somatosensory noise would be tuned down, resulting in a transient vanishing of tinnitus, an effect referred to as residual inhibition.

Brain-optimized extraction of complex sound features that drive continuous auditory perception.

Understanding how the human brain processes auditory input remains a challenge. Traditionally, a distinction between lower- and higher-level sound features is made, but their definition depends on a specific theoretical framework and might not match the neural representation of sound. Here, we postulate that constructing a data-driven neural model of auditory perception, with a minimum of theoretical assumptions about the relevant sound features, could provide an alternative approach and possibly a better match to the neural responses. We collected electrocorticography recordings from six patients who watched a long-duration feature film. The raw movie soundtrack was used to train an artificial neural network model to predict the associated neural responses. The model achieved high prediction accuracy and generalized well to a second dataset, where new participants watched a different film. The extracted bottom-up features captured acoustic properties that were specific to the type of sound and were associated with various response latency profiles and distinct cortical distributions. Specifically, several features encoded speech-related acoustic properties with some features exhibiting shorter latency profiles (associated with responses in posterior perisylvian cortex) and others exhibiting longer latency profiles (associated with responses in anterior perisylvian cortex). Our results support and extend the current view on speech perception by demonstrating the presence of temporal hierarchies in the perisylvian cortex and involvement of cortical sites outside of this region during audiovisual speech perception.

Penetration state identification of lap joints in gas tungsten arc welding process based on two channel arc sounds

Abstract Lap joints commonly exist in the welding processes during manufacturing of automobiles and ships. Their asymmetric structures make it extraordinary difficult to monitor the penetration states online. In this paper, a two-channel arc sound signal acquisition device is developed and a binaural auditory perception model is proposed to identify the penetration states of lap joints. Firstly, an auditory peripheral model is built to filter the two-channel arc sound signals and decompose them into a series of frequency bands. After that, with simulating the energy transformation and lateral inhabitation of auditory system, the frequency distributions of the two-channel arc sound signals are obtained. Subsequently, the energy differences of the two-channel sounds are obtained and decomposed into multi-scales by an auditory cortex model. Finally, a support vector machine is built to recognize the lap bead penetration states. This study provides a new feasible approach to identify penetration states of lap weld joints.

Weld bead penetration state recognition in GMAW process based on a central auditory perception model

Arc sound signals contain abundant information about the welding process, so they are widely adopted to monitor the penetration states of weld bead online. However, the arc sounds are vulnerable to back ground noises, which limits the practical application of them in industry. In this study, a central auditory perception model was proposed. This model through simulating human auditory system to identify the penetration states of weld bead in GMAW process. Firstly, the auditory spectrums of the arc sounds are obtained through auditory peripheral model. Then the shape of auditory spectrums is analyzed by an auditory cortex model. Finally, according to the auditory cortical response, character vectors are built and used to identify the penetration states. The results show that the proposed method has a high accuracy rate in the identification of penetration states of weld bead. This research provides a new method for the online monitoring of welding quality.

Application of a pitch perception model to investigate the effect of stimulation field spread on the pitch ranking abilities of cochlear implant recipients

Although many cochlear implant (CI) recipients perceive speech very well in favorable conditions, they still have difficulty with music, speech in noisy environments, and tonal languages. Studies show that CI users’ performance in these tasks are correlated with their ability to perceive pitch. The spread of stimulation field from the electrodes to the auditory nerve is one of the factors affecting performance. This study proposes a model of auditory perception to predict the performance of CI users in pitch ranking tasks using an existing sound processing scheme. The model is then used as a platform to investigate the effect of stimulation field spread on performance.

A New Technique for Functional Imaging in Songbirds and Beyond

The quantitative study of cerebral hemodynamics requires measurement methods with high spatial and temporal resolution. In the best case, these methods are noninvasive or even contact free. Mottin et al (2010, this issue) propose picosecond optical tomography, or POT, to quantitatively image hemodynamics in a contact-free way. POT relies on white laser light spectral absorption in the near-infrared measured with a streak camera. The tens-of-femtosecond temporal resolution of the streak camera enables depth-specific imaging of oxy- and deoxy-hemoglobin concentration with submillimeter resolution. The authors validate their technique on a songbird model of auditory perception and show results consistent with previous findings that were obtained using functional magnetic resonance imaging (MRI). Furthermore, they shed new light onto brain oxygen recoupling in these tachymetabolic animals, results that would be difficult to obtain with conventional functional MRI. The songbird model is an interesting choice for demonstrating the validity of POT, as songbirds provide a versatile model for studying neuronal plasticity (Brenowitz, 2004). This includes the ability to generate a significant amount of new neurons in adult life (Nottebohm, 2002), as well as the rare trait of vocal learning (Mooney, 2009). Songbirds produce behavioral patterns of high complexity (song) that can be precisely recorded and related to neuronal dynamics from time scales of milliseconds to a time spanning several generations (Hahnloser et al, 2002; Feher et al, 2009). It might be possible that POT adds information to more classical methods of birdsong research based on electrical recordings or functional MRI by virtue of its quantitative nature. For example, could it be used to get a better picture of how the song system works as a network of connected song nuclei by quantitatively analyzing the hemodynamic response dynamics? Besides further applications in songbird research, these advantages could also play out in functional studies of neurovascular plasticity, as has been observed for example after focal cerebral injury (Whitaker et al, 2007), or in neurovascular repair mechanisms after induction of small-scale cortical vascular lesions (Nishimura et al, 2006) that are believed to have a function in aging-related cognitive decline. As they point out at the end of the article, this is a field worth studying for Mottin et al too, by using their technology to investigate a proposed link between brain aging and changes in vascular reactivity found by functional MRI. However, it remains to be seen if POT can compete with the very powerful imaging techniques already used in this field, such as multiphoton microscopy and voltage sensitive dye imaging (Brown et al, 2009). Finally, the authors propose a way to expand POT into a true two-dimensional imaging method (so far it can only image line segments) by sweeping the optical field over the sample. In the long term, functional imaging methods like this could, once developed for human applications, become more relevant clinically (Prakash et al, 2009). As functional MRI maps of the human cortex display a considerable amount of interindividual variability, it is necessary to map, for example, eloquent sensorimotor areas before or during brain tumor surgery. Often, a combination of electrocortical stimulation mapping on the exposed brain and functional MRI is used to identify eloquent sensorimotor areas and thus to reduce surgical risks (Holodny, 2008). It can be envisioned that optical imaging methods based on POT could be used in the operating room to rapidly map individual eloquent cortical areas during surgery.

Psychoacoustic measures of stop production in Cantonese, Greek, English, Japanese, and Korean.

Spectral analyses of stop bursts have revealed that the place of articulation can be predicted based on both invariant and time‐varying cues present within the acoustic signal (e.g., Stevens and Blumstein, 1978; Forrest et al., 1988). While prior studies have had some success in uncovering such cues for American English, it is not clear whether these parameters are equally pertinent in stoping consonant classification in other languages. Furthermore, one of the main limitations of a linear acoustic analysis is that it imposes different scales of loudness and frequency on the acoustic signal than does the human ear, thus generating power spectra with different frequency distributions than are produced by the auditory system (e.g., Zwicker 1961; Kewley‐Port, 1983). In the present study of word‐initial stops produced in Cantonese, English, Greek, Japanese, and Korean, measures derived from a psychoacoustic model of auditory perception were developed in order to more accurately isolate the front cavity resonances of the burst transient. For all languages, peak amplitude frequency and a peak compactness measure successfully distinguished alveolar from velar stops with at least 74% accuracy and were more accurate at stop classification than were spectral moments measures.

Pitch strength decreases as F0 and harmonic resolution increase in complex tones composed exclusively of high harmonics

A melodic pitch experiment was performed to demonstrate the importance of time-interval resolution for pitch strength. The experiments show that notes with a low fundamental (75 Hz) and relatively few resolved harmonics support better performance than comparable notes with a higher fundamental (300 Hz) and more resolved harmonics. Two four note melodies were presented to listeners and one note in the second melody was changed by one or two semitones. Listeners were required to identify the note that changed. There were three orthogonal stimulus dimensions: F0 (75 and 300 Hz); lowest frequency component (3, 7, 11, or 15); and number of harmonics (4 or 8). Performance decreased as the frequency of the lowest component increased for both F0's, but performance was better for the lower F0. The spectral and temporal information in the stimuli were compared using a time-domain model of auditory perception. It is argued that the distribution of time intervals in the auditory nerve can explain the decrease in performance as F0, and spectral resolution increase. Excitation patterns based on the same time-interval information do not contain sufficient resolution to explain listener's performance on the melody task.

Classification of impulsive-source active-sonar echoes based on a model of auditory perception

Models of auditory perception are investigated as the basis for a biomimetic classifier of impulsive-source active-sonar echoes. Multidimensional scaling estimates the perceptual space in which listeners perform classification [J. E. Summers et al., J. Acoust. Soc. Am. 120, 3125 (A) (2006)]. In the resulting space of perceptual dimensions, stimuli form distinct clusters and target is discriminated from clutter along a single perceptual dimension. Dimensions in this space do not correspond to features having simple algorithmic representations. Consequently, conventional methods to develop a mapping from signal space to feature space fail. Instead, dimensions reflect untrained categorical perception manifested through the mixtures of top-down and bottom-up processes used by listeners: A high-level cognitive process for interclass dissimilarity ratings and a low-level signal-based process for intraclass comparisons. Behaving as expert systems, listeners rapidly assign stimuli to categories based on prior experiences, a process analogous to the statistical description of classes in the class-specific method. In contrast, within-class comparisons reflect signal-derived features found to be most efficacious for differentiating between the signals, a process similar to generation of features by singular-value decomposition. Implications of these findings for design of hybrid generative/discriminative human-mimetic classifier architectures are discussed. [Work supported by ONR.]

Sequential processing of interaural timing differences for sound source segregation and spatial localization: Evidence from event‐related cortical potentials

Cortical processing of interaural timing differences (ITDs) was investigated with event-related potential (ERP) measurements in 16 human participants who were required in separate tasks to detect or to spatially localize dichotically embedded pitches. ITDs elicited three ERP components labeled ORN, N2, and P400. The ORN occurred at a latency of 150-250 ms and was elicited by ITDs regardless of location or task. In contrast, the N2 response (250-350 ms) was strongly modulated by location and showed larger amplitudes for the localization task than for the detection task. Finally, ITDs in the detection task elicited a P400 at a latency of 400-500 ms, but this response was entirely absent from ERPs elicited by identical stimuli in the localization task. These results are consistent with a sequential model of auditory perception in which segregation of concurrent sounds is followed by domain-specific processing of object location and identity.

PEMO-Q—A New Method for Objective Audio Quality Assessment Using a Model of Auditory Perception

A new method for the objective assessment and prediction of perceived audio quality is introduced. It represents an expansion of the speech quality measure q <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</sub> , introduced by Hansen and Kollmeier, and is based on a psychoacoustically validated, quantitative model of the "effective" peripheral auditory processing by Dau et al. To evaluate the audio quality of a given distorted signal relative to a corresponding high-quality reference signal, the auditory model is employed to compute "internal representations" of the signals, which are partly assimilated in order to account for assumed cognitive aspects. The linear cross correlation coefficient of the assimilated internal representations represents the perceptual similarity measure (PSM). PSM shows good correlations with subjective quality ratings if different types of audio signals are considered separately, whereas a better accuracy of signal-independent quality prediction is achieved by a second quality measure PSM <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</sub> represented by the fifth percentile of the sequence of instantaneous audio quality PSM(t). The new measures were evaluated using a large database of subjective listening tests that were originally carried out on behalf of the International Telecommunication Union (ITU) and Moving Pictures Experts Group (MPEG) for the evaluation of various low bit-rate audio codecs. Additional tests with data unknown in the development phase of the model were carried out. Except for linear distortions, the new method shows a higher prediction accuracy than the ITU-R recommendation BS.1387 ("PEAQ") for the tested data

Sub-band SNR estimation using auditory feature processing

In this paper a new approach is presented for estimating the long-term speech-to-noise ratio (SNR) in individual frequency bands that is based on methods known from automatic speech recognition (ASR). It uses a model of auditory perception as front end, physiologically and psychoacoustically motivated sigma–pi cells as secondary features, and a linear or non-linear neural network as classifier. A non-linear neural network back end is capable of estimating the SNR in time segments of 1 s with a root-mean-square error of 5.68 dB on unknown test material. This performance is obtained on a large set of natural types of noise, containing instationary signals and alarm sounds. However, the SNR estimation works best for more stationary types of noise. The individual components of the estimation algorithms are examined with respect to their importance for the estimation accuracy. The algorithm presented in this paper yields similar or better results with comparable computational effort relative to other methods known from the literature for short-term SNR estimation. The new approach is purely based on slow spectro-temporal modulations and is therefore a valuable contribution to both, digital hearing-aids and ASR systems.

Combining speech enhancement and auditory feature extraction for robust speech recognition

A major deficiency in state-of-the-art automatic speech recognition (ASR) systems is the lack of robustness in additive and convolutional noise. The model of auditory perception (PEMO), developed by Dau et al. (T. Dau, D. Püschel, A. Kohlrausch, J. Acoust. Soc. Am. 99 (6) (1996) 3615–3622) for psychoacoustical purposes, partly overcomes these difficulties when used as a front end for automatic speech recognition. To further improve the performance of this auditory-based recognition system in background noise, different speech enhancement methods were examined, which have been evaluated in earlier studies as components of digital hearing aids. Monaural noise reduction, as proposed by Ephraim and Malah (Y. Ephraim, D. Malah, IEEE Trans. Acoust. Speech Signal Process. ASSP-32 (6) (1984) 1109–1121) was compared to a binaural filter and dereverberation algorithm after Wittkop et al. (T. Wittkop, S. Albani, V. Hohmann, J. Peissig, W. Woods, B. Kollmeier, Acustica United with Acta Acustica 83 (4) (1997) 684–699). Both noise reduction algorithms yield improvements in recognition performance equivalent to up to 10 dB SNR in non-reverberant conditions for all types of noise, while the performance in clean speech is not significantly affected. Even in real-world reverberant conditions the speech enhancement schemes lead to improvements in recognition performance comparable to an SNR gain of up to 5 dB. This effect exceeds the expectations as earlier studies found no increase in speech intelligibility for hearing-impaired human subjects.

A model of auditory perception as front end for automatic speech recognition.

A front end for automatic speech recognizers is proposed and evaluated which is based on a quantitative model of the "effective" peripheral auditory processing. The model simulates both spectral and temporal properties of sound processing in the auditory system which were found in psychoacoustical and physiological experiments. The robustness of the auditory-based representation of speech was evaluated in speaker-independent, isolated word recognition experiments in different types of additive noise. The results show a higher robustness of the auditory front end in noise, compared to common mel-scale cepstral feature extraction. In a second set of experiments, different processing stages of the auditory front end were modified to study their contribution to robust speech signal representation in detail. The adaptive compression stage which enhances temporal changes of the input signal appeared to be the most important processing stage towards robust speech representation in noise. Low-pass filtering of the fast fluctuating envelope in each frequency band further reduces the influence of noise in the auditory-based representation of speech.

Evaluation of monaural and binaural speech enhancement for robust auditory-based automatic speech recognition

A major deficiency in state-of-the-art automatic speech recognition systems is the lack of robustness in additive and convolutive noise. The model of auditory perception, as developed by Dau et al. [J. Acoust. Soc. Am. 99, 3615–3622 (1996)] for psychoacoustical purposes, partly overcomes these difficulties when used as a front-end for speech recognition. Especially in combination with locally-recurrent neural networks (LRNN) the model output, called ‘‘internal representation’’ had been shown to provide highly robust feature vectors [Tchorz and Kollmeier, J. Acoust. Soc. Am. (submitted)]. To further improve the performance of this auditory-based LRNN recognition system in background noise, different speech enhancement methods were examined. The minimum mean-square error (MMSE) short-term spectral amplitude estimator (STSA), as proposed by Ephraim and Malah [IEEE Trans. Acoust., Speech, Signal Process. 32, 1109–1121 (1984)], was compared to a binaural Wiener filter [Wittkop et al., this meeting], based on directional and coherence cues. Both noise reduction algorithms yield highly improved recognition rates in nonreverberant noisy conditions, while the performance in clean speech is not significantly affected. The algorithms were also evaluated in real-world reverberant conditions with speech-simulating noise and jammer speech.

Detecting time and amplitude jitter in a click train

A series of 2IFC experiments was performed with click trains to determine (a) the temporal jitter and (b) the amplitude jitter necessary to discriminate a regular click train from a jittered click train. The period of the regular click train was varied from 10 ms, where the sound is heard as a 100-Hz buzzy tone, up to 120 ms, where it is heard as a stream of discreet clicks. A fixed percentage jitter was applied to the period (or the amplitude) of each click in the train with positive jitter, negative jitter, and no jitter being given equal probability. The results show that the auditory system is much more sensitive to time jitter (around 12%) than to amplitude jitter (around 50%), and there is a ‘‘hump’’ in the Weber fraction around periods of 40–50 ms between the ‘‘buzzy tone’’ and ‘‘distinct clicks’’ regions. The results are interpreted in terms of the auditory image model of auditory perception [Pattersonet al., J. Acoust. Soc. Am. 98, 1890–1894 (1995)] which preserves phase-locking information in a buffer which is limited to time intervals less than about 40 ms.

On the compression obtainable with four-tap wavelets

The performance obtainable with four-tap wavelet filters for low bit rate audio coding is presented. For the investigation and comparison of the performance of these wavelet filters, a codec model has been designed and implemented based on wavelet packet algorithm and the model of auditory perception.