Spectral Cues Research Articles

Happy new ears: rapid adaptation to novel spectral cues in vertical sound localization.

Happy new ears: rapid adaptation to novel spectral cues in vertical sound localization.

Contribution of Spectral and Temporal Cues in Timbre Recognition among Normal-Hearing and Hearing- Impaired People

Background and Aim: Timbre perception is a multidimensional feature of music perception and hearing loss may deteriorate timbre and instrument recognition. Deficits in timbre recognition may originate from poor temporal or spectral coding of musical notes subsequent to hearing loss However, it is not well understood which of these features are more dominant in instrument recognition with normal hearing and hearing loss. This study aimed to evaluate the relative importance of temporal and spectral cues instrument recognition in people with and without nearing loss. Methods: Two groups of adults with normal-hearing and hearing loss were tested with the timbre subtest of the Clinical Assessment of Music Perception (CAMP). Originally, a series of notes is played with eight musical instruments and participants were asked to identify the played instrument among them. In two other conditions, either the rise time or spectral cues of the same notes were masked using a special masking technique. Among eight instruments. Results: Instrument recognition scores were not significantly different between the original and spectral-manipulated conditions but were lower at the temporal-manipulated condition. The difference between the two groups was significant (p<0.05), where the normal-hearing group recognized the musical instruments significantly better than participants with hearing loss group (p<0.05). Conclusion: Temporal cues may have greater importance on timbre recognition regardless of hearing status. This suggests that the interventions and assistive devices which are designed to improve timbre recognition for people with hearing loss should prioritize improvement in temporal coding. Keywords: Timbre recognition; hearing loss; music; musical instrument; temporal; spectral

DNN-based HRTF individualization for accurate spectral cues using a compact PRTF

Head-Related Transfer Function (HRTF) plays a critical role in how the auditory system perceives spatial information. The spectral cues embedded in HRTF are vital for accurately determining the elevation of sound sources. In existing approaches, deep neural networks (DNNs) have been utilized to predict the magnitude spectra of HRTF from images of the pinna, typically employing the HRTF log-magnitude as the output during training. However, HRTF encompasses the acoustic characteristics of both the head and torso, exhibiting direction-dependent patterns that pose challenges in reconstructing its spectral cues. To address this complexity, we propose an innovative method for HRTF individualization. Our model uses Pinna-Related Transfer Function (PRTF) as the output during training, which helps alleviate the impact of sound reflections from the head and torso in the head-related impulse response (HRIR). Our experimental findings, based on an HRTF dataset, illustrate that our proposed model excels in reconstructing the first and second spectral cues. Furthermore, it outperforms previous deep learning models in terms of log spectral distortion (LSD).

Evaluating on the spatial localization cues of binaural sound pressure recorded via behind-the-ear microphones in a hearing aid

Spatial hearing ability enables people with normal hearing to localize a sound source and utilizes the cocktail party effect to detect the target speech information in a noise environment. Hearing-impairment causes not only hearing loss but also the degradation of spatial auditory ability. Hearing aid is desired to improve the spatial auditory ability of some hearing-impaired people, in addition to compensating hearing loss. However, the microphones for conventional behind-the-ear (BTE) hearing aid are located behind the pinnae rather than in the entrances of ear canal, which may cause distortion in spatial cues in the recorded binaural signals. The present work comprehensively evaluates the spatial hearing cues recorded by BTE-microphones. The head-related transfer functions (HRTF) for a KEMAR artificial head and eight human subjects, at three BTE positions and the entrances of blocked ear-canal, are measured respectively. Analysis from the HRTF data indicates that BTE microphone recording basically reserves the interaural time difference and interaural level difference as lateral-direction information. It also reserves the dynamic cues, but obvious distortions appear in the spectral cues at high frequencies. However, due to the redundancy among different spatial information, it can be inferred that dynamic cues alone may provide appropriate front-back and vertical-direction information.

Auditory Processing of Speech and Nonspeech in People Who Stutter.

We investigated speech and nonspeech auditory processing of temporal and spectral cues in people who do and do not stutter. We also asked whether self-reported stuttering severity was predicted by performance on the auditory processing measures. People who stutter (n = 23) and people who do not stutter (n = 28) completed a series of four auditory processing tasks online. These tasks consisted of speech and nonspeech stimuli differing in spectral or temporal cues. We then used independent-samples t-tests to assess differences in phonetic categorization slopes between groups and linear mixed-effects models to test differences in nonspeech auditory processing between stuttering and nonstuttering groups, and stuttering severity as a function of performance on all auditory processing tasks. We found statistically significant differences between people who do and do not stutter in phonetic categorization of a continuum differing in a temporal cue and in discrimination of nonspeech stimuli differing in a spectral cue. A significant proportion of variance in self-reported stuttering severity was predicted by performance on the auditory processing measures. Taken together, these results suggest that people who stutter process both speech and nonspeech auditory information differently than people who do not stutter and may point to subtle differences in auditory processing that could contribute to stuttering. We also note that these patterns could be the consequence of listening to one's own speech, rather than the cause of production differences.

Evaluating Changes in Adult Cochlear Implant Users' Brain and Behavior Following Auditory Training.

To describe the effects of two types of auditory training on both behavioral and physiological measures of auditory function in cochlear implant (CI) users, and to examine whether a relationship exists between the behavioral and objective outcome measures. This study involved two experiments, both of which used a within-subject design. Outcome measures included behavioral and cortical electrophysiological measures of auditory processing. In Experiment I, 8 CI users participated in a music-based auditory training. The training program included both short training sessions completed in the laboratory as well as a set of 12 training sessions that participants completed at home over the course of a month. As part of the training program, study participants listened to a range of different musical stimuli and were asked to discriminate stimuli that differed in pitch or timbre and to identify melodic changes. Performance was assessed before training and at three intervals during and after training was completed. In Experiment II, 20 CI users participated in a more focused auditory training task: the detection of spectral ripple modulation depth. Training consisted of a single 40-minute session that took place in the laboratory under the supervision of the investigators. Behavioral and physiologic measures of spectral ripple modulation depth detection were obtained immediately pre- and post-training. Data from both experiments were analyzed using mixed linear regressions, paired t tests, correlations, and descriptive statistics. In Experiment I, there was a significant improvement in behavioral measures of pitch discrimination after the study participants completed the laboratory and home-based training sessions. There was no significant effect of training on electrophysiologic measures of the auditory N1-P2 onset response and acoustic change complex (ACC). There were no significant relationships between electrophysiologic measures and behavioral outcomes after the month-long training. In Experiment II, there was no significant effect of training on the ACC, although there was a small but significant improvement in behavioral spectral ripple modulation depth thresholds after the short-term training. This study demonstrates that auditory training improves spectral cue perception in CI users, with significant perceptual gains observed despite cortical electrophysiological responses like the ACC not reliably predicting training benefits across short- and long-term interventions. Future research should further explore individual factors that may lead to greater benefit from auditory training, in addition to optimization of training protocols and outcome measures, as well as demonstrate the generalizability of these findings.

Cortical signatures of auditory looming bias show cue-specific adaptation between newborns and young adults

Adaptive biases in favor of approaching, or “looming”, sounds have been found across ages and species, thereby implicating the potential of their evolutionary origin and universal basis. The human auditory system is well-developed at birth, yet spatial hearing abilities further develop with age. To disentangle the speculated inborn, evolutionary component of the auditory looming bias from its learned counterpart, we collected high-density electroencephalographic data across human adults and newborns. As distance-motion cues we manipulated either the sound’s intensity or spectral shape, which is pinna-induced and thus prenatally inaccessible. Through cortical source localisation we demonstrated the emergence of the bias in both age groups at the level of Heschl’s gyrus. Adults exhibited the bias in both attentive and inattentive states; yet differences in amplitude and latency appeared based on attention and cue type. Contrary to the adults, in newborns the bias was elicited only through manipulations of intensity and not spectral cues. We conclude that the looming bias comprises innate components while flexibly incorporating the spatial cues acquired through lifelong exposure.

A Study on the Utilizing of Durational and Spectral Cues in the Perception of English /I-i:/ Contrast by Chinese Learners

A Study on the Utilizing of Durational and Spectral Cues in the Perception of English /I-i:/ Contrast by Chinese Learners

Spectral consistency in sound sequence affects perceptual accuracy in discriminating subdivided rhythmic patterns.

Musical compositions are distinguished by their unique rhythmic patterns, determined by subtle differences in how regular beats are subdivided. Precise perception of these subdivisions is essential for discerning nuances in rhythmic patterns. While musical rhythm typically comprises sound elements with a variety of timbres or spectral cues, the impact of such spectral variations on the perception of rhythmic patterns remains unclear. Here, we show that consistency in spectral cues affects perceptual accuracy in discriminating subdivided rhythmic patterns. We conducted online experiments using rhythmic sound sequences consisting of band-passed noise bursts to measure discrimination accuracy. Participants were asked to discriminate between a swing-like rhythm sequence, characterized by a 2:1 interval ratio, and its more or less exaggerated version. This task was also performed under two additional rhythm conditions: inversed-swing rhythm (1:2 ratio) and regular subdivision (1:1 ratio). The center frequency of the band noises was either held constant or alternated between two values. Our results revealed a significant decrease in discrimination accuracy when the center frequency was alternated, irrespective of the rhythm ratio condition. This suggests that rhythm perception is shaped by temporal structure and affected by spectral properties.

Speech adaptation in conversation: Effects of segment cue-weighting strategy for non-native speakers

Research has investigated how speakers make phonetic adaptations by adjusting their speech sounds to either converge to or diverge from their interlocutor’s. Less is known about the dynamic adaptive strategies non-native speakers use to improve intelligibility when interacting with native speakers, particularly how their strategies change over the course of spontaneous conversations. The current study examines phonetic adaptations in English words contrasting tense and lax vowels (e.g., sheep-ship) in unscripted conversations between non-native Japanese English and native English speakers during an engaging computer game task. Japanese speakers have previously been found to rely on temporal cues (vowel length) for tensity distinctions due to their L1 cue-weighting strategy rather than spectral cues (vowel quality) that are predominantly used in English. Acoustic analyses are conducted to examine changes in non-native vowel productions before, during and after the conversation task. We predict that, in an attempt to overcome miscommunication, non-native speakers may initially lengthen the tense vowels to distinguish them from their lax counterparts. As the conversation progresses, non-native speakers may adapt to a more native-like cue-weighting pattern by shifting to spectral distinctions. Results are discussed in terms of adaptations to cue-weighting strategies for intelligibility gains by interlocutors of different linguistic backgrounds.

Factors influencing the minimum audible change in head orientation of a talker

Humans can detect changes in the head orientation (a.k.a. facing angle) of a talker using only auditory cues. Head orientation cues are beneficial for determining whether one is the intended recipient of an utterance and for segregating a target talker from background talkers. Because human talkers are directional sound sources, binaural cues and monoaural spectral cues can both contribute to talker head orientation perception. In this study, we assessed listeners’ sensitivity to talker head orientation changes using only monaural cues. We tested several factors that could influence the minimum audible change in head orientation: talker and gender (two male, two female), stimulus bandwidth (full-band versus low-pass filtered at 8 or 10 kHz), transducer (loudspeaker versus headphone), stimulus uncertainty (interleaved versus blocked presentation of four talkers), and vocal production mode (speech versus singing). Bandlimiting at 8 or 10 kHz led to worse performance, as did increasing stimulus uncertainty. The effect of transducer was very limited. Performance with speech was better than that with singing. The effect of talker was large and consistent, suggesting individual variability in speech directivity patterns may affect head orientation cues. [Work supported by NIH grant R01DC019745.]

Reduced Channel Interaction Improves Timbre Recognition Under Vocoder Simulation of Cochlear Implant Processing.

This study aimed to investigate the influence of the number of channels and channel interaction on timbre perception in cochlear implant (CI) processing. By utilizing vocoder simulations of CI processing, the effects of different numbers of channels and channel interaction were examined to assess their impact on timbre perception, an essential aspect of music and auditory performance. Fourteen CI recipients, with at least 1 year of CI device use, and two groups (N = 16 and N = 19) of normal hearing (NH) participants completed a timbre recognition (TR) task. NH participants were divided into two groups, with each group being tested on different aspects of the study. The first group underwent testing with varying numbers of channels (8, 12, 16, and 20) to determine an ideal number that closely reflected the TR performance of CI recipients. Subsequently, the second group of NH participants participated in the assessment of channel interaction, utilizing the identified ideal number of 20 channels, with three conditions: low interaction (54 dB/octave), medium interaction (24 dB/octave), and high interaction (12 dB/octave). Statistical analyses, including repeated-measures analysis of variance and pairwise comparisons, were conducted to examine the effects. The number of channels did not demonstrate a statistically significant effect on TR in NH participants ( p > 0.05). However, it was observed that the condition with 20 channels closely resembled the TR performance of CI recipients. In contrast, channel interaction exhibited a significant effect ( p < 0.001) on TR. Both the low interaction (54 dB/octave) and high interaction (12 dB/octave) conditions differed significantly from the actual CI recipients' performance. Timbre perception, a complex ability reliant on highly detailed spectral resolution, was not significantly influenced by the number of channels. However, channel interaction emerged as a significant factor affecting timbre perception. The differences observed under different channel interaction conditions suggest potential mechanisms, including reduced spectro-temporal resolution and degraded spectral cues. These findings highlight the importance of considering channel interaction and optimizing CI processing strategies to enhance music perception and overall auditory performance for CI recipients.

Spatial release from masking in the median plane with non-native speakers using individual and mannequin head related transfer functions.

Spatial release from masking (SRM) in speech-on-speech tasks has been widely studied in the horizontal plane, where interaural cues play a fundamental role. Several studies have also observed SRM for sources located in the median plane, where (monaural) spectral cues are more important. However, a relatively unexplored research question concerns the impact of head-related transfer function (HRTF) personalisation on SRM, for example, whether using individually-measured HRTFs results in better performance if compared with the use of mannequin HRTFs. This study compares SRM in the median plane in a speech-on-speech virtual task rendered using both individual and mannequin HRTFs. SRM is obtained using English sentences with non-native English speakers. Our participants show lower SRM performances compared to those found by others using native English participants. Furthermore, SRM is significantly larger when the source is spatialised using the individual HRTF, and this effect is more marked for those with lower English proficiency. Further analyses using a spectral distortion metric and the estimation of the better-ear effect, show that the observed SRM can only partially be explained by HRTF-specific factors and that the effect of the familiarity with individual spatial cues is likely to be the most significant element driving these results.

Fusion of dichotic consonants in normal-hearing and hearing-impaired listenersa).

Hearing-impaired (HI) listeners have been shown to exhibit increased fusion of dichotic vowels, even with different fundamental frequency (F0), leading to binaural spectral averaging and interference. To determine if similar fusion and averaging occurs for consonants, four natural and synthesized stop consonants (/pa/, /ba/, /ka/, /ga/) at three F0s of 74, 106, and 185 Hz were presented dichotically-with ΔF0 varied-to normal-hearing (NH) and HI listeners. Listeners identified the one or two consonants perceived, and response options included /ta/ and /da/ as fused percepts. As ΔF0 increased, both groups showed decreases in fusion and increases in percent correct identification of both consonants, with HI listeners displaying similar fusion but poorer identification. Both groups exhibited spectral averaging (psychoacoustic fusion) of place of articulation but phonetic feature fusion for differences in voicing. With synthetic consonants, NH subjects showed increased fusion and decreased identification. Most HI listeners were unable to discriminate the synthetic consonants. The findings suggest smaller differences between groups in consonant fusion than vowel fusion, possibly due to the presence of more cues for segregation in natural speech or reduced reliance on spectral cues for consonant perception. The inability of HI listeners to discriminate synthetic consonants suggests a reliance on cues other than formant transitions for consonant discrimination.

The Role of Temporal and Spectral Cues in Non-native Speech Production:

This study investigated the role of durational and spectral cues in second language tense and lax vowel contrasts produced by non-native speakers. To test previous claims that speakers primarily rely on durational cues over spectral cues to distinguish L2 tense and lax vowel pairs, citation style speech data were collected from 16 native speakers of Bangla; participants were all undergraduate students. The data were collected via a shadowing task where participants listened to a carefully constructed list of English words in random order and repeated each word immediately after they heard them. The utterances were recorded via a Zoom H1n voice recorder. Collected speech data were annotated and processed using the phonetic analysis software Praat and the semi-automatic annotation toolkit DARLA; statistical analyses were performed using R statistical computing software. Results indicate that Bangla speakers do not emphasize on durational cues to differentiate English tense-lax vowel pairs, contrary to the general patterns reported from other languages; rather, they prefer the spectral cues over the durational cues.

Analysis and Experiment on the Limitations of Static and Dynamic Transaural Reproduction with Two Frontal Loudspeakers

By duplicating the binaural pressures of an actual source, transaural reproduction with two frontal loudspeakers is expected to recreate a virtual source in arbitrary direction. However, experiments indicated that in static transaural reproduction, the perceived virtual source is usually limited to the frontalhorizontal plane. The reasons for this limitation, as guessed, are that, in static reproduction, the dynamic cues for front-back and vertical localisation are incorrect, and the high-frequency spectral cues are unstable with head movement. To validate this hypothesis, the variations of ITD (interaural time difference) caused by head turning in both static and dynamic transaural reproductions are analysed. The results indicate that dynamic reproduction is able to create appropriate low-frequency ITD variations, and the static transaural reproduction is unable to do so. Psychoacoustic experiments are conducted to compare virtual source localisation in static and dynamic reproductions. The results indicate that dynamic reproduction is able to recreate the front, back, and vertical virtual source for low-pass stimuli below 3 kHz, while for full audible bandwidth stimulus, appropriate low-frequency dynamic cue and unstable high-frequency spectral cues in dynamic reproduction result in two splitting virtual sources.

Australian English listeners' perception of Japanese vowel length reveals underlying phonological knowledge.

Speech perception patterns are strongly influenced by one's native phonology. It is generally accepted that native English listeners rely primarily on spectral cues when perceiving vowels, making limited use of duration cues because English lacks phonemic vowel length. However, the literature on vowel perception by English listeners shows a marked bias toward American English, with the phonological diversity among different varieties of English largely overlooked. The current study investigates the perception of Japanese vowel length contrasts by native listeners of Australian English, which is reported to use length to distinguish vowels unlike most other varieties of English. Twenty monolingual Australian English listeners participated in a forced-choice experiment, where they categorized Japanese long and short vowels as most similar to their native vowel categories. The results showed a general tendency for Japanese long and short vowels (e.g., /ii, i/) to be categorized as Australian English long and short vowels (e.g., /i:, ɪ/ as in "heed," "hid"), respectively, which contrasts with American English listeners' categorization of all Japanese vowels as tense regardless of length (e.g., /ii, i/ as both "heed") as reported previously. Moreover, this duration-based categorization was found not only for Australian English categories that contrast in duration alone (e.g., /ɐ:, ɐ/ as in "hard," "hud") but also for those that contrast in both duration and spectra (e.g., /o:, ɔ/ as in "hoard," "hod"), despite their spectral mismatch from the corresponding Japanese vowels (e.g., /aa, a/ and /oo, o/). The results, therefore, suggest that duration cues play a prominent role across all vowel categories-even nonnative-for Australian English listeners. The finding supports a feature-based framework of speech perception, where phonological features like length are shared across multiple categories, rather than the segment-based framework that is currently dominant, which regards acoustic cues like duration as being tied to a specific native segmental category. Implications for second and foreign language learning are discussed.

Allocation of cognitive resources for older adults with normal or impaired hearing when listening to temporally degraded and masked speech

Hearing impairment may affect how older listeners allocate cognitive resources when communicating in challenging environments. Older adults with normal or impaired hearing completed three speech recognition studies. The studies consisted of 15-16 measures of temporally filtered speech with (1) degraded spectral cues, (2) competing speech-modulated noise, and (3) speech vocoding in speech-modulated noise. Generalization was assessed with measures of speech interruption, segregation, and multitalker masking. To account for audibility differences, all listeners received spectral shaping fit to their hearing thresholds. Measures in each study were reduced to a single principal component (PC) that represented speech recognition abilities. Participants also completed a cognitive test battery that yielded domain-specific PCs and a measure of speech glimpsing. For each study, hierarchical linear regression analysis was completed separately for subgroups of older adults with normal or impaired hearing using cognitive PCs as predictor variables, followed by speech glimpsing, age, and hearing thresholds. Vocabulary knowledge was the best predictor of speech recognition for older adults with normal hearing, whereas working memory consistently explained ∼30%-40% of the variance in recognition for older adults with hearing loss. Thus, even with sufficient audibility, hearing impairment may alter allocation of cognitive resources for older adults listening to degraded speech.

Spectral and temporal cues used by echolocating bottlenose dolphins to discriminate changes in inter-highlight intervals

Echolocating dolphins use spectral cues to discriminate complex echoes from targets with multiple reflective surfaces; however, the specific cues involved are not clear. This study investigated the role of spectral interference patterns on the dolphin’s ability to discriminate two-highlight echoes. In task one of the study, two dolphins were trained to echolocate, listen to returning electronic (“phantom”) echoes with two identical highlights, and produce a conditioned acoustic response if the inter-highlight interval (IHI) increased. The amount that the IHI increased varied across trials. Task two replicated task one but applied a random phase shift to all highlights. This varied the specific locations of notches along the frequency axis in the complex echo spectrum but preserved the notch spacing (equal to the inverse of the IHI). For IHIs less than 250 μs, discrimination thresholds increased with IHI and were larger with random phase shifts. For IHIs greater than 250 μs, thresholds plateaued and were similar for both tasks. Results agree with those from a previous study with passively listening dolphins. Namely, echolocating dolphins can use the spacing of notches in spectral interference patterns to detect IHI changes within the ∼250-μs temporal window, but temporal cues are used at longer IHIs. [Work supported by ONR]

Auditory vertical localization in the median plane with conflicting dynamic interaural time difference and other elevation cues.

Both dynamic variation of interaural time difference (ITD) and static spectral cues provide information for front-back discrimination and vertical localization. However, the contributions of the two cues are still unclear. The static spectral cue has conventionally been regarded as the dominant one. In the present work, psychoacoustic experiments were conducted to examine the contribution of dynamic ITD and static spectral cues to vertical localization in the median plane. By modifying the head-related transfer functions used in a dynamic virtual auditory display, binaural signals with conflicting dynamic ITD and spectral cues that were either static or dynamically modified according to instantaneous head position were created. The results indicated that the dynamic ITD and static spectral cues contribute to vertical localization at low and high frequencies, respectively. For full a bandwidth stimulus, conflicting dynamic ITD and static spectral cues usually result in two separated virtual sources at different elevations corresponding to the spatial information conveyed by the low- and high-frequency bands, respectively. In most cases, no fused localization occurs in the high-level cognition system. Therefore, dynamic ITD and static spectral cues contribute to vertical localization at different frequency ranges, and neither of them dominates vertical localization in the case of wideband stimuli.