Sound Segregation Research Articles

Study of speech intelligibility in anechoic and reverberant environments using Slovak language

Binaural aspects play a significant role in speech intelligibility, with listeners demonstrating better performance in scenarios where the signal and noise sources are spatially separated, leveraging interaural time and level differences for improved sound segregation and localization. Thus, the question is: How do binaural aspects influence speech intelligibility in different spatial configurations of signal and noise? In this paper, speech intelligibility is examined in an anechoic and reverberant environment, specifically focusing on three configurations of signal and noise: (1) signal and noise both coming from the front at a distance of 1-meter, (2) signal from the front and noise from the right side (90 degrees) at a 1-meter distance, and (3) signal from the front and noise from the back (180 degrees) at a 1-meter distance. The study employs two different noises-speech-weighted noise and multi-talker noise-during listening tests conducted in the Slovak language. Furthermore, the study compares the outcomes with a similar experiment conducted a few years ago in the Dutch language, providing valuable insights into language-specific variations in speech perception under different acoustic environments.

Deficits in congenital amusia: Pitch, music, speech, and beyond

Congenital amusia is a neurodevelopmental disorder characterized by deficits of music perception and production, which are related to altered pitch processing. The present study used a wide variety of tasks to test potential patterns of processing impairment in individuals with congenital amusia (N = 18) in comparison to matched controls (N = 19), notably classical pitch processing tests (i.e., pitch change detection, pitch direction of change identification, and pitch short-term memory tasks) together with tasks assessing other aspects of pitch-related auditory cognition, such as emotion recognition in speech, sound segregation in tone sequences, and speech-in-noise perception. Additional behavioral measures were also collected, including text reading/copying tests, visual control tasks, and a subjective assessment of hearing abilities. As expected, amusics' performance was impaired for the three pitch-specific tasks compared to controls. This deficit of pitch perception had a self-perceived impact on amusics’ quality of hearing. Moreover, participants with amusia were impaired in emotion recognition in vowels compared to controls, but no group difference was observed for emotion recognition in sentences, replicating previous data. Despite pitch processing deficits, participants with amusia did not differ from controls in sound segregation and speech-in-noise perception. Text reading and visual control tests did not reveal any impairments in participants with amusia compared to controls. However, the copying test revealed more numerous eye-movements and a smaller memory span. These results allow us to refine the pattern of pitch processing and memory deficits in congenital amusia, thus contributing further to understand pitch-related auditory cognition. Together with previous reports suggesting a comorbidity between congenital amusia and dyslexia, the findings call for further investigation of language-related abilities in this disorder even in the absence of neurodevelopmental language disorder diagnosis.

Machine learning for efficient segregation and labeling of potential biological sounds in long-term underwater recordings

Studying marine soundscapes by detecting known sound events and quantifying their spatio-temporal patterns can provide ecologically relevant information. However, the exploration of underwater sound data to find and identify possible sound events of interest can be highly time-intensive for human analysts. To speed up this process, we propose a novel methodology that first detects all the potentially relevant acoustic events and then clusters them in an unsupervised way prior to manual revision. We demonstrate its applicability on a short deployment. To detect acoustic events, a deep learning object detection algorithm from computer vision (YOLOv8) is re-trained to detect any (short) acoustic event. This is done by converting the audio to spectrograms using sliding windows longer than the expected sound events of interest. The model detects any event present on that window and provides their time and frequency limits. With this approach, multiple events happening simultaneously can be detected. To further explore the possibilities to limit the human input needed to create the annotations to train the model, we propose an active learning approach to select the most informative audio files in an iterative manner for subsequent manual annotation. The obtained detection models are trained and tested on a dataset from the Belgian Part of the North Sea, and then further evaluated for robustness on a freshwater dataset from major European rivers. The proposed active learning approach outperforms the random selection of files, both in the marine and the freshwater datasets. Once the events are detected, they are converted to an embedded feature space using the BioLingual model, which is trained to classify different (biological) sounds. The obtained representations are then clustered in an unsupervised way, obtaining different sound classes. These classes are then manually revised. This method can be applied to unseen data as a tool to help bioacousticians identify recurrent sounds and save time when studying their spatio-temporal patterns. This reduces the time researchers need to go through long acoustic recordings and allows to conduct a more targeted analysis. It also provides a framework to monitor soundscapes regardless of whether the sound sources are known or not.

Effects of sound segregation cues on multi-sound intensity discrimination.

The effects of sound segregation cues on the sensitivity to intensity increments were explored. Listeners indicated whether the second and fourth sounds (harmonic complexes) within a five-sound sequence were increased in intensity. The target sound had a fundamental frequency of 250 Hz. In different conditions, nontarget sounds had different fundamental frequencies, different spectral shapes, and unique frequency regions relative to the target. For targets more intense than nontargets, nontarget characteristics did not affect thresholds. For targets less intense than the nontargets, thresholds improved when the targets and nontargets had unique frequency regions.

The Effect of Combined Auditory Training on Concurrent Sound Segregation in Young old: A Single-Blinded Randomized Clinical Trial.

The online version contains supplementary material available at 10.1007/s12070-023-03923-x.

A biologically oriented algorithm for spatial sound segregation.

Listening in an acoustically cluttered scene remains a difficult task for both machines and hearing-impaired listeners. Normal-hearing listeners accomplish this task with relative ease by segregating the scene into its constituent sound sources, then selecting and attending to a target source. An assistive listening device that mimics the biological mechanisms underlying this behavior may provide an effective solution for those with difficulty listening in acoustically cluttered environments (e.g., a cocktail party). Here, we present a binaural sound segregation algorithm based on a hierarchical network model of the auditory system. In the algorithm, binaural sound inputs first drive populations of neurons tuned to specific spatial locations and frequencies. The spiking response of neurons in the output layer are then reconstructed into audible waveforms via a novel reconstruction method. We evaluate the performance of the algorithm with a speech-on-speech intelligibility task in normal-hearing listeners. This two-microphone-input algorithm is shown to provide listeners with perceptual benefit similar to that of a 16-microphone acoustic beamformer. These results demonstrate the promise of this biologically inspired algorithm for enhancing selective listening in challenging multi-talker scenes.

Level differences impact the fusion of concurrent vowels dissimilarly within versus across ears.

This study investigated how level differences affect the fusion and identification of dichotically and monaurally presented concurrent vowel pairs where the vowels differed in level by 0, 4, 8, or 12 dB. With dichotic presentation, there was minimal variation in fusion and identification-vowels were nearly always fused and were identified consistently across level differences. Conversely, with monaural presentation, fusion and identification varied systematically across level differences-with the more intense vowel dominating fused percepts. The dissimilar effect of level difference for dichotic versus monaural presentation may arise from differences in energetic masking and/or divergent mechanisms underlying sound segregation and integration.

Investigating the role of harmonic cancellation in speech-on-speech masking

This study investigated the role of harmonic cancellation in the intelligibility of speech in “cocktail party” situations. While there is evidence that harmonic cancellation plays a role in the segregation of simple harmonic sounds based on fundamental frequency (F0), its utility for mixtures of speech containing non-stationary F0s and unvoiced segments is unclear. Here we focused on the energetic masking of speech targets caused by competing speech maskers. Speech reception thresholds were measured using seven maskers: speech-shaped noise, monotonized and intonated harmonic complexes, monotonized speech, noise-vocoded speech, reversed speech and natural speech. These maskers enabled an estimate of how the masking potential of speech is influenced by harmonic structure, amplitude modulation and variations in F0 over time. Measured speech reception thresholds were compared to the predictions of two computational models, with and without a harmonic cancellation component. Overall, the results suggest a minor role of harmonic cancellation in reducing energetic masking in speech mixtures.

Blind separation of speech from aortic regurgitation signals using Dhoulath’s method

ABSTRACT Conducting auscultation of traumatically distressed patients has always been demanding for medical professionals. The challenge calls for an innovative solution enabling doctors to conduct precise diagnoses despite other sound interference. This suggested study presents an entirely non-invasive and convenient method designed to aid doctors in routine diagnostic procedures. This study is centred on the segregation of aortic regurgitation heart sounds from speech. The mixture utilised for the study is a combination of speech and aortic regurgitation signals. The method applied for the study is a revised procedure of Blind Source component separation utilising a solo sensor method. With this technique, doctors are not compelled to prevent patients from articulating their pain or discomfort while diagnosing heart sounds. Doctors can offer a consoling word to patients while the auscultation is in progress without worrying about how the speech sounds affect the diagnosis. For babies, timely detection of heart-related issues can be life-saving. With Dhoulath’s method, the distressing sounds of a baby’s cries can be effectively separated, thereby offering doctors clear audio of heartbeats. The study was conducted to ascertain if heartbeats can be segregated from the signals of speech or cries. This segregation procedure has succeeded in arriving at an enhanced level of clarity.

EEG Responses to auditory figure-ground perception

Speech-in-noise difficulty is commonly reported among hearing-impaired individuals. Recent work has established generic behavioural measures of sound segregation and grouping that are related to speech-in-noise processing but do not require language. In this study, we assessed potential clinical electroencephalographic (EEG) measures of central auditory grouping (stochastic figure-ground test) and speech-in-noise perception (speech-in-babble test) with and without relevant tasks. Auditory targets were presented within background noise (16 talker-babble or randomly generated pure-tones) in 50% of the trials and composed either a figure (pure-tone frequency chords repeating over time) or speech (English names), while the rest of the trials only had background noise. EEG was recorded while participants were presented with the target stimuli (figure or speech) under different attentional states (relevant task or visual-distractor task). EEG time-domain analysis demonstrated enhanced negative responses during detection of both types of auditory targets within the time window 150-350 ms but only figure detection produced significantly enhanced responses under the distracted condition. Further single-channel analysis showed that simple vertex-to-mastoid acquisition defines a very similar response to more complex arrays based on multiple channels. Evoked-potentials to the generic figure-ground task therefore represent a potential clinical measure of grouping relevant to real-world listening that can be assessed irrespective of language knowledge and expertise even without a relevant task.

Long latency auditory evoked potentials and object-related negativity based on harmonicity in hearing-impaired children

Hearing-impaired children (HIC) have difficulty understanding speech in noise, which may be due to difficulty parsing concurrent sound object based on harmonicity cues. Using long latency auditory evoked potentials (LLAEPs) and object-related negativity (ORN), a neural metric of concurrent sound segregation, this study investigated the sensitivity of HIC in processing harmonic relation. The participants were 14 normal-hearing children (NHC) with an average age of 7.82 ± 1.31 years and 17 HIC with an average age of 7.98 ± 1.25 years. They were presented with a sequence of 200 Hz harmonic complex tones that had either all harmonic in tune or the third harmonic mistuned by 2%, 4%, 8%, and 16% of its original value while neuroelectric brain activity was recorded. The analysis of scalp-recorded LLAEPs revealed lower N2 amplitudes elicited by the tuned stimuli in HIC than control. The ORN, isolated in difference wave between LLAEP elicited by tuned and mistuned stimuli, was delayed and smaller in HIC than NHC. This study showed that deficits in processing harmonic relation in HIC, which may contribute to their difficulty in understanding speech in noise. As a result, top-down and bottom-up rehabilitations aiming to improve processing of basic acoustic characteristics, including harmonics are recommended for children with hearing loss.

Fusion of dichotic vowels with level differences across ears in normal-hearing listeners

Fundamental frequency (voice pitch) is an important cue for sound segregation. Previous studies show that normal-hearing (NH) listeners fuse pure tones with similar pitches. When two different vowels with the same fundamental frequencies are presented dichotically to the two ears, NH listeners often fuse the two vowels and perceive one vowel. Sometimes the perceived vowel is not one of the two vowels presented. Interaural level difference is an important cue for sound localization, which also helps with sound segregation. The current study introduced level differences into the dichotic vowel pair to examine how level differences affect vowel fusion. Dichotic vowel pairs were selected from synthetic vowels /i/, /ae/, /ɔ/, and /ʊ/ with the fundamental frequency of 109.9Hz. The level difference varied between -8 and 8 dB. The results showed that there was no significant effect of level difference on fusion in NH listeners. However, for some vowel pairs, the single fused vowel perceived changed significantly with level difference, indicating that internal representations of fused vowels can shift with level differences. This research was funded by NIH-NIDCD-R01-DC013307.

Binding the Acoustic Features of an Auditory Source through Temporal Coherence.

Numerous studies have suggested that the perception of a target sound stream (or source) can only be segregated from a complex acoustic background mixture if the acoustic features underlying its perceptual attributes (e.g., pitch, location, and timbre) induce temporally modulated responses that are mutually correlated (or coherent), and that are uncorrelated (incoherent) from those of other sources in the mixture. This “temporal coherence” hypothesis asserts that attentive listening to one acoustic feature of a target enhances brain responses to that feature but would also concomitantly (1) induce mutually excitatory influences with other coherently responding neurons, thus enhancing (or binding) them all as they respond to the attended source; by contrast, (2) suppressive interactions are hypothesized to build up among neurons driven by temporally incoherent sound features, thus relatively reducing their activity. In this study, we report on EEG measurements in human subjects engaged in various sound segregation tasks that demonstrate rapid binding among the temporally coherent features of the attended source regardless of their identity (pure tone components, tone complexes, or noise), harmonic relationship, or frequency separation, thus confirming the key role temporal coherence plays in the analysis and organization of auditory scenes.

The effect of harmonic training on speech perception in noise in hearing-impaired children

ObjectiveSpeech perception in noise is a highly challenging situation experienced by hearing-impaired children (HIC). Despite advances in hearing aid technologies, speech perception in noise still poses challenges. Pitch-based training improves pitch discrimination and speech perception and may facilitate concurrent sound segregation. Considering the role of harmonics in the analysis of concurrent sounds, we performed a harmonic assessment, examined the role of harmonic training in the rehabilitation of moderate-to-severe HIC, and investigated its effect on their speech perception in noise. MethodsThe participants were 57 normally hearing children (NHC) with a mean age of 7.73 ± 1.57 years and 18 HIC with a mean age of 7.94 ± 1.47 years. The two groups were compared in terms of harmonic assessment, the Pitch Pattern Sequence Test (PPST), the Consonant-Vowel in Noise (CV in noise) test, and the Bamford-Kowal Bench (BKB) test. Subsequently, the HIC underwent harmonic training, and the results of the pre- and post-harmonic training assessments were compared. ResultsHIC displayed poorer harmonic discrimination than NHC at all harmonics (P < 0.05). They also showed lower scores in PPST, CV in noise, and BKB tests compared to NHC (P < 0.05). Harmonic training led to HIC's better performance in harmonic assessment, PPST, and CV in noise test (P < 0.05). However, the BKB test results pre- and post-training did not significantly differ (P > 0.05). ConclusionHarmonic training plays a significant role in improving the HIC's temporal processing of the PPST and CV in noise test; therefore, it can serve as a rehabilitation method to enhance temporal processing and auditory scene analysis.

Infants' use of isolated and combined temporal cues in speech sound segregation.

This paper investigates infants' and adults' use of envelope cues and combined onset asynchrony and envelope cues in the segregation of concurrent vowels. Listeners heard superimposed vowel pairs consisting of two different vowels spoken by a male and a female talker and were trained to respond to one specific target vowel, either the male /u:/ or male /i:/. Vowel detection was measured in three conditions. In the baseline condition the two superimposed vowels had similar amplitude envelopes and synchronous onset. In the envelope cue condition, the amplitude envelopes of the two vowels differed. In the combined cue condition, both the onset time and amplitude envelopes of the two vowels differed. Seven-month-old infants' concurrent vowel segregation improved both with envelope and with combined onset asynchrony and envelope cues to the same extent as adults'. A preliminary investigation with 3-month-old infants suggested that neither envelope cues nor combined asynchrony and envelope cues improved their ability to detect the target vowel. Taken together, these results suggest that envelope and combined onset-asynchrony cues are available to infants as they attempt to process competing speech sounds, at least after 7 months of age.

Perceptual fusion of musical notes by native Amazonians suggests universal representations of musical intervals

Music perception is plausibly constrained by universal perceptual mechanisms adapted to natural sounds. Such constraints could arise from our dependence on harmonic frequency spectra for segregating concurrent sounds, but evidence has been circumstantial. We measured the extent to which concurrent musical notes are misperceived as a single sound, testing Westerners as well as native Amazonians with limited exposure to Western music. Both groups were more likely to mistake note combinations related by simple integer ratios as single sounds (‘fusion’). Thus, even with little exposure to Western harmony, acoustic constraints on sound segregation appear to induce perceptual structure on note combinations. However, fusion did not predict aesthetic judgments of intervals in Westerners, or in Amazonians, who were indifferent to consonance/dissonance. The results suggest universal perceptual mechanisms that could help explain cross-cultural regularities in musical systems, but indicate that these mechanisms interact with culture-specific influences to produce musical phenomena such as consonance.

Hearing in 3D: Directional Auditory Sensitivity of Northern Saw-Whet Owls (Aegolius acadicus).

Northern saw-whet owls (Aegolius acadicus) are nocturnal predators that are able to acoustically localize prey with great accuracy; an ability that is attributed to their unique asymmetrical ear structure. While a great deal of research has focused on open loop sound localization prior to flight in owls (primarily barn owls), directional sensitivity of the ears may also be important in locating moving prey on the wing. Furthermore, directionally sensitive ears may also reduce the effects of masking noise, either from the owls' wings during flight or environmental noise (e.g., wind and leaf rustling), by enhancing spatial segregation of target sounds and noise sources. Here, we investigated auditory processing of Northern saw-whet owls in three-dimensional space using auditory evoked potentials (AEPs). We simultaneously evoked auditory responses in two channels (right and left ear) with broadband clicks from a sound source that could be manipulated in space. Responses were evoked from 66 spatial locations, separated by 30° increments in both azimuth and elevation. We found that Northern saw-whet owls had increased sensitivity to sound sources directly in front of and above their beaks and decreased sensitivity to sound sources below and behind their heads. The spatial region of highest sensitivity extends from the lower beak to the crown of the head and 30° left or right of the median plane, dropping off beyond those margins. Directional sensitivity is undoubtedly useful during foraging and predator evasion, and may also reduce the effect of masking noise from the wings during flight due to the spatial segregation of the noise and targets of interest.

Electro-Haptic Enhancement of Spatial Hearing in Cochlear Implant Users

Cochlear implants (CIs) have enabled hundreds of thousands of profoundly hearing-impaired people to perceive sounds by electrically stimulating the auditory nerve. However, CI users are often very poor at locating sounds, which leads to impaired sound segregation and threat detection. We provided missing spatial hearing cues through haptic stimulation to augment the electrical CI signal. We found that this “electro-haptic” stimulation dramatically improved sound localisation. Furthermore, participants were able to effectively integrate spatial information transmitted through these two senses, performing better with combined audio and haptic stimulation than with either alone. Our haptic signal was presented to the wrists and could readily be delivered by a low-cost wearable device. This approach could provide a non-invasive means of improving outcomes for the vast majority of CI users who have only one implant, without the expense and risk of a second implantation.

Orienting Attention to Auditory and Visual Short-term Memory: The Roles of Age, Hearing Loss, and Cognitive Status

ABSTRACTBackground/Study Context: Attention can be reflectively oriented to a visual or auditory representation in short-term memory, but it is not clear how aging and hearing acuity affects reflective attention. The purpose of the present study was to examine whether performance in auditory and visual reflective attention tasks varies as a function of participants’ age and hearing status.Methods: Young (19 to 33 years) and older adults with normal or mild to moderate hearing loss (62–90 years) completed a delayed match-to-sample task in which participants were first presented with a memory array of four different digits to hold in memory. Two digits were presented visually (left and right hemifield), and two were presented aurally (left and right ears simultaneously). During the retention interval, participants were presented with a cue (dubbed retro-cue), which could be either uninformative or indicated to the participants to retrospectively orient their attention to either auditory short-term memory (ASTM) or visual short-term memory (VSTM). The cue was followed by another delay, after which a single item was presented (i.e., test probe) for comparison (match or no match) with the items held in ASTM and/or VSTM.Results: Overall, informative retro-cue yielded faster response time than uninformative retro-cue. The retro-cue benefit in response time was comparable for auditory and visual-orienting retro-cue and similar in young and older adults. Regression analyses showed that only the auditory-orienting retro-cue benefit was predicted by hearing status rather than age per se.Conclusion: Both younger and older adults can benefit from visual and auditory-orienting retro-cues, but the auditory-orienting retro-cue benefit decreases with poorer hearing acuity. This finding highlights changes in cognitive processes that come with age even in those with just mild-to-moderate hearing loss, and suggest that older adults’ performance in working memory tasks is sensitive to low level auditory scene analysis (i.e., concurrent sound segregation).

A biologically oriented sound segregation algorithm

A biologically oriented sound segregation algorithm