Detection Of Speech In Noise Research Articles

Impact of Unilateral Hearing Loss on Behavioral and Evoked Potential Measures of Auditory Function in Adults.

A unilateral hearing loss (UHL) can have a significant functional and social impact on children and adults, affecting their quality of life. In adults, UHL is typically associated with difficulties understanding speech in noise and sound localization, and UHL increases the self-perception of auditory disability for a range of listening situations. Furthermore, despite evidence for the negative effects of reduced unilateral auditory input on the neural encoding of binaural cues, the perceptual consequences of these changes are still not well understood. Determine effects of UHL on auditory abilities and speech-evoked cortical auditory evoked potentials (CAEPs). CAEPs, sound localization, speech perception in noise and self-perception of auditory abilities (speech, spatial, and qualities hearing scale) were assessed. Thirteen adults with UHL with a range of etiologies, duration of hearing loss, and severity and a control group of eleven binaural listeners with normal hearing. Participants with UHL varied greatly in their ability to localize sound and reported speech recognition and listening effort were the greatest problem. There was a greater effect of right ear than left ear hearing loss on N1 amplitude hemispheric asymmetry and N1 latencies evoked by speech syllables in noise. As duration of hearing loss increased, contralateral dominance (N1 amplitude asymmetry) decreased. N1 amplitudes correlated with speech scores, larger N1 amplitudes were associated with better speech recognition in noise scores. N1 latencies are delayed (in the better ear) and amplitude hemisphere asymmetry differed across UHL participants as function of side of deafness, mainly for right-sided deafness. UHL affects a range of auditory abilities, including speech detection in noise, sound localization, and self-perceived hearing disability. CAEPs elicited by speech sounds are sensitive enough to evidence changes within the auditory cortex due to an UHL.

Audiovisual speech detection benefits: A comparison across psychophysical approaches

Visual speech reduces uncertainty as to when auditory speech will occur, thereby enhancing detection of speech in noise. These detection benefits reflect a low-level perceptual mechanism of AV speech enhancement that is independent of the linguistic nature of speech. Audiovisual speech detection benefits have primarily been assessed using two-interval forced-choice paradigms. Results from AV speech detection experiments using three different psychophysical approaches will be presented. These include a single-interval yes/no task, a go/no-go task, and a two-interval forced choice task. In all experiments, adults with normal hearing complete an auditory-only condition with a still image of the talker and an audiovisual condition with a synchronous and congruent visual speech signal. The audiovisual condition includes foil trials/intervals that force participants to use the auditory information to make the decision. Adults demonstrate significant audiovisual benefit to speech detection across all three task,...

Speech Detection in Noise for Young Bilaterally Implanted Children: Is There Evidence of Binaural Benefit Over the Shadowed Ear Alone?

To measure binaural benefit over the shadowed ear alone for young bilateral cochlear implant (CI) users. It was hypothesized that children who received bilateral CIs at a young age (<4 years), and had significant bilateral experience, would demonstrate lower detection thresholds for speech sounds in background noise in the bilateral CI over the unilateral CI condition when the added CI was ipsilateral to the noise source. Children receiving bilateral CIs at the Eye and Ear Hospital Clinic in Melbourne were invited to participate in a wider research project evaluating outcomes; those participating in the wider project who were bilaterally implanted by 4 years and were approximately 2 years postoperative were included in the present study. For 20 participants, detection signal to noise ratios (SNRs) were measured for speech presented from in front and noise from 90° in at least 3 of 4 device/noise conditions, namely left CI/noise right and right CI/noise left, plus bilateral CIs/noise right and bilateral CIs/noise left. As some participants could only complete testing in 3 conditions within the 1 test block, the unilateral versus bilateral comparison was performed for 1 CI (i.e., 1 noise direction) for 15 participants and for both CIs (i.e., noise left and noise right) for 5 participants. Group analysis indicated no significant difference in detection SNR between the unilateral and bilateral CI conditions when adding the left CI or right CI (for the overall group) or when adding the first or second CI (for the 15 participants with sequential bilateral CIs). Separate analyses indicated no significant difference in detection SNR between the unilateral and bilateral CI conditions for the majority of individuals; this occurred irrespective of whether the analysis indicated that the CI added in the bilateral condition was poorer-performing, better-performing, or not significantly different compared with the other CI. Four individuals demonstrated a significant improvement in the bilateral condition when the CI added in the bilateral condition was a better-performing (n = 1), poorer-performing (n = 2), or not significantly different CI (n = 1). There was no relationship between the detection SNR difference between each CI and the detection SNR difference between the unilateral and bilateral conditions. The hypothesis of a lower detection SNR in the bilateral condition was not supported by the group results or by the results for the majority of individuals. For the 4 participants who did demonstrate benefit over the shadowed ear alone, that benefit cannot be separated from the potential benefit gained as a result of the CI added in the bilateral condition being the better-performing CI for 1 of the 4. Variation in outcomes could not be related to demographic factors for this group, which was relatively homogeneous for age at bilateral CI and experience; an older, more experienced group may demonstrate greater binaural benefit in these conditions. These results can be used during counseling for families regarding postoperative expectations for young children, especially in the first 2 years.

Uninformative dynamic visual stimuli aid in segregating two similar acoustic stimuli, but not in detecting a single stimulus in noise

Both the comprehension and detection of speech in noise are improved when the listener sees the talker’s mouth. There are multiple reasons for this, from basic physical temporal correlations to higher order linguistic cues; we have recently performed several experiments investigating the former. They were based on artificial stimuli with speech-like dynamics but no linguistic information. Auditory stimuli were a tone or tone complex with randomly modulated amplitude. Visual stimuli were a disc with a randomly modulated radius. We manipulated the correlation between the visual stimulus and each auditory stimulus. In all experiments, the visual stimulus provided no information about the task. In the first study, we presented two competing auditory stimuli and had listeners respond to events in the target stimulus (brief pitch or timbre fluctuations). Performance was better when the visual stimulus matched the auditory target than when it matched the masker. The second study employed a two-interval two-alter...

Whispered Speech Detection in Noise Using Auditory-Inspired Modulation Spectrum Features

Robustness to ambient noise, varying vocal effort, and availability of only short-duration test utterances represent big challenges for developers of automated speech-enabled applications. Recent studies have proposed the use of vocal effort-matched speaker models as a potential solution to such challenges. However, detecting whispered speech in extremely noisy environments is not a trivial task. This letter proposes the use of auditory-inspired modulation spectral-based features as a method of separating speech from environment-based components, thus resulting in accurate whispered speech detection at signal-to-noise ratios as low as 0 dB. Experimental results show the proposed detection algorithm outperforming two benchmark approaches.

Speech Detection in Noise and Spatial Unmasking in Children With Simultaneous Versus Sequential Bilateral Cochlear Implants

To measure speech detection in noise performance for children with bilateral cochlear implants (BiCI), to compare performance in children with simultaneous implant versus those with sequential implant, and to compare performance to normal-hearing children. Prospective cohort study. Tertiary academic pediatric center. Children with early-onset bilateral deafness and 2-year BiCI experience, comprising the "sequential" group (>2 yr interimplantation delay, n = 12) and "simultaneous group" (no interimplantation delay, n = 10) and normal-hearing controls (n = 8). Thresholds to speech detection (at 0-degree azimuth) were measured with noise at 0-degree azimuth or ± 90-degree azimuth. Spatial unmasking (SU) as the noise condition changed from 0-degree azimuth to ± 90-degree azimuth and binaural summation advantage (BSA) of 2 over 1 CI. Speech detection in noise was significantly poorer than controls for both BiCI groups (p < 0.0001). However, the SU in the simultaneous group approached levels found in normal controls (7.2 ± 0.6 versus 8.6 ± 0.6 dB, p > 0.05) and was significantly better than that in the sequential group (3.9 ± 0.4 dB, p < 0.05). Spatial unmasking was unaffected by the side of noise presentation in the simultaneous group but, in the sequential group, was significantly better when noise was moved to the second rather than the first implanted ear (4.8 ± 0.5 versus 3.0 ± 0.4 dB, p < 0.05). This was consistent with a larger BSA from the sequential group's second rather than first CI. Children with simultaneously implanted BiCI demonstrated an advantage over children with sequential implant by using spatial cues to improve speech detection in noise.

Spatial hearing of normally hearing and cochlear implanted children

ObjectiveSpatial hearing uses both monaural and binaural mechanisms that require sensitive hearing for normal function. Deaf children using either bilateral (BCI) or unilateral (UCI) cochlear implants would thus be expected to have poorer spatial hearing than normally hearing (NH) children. However, the relationship between spatial hearing in these various listener groups has not previously been extensively tested under ecologically valid conditions using a homogeneous group of children who are UCI users. We predicted that NH listeners would outperform BCI listeners who would, in turn, outperform UCI listeners. MethodsWe tested two methods of spatial hearing to provide norms for NH and UCI using children and preliminary data for BCI users. NH children (n=40) were age matched (6–15 years) to UCI (n=12) and BCI (n=6) listeners. Testing used a horizontal ring of loudspeakers within a booth in a hospital outpatient clinic. In a ‘lateral release’ task, single nouns were presented frontally, and masking noises were presented frontally, or 90° left or right. In a ‘localization’ task, allowing head movements, nouns were presented from loudspeakers separated by 30°, 60° or 120° about the midline. ResultsNormally hearing children improved with age in speech detection in noise, but not in quiet or in lateral release. Implant users performed more poorly on all tasks. For frontal signals and noise, UCI and BCI listeners did not differ. For lateral noise, BCI listeners performed better on both sides (within ∼2dB of NH), whereas UCI listeners benefited only when the noise was opposite the unimplanted ear. Both the BCI and, surprisingly, the UCI listeners performed better than chance at all loudspeaker separations on the ecologically valid, localization task. However, the BCI listeners performed about twice as well and, in two cases, approached the performance of NH children. ConclusionChildren using either UCI or BCI have useful spatial hearing. BCI listeners gain benefits on both sides, and localize better, but not as well as NH listeners.

Speech detection and localization results and clinical outcomes for children receiving sequential bilateral cochlear implants before four years of age

The aim of this study was to describe the adaptation to bilateral cochlear implant use and the perceptual benefits demonstrated by 10 children who were successful users of a first implant when a second was received before four years of age. Although one subject rejected the second implant at switch-on, the nine subjects who accepted the device adapted easily to bilateral implant use and developed useful listening skills with the second implant. Tests of localization (left versus right) and speech detection in noise were administered in the unilateral and bilateral conditions, usually after six months experience. All subjects demonstrated some bilateral benefit on speech detection testing (mostly due to a headshadow effect), and the majority localized left versus right. Results suggested that outcomes may be negatively impacted by increased age at the time of second implant switch-on. The majority of the subjects adapted well to bilateral implant use within six months and demonstrated some perceptual benefit and, according to subjective parent reports, improved daily functioning; however, device rejection must be discussed pre-operatively as a possibility.

Investigating the audio–visual speech detection advantage

Seeing the moving face of the talker permits better detection of speech in noise compared to auditory only presentation, an Audio–Visual (AV) facilitation effect. Three experiments that used a masked speech detection task are reported. The experiments were designed to contrast two accounts of the AV facilitation effect (AV peak listening and AV grouping). In each experiment a different manipulation of the relationship between the auditory and visual signals was employed. The first experiment manipulated the sequencing of the visual and auditory information by presenting the displays time normal or time reversed. The results showed that AV facilitation only occurred for the time normal presentations where there was a high correlation between the AV signals. Experiment 2 examined the impact on AV facilitation of shifting the auditory signals earlier in time than its normal position (again with time normal and time reversed presentation). It was found that shifting the auditory component abolished the AV effect for both the time normal and reversed conditions. The final experiment examined the AV detection advantage using another situation in which the relationship between the AV signals differed. Two versions of AV speech produced by a virtual talker were investigated. In one version, based on text-to-speech synthesis, the video and auditory signals were more rapid than the human talker of Experiment 1. In the other version, the signals were lengthened to match the durations of the human talker. A small but reliable AV facilitation effect was only found for the second version. The results are consistent with a cross-modal peak listening account and are discussed in terms of constraints on the integration of auditory and visual speech.

Auditory speech detection in noise enhanced by lipreading

Audiovisual speech stimuli have been shown to produce a variety of perceptual phenomena. Enhanced detectability of acoustic speech in noise, when the talker can also be seen, is one of those phenomena. This study investigated whether this enhancement effect is specific to visual speech stimuli or can rely on more generic non-speech visual stimulus properties. Speech detection thresholds for an auditory /ba/ stimulus were obtained in a white noise masker. The auditory /ba/ was presented adaptively to obtain its 79.4% detection threshold under five conditions. In Experiment 1, the syllable was presented (1) auditory-only (AO) and (2) as audiovisual speech (AVS), using the original video recording. Three types of synthetic visual stimuli were also paired synchronously with the audio token: (3) A dynamic Lissajous (AVL) figure whose vertical extent was correlated with the acoustic speech envelope; (4) a dynamic rectangle (AVR) whose horizontal extent was correlated with the speech envelope; and (5) a static rectangle (AVSR) whose onset and offset were synchronous with the acoustic speech onset and offset. Ten adults with normal hearing and vision participated. The results, in terms of dB signal-to-noise ratio (SNR), were AVS < (AVL ≈ AVR ≈ ASR) < AO. That is, AVS was significantly easiest to detect, there was no difference among the synthesized visual stimuli, and all audiovisual conditions resulted in significantly lower thresholds than AO. To determine the advantage of the AVS stimulus, in Experiment 2, a preliminary mouth gesture was edited from the video speech token. This manipulation defeated the advantage for both the original and the edited AVS stimulus, while the audiovisual detection enhancement persisted. Overall, the results showed enhanced auditory speech detection with visual stimuli but no advantage for a fine-grained correlation between acoustic and optical speech signals.

Feature extraction in human speech recognition

In 1955 George Miller and Patricia Nicely explored the human speech code and discovered that a five-channel feature code is used to extract phonemes from speech. Their mutual information analysis of the confusion matrices of filtered noisy speech showed that the five channels were virtually independent of each other, accounting for the robustness of speech communication quantified by Miller, Heise, and Lichten in 1951. While the 1955 analysis was a major step forward, it did not lead to practical results, since there are no known methods for extracting these features from the speech. This weakness is the basis of the present research program, to define the signal processing required to robustly extract features from narrow bands of speech. Our emphasis is to work in narrow bands, starting from critical bands. In this paper we discuss the first step of this process, which is to measure (a) the jnd for the detection of speech in noise and (b) the jnd for residue pitch given narrow bands of speech in noise. We have been investigating these features with the ultimate goal of trying to build a speech recognition machine based on distinctive feature signal processing.

The use of visible speech cues for improving auditory detection of spoken sentences.

Classic accounts of the benefits of speechreading to speech recognition treat auditory and visual channels as independent sources of information that are integrated fairly early in the speech perception process. The primary question addressed in this study was whether visible movements of the speech articulators could be used to improve the detection of speech in noise, thus demonstrating an influence of speechreading on the ability to detect, rather than recognize, speech. In the first experiment, ten normal-hearing subjects detected the presence of three known spoken sentences in noise under three conditions: auditory-only (A), auditory plus speechreading with a visually matched sentence (AV(M)) and auditory plus speechreading with a visually unmatched sentence (AV(UM). When the speechread sentence matched the target sentence, average detection thresholds improved by about 1.6 dB relative to the auditory condition. However, the amount of threshold reduction varied significantly for the three target sentences (from 0.8 to 2.2 dB). There was no difference in detection thresholds between the AV(UM) condition and the A condition. In a second experiment, the effects of visually matched orthographic stimuli on detection thresholds was examined for the same three target sentences in six subjects who participated in the earlier experiment. When the orthographic stimuli were presented just prior to each trial, average detection thresholds improved by about 0.5 dB relative to the A condition. However, unlike the AV(M) condition, the detection improvement due to orthography was not dependent on the target sentence. Analyses of correlations between area of mouth opening and acoustic envelopes derived from selected spectral regions of each sentence (corresponding to the wide-band speech, and first, second, and third formant regions) suggested that AV(M) threshold reduction may be determined by the degree of auditory-visual temporal coherence, especially between the area of lip opening and the envelope derived from mid- to high-frequency acoustic energy. Taken together, the data (for these sentences at least) suggest that visual cues derived from the dynamic movements of the fact during speech production interact with time-aligned auditory cues to enhance sensitivity in auditory detection. The amount of visual influence depends in part on the degree of correlation between acoustic envelopes and visible movement of the articulators.

The use of visible speech cues (speechreading) for directing auditory attention: Reducing temporal and spectral uncertainty in auditory detection of spoken sentences

It is well established that auditory-visual speech recognition is far superior to auditory-only speech recognition. Classic accounts of the benefits of speechreading to speech recognition treat auditory and visual channels as independent sources of information that are integrated early in the speech perception process, most likely at a precategorical stage. The question addressed in this study was whether visible movements of the speech articulators could be used to improve the detection of speech in noise, thus demonstrating an influence of speechreading on the processing of low-level auditory cues. Subjects were required to detect the presence of spoken sentences in noise under three conditions: auditory-only, auditory-visual with a visually matched sentence, and auditory-visual with a visually unmatched sentence. The potential benefits of congruent visual speech cues to auditory detection will be discussed in terms of a reduction of signal uncertainty, in both temporal and spectral domains. In effect, the visual channel may serve to inform the auditory system about when (in time) and where (in frequency) to expect changes in the acoustic signal. [Work supported by NIH Grant DC00792.]

Can simultaneous presentation of print influence the detection of speech in noise

When spoken words are masked by noise having the same amplitude envelope, subjects report they hear the word much more clearly when they see its printed version at the same time. Using signal detection methodology, it was investigated whether this subjective impression reflects a change in perceptual sensitivity or in bias. Signal‐plus‐noise and noise‐only trials were accompanied by matching print, nonmatching (but structurally similar) print, or a neutral visual stimulus. These results revealed a strong bias effect: The matching visual input apparently made the masking noise sound more speechlike, but it did not improve the detectability of the speech. (However, reaction times of correct detections were reliably shorter in the matching condition, suggesting perhaps a subliminal facilitation.) The bias effect was much smaller when nonwords were substituted for the words. Thus it seems that subjects automatically detect correspondences between speech amplitude envelopes and printed stimuli, and they do this more efficiently when the printed stimuli are real words. This supports the hypothesis, much discussed in the reading literature, that printed words are immediately translated into an internal representation having speechlike characteristics. [Work supported by NICHD.]

Infants' detection of speech in noise.

Localization responses to a speech phrase masked by white noise were obtained from infants 6, 12, 18, and 24 months of age and from adults. The masking noise was presented continuously from two loudspeakers located 45 degrees to each side of the infant. During a trial the speech phrase was presented through one of the loudspeakers. A head turn to the signal (correct response) was rewarded by activating an animated toy on top of the speaker. The intensity of the signal was varied over trials (method of constant stimuli) to determine thresholds (defined as the intensity corresponding to 65% correct head turns) at each of two levels of masking noise, 42 and 60 dBC. Thresholds for the speech signal were comparable across all infant groups for both levels of masking noise. Increasing the masking noise from 42 to 60 dBC resulted in a threshold shift of comparable magnitude for infants and adults. However, adult thresholds were approximately 10-12 dB lower than those of infants at both masking levels.

Speech Detection in Noise

Exploratory work to optimize the signal-to-noise ratio and the wearability of speech detection devices for use in high-altitude flight has led to quantitative determination of the relative capabilities of head contact devices in relation to acoustic detectors. One of the discoveries of the exploratory work was that an accelerometer fastened to an upper front tooth will detect highly intelligible speech and discriminate strongly against external noise. Data is presented on the acceleration spectrum of the teeth as a result of speech and external noise. The signal-to-noise ratio of the tooth microphone is compared to other speech reception systems under the same conditions. Refinement of the attachment of the accelerometer to the teeth is discussed. (This research was supported by U. S. Air Force contract.)