Speech Intelligibility In Reverberation Research Articles

Speech intelligibility in reverberation based on audio-visual scenes recordings reproduced in a 3D virtual environment

Audio-visual scenes were collected in a medium-sized reverberant conference hall through in-field 3rd-order ambisonics impulse response recordings and 360-degree stereoscopic videos. The visual scenes included cues of the room and the location of the sound sources, without lip-sync-related cues. Speech intelligibility tests based on seven audio-visual scenes were administered inside an immersive virtual 3D environment reproduced through a spherical 16-speaker array synched with a head-mounted display. Forty normal-hearing subjects were engaged to test the effects on speech intelligibility of a talker in front of the listener and amplified by two lateral symmetrical loudspeakers, in the case of (i) different listener-to-talker distances, (ii) one-talker noise at various azimuth angles around the listener, (iii) high reverberation with –5 dB signal-to-noise ratio, (iv) self-motion, and (v) visual cues. We conducted tests in four configurations, that is, audio-visual and audio-only, both with self-motion and in the static condition. The static audio-only tests scored the highest speech intelligibility, followed by a tie between audio-visual with self-motion and in the static condition. Speech intelligibility decreased as the target-to-listener distance increased in all the noisy scenes. Additionally, speech intelligibility increased when the noise azimuth was at 120° compared to both 180° and 0° , with the talker at approximately 8 m from the listener. The advantage of the spatial separation of the noise signal in reverberation is evident in the case of the audio-visual with self-motion test. This suggests a spatial release from masking in the presence of reverberation, one-talker-interfering noise and within an more ecological scene.

An adaptive multichannel speech dereverberation approach based on robust estimation

Room reverberation usually produces adverse influence on the quality of speech in human-machine interaction scenarios. To improve the clarity and intelligibility of reverberant speech, an adaptive multichannel speech dereverberation approach is proposed in this work. Unlike the traditional least-squares cost function for speech dereverberation, a robust estimator with an adaptive parameter is adopted to define a flexible cost function of the adaptive speech dereverberation algorithm. As a consequence, the developed adaptive filtering algorithm can dynamically track the non-stationarity and non-Gaussian properties of speech so that the reverberation component of speech is effectively suppressed. Numerical simulation experiments validate the dereverberation performance of the proposed approach.

Speech Intelligibility in Reverberation is Reduced During Self-Rotation.

Speech intelligibility in cocktail party situations has been traditionally studied for stationary sound sources and stationary participants. Here, speech intelligibility and behavior were investigated during active self-rotation of standing participants in a spatialized speech test. We investigated if people would rotate to improve speech intelligibility, and we asked if knowing the target location would be further beneficial. Target sentences randomly appeared at one of four possible locations: 0°, ± 90°, 180° relative to the participant's initial orientation on each trial, while speech-shaped noise was presented from the front (0°). Participants responded naturally with self-rotating motion. Target sentences were presented either without (Audio-only) or with a picture of an avatar (Audio-Visual). In a baseline (Static) condition, people were standing still without visual location cues. Participants' self-orientation undershot the target location and orientations were close to acoustically optimal. Participants oriented more often in an acoustically optimal way, and speech intelligibility was higher in the Audio-Visual than in the Audio-only condition for the lateral targets. The intelligibility of the individual words in Audio-Visual and Audio-only increased during self-rotation towards the rear target, but it was reduced for the lateral targets when compared to Static, which could be mostly, but not fully, attributed to changes in spatial unmasking. Speech intelligibility prediction based on a model of static spatial unmasking considering self-rotations overestimated the participant performance by 1.4 dB. The results suggest that speech intelligibility is reduced during self-rotation, and that visual cues of location help to achieve more optimal self-rotations and better speech intelligibility.

Listener Factors Associated with Individual Susceptibility to Reverberation.

Reverberation is a source of acoustic degradation, present to varying extents in many everyday listening environments. The presence of reverberation decreases speech intelligibility, especially for listeners with hearing impairment. There is substantial variability in how susceptible individuals with hearing impairment are to the effects of reverberation (i.e., how intelligible reverberant speech is to a listener). Relatively little is known about the listener factors which drive that susceptibility. To identify listener factors that are associated with an individual's susceptibility to reverberation. Another purpose was to investigate how these listener factors are associated with reverberant susceptibility in relation to the amount of reverberation. The listener factors investigated were degree of hearing loss, age, temporal envelope sensitivity, and working memory capacity. This study used a correlational design to investigate the association between different listener factors and speech intelligibility with varying amounts of reverberation. Thirty-three older adults with sensorineural hearing loss participated in the study. Listener temporal envelope sensitivity was measured using a gap detection threshold task. Listener working memory capacity was measured using the Reading Span Test. Intelligibility of reverberant speech was measured using a set of low-context sentence materials presented at 70 dB SPL without individual frequency shaping. Sentences were presented at a range of realistic reverberation times, including no reverberation (0.0 sec), moderate reverberation (1.0 sec), and severe reverberation (4.0 sec). Stepwise linear regression analyses were conducted to model speech intelligibility using individual degree of hearing loss, age, temporal envelope sensitivity, and working memory capacity. A separate stepwise linear regression model was conducted to model listener speech intelligibility at each of the three levels of reverberation. As the amount of reverberation increased, listener speech intelligibility decreased and variability in scores among individuals increased. Temporal envelope sensitivity was most closely associated with speech intelligibility in the no reverberation condition. Both listener age and degree of hearing loss were significantly associated with speech intelligibility in the moderate reverberation condition. Both listener working memory capacity and age were significantly associated with speech intelligibility in the severe reverberation condition. The results suggest that suprathreshold listener factors can be used to best predict speech intelligibility across a range of reverberant conditions. However, which listener factor(s) to consider when predicting a listener's susceptibility to reverberation depends on the amount of reverberation in an environment. Clinicians may be able to use different listener factors to identify individuals who are more susceptible to reverberation and would be more likely to have difficulty communicating in reverberant environments.

Perceived listening effort and speech intelligibility in reverberation and noise for hearing-impaired listeners

Objective: The purpose of this study was to assess perceived listening effort and speech intelligibility in reverberant and noisy conditions for hearing-impaired listeners for conditions that are similar according to the speech transmission index (STI). Design: Scaled listening effort was measured in four different conditions at five different STI generated using various relative contributions of noise and reverberant interferences. Intelligibility was measured for a subset of conditions. Study sample: Twenty mildly to moderately hearing-impaired listeners. Results: In general, listening effort decreased and speech intelligibility increased with increasing STI. For simulated impulse responses consisting of white Gaussian noise exponentially decaying in time, a good agreement between conditions of different relative contributions of noise and reverberation was found. For real impulse responses, the STI slightly overestimated the effect of reverberation on the perceived listening effort and underestimated its effect on speech intelligibility. Including the average hearing loss in the calculation of the STI led to a better agreement between STI predictions and subjective data. Conclusion: Speech intelligibility and listening effort provide complementary tools to evaluate speech perception over a broad range of acoustic scenarios. In addition, when incorporating hearing loss information the STI provides a rough prediction of listening effort in these acoustic scenarios.

An example of dissociation between speech intelligibility and perceived reverberation

It is well known that reverberation can affect the intelligibility of speech. Psychophysical and computational results have demonstrated that the relationship is inverse: an increase in the amount of reverberation results in a decrease in intelligibility. From the architectural acoustics literature, it is also well known that there is a direct relationship between the physical amount of reverberation and perceived reverberation. It therefore might be assumed that perceived reverberation and intelligibility are inversely related, although here a situation is demonstrated in which the two are effectively independent of one another. Using virtual auditory space techniques to simulate reverberant sound field listening, it is shown that when reverberant sound level is artificially decreased in the ipsilateral ear and naturally preserved in the contralateral ear, perceived reverberation is unaffected, but speech intelligibility is markedly improved. This dissociation likely results from the differential monaural and binaural aspects of reverberation, and is consistent with the idea that perceived reverberation is multidimensional. These results also suggest a potential binaural approach to the application of improving speech intelligibility in reverberation that does not limit the positive sound quality benefits of reverberation.

Intelligibility of Reverberant Speech with Amplification: Limitation of Speech Intelligibility Metrics, and a Preliminary Examination of an Alternative Approach

This study examines the effect of speech level on intelligibility in different reverberation conditions, and explores the potential of loudness-based reverberation parameters proposed by Lee et al. [J. Acoust. Soc. Am., 131(2), 1194-1205 (2012)] to explain the effect of speech level on intelligibility in various reverberation conditions. Listening experiments were performed with three speech levels (LAeq of 55 dB, 65 dB and 75 dB) and three reverberation conditions (T20 of 1.0 s, 1.9 s and 4.0 s), and subjects listened to speech stimuli through headphones. Collected subjective data were compared with two conventional speech intelligibility parameters (Speech Intelligibility Index and Speech Transmission Index) and two loudness-based reverberation parameters (EDTN and TN). Results reveal that the effect of speech level on intelligibility changes with a room’s reverberation conditions, and that increased level results in reduced intelligibility in highly reverberant conditions. EDTN and TN explain this finding better than do STI and SII, because they consider many psychoacoustic phenomena important for the modeling of the effect of speech level varying with reverberation.

Speech intelligibility in reverberation with ideal binary masking: Effects of early reflections and signal-to-noise ratio threshold

Ideal binary masking is a signal processing technique that separates a desired signal from a mixture by retaining only the time-frequency units where the signal-to-noise ratio (SNR) exceeds a predetermined threshold. In reverberant conditions there are multiple possible definitions of the ideal binary mask in that one may choose to treat the target early reflections as either desired signal or noise. The ideal binary mask may therefore be parameterized by the reflection boundary, a predetermined division point between early and late reflections. Another important parameter is the local SNR threshold used in labeling the time-frequency units as either target or background. Two experiments were designed to assess the impact of these two parameters on speech intelligibility with ideal binary masking for normal-hearing listeners in reverberant conditions. Experiment 1 shows that in order to achieve intelligibility improvements only the early reflections should be preserved by the binary mask. Moreover, it shows that the effective SNR should be accounted for when deciding the local threshold optimal range. Experiment 2 shows that with long reverberation times, intelligibility improvements are only obtained when the reflection boundary is 100 ms or less. Also, the experiment suggests that binary masking can be used for dereverberation.

Blind binary masking for reverberation suppression in cochlear implants

A monaural binary time-frequency (T-F) masking technique is proposed for suppressing reverberation. The mask is estimated for each T-F unit by extracting a variance-based feature from the reverberant signal and comparing it against an adaptive threshold. Performance of the estimated binary mask is evaluated in three moderate to relatively high reverberant conditions (T60 = 0.3, 0.6, and 0.8 s) using intelligibility listening tests with cochlear implant users. Results indicate that the proposed T-F masking technique yields significant improvements in intelligibility of reverberant speech even in relatively high reverberant conditions (T60 = 0.8 s). The improvement is hypothesized to result from the recovery of the vowel/consonant boundaries, which are severely smeared in reverberation.

Prior listening exposure to a reverberant room improves open-set intelligibility of high-variability sentences

Previous studies have demonstrated that speech understanding in reverberant rooms improves when listeners are given prior exposure to the room. Results from these room-adaptation studies are limited, however, because they were conducted with materials that are not representative of the high acoustic variability observed in speech signals during everyday communication. Here, room adaptation effects were measured using an open-set speech corpus with high lexical and indexical variability and virtual auditory space techniques to simulate binaural listening in rooms. Room adaptation effects of comparable magnitude to previous studies were observed, suggesting general importance for facilitating speech intelligibility in reverberation.

Candidacy for Bilateral Hearing Aids: A Retrospective Multicenter Study

The goal of this study was to find factors for refining candidacy criteria for bilateral hearing aid fittings. Clinical files of 1,000 consecutive hearing aid fittings were analyzed. Case history, audiometric, and rehabilitation data were collected from clinical files, and an extensive questionnaire on long-term outcome measures was conducted. After at least 2 years of hearing aid use, 505 questionnaires were returned. In order to compare differences in benefits between unilateral and bilateral fittings, two subgroups were composed in which most relevant variables (age, degree of hearing loss, and audiometric asymmetry) were matched for unilateral fittings (n=212) and bilateral fittings (n=477). The bilateral group had significantly higher benefit scores than the unilateral group for detection, speech intelligibility in reverberation, and localization, but poorer scores for comfort of loud sounds. The benefit of bilateral hearing aids was not significantly related to the level of technology of the hearing aids. The analysis of the relation between objective parameters and the subjective outcome measures showed that candidacy for a successful bilateral fitting could not be predicted from age, maximum speech intelligibility, employment, exposure to background noise, or social activities.

Speech segregation in rooms: Effects of reverberation on both target and interferer

Speech reception thresholds were measured to investigate the influence of a room on speech segregation between a spatially separated target and interferer. The listening tests were realized under headphones. A room simulation allowed selected positioning of the interferer and target, as well as varying the absorption coefficient of the room internal surfaces. The measurements involved target sentences and speech-shaped noise or 2-voice interferers. Four experiments revealed that speech segregation in rooms was not only dependent on the azimuth separation of sound sources, but also on their direct-to-reverberant energy ratio at the listening position. This parameter was varied for interferer and target independently. Speech intelligibility decreased as the direct-to-reverberant ratio of sources was degraded by sound reflections in the room. The influence of the direct-to-reverberant ratio of the interferer was in agreement with binaural unmasking theories, through its effect on interaural coherence. The effect on the target occurred at higher levels of reverberation and was explained by the intrinsic degradation of speech intelligibility in reverberation.

Effects of listening environments, listeners and preprocessing parameters on steady-state suppression for improving speech intelligibility in reverberation

Effects of listening environments, listeners and preprocessing parameters on steady-state suppression for improving speech intelligibility in reverberation

The effect of a preprocessing approach improving speech intelligibility in reverberation considering a public-address system and room acoustics

This study evaluates a preprocessing approach for reducing reverberation effects when the input signal is not ideal, dry speech, but rather a realistic speech signal picked up by a close microphone in a room. And this study shows how the situation affects the input and the further approach compared to a dry signal as the input. Steady-state suppression, as described by Arai et al. [Acoust. Sci. Technol. 23, 229–232 (2002)], was used as a preprocessing approach that processes a speech signal before it is radiated from loudspeakers. A lecture was simulated in two different halls (reverberation times of 1.2 and 1.8 s) in which public address systems were installed. The simulation software CATT-Acoustic was used and impulse responses were calculated for the input to the preprocessing approach and for a listener position. Stimuli for a syllable identification test were prepared by convolving speech signals with the calculated impulse responses. Speech signals were given with and without steady-state suppression. The inclusion of natural and electroacoustical impulse responses makes the study of steady-state suppression more realistic and tests its robustness. [Work supported by JSPS (176911).]

Improving syllable identification by a preprocessing method reducing overlap-masking in reverberant environments

Overlap-masking degrades speech intelligibility in reverberation [R. H. Bolt and A. D. MacDonald, J. Acoust. Soc. Am. 21(6), 577-580 (1949)]. To reduce the effect of this degradation, steady-state suppression has been proposed as a preprocessing technique [Arai et al., Proc. Autumn Meet. Acoust. Soc. Jpn., 2001; Acoust. Sci. Tech. 23(8), 229-232 (2002)]. This technique automatically suppresses steady-state portions of speech that have more energy but are less crucial for speech perception. The present paper explores the effect of steady-state suppression on syllable identification preceded by /a/ under various reverberant conditions. In each of two perception experiments, stimuli were presented to 22 subjects with normal hearing. The stimuli consisted of mono-syllables in a carrier phrase with and without steady-state suppression and were presented under different reverberant conditions using artificial impulse responses. The results indicate that steady-state suppression statistically improves consonant identification for reverberation times of 0.7 to 1.2 s. Analysis of confusion matrices shows that identification of voiced consonants, stop and nasal consonants, and bilabial, alveolar, and velar consonants were especially improved by steady-state suppression. The steady-state suppression is demonstrated to be an effective preprocessing method for improving syllable identification by reducing the effect of overlap-masking under specific reverberant conditions.

Modulation enhancement of speech by a pre-processing algorithm for improving intelligibility in reverberant environments

Most listeners have difficulty understanding speech in reverberant conditions. The purpose of this study is to investigate whether it is possible to reduce the degree of degradation of speech intelligibility in reverberation through the development of an algorithm. The modulation spectrum is the spectral representation of the temporal envelope of the speech signal. That of clean speech is dominated by components between 1 and 16 Hz centered at 4 Hz which is the most important range for human perception of speech. In reverberant conditions, the modulation spectrum of speech is shifted toward the lower end of the modulation frequency range. In this study, we proposed to enhance the important modulation spectral components prior to distortion of speech by reverberation. Word intelligibility in a carrier sentence was tested with the newly developed algorithm including two different filter designs in three reverberant conditions. The reverberant speech was simulated by convoluting clean speech with impulse responses measured in the actual halls. The experimental results show that modulation filtering incorporated into a pre-processing algorithm improves intelligibility for normal hearing listeners when (1) the modulation filters are optimal for a specific reverberant condition (i.e., T 60 = 1.1 s), and (2) consonants are preceded by highly powered segments. Under shorter (0.7 s) and longer (1.6 s) reverberation times, the modulation filtering in the current experiments, an Empirically-Designed (E-D) filter and a Data-Derived (D-D) filter, caused a slight performance decrement respectively. The results of this study suggest that further gains in intelligibility may be accomplished by re-design of the modulation filters suitable for other reverberant conditions.

Binaural and monaural speech intelligibility in reverberation.

Forty-eight subjects with normal hearing were tested to determine binaural and monaural near- and far-ear speech discrimination in various reverberant conditions. One half of the group listened to the recorded PB words in quiet, while the other half received the words with a competing message. Binaural superiority was maintained throughout the study, except in the anechoic-quiet condition. Interaural time differences make a significant contribution to speech intelligibility; and the monaural far ear supplies valuable aid to binaural listening.