Target Speech Research Articles

Design and evaluation of personal audio systems based on speech privacy constraints.

Personal audio refers to the generation of spatially distinct sound zones that allow individuals within a shared space to listen to their own audio material without affecting, or being affected, by others. Recent interest in such systems has focussed on their performance in public spaces where speech privacy is desirable. To achieve this goal, speech is focussed towards the target listener and a masking signal is focussed into the area where the target speech signal could otherwise be overheard. An effective masking signal must substantially reduce the intelligibility in this region without becoming an annoyance to those nearby. To assess these perceptual requirements, listening tests were carried out using two examples of loudspeaker arrays with different spatial aliasing characteristics, to determine the impacts of different masking signal spectra on speech intelligibility and subjective preference. The results of these tests were used, alongside objective and subjective metrics, to form a design specification for private personal audio systems.

The Effect of Musical Training and Working Memory in Adverse Listening Situations.

Speech-in-noise (SIN) perception is essential for everyday communication. In most communication situations, the listener requires the ability to process simultaneous complex auditory signals to understand the target speech or target sound. As the listening situation becomes more difficult, the ability to distinguish between speech and noise becomes dependent on recruiting additional cognitive resources, such as working memory (WM). Previous studies have explored correlations between WM and SIN perception in musicians and nonmusicians, with mixed findings. However, no study to date has examined the speech perception abilities of musicians and nonmusicians with similar WM capacity. The objectives of this study were to investigate (1) whether musical experience results in improved listening in adverse listening situations, and (2) whether the benefit of musical experience can be separated from the effect of greater WM capacity. Forty-nine young musicians and nonmusicians were assigned to subgroups of high versus low WM, based on the performance on the backward digit span test. To investigate the effects of music training and WM on SIN perception, performance was assessed on clinical tests of speech perception in background noise. Listening effort (LE) was assessed in a dual-task paradigm and via self-report. We hypothesized that musicians would have an advantage when listening to SIN, at least in terms of reduced LE. There was no statistically significant difference between musicians and nonmusicians, and no significant interaction between music training and WM on any of the outcome measures used in this study. However, a significant effect of WM on SIN ability was found on both the Quick Speech-In-Noise test (QuickSIN) and the Hearing in Noise Test (HINT) tests. The results of this experiment suggest that music training does not provide an advantage in adverse listening situations either in terms of improved speech understanding or reduced LE. While musicians have been shown to have heightened basic auditory abilities, the effect on SIN performance may be more subtle. Our results also show that regardless of prior music training, listeners with high WM capacity are able to perform significantly better on speech-in-noise tasks.

Informational masking of speech by acoustically similar intelligible and unintelligible interferers.

Masking experienced when target speech is accompanied by a single interfering voice is often primarily informational masking (IM). IM is generally greater when the interferer is intelligible than when it is not (e.g., speech from an unfamiliar language), but the relative contributions of acoustic-phonetic and linguistic interference are often difficult to assess owing to acoustic differences between interferers (e.g., different talkers). Three-formant analogues (F1+F2+F3) of natural sentences were used as targets and interferers. Targets were presented monaurally either alone or accompanied contralaterally by interferers from another sentence (F0 = 4 semitones higher); a target-to-masker ratio (TMR) between ears of 0, 6, or 12 dB was used. Interferers were either intelligible or rendered unintelligible by delaying F2 and advancing F3 by 150 ms relative to F1, a manipulation designed to minimize spectro-temporal differences between corresponding interferers. Target-sentence intelligibility (keywords correct) was 67% when presented alone, but fell considerably when an unintelligible interferer was present (49%) and significantly further when the interferer was intelligible (41%). Changes in TMR produced neither a significant main effect nor an interaction with interferer type. Interference with acoustic-phonetic processing of the target can explain much of the impact on intelligibility, but linguistic factors-particularly interferer intrusions-also make an important contribution to IM.

Spatial Hearing as a Function of Presentation Level in Moderate-to-Severe Unilateral Conductive Hearing Loss.

Patients with moderate-to-severe unilateral conductive hearing loss (UCHL) can make use of binaural difference cues when stimuli are presented at a high enough intensity to provide audibility in the affected ear. Spatial hearing is essential for listening in complex environments and sound source localization. Patients with UCHL have decreased access to binaural difference cues, resulting in poorer spatial hearing abilities compared with listeners with normal hearing. Twelve patients with moderate-to-severe UCHL, most due to atresia (83.3%), and 12 age-matched controls with normal hearing bilaterally participated in this study. Outcome measures included: 1) spatial release from masking, and 2) sound source localization. Speech reception thresholds were measured with target speech (Pediatric AzBio sentences) presented at 0 degree and a two-talker masker that was either colocated with the target (0 degree) or spatially separated from the target (symmetrical, ±90 degrees). Spatial release from masking was quantified as the difference between speech reception thresholds in these two conditions. Localization ability in the horizontal plane was assessed in a 180 degree arc of 11 evenly-spaced loudspeakers. These two tasks were completed at 50 and 75 dB SPL. Both children and adults with UCHL performed more poorly than controls when recognizing speech in a spatially separated masker or localizing sound; however, this group difference was larger at 50 than 75 dB SPL. Patients with UCHL experience improved spatial hearing with the higher presentation level, suggesting that the auditory deprivation associated with a moderate-to-severe UCHL does not preclude exposure to-or use of-binaural difference cues.

Measuring access to high-modulation-rate envelope speech cues in clinically fitted auditory prostheses.

The signal processing used to increase intelligibility within the hearing-impaired listener introduces distortions in the modulation patterns of a signal. Trade-offs have to be made between improved audibility and the loss of fidelity. Acoustic hearing impairment can cause reduced access to temporal fine structure (TFS), while cochlear implant processing, used to treat profound hearing impairment, has reduced ability to convey TFS, hence forcing greater reliance on modulation cues. Target speech mixed with a competing talker was split into 8-22 frequency channels. From each channel, separate low-rate (EmodL, <16 Hz) and high-rate (EmodH, <300 Hz) versions of the envelope modulation were extracted, which resulted in low or high intelligibility, respectively. The EModL modulations were preserved in channel valleys and cross-faded to EModH in channel peaks. The cross-faded signal modulated a tone carrier in each channel. The modulated carriers were summed across channels and presented to hearing aid (HA) and cochlear implant users. Their ability to access high-rate modulation cues and the dynamic range of this access was assessed. Clinically fitted hearing aids resulted in 10% lower intelligibility than simulated high-quality aids. Encouragingly, cochlear implantees were able to extract high-rate information over a dynamic range similar to that for the HA users.

Dual-Channel Speech Enhancement Based On Speech Presence Probability

This paper introduces technology to improve sound quality, which serves the needs of media and entertainment. Major challenging problem in the speech processing applications like mobile phones, hands-free phones, car communication, teleconference systems, hearing aids, voice coders, automatic speech recognition and forensics etc., is to eliminate the background noise. Speech enhancement algorithms are widely used for these applications in order to remove the noise from degraded speech in the noisy environment. Hence, the conventional noise reduction methods introduce more residual noise and speech distortion. So, it has been found that the noise reduction process is more effective to improve the speech quality but it affects the intelligibility of the clean speech signal. In this paper, we introduce a new model of coherence-based noise reduction method for the complex noise environment in which a target speech coexists with a coherent noise around. From the coherence model, the information of speech presence probability is added to better track noise variation accurately; and during the speech presence and speech absent period, adaptive coherence-based method is adjusted. The performance of suggested method is evaluated in condition of diffuse and real street noise, and it improves the speech signal quality less speech distortion and residual noise.

Autonomic Nervous System Correlates of Speech Categorization Revealed Through Pupillometry.

Human perception requires the many-to-one mapping between continuous sensory elements and discrete categorical representations. This grouping operation underlies the phenomenon of categorical perception (CP)—the experience of perceiving discrete categories rather than gradual variations in signal input. Speech perception requires CP because acoustic cues do not share constant relations with perceptual-phonetic representations. Beyond facilitating perception of unmasked speech, we reasoned CP might also aid the extraction of target speech percepts from interfering sound sources (i.e., noise) by generating additional perceptual constancy and reducing listening effort. Specifically, we investigated how noise interference impacts cognitive load and perceptual identification of unambiguous (i.e., categorical) vs. ambiguous stimuli. Listeners classified a speech vowel continuum (/u/-/a/) at various signal-to-noise ratios (SNRs [unmasked, 0 and −5 dB]). Continuous recordings of pupil dilation measured processing effort, with larger, later dilations reflecting increased listening demand. Critical comparisons were between time-locked changes in eye data in response to unambiguous (i.e., continuum endpoints) tokens vs. ambiguous tokens (i.e., continuum midpoint). Unmasked speech elicited faster responses and sharper psychometric functions, which steadily declined in noise. Noise increased pupil dilation across stimulus conditions, but not straightforwardly. Noise-masked speech modulated peak pupil size (i.e., [0 and −5 dB] > unmasked). In contrast, peak dilation latency varied with both token and SNR. Interestingly, categorical tokens elicited earlier pupil dilation relative to ambiguous tokens. Our pupillary data suggest CP reconstructs auditory percepts under challenging listening conditions through interactions between stimulus salience and listeners’ internalized effort and/or arousal.

Targeted Speech Adversarial Example Generation With Generative Adversarial Network

Although neural network-based speech recognition models have enjoyed significant success in many acoustic systems, they are susceptible to be attacked by the adversarial examples. In this work, we make first step towards using generative adversarial network (GAN) for constructing the targeted speech adversarial examples. Specifically, we integrate the target speech recognition network with GAN framework, which can then be formulated as a three-party game. The generator in GAN aims at generating perturbation that could make the target network misclassified to a specific target, while simultaneously fooling the discriminator treating the adversarial example as a beguine one. The discriminator is to distinguish the crafted examples from the geniue samples. The classification error of the target network is back-propagated via gradient flow to the generator for updating. The target network is responsible for back-propagating the classification error via gradients to the generator for updating, but the target network itself is freezed. With the carefully designed network architecture, loss function and training strategy, we successfully train a generator that could generate the adversarial perturbation for a given speech clip and a target label. Experiential results show that the generated adversarial examples could effectively fool the state-of-the-art speech classification networks, while attaining an acceptable auditory perception quality. In addition, our proposed method runs much faster than the prevalent optimization-based schemes. To facilitate reproducible research, codes, models and data are publicly available at https://github.com/winterwindwang/SpeechAdvGan.

Masking Release for Speech-in-Speech Recognition Due to a Target/Masker Sex Mismatch in Children With Hearing Loss.

The goal of the present study was to compare the extent to which children with hearing loss and children with normal hearing benefit from mismatches in target/masker sex in the context of speech-in-speech recognition. It was hypothesized that children with hearing loss experience a smaller target/masker sex mismatch benefit relative to children with normal hearing due to impairments in peripheral encoding, variable access to high-quality auditory input, or both. Eighteen school-age children with sensorineural hearing loss (7 to 15 years) and 18 age-matched children with normal hearing participated in this study. Children with hearing loss were bilateral hearing aid users. Severity of hearing loss ranged from mild to severe across participants, but most had mild to moderate hearing loss. Speech recognition thresholds for disyllabic words presented in a two-talker speech masker were estimated in the sound field using an adaptive, forced-choice procedure with a picture-pointing response. Participants were tested in each of four conditions: (1) male target speech/two-male-talker masker; (2) male target speech/two-female-talker masker; (3) female target speech/two-female-talker masker; and (4) female target speech/two-male-talker masker. Children with hearing loss were tested wearing their personal hearing aids at user settings. Both groups of children showed a sex-mismatch benefit, requiring a more advantageous signal to noise ratio when the target and masker were matched in sex than when they were mismatched. However, the magnitude of sex-mismatch benefit was significantly reduced for children with hearing loss relative to age-matched children with normal hearing. There was no effect of child age on the magnitude of sex-mismatch benefit. The sex-mismatch benefit was larger for male target speech than for female target speech. For children with hearing loss, the magnitude of sex-mismatch benefit was not associated with degree of hearing loss or aided audibility. The findings from the present study indicate that children with sensorineural hearing loss are able to capitalize on acoustic differences between speech produced by male and female talkers when asked to recognize target words in a competing speech masker. However, children with hearing loss experienced a smaller benefit relative to their peers with normal hearing. No association between the sex-mismatch benefit and measures of unaided thresholds or aided audibility were observed for children with hearing loss, suggesting that reduced peripheral encoding is not the only factor responsible for the smaller sex-mismatch benefit relative to children with normal hearing.

Auditory Attention and Spatial Unmasking in Children With Cochlear Implants.

The ability to attend to target speech in background noise is an important skill, particularly for children who spend many hours in noisy environments. Intelligibility improves as a result of spatial or binaural unmasking in the free-field for normal-hearing children; however, children who use bilateral cochlear implants (BiCIs) demonstrate little benefit in similar situations. It was hypothesized that poor auditory attention abilities might explain the lack of unmasking observed in children with BiCIs. Target and interferer speech stimuli were presented to either or both ears of BiCI participants via their clinical processors. Speech reception thresholds remained low when the target and interferer were in opposite ears, but they did not show binaural unmasking when the interferer was presented to both ears and the target only to one ear. These results demonstrate that, in the most extreme cases of stimulus separation, children with BiCIs can ignore an interferer and attend to target speech, but there is weak or absent binaural unmasking. It appears that children with BiCIs mostly experience poor encoding of binaural cues rather than deficits in ability to selectively attend to target speech.

Spatially Correct Rate-Constrained Noise Reduction for Binaural Hearing Aids in Wireless Acoustic Sensor Networks

Compared to monaural hearing aids (HAs), binaural hearing aid systems, in which there is a communication link between the two devices, have improved noise reduction capabilities and the ability to preserve binaural spatial information. However, the limited HA battery lifetime puts constraints on the amount of information that can be shared between the two devices. In other words, the rate of transmission between the devices is an important constraint that needs to be considered, while preserving the spatial information. In this article, a linearly constrained noise reduction problem is proposed, which jointly finds the optimal rate allocation and the optimal estimation (beamforming) weights across all sensors and frequencies, while preserving the binaural spatial cues of point sources. The proposed method considers a rate constraint together with linear constraints to preserve the binaural spatial cues of point sources. Minimizing the mean square error on the estimated target speech at the left and the right side beamformers, the optimal weights are found to be rate-constrained linearly constrained minimum variance (LCMV) filters, and the optimal rates are found to be the solutions to a set of reverse water filling problems. The performance of the proposed method is evaluated using the averaged binaural signal-to-noise ratio (SNR), the interaural level difference (ILD) error and the interaural time difference (ITD) error. The results show that the proposed method outperforms spatially correct noise reduction approaches that use naive/random rate allocation strategies.

Masking Effects in the Perception of Multiple Simultaneous Talkers in Normal-Hearing and Cochlear Implant Listeners.

For normal-hearing (NH) listeners, monaural factors, such as voice pitch cues, may play an important role in the segregation of speech signals in multitalker environments. However, cochlear implant (CI) users experience difficulties in segregating speech signals in multitalker environments in part due to the coarse spectral resolution. The present study examined how the vocal characteristics of the target and masking talkers influence listeners’ ability to extract information from a target phrase in a multitalker environment. Speech recognition thresholds (SRTs) were measured with one, two, or four masker talkers for different combinations of target-masker vocal characteristics in 10 adult Mandarin-speaking NH listeners and 12 adult Mandarin-speaking CI users. The results showed that CI users performed significantly poorer than NH listeners in the presence of competing talkers. As the number of masker talkers increased, the mean SRTs significantly worsened from –22.0 dB to –5.2 dB for NH listeners but significantly improved from 5.9 dB to 2.8 dB for CI users. The results suggest that the flattened peaks and valleys with increased numbers of competing talkers may reduce NH listeners’ ability to use dips in the spectral and temporal envelopes that allow for “glimpses” of the target speech. However, the flattened temporal envelope of the resultant masker signals may be less disruptive to the amplitude contour of the target speech, which is important for Mandarin-speaking CI users’ lexical tone recognition. The amount of masking release was further estimated by comparing SRTs between the same-sex maskers and the different-sex maskers. There was a large amount of masking release in NH adults (12 dB) and a small but significant amount of masking release in CI adults (2 dB). These results suggest that adult CI users may significantly benefit from voice pitch differences between target and masker speech.

The effect of an active transcutaneous bone conduction device on spatial release from masking

Objective: The aim was to quantify the effect of the experimental active transcutaneous Bone Conduction Implant (BCI) on spatial release from masking (SRM) in subjects with bilateral or unilateral conductive and mixed hearing loss.Design: Measurements were performed in a sound booth with five loudspeakers at 0°, +/−30° and +/−150° azimuth. Target speech was presented frontally, and interfering speech from either the front (co-located) or surrounding (separated) loudspeakers. SRM was calculated as the difference between the separated and the co-located speech recognition threshold (SRT).Study Sample: Twelve patients (aged 22–76 years) unilaterally implanted with the BCI were included.Results: A positive SRM, reflecting a benefit of spatially separating interferers from target speech, existed for all subjects in unaided condition, and for nine subjects (75%) in aided condition. Aided SRM was lower compared to unaided in nine of the subjects. There was no difference in SRM between patients with bilateral and unilateral hearing loss. In aided situation, SRT improved only for patients with bilateral hearing loss.Conclusions: The BCI fitted unilaterally in patients with bilateral or unilateral conductive/mixed hearing loss seems to reduce SRM. However, data indicates that SRT is improved or maintained for patients with bilateral and unilateral hearing loss, respectively.

Mask-based blind source separation and MVDR beamforming in ASR

This paper presents a front-end enhancement system for automatic speech recognition to address the cocktail party problem. Cocktail party problem is focus on recognizing the target speech when multiple speakers talk in the noisy real-environments. Many conventional techniques have been proposed. In this work, we propose a new framework to integrate the conventional blind source separation and minimum variance distortionless response beamformer for the speech enhancement and source separation of the recent CHiME-5 challenge. In our experiments, we found that the time–frequency (T–F) mask estimation strategy based on the BSS algorithm should be different for speech enhancement and source separation. The main difference is that whether we need to account for background noise as an additional class during T–F mask estimation. Experimental results showed that the proposed framework was very beneficial to improve the speech recognition performance on the Single-array-track of CHiME-5. We obtained relative 13.5% WER reduction than the official baseline system by only improving the front-end speech enhancement framework.

Prosodic Analysis and Enhancement of Dysarthric Speech

Dysarthria is a disorder that is caused in the nervous system. It is caused by damage in some parts of the brain such as cerebellum. Because of the damage in brain it causes weakness in muscles used for speech therefore it happens as in mumbled, slurred or slow speech that human and the machine find it difficult to understand such slurred speech. The Automatic Speech Recognizers which were designed for speech intelligibility perform poorly on dysarthric speech recognition. This paper focuses on the transformation of voice for dysarthria to enhance its intelligibility Formant tracking, pitch and energy estimation with durational cues from dysarthric speech facilitate the modification of these trajectories to more closely approximate the desired intelligible target speech. The transformation of the speech is done by using formant re-synthesis, pitch change and duration morphing. The results of such transformation results indicate that the transformation of the pitch and duration step enhances the intelligibility of dysarthric speech and make it easy to understand for humans and machines.

Modulation of Speech-in-Noise Comprehension Through Transcranial Current Stimulation With the Phase-Shifted Speech Envelope.

Neural activity tracks the envelope of a speech signal at latencies from 50 ms to 300 ms. Modulating this neural tracking through transcranial alternating current stimulation influences speech comprehension. Two important variables that can affect this modulation are the latency and the phase of the stimulation with respect to the sound. While previous studies have found an influence of both variables on speech comprehension, the interaction between both has not yet been measured. We presented 17 subjects with speech in noise coupled with simultaneous transcranial alternating current stimulation. The currents were based on the envelope of the target speech but shifted by different phases, as well as by two temporal delays of 100 ms and 250 ms. We also employed various control stimulations, and assessed the signal-to-noise ratio at which the subject understood half of the speech. We found that, at both latencies, speech comprehension is modulated by the phase of the current stimulation. However, the form of the modulation differed between the two latencies. Phase and latency of neurostimulation have accordingly distinct influences on speech comprehension. The different effects at the latencies of 100 ms and 250 ms hint at distinct neural processes for speech processing.

The Dynamics of Attention Shifts Among Concurrent Speech in a Naturalistic Multi-speaker Virtual Environment.

Focusing attention on one speaker on the background of other irrelevant speech can be a challenging feat. A longstanding question in attention research is whether and how frequently individuals shift their attention towards task-irrelevant speech, arguably leading to occasional detection of words in a so-called unattended message. However, this has been difficult to gauge empirically, particularly when participants attend to continuous natural speech, due to the lack of appropriate metrics for detecting shifts in internal attention. Here we introduce a new experimental platform for studying the dynamic deployment of attention among concurrent speakers, utilizing a unique combination of Virtual Reality (VR) and Eye-Tracking technology. We created a Virtual Café in which participants sit across from and attend to the narrative of a target speaker. We manipulated the number and location of distractor speakers by placing additional characters throughout the Virtual Café. By monitoring participant’s eye-gaze dynamics, we studied the patterns of overt attention-shifts among concurrent speakers as well as the consequences of these shifts on speech comprehension. Our results reveal important individual differences in the gaze-pattern displayed during selective attention to speech. While some participants stayed fixated on a target speaker throughout the entire experiment, approximately 30% of participants frequently shifted their gaze toward distractor speakers or other locations in the environment, regardless of the severity of audiovisual distraction. Critically, preforming frequent gaze-shifts negatively impacted the comprehension of target speech, and participants made more mistakes when looking away from the target speaker. We also found that gaze-shifts occurred primarily during gaps in the acoustic input, suggesting that momentary reductions in acoustic masking prompt attention-shifts between competing speakers, in line with “glimpsing” theories of processing speech in noise. These results open a new window into understanding the dynamics of attention as they wax and wane over time, and the different listening patterns employed for dealing with the influx of sensory input in multisensory environments. Moreover, the novel approach developed here for tracking the locus of momentary attention in a naturalistic virtual-reality environment holds high promise for extending the study of human behavior and cognition and bridging the gap between the laboratory and real-life.

Congruent audiovisual speech enhances auditory attention decoding with EEG

Objective. The auditory attention decoding (AAD) approach can be used to determine the identity of the attended speaker during an auditory selective attention task, by analyzing measurements of electroencephalography (EEG) data. The AAD approach has the potential to guide the design of speech enhancement algorithms in hearing aids, i.e. to identify the speech stream of listener’s interest so that hearing aids algorithms can amplify the target speech and attenuate other distracting sounds. This would consequently result in improved speech understanding and communication and reduced cognitive load, etc. The present work aimed to investigate whether additional visual input (i.e. lipreading) would enhance the AAD performance for normal-hearing listeners. Approach. In a two-talker scenario, where auditory stimuli of audiobooks narrated by two speakers were presented, multi-channel EEG signals were recorded while participants were selectively attending to one speaker and ignoring the other one. Speakers’ mouth movements were recorded during narrating for providing visual stimuli. Stimulus conditions included audio-only, visual input congruent with either (i.e. attended or unattended) speaker, and visual input incongruent with either speaker. The AAD approach was performed separately for each condition to evaluate the effect of additional visual input on AAD. Main results. Relative to the audio-only condition, the AAD performance was found improved by visual input only when it was congruent with the attended speech stream, and the improvement was about 14 percentage points on decoding accuracy. Cortical envelope tracking activities in both auditory and visual cortex were demonstrated stronger for the congruent audiovisual speech condition than other conditions. In addition, a higher AAD robustness was revealed for the congruent audiovisual condition, with reduced channel number and trial duration achieving higher accuracy than the audio-only condition. Significance. The present work complements previous studies and further manifests the feasibility of the AAD-guided design of hearing aids in daily face-to-face conversations. The present work also has a directive significance for designing a low-density EEG setup for the AAD approach.

Infants and Adults Use Visual Cues to Improve Detection and Discrimination of Speech in Noise.

Purpose This study assessed the extent to which 6- to 8.5-month-old infants and 18- to 30-year-old adults detect and discriminate auditory syllables in noise better in the presence of visual speech than in auditory-only conditions. In addition, we examined whether visual cues to the onset and offset of the auditory signal account for this benefit. Method Sixty infants and 24 adults were randomly assigned to speech detection or discrimination tasks and were tested using a modified observer-based psychoacoustic procedure. Each participant completed 1-3 conditions: auditory-only, with visual speech, and with a visual signal that only cued the onset and offset of the auditory syllable. Results Mixed linear modeling indicated that infants and adults benefited from visual speech on both tasks. Adults relied on the onset-offset cue for detection, but the same cue did not improve their discrimination. The onset-offset cue benefited infants for both detection and discrimination. Whereas the onset-offset cue improved detection similarly for infants and adults, the full visual speech signal benefited infants to a lesser extent than adults on the discrimination task. Conclusions These results suggest that infants' use of visual onset-offset cues is mature, but their ability to use more complex visual speech cues is still developing. Additional research is needed to explore differences in audiovisual enhancement (a) of speech discrimination across speech targets and (b) with increasingly complex tasks and stimuli.

Masker design for real-time informational masking with mitigated annoyance

An alternative design of masker for real-time speech masking is proposed. Time-reversed speech, which is also adopted in the proposed design, has been commonly used for speech masking because of its efficient performance in making the target speech unintelligible using informational masking. However, previous studies revealed that a time-reversed speech causes more annoyance and cognitive distraction for the listeners. Meanwhile, although generating time-reversed speech from the target speech would help improve masking effectiveness, it would require real-time processing as the target speech would not be available beforehand. The proposed masker design utilises techniques known for minimising discontinuities in the waveform of synthesised speech in order to minimise annoyance and distraction to the listeners. The design also avoids processes that hinders real-time generation of the masker.Results of subjective listening tests reveal that the proposed design is able to compromise the level of annoyance and the masking effect. Only marginal improvement is observed in the level of distraction. Further attempt to mitigate annoyance of the proposed masker by adding artificial reverberation did not help as it also reduces the masking effect significantly.