Target Speech Research Articles

Distinct roles of SNR, speech Intelligibility, and attentional effort on neural speech tracking in noise.

Robust neural encoding of speech in noise is influenced by several factors, including signal-to-noise ratio (SNR), speech intelligibility (SI), and attentional effort (AE). Yet, the interaction and distinct role of these factors remain unclear. In this study, fourteen native English speakers performed selective speech listening tasks at various SNR levels while EEG responses were recorded. Attentional performance was assessed using a repeated word detection task, and attentional effort was inferred from subjects' gaze velocity. Results indicate that both SNR and SI enhance neural tracking of target speech, with distinct effects influenced by the previously overlooked role of attentional effort. Specifically, at high levels of SI, increasing SNR leads to reduced attentional effort, which in turn decreases neural speech tracking. Our findings highlight the importance of differentiating the roles of SNR, SI, and AE in neural speech processing and advance our understanding of how noisy speech is processed in the auditory pathway.

Acoustic beamforming using maximum likelihood criteria with Laplacian model of the desired signal

Acoustic beamforming is a technique used in audio engineering and signal processing to filter sound waves in the spatial domain. Usually, an array of microphones is used to capture sound from different directions. These array signals are then combined to reinforce each other in the desired direction while cancelling the noise or interference from other directions. An important application of acoustic beamforming is the cocktail party scenario where multiple people speak simultaneously in a noisy room. To capture the speech of only the desired speaker, acoustic beamforming is used. Usually, in such cases, the target speech is modelled as a zero-mean complex Gaussian distributed random variable. However, the target speech coefficients are sparse in the time-frequency domain. Hence, in our work, we model the speech coefficients using a zero-mean circular Laplacian distribution. After modelling the target speech, we formulate a beamformer based on the maximum likelihood criteria. We add a distortionless constraint to the proposed beamformer to further improve performance. The final solution of the proposed beamformer encourages sparsity, indicating that it models the target speech better than the complex Gaussian distribution. Simulations show the effectiveness of the proposed beamformer in capturing the target speech and rejecting interfering speakers.

Impact of contextual and lip-sync-related visual cues on speech intelligibility through immersive audio-visual scene recordings in a reverberant conference room

Recent hearing research has benefitted from the latest Virtual Reality systems that allowed the reproduction of immersive Audio-Visual scenarios to achieve more ecological listening tests. Indeed, efforts have been spent to identify the aspects that convey actual ecological validity, particularly investigating the effects of visual cues and self-motion on Speech Intelligibility through tests mainly based on simulated scenes. However, work must still be addressed when sceneries developed through real recordings inside reverberant environments are concerned. This study used 3rd-order ambisonics recordings and stereoscopic 360° videos inside a reverberant conference hall to create three virtual audio-visual scenes where speech intelligibility tests were performed, introducing informational noise from different angles. A 16-speaker spherical array synced with a headmounted display was used to administer the immersive tests to 50 normal-hearing subjects. Firstly, tests only composed of the auditory scenes were compared, based on the achieved scores, with tests also providing contextual and positional source-related visual cues, both with and without self-motion, for a total of four different test configurations. Then, to complete the investigation of the visual cues' impact on speech intelligibility, ten normal-hearing subjects were recruited to performaudio-visual tests incorporating lip-sync-related visual cues for the target speech.

Deep-learning-based speech enhancement under rough-focus conditions with optical laser microphone

Optical laser microphones have attracted attention for acoustic systems capable of recording the target speech from a distance. An optical laser microphone measures the speech-induced vibration by focusing the laser beam on the surface of the vibrating object. A recording method called rough-focus recording using an unfocused laser beam, enables wide-range recording and robust recording against changes in focal length. However, with rough-focus recording, the broad laser beam coverage leads to insufficient intensity of the reflected laser for accurate acoustical signal measurement, resulting in speech-quality degradation, such as the inclusion of noise, and attenuation of high-frequency components in the acquired speech. To solve this problem, deep-learning-based speech-enhancement methods for optical laser microphones have been proposed. Such methods require separate models for different focus settings, exhibiting a lack of adaptability to changing focus settings. We propose a speech-enhancement method for training a single model for various focus settings. This model is trained with speech signals recorded across different focus settings to enhance speech recorded in various focus settings. Experimental results indicate that the this model trained with the proposed method performs equivalent to or better than a model trained with the conventional models.

Neural encoding for spatial release from informational masking and its correlation with behavioral metrics.

The central auditory system encompasses two primary functions: identification and localization. Spatial release from masking (SRM) highlights speech recognition in competing noise and improves the listening experience when a spatial cue is introduced between noise and target speech. This assessment focuses on the integrity of auditory function and holds clinical significance. However, infants or pre-lingual subjects sometimes provide less reliable results. This study investigates the value of cortical auditory evoked potentials (CAEPs) onset and acoustic change complex (ACC) as an objective measurement of SRM. Thirty normal-hearing young adults (11 males) were recruited. We found the spatial separation of signals and noise (±90° symmetrically) resulted in a signal-to-noise ratio (SNR) improvement of 9.00 ± 1.71 dB behaviorally. It significantly enhanced cortical processing at all SNR levels, shortened CAEP latencies, and increased amplitudes, resulting in a greater number of measurable peaks for ACC. SRM showed mild to moderate correlations with the differences between two conditions in CAEP measures. The regression model combining N1'-P2' amplitude at 5 dB SNR (R2 = 0.26), P1 amplitude at 0 dB SNR (R2 = 0.14), and P1 latency at -5 dB SNR (R2 = 0.15), explained 45.3% of the variance in SRM. Our study demonstrates that introducing spatial cues can improve speech perception and enhance central auditory processing in normal-hearing young adults. CAEPs may contribute to predictions about SRM and hold potential for practical application.NEW & NOTEWORTHY The neural encoding of spatial release from masking (SRM) can be observed in normal-hearing young adults. Spatial separation between target and masker improves speech perception in noise and enhances central auditory processing. The behavioral results showed mild-to-moderate correlations with electrophysiological measures, with acoustic change complex (ACC) amplitude being a better indicator than onset components. Cortical auditory evoked potentials (CAEPs) may contribute to predictions about spatial release from masking, especially when behavioral tests are less reliable.

Speech targeting and constituency representation in open-list electoral systems

This study investigates the localistic behavior of legislators elected under open-list proportional representation (OLPR), focusing on the impacts of district magnitude, intra-party competition, electorate size, and the presence of a national tier. We examine the Ecuadorian case, where institutional reformers implemented a national tier to offset the parochial tendencies of lower-tier provincial legislators yet retained OLPR for both tiers. Our study, which analyzes a 12-year dataset of congressional speeches, challenges the expectation that national-tier members are less localistic than their provincial counterparts and shows that electoral incentives drive legislators’ geographical focus. Contrary to conventional expectations, we find no evidence that increased intra-party competition is associated with more localistic behavior. However, there is consistent support for the hypothesis that smaller electoral constituencies amplify localistic behavior.

Illusory interparty disagreement: Partisans agree on what hate speech to censor but do not know it

Whether and when to censor hate speech are long-standing points of contention in the US. The latest iteration of these debates entails grappling with content regulation on social media in an age of intense partisan polarization. But do partisans disagree about what types of hate speech to censor on social media or do they merely differ on how much hate speech to censor? And do they understand out-party censorship preferences? We examine these questions in a nationally representative conjoint survey experiment (participant N = 3,357; decision N = 40,284). We find that, although Democrats support more censorship than Republicans, partisans generally agree on what types of hate speech are most deserving of censorship in terms of the speech's target, source, and severity. Despite this substantial cross-party agreement, partisans mistakenly believe that members of the other party prioritize protecting different targets of hate speech. For example, a major disconnect between the two parties is that Democrats overestimate and Republicans underestimate the other party's willingness to censor speech targeting Whites. We conclude that partisan differences on censoring hate speech are largely based on free speech values and misperceptions rather than identity-based social divisions.

Estimation of the Cognitive Functioning of the Elderly by AI Agents: A Comparative Analysis of the Effects of the Psychological Burden of Intervention.

In recent years, an increasing number of studies have begun to use conversational data in spontaneous speech to estimate cognitive function in older people. The targets of spontaneous speech with older people used to be physicians and licensed psychologists, but it is now possible to have conversations with fully automatic AI agents. However, it has not yet been clarified what difference there is in conversational communication with older people when the examiner is a human or an AI agent. This study explored the psychological burden experienced by elderly participants during cognitive function assessments, comparing interactions with human and AI conversational partners. Thirty-four participants, averaging 78.71 years of age, were evaluated using the Mini-Mental State Examination (MMSE), the Visual Analogue Scale (VAS), and the State-Trait Anxiety Inventory (STAI). The objective was to assess the psychological impact of different conversational formats on the participants. The results indicated that the mental strain, as measured by VAS and STAI scores, was significantly higher during the MMSE sessions compared to other conversational interactions (p < 0.01). Notably, there was no significant difference in the mental burden between conversations with humans and AI agents, suggesting that AI-based systems could be as effective as human interaction in cognitive assessments.

A lightweight speech enhancement network fusing bone- and air-conducted speech.

Air-conducted (AC) microphones capture the high-quality desired speech and ambient noise, whereas bone-conducted (BC) microphones are immune to ambient noise but only capture band limited speech. This paper proposes a speech enhancement model that leverages the merits of BC and AC speech. The proposed model takes the spectrogram of BC and AC speech as input and fuses them by an attention-based feature fusion module. The backbone network of the proposed model uses the fused signals to estimate mask of the target speech, which is then applied to the noisy AC speech to recover the target speech. The proposed model adopts a lightweight design of densely gated convolutional attention network (DenGCAN) as the backbone network, which contains encoder, bottleneck layers, and decoder. Furthermore, this paper improves an attention gate and integrates it into skip-connections of DenGCAN, which allows the decoder to focus on the key areas of the feature map extracted by the encoder. As the DenGCAN adopts self-attention mechanism, the proposed model has the potential to improve noise reduction performance at the expense of an increased input-output latency. Experimental results demonstrate that the enhanced speech of the proposed model achieves an average 1.870 wideband-PESQ improvement over the noisy AC speech.

Air Traffic Control Speech Enhancement Method Based on Improved DNN-IRM

The quality of air traffic control speech is crucial. However, internal and external noise can impact air traffic control speech quality. Clear speech instructions and feedback help optimize flight processes and responses to emergencies. The traditional speech enhancement method based on a deep neural network and ideal ratio mask (DNN-IRM) is prone to distortion of the target speech in a strong noise environment. This paper introduces an air traffic control speech enhancement method based on an improved DNN-IRM. It employs LeakyReLU as an activation function to alleviate the gradient vanishing problem, improves the DNN network structure to enhance the IRM estimation capability, and adjusts the IRM weights to reduce noise interference in the target speech. The experimental results show that, compared with other methods, this method improves the perceptual evaluation of speech quality (PESQ), short-term objective intelligibility (STOI), scale-invariant signal-to-noise ratio (SI-SNR), and speech spectrogram clarity. In addition, we use this method to enhance real air traffic control speech, and the speech quality is also improved.

Triple-0: Zero-shot denoising and dereverberation on an end-to-end frozen anechoic speech separation network.

Speech enhancement is crucial both for human and machine listening applications. Over the last decade, the use of deep learning for speech enhancement has resulted in tremendous improvement over the classical signal processing and machine learning methods. However, training a deep neural network is not only time-consuming; it also requires extensive computational resources and a large training dataset. Transfer learning, i.e. using a pretrained network for a new task, comes to the rescue by reducing the amount of training time, computational resources, and the required dataset, but the network still needs to be fine-tuned for the new task. This paper presents a novel method of speech denoising and dereverberation (SD&D) on an end-to-end frozen binaural anechoic speech separation network. The frozen network requires neither any architectural change nor any fine-tuning for the new task, as is usually required for transfer learning. The interaural cues of a source placed inside noisy and echoic surroundings are given as input to this pretrained network to extract the target speech from noise and reverberation. Although the pretrained model used in this paper has never seen noisy reverberant conditions during its training, it performs satisfactorily for zero-shot testing (ZST) under these conditions. It is because the pretrained model used here has been trained on the direct-path interaural cues of an active source and so it can recognize them even in the presence of echoes and noise. ZST on the same dataset on which the pretrained network was trained (homo-corpus) for the unseen class of interference, has shown considerable improvement over the weighted prediction error (WPE) algorithm in terms of four objective speech quality and intelligibility metrics. Also, the proposed model offers similar performance provided by a deep learning SD&D algorithm for this dataset under varying conditions of noise and reverberations. Similarly, ZST on a different dataset has provided an improvement in intelligibility and almost equivalent quality as provided by the WPE algorithm.

The Role of Gender in Hate Speech Targeting Politicians: Evidence from Finnish Twitter

The Role of Gender in Hate Speech Targeting Politicians: Evidence from Finnish Twitter

Gradient speech change during intervention for school-aged children and adults with cleft palate ± lip

ABSTRACT Gradient speech change, where speech sound production develops in a broadly step-wise fashion towards the standard adult form, is a well-recognised phenomenon in children developing typical speech, but is much less studied in speakers with developmental speech sound disorders. Instrumental techniques, such as electropalatography (EPG), may be useful for identifying gradient speech change and may supplement phonetic transcription in important ways. This study investigated whether gradient speech change occurred in six participants with cleft palate ± lip undergoing intervention within a usage-based phonology framework (2/6 participants with speech distortions; 4/6 with pattern-based speech substitutions; combined total of 25 speech sounds targeted for intervention). Participants received weekly therapy in a hospital setting and were aged 10–27 years. Gradient speech change with target speech sounds was examined using EPG analysis, which was undertaken after every fifth session of therapy. The presence of gradient change was determined by visually examining EPG palatograms and EPG indices for target speech sounds across successive EPG test points. This study found gradient speech change occurred in 22/25 target sounds over the course of intervention. This gradient change occurred for both speech distortions and pattern-based speech substitutions. The remaining 3/25 target sounds showed categorical change. Usage-based phonology was suggested as a theory with potential for explaining gradient speech change, with both typical and atypical speech, and with speech distortions and pattern-based speech substitutions. This finding adds to other research showing that the objective data provided by instrumental techniques, such as EPG, may be a valuable complement to phonetic transcription.

X-TF-GridNet: A time–frequency domain target speaker extraction network with adaptive speaker embedding fusion

Target speaker extraction (TSE) which has the capability to directly extract desired speech given enrollment utterances of the target speaker has attracted more and more attention for its potential applications in solving the cocktail-party problem. Despite the considerable progress made by existing time-domain methods, which have become the dominant approach for TSE, these methods often significantly degrade their performance under more realistic conditions. This paper proposes an innovative approach in the time–frequency (T–F) domain, namely X-TF-GridNet, which uses complex spectrum mapping to extract the real and imaginary (RI) components of the target speech. Specifically, the TF-GridNet block was designed to serve as the primary speaker extractor module. Our proposed method boasts two key extensions: first, a U2-Net style network adeptly extracts robust fixed speaker embeddings, which could efficiently capture and represent target speaker information. Second, an adaptive embedding fusion (AEA) mechanism ensures the effective utilization of target speaker information, which makes the backbone extractor focus on the speech of interest. Additionally, we also introduced a multi-task learning framework, comprising two distinct loss functions, to explicitly enhance both the discriminative speaker embeddings for the reference speech and the overall quality of the target speech. We conducted extensive ablation studies and quantitative comparisons against previous TSE methods on both the WSJ0-2mix and its noisy and reverberant counterparts. Our proposed method achieved a commendable SI-SDR of 19.7 dB with a moderate model size on the WSJ0-2mix dataset, and the SI-SDR can be improved to 20.7 dB with a larger model. Experimental results demonstrated that, compared with existing time-domain approaches, our proposed method not only achieved competitive performance across multiple objective metrics but also mitigated speaker confusion errors under more challenging conditions, including various interferences such as noises and reverberation.

Digital Echoes of Discrimination: Investigating the Reception of Hate Speech Against Korean Chinese in Online Communities

abstract: This study aims to investigate the circulation, consumption, and learning of anonymous users’ hatred in online communities, focusing on the reception process of hate speech targeting Korean Chinese people, a group long discriminated against by native Koreans and the media. It identifies critical factors shaping attitudes toward Korean Chinese (i.e., Chaoxianzu), defining hate as long-term sentiments resulting from repeated immediate emotions triggered by specific incidents and social identities. This study adopts a grounded theory approach, employing interviews to construct a paradigm model that captures the authentic perspectives of online community users. Following the coding process, 17 categories and 38 subcategories emerged. Online audiences consolidate or maintain their attitudes through admission, resistance, exploration, and reproduction after exposure to hate expressions toward Korean Chinese. Self-intervention, personal experience, quality of verbal stimulation, and patriotism function as interventional conditions or environmental contexts throughout this process.

Gated Cross-Attention for Universal Speaker Extraction: Toward Real-World Applications

Current target-speaker extraction (TSE) models have achieved good performance in separating target speech from highly overlapped multi-talker speech. However, in real-world applications, multi-talker speech is often sparsely overlapped, and the target speaker may be absent from the speech mixture, making it difficult for the model to extract the desired speech in such situations. To optimize models for various scenarios, universal speaker extraction has been proposed. However, current models do not distinguish between the presence or absence of the target speaker, resulting in suboptimal performance. In this paper, we propose a gated cross-attention network for universal speaker extraction. In our model, the cross-attention mechanism learns the correlation between the target speaker and the speech to determine whether the target speaker is present. Based on this correlation, the gate mechanism enables the model to focus on extracting speech when the target is present and filter out features when the target is absent. Additionally, we propose a joint loss function to evaluate both the reconstructed target speech and silence. Experiments on the WSJ0-2mix-extr and LibriMix datasets show that our proposed method achieves superior performance over comparison approaches in terms of SI-SDR and WER.

Speech emotion recognition in real static and dynamic human-robot interaction scenarios

The use of speech-based solutions is an appealing alternative to communicate in human-robot interaction (HRI). An important challenge in this area is processing distant speech which is often noisy, and affected by reverberation and time-varying acoustic channels. It is important to investigate effective speech solutions, especially in dynamic environments where the robots and the users move, changing the distance and orientation between a speaker and the microphone. This paper addresses this problem in the context of speech emotion recognition (SER), which is an important task to understand the intention of the message and the underlying mental state of the user. We propose a novel setup with a PR2 robot that moves as target speech and ambient noise are simultaneously recorded. Our study not only analyzes the detrimental effect of distance speech in this dynamic robot-user setting for speech emotion recognition but also provides solutions to attenuate its effect. We evaluate the use of two beamforming schemes to spatially filter the speech signal using either delay-and-sum (D&S) or minimum variance distortionless response (MVDR). We consider the original training speech recorded in controlled situations, and simulated conditions where the training utterances are processed to simulate the target acoustic environment. We consider the case where the robot is moving (dynamic case) and not moving (static case). For speech emotion recognition, we explore two state-of-the-art classifiers using hand-crafted features implemented with the ladder network strategy and learned features implemented with the wav2vec 2.0 feature representation. MVDR led to a signal-to-noise ratio higher than the basic D&S method. However, both approaches provided very similar average concordance correlation coefficient (CCC) improvements equal to 116 % with the HRI subsets using the ladder network trained with the original MSP-Podcast training utterances. For the wav2vec 2.0-based model, only D&S led to improvements. Surprisingly, the static and dynamic HRI testing subsets resulted in a similar average concordance correlation coefficient. Finally, simulating the acoustic environment in the training dataset provided the highest average concordance correlation coefficient scores with the HRI subsets that are just 29 % and 22 % lower than those obtained with the original training/testing utterances, with ladder network and wav2vec 2.0, respectively.

CHUNAV: Analyzing Hindi Hate Speech and Targeted Groups in Indian Election Discourse

In the ever-evolving landscape of online discourse and political dialogue, the rise of hate speech poses a significant challenge to maintaining a respectful and inclusive digital environment. The context becomes particularly complex when considering the Hindi language—a low-resource language with limited available data. To address this pressing concern, we introduce the CHUNAV dataset—a collection of 11,457 Hindi tweets gathered during assembly elections in various states. CHUNAV is purpose-built for hate speech categorization and the identification of target groups. The dataset is a valuable resource for exploring hate speech within the distinctive socio-political context of Indian elections. The tweets within CHUNAV have been meticulously categorized into “Hate” and “Non-Hate” labels, and further subdivided to pinpoint the specific targets of hate speech, including “Individual”, “Organization”, and “Community” labels (as shown in Figure 1). Furthermore, this paper presents multiple benchmark models for hate speech detection, along with an innovative ensemble and oversampling-based method. The paper also delves into the results of topic modeling, all aimed at effectively addressing hate speech and target identification in the Hindi language. This contribution seeks to advance the field of hate speech analysis and foster a safer and more inclusive online space within the distinctive realm of Indian Assembly Elections.

The role of long-term target and masker talker familiarity in children's speech-in-speech recognition.

This study investigated the influence of long-term talker familiarity on speech-in-speech recognition in school-age children, with a specific emphasis on the role of familiarity with the mother's voice as either the target or masker speech. Open-set sentence recognition was measured adaptively in a two-talker masker. Target and masker sentences were recorded by the adult mothers of the child participants. Each child heard sentences spoken by three adult female voices during testing; their own mother's voice (familiar voice) and two unfamiliar adult female voices. Twenty-four school age children (8-13 years) with normal hearing. When the target speech was spoken by a familiar talker (the mother), speech recognition was significantly better compared to when the target was unfamiliar. When the masker was spoken by the familiar talker, there was no difference in performance relative to the unfamiliar masker condition. Across all conditions, younger children required a more favorable signal-to-noise ratio than older children. Implicit long-term familiarity with a talker consistently improves children's speech-in-speech recognition across the age range tested, specifically when the target talker is familiar. However, performance remains unaffected by masker talker familiarity. Additionally, while target familiarity is advantageous, it does not entirely eliminate children's increased susceptibility to competing speech.

Hate Speech Against Asian American Youth: Pre-Pandemic Trends and The Role of School Factors

Although hate speech against Asian American youth has intensified in recent years—fueled, in part, by anti-Asian rhetoric associated with the COVID-19 pandemic—the phenomenon remains largely understudied at scale and in relation to the role of schools prior to the pandemic. This study describes the prevalence of hate speech against Asian American adolescents in the US between 2015 and 2019 and investigates how school-related factors are associated with whether Asian American youth are victims of hate speech at school. Analyses are based on a sample of 938 Asian American adolescents (Mage = 14.8; 48% female) from the three most recently available waves (2015, 2017, and 2019) of the School Crime Supplement to the National Crime Victimization Survey. On average, approximately 7% of Asian Americans were targets of hate speech at school between 2015 and 2019, with rates remaining stable over time. Findings also indicate that students had lower odds of experiencing hate speech if they attended schools with a stronger authoritative school climate, which is characterized by strict, yet fair disciplinary rules coupled with high levels of support from adults. On the other hand, Asian American youth faced higher odds of experiencing hate speech if they were involved in school fights. Authoritative school climate and exposure to fights are malleable and can be shaped directly by broader school climate related policies, programs and interventions. Accordingly, efforts to promote stronger authoritative climates and reduce exposure to physical fights hold considerable potential in protecting Asian American youth from hate speech at school.