Speech Sources Research Articles

An optimized approach to speech transcription using blind source separation and speech-to-text machine learning models

The use of speech-to-text transcription has a multitude of applications in various industries, including accessibility support, language processing, and automatic subtitling. In recent years, there has been greater interest in incorporating automatic speech source separation features to improve the accuracy and efficiency of transcription mechanisms. This paper aims to design a transcription mechanism that utilizes DUET algorithm to separate speech sources in a stereo setup. The separated sources are then transcribed into text using a machine learning model. The study evaluates the effectiveness of this approach using a dataset of speech recordings. The results of the study indicate high accuracy in speech separation and transcription, highlighting the potential of this approach for practical applications. However, the study also revealed potential issues with the mechanism, indicating the need for further exploration and refinement. These findings indicate the potential of the proposed approach for practical applications, and propose insight for further development and researches in this area.

Method and practice of microphone array speech source localization based on sound propagation modeling

Abstract This paper realizes the speech source localization for microphone arrays based on the sound propagation model. According to the actual environment and location of the sound source, this paper divides the sound source into far-field source and near-field source and constructs the far-field sound model and near-field sound model applicable to the microphone array. The TDOA time-delayed localization algorithm is employed to locate the voice source of the microphone array by judging the sound far and near the field. In the localization test, this paper selects microphones to form an array according to the actual needs and preprocesses the sound signal data required for practice. The preprocessing data and sound source localization practice prove that the microphone array speech source localization algorithm used in this paper can effectively estimate the actual position of the sound source, and the absolute error between its estimated sound source position and the actual sound source position is only about 0.3m.

Emotional Language Processing in Bilingualism: Subjective Affect and Prosodic Markers in Simultaneous Interpreting

Emotion, as part of the overall sensorimotor, introspective, and affective system, is an essential part of language comprehension within the framework of embodied semantics. As emotional state influences semantic and syntactic processing, emotional language processing has been shown to modulate mood as well. The reciprocal relationship between language and emotion has also been informative in bilingualism. Here we take a relatively underresearched type of bilingual processing, simultaneous interpreting, as a case of extreme bilingualism and investigate the effect of emotional language rendering in the L1 on subjective affect and prosodic markers of L2 output. 18 trainee interpreters were asked to simultaneously interpret three speeches in Turkish that varied in emotionality, valence (negative, neutral, and positive), and difficulty in English. Responses to emotional language processing were analysed based on participants’ self-reported positive and negative affect using the Positive and Negative Affect Schedule (PANAS) and three prosodic parameters (intensity, pitch, and fluency). Results showed that interpreting emotionally negative speech increased negative affect, whereas interpreting emotionally positive speech did not modify positive affect. Intensity generally reflected cognitive load. Pitch and fluency, in particular, were more sensitive to changes in the valence of the source speech.

Voice Cloning Using Transfer Learning with Audio Samples

Voice cloning refers to the artificial replication of a certain human voice. Several deep learning approaches were studied for voice cloning. After studying learning approaches, a cloning system was offered that creates natural-sounding audio samples within few seconds of source speech from the target speaker. From a speaker verification challenge to text-to-speech synthesis with multi-speaker capability, the current study used a transfer learning technique. In a zero-shot mode, this system creates speech sounds in the voices of various speakers, even individuals who were not seen during the training process. The current study used latent embedding’s to encode speaker-specific information, enabling additional model parameters to be pooled across all speakers. The speaker modelling stage was separated from voice synthesis by training a discrete speaker-discriminative encoder network. This is because networks require distinct types of input, disconnection enables each to be trained using separate datasets. When employed for zero-shot adaptability to unknown speakers, an embedding-based technique for voice cloning enhances speaker resemblance. Furthermore, it reduces computational resource needs which may be advantageous for use-cases requiring minimal resource deployment.

Differences in speech intelligibility in noise under Ambisonics-based virtual acoustic environments with varying sound recording/rendering methods

The present study investigates the differences in speech intelligibility in noise under Ambisonics-based virtual acoustic environments realised by various recording/rendering methods. Subjective listening tests were conducted under four different implementations of Ambisonics-based virtual acoustic environments reproducing a seminar room using different modes of recording and rendering techniques. For recording, both first and higher order Ambisonics microphone arrays were utilised while for rendering headphones (binaural) and 32-ch loudspeaker array were used. The results were also compared against the data collected in the real room. The experimental results suggest the speech intelligibility under virtual acoustic environments does not differ significantly regardless of the modes of recording and rendering, however it was generally lower than that in their original room. The effect of spatial release from masking was also observed under the virtual acoustic environments when either/both the higher order Ambisonics microphone array or/and the 32-channel loudspeaker array were utilised for the implementation, however, it was limited to the case when the speech source was located close to the listener.

Any-to-One Non-Parallel Voice Conversion System Using an Autoregressive Conversion Model and LPCNet Vocoder

We present an any-to-one voice conversion (VC) system, using an autoregressive model and LPCNet vocoder, aimed at enhancing the converted speech in terms of naturalness, intelligibility, and speaker similarity. As the name implies, non-parallel any-to-one voice conversion does not require paired source and target speeches and can be employed for arbitrary speech conversion tasks. Recent advancements in neural-based vocoders, such as WaveNet, have improved the efficiency of speech synthesis. However, in practice, we find that the trajectory of some generated waveforms is not consistently smooth, leading to occasional voice errors. To address this issue, we propose to use an autoregressive (AR) conversion model along with the high-fidelity LPCNet vocoder. This combination not only solves the problems of waveform fluidity but also produces more natural and clear speech, with the added capability of real-time speech generation. To precisely represent the linguistic content of a given utterance, we use speaker-independent PPG features (SI-PPG) computed from an automatic speech recognition (ASR) model trained on a multi-speaker corpus. Next, a conversion model maps the SI-PPG to the acoustic representations used as input features for the LPCNet. The proposed autoregressive structure enables our system to produce the following prediction step outputs from the acoustic features predicted in the previous step. We evaluate the effectiveness of our system by performing any-to-one conversion pairs between native English speakers. Experimental results show that the proposed method outperforms state-of-the-art systems, producing higher speech quality and greater speaker similarity.

Experimental Approaches to the Study of Sound Sources Localization by Distance in Hearing Pathology

The overview presents modern experimental approaches that are used to assess the ability to localize sound sources by distance in hearing pathology. The influence of the typical manifestations of hearing impairment on the processes of identifying the main localization cues – the signal level, the spectral features, binaural characteristics in auditory perception of the distance to stationary and moving sound or speech sources is considered. The review introduce to the results of the authors' own research and literature data on changes in hearing resolution by distance in sensorineural hearing loss, unilateral hearing loss, central auditory disorders, including age-related aspects of the problem. The compensatory potential of the auditory spatial function in non-invasive and invasive hearing aids, as well as its training with elements of acoustic virtual reality, is described. A methodical approach to forming of spatial scenes available for implementation in clinical practice is proposed.

Design and integration of acoustic treatment to the Former Stock Exchange—Cathedral Room

The Gothic Bank Cathedral Room within the Former Stock Exchange located in Melbourne is architecturally significant and largely original displaying ornate features for a historical public business arena. The Cathedral Room was the main trading floor for the Stock Exchange and was designed to represent the interior of a church, highlighted by the vaulted roof, columns, and the stained-glass windows. Through the guidance of Heritage Council Victoria, the Cathedral Room was granted permission to be converted into a commercial dining room. This paper details the methodology adopted to reality capture the interiors of the Cathedral Room by deploying LiDAR laser scanning, to develop a room acoustics model, and to investigate mitigation options. Furthermore, it explores the link between applying appropriate acoustic targets, guiding stakeholders, and delivering the project, which is essential to operate a successful commercial business to meet the expectations of their cliental. Arup developed an Odeon model, which allowed for the placement of speech sources and receivers and with integrated mitigation options. The modeling was presented to stakeholders in Arup’s SoundLab, an ambisonic sound studio, allowing the stakeholders to listen to the untreated space and compare it to various levels of mitigation prior to progressing to detail design phase.

Cascade algorithms for combined acoustic feedback cancelation and noise reduction

This paper presents three cascade algorithms for combined acoustic feedback cancelation (AFC) and noise reduction (NR) in speech applications. A prediction error method (PEM)-based adaptive feedback cancelation (PEM-based AFC) algorithm is used for the AFC stage, while a multichannel Wiener filter (MWF) is applied for the NR stage. A scenario with M microphones and 1 loudspeaker is considered, without loss of generality. The first algorithm is the baseline algorithm, namely the cascade M-channel rank-1 MWF and PEM-AFC, where a NR stage is performed first using a rank-1 MWF followed by a single-channel AFC stage using a PEM-based AFC algorithm. The second algorithm is the cascade (M+1)-channel rank-2 MWF and PEM-AFC, where again a NR stage is applied first followed by a single-channel AFC stage. The novelty of this algorithm is to consider an (M+1)-channel data model in the MWF formulation with two different desired signals, i.e., the speech component in the reference microphone signal and in the loudspeaker signal, both defined by the speech source signal but not equal to each other. The two desired signal estimates are later used in a single-channel PEM-based AFC stage. The third algorithm is the cascade M-channel PEM-AFC and rank-1 MWF where an M-channel AFC stage is performed first followed by an M-channel NR stage. Although in cascade algorithms where NR is performed first and then AFC the estimation of the feedback path is usually affected by the NR stage, it is shown here that by performing a rank-2 approximation of the speech correlation matrix this issue can be avoided and the feedback path can be correctly estimated. The performance of the algorithms is assessed by means of closed-loop simulations where it is shown that for the considered input signal-to-noise ratios (iSNRs) the cascade (M+1)-channel rank-2 MWF and PEM-AFC and the cascade M-channel PEM-AFC and rank-1 MWF algorithms outperform the cascade M-channel rank-1 MWF and PEM-AFC algorithm in terms of the added stable gain (ASG) and misadjustment (Mis) as well as in terms of perceptual metrics such as the short-time objective intelligibility (STOI), perceptual evaluation of speech quality (PESQ), and signal distortion (SD).

Creating speech zones with self-distributing acoustic swarms

Imagine being in a crowded room with a cacophony of speakers and having the ability to focus on or remove speech from a specific 2D region. This would require understanding and manipulating an acoustic scene, isolating each speaker, and associating a 2D spatial context with each constituent speech. However, separating speech from a large number of concurrent speakers in a room into individual streams and identifying their precise 2D locations is challenging, even for the human brain. Here, we present the first acoustic swarm that demonstrates cooperative navigation with centimeter-resolution using sound, eliminating the need for cameras or external infrastructure. Our acoustic swarm forms a self-distributing wireless microphone array, which, along with our attention-based neural network framework, lets us separate and localize concurrent human speakers in the 2D space, enabling speech zones. Our evaluations showed that the acoustic swarm could localize and separate 3-5 concurrent speech sources in real-world unseen reverberant environments with median and 90-percentile 2D errors of 15 cm and 50 cm, respectively. Our system enables applications like mute zones (parts of the room where sounds are muted), active zones (regions where sounds are captured), multi-conversation separation and location-aware interaction.

‘Well, taakin about he da bring inta me yead wat I promised var ta tell ee about’: representations of south-western speech in nineteenth-century dialect writing

This article explores representations of south-western speech in nineteenth-century dialect writing. It draws on a selection of specimens from theSalamanca Corpusin order to determine what they can tell us about the language of south-western speakers at this time. By focusing on periphrasticdoand pronoun exchange, I argue that representations of south-western dialects can be taken as a missing link in the history of these two grammatical features. In fact, the analysis of their distribution and frequency, which this article explores in dialect writing for the first time, shows that they accord with later evidence to an interesting degree. At the same time, the data are placed within the third-wave sociolinguistic models of enregisterment and indexicality so as to show that the conscious representation of these morphosyntactic features reflects contemporary perceptions about their use in south-western dialects while they reveal indexical associations between place, speaker and speech. This article thus seeks to contribute to the history of south-western dialects, while underscoring the validity of dialect writing as a source of Late Modern English speech where the structural and ideological dimensions of dialect intersect.

Can you amuse the audience through an interpreter?

Abstract In this article, I investigate how interpreters handle humorous utterances during plenary debates of the European Parliament, focusing on the input by one Polish Member of the European Parliament (MEP), Janusz Korwin-Mikke. The source speeches (in Polish or English) are analysed bottom-up to identify the types of humour favoured by the speaker. The most frequent ones are irony, ad hominem arguments with an element of ridicule, absurdity, and shifts in register. Subsequently, a pragmatically oriented comparative analysis is conducted to assess whether and how individual instances of humour are transferred by interpreters. Additionally, possible side effects are considered, such as shifts accompanying transferred humour and message incoherence resulting from humour loss. Register humour is typically removed by interpreters. The successful handling of absurdity relies mainly on compression and often fails, while ad hominem and irony appear to be relatively less challenging to interpret. Interestingly, irony is occasionally added by interpreters, either to boost the speaker’s comical intent or to distance themselves from his views.

YouTube™ as an information source for speech and language disorders

Aims: The purpose of this study is to evaluate the quality of the information provided by the top 100 videos when searching for "speech and language disorders" on YouTube™.
 Methods: The results of the YouTube™ search were examined using the keyword “speech and language disorders”. The search was limited to the first 100 videos. Modified DISCERN, Global Quality Score (GQS), Video Information and Quality Index (VIQI), and the Journal of American Medical Association (JAMA) criteria were used for the evaluation of the videos. The Kruskal-Wallis H Test, Mann-Whitney U, and Spearman's RHO correlation were used for statistical analyses. The significance level was taken as p

The Benefit of Binaural Hearing Among Listeners with Sensorineural Hearing Loss

The performance of binaural processing may be disturbed in the presence of hearing loss, especially of sensorineural type. To assess the impact of hearing loss on speech perception in noise regarding binaural processing, series of speech recognition measurements in controlled laboratory conditions were carried out. The spatial conditions were simulated using dummy head recordings played back on headphones. The Intelligibility Level Difference (ILD) was determined by measuring the change in the speech reception thresholds (SRT) between two configurations of a masking signal source (N) and a speech source (S), namely the S0N90 condition (where numbers stand for angles in horizontal plane) and the co-located condition (S0N0). To disentangle the head shadow effect (better ear effect) from binaural processing in the brain, the difference between binaural and monaural S0N90 condition (so-called Binaural Intelligibility Level Difference, BILD) value was calculated. Measurements were performed with a control group of normal-hearing listeners and a group of sensorineural hearing-impaired subjects. In all conditions performance of the hearing-impaired listeners was significantly lower than normal-hearing ones, resulting in higher SRT values (3 dB difference in the S0N0 configuration, 7.6 dB in S0N90 and 5 dB in monaural S0N90). The SRT improvement due to the spatial separation of target and masking signal (ILD) was also higher in the control group (8.1 dB) than in hearing-impaired listeners (3.5 dB). Moreover, a significant deterioration of the binaural processing described by BILD was found in people with sensorineural deficits. This parameter for normal-hearing listeners reached a value of 3 to 6 dB (4.6 dB on average) and decreased more than two times in the hearing-impaired group to 1.9 dB on average (with a deviation of 1.4 dB). These findings could not be explained by individual average hearing threshold (standard in audiological diagnostics) only. The outcomes indicate that there is a contribution of suprathershold deficits and it may be useful to consider binaural SRT measurements in noise in addition to the pure tone audiometry resulting in better diagnostics and hearing aid fitting.

Target Selection Strategies for Demucs-Based Speech Enhancement

The Demucs-Denoiser model has been recently shown to achieve a high level of performance for online speech enhancement, but assumes that only one speech source is present in the fed mixture. In real-life multiple-speech-source scenarios, it is not certain which speech source will be enhanced. To correct this issue, two target selection strategies for the Demucs-Denoiser model are proposed and evaluated: (1) an embedding-based strategy, using a codified sample of the target speech, and (2) a location-based strategy, using a beamforming-based prefilter to select the target that is in front of a two-microphone array. In this work, it is shown that while both strategies improve the performance of the Demucs-Denoiser model when one or more speech interferences are present, they both have their pros and cons. Specifically, the beamforming-based strategy achieves overall a better performance (increasing the output SIR between 5 and 10 dB) compared to the embedding-based strategy (which only increases the output SIR by 2 dB and only in low-input-SIR scenarios). However, the beamforming-based strategy is sensitive against the location variation of the target speech source (decreasing the output SIR by 10 dB if the target speech source is located only 0.1 m from its expected position), which the embedding-based strategy does not suffers from.

A Post-Filtering for Performance of Differential Microphone Array

Microphone array (MA) is a perspective technique for speech enhancement, because of its simplicity and lowcomputation. MA can be decomposed into a concatenation of a spatial filtering, which provides beampattern toward a certain target directional speech source while attenuating noise, interference or third-party speaker from other directions, and a post-filtering, which suppresses the remaining noise at the final output signal. Differential Microphone Array (DIF) is one basic MA, which has many advantages for extracting the desired speech and null-steering to direction of noise. Nevertheless, in real situation, the performance of DIF often deteriorated due to phase error, imprecise the direction-ofarrival (DOA) of interest signal, microphone mismatches or imperfect distribution of MA, which significantly degrade the performance and the speech quality of DIF. In this article, the author proposed a Post-Filtering (PF), which based on the properties of considered recording environment DOA to enhance the speech quality of target speaker and reduce the other non-target speaker at the opposite direction. Experimental results were verified in real scenario with dialogue between two speakers, which stand opposite direction, and show the effectiveness of the suggested PF when compared with the author’s previous work.

Non-Special Loudspeakers as Speech Test Sources in Natural Acoustics Speech Intelligibility Investigations

Objective speech intelligibility estimations undertaken in natural acoustics speech communications (NAS) scenarios require the utilization of a speech source that approximates the acoustic characteristics of a human talker. Only a limited number of special speech sources that conform to the specifications in the relevant guidelines are available in the market; however, they can be deemed expensive by professional practitioners and other users. Non-special and affordable loudspeakers are often used in NAS investigations in place of standardized special speech sources without the knowledge of their suitability and results validity. This study aims to examine the suitability of a range of representative common and affordable non-special loudspeakers as a potential alternative to standardized speech sources in NAS indicative or pilot investigations. Frequency response and Speech Transmission Index Public Address (STIPA) experimental results obtained from a reference standardized speech source were compared against results from various non-special loudspeakers measured utilizing diverse and real-world representative combinations of NAS acoustic conditions under controlled laboratory conditions. STIPA mean absolute errors for the alternative speech sources were generally lower than the STIPA method uncertainty and one Just Noticeable Difference (0.03 STI). The findings of this study will inform practitioners of the suitability of affordable loudspeakers when standardized special test loudspeakers are not available.

VideoDubber: Machine Translation with Speech-Aware Length Control for Video Dubbing

Video dubbing aims to translate the original speech in a film or television program into the speech in a target language, which can be achieved with a cascaded system consisting of speech recognition, machine translation and speech synthesis. To ensure the translated speech to be well aligned with the corresponding video, the length/duration of the translated speech should be as close as possible to that of the original speech, which requires strict length control. Previous works usually control the number of words or characters generated by the machine translation model to be similar to the source sentence, without considering the isochronicity of speech as the speech duration of words/characters in different languages varies. In this paper, we propose VideoDubber, a machine translation system tailored for the task of video dubbing, which directly considers the speech duration of each token in translation, to match the length of source and target speech. Specifically, we control the speech length of generated sentence by guiding the prediction of each word with the duration information, including the speech duration of itself as well as how much duration is left for the remaining words. We design experiments on four language directions (German -> English, Spanish -> English, Chinese English), and the results show that VideoDubber achieves better length control ability on the generated speech than baseline methods. To make up the lack of real-world datasets, we also construct a real-world test set collected from films to provide comprehensive evaluations on the video dubbing task.

Single-channel speech separation using soft-minimum permutation invariant training

The goal of speech separation is to extract multiple speech sources from a single microphone recording. Recently, with the advancement of deep learning and availability of large datasets, speech separation has been formulated as a supervised learning problem. These approaches aim to learn discriminative patterns of speech, speakers, and background noise using a supervised learning algorithm, typically a deep neural network. A long-lasting problem in supervised speech separation is finding the correct label for each separated speech signal, referred to as label permutation ambiguity. Permutation ambiguity refers to the problem of determining the output-label assignment between the separated sources and the available single-speaker speech labels. Finding the best output-label assignment is required for calculation of separation error, which is later used for updating parameters of the model. Recently, Permutation Invariant Training (PIT) has been shown to be a promising solution in handling the label ambiguity problem. However, the overconfident choice of the output-label assignment by PIT results in a sub-optimal trained model. In this work, we propose a probabilistic optimization framework to address the inefficiency of PIT in finding the best output-label assignment. Our proposed method entitled trainable Soft-minimum PIT is then employed on the same Long-Short Term Memory (LSTM) architecture used in Permutation Invariant Training (PIT) speech separation method. The results of our experiments show that the proposed method outperforms conventional PIT speech separation significantly (p-value <0.01) by +1dB in Signal to Distortion Ratio (SDR) and +1.5dB in Signal to Interference Ratio (SIR).

Separation of Multiple Speech Sources in Reverberant Environments Based on Sparse Component Enhancement

Separation of Multiple Speech Sources in Reverberant Environments Based on Sparse Component Enhancement