Speech Sources Research Articles

Separating Multi Speeches in Intelligent Humanoid Robot using FastICA

The main objective of our research is to develop an intelligent humanoid robot for teaching children by listening and answering the questions. In our previous research, we have designed a humanoid robot that can detect human face and receive commands by using speech recognition. Our robot is based on Bioloid GP robot and Raspberry Pi2 as control system. In this study, we would like to expand the capability of the robot system in order to isolate the speech of one speaker from all the other sounds. The problem for separating multi speeches from stereo audio record is called as Blind Speech Separation (BSS). We propose FastICA algorithm to solve the BSS problem. FastICA is an efficient algorithm to separate several signals based on Independent Component Analysis (ICA) algorithm. Some assumption must be met to use FastICA, that is the number of mixtures are equal to the number of sources and the sources are linearly independent from each other. To evaluate the algorithm, we use several simulations based on two speech sources and its mixing matrix. Our simulation shows FastICA algorithm can solve BSS problem by separating two sound signals, but its linearly independent assumption makes it difficult to implement in our humanoid robot

Interaural Coherence Preservation for Binaural Noise Reduction Using Partial Noise Estimation and Spectral Postfiltering

The objective of binaural speech enhancement algorithms is to reduce the undesired noise component, while preserving the desired speech source and the binaural cues of all sound sources. For the scenario of a single desired speech source in a diffuse noise field, an extension of the binaural multichannel Wiener filter (MWF), namely the MWF-IC, has been recently proposed, which aims to preserve the interaural coherence (IC) of the noise component. However, due to the large complexity of the MWF-IC, in this paper we propose several alternative algorithms at a lower computational complexity. First, we consider a quasi-distortionless version of the MWF-IC, denoted as minimum-variance-distortionless response (MVDR-IC). Second, we propose to preserve the IC of the noise component using the binaural MWF with partial noise estimation (MWF-N) and the binaural MVDR beamformer with partial noise estimation (MVDR-N), for which closed-form expressions exist. In addition, we show that for the MVDR-N a closed-form expression can be derived for the tradeoff parameter yielding a desired magnitude squared coherence (MSC) for the output noise component. Since contrary to the MWF-IC and the MWF-N the MVDR-IC and the MVDR-N do not take into account the spectro-temporal properties of the speech and the noise components, we propose to apply a spectral postfilter to the filter outputs, improving the noise reduction performance. The performance of all algorithms is compared in several diffuse noise scenarios. The simulation results show that both the MVDR-IC and the MVDR-N are able to preserve the MSC of the noise component, while generally the MVDR-IC shows a slightly better noise reduction performance at a larger complexity. Further, simulation results show that applying a spectral postfilter leads to a very similar performance for all considered algorithms in terms of noise reduction and speech distortion.

Spatial Processing of Broadband Signals Using Speech Messages as Examples

This study is aimed at extracting the speech signal from a certain spatial point with maximum signal-to-noise ratio if interference is caused by other speech sources and the signals are recorded by an array of multipositional omnidirectional microphones. It is proved that the joint use of the algorithms for introducing exact time delays and forming optimal weight coefficients of the microphones considerably increases the useful-signal power compared with the interference power.

Sound source localization and speech enhancement with sparse Bayesian learning beamforming.

Speech localization and enhancement involves sound source mapping and reconstruction from noisy recordings of speech mixtures with microphone arrays. Conventional beamforming methods suffer from low resolution, especially with a limited number of microphones. In practice, there are only a few sources compared to the possible directions-of-arrival (DOA). Hence, DOA estimation is formulated as a sparse signal reconstruction problem and solved with sparse Bayesian learning (SBL). SBL uses a hierarchical two-level Bayesian inference to reconstruct sparse estimates from a small set of observations. The first level derives the posterior probability of the complex source amplitudes from the data likelihood and the prior. The second level tunes the prior towards sparse solutions with hyperparameters which maximize the evidence, i.e., the data probability. The adaptive learning of the hyperparameters from the data auto-regularizes the inference problem towards sparse robust estimates. Simulations and experimental data demonstrate that SBL beamforming provides high-resolution DOA maps outperforming traditional methods especially for correlated or non-stationary signals. Specifically for speech signals, the high-resolution SBL reconstruction offers not only speech enhancement but effectively speech separation.

Development of Novel Hearing Aids by Using Image Recognition Technology.

Speech is easily affected by different background noise in real environment to reduce the speech intelligibility, in particular, for hearing impaired listeners. In order to improve the above issue, several hearing aids have been developed to enhance the speech signal in noisy environment. Most of current hearing aids were designed to enhance the component of speech and suppress the component of noise. However, it is difficult to separate other speech sources. Adaptive signal enhancement with the beamforming technique might improve the above issue. However, how to distinguish the location of the desired speaker effectively is still a difficult challenge for adaptive beamforming method. A novel concept of hearing aid was proposed in this study. Different from the beamforming-based hearing aids, which use the cross-correlation-coefficient method to estimate time difference of arrival (TDOA), an image recognition technology was used to estimate the location of the desired speaker to obtain the more precise TDOA. An adaptive signal enhancement was also used to enhance the noisy speech sound. From the experimental results, the proposed system could provide a smaller absolute error of TDOA less than 1.25 × 10-4 ms, and a clear speech sound from the target speaker who the user wants to listen to.

Translating like a conduit? A sociosemiotic analysis of modality in Chinese government press conference interpreting

AbstractThis paper is a corpus-based sociosemiotic inquiry into the translation of linguistic modality in government press conferences in the Chinese context, with an eye to its indication of interpreter’s identity. Viewing translation (including interpreting) as a process of social semiosis, the paper draws on theoretical insights from Systemic Functional Linguistics (SFL) and applies them to the analysis of modality in both English and Chinese – the language pair that concerns the present research. Results of the study show that, while modality distribution in the two languages are basically maintained at the same level, interchangeable uses between volitive and obligatory subtypes of modality plus the general increase of modality value in interpreted vis-à-vis source speeches indicate that interpreters are not deprived of mediating latitude which is believed to contradict their prescriptive stereotypes. Also, exemplary parallel concordance analysis of modality reveals that interpreters adopt various solutions to translating the same modal element. Further, the paper proposes a taxonomy for the analysis of modality shifts in interpreter-mediated encounters, with illustrative cases of each subclass examined and discussed. The findings are expected to shed light on the interpreter’s identity in political institutional settings.

A Conditional Generative Model for Speech Enhancement

Deep learning-based speech enhancement approaches like deep neural networks (DNN) and Long Short-Term Memory (LSTM) have already demonstrated superior results to classical methods. However, these methods do not take full advantage of temporal context information. While DNN and LSTM consider temporal context in the noisy source speech, it does not do so for the estimated clean speech. Both DNN and LSTM also have a tendency to over-smooth spectra, which causes the enhanced speech to sound muffled. This paper proposes a novel architecture to address both issues, which we term a conditional generative model (CGM). By adopting an adversarial training scheme applied to a generator of deep dilated convolutional layers, CGM is designed to model the joint and symmetric conditions of both noisy and estimated clean spectra. We evaluate CGM against both DNN and LSTM in terms of Perceptual Evaluation of Speech Quality (PESQ) and Short-Time Objective Intelligibility (STOI) on TIMIT sentences corrupted by ITU-T P.501 and NOISEX-92 noise in a range of matched and mismatched noise conditions. Results show that both the CGM architecture and the adversarial training mechanism lead to better PESQ and STOI in all tested noise conditions. In addition to yielding significant improvements in PESQ and STOI, CGM and adversarial training both mitigate against over-smoothing.

Modeling Cochlear Implant processing and bilateral performance in acoustic environments with multiple sources

The work reported here focuses on the use of cochlear implants in situations with multiple sources, especially speech sources, and in other difficult environments. After a review of previous measurements and models of cochlear implant (CI) stimulation, recent modeling studies of speech processing in conditions with multiple inputs in complex acoustic environments will be discussed. As noted above, complex environments considered include conditions with multiple speech sources, and this talk focuses on effects of spatial separation and voice pitch differences in confusions related to source separation. These effects include both energetic and informational masking. Multiple implant factors will be addressed, including interaural variability in the timing of the left and right stimuli, the effects of independent left and right automatic gain control, and the interactions with and basic role of pulse rate in the implants. [Work supported by NIDCD 5 R01 DC000100.]

Characteristics of Real-World Signal to Noise Ratios and Speech Listening Situations of Older Adults With Mild to Moderate Hearing Loss.

The first objective was to determine the relationship between speech level, noise level, and signal to noise ratio (SNR), as well as the distribution of SNR, in real-world situations wherein older adults with hearing loss are listening to speech. The second objective was to develop a set of prototype listening situations (PLSs) that describe the speech level, noise level, SNR, availability of visual cues, and locations of speech and noise sources of typical speech listening situations experienced by these individuals. Twenty older adults with mild to moderate hearing loss carried digital recorders for 5 to 6 weeks to record sounds for 10 hours per day. They also repeatedly completed in situ surveys on smartphones several times per day to report the characteristics of their current environments, including the locations of the primary talker (if they were listening to speech) and noise source (if it was noisy) and the availability of visual cues. For surveys where speech listening was indicated, the corresponding audio recording was examined. Speech-plus-noise and noise-only segments were extracted, and the SNR was estimated using a power subtraction technique. SNRs and the associated survey data were subjected to cluster analysis to develop PLSs. The speech level, noise level, and SNR of 894 listening situations were analyzed to address the first objective. Results suggested that as noise levels increased from 40 to 74 dBA, speech levels systematically increased from 60 to 74 dBA, and SNR decreased from 20 to 0 dB. Most SNRs (62.9%) of the collected recordings were between 2 and 14 dB. Very noisy situations that had SNRs below 0 dB comprised 7.5% of the listening situations. To address the second objective, recordings and survey data from 718 observations were analyzed. Cluster analysis suggested that the participants' daily listening situations could be grouped into 12 clusters (i.e., 12 PLSs). The most frequently occurring PLSs were characterized as having the talker in front of the listener with visual cues available, either in quiet or in diffuse noise. The mean speech level of the PLSs that described quiet situations was 62.8 dBA, and the mean SNR of the PLSs that represented noisy environments was 7.4 dB (speech = 67.9 dBA). A subset of observations (n = 280), which was obtained by excluding the data collected from quiet environments, was further used to develop PLSs that represent noisier situations. From this subset, two PLSs were identified. These two PLSs had lower SNRs (mean = 4.2 dB), but the most frequent situations still involved speech from in front of the listener in diffuse noise with visual cues available. The present study indicated that visual cues and diffuse noise were exceedingly common in real-world speech listening situations, while environments with negative SNRs were relatively rare. The characteristics of speech level, noise level, and SNR, together with the PLS information reported by the present study, can be useful for researchers aiming to design ecologically valid assessment procedures to estimate real-world speech communicative functions for older adults with hearing loss.

An optimization model for electroencephalography-assisted binaural beamforming

In a multi-speakers noisy environment, the incoherent correlation between the electroencephalography (EEG) signal and the attended speech envelope provides a user-informed way to design the beamformer for preserving the attended speech and suppressing others. In this work, we exploit such incoherent correlation property in terms of Pearson correlation and propose a unified optimization model for simultaneously designing beamformer and aligning attention preference of the user to each speech source. To balance different design considerations, the proposed optimization formulation makes a trade-off among aligning attention preference, controlling speech distortion, and reducing noise in a weighted manner in the objective function. Specifically, the attention preference is aligned by maximizing the Pearson correlation between the envelope of beamforming output and the linearly transformed EEG signal. To control speech distortion with flexibility, the spatial response of the beamformer to each source is penalized in a min-max sense. And the noise is further reduced by minimizing its mean squares at beamforming output. Experiments on collected EEG signal in two speakers environment demonstrate the effectiveness of the proposed model.

Spatial Selectivity of Hearing in Speech Recognition in Speech-shaped Noise Environment

A variety of perceptual features can be used for the successful separation of information flows and higher speech intelligibility. The binaural system based on the spatial allocation of speech signals plays the most important role in attaining this goal. This review discusses how the mechanisms of spatial hearing provide selective attention to a target speech source and promote the recognition of a masked target signal.

Multiple Speech Source Separation Using Inter-Channel Correlation and Relaxed Sparsity

In this work, a multiple speech source separation method using inter-channel correlation and relaxed sparsity is proposed. A B-format microphone with four spatially located channels is adopted due to the size of the microphone array to preserve the spatial parameter integrity of the original signal. Specifically, we firstly measure the proportion of overlapped components among multiple sources and find that there exist many overlapped time-frequency (TF) components with increasing source number. Then, considering the relaxed sparsity of speech sources, we propose a dynamic threshold-based separation approach of sparse components where the threshold is determined by the inter-channel correlation among the recording signals. After conducting a statistical analysis of the number of active sources at each TF instant, a form of relaxed sparsity called the half-K assumption is proposed so that the active source number in a certain TF bin does not exceed half the total number of simultaneously occurring sources. By applying the half-K assumption, the non-sparse components are recovered by regarding the extracted sparse components as a guide, combined with vector decomposition and matrix factorization. Eventually, the final TF coefficients of each source are recovered by the synthesis of sparse and non-sparse components. The proposed method has been evaluated using up to six simultaneous speech sources under both anechoic and reverberant conditions. Both objective and subjective evaluations validated that the perceptual quality of the separated speech by the proposed approach outperforms existing blind source separation (BSS) approaches. Besides, it is robust to different speeches whilst confirming all the separated speeches with similar perceptual quality.

Evaluation of Two-Channel Source Separation Using Exploratory Projection Pursuit Technique

Difficulty of understanding speech in noise among the elderly necessitates the need for Auditory Training which has made a renewal of interest in the last decade with the auditory training applications. This interest is perhaps spurred by advances in computer-based technology. In computer-based auditory training, speech signals are considered as auditory training stimuli where input speech signals need to be verified prior to training as the speech signals are mixed with noise signals. Computer-based Auditory Training System can be embedded with input speech verifying module. Input speech verifying module is employed with speech and noise separator simulator. This simulator needs to guarantee accurate separation of speech from noise signals. Therefore, in this research, Exploratory Projection Pursuit (EPP) technique under semi-Blind Source Separation (BSS) method is intended to separate the speech source signals which are mixed with competing speech (multitalker speech babble). This training uses Malay language based sentences which differ in word length and hence number of sample values. The experimental simulation considers two-channel random, linear mixing of speech sources and competing speech. The aim of this study is to evaluate the performance of source separation using the anticipated EPP technique for various sample values of speech signals which varies in time duration due to word length dissimilarity. Simulation results show that EPP technique is feasible for source separation. As a consequence, high correlation value of r ≥ 0.99 is obtained between extracted speech signal and original speech signal for all categories of speech signals. It is further verified by the maximum nongaussianity of extracted speech signal which has high kurtosis value of 32 approximately. DOI : http://dx.doi.org./10.17576/JSKM-2018-28

Turn an Ear to Hear: How Hearing-Impaired Listeners Can Exploit Head Orientation to Enhance Their Speech Intelligibility in Noisy Social Settings.

Turning an ear toward the talker can enhance spatial release from masking. Here, with their head free, listeners attended to speech at a gradually diminishing signal-to-noise ratio and with the noise source azimuthally separated from the speech source by 180° or 90°. Young normal-hearing adult listeners spontaneously turned an ear toward the speech source in 64% of audio-only trials, but a visible talker’s face or cochlear implant (CI) use significantly reduced this head-turn behavior. All listener groups made more head movements once instructed to explore the potential benefit of head turns and followed the speech to lower signal-to-noise ratios. Unilateral CI users improved the most. In a virtual restaurant simulation with nine interfering noises or voices, hearing-impaired listeners and simulated bilateral CI users typically obtained a 1 to 3 dB head-orientation benefit from a 30° head turn away from the talker. In diffuse interference environments, the advice to U.K. CI users from many CI professionals and the communication guidance available on the Internet most often advise the CI user to face the talker head on. However, CI users would benefit from guidelines that recommend they look sidelong at the talker with their better hearing or implanted ear oriented toward the talker.

Multiple Speech Source Separation with Non-Sparse Components Recovery by Using Dual Similarity Determination

Multiple Speech Source Separation with Non-Sparse Components Recovery by Using Dual Similarity Determination

A Case Study of Arabic-Speaking Undergraduate Trainee Interpreters’ Strategies: A Model for Classroom Practice

This paper reports the results of a case study of the strategies used by Arabic-speaking undergraduate simultaneous interpreter (SI) trainees while interpreting in booths under test conditions. The study was discourse-based as it adopted the main premise of the Information Processing approach to SI (Hodzik & Williams 2017, p. 2) which considers interpreting a language-dependent activity. The data consisted of a sample of the aforementioned trainees’ recorded interpretations of a video lecture on solar energy. The data analysis was based on Barik’s (1975, 2002) model of omission, addition and phrasing changes. The results show that the most frequent errors made and/or strategies used by the study subjects are delay omission and comprehension omission. This paper also presents a discourse-based module for SI training. The module is based on the premise that SI comprises an analysis of the comprehension phase of the source speech and the production phase of source speech messages in the target language. The material utilizes the main tenets of Setton’s (1993, 1998, 2002) model of comprehension and production and Seleskovitch’s (2008) model of teaching interpreting. This study thus sought to integrate research on SI into classroom practices.

Separation of multiple speech sources by recovering sparse and non-sparse components from B-format microphone recordings

This paper proposes a blind source separation (BSS) method for recovering multiple speech sources from sound fields recorded by a B-format microphone. This microphone provides a four channel representation that can be used to derive the direction of arrival (DOA) of spatially distinct time-frequency (TF) components. Such sparse components correspond to bins where only one speech source is active and are identified based on the inter-correlation among the mixture signals. They are recovered via a degenerate unmixing estimation technique (DUET)-like method. Proposed is a “local-zone stationarity” assumption, where the amplitude of a speech signal remains approximately constant within a small band of TF components. This assumption is validated through statistical analysis of a quantitative measure of stationarity. Under this assumption, the non-sparse components (TF points where more than one speech source is active) are recovered via a Wiener-filter-like approach where the separated sparse components is utilized as a guide. The final separated sources are obtained by combining the separated sparse and non-sparse components. Both objective and subjective evaluations show that the proposed method achieves better separation quality compared to some existing BSS approaches where up to six simultaneous speech sources are considered.

A non-intrusive method for estimating binaural speech intelligibility from noise-corrupted signals captured by a pair of microphones

A non-intrusive method is introduced to predict binaural speech intelligibility in noise directly from signals captured using a pair of microphones. The approach combines signal processing techniques in blind source separation and localisation, with an intrusive objective intelligibility measure (OIM). Therefore, unlike classic intrusive OIMs, this method does not require a clean reference speech signal and knowing the location of the sources to operate. The proposed approach is able to estimate intelligibility in stationary and fluctuating noises, when the noise masker is presented as a point or diffused source, and is spatially separated from the target speech source on a horizontal plane. The performance of the proposed method was evaluated in two rooms. When predicting subjective intelligibility measured as word recognition rate, this method showed reasonable predictive accuracy with correlation coefficients above 0.82, which is comparable to that of a reference intrusive OIM in most of the conditions. The proposed approach offers a solution for fast binaural intelligibility prediction, and therefore has practical potential to be deployed in situations where on-site speech intelligibility is a concern.

Improvement of phone recognition accuracy using speech mode classification

In this work, we have developed a speech mode classification model for improving the performance of phone recognition system (PRS). In this paper, we have explored vocal tract system, excitation source and prosodic features for development of speech mode classification (SMC) model. These features are extracted from voiced regions of a speech signal. In this study, conversation, extempore, and read speech are considered as three different modes of speech. The vocal tract component of speech is extracted using Mel-frequency cepstral coefficients (MFCCs). The excitation source features are captured through Mel power differences of spectrum in sub-bands (MPDSS) and residual Mel-frequency cepstral coefficients (RMFCCs) of the speech signal. The prosody information is extracted from pitch and intensity. Speech mode classification models are developed using above described features independently, and in fusion. The experiments carried out on Bengali speech corpus to analyze the accuracy of the speech mode classification model using the artificial neural network (ANN), naive Bayes, support vector machines (SVMs) and k-nearest neighbor (KNN). We proposed four classification models which are combined using maximum voting approach for optimal performance. From the results, it is observed that speech mode classification model developed using the fusion of vocal tract system, excitation source and prosodic features of speech, yields the best performance of 98%. Finally, the proposed speech mode classifier is integrated to the PRS, and the accuracy of phone recognition system is observed to be improved by 11.08%.

Deep recurrent neural networks based binaural speech segregation for the selection of closest target of interest

An auditory attention model that consists of binaural source segregation and also full localization of a target speech signal in a multi-talker environment is presented. The joint acoustic features, such as monaural, binaural and direct to reverberant ratio (DRR) that are successfully incorporated into deep recurrent neural network (DRNN) based joint discriminative model for the speech source segregation process. The monaural and binaural features are extracted from binaural speech mixtures of two speakers by using mean Hilbert envelope coefficients (MHEC) and interaural time, and level differences, respectively. The performance of deep recurrent network based speech segregation is validated in terms of signal to interference, signal to distortion and signal to artifacts and compared with existing architectures, including deep neural network (DNN). The proposed system is observed and found to be more suitable than monaural speech segregation especially when the desired target and interfering sources are located at different positions. The study also proposes full localization of segregated speech source that created the possibility to select the desired speaker of interest from an input acoustic speech mixture in a reverberant environment. The developed system has the capability to handle binaural segregation problem in multi-source and reverberation conditions. The auditory attention model provides accurate information about speech sources even when the desired targets are located at 2 m and above with higher reverberation time.