Mixed Speech Signals Research Articles

A speech separation model improved based on Conv-TasNet network

Abstract In the field of single-channel speech separation, the extraction and separation of features from mixed audio have always been the focus and difficulty of research. Currently, mainstream methods mainly suffer from poor generalization ability and issues such as inadequate feature extraction, which leads to the models’ inferior separation capability. This paper proposes an improved DConv-TasNet network model, focusing on the optimization of the encoder/decoder modules and separation modules and utilizing deep dilated encoders/decoders to extract features from mixed speech signals. It enhances feature extraction capability and generalization compared to conventional encoders/decoders. In terms of the separation module, improvements were made to the convolutional blocks within the module by enhancing feature extraction in the channel dimension, leading to improved performance of the separation network. Validation of the model’s performance was conducted using the WSJ0-Mix2 dataset, demonstrating superior performance compared to the Conv-TasNet network.

Speech Signal Extraction Method Based on Bayesian Estimation Using Air- and Bone-Conduction Sound in Speech Confusion

With the recent development of digital information technology, speech information recognition system has improved dramatically and is being applied in a wide range of fields. In order to improve the system efficiency, countermeasures measures against noise from the surrounding environment have become an important issue. In the previous study, we derived an algorithm to estimate speech signals by using air-conducted speech signal mixed with noise as observation and bone-conducted speech signal. In this research, we propose an algorithm to obtain more accurate estimation results by using not only air-conducted speech signal mixed with noise but also bone-conducted speech signal as observation values. The effectiveness of the proposed method was confirmed experimentally by applying it to mixed speech signals actually measured in surrounding noisy environment.

Single-channel speech separation method based on attention mechanism

In order to improve the efficiency of speech interaction so that both humans and machines can hear it clearly, to solve the problem of single channel speech separation, a singlechannel speech separation method combined with attention mechanism is proposed. Based on the time-domain speech separation network, the method uses Encoder-separator-decoder framework and improves the Separator structure of the framework. First, the encoder transforms the input mixed speech signal into the latent space and extracts the feature of the speech signal to obtain the deep representation of the speech. Then, the deep attention features of speech were extracted by the splitter combined with the attention mechanism, and the mask was estimated for each independent sound source. Finally, the decoder inversely transforms the separated source signals back to the time domain. Experiments on WSJ0 speech separation data set show that the proposed method can effectively improve the SI-SNR of mixed speech separation, which is 0.31dB higher than the original time-domain network, and 0.94dB higher for the mixed case of boys and girls.

Generative adversarial networks for single channel separation of convolutive mixed speech signals

The suppression of interference for speech recognition is of great significance in noisy situation, especially in single channel receiving mode, the suppression of interference is much more difficult. In this paper, we propose a generative adversarial network (GAN) based method for single channel dereverberation and speech separation. Different from the existing methods, our method considers the influence of strong reverberation on the observed signals. The proposed network involves two parts: reverberation suppression and target speech enhancement. Firstly, we use an improved CyclyGAN to compensate the multi-path effect on both target speech and interference. Secondly, we propose a differentialGAN to extract both target speech and interference while the interference enhancement network can indirectly improve the performance of target speech enhancement network. We use the real and imaginary parts of the complex spectrum as the feature vector, which avoids the phase mismatch during signal recovery. Simulation results show that our method is superior to its competitors in terms of multiple metrics in severe reverberation environment.

Speech Separation Using Convolutional Neural Network and Attention Mechanism

Speech information is the most important means of human communication, and it is crucial to separate the target voice from the mixed sound signals. This paper proposes a speech separation model based on convolutional neural networks and attention mechanism. The magnitude spectrum of the mixed speech signals, as the input, has its high dimensionality. By analyzing the characteristics of the convolutional neural network and attention mechanism, it can be found that the convolutional neural network can effectively extract low-dimensional features and mine the spatiotemporal structure information in the speech signals, and the attention mechanism can reduce the loss of sequence information. The accuracy of speech separation can be improved effectively by combining two mechanisms. Compared to the typical speech separation model DRNN-2 + discrim, this method achieves 0.27 dB GNSDR gain and 0.51 dB GSIR gain, which illustrates that the speech separation model proposed in this paper has achieved an ideal separation effect.

Enhanced forensic speaker verification performance using the ICA-EBM algorithm under noisy and reverberant environments

Forensic speaker verification performance reduces significantly under high levels of noise and reverberation. Multiple channel speech enhancement algorithms, such as independent component analysis by entropy bound minimization (ICA-EBM), can be used to improve noisy forensic speaker verification performance. Although the ICA-EBM was used in previous studies to separate mixed speech signals under clean conditions, the effectiveness of using the ICA-EBM for improving forensic speaker verification performance under noisy and reverberant conditions has not been investigated yet. In this paper, the ICA-EBM algorithm is used to separate the clean speech from noisy speech signals. Features from the enhanced speech are obtained by combining the feature-warped mel frequency cepstral coefficients with similar features extracted from the discrete wavelet transform. The identity vector (i-vector) length normalized Gaussian probabilistic linear discriminant analysis is used as a classifier. The Australian Forensic Voice Comparison and QUT-NOISE corpora were used to evaluate forensic speaker verification performance under noisy and reverberant conditions. Simulation results demonstrate that forensic speaker verification performance based on ICA-EBM improves compared with that of the traditional independent component analysis under different types of noise and reverberation environments. For surveillance recordings corrupted with different types of noise (CAR, STREET and HOME) at − 10 dB signal to noise ratio, the average equal error rate of the proposed method based on ICA-EBM is better than that of the traditional ICA by 12.68% when the interview recordings are kept clean, and 7.25% when the interview recordings have simulated room reverberations.

Mental Workload Classification Method Based on EEG Independent Component Features

Excessive mental workload will reduce work efficiency, but low mental workload will cause a waste of human resources. It is very significant to study the mental workload status of operators. The existing mental workload classification method is based on electroencephalogram (EEG) features, and its classification accuracy is often low because the channel signals recorded by the EEG electrodes are a group of mixed brain signals, which are similar to multi-source mixed speech signals. It is not wise to directly analyze the mixed signals in order to distinguish the feature of EEG signals. In this study, we propose a mental workload classification method based on EEG independent components (ICs) features, which borrows from the blind source separation (BSS) idea of mixed speech signals. This presented method uses independent component analysis (ICA) to obtain pure signals, i.e., ICs. The energy features of ICs are directly extracted for classifying the mental workload, since this method directly uses ICs energy features for feature extraction. Compared with the existing solution, the proposed method can obtain better classification results. The presented method might provide a way to realize a fast, accurate, and automatic mental workload classification.

Adaptive Speech Separation Based on Beamforming and Frequency Domain-Independent Component Analysis

Voice signals acquired by a microphone array often include considerable noise and mutual interference, seriously degrading the accuracy and speed of speech separation. Traditional beamforming is simple to implement, but its source interference suppression is not adequate. In contrast, independent component analysis (ICA) can improve separation, but imposes an iterative and time-consuming process to calculate the separation matrix. As a supporting method, principle component analysis (PCA) contributes to reduce the dimension, retrieve fast results, and disregard false sound sources. Considering the sparsity of frequency components in a mixed signal, we propose an adaptive fast speech separation algorithm based on multiple sound source localization as preprocessing to select between beamforming and frequency domain ICA according to different mixing conditions per frequency bin. First, a fast positioning algorithm allows calculating the maximum number of components per frequency bin of a mixed speech signal to prevent the occurrence of false sound sources. Then, PCA reduces the dimension to adaptively adjust the weight of beamforming and ICA for speech separation. Subsequently, the ICA separation matrix is initialized based on the sound source localization to notably reduce the iteration time and mitigate permutation ambiguity. Simulation and experimental results verify the effectiveness and speedup of the proposed algorithm.

A General Nonstationary and Time-Varying Mixed Signal Blind Source Separation Method Based on Online Gaussian Process

Most nonstationary and time-varying mixed source separation algorithms are based on the model of instantaneous mixtures. However, the observation signal is a convolutional mixed source in reverberation environment, such as mobile voice received by indoor microphone arrays. In this paper, a time-varying convolution blind source separation (BSS) algorithm for nonstationary signals is proposed, which can separate both time-varying instantaneous mixtures and time-varying convolution mixtures. We employ the variational Bayesian (VB) inference method with Gaussian process (GP) prior for separating the nonstationary source frame by frame from the time-varying convolution signal, in which the prior information of the mixing matrix and the source signal are obtained by the Gaussian autoregressive method, and the posterior distributions of parameters (source signal and mixing matrix) are obtained by the VB learning. In the learning process, the learned parameters and hyperparameters are propagated to the next frame for VB inference as the prior which is combined with the likelihood function to get the posterior distribution. The experimental results show that the proposed algorithm is effective for separating time-varying mixed speech signals.

Joint dictionary learning using a new optimization method for single-channel blind source separation

Cross projection is often produced between sub-dictionaries when a mixed speech signal is represented over a joint dictionary in single-channel blind source separation (SCBSS), which leads to poor separation performance. To solve this problem, we introduce a new optimization function of joint dictionary learning for SCBSS, which trains the identity sub-dictionaries and common sub-dictionary simultaneously. The existence of a common sub-dictionary can effectively avoid one source signal being represented by the interferer sub-dictionaries. The new optimization function presented in this paper effectively inhibits the cross projection via enforcing constraint and cross-penalty between the identity sub-dictionaries and common sub-dictionary. The optimization of the objective function is introduced in this work, which involves three steps: initial joint dictionary, coding update, and dictionary update. In the speech separation stage, the source separation is achieved by an identity sub-dictionary, the common sub-dictionary and the sparse coefficients of a single mixed signal over the joint dictionary. Experimental results verify that our algorithm works much better than the current algorithms.

Frequency Selection Based Separation of Speech Signals with Reduced Computational Time Using Sparse NMF

Abstract Application of wavelet decomposition is described to speed up the mixed speech signal separation with the help of non-negative matrix factorisation (NMF). It is assumed that the basis vectors of training data of individual speakers had been recorded. In this paper, the spectrogram magnitude of a mixed signal has been factorised with the help of NMF with consideration of sparseness of speech signals. The high frequency components of signal contain very small amount of signal energy. By rejecting the high frequency components, the size of input signal is reduced, which reduces the computational time of matrix factorisation. The signal of lower energy has been separated by using wavelet decomposition. The present work is done for wideband microphone speech signal and standard audio signal from digital video equipment. This shows an improvement in the separation capability using the proposed model as compared with an existing one in terms of correlation between separated and original signals. Obtained signal to distortion ratio (SDR) and signal to interference ratio (SIR) are also larger as compare of the existing model. The proposed model also shows a reduction in computational time, which results in faster operation.

An unsupervised approach for co-channel speech separation using Hilbert–Huang transform and Fuzzy C-Means clustering

In this paper we discuss an unsupervised approach for co-channel speech separation where two speakers are speaking simultaneously over same channel. We propose a two stage separation process where the initial stage is based on empirical mode decomposition (EMD) and Hilbert transform generally known as Hilbert---Huang transform. EMD decomposes the mixed signal into oscillatory functions known as intrinsic mode functions. Hilbert transform is applied to find the instantaneous amplitudes and Fuzzy C-Means clustering is applied to group the speakers at initial stage. In second stage of separation speaker groups are transformed into time---frequency domain using short time Fourier transform (STFT). Time---frequency ratio's are computed by dividing the STFT matrix of mixed speech signal and STFT matrix of stage1 recovered speech signals. Histogram of the ratios obtained can be used to estimate the ideal binary mask for each speaker. These masks are applied to the speech mixture and the underlying speakers are estimated. Masks are estimated from the speech mixture and helps in imputing the missing values after stage1 grouping of speakers. Results obtained show significant improvement in objective measures over other existing single-channel speech separation methods.

A binaural model using equalization/cancellation and simulated head movements to localize and extract one speaker from a mixture

This model takes a mixture of two simultaneous speech signals at unique azimuth positions to extract either speaker using the equalization/cancelation (EC) method. Head-related transfer functions are used to spatialize the two sound sources. The model localizes the sources by analyzing interaural time differences and then virtually rotates its head to find the position for the best signal-to-noise ratio. Next, the model segments the mixed speech signal in time and frequency bins, and uses an EC algorithm in each bin to compensate the target signal from the mixture. From the residual non-cancelled energy, it generates a binary map and overlays this on the spectrogram. The ability of the model to cancel out the target signal determines the bins where the target is actually present. The signal is then reconstructed in time and frequency, leaving only one desired target signal. The model achieves signal-to-noise ratios of up to 80 dB. [This material is based upon work supported by the National Science Foundation under Grant Nos. 1320059 and 1539276.]

High-speed separation of anechoic mixtures of speech signals using fusion of iterative and closed-forms separation method

We introduce the fusion of iterative and closed-forms separation (FICS) method for high-speed separation of anechoic mixed speech signals. This method is performed in two stages: 1) iterative-form separation and 2) closed-form separation. This algorithm significantly improves the separation quality simply due to incorporating only some specific frequency bins into computations. We apply the FICS method to the frequency-domain independent component analysis (ICA) to evaluate its performance in increasing the signal separation speed for anechoic mixtures. Simulation results show that for speech signals and anechoic conditions, the proposed algorithm is on average 65 times faster than ICA while preserving the separation quality. It also outperforms FastICA, JADE, and SOBI in terms of separation quality and speed.

High-speed blind speech separation using FICS method

In this paper, we introduce the fusion of iterative and closed-form separation (FICS) method for high-speed separation of mixed speech signals. This method is performed in two stages: (i) iterative-form separation and (ii) closed-form separation. This algorithm significantly improves the separation quality simply by incorporating only some specific frequency bins into computations. We apply the FICS method to the frequency-domain independent component analysis (ICA) to evaluate its performance in increasing the signal separation speed. Simulation results show that for speech signals, the proposed algorithm is on average 40 times faster than the ICA, while preserving the separation quality. Also, it outperforms the FastICA, JADE, and SOBI in terms of separation quality. © 2014 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

An Introduction to Energy-Based Blind Separating Algorithm for Speech Signals

We introduce the Energy-Based Blind Separating (EBS) algorithm for extremely fast separation of mixed speech signals without loss of quality, which is performed in two stages: iterative-form separation and closed-form separation. This algorithm significantly improves the separation speed simply due to incorporating only some specific frequency bins into computations. Simulation results show that, on average, the proposed algorithm is 43 times faster than the independent component analysis (ICA) for speech signals, while preserving the separation quality. Also, it outperforms the fast independent component analysis (FastICA), the joint approximate diagonalization of eigenmatrices (JADE), and the second-order blind identification (SOBI) algorithm in terms of separation quality.

An Introduction to Energy-Based Blind Separating Algorithm for Speech Signals

We introduce the Energy-Based Blind Separating (EBS) algorithm for extremely fast separation of mixed speech signals without loss of quality, which is performed in two stages: iterative-form separation and closed-form separation. This algorithm significantly improves the separation speed simply due to incorporating only some specific frequency bins into computations. Simulation results show that, on average, the proposed algorithm is 43 times faster than the independent component analysis (ICA) for speech signals, while preserving the separation quality. Also, it outperforms the fast independent component analysis (FastICA), the joint approximate diagonalization of eigenmatrices (JADE), and the second-order blind identification (SOBI) algorithm in terms of separation quality.

Blind Source Separation of Noisy Mixed Speech Signals

In this paper, a new method for blind source separation of the noisy mixed speech signals is introduced. Firstly, the adaptive spectral subtraction is adopted to eliminate noise of noisy mixed speech signals. Secondly, the FASTICA algorithm is used to separate denoised mixed speech signals .Finally, wavelet transform is applied to remove the residual noise, and then the estimation of each speech source signal can be got.

가변 스텝 크기 적응 필터와 음성 검출기를 이용한 보청기용 피드백 제거 알고리즘

Acoustic feedback is perceived as whistling or howling, which is a major complaint of hearing-aids users. Acoustic feedback cancellation is important in hearing-aids because acoustic feedback degrades performance of the hearing aid device by reducing maximum insertion gain. Adaptive systems for estimate acoustic feedback path and feedback suppression algorithms have been proposed in order to solve this problem. A typical feedback cancellation algorithm is LMS(least mean squares) because of its computational efficiency. However it has problem of convergence performance in high correlated input signal. In this paper, we propose a new variable step-size normalized LMS(least mean squares) algorithm using VAD(voice activity detection) to overcome the limitation of the LMS algorithm. The VAD algorithm is GSAP(global speech absence probability) and the feedback cancellation algorithm is normalized LMS. The proposed algorithm applies different step-size between voice and non-voice using VAD, for high stability, fast convergence speed and low misalignment when correlated inputs, such as speech. The result of simulation with white noise mixed speech signal, the proposed algorithm shows high performance then traditional algorithm in terms of stability, convergence speed and misalignment.

Group Delay Based Methods for Speaker Segregation and its Application in Multimedia Information Retrieval

A novel method of single channel speaker segregation using the group delay cross correlation function is proposed in this paper. The group delay function, which is the negative derivative of the phase spectrum, yields robust spectral estimates. Hence the group delay spectral estimates are first computed over frequency sub-bands after passing the speech signal through a bank of filters. The filter bank spacing is based on a multi-pitch algorithm that computes the pitch estimates of the competing speakers. An affinity matrix is then computed from the group delay spectral estimates of each frequency sub-band. This affinity matrix represents the correlations of the different sub-bands in the mixed broadband speech signal. The grouping of correlated harmonics present in the mixed speech signal is then carried out by using a new iterative graph cut method. The signals are reconstructed from the respective harmonic groups which represent individual speakers in the mixed speech signal. Spectrographic masks are then applied on the reconstructed signals to refine their perceptual quality. The quality of separated speech is evaluated using several objective and subjective criteria. Experiments on multi-speaker automatic speech recognition are conducted using mixed speech data from the GRID corpus. A cell phone based multimedia information retrieval system (MIRS) for multi-source meeting environments are also developed.