Single-channel Source Separation Research Articles

One to multiple mapping dual learning: Learning multiple signals from one mixture

Single channel blind source separation (SCBSS) refers to separating multiple sources from a mixture collected by a single sensor. Existing methods for SCBSS have limited performance in separating multiple sources and generalization. To address these problems, an algorithm is proposed in this paper to separate multiple sources from a mixture by designing a parallel dual generative adversarial network (PDualGAN) that can build the relationship between a mixture and the corresponding multiple sources to achieve one-to-multiple cross-domain mapping. This algorithm can be applied to a variety of mixtures including both instantaneous and convolutive mixtures. In addition, new datasets for single channel source separation are created which include the mixtures and corresponding sources for this study. Experiments were performed on four different datasets including both one-dimensional and two-dimensional signals. Experimental results show that the proposed algorithm outperforms state-of-the-art algorithms, measured with peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), source-to-distortion ratio (SDR), source-to-interferences ratio (SIR), relative root mean squared error (RRMSE) and correlation.

SFSRNet: Super-resolution for Single-Channel Audio Source Separation

The problem of single-channel audio source separation is to recover (separate) multiple audio sources that are mixed in a single-channel audio signal (e.g. people talking over each other). Some of the best performing single-channel source separation methods utilize downsampling to either make the separation process faster or make the neural networks bigger and increase accuracy. The problem concerning downsampling is that it usually results in information loss. In this paper, we tackle this problem by introducing SFSRNet which contains a super-resolution (SR) network. The SR network is trained to reconstruct the missing information in the upper frequencies of the audio signal by operating on the spectrograms of the output audio source estimations and the input audio mixture. Any separation method where the length of the sequence is a bottleneck in speed and memory can be made faster or more accurate by using the SR network. Based on the WSJ0-2mix benchmark where estimations of the audio signal of two speakers need to be extracted from the mixture, in our experiments our proposed SFSRNet reaches a scale-invariant signal-to-noise-ratio improvement (SI-SNRi) of 24.0 dB outperforming the state-of-the-art solution SepFormer which reaches an SI-SNRi of 22.3 dB.

Single channel source separation using time–frequency non-negative matrix factorization and sigmoid base normalization deep neural networks

Single channel source separation using time–frequency non-negative matrix factorization and sigmoid base normalization deep neural networks

Learning a discriminative high-fidelity dictionary for single channel source separation

Sparse-representation-based single-channel source separation, which aims to recover each source's signal using its corresponding sub-dictionary, has attracted many scholars' attention. The basic premise of this model is that each sub-dictionary possesses discriminative information about its corresponding source, and this information can be used to recover almost every sample from that source. However, in a more general sense, the samples from a source are composed not only of discriminative information but also common information shared with other sources. This paper proposes learning a discriminative high-fidelity dictionary to improve the separation performance. The innovations are threefold. Firstly, an extra sub-dictionary was combined into a conventional union dictionary to ensure that the source-specific sub-dictionaries can capture only the purely discriminative information for their corresponding sources because the common information is collected in the additional sub-dictionary. Secondly, a task-driven learning algorithm is designed to optimize the new union dictionary and a set of weights that indicate how much of the common information should be allocated to each source. Thirdly, a source separation scheme based on the learned dictionary is presented. Experimental results on a human speech dataset yield evidence that our algorithm can achieve better separation performance than either state-of-the-art or traditional algorithms.

Signal-to-signal neural networks for improved spike estimation from calcium imaging data.

Spiking information of individual neurons is essential for functional and behavioral analysis in neuroscience research. Calcium imaging techniques are generally employed to obtain activities of neuronal populations. However, these techniques result in slowly-varying fluorescence signals with low temporal resolution. Estimating the temporal positions of the neuronal action potentials from these signals is a challenging problem. In the literature, several generative model-based and data-driven algorithms have been studied with varied levels of success. This article proposes a neural network-based signal-to-signal conversion approach, where it takes as input raw-fluorescence signal and learns to estimate the spike information in an end-to-end fashion. Theoretically, the proposed approach formulates the spike estimation as a single channel source separation problem with unknown mixing conditions. The source corresponding to the action potentials at a lower resolution is estimated at the output. Experimental studies on the spikefinder challenge dataset show that the proposed signal-to-signal conversion approach significantly outperforms state-of-the-art-methods in terms of Pearson’s correlation coefficient, Spearman’s rank correlation coefficient and yields comparable performance for the area under the receiver operating characteristics measure. We also show that the resulting system: (a) has low complexity with respect to existing supervised approaches and is reproducible; (b) is layer-wise interpretable, and (c) has the capability to generalize across different calcium indicators.

Sampled and discretized of short-time Fourier transform and non-negative matrix factorization: the single-channel source separation case

The Short-time Fourier transform (STFT) is a popular time-frequency representation in many source separation problems. In this work, the sampled and discretized version of Discrete Gabor Transform (DGT) is proposed to replace STFT within the single-channel source separation problem of the Non-negative Matrix Factorization (NMF) framework. The result shows that NMF-DGT is better than NMF-STFT according to Signal-to-Interference Ratio (SIR), Signal-to-Artifact Ratio (SAR), and Signal-to-Distortion Ratio (SDR). In the supervised scheme, NMF-DGT has a SIR of 18.60 dB compared to 16.24 dB in NMF-STFT, SAR of 13.77 dB to 13.69 dB, and SDR of 12.45 dB to 11.16 dB. In the unsupervised scheme, NMF-DGT has a SIR of 0.40 dB compared to 0.27 dB by NMF-STFT, SAR of -10.21 dB to -10.36 dB, and SDR of -15.01 dB to -15.23 dB.

Single-channel blind source separation based on attentional generative adversarial network

Blind single-channel source separation is a long-standing machine learning and signal processing problem. Traditional blind source separation (BSS) algorithms were proposed to solve this task utilizing multiple signal constraints. Generative adversarial network (GAN) are free from statistical constraints and samples, but the role of adversarial training in the task of BSS has not been fully demonstrated. Therefore, a new separation network model that enables to learn the known separated signal distribution from the stepwise fine estimation of the unknown mixture distribution was presented in this paper, and a self-attention mechanism was introduced to solve the problem of the blurring details of the generated image by the generator which preserves image details in the process of image separation. Compared with the existing single-channel blind source separation algorithm based on generative adversarial network-neural egg separation (NES), the detailed information of this new separation algorithm is more prominent, and the source signal in the mixed image has been separated more effectively, and has better separation performance than the classic blind source separation algorithms.

Single Channel Source Separation with ICA-Based Time-Frequency Decomposition.

This paper relates to the separation of single channel source signals from a single mixed signal by means of independent component analysis (ICA). The proposed idea lies in a time-frequency representation of the mixed signal and the use of ICA on spectral rows corresponding to different time intervals. In our approach, in order to reconstruct true sources, we proposed a novelty idea of grouping statistically independent time-frequency domain (TFD) components of the mixed signal obtained by ICA. The TFD components are grouped by hierarchical clustering and k-mean partitional clustering. The distance between TFD components is measured with the classical Euclidean distance and the distance of Gaussian distribution introduced by as. In addition, the TFD components are grouped by minimizing the negentropy of reconstructed constituent signals. The proposed method was used to separate source signals from single audio mixes of two- and three-component signals. The separation was performed using algorithms written by the authors in Matlab. The quality of obtained separation results was evaluated by perceptual tests. The tests showed that the automated separation requires qualitative information about time-frequency characteristics of constituent signals. The best separation results were obtained with the use of the distance of Gaussian distribution, a distance measure based on the knowledge of the statistical nature of spectra of original constituent signals of the mixed signal.

Online/offline score informed music signal decomposition: application to minus one

In this paper, we propose a score-informed source separation framework based on non-negative matrix factorization (NMF) and dynamic time warping (DTW) that suits for both offline and online systems. The proposed framework is composed of three stages: training, alignment, and separation. In the training stage, the score is encoded as a sequence of individual occurrences and unique combinations of notes denoted as score units. Then, we proposed a NMF-based signal model where the basis functions for each score unit are represented as a weighted combination of spectral patterns for each note and instrument in the score obtained from a trained a priori over-completed dictionary. In the alignment stage, the time-varying gains are estimated at frame level by computing the projection of each score unit basis function over the captured audio signal. Then, under the assumption that only a score unit is active at a time, we propose an online DTW scheme to synchronize the score information with the performance. Finally, in the separation stage, the obtained gains are refined using local low-rank NMF and the separated sources are obtained using a soft-filter strategy. The framework has been evaluated and compared with other state-of-the-art methods for single channel source separation of small ensembles and large orchestra ensembles obtaining reliable results in terms of SDR and SIR. Finally, our method has been evaluated in the specific task of acoustic minus one, and some demos are presented.

Single-channel source separation and parameters estimation of multi-component BPSK/QPSK signal based on 3-D EVR spectrum and wavelet analysis

This paper proposes an approach of single-channel source separation and parameter estimation of multi-component rectangular pulse-shaped BPSK/QPSK (binary/quadrature phase-shift keying) signal. By structuring multi-dimensional matrix from the observed signal, we may then apply 3-D EVR spectrum to determine the delay time and symbol period and therefore to estimate the basic waveform. A cost function is designed to estimate the carrier frequency and initial phase, and the periodic ambiguity problem is solved through wavelet analysis to get accurate symbol period. Finally, the component signal can be reconstructed with estimated parameters, and the separation stops adaptively by the SNR value which is estimated with the method of subspace-based decomposition. Simulation results confirmed the effectiveness of the proposed method.

Detection of Multiple Movers Based on Single Channel Source Separation of Their Micro-Dopplers

Studies have demonstrated the usefulness of micro-Doppler signatures for classifying dynamic radar targets such as humans, helicopters, and wind turbines. However, these classification works are based on the assumption that the propagation channel consists of only a single moving target. When multiple targets move simultaneously in the channel, the micro-Dopplers, in their radar backscatter, superimpose thereby distorting the signatures. In this paper, we propose a method to detect multiple targets that move simultaneously in the propagation channel. We first model the micro-Doppler radar signatures of different movers using dictionary learning techniques. Then, we use a sparse coding algorithm to separate the aggregate radar backscatter signal from multiple targets into their individual components. We demonstrate that the disaggregated signals are useful for accurately detecting multiple targets.

Audio signal separation through complex tensor factorization: Utilizing modulation frequency and phase information

I propose a complex-valued tensor factorization algorithm for audio-source separation to exploit not only amplitude but phase information of audio signals in the modulation frequency (MF) domain. The proposed algorithm is extended from complex non-negative matrix factorization, which is capable of decomposing an arbitrary complex matrix such as the complex spectrum in the acoustic frequency domain. The proposed method enables us to factorize an arbitrary complex tensor of order 3. The detailed performance of the proposed algorithm for single-channel source separation is investigated through numerical experiments. I examine the quantitative contributions of the MF domain and phase information examined by additionally presenting three tensor factorization algorithms and using five objective indices for source separation.

Impact of phase estimation on single-channel speech separation based on time-frequency masking.

Time-frequency masking is a common solution for the single-channel source separation (SCSS) problem where the goal is to find a time-frequency mask that separates the underlying sources from an observed mixture. An estimated mask is then applied to the mixed signal to extract the desired signal. During signal reconstruction, the time-frequency-masked spectral amplitude is combined with the mixture phase. This article considers the impact of replacing the mixture spectral phase with an estimated clean spectral phase combined with the estimated magnitude spectrum using a conventional model-based approach. As the proposed phase estimator requires estimated fundamental frequency of the underlying signal from the mixture, a robust pitch estimator is proposed. The upper-bound clean phase results show the potential of phase-aware processing in single-channel source separation. Also, the experiments demonstrate that replacing the mixture phase with the estimated clean spectral phase consistently improves perceptual speech quality, predicted speech intelligibility, and source separation performance across all signal-to-noise ratio and noise scenarios.

Non-negative matrix factorization for speech/music separation using source dependent decomposition rank, temporal continuity term and filtering

Abstract Non-negative matrix factorization (NMF) is a recently well-known method for separating speech from music signal as a single channel source separation problem. In this approach, spectrogram of each source signal is factorized as a multiplication of two matrices known as basis and weight matrices. To obtain a good estimation of signal spectrogram, weight and basis matrices are updated based on a cost function, iteratively. In standard NMF, each frame of signal is considered as an independent observation and this assumption is a drawback for NMF. For overcoming this weakness, a regularization term is added to the cost function to consider spectral temporal continuity. Furthermore, in the standard NMF, the same decomposition rank is usually used for different sources. In this paper, in accompany with using a regularization term, we propose to apply a filter to the signals estimated by NMF. The filter is constructed by signals which are estimated using a regularized NMF method. Moreover, we propose to use different decomposition ranks for speech and music signals as different sources. Experimental results on one hour of speech and music signals show that the proposed method increases signal to inference ratio (SIR) values for speech and music signals in comparison to conventional NMF methods.

Single channel source separation using graph sparse NMF and adaptive dictionary learning

Single channel source separation using graph sparse NMF and adaptive dictionary learning

Blind Separation of Audio Mixtures Through Nonnegative Tensor Factorization of Modulation Spectrograms

This paper presents an algorithm for unsupervised single-channel source separation of audio mixtures. The approach specifically addresses the challenging case of separation where no training data are available. By representing mixtures in the modulation spectrogram (MS) domain, we exploit underlying similarities in patterns present across frequency. A three-dimensional tensor factorization is able to take advantage of these redundant patterns, and is used to separate a mixture into an approximated sum of components by minimizing a divergence cost. Furthermore, we show that the basic tensor factorization can be extended with convolution in time being used to improve separation results and provide update rules to learn components in such a manner. Following factorization, sources are reconstructed in the audio domain from estimated components using a novel approach based on reconstruction masks that are learned using MS activations, and then applied to a mixture spectrogram. We demonstrate that the proposed method produces superior separation performance to a spectrally based nonnegative matrix factorization approach, in terms of source-to-distortion ratio. We also compare separation with the perceptually motivated interference-related perceptual score metric and identify cases with higher performance.

Single-channel source separation of multi-component radar signal based on EVD and ICA

This study explored a novel method based on eigenvalue decomposition (EVD) and independent component analysis (ICA) to separate the multi-component radar signal in the single channel. By exploiting the generalized periodicity of the radar signal, the proposed method structures the multi-dimensional matrix from the observed signal in single-channel through EVD, then applies ICA to the matrix to determine the basic waveform of each component, and finally reconstructs the component signals. Simulation results confirmed the effectiveness of the proposed method and compared it with other methods, although the performance of proposed approach is a bit worse than some other method when processing radar signals, the most outstanding advantage of the proposed approach is that it does not require any other known conditions, and it can recover the component signals with a satisfactory level, so it can yet be regarded as an effective method.

Exploiting cyclostationary difference in anti-jamming method based on single channel source separation

Traditional time/frequency domain filter-based methods would fail due to the serious overlapping of the communication signal and jamming in time/frequency domain. In order to solve this problem, an anti-jamming algorithm based on single channel source separation by exploiting cyclostationary difference is proposed for direct sequence spread spectrum (DSSS) communication. Firstly, the cyclostationary difference among source signals is used to establish the single channel source separation model based on partial priori information of the DSSS signal. Secondly, cost function of the algorithm is constructed by the training sequences and the minimum mean square error criterion; after that, the optimal separation matrix of the pilot mixed signal can be calculated based on this cost function. Thirdly, the purpose of anti-jamming for DSSS communication can be achieved by separating the communication signal from the service mixed signal based on the optimal separation matrix calculated before. Simulation results show that, compared with the unseparated signal of high bit error rate (BER) close to 0.5, the proposed antijamming algorithm is more effective in anti-jamming with the BER as low as 10–3, which is not only applicable to narrowband jamming such as the single-tone/multi-tone jamming, but also applicable to broadband jamming such as the sweep-spot jamming.

Online Noisy Single-Channel Source Separation Using Adaptive Spectrum Amplitude Estimator and Masking

A novel single-channel source separation method is presented to recover the original signals given only a single observed mixture in noisy environment. The proposed separation method is an online adaptive process and independent of parameters initialization. In this paper, a noisy pseudo-stereo mixing model is developed by formulating an artificial mixture from the observed mixture where the signals are modeled by the autoregressive process. The proposed demixing process composes of two steps: First, the noisy mixing model is enhanced by selecting the time-frequency (TF) units of signal presence and computing the mixture spectral amplitude, and second, an adaptive estimation of the parameters associated with each source is computed frame-by-frame, which is then used to construct a TF mask for the separation process. To assess the performance of the proposed method, noisy mixtures of real-audio sources with nonstationary noise have been conducted under various SNRs. Experiments show that the proposed algorithm has yielded superior separation performance especially in low input SNR compared with existing methods.

Representation Learning for Single-Channel Source Separation and Bandwidth Extension

In this paper, we use deep representation learning for model-based single-channel source separation (SCSS) and artificial bandwidth extension (ABE). Both tasks are ill-posed and source-specific prior knowledge is required. In addition to well-known generative models such as restricted Boltzmann machines and higher order contractive autoencoders two recently introduced deep models, namely generative stochastic networks (GSNs) and sum-product networks (SPNs), are used for learning spectrogram representations. For SCSS we evaluate the deep architectures on data of the 2 $^{\rm nd}$ CHiME speech separation challenge and provide results for a speaker dependent, a speaker independent, a matched noise condition and an unmatched noise condition task. GSNs obtain the best PESQ and overall perceptual score on average in all four tasks. Similarly, frame-wise GSNs are able to reconstruct the missing frequency bands in ABE best, measured in frequency-domain segmental SNR. They outperform SPNs embedded in hidden Markov models and the other representation models significantly.