Source Separation Performance Research Articles

Application of Unet-SE-Bisru Algorithm for Music Signal Processing in Music Source Separation

At present, the use of time-domain deep learning for end-to-end neural network models has the problem of long training time and poor performance in music source separation. To address this issue, a U-network squeezing excitation bidirectional simple recursive unit model was proposed based on the deep extractor model. Replace Unet SE Bisru with Unet SE Bisru in the following text. This model improves the bidirectional long short-term memory network into a bidirectional simple recurrent unit, and then introduces attention mechanisms in the generalized encoding and decoding layers. The squeezing excitation block is used to selectively extract features based on the type of audio to be separated. Finally, group normalization is added after onedimensional convolution, And its effectiveness was verified. The experimental results show that the signal noise distortion ratio in the improved model is 5.68 decibels compared to the bidirectional simple recursive unit value, which is higher than the 5.55 decibels of bidirectional long short-term memory. After adding the squeezing excitation module, the overall increase is about 0.1-0.5 decibels. In addition, in the model comparison, the three indicators of the improved model with the same number of channels were 5.68 decibels, 5.91 decibels, and 11.28 decibels, respectively, higher than the benchmark model. Compared with other music source classification models, the improved model has better comprehensive separation performance. Although some indicators are lower than the comparison model, the signal noise distortion ratio of drum and bass is 6.11 decibels and 6.36 decibels, which is better than the comparison model. Overall, the improved model has high performance in music source separation for music signal processing and can be effectively applied in practical music source separation.

Research on Mixed Matrix Estimation Algorithm Based on Improved Sparse Representation Model in Underdetermined Blind Source Separation System

The estimation accuracy of the mixed matrix is very important to the performance of the underdetermined blind source separation (UBSS) system. To improve the estimation accuracy of the mixed matrix, the sparsity of the mixed signal is required. The novel fractional domain time–frequency plane is obtained by rotating the time–frequency plane after the short-time Fourier transform. This plane represents the fine characteristics of the mixed signal in the time domain and the frequency domain. The rotation angle is determined by global searching for the minimum L1 norm to make the mixed signal sufficiently sparse. The obtained time–frequency points do not need single source point detection, reducing the calculation amount of the original algorithm, and the insensitivity to noise in the fractional domain improves the robustness of the algorithm in the noise environment. The simulation results show that the sparsity of the mixed signal and the estimation accuracy of the mixed matrix are improved. Compared with the existing mixed matrix estimation algorithms, the proposed method is effective.

FastMVAE2: On Improving and Accelerating the Fast Variational Autoencoder-Based Source Separation Algorithm for Determined Mixtures

This article proposes a new source model and training scheme to improve the accuracy and speed of the multichannel variational autoencoder (MVAE) method. The MVAE method is a recently proposed powerful multichannel source separation method. It consists of pretraining a source model represented by a conditional VAE (CVAE) and then estimating separation matrices along with other unknown parameters so that the log-likelihood is non-decreasing given an observed mixture signal. Although the MVAE method has been shown to provide high source separation performance, one drawback is the computational cost of the backpropagation steps in the separation-matrix estimation algorithm. To overcome this drawback, a method called “FastMVAE” was subsequently proposed, which uses an auxiliary classifier VAE (ACVAE) to train the source model. By using the classifier and encoder trained in this way, the optimal parameters of the source model can be inferred efficiently, albeit approximately, in each step of the algorithm. However, the generalization capability of the trained ACVAE source model was not satisfactory, which led to poor performance in situations with unseen data. To improve the generalization capability, this article proposes a new model architecture (called the “ChimeraACVAE” model) and a training scheme based on knowledge distillation. The experimental results revealed that the proposed source model trained with the proposed loss function achieved better source separation performance with less computation time than FastMVAE. We also confirmed that our methods were able to separate 18 sources with a reasonably good accuracy.

Music Source Separation With Band-Split RNN

The performance of music source separation (MSS) models has been greatly improved in recent years thanks to the development of novel neural network architectures and training pipelines. However, recent model designs for MSS were mainly motivated by other audio processing tasks or other research fields, such as speech enhancement and speech separation, while the intrinsic characteristics and patterns of the music signals were not fully discovered. In this paper, we propose band-split RNN (BSRNN), a frequency-domain model that explictly splits the spectrogram of the mixture into subbands and perform interleaved band-level and sequence-level modeling. The choices of the bandwidths of the subbands can be determined by a priori knowledge or expert knowledge on the characteristics of the target source in order to optimize the performance on a certain type of target musical instrument. To better make use of unlabeled data, we also describe a semi-supervised model finetuning pipeline that can further improve the performance of the model. Experiment results show that BSRNN trained only on MUSDB18-HQ dataset significantly outperforms several top-ranking models in Music Demixing (MDX) Challenge 2021, and the semi-supervised finetuning stage further improves the performance on all four instrument tracks.

Combined LOFAR and DEMON Spectrums for Simultaneous Underwater Acoustic Object Counting and F0 Estimation

In a typical underwater acoustic target detection mission, we have to estimate the target number (N), perform source separation when N>1, and consequently predict the motion parameters such as fundamental frequency (F0) from separated noises for each target. Although deep learning methods have been adopted in each task, their successes strongly depend on the feed-in features. In this paper, we evaluate several time-frequency features and propose a universal feature extraction strategy for object counting and F0 estimation simultaneously, with a convolutional recurrent neural network (CRNN) as the backbone. On one hand, LOFAR and DEMON are feasible for low-speed and high-speed analysis, respectively, and are combined (LOFAR + DEMON) to cope with full-condition estimation. On the other hand, a comb filter (COMB) is designed and applied to the combined spectrum for harmonicity enhancement, which will be further streamed into the CRNN for prediction. Experiments show that (1) in the F0 estimation task, feeding the filtered combined feature (LOFAR + DEMON + COMB) into the CRNN achieves an accuracy of 98% in the lake trial dataset, which is superior to LOFAR + COMB (83%) or DEMON + COMB (94%) alone, demonstrating that feature combination is plausible. (2) In a counting task, the prediction accuracy of the combined feature (LOFAR + DEMON, COMB included or excluded) is comparable to the state-of-the-art on simulation dataset and dominates the rest on the lake trial dataset, indicating that LOFAR + DEMON can be used as a common feature for both tasks. (3) The inclusion of COMB accelerates the convergence speed of the F0 estimation task, however, it penalizes the counting task by a depression of 13% on average, partly due to the merging effects brought in by the broadband filtering of COMB.

Using Deep Neural Networks for On-Load Tap Changer Audio-Based Diagnostics

This paper proposes a sound separation methodology based on deep learning neural network (DLNN) to extract useful diagnostic material in non-invasive audio-based On-Load Tap Changer (OLTC) diagnostics. The proposed methodology has been experimentally verified on both artificial mixtures (created by reproducing the targeted data by the speakers placed next to the active transformers) and actual mixtures (recorded by the microphone during OLTC live operation in the field). The results show that the method produces high-quality estimates (correlation to referent fingerprints <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rho &gt; 0.9$</tex-math></inline-formula> ) compared to other sound separation methods, e.g. Independent Component Analysis (ICA). The proposed framework can also perform source separation from a monaural mixture (mixture recorded with a single microphone only), which is impossible for ICA methods. Moreover, the results show that DLNN trained with healthy OLTC data produces diagnostically valuable estimates even when fed with a faulty OLTC audio mixture. For that reason, once trained, the DLNN can produce the diagnostic signal estimates from monaural mixtures that can be used with existing vibration-based diagnostic methods.

ODAS: Open embeddeD Audition System.

Artificial audition aims at providing hearing capabilities to machines, computers and robots. Existing frameworks in robot audition offer interesting sound source localization, tracking and separation performance, although involve a significant amount of computations that limit their use on robots with embedded computing capabilities. This paper presents ODAS, the Open embeddeD Audition System framework, which includes strategies to reduce the computational load and perform robot audition tasks on low-cost embedded computing systems. It presents key features of ODAS, along with cases illustrating its uses in different robots and artificial audition applications.

Music Demixing Challenge 2021

Music source separation has been intensively studied in the last decade and tremendous progress with the advent of deep learning could be observed. Evaluation campaigns such as MIREX or SiSEC connected state-of-the-art models and corresponding papers, which can help researchers integrate the best practices into their models. In recent years, the widely used MUSDB18 dataset played an important role in measuring the performance of music source separation. While the dataset made a considerable contribution to the advancement of the field, it is also subject to several biases resulting from a focus on Western pop music and a limited number of mixing engineers being involved. To address these issues, we designed the Music Demixing Challenge on a crowd-based machine learning competition platform where the task is to separate stereo songs into four instrument stems (Vocals, Drums, Bass, Other). The main differences compared with the past challenges are 1) the competition is designed to more easily allow machine learning practitioners from other disciplines to participate, 2) evaluation is done on a hidden test set created by music professionals dedicated exclusively to the challenge to assure the transparency of the challenge, i.e., the test set is not accessible from anyone except the challenge organizers, and 3) the dataset provides a wider range of music genres and involved a greater number of mixing engineers. In this paper, we provide the details of the datasets, baselines, evaluation metrics, evaluation results, and technical challenges for future competitions.

Blind Source Separation Based on Quantum Slime Mould Algorithm in Impulse Noise

In order to resolve engineering problems that the performance of the traditional blind source separation (BSS) methods deteriorates or even becomes invalid when the unknown source signals are interfered by impulse noise with a low signal-to-noise ratio (SNR), a more effective and robust BSS method is proposed. Based on dual-parameter variable tailing (DPVT) transformation function, moving average filtering (MAF), and median filtering (MF), a filtering system that can achieve noise suppression in an impulse noise environment is proposed, noted as MAF-DPVT-MF. A hybrid optimization objective function is designed based on the two independence criteria to achieve more effective and robust BSS. Meanwhile, combining quantum computation theory with slime mould algorithm (SMA), quantum slime mould algorithm (QSMA) is proposed and QSMA is used to solve the hybrid optimization objective function. The proposed method is called BSS based on QSMA (QSMA-BSS). The simulation results show that QSMA-BSS is superior to the traditional methods. Compared with previous BSS methods, QSMA-BSS has a wider applications range, more stable performance, and higher precision.

Single-Channel Blind Source Separation of Spatial Aliasing Signal Based on Stacked-LSTM.

Aiming at the problem of insufficient separation accuracy of aliased signals in space Internet satellite-ground communication scenarios, a stacked long short-term memory network (Stacked-LSTM) separation method based on deep learning is proposed. First, the coding feature representation of the mixed signal is extracted. Then, the long sequence input is divided into smaller blocks through the Stacked-LSTM network with the attention mechanism of the SE module, and the deep feature mask of the source signal is trained to obtain the Hadamard product of the mask of each source and the coding feature of the mixed signal, which is the encoding feature representation of the source signal. Finally, characteristics of the source signal is decoded by 1-D convolution to to obtain the original waveform. The negative scale-invariant source-to-noise ratio (SISNR) is used as the loss function of network training, that is, the evaluation index of single-channel blind source separation performance. The results show that in the single-channel separation of spatially aliased signals, the Stacked-LSTM method improves SISNR by 10.09∼38.17 dB compared with the two classic separation algorithms of ICA and NMF and the three deep learning separation methods of TasNet, Conv-TasNet and Wave-U-Net. The Stacked-LSTM method has better separation accuracy and noise robustness.

Joint-diagonalizability-constrained multichannel nonnegative matrix factorization based on time-variant multivariate complex sub-Gaussian distribution

Multichannel nonnegative matrix factorization (MNMF) is a common blind source separation technique that employs full-rank spatial covariance matrices (SCMs). The full-rank SCMs can simulate reverberant mixing systems where the sources are spatially spread. In conventional MNMF, spectrograms of observed signals are modeled by some types of distribution, e.g., the Gaussian distribution and Student’s t distribution. However, MNMF based on the sub-Gaussian distribution has not been proposed because its cost function is difficult to minimize. In this paper, we address the statistical model extension of MNMF to the sub-Gaussian distribution to improve the source separation accuracy. In the proposed method, the generalized Gaussian distribution is utilized as the sub-Gaussian model. Moreover, to design an auxiliary function for the proposed cost function, we introduce the joint-diagonalizability constraint to SCMs similarly to FastMNMF. Two types of update rule for the proposed MNMF are derived on the basis of the majorization-minimization (MM) and majorization-equalization (ME) algorithms. Since the optimization speed of each parameter affects the source separation performance, we experimentally analyze the best combination of MM- and ME-algorithm-based update rules in the proposed method. Experiments of blind source separation reveal that the proposed MNMF based on the sub-Gaussian model can outperform conventional methods.

Music Source Separation Using ASPP Based on Coupled U-Net Model

Noise has established itself as one of the factors that interfere with modern human life, and various noise canceling techniques have been studied to prevent noise. While the old era's noise-canceling technique focused on the physical soundproofing technique, multiple studies have been conducted on the active noise canceling technique that removes only the activated noise in the current era. Active noise canceling (ANC) or digital noise-canceling technology is based on the sound source separation method. This leads to sound source separation technology, which refers to the technology to separate individual sound signals from mixture sounds. Most of the source separation technologies focus on improving speech, not noise reduction. This technology makes it possible to obtain desired sound information more accurately and further improves noise-canceling technology by eliminating unwanted sound information. To provide deeper capability and more enhanced sound separation than the existing structure, we are focused on coupled U-Net model and Atrous spatial pyramid pooling technique (ASPP). This paper presents the music source separation method that combined Coupled U-Net structure with Atrous spatial pyramid pooling technique. To prove the proposed source separation method, we compared GNSDR, GSIR, and GSAR using MIR-1K, a data set that can evaluate the performance of the music source separation. Performance results show that the proposed source separation method overcame other methods' disadvantages and strengthened the feature map.

High resolution sub-band decomposition underdetermined blind signal separation using virtual sensor based ICA method for low latency applications

Underdetermined blind source separation (UBSS) objective is to recover the source signals from a number of mixtures without any information about the mixing system. Technologies such as teleconferencing, hearing aids and hands-free telephony require real time processing, as long delays are considered intolerable in interactive two-way communication. This is a major challenge in BSS, as algorithms generally require significant amounts of data (several seconds of data or more) to generate sufficient statistics for separation. In this paper, we propose a hybrid algorithm which combines sub-band decomposition and well-known Independent Component Analysis (ICA) based algorithm, Extended-Infomax. We first employ sub-band decomposition algorithm in sparse time domain to compensate low data efficiency of short time block lengths and estimate the mixing matrix. Subsequently, the proposed virtual sensor based underdetermined Extended-Infomax source model is used to estimate the source signals. As the separation evaluation results are so sensitive to the mixing matrix estimation outcome, achieving nearly optimum results in the first phase of the proposed algorithm significantly improves the results of source separation performance. A weighted version of k-plane clustering algorithm is derived to obtain the mixing coefficients. Experimental evaluations reveal the effectiveness of our proposed method over the state-of-the-art techniques.

Blind Source Separation Method Based on Neural Network with Bias Term and Maximum Likelihood Estimation Criterion.

Convergence speed and steady-state source separation performance are crucial for enable engineering applications of blind source separation methods. The modification of the loss function of the blind source separation algorithm and optimization of the algorithm to improve its performance from the perspective of neural networks (NNs) is a novel concept. In this paper, a blind source separation method, combining the maximum likelihood estimation criterion and an NN with a bias term, is proposed. The method adds L2 regularization terms for weights and biases to the loss function to improve the steady-state performance and designs a novel optimization algorithm with a dual acceleration strategy to improve the convergence speed of the algorithm. The dual acceleration strategy of the proposed optimization algorithm smooths and speeds up the originally steep, slow gradient descent in the parameter space. Compared with competing algorithms, this strategy improves the convergence speed of the algorithm by four times and the steady-state performance index by 96%. In addition, to verify the source separation performance of the algorithm more comprehensively, the simulation data with prior knowledge and the measured data without prior knowledge are used to verify the separation performance. Both simulation results and validation results based on measured data indicate that the new algorithm not only has better convergence and steady-state performance than conventional algorithms, but it is also more suitable for engineering applications.

Integrating Dilated Convolution into DenseLSTM for Audio Source Separation

Herein, we proposed a multi-scale multi-band dilated time-frequency densely connected convolutional network (DenseNet) with long short-term memory (LSTM) for audio source separation. Because the spectrogram of the acoustic signal can be thought of as images as well as time series data, it is suitable for convolutional recurrent neural network (CRNN) architecture. We improved the audio source separation performance by applying the dilated block with a dilated convolution to CRNN architecture. The dilated block has the role of effectively increasing the receptive field in the spectrogram. In addition, it was designed in consideration of the acoustic characteristics that the frequency axis and the time axis in the spectrogram are changed by independent influences such as speech rate and pitch. In speech enhancement experiments, we estimated the speech signal using various deep learning architectures from a signal in which the music, noise, and speech were mixed. We conducted the subjective evaluation on the estimated speech signal. In addition, speech quality, intelligibility, separation, and speech recognition performance were also measured. In music signal separation, we estimated the music signal using several deep learning architectures from the mixture of the music and speech signal. After that, the separation performance and music identification accuracy were measured using the estimated music signal. Overall, the proposed architecture shows the best performance compared to other deep learning architectures not only in speech experiments but also in music experiments.

Sparse Representations With Legendre Kernels for DOA Estimation and Acoustic Source Separation

Recording multiple sound sources in a reverberant environment results in convolutive mixtures. Sound sources can be extracted from microphone array recordings of such mixtures using acoustic source separation techniques. Acoustic source separation using recordings obtained from rigid spherical microphone arrays (RSMA) benefit from the representation of sound fields as series of spherical harmonics. More specifically, RSMAs afford increased flexibility in acoustic beamforming and spatial filtering. We propose a data-driven DOA estimation and acoustic source separation method based on a dictionary-based sparse decomposition of sound fields. The proposed method involves identifying the time-frequency bins with contributions from a single source only and those with sensor noise or diffuse sound field components. The former set of bins is used in DOA estimation and beamforming in the sparse decomposition domain. The latter set is used to calculate the diffuse field covariance matrix used in Wiener post-filtering to improve the source separation performance further. We demonstrate the utility of the proposed method via extensive objective and subjective evaluations.

Blind Separation for Multiple Moving Sources With Labeled Random Finite Sets

This paper proposes a novel solution for separating an unknown and time-varying number of moving acoustic sources in a blind setting using multiple microphone arrays. A standard steered-response power phase transform method is applied to extract source position measurements, which inevitably contain noise, false detections, missed detections, and are not labeled with the source identities. The imperfect measurements lead to the space-time permutation problem, as there is no information on how the measurements are associated to the sources in space, nor how the measurements are connected across time, if at all. To solve this problem, a labeled random finite set tracking framework is adopted to jointly estimate the source positions and their labels or identities. Based on these trajectory estimates, a corresponding set of time-varying generalized side-lobe cancellers is constructed to perform source separation. The overall algorithm operates in a block-wise or an online fashion and is scalable with the number of microphone arrays. The quality of the measurements, tracking, and separation, are evaluated respectively, with the OSPA metric, OSPA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(2)</sup> metric, and ITU-T P.835 based listening tests, on both real-world and simulated data.

Analytical Comparison of Classification Models for Raga Identification in Carnatic Classical Instrumental Polyphonic Audio

“MUSIC is the divine way of portraying the most beautiful about this world”. With that being said, the diversity in this language of music is immense, to say the least. Broadly, one would be well aware of the classification between Indian classical music and western music. In music Information Retrieval (MIR), raga classification has a tremendous role in understanding the fundamentals of Indian classical music and in a multitude of other tasks like database organisation of music files to music recommendation systems. The paper incorporates a variety of techniques like ANN, CNN, Bi LSTM and XGBoost models for the task of Raga Identification from a Carnatic Classical Instrumental audio (CCIA). The work is initially carried out on a set of 10 ragas and then extended to largely available 15 ragas of the dataset. The data samples for the same were obtained from the standard data set. This task showed state-of-the-art results with an accuracy of 97% for a set of 15 Ragas. The astounding results were obtained without performing source separation on the musical audio track. The process was carried out on the Ragas pertaining to Carnatic Classical music, a division of Indian classical music.

FastMVAE: A Fast Optimization Algorithm for the Multichannel Variational Autoencoder Method

This paper proposes a fast optimization algorithm for the multichannel variational autoencoder (MVAE) method, a recently proposed powerful multichannel source separation technique. The MVAE method can achieve good source separation performance thanks to a convergence-guaranteed optimization algorithm and the idea of jointly performing multi-speaker separation and speaker identification. However, one drawback is the high computational cost of the optimization algorithm. To overcome this drawback, this paper proposes using an auxiliary classifier VAE, an information-theoretic extension of the conditional VAE (CVAE), to train the generative model of the source spectrograms and using it to efficiently update the parameters of the source spectrogram models at each iteration of the source separation algorithm. We call the proposed algorithm “FastMVAE” (or fMVAE for short). Experimental evaluations revealed that the proposed fast algorithm can achieve high source separation performance in both speaker-dependent and speaker-independent scenarios while significantly reducing the computational time compared to the original MVAE method by more than 90% on both GPU and CPU. However, there is still room for improvement of about 3 dB compared to the original MVAE method.

An Adaptive Algorithm for Calculating Crosstalk Error for Blind Source Separation

The crosstalk error is widely used to evaluate the performance of blind source separation. However, it needs to know the global separation matrix in advance, and it is not robust. In order to solve these problems, a new adaptive algorithm for calculating crosstalk error is presented, which calculates the crosstalk error by a cost function of least squares criterion, and the robustness of the crosstalk error is improved by introducing the position information of the maximum value in the global separation matrix. Finally, the method is compared with the conventional RLS algorithms in terms of performance, robustness and convergence rate. Furthermore, its validity is verified by simulation experiments and real world signals experiments.