Demixing Matrix Research Articles

Dimension reduction as a deflation method in ICA

In independent component analysis (ICA), when using the one-unit contrast function optimization approach to estimate independent components one by one, the constraint of uncorrelatedness between independent components prevents the algorithm from converging to the previously found components. A popular way to achieve uncorrelatedness is the Gram-Schmidt-like decorrelation scheme. In fact, uncorrelatedness between independent components can be achieved by reducing the degree of freedom in the unknown parameter set of the de-mixing matrix. In this letter, we propose to exploit the dimension-reduction technique to exactly enforce uncorrelatedness between difference independent components. The advantage of this method is that dimension reduction of the observations and de-mixing weight vectors makes the computation complexity lower and produces a faster convergence. Hence, our method results in a faster algorithm in computation of ICA.

Fetal heart rate monitoring from maternal body surface potentials using independent component analysis

ABSTRACTThe fetal heart rate is indispensable for monitoring the health of unborn cattle fetuses. To monitor the fetal heart rate, a method employing independent component analysis (ICA) to extract the fetal electrocardiogram (fECG) from potentials measured on the maternal body surface and composed of a mixture of the maternal ECG (mECG), fECG, baseline drift and noise is described. A mixing of the raw data was simplified using a linear time‐invariant model. To separate the fECG from the mECG, baseline drift, and noise, an ICA strategy was applied, using a hyperbolic tangent as the contrast function and treating mutual information with the minimization principle to find the optimum demixing matrix to derive the fECG from the measured signals. After the feasibility of this method was shown on simulated signals obtained by randomly mixing pure fECG, pure mECG, low frequency sinusoidal drift and noise, real signals from three cloned pregnant Holstein cows with 157, 177 and 224‐day gestation periods were used to verify the separation method. The results show that the fECG, mECG, low‐frequency sinusoidal drift and noise can be clearly segregated in simulations, and that the fECG, mECG, baseline drift and noise can be successfully derived from real signals. The ICA approach has great potential in effectively detecting the fECG from maternal body surface potentials.

BLIND SEPARATION OF MIXED KURTOSIS SIGNED SIGNALS USING PARTIAL OBSERVATIONS AND LOW COMPLEXITY ACTIVATION FUNCTIONS

Although several highly accurate blind source separation algorithms have already been proposed in the literature, these algorithms must store and process the whole data set which may be tremendous in some situations. This makes the blind source separation infeasible and not realisable on VLSI level, due to a large memory requirement and costly computation. This paper concerns the algorithms for solving the problem of tremendous data sets and high computational complexity, so that the algorithms could be run on-line and implementable on VLSI level with acceptable accuracy. Our approach is to partition the observed signals into several parts and to extract the partitioned observations with a simple activation function performing only the "shift-and-add" micro-operation. No division, multiplication and exponential operations are needed. Moreover, obtaining an optimal initial de-mixing weight matrix for speeding up the separating time will be also presented. The proposed algorithm is tested on some benchmarks available online. The experimental results show that our solution provides comparable efficiency with other approaches, but lower space and time complexity.

A novel adaptive learning rate sequential blind source separation algorithm

A new member of the family of natural gradient algorithms for on-line blind separation of independent sources is proposed. The method is based upon an adaptive step-size which varies in sympathy with the dynamics of the input signals and properties of the de-mixing matrix, and is robust to the perturbations in the initial value of the learning rate parameter. As a result, the convergence speed is significantly improved, especially in non-stationary mixing environments. Simulations support the expected improvement in convergence speed of the approach.

Eliminating indeterminacy in ICA

This paper presents a method to eliminate inherent indeterminacy of permutation and dilation existing in the classical independent component analysis (ICA). The method incorporates additional requirements or a priori information as constraints in the ICA contrast function. We illustrate how this approach sorts independent components (ICs) according to some statistic and normalizes the demixing matrix or the energies of separated ICs. With some prior knowledge, the algorithm is able to identify and extract the original sources perfectly from their mixtures. The experiments with simulated random signals and real audio signals demonstrate the versatility of eliminating indeterminacy in the ICA. An application separating functional magnetic resonance imaging (fMRI) data into activation maps ordered according to their sparseness is also presented.

Blind source separation using Renyi's α-marginal entropies

We have recently suggested the minimization of a nonparametric estimator of Renyi's mutual information as a criterion for blind source separation. Using a two-stage topology, consisting of spatial whitening and a series of Givens rotations, the cost function reduces to the sum of marginal entropies, just like in the Shannon's entropy case. Since we use a Parzen window density estimator and eliminate the joint entropy by employing an orthonormal demixing matrix, the problems of probability density function inaccuracy due to truncation of series expansion and the estimation of joint pdfs in high-dimensional spaces (given the typical paucity of data) are avoided, respectively. In our previous formulation, the algorithm was restricted to Renyi's second-order entropy and Gaussian kernels for the Parzen window estimator. The present work extends the previous results by formulating a new estimation methodology for Renyi's entropy, which allows the designer to choose any order of entropy and any suitable kernel function. Simulations illustrate that the proposed method compares favorably to Hyvarinen's FastICA, Bell and Sejnowski's Infomax and Common's minimum of mutual information.

Second Order Nonstationary Source Separation

This paper addresses a method of blind source separation that jointly exploits the nonstationarity and temporal structure of sources. The method needs only multiple time-delayed correlation matrices of the observation data, each of which is evaluated at different time-windowed data frame, to estimate the demixing matrix. The method is insensitive to the temporally white noise since it is based on only time-delayed correlation matrices (with non-zero time-lags) and is applicable to the case of either nonstationary sources or temporally correlated sources. We also discuss the extension of some existing methods with the overview of second-order blind source separation methods. Extensive numerical experiments confirm the validity and high performance of the proposed method.

Independent component analysis using Potts models

We explore the extending application of Potts encoding to the task of independent component analysis, which primarily deals with the problem of minimizing the Kullback-Leibler divergence between the joint distribution and the product of all marginal distributions of output components. The competitive mechanism of Potts neurons is used to encode the overlapping projections from observations to output components. Based on these projections, the marginal distributions and the entropy of output components are made tractable for computation and the adaptation of the de-mixing matrix toward independent output components is obtained. The Potts model for ICA is well formulated by an objective function subject to a set of constraints, which leads to a novel energy function. A hybrid of the mean field annealing and the gradient descent method is applied to the energy function. Our approach to independent component analysis presents a new criterion for ICA. The performance of the Potts model for ICA given by our numerical simulations is encouraging.

10.1162/jmlr.2003.4.7-8.1393

The learning dynamics of on-line independent component analysis is analysed in the limit of large data dimension. We study a simple Hebbian learning algorithm that can be used to separate out a small number of non-Gaussian components from a high-dimensional data set. The de-mixing matrix parameters are confined to a Stiefel manifold of tall, orthogonal matrices and we introduce a natural gradient variant of the algorithm which is appropriate to learning on this manifold. For large input dimension the parameter trajectory of both algorithms passes through a sequence of unstable fixed points, each described by a diffusion process in a polynomial potential. Choosing the learning rate too large increases the escape time from each of these fixed points, effectively trapping the learning in a sub-optimal state. In order to avoid these trapping states a very low learning rate must be chosen during the learning transient, resulting in learning time-scales of O(N2) or O(N3) iterations where N is the data dimension. Escape from each sub-optimal state results in a sequence of symmetry breaking events as the algorithm learns each source in turn. This is in marked contrast to the learning dynamics displayed by related on-line learning algorithms for multilayer neural networks and principal component analysis. Although the natural gradient variant of the algorithm has nice asymptotic convergence properties, it has an equivalent transient dynamics to the standard Hebbian algorithm.

Neural network for demixing super-Gaussian signals

A new method for separating linear mixtures of statistically independent signals with super-Gaussian probability distributions, using a simple neural network, is proposed. The procedure is based on geometric properties, and it is shown that the maxima of the mixed density distribution belong to straight lines, the direction vectors of which, when taken as columns of a matrix, comprise a demixing matrix. The results obtained with synthetic mixtures of real speech signals are shown.

Self-stabilized gradient algorithms for blind source separation with orthogonality constraints

Recently, developments in self-stabilized algorithms for gradient adaptation of orthonormal matrices have resulted in simple but powerful principal and minor subspace analysis methods. In this paper, we extend these ideas to develop algorithms for instantaneous prewhitened blind separation of homogeneous signal mixtures. Our algorithms are proven to be self-stabilizing to the Stiefel manifold of orthonormal matrices, such that the rows of the adaptive demixing matrix do not need to be periodically reorthonormalized. Several algorithm forms are developed, including those that are equivariant with respect to the prewhitened mixing matrix. Simulations verify the excellent numerical properties of the proposed methods for the blind source separation task.

Robust techniques for independent component analysis (ICA) with noisy data

In this contribution, we propose approaches to independent component analysis (ICA) when the measured signals are contaminated by additive noise. We extend existing adaptive algorithms with equivariant properties in order to considerably reduce the bias in the demixing matrix caused by measurement noise. Moreover, we describe a novel recurrent dynamic neural network for simultaneous estimation of the unknown mixing matrix, blind source separation, and reduction of noise in the extracted output signals. We discuss the optimal choice of nonlinear activation functions for various noise distributions assuming a generalized Gaussian-distributed noise model. Computer simulations of a selected approach are provided that confirm its usefulness and excellent performance.

Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information

There are two major approaches for blind separation: maximum entropy (ME) and minimum mutual information (MMI). Both can be implemented by the stochastic gradient descent method for obtaining the demixing matrix. The MI is the contrast function for blind separation; the entropy is not. To justify the ME, the relation between ME and MMI is first elucidated by calculating the first derivative of the entropy and proving that the mean subtraction is necessary in applying the ME and at the solution points determined by the MI, the ME will not update the demixing matrix in the directions of increasing the cross-talking. Second, the natural gradient instead of the ordinary gradient is introduced to obtain efficient algorithms, because the parameter space is a Riemannian space consisting of matrices. The mutual information is calculated by applying the Gram-Charlier expansion to approximate probability density functions of the outputs. Finally, we propose an efficient learning algorithm that incorporates with an adaptive method of estimating the unknown cumulants. It is shown by computer simulation that the convergence of the stochastic descent algorithms is improved by using the natural gradient and the adaptively estimated cumulants.