Independent Component Analysis Problem Research Articles

Decentralized nonconvex optimization with guaranteed privacy and accuracy

Privacy protection and nonconvexity are two challenging problems in decentralized optimization and learning involving sensitive data. Despite some recent advances addressing each of the two problems separately, no results have been reported that have theoretical guarantees on both privacy protection and saddle/maximum avoidance in decentralized nonconvex optimization. We propose a new algorithm for decentralized nonconvex optimization that can enable both rigorous differential privacy and saddle/maximum avoiding performance. The new algorithm allows the incorporation of persistent additive noise to enable rigorous differential privacy for data samples, gradients, and intermediate optimization variables without losing provable convergence, and thus circumventing the dilemma of trading accuracy for privacy in differential privacy design. More interestingly, the algorithm is theoretically proven to be able to efficiently guarantee accuracy by avoiding convergence to local maxima and saddle points, which has not been reported before in the literature on decentralized nonconvex optimization. The algorithm is efficient in both communication (it only shares one variable in each iteration) and computation (it is encryption-free), and hence is promising for large-scale nonconvex optimization and learning involving high-dimensional optimization parameters. Numerical experiments for both a decentralized estimation problem and an Independent Component Analysis (ICA) problem confirm the effectiveness of the proposed approach.

Backward scattering suppression in an underwater LiDAR signal processing based on CEEMDAN-fast ICA algorithm.

A new signal-processing method to realize blind source separation (BSS) in an underwater lidar-radar system based on complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and independent component analysis (ICA) is presented in this paper. The new statistical signal processing approach can recover weak target reflections from strong backward scattering clutters in turbid water, thus greatly improve the ranging accuracy. The proposed method can overcome the common problem of ICA, i.e. the number of observations must be equal to or larger than the number of sources to be separated, therefore multiple independent observations are required, which normally is realized by repeating the measurements in identical circumstances. In the new approach, the observation matrix for ICA is constructed by CEEMDAN from a single measurement. BSS can be performed on a single measurement of the mixed source signals. The CEEMDAN-ICA method avoid the uncertainty induced by the change of measurement circumstances and reduce the errors in ICA algorithm. In addition, the new approach can also improve the detection efficiency because the number of measurement is reduced. The new approach was tested in an underwater lidar-radar system. A mirror and a white Polyvinyl chloride (PVC) plate were used as target, respectively. Without using the CEEMDAN- Fast ICA, the ranging error with the mirror was 12.5 cm at 2 m distance when the attenuation coefficient of the water was 7.1 m-1. After applying the algorithm, under the same experimental conditions, the ranging accuracy was improved to 4.33 cm. For the PVC plate, the ranging errors were 5.01 cm and 21.54 cm at 3.75 attenuation length with and without the algorithm respectively. In both cases, applying this algorithm can significantly improve the ranging accuracy.

Decoupling of Multivariable Control Systems Using Mod-ICA / Dissociação de sistemas de controle multivariável usando Mod-ICA

Sensors usage in process control systems is of vital importance for industrial plants proper operation and monitoring. In turn, process signals may have interference from other sources, and in some cases, it is not possible to observe the individual signs of the sources directly. Because of this, techniques of processing and separation of signals has been used to extract the information from the sources contained in mixed signals. The main signal separation techniques are associated with the technique ICA (Independent Component Analysis), which has undergone significant evolution since its creation in the 80’s. This growth had also received contributions from PCA technique (Principal Component Analysis), and the development of computational processing power. However, these methods have two fundamental problems, which are the deviation of amplitude, and phase change, with issues such as limiting its use in control systems. Thus, this work aims to present a solution to the amplitude problem of ICA's techniques for use in obtaining uncoupling of multivariable systems. The proposed correction, based on the stage of whitening of ICA algorithms, generating the technique MOD-ICA, has been used as an alternative to the breaking of the correlation between variables of multivariate systems, with the goal of achieving the decoupling of MIMO (Multiple Input Multiple Output) systems. In the study case, proposed in this work, it was observed a better estimation of the parameters of decoupling models for the control system. The variables, after using the ICA modified technique, are independent and do not present the influence of disturbances arising from other variables in the process, resulting in a more robust control system for process variations.

An Efficient Convolutional Blind Source Separation Algorithm for Speech Signals under Chaotic Masking

As the main method of information transmission, it is particularly important to ensure the security of speech communication. Considering the more complex multipath channel transmission situation in the wireless communication of speech signals and separating or extracting the source signal from the convolutional signal are crucial steps in obtaining source information. In this paper, chaotic masking technology is used to guarantee the transmission safety of speech signals, and a fast fixed-point independent vector analysis algorithm is used to solve the problem of convolutional blind source separation. First, the chaotic masking is performed before the speech signal is sent, and the convolutional mixing process of multiple signals is simulated by impulse response filter. Then, the observed signal is transformed to the frequency domain by short-time Fourier transform, and instantaneous blind source separation is performed using a fast fixed-point independent vector analysis algorithm. The algorithm can preserve the high-order statistical correlation between frequencies to solve the permutation ambiguity problem in independent component analysis. Simulation experiments show that this algorithm can efficiently complete the blind extraction of convolutional signals, and the quality of recovered speech signals is better. It provides a solution for the secure transmission and effective separation of speech signals in multipath transmission channels.

Performance analysis for complex-valued FastICA and its improvement based on the Tukey M-estimator

Independent component analysis (ICA) is increasingly utilized to modern digital signal processing. Complex-valued FastICA, a fast fixed-point algorithm for ICA, is one of the most non-trivial algorithms for solving the ICA problems in the complex domain. Hitherto, there have been several attempts to give performance analysis for complex-valued FastICA. Rigorous theoretical analysis, however, still has room for improvement further. Consequently, the purposes of this paper are threefold: Firstly, the uniformity of the complex-valued FastICA estimator is constructed for the first time. Secondly, the stability of the complex-valued ICA algorithm is rigorously deduced based on the augmented generating matrix. Meanwhile, the local convergence of complex-valued FastICA algorithm is derived based on circular source signals. Finally, for improving the performance of separation, we select a novel alternative for nonlinearity based on the Tukey M-estimator in the complex-valued FastICA algorithm. Further, we prove the existence of local optimal solution and stability of the complex ICA problem based on the Tukey M-estimator. Simulations are presented to demonstrate the accuracy of our analysis. Additionally, the experimental results with synthetic data and complex-valued wind signal show the superiorities of the improved method.

Resonance-based sparse improved fast independent component analysis and its application to the feature extraction of planetary gearboxes

A resonance-based sparse improved fast independent component analysis (ICA) (RSIFICA) is proposed to extract the fault characteristics of planetary gearbox. First, the signal is decomposed using resonance sparse signal decomposition (RSSD). Second, high-resonance components were retained while others were eliminated. Finally, the signal after dimension reduction was analyzed using ICA, and the fault characteristic frequency was extracted through envelope spectrum analysis. In this process, the preset Q parameter of RSSD is optimized on the basis of fuzzy entropy and ant-lion optimization algorithm. The accuracy of RSSD was improved by performing time-frequency entropy component selection. FastICA was improved, and the slow convergence problem of ICA was solved. Results showed that RSIFICA could extract the fault characteristic frequency accurately, and the calculation efficiency of FastICA increased by 21.49 %. In terms of extracting the fault features, its performance could be better than EMD-FastICA.

A Method for Blind Source Separation of Multichannel Electromagnetic Radiation in the Field

Considering the multichannel instability, spectral overlap and strong interference of electromagnetic radiation signals in the integrated electric propulsion systems of ships, a new method is proposed which combines multivariate empirical mode decomposition (MEMD) with independent component analysis (ICA) for synchronous blind source separation of multichannel electromagnetic radiation in the field. In order to construct virtual channels, noise-aided MEMD is first applied to decompose multichannel data in this approach. Then the comprehensive screening algorithm is used to filter the intrinsic mode functions (IMFs) produced by the decomposition procedure. Finally, the new multivariate input signal is analyzed after screening using ICA to obtain the original electromagnetic radiation source signal. This method is able to effectively addresses several limitations of EMD including the mode mixing problem, its inability to handle multichannel data, as well as the indeterminacy problem in ICA. The efficacy of the MEMD-ICA algorithm was evaluated using simulated signals and real world data from a cruise ship, and comparing ensemble empirical mode decomposition-ICA(EEMD-ICA).

A Modified RLS Algorithm for ICA with Weighted Orthogonal Constraint

Independent component analysis (ICA), as an important data processing technique, is widely employed in many areas. The objective of the ICA is to recover independent components from observed signals. Several algorithms, such as equivariant adaptive separation via independence algorithm, least-mean-square (LMS)-type algorithms and recursive least-squares (RLS)-type learning rules, are proposed to solve the ICA problem. In the present paper, a modified RLS algorithm for ICA with weighted orthogonal constraint is developed to implement source separation based on the local convergence analysis of the available algorithm. Comparative experiment results demonstrate that the proposed algorithm is better than existing learning rules in the aspect of the accuracy of separation and stability.

Independent component analysis employing exponentials of sparse antisymmetric matrices

Abstract Independent component analysis (ICA) is a standard method for separating a multivariate signal into additive components that are non-Gaussian and independent from each other. This paper introduced a novel algorithm to perform ICA employing matrix exponentials, which performs similarly to geodesic based methods but based on a different insight. First, we showed that the ICA problem can be formulated as an optimization problem in the space of orthogonal matrices whose determinants are one, which can be further transformed into an equivalent problem in the space of antisymmetric matrices. Then, an efficient approach was presented for iteratively solving this problem using the antisymmetric matrices with one or more nonzero columns and rows. Especially, we proved that in the sense of local optimization it is sufficient to employ antisymmetric matrices with only one nonzero column and row. The analytical expressions of exponentials of such special antisymmetric matrices were also explicitly established in this paper. Compared to other competing algorithms, experimental results indicated that the proposed method can achieve separation with superior performance in term of the precision and running speed.

A Copula Based ICA Algorithm and Its Application to Time Series Clustering

Independent component analysis (ICA) is a method to recover the original independent variables from the linear transformations of the observations. Most of ICA algorithms are formulated as an optimization of a contrast function which minimizes the cross-dependency among the components. In this paper, we propose an innovative algorithm for performing ICA problem which uses a contrast function based on the Hoeffding’s measure of pairwise dependence. This measure takes its minimum if and only if the random variables are independent, and takes its maximum if and only if one of the variable is a function of the other. Since the Hoeffding’s index is computed based on the rank values rather than the actual values of the data, it is significantly robust to the outliers and performs well even in the presence of noise. The proposed algorithm is evaluated using simulated data. The algorithm is utilized as a pre-processing method for clustering of trends in time series data. This pre-processing technique establish new components from original observations which have adequate information trend of time series. For illustrative purposes, the proposed methodology is applied to clustering of two real data sets involving financial time series.

Independent component analysis: A statistical perspective

Independent component analysis (ICA) is a data analysis tool that can be seen as a refinement of principal component analysis or factor analysis. ICA recovers the structures in the data which stay hidden if only the covariance matrix is used in the analysis. The ICA problem is formulated as a latent variable model where the observed variables are linear combinations of unobserved mutually independent non‐Gaussian variables. The goal is to recover linear transformations back to these latent independent components (ICs). As a statistical tool, the unmixing procedure is expressed as a functional in a relevant semiparametric model which further allows a careful formulation of the inference problem and the comparison of competing estimation procedures. For most approaches, the ICs are found in two steps, (a) by standardizing the random vector and then (b) by rotating the standardized vector to the ICs. In the projection pursuit, the ICs can be found either one‐by‐one or simultaneously and this is discussed in detail when the convex combination of the squared third and fourth cumulants is used as a projection index. Alternative projection indices and their use are also explained. The classical fourth‐order blind identification (FOBI) and joint approximate diagonalization of eigenmatrices (JADE) are described as well. The statistical tools for the comparison of consistent and asymptotically multivariate normal unmixing matrix estimates are discussed. Finally, recent extensions for times series, matrix‐ and tensor‐valued and functional data are reviewed.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms Statistical Models > Multivariate Models Statistical and Graphical Methods of Data Analysis > Dimension Reduction Statistical and Graphical Methods of Data Analysis > Information Theoretic Methods

Harmonic separation from grid voltage using ensemble empirical-mode decomposition and independent component analysis

Summary Harmonics and subharmonics in power systems distort grid voltage, reduce the quality of power, and affect the security of the power grid. Rapid and accurate harmonic separation from grid voltage is the crucial technology to ensure that power systems operate safely and stably. The blind source separation method based on independent component analysis has been used to separate the components of grid voltage. As the grid voltage is acquired in only a single channel, harmonic separation from it is classed as a single-channel independent component analysis problem. Hence, this paper proposes a method for the harmonic separation of single-channel grid voltage that combines ensemble empirical-mode decomposition and FastICA. Compared with 2 traditional methods, the fast Fourier transform and the discrete wavelet transform, the proposed method is superior in that it does not require prior knowledge of the frequency of the original source, does not feature mode mixing, and is more robust against noise. Results obtained from both synthetic and real-life signals demonstrated the excellent performance of the proposed method.

Comparison and applicability study of blood volume pulse extraction based on facial video

Blind source separation technique based on independent component analysis (ICA) can separate blood volume pulse (BVP) from the facial video and then realize the telemetry of heart rate, blood oxygen saturation, respiratory rate and other vital signs parameters. However, the superiority of ICA in BVP extraction has not been demonstrated in the existing researches. Some researchers suggested using traditional G-channel method for BVP extraction (G-BVP) instead of ICA method (ICA-BVP). This study investigated the applicability of ICA-BVP comparatively. To solve the inherent permutation problem of ICA, a spectral kurtosis-based method was proposed for BVP identification. The experimental results based on the facial video datasets from 9 subjects shows that ICA-BVP method has apparent advantages in motion artifacts attenuation and ambient light changes elimination. The kurtosis-based method achieved a good performance in BVP identification and dynamic heart rate (HR) estimation. In practical application, the proposed ICA-BVP method could present a better stability and accuracy in vital signs parameters extraction.

Independent Component Analysis Via Copula Techniques

Independent component analysis (ICA) is a modern factor analysis tool de- veloped in the last two decades. Given p-dimensional data, we search for that linear combination of data which creates (almost) independent components. Here copulae are used to model the p-dimensional data and then independent components are found by optimizing the copula parameters. Based on this idea, we propose the COPICA method for searching independent components. We illustrate this method using several blind source separation examples, which are mathematically equivalent to ICA problems. Finally performances of our method and FastICA are compared to explore the advantages of this method.

Simple technique to determine the Givens‐rotation matrix in the two‐source ICA problem for skewed sources

The estimation of the Givens-rotation matrix or the Jacobi angle has always been a fundamental problem in independent component analysis. To estimate the angle, many approaches which use the optimal means and statistics of observation data have been presented during the past few decades. It is showed in this Letter that as long as one of two sources is asymmetrical, the Givens-rotation matrix can be solved from two weighted covariance matrices constructed by observations. Simulations illustrate the effectiveness of the approach.

Fixed-point ICA algorithm for blind separation of complex mixtures containing both circular and noncircular sources

Fixed-point algorithms are widely used for independent component analysis (ICA) owing to its good convergence. However, most existing complex fixed-point ICA algorithms are limited to the case of circular sources and result in phase ambiguity, that restrict the practical applications of ICA. To solve these problems, this paper proposes a two-stage fixed-point ICA (TS-FPICA) algorithm which considers complex signal model. In this algorithm, the complex signal model is converted into a new real signal model by utilizing the circular coefficients contained in the pseudo-covariance matrix. The algorithm is thus valid to noncircular sources. Moreover, the ICA problem under the new model is formulated as a constrained optimization problem, and the real fixed-point iteration is employed to solve it. In this way, the phase ambiguity resulted by the complex ICA is avoided. The computational complexity and convergence property of TS-FPICA are both analyzed theoretically. Simulation results show that the proposed algorithm has better separation performance and without phase ambiguity in separated signals compared with other algorithms. TS-FPICA convergences nearly fast as the other fixed-point algorithms, but far faster than the joint diagonalization method, e.g. joint approximate diagonalization of eigenmatrices (JADE).

A study of the fixed points and spurious solutions of the deflation-based FastICA algorithm

The FastICA algorithm is one of the most popular algorithms in the domain of independent component analysis (ICA). Despite its success, it is observed that FastICA occasionally yields outcomes that do not correspond to any true solutions (known as demixing vectors) of the ICA problem. These outcomes are commonly referred to as spurious solutions. Although FastICA is a well-studied ICA algorithm, the occurrence of spurious solutions is not yet completely understood by the ICA community. In this contribution, we aim at addressing this issue. In the first part of this work, we are interested in characterizing the relationship between demixing vectors, local optimizers of the contrast function and (attractive or unattractive) fixed points of FastICA algorithm. We will show that there exists an inclusion relationship between these sets. In the second part, we investigate the possible scenarios where spurious solutions occur. It will be shown that when certain bimodal Gaussian mixture distributions are involved, there may exist spurious solutions that are attractive fixed points of FastICA. In this case, popular nonlinearities such as "Gauss" or "tanh" tend to yield spurious solutions, whereas "kurtosis" gives much more reliable results.

Range-based ICA using a nonsmooth quasi-newton optimizer for electroencephalographic source localization in focal epilepsy.

Independent component analysis (ICA) aims at separating a multivariate signal into independent nongaussian signals by optimizing a contrast function with no knowledge on the mixing mechanism. Despite the availability of a constellation of contrast functions, a Hartley-entropy-based ICA contrast endowed with the discriminacy property makes it an appealing choice as it guarantees the absence of mixing local optima. Fueled by an outstanding source separation performance of this contrast function in practical instances, a succession of optimization techniques has recently been adopted to solve the ICA problem. Nevertheless, the nondifferentiability of the considered contrast restricts the choice of the optimizer to the class of derivative-free methods. On the contrary, this letter concerns a Riemannian quasi-Newton scheme involving an explicit expression for the gradient to optimize the contrast function that is differentiable almost everywhere. Furthermore, the inexact line search insisting on the weak Wolfe condition and a terminating criterion befitting the partly smooth function optimization have been generalized to manifold settings, leaving the previous results intact. The investigations with diversified images and the electroencephalographic (EEG) data acquired from 45 focal epileptic subjects demonstrate the efficacy of our approach in terms of computational savings and reliable EEG source localization, respectively. Additional experimental results are available in the online supplement.

Semi-nonnegative joint diagonalization by congruence and semi-nonnegative ICA

In this paper, we focus on the Joint Diagonalization by Congruence (JDC) decomposition of a set of matrices, while imposing nonnegative constraints on the joint diagonalizer. The latter will be referred to the semi-nonnegative JDC fitting problem. This problem appears in semi-nonnegative Independent Component Analysis (ICA), say ICA involving nonnegative static mixtures, such as those encountered for instance in image processing and in magnetic resonance spectroscopy. In order to achieve the semi-nonnegative JDC decomposition, we propose two novel algorithms called ELS-ALSexp and CGexp, which optimize an unconstrained problem obtained by means of an exponential change of variable. The proposed methods are based on the line search strategy for which an analytic global plane search procedure has been considered. All derivatives have been jointly calculated in matrix form using the algebraic basis for matrix calculus and product operator properties. Our algorithms have been tested on synthetic arrays and the semi-nonnegative ICA problem is illustrated through simulations in magnetic resonance spectroscopy and in image processing. The numerical results show the benefit of using a priori information, such as nonnegativity.

A Single-Channel ICA-R Method for Speech Signal Denoising combining EMD and Wavelet

According to the problem of speech signal denoising, we propose a novel method in this paper, which combines empirical mode decomposition (EMD), wavelet threshold denoising and independent component analysis with reference (ICA-R). Because there is only one mixed recording, it is a single-channel independent component analysis (SCICA) problem in fact, which is hard to solve by traditional ICA methods. EMD is exploited to expand the single-channel received signal into several intrinsic mode functions (IMFs) in advance, therefore traditional ICA of multi-dimension becomes applicable. First, the received signal is segmented to reduce the processing delay. Secondly, wavelet thresholding is applied to the noise-dominated IMFs. Finally, fast ICA-R is introduced to extract the object speech component from the processed IMFs, whose reference signal is constructed by assembling the high-order IMFs. The simulations are carried out under different noise levels and the performance of the proposed method is compared with EMD, wavelet thresholding, EMD-wavelet and EMD-ICA approaches. Simulation results indicate that the proposed method exhibit superior denoising performance especially when signal-to-noise ratio is low, with a half shorter running time.