Classical Independent Component Analysis Research Articles

Direct ICA on data tensor via random matrix modeling

Independent Component Analysis (ICA) is a fundamental method for Blind Source Separation (BSS). Classical ICA takes data matrix input formed by vector data. This paper focuses on ICA for BSS with third-order data tensor input formed by matrix data, such as 2D images. Two approaches exist for this problem. The first approach reshapes each matrix into a vector to apply classical ICA, with structural information lost. The second approach unfolds a data tensor into a data matrix along different modes to perform classical ICA mode-wise, which partially preserves structures but has strong or ill BSS assumptions. This paper proposes a third approach via RAndom Matrix ICA (RAMICA) modeling. RAMICA works on data tensor directly, without vectorization or unfolding, and preserves row or column structures under more general BSS assumptions. We develop the RAMICA model, algorithm, and related theories via defining new statistics for random matrices and new procedures for whitening and independent component estimation. We study the identifiability, higher-order extension, and relationships with existing methods. Experiments on both synthetic and real data show superior BSS performance of RAMICA over competing methods and offer insights on the trade-offs between different factors.

VLSI Design of a Fast One-Stage Independent Component Extracting System Based on ICA-R Algorithm

Independent component analysis (ICA) is an efficient blind source separation technique, but the extracted independent components are randomly permuted in classic ICA algorithms, and subsequent identification is required to find the desired component. Such a two-stage method causes inefficiency. By utilizing the prior information, ICA with reference (ICA-R) algorithm can extract the desired source signal in one stage without subsequent processing. There are many theoretical extensions on ICA-R, but few hardware implementations can be found. Therefore, the efficient VLSI design of a fast one-stage independent component extracting system based on ICA-R algorithm is presented in this paper. The proposed system consists of Preprocess module and Iteration module, which are designed highly parallel and pipelined to accelerate the extraction process. The designed system is implemented in Kintex-7 FPGA, and its performance is verified using synthesized signal. Experiment results show that the presented system can extract the desired components in one stage without subsequent identification, and the highly parallel circuit structure of the system speeds up the component extracting process by [Formula: see text] compared to software implementation.

Fast automated on‐chip artefact removal of EEG for seizure detection based on ICA‐R algorithm and wavelet denoising

Portable automatic seizure detection systems can greatly improve the quality of life of epileptic patients. To improve the performance of seizure detection, independent component analysis (ICA) is implemented in these systems to extract artefacts of electroencephalogram (EEG), and then wavelet denoising method is used to remove the artefacts. However, classical ICA requires post-identification of the components containing artefacts, which cause inefficiency. In this study, integrated circuit implementation of fast ICA with reference algorithm and wavelet denoising method is carried out to enable on-chip artefact extraction and removal without post-identification. This system consists of extraction and removal module, which are designed highly parallel to speed up computation, and therefore, save time for seizure detection. The presented system is verified on Kintex-7 field-programmable gate array using synthesised signal and real EEG data. Experiment results show that the designed system is fully functional and speeds up the artefact removal process.

Intelligent approach for large-scale data mining

Large-scale data mining has become a very difficult issue using traditional methods because the data complexity is very high. In the proposed approach, an integration of three methods; Optimised Principal Component Analysis (OPCA), Optimised Enhanced Extreme Learning Machine (OEELM), and stratified sampling, called OPCA-EELM2SS, is presented to provide intelligent and enhanced large-scale data mining. OPCA provides a good representation of large-scale data sets by using the Stratified Sample (SS). By using OEELM, the optimal number of Hidden Nodes (HNs) in ELM is exploited to build a single hidden layer feedforward neural network (SLFN). The proposed approach is tested by using nineteen benchmark data sets. The experimental results demonstrate the effectiveness of the proposed approach by performing different experiments for classical PCA and Independent Component Analysis (ICA), which are integrated with the enhanced ELM using different evaluation criteria. For more reliability, the proposed approach is compared with many previous methods.

Minimum BER Criterion and Adaptive Moment Estimation Based Enhanced ICA for Wireless Communications

This paper concentrates on investigating an enhanced independent component analysis (ICA) method for blind separation of signals corrupted by noise in wireless communications. Because of the traditional classical ICA methods that always have an inadequate capacity of anti-noise or insufficient separable ability in noise circumstance without satisfying practical application requirements. For this reason, two mechanisms are conducted to establish the modified cost function and fulfill the optimization assignment in the process of constituting an enhanced ICA algorithm. This proposed algorithm can benefit tremendously from the derived minimum bit error rate (BER) criterion and the novel adaptive moment estimation (Adam) optimization approach. In this work, firstly, the novel cost function is obtained according to minimum BER fused into maximum likelihood (ML) principle-based ICA cost function. Furthermore, by utilizing Adam processing, the task of blind separation of mixed signals is implemented via optimizing this modified cost function. Lastly, theoretical analysis and experiment results corroborate the better effectiveness and robustness of the proposed enhanced ICA algorithm compared with a series of popular representative ICA methods.

MAICA: an ICA-based method for source separation in a low-channel EEG recording

Objective. The paper aims to present a method that enables the application of independent component analysis (ICA) to a low-channel EEG recording. The idea behind the method (called moving average ICA or MAICA) is to extend the original low-sensor matrix of signals by applying a set of zero-phase moving average filters to each of the recorded signals. Approach. The paper discusses the theoretical background of the MAICA algorithm and verifies its usefulness under three exemplary settings: (i) a pure mathematic system composed of ten source sinusoids; (ii) real EEG data recorded from 64 channels; (iii) real EEG data recorded from five subjects during 200 trials with motor imagery brain–computer interface. Main results. The first system shows that MAICA is able to decompose two mixed signals (composed of ten source sinusoids) into ten components with an extremely high correlation between the source patterns and identified components (99%–100%). The second system shows that when used over five channels, MAICA is able to recognize more artefact components than those recognized by classic ICA used over 64 channels. Finally, the third system demonstrates that MAICA is capable of working in an online mode without significant delays; the additional time needed to run MAICA for one trial was less than 6ms in the survey reported in the paper. Significance. The method presented in the paper should have a significant impact on all areas of medical signal processing where a large number of known and/or unknown patterns have to be retrieved in real time from complex signals recorded from a small number of external/internal body sensors.

To Explore the Potentials of Independent Component Analysis in Brain-Computer Interface of Motor Imagery.

This paper is focused on the experimental approach to explore the potential of independent component analysis (ICA) in the context of motor imagery (MI)-based brain-computer interface (BCI). We presented a simple and efficient algorithmic framework of ICA-based MI BCI (ICA-MIBCI) for the evaluation of four classical ICA algorithms (Infomax, FastICA, Jade, and Sobi) as well as a simplified Infomax (sInfomax). Two novel performance indexes, self-test accuracy and the number of invalid ICA filters, were employed to assess the performance of MIBCI based on different ICA variants. As a reference method, common spatial pattern (CSP), a commonly-used spatial filtering method, was employed for the comparative study between ICA-MIBCI and CSP-MIBCI. The experimental results showed that sInfomax-based spatial filters exhibited significantly better transferability in session to session and subject to subject transfer as compared to CSP-based spatial filters. The online experiment was also introduced to demonstrate the practicability and feasibility of sInfomax-based MIBCI. However, four classical ICA variants, especially FastICA, Jade, and Sobi, performed much worse as compared to sInfomax and CSP in terms of classification accuracy and stability. We consider that conventional ICA-based spatial filtering methods tend to be overfitting while applied to real-life electroencephalogram data. Nevertheless, the sInfomax-based experimental results indicate that ICA methods have a great space for improvement in the application of MIBCI. We believe that this paper could bring forth new ideas for the practical implementation of ICA-MIBCI.

SCTICA: Sub-packet constrained temporal ICA method for fMRI data analysis

Independent component analysis (ICA) has become a widely used method for functional magnetic resonance imaging (fMRI) data analysis. However, spatial ICA usually performs better than temporal ICA with regard to the stability and accuracy of functional connectivity detection, and temporal ICA is often not feasible when it is applied to the analysis of real fMRI data of the whole brain because of the excessive spatial dimensions. In this paper, to overcome these problems, we propose a sub-packet constrained temporal ICA (SCTICA) method to take advantage of the a priori information using a multi-objective optimization framework with the Newton iterative algorithm. Moreover, a splitting strategy is presented to improve the feasibility of the temporal ICA for whole brain fMRI data analysis. The experimental results of real data show that the splitting strategy improved the ability of the temporal ICA to analyze whole brain fMRI data. Furthermore, the experimental results also demonstrated that the proposed SCTICA method can not only improve the stability of the temporal ICA, but can also improve the functional connectivity detection ability compared with the classical ICA and ICA with a priori information methods. In brief, the proposed SCTICA method overcomes the problem that prevents temporal ICA from being applied to fMRI data of the whole brain, and the functional connectivity detection performance is greatly improved compared with that of traditional methods.

A New Constrained Spatiotemporal ICA Method Based on Multi-Objective Optimization for fMRI Data Analysis.

Compared with independent component analysis (ICA), constrained ICA (CICA) has unique advantages in functional magnetic resonance image (fMRI) data analysis by introducing some priori information into the estimation process. However, there are still some controversies in the current CICA methods, such as how to choose the threshold parameter to restrain the similarity, and how to reduce the accuracy requirements for a priori information. In this paper, we propose a new constrained spatiotemporal ICA (CSTICA) method based on the framework of multi-objective optimization, where the inequality constraint of the traditional CICA method is transformed into the objective optimization function of the CSTICA, and both temporal and spatial a priori information are included simultaneously. The simulated, hybrid, and real fMRI data experiments are designed to evaluate the performance of the proposed CSTICA method in comparison with the classical ICA and CICA methods. Compared with the traditional CICA methods, the CSTICA has circumvented the problem of threshold parameter selection. Furthermore, the experimental results demonstrate that the source recovery ability of the CSTICA has been improved to a certain extent especially in the cases of a priori information with low accuracies. Meanwhile, the results also indicate that the CSTICA reduces dependency on the accuracy of a priori information.

Automatic ocular artifacts removal in EEG using deep learning

Abstract Ocular artifacts (OAs) are one the most important form of interferences in the analysis of electroencephalogram (EEG) research. OAs removal/reduction is a key analysis before the processing of EEG signals. For classic OAs removal methods, either an additional electrooculogram (EOG) recording or multi-channel EEG is required. To address these limitations of existing methods, this paper investigates the use of deep learning network (DLN) to remove OAs in EEG signals. The proposed method consists of offline stage and online stage. In the offline stage, training samples without OAs are intercepted and used to train an DLN to reconstruct the EEG signals. The high-order statistical moments information of EEG is therefore learned. In the online stage, the trained DLN is used as a filter to automatically remove OAs from the contaminated EEG signals. Compared with the exiting methods, the proposed method has the following advantages: (i) nonuse of additional EOG reference signals, (ii) any few number of EEG channels can be analyzed, (iii) time saving, and (iv) the strong generalization ability, etc. In this paper, both public database and lab individual data for EEG analysis are used, we compared the proposed method with the classic independent component analysis (ICA), kurtosis-ICA (K-ICA), Second-order blind identification (SOBI) and a shallow network method. Experimental results show that the proposed method performs better even for very noisy EEG.

Blind quantum source separation: Quantum-processing qubit uncoupling systems based on disentanglement

Blind Quantum Source Separation (BQSS) deals with multi-qubit states, called “mixed states”, obtained by applying an unknown “mixing function” (which typically corresponds to undesired coupling, e.g. between qubits implemented as close electron spins 1/2) to unknown multi-qubit “source states”, which are product states (and pure in the simplest case, considered in this paper). Some other properties are also possibly requested from these source states and/or mixing function. Using mixed states, BQSS systems aim at restoring (the information contained in) source states, during the second phase of their operation (“inversion phase”). To this end, they estimate the unmixing function (inverse of mixing function), during the first phase of their operation (“adaptation phase”). Most previously reported BQSS systems first convert mixed states into classical-form data, that they then process with classical means. Besides, they estimate the unmixing function by using statistical methods related to classical Independent Component Analysis. On the contrary, the new BQSS systems proposed here use only quantum-form data and quantum processing in the inversion phase, and they use classical-form data during the adaptation phase only. Moreover, their unmixing function estimation methods are essentially based on using unentangled source states during that phase. They mainly consist of disentangling the output quantum state of the separating system (for a few source states). Afterwards, they can also restore entangled source states. They yield major improvements over previous systems, concerning restored source parameters, associated indeterminacies and approximations, number of source states required for adaptation, numbers of source state preparations in adaptation and inversion phases. Numerical tests confirm that they accurately restore quantum source states.

On Independent Component Analysis with Stochastic Volatility Models

Consider a multivariate time series where each component series is assumed to be a linear mixture of latent mutually independent stationary time series. Classical independent component analysis (ICA) tools, such as fastICA, are often used to extract latent series, but they don't utilize any information on temporal dependence. Also financial time series often have periods of low and high volatility. In such settings second order source separation methods, such as SOBI, fail. We review here some classical methods used for time series with stochastic volatility, and suggest modifications of them by proposing a family of vSOBI estimators. These estimators use different nonlinearity functions to capture nonlinear autocorrelation of the time series and extract the independent components. Simulation study shows that the proposed method outperforms the existing methods when latent components follow GARCH and SV models. This paper is an invited extended version of the paper presented at the CDAM 2016 conference.

A new method for independent component analysis with priori information based on multi-objective optimization

BackgroundCurrently the problem of incorporating priori information into an independent component analysis (ICA) model is often solved under the framework of constrained ICA, which utilizes the priori information as a reference signal to form a constraint condition and then introduce it into classical ICA. However, it is difficult to pre-determine a suitable threshold parameter to constrain the closeness between the output signal and the reference signal in the constraint condition. New methodIn this paper, a new model of ICA with priori information as a reference signal is established on the framework of multi-objective optimization, where an adaptive weighted summation method is introduced to solve this multi-objective optimization problem with a new fixed-point learning algorithm. ResultsThe experimental results of fMRI hybrid data and task-related data on the single-subject level have demonstrated that the proposed method has a better overall performance on the recover abilities of both spatial source and time course. Comparison with existing methodsAt the same time, compared with traditional ICA with reference methods and classical ICA method, the experimental results of resting-state fMRI data on the group-level have showed that the group independent component calculated by the proposed method has a higher correlation with the corresponding independent component of each subject through T-test. ConclusionsThe proposed method does not need us to select a threshold parameter to constrain the closeness between the output signal and the reference signal. In addition, the performance of functional connectivity detection has a great improvement in comparison with traditional methods.

A Penalized Semialgebraic Deflation ICA Algorithm for the Efficient Extraction of Interictal Epileptic Signals.

As a noninvasive technique, electroencephalography (EEG) is commonly used to monitor the brain signals of patients with epilepsy such as the interictal epileptic spikes. However, the recorded data are often corrupted by artifacts originating, for example, from muscle activities, which may have much higher amplitudes than the interictal epileptic signals of interest. To remove these artifacts, a number of independent component analysis (ICA) techniques were successfully applied. In this paper, we propose a new deflation ICA algorithm, called penalized semialgebraic unitary deflation (P-SAUD) algorithm, that improves upon classical ICA methods by leading to a considerably reduced computational complexity at equivalent performance. This is achieved by employing a penalized semialgebraic extraction scheme, which permits us to identify the epileptic components of interest (interictal spikes) first and obviates the need of extracting subsequent components. The proposed method is evaluated on physiologically plausible simulated EEG data and actual measurements of three patients. The results are compared to those of several popular ICA algorithms as well as second-order blind source separation methods, demonstrating that P-SAUD extracts the epileptic spikes with the same accuracy as the best ICA methods, but reduces the computational complexity by a factor of 10 for 32-channel recordings. This superior computational efficiency is of particular interest considering the increasing use of high-resolution EEG recordings, whose analysis requires algorithms with low computational cost.

Multi-User Detection based on PSO-KICA

Multi-user detection (MUD) is used to reduce multiple access interference (MAI) and promote system performance and capacity, which is one of the core technologies for CDMA system. In this paper, some questions of blind MUD based on independent component analysis (ICA) are introduced firstly, and then a k ernel independent component analysis (KICA) algorithm which brings in a new hybrid kernel function is proposed. In addition, the particle swarm optimization (PSO) algorithm is studied and it is devoted to the optimizing process of the KICA algorithm. The improved PSO-KICA algorithm can solve the issues that the objective function falls into the local optimum. The experimental results show that the PSO-KICA algorithm has the smaller bit error rate and good convergent speed than the classical ICA algorithm.

A fast blind source separation algorithm based on the temporal structure of signals

Classical independent component analysis (ICA) has been reasonably successful; however, the performance and the convergence of the conventional ICA algorithms have reached limitations of further improvement since they utilize only the statistical independency among the sources. For circumventing this situation, in this paper, we incorporate some other kinds of temporal priori information, i.e., the generalized autocorrelation and the nonlinear predictability of each source, and make a convex combination of them to formulate a novel cost function for blind source separation (BSS). With this cost function, a fixed-point BSS algorithm is developed. This algorithm inherits the advantages of the well-known FastICA algorithm of ICA, which converges fast and does not need to choose any learning step sizes. Its higher separation accuracy is verified by numerical experiments. Meanwhile, we also give the consistency analysis and prove convergence properties of the algorithm, which has a (locally) consistent estimator and at least quadratic convergence.

An Improved ICA Algorithm Based on the Negative Entropy and Simulated Annealing Algorithm

The Independent Component Analysis (ICA) is a classical algorithm for exploring statistically independent non-Gaussian signals from multi-dimensional data, which has a wide range of applications in engineering, for instance, the blind source separation. The classical ICA measures the Gaussian characteristic by kurtosis, which has the following two disadvantages. Firstly, the kurtosis relies on the value of samples, and is not robust to outliers. Secondly, the algorithm often falls into local optima. To address these drawbacks, we replace the kurtosis by negative entropy, utilize the simulated annealing algorithm for optimization, and finally propose an improved ICA algorithm. Experimental results demonstrate that the proposed algorithm outperforms the classical ICA in its robustness to outliers and convergent rate.

A Hierarchical Model for Probabilistic Independent Component Analysis of Multi-Subject fMRI Studies

An important goal in fMRI studies is to decompose the observed series of brain images to identify and characterize underlying brain functional networks. Independent component analysis (ICA) has been shown to be a powerful computational tool for this purpose. Classic ICA has been successfully applied to single-subject fMRI data. The extension of ICA to group inferences in neuroimaging studies, however, is challenging due to the unavailability of a pre-specified group design matrix. Existing group ICA methods generally concatenate observed fMRI data across subjects on the temporal domain and then decompose multi-subject data in a similar manner to single-subject ICA. The major limitation of existing methods is that they ignore between-subject variability in spatial distributions of brain functional networks in group ICA. In this article, we propose a new hierarchical probabilistic group ICA method to formally model subject-specific effects in both temporal and spatial domains when decomposing multi-subject fMRI data. The proposed method provides model-based estimation of brain functional networks at both the population and subject level. An important advantage of the hierarchical model is that it provides a formal statistical framework to investigate similarities and differences in brain functional networks across subjects, for example, subjects with mental disorders or neurodegenerative diseases such as Parkinson's as compared to normal subjects. We develop an EM algorithm for model estimation where both the E-step and M-step have explicit forms. We compare the performance of the proposed hierarchical model with that of two popular group ICA methods via simulation studies. We illustrate our method with application to an fMRI study of Zen meditation.

Separation of instantaneous mixtures of a particular set of dependent sources using classical ICA methods

This article deals with the problem of blind source separation in the case of a linear and instantaneous mixture. We first investigate the behavior of known independent component analysis (ICA) methods in the case where the independence assumption is violated: specific dependent sources are introduced and it is shown that, depending on the source vector, the separation may be successful or not. For sources which are a probability mixture of the previous dependent ones and of independent sources, we introduce an extended ICA model. More generally, depending on the value of a hidden latent process at the same time, the unknown components of the linear mixture are assumed either mutually independent or dependent. We propose for this model a separation method which combines: (i) a classical ICA separation performed using the set of samples whose components are conditionally independent, and (ii) a method for estimation of the latent process. The latter task is performed by iterative conditional estimation (ICE). It is an estimation technique in the case of incomplete data, which is particularly appealing because it requires only weak conditions.

Second-Order Multidimensional ICA: Performance Analysis

Independent component analysis (ICA) and blind source separation (BSS) deal with extracting a number of mutually independent elements from a set of observed linear mixtures. Motivated by various applications, this paper considers a more general and more flexible model: the sources can be partitioned into groups exhibiting dependence within a given group but independence between two different groups. We argue that this is tantamount to considering multidimensional components as opposed to the standard ICA case which is restricted to one-dimensional components. The core of the paper is devoted to the statistical analysis of the blind separation of multidimensional components based on second-order statistics, in a piecewise-stationary model. We develop the likelihood and the associated estimating equations for the Gaussian case. We obtain closed-form expressions for the Fisher information matrix and the Cramer-Rao bound of the de-mixing parameters, as well as the mean-square error (MSE) of the component estimates. The derived MSE is valid also for non-Gaussian data. Our analysis is verified through numerical experiments, and its performance is compared to classical ICA in various dependence scenarios, quantifying the gain in the accuracy of component recovery in presence of multidimensional components.