Extract Independent Components Research Articles

An Anomaly Detection Framework Based on ICA and Bayesian Classification for IaaS Platforms

Infrastructure as a Service (IaaS) encapsulates computer hardware into a large amount of virtual and manageable instances mainly in the form of virtual machine (VM), and provides rental service for users. Currently, VM anomaly incidents occasionally occur, which leads to performance issues and even downtime. This paper aims at detecting anomalous VMs based on performance metrics data of VMs. Due to the dynamic nature and increasing scale of IaaS, detecting anomalous VMs from voluminous correlated and non-Gaussian monitored performance data is a challenging task. This paper designs an anomaly detection framework to solve this challenge. First, it collects 53 performance metrics to reflect the running state of each VM. The collected performance metrics are testified not to follow the Gaussian distribution. Then, it employs independent components analysis (ICA) instead of principal component analysis (PCA) to extract independent components from collected non-Gaussian performance metric data. For anomaly detection, it employs multi-class Bayesian classification to determine the current state of each VM. To evaluate the performance of the designed detection framework, four types of anomalies are separately or jointly injected into randomly selected VMs in a campus-wide testbed. The experimental results show that ICA-based detection mechanism outperforms PCA-based and LDA-based detection mechanisms in terms of sensitivity and specificity.

Dynamical analysis of contrastive divergence learning: Restricted Boltzmann machines with Gaussian visible units

The restricted Boltzmann machine (RBM) is an essential constituent of deep learning, but it is hard to train by using maximum likelihood (ML) learning, which minimizes the Kullback–Leibler (KL) divergence. Instead, contrastive divergence (CD) learning has been developed as an approximation of ML learning and widely used in practice. To clarify the performance of CD learning, in this paper, we analytically derive the fixed points where ML and CDn learning rules converge in two types of RBMs: one with Gaussian visible and Gaussian hidden units and the other with Gaussian visible and Bernoulli hidden units. In addition, we analyze the stability of the fixed points. As a result, we find that the stable points of CDn learning rule coincide with those of ML learning rule in a Gaussian–Gaussian RBM. We also reveal that larger principal components of the input data are extracted at the stable points. Moreover, in a Gaussian–Bernoulli RBM, we find that both ML and CDn learning can extract independent components at one of stable points. Our analysis demonstrates that the same feature components as those extracted by ML learning are extracted simply by performing CD1 learning. Expanding this study should elucidate the specific solutions obtained by CD learning in other types of RBMs or in deep networks.

Identification and Automatic ICA-PSO Two-Class Classification of Time Series RSN in Shaky Hand Syndrome

Neuro-imaging techniques are used to extract and assess brain enactments. As brain activations are free to advance in an eccentric way, data driven methods are exploited for functional localization. Three motor imbalance subjects whose scan size were 128 × 128 × 23 and an aggregate of 110 volumes joined by three scans for nine acquisitions, who are on a normal age of 65 years and a set of 10 subject’s simulated data was subjected to examination. To fully exploit the potential, advanced signal processing methods are applied on acquired resting state functional MRI (rsfMRI) and SimTB simulated rsfMRI. An algorithm called Independent Component Analysis-Particle Swam Optimization two-class classifier for decision support is implemented. The algorithm pre-process each simulated and real time rsfMRI scans, extract independent components(IC) from smoothed output, select eigen vector for optimized minimum misclassification from both time series data and perform 2-class classification using k-means clustering. The proposed algorithm aided the classification of about 87.5% of the functional localization of shaky hand subjects of acquired rsfMRI data. The number of highly activated voxels in the sensory motor network is more in shaky hand subjects.

On - line Fault Detection and Diagnosis of Sequencing Batch Reactor Using MKICA

Considering the data of Sequencing Batch Reactor (SBR) having the characteristics of non-Gaussian distribution and highly nonlinearity, this research applies Multi-way Kernel Independent Component Analysis (MKICA) to the on-line process monitoring of SBR. Meanwhile, a novel contribution analysis scheme named bar plot is developed for MKICA to diagnose faults. Above all, the three-dimensional data of SBR is expanded into two-dimensional by a new data expanding method; then, Kernel Principal Component Analysis (KPCA) is utilized to map the two-dimensional data into a high dimensional feature space, and make use of Independent Component Analysis (ICA) to extract Independent Components (ICs) in feature space; finally, if MKICA detects a fault occurs during on-line monitoring stage, the bar plot is used to identify the variables causing the fault. The method is successfully applied to an 80L lab-scale SBR. The experimental results demonstrate that, compared with traditional MICA, the proposed method exhibit better performance in fault detection and diagnose.

A multi-stage control chart pattern recognition scheme based on independent component analysis and support vector machine

Recognition of unnatural control chart patterns (CCPs) is an important issue because the unnatural CCPs can be associated with specific assignable causes negatively affecting the manufacturing process. By assuming that an unnatural CCP is a combination of normal pattern and process disturbance, a multi-stage control chart pattern recognition scheme which integrates independent component analysis (ICA) and support vector machine (SVM) is proposed in this study. The proposed multi-stage ICA-SVM scheme first uses ICA to extract independent components (ICs) from the monitoring process data containing CCPs. The normal pattern and process disturbance hidden in the process data can be discovered in the ICs. Then, the IC representing the process disturbance can be identified. Finally, the identified IC and the data of the monitoring process are used as input variables to develop three different SVM models for CCP recognition. The simulation results show that the proposed multi-stage ICA-SVM scheme not only produces accurate and stable recognition results but also has better classification accuracy than four competing models.

Multivariate industrial process monitoring based on the integration method of canonical variate analysis and independent component analysis

Tennessee Eastman (TE) process is a typical multivariate chemical process. It has some characteristics of complexity and nonlinearity. Therefore, it is an ideal research platform substituted for the real industrial process whose data is difficult to be achieved. Many scholars have done a lot of studies on monitoring approaches and applied these methods on the platform. However, it is not an easy work to obtain some ideal simulation results on detecting some special faults in TE process, such as the fault 3. In this paper, an integration of canonical variate analysis and independent component analysis method (CV-ICA) is proposed. It combines the advantages of canonical variate analysis (CVA) and independent component analysis (ICA) to solve these problems. CV-ICA applies CVA to calculate the canonical variates from the process data, and then employs ICA to extract independent components (ICs). The monitoring simulation demonstrates the availability of the proposed method.

Dynamic changes of ICA-derived EEG functional connectivity in the resting state.

An emerging issue in neuroscience is how to identify baseline state(s) and accompanying networks termed "resting state networks" (RSNs). Although independent component analysis (ICA) in fMRI studies has elucidated synchronous spatiotemporal patterns during cognitive tasks, less is known about the changes in EEG functional connectivity between eyes closed (EC) and eyes open (EO) states, two traditionally used baseline indices. Here we investigated healthy subjects (n = 27) in EC and EO employing a four-step analytic approach to the EEG: (1) group ICA to extract independent components (ICs), (2) standardized low-resolution tomography analysis (sLORETA) for cortical source localization of IC network nodes, followed by (3) graph theory for functional connectivity estimation of epochwise IC band-power, and (4) circumscribing IC similarity measures via hierarchical cluster analysis and multidimensional scaling (MDS). Our proof-of-concept results on alpha-band power demonstrate five statistically clustered groups with frontal, central, parietal, occipitotemporal, and occipital sources. Importantly, during EO compared with EC, graph analyses revealed two salient functional networks with frontoparietal connectivity: a more medial network with nodes in the mPFC/precuneus which overlaps with the "default-mode network" (DMN), and a more lateralized network comprising the middle frontal gyrus and inferior parietal lobule, coinciding with the "dorsal attention network" (DAN). Furthermore, a separate MDS analysis of ICs supported the emergence of a pattern of increased proximity (shared information) between frontal and parietal clusters specifically for the EO state. We propose that the disclosed component groups and their source-derived EEG functional connectivity maps may be a valuable method for elucidating direct neuronal (electrophysiological) RSNs in healthy people and those suffering from brain disorders.

EEG correlates of haptic feedback in a visuomotor tracking task

This study investigates the temporal brain dynamics associated with haptic feedback in a visuomotor tracking task. Haptic feedback with deviation-related forces was used throughout tracking experiments in which subjects' behavioral responses and electroencephalogram (EEG) data were simultaneously measured. Independent component analysis was employed to decompose the acquired EEG signals into temporally independent time courses arising from distinct brain sources. Clustering analysis was used to extract independent components that were comparable across participants. The resultant independent brain processes were further analyzed via time–frequency analysis (event-related spectral perturbation) and event-related coherence (ERCOH) to contrast brain activity during tracking experiments with or without haptic feedback. Across subjects, in epochs with haptic feedback, components with equivalent dipoles in or near the right motor region exhibited greater alpha band power suppression. Components with equivalent dipoles in or near the left frontal, central, left motor, right motor, and parietal regions exhibited greater beta-band power suppression, while components with equivalent dipoles in or near the left frontal, left motor, and right motor regions showed greater gamma-band power suppression relative to non-haptic conditions. In contrast, the right occipital component cluster exhibited less beta-band power suppression in epochs with haptic feedback compared to non-haptic conditions. The results of ERCOH analysis of the six component clusters showed that there were significant increases in coherence between different brain networks in response to haptic feedback relative to the coherence observed when haptic feedback was not present. The results of this study provide novel insight into the effects of haptic feedback on the brain and may aid the development of new tools to facilitate the learning of motor skills.

Independent component analysis (ICA) of generalized spike wave discharges in fMRI: Comparison with general linear model‐based EEG‐fMRI

Most EEG-fMRI studies in epileptic patients are analyzed using the general linear model (GLM), which assumes a known hemodynamic response function (HRF) to epileptic spikes. In contrast, independent component analysis (ICA) can extract blood-oxygenation level dependent (BOLD) responses without imposing constraints on the HRF. ICA might therefore detect responses that vary in time and shape, and that are not detected in the GLM analysis. In this study, we compared the findings of ICA and GLM analyses in 12 patients with idiopathic generalized epilepsy. Spatial ICA was used to extract independent components from the functional magnetic resonance imaging (fMRI) data. A deconvolution method identified component time courses significantly related to the generalized EEG discharges, without constraining the shape of the HRF. The results from the ICA analysis were compared to those from the GLM analysis. GLM maps and ICA maps showed significant correlation and revealed BOLD responses in the thalamus, caudate nucleus, and default mode areas. In patients with a low rate of discharges per minute, the GLM analysis detected BOLD signal changes within the thalamus and the caudate nucleus that were not revealed by the ICA. In conclusion, ICA is a viable alternative technique to GLM analyses in EEG-fMRI studies related to generalized discharges. This study demonstrated that the BOLD response largely resembles the standard HRF and that GLM analysis is adequate. However, ICA is more dependent on a sufficient number of events than GLM analysis.

Scheffé-type paired comparison models for examining correlations between individual preferences for alternatives

It is difficult to estimate and examine correlations between individual preferences for alternatives using the present Scheffé-type paired comparison models. In this paper, we propose two models that address individual preferences for alternatives. One is a simple model that makes it possible to estimate correlations between individual preferences. The other is an improved model that makes it possible to extract independent components from those correlations. Paired comparison data were collected in a survey about preferences for several new product names. Analysis of this data shows that the proposed models enabled the estimation not only of average preferences for alternatives, but also correlations between individual preferences and loading matrices for independent components. The effectiveness of the proposed methods was confirmed by the interpretations of those estimates.

Multivariate statistical process monitoring using an improved independent component analysis

An approach for multivariate statistical monitoring based on kernel independent component analysis (Kernel ICA) is presented. Different from the recently developed KICA which means kernel principal component analysis (KPCA) plus independent component analysis (ICA), Kernel ICA is an improvement of ICA and uses contrast functions based on canonical correlations in a reproducing kernel Hilbert space. The basic idea is to use Kernel ICA to extract independent components and later to provide enhanced monitoring of multivariate processes. I 2 (the sum of the squared independent scores) and squared prediction error (SPE) are adopted as statistical quantities. Besides, kernel density estimation (KDE) is described to calculate the confidence limits. The proposed monitoring method is applied to fault detection in the simulation benchmark of the wastewater treatment process and the Tennessee Eastman process, the simulation results clearly show the advantages of Kernel ICA monitoring in comparison to ICA and KICA monitoring.

Nonlinear extraction of independent components of natural images using radial gaussianization.

We consider the problem of efficiently encoding a signal by transforming it to a new representation whose components are statistically independent. A widely studied linear solution, known as independent component analysis (ICA), exists for the case when the signal is generated as a linear transformation of independent nongaussian sources. Here, we examine a complementary case, in which the source is nongaussian and elliptically symmetric. In this case, no invertible linear transform suffices to decompose the signal into independent components, but we show that a simple nonlinear transformation, which we call radial gaussianization (RG), is able to remove all dependencies. We then examine this methodology in the context of natural image statistics. We first show that distributions of spatially proximal bandpass filter responses are better described as elliptical than as linearly transformed independent sources. Consistent with this, we demonstrate that the reduction in dependency achieved by applying RG to either nearby pairs or blocks of bandpass filter responses is significantly greater than that achieved by ICA. Finally, we show that the RG transformation may be closely approximated by divisive normalization, which has been used to model the nonlinear response properties of visual neurons.

Spiking Neurons Can Learn to Solve Information Bottleneck Problems and Extract Independent Components

Independent component analysis (or blind source separation) is assumed to be an essential component of sensory processing in the brain and could provide a less redundant representation about the external world. Another powerful processing strategy is the optimization of internal representations according to the information bottleneck method. This method would allow extracting preferentially those components from high-dimensional sensory input streams that are related to other information sources, such as internal predictions or proprioceptive feedback. However, there exists a lack of models that could explain how spiking neurons could learn to execute either of these two processing strategies. We show in this article how stochastically spiking neurons with refractoriness could in principle learn in an unsupervised manner to carry out both information bottleneck optimization and the extraction of independent components. We derive suitable learning rules, which extend the well-known BCM rule, from abstract information optimization principles. These rules will simultaneously keep the firing rate of the neuron within a biologically realistic range.

Methods of Collection of Biological Information for Fatigue Evaluation during Visual Display Terminals(VDT) Operation

A method for the analysis of fatigue parameters has been rarely researched in VDT operation. Up to now, fatigue was evaluated by changing of biological information. If signals regarding fatigue are detected, fatigue can be measured. The purpose of this study proposed experiment and analysis method to extract parameters related to fatigue from the biological information during VDT operation using the Independent Component Analysis (ICA). An experiment had 11 subjects. As for the experiment were light loaded VDT operation and heavy loaded VDT operation. A measurement item were amount of work, a mistake number, subjective symptom, surface skin temperature (forehead and apex nasi), heart rate, skin blood flow of forearm and respiratory rate. In the heavy loaded operation group, mistake number and subjective symptom score were increased to compare with the other. And Two-factor ANOVA was used for analysis. The result of mistake number was confirmed that heavy loaded. After the moving averages of waveshape were calculated, it was made to extract independent components by using the ICA. The results of the ICA suggest that the independent components increase according to accumulation of fatigue. Thus, the independent components would be a possible parameter of fatigue. However, further experiments should continue in order to obtain the conclusive finding of our research.

Independent component analysis of interictal fMRI in focal epilepsy: Comparison with general linear model-based EEG-correlated fMRI

The general linear model (GLM) has been used to analyze simultaneous EEG–fMRI to reveal BOLD changes linked to interictal epileptic discharges (IED) identified on scalp EEG. This approach is ineffective when IED are not evident in the EEG. Data-driven fMRI analysis techniques that do not require an EEG derived model may offer a solution in these circumstances. We compared the findings of independent components analysis (ICA) and EEG-based GLM analyses of fMRI data from eight patients with focal epilepsy. Spatial ICA was used to extract independent components (IC) which were automatically classified as either BOLD-related, motion artefacts, EPI-susceptibility artefacts, large blood vessels, noise at high spatial or temporal frequency. The classifier reduced the number of candidate IC by 78%, with an average of 16 BOLD-related IC. Concordance between the ICA and GLM-derived results was assessed based on spatio-temporal criteria. In each patient, one of the IC satisfied the criteria to correspond to IED-based GLM result. The remaining IC were consistent with BOLD patterns of spontaneous brain activity and may include epileptic activity that was not evident on the scalp EEG. In conclusion, ICA of fMRI is capable of revealing areas of epileptic activity in patients with focal epilepsy and may be useful for the analysis of EEG–fMRI data in which abnormalities are not apparent on scalp EEG.

Source separation using single channel ICA

Many researchers have recently used independent component analysis (ICA) to generate codebooks or features for a single channel of data. We examine the nature of these codebooks and identify when such features can be used to extract independent components from a stationary scalar time series. This question is motivated by empirical work that suggests that single channel ICA can sometimes be used to separate out important components from a time series. Here we show that as long as the sources are reasonably spectrally disjoint then we can identify and approximately separate out individual sources. However, the linear nature of the separation equations means that when the sources have substantially overlapping spectra both identification using standard ICA and linear separation are no longer possible.

Extraction of Unique Independent Components for Nonlinear Mixture of Sources

In this paper, a neural network solution to extract independent components from nonlinearly mixed signals is proposed. Firstly, a structurally constrained mixing model is introduced to extend the recently proposed mono-nonlinearity mixing model, allowing that different nonlinear distortion are applied to each source signal. Based on this nonlinear mixing model, a novel demixing system characterized by polynomial neural network is then proposed for recovering the original sources. The parameter learning algorithm is derived mathematically based on the minimum mutual information principle. It is shown that unique extraction of independent components can be achieved by optimizing the mutual information cost function under both model structure and signal constraints. In this framework, the theory of series reversion is developed with the aim to perform dual optimization on the polynomials of the proposed demixing system. Finally, simulation results are presented to verify the efficacy of the proposed approach.

Extracting underlying meaningful features and canceling noise using independent component analysis for direct marketing

As the Internet spreads widely, it has become easier for companies to obtain and utilize valuable information on their customers. Nevertheless, many of them have difficulty in using the information effectively because of the huge amount of data from their customers that must to be analyzed. In addition, the data usually contains much noise due to anonymity of the Internet. Consequently, extracting the underlying meanings and canceling the noise of the collected customer data are crucial for the companies to implement their strategies for customer relationship management. As a novel solution, we propose the use of independent component analysis (ICA). ICA is a multivariate statistical tool which extracts independent components or sources of information, given only observed data that are assumed to be linear mixtures of some unknown sources. Moreover, ICA is able to reduce the dimension of the observed data, especially noisy variables. To validate the usefulness of ICA, we applied it to a real-world one-to-one marketing case. In this study, we used ICA as a preprocessing tool, and made a prediction for potential buyers using artificial neural networks (ANNs). We also applied PCA as a comparative model for ICA. The experimental results showed that ICA-preprocessed ANN outperformed all the comparative classifiers without preprocessing as well as PCA-preprocessed ANN.

Fuzzy local independent component analysis with external criteria and its application to knowledge discovery in databases

Local feature values derived by hybrid approaches to fuzzy clustering and multivariate data analysis have been used for knowledge discovery in databases (KDD). They often, however, fail to reveal intrinsic structure because observed variables are easily influenced by external variables. This paper proposes an enhanced technique of local independent component analysis (Local ICA), which extracts independent components uncorrelated to some external criteria. The new technique is applied to knowledge discovery from POS transaction data with the goal of the analysis being to reveal the relationship between the number of customers and days of the week.

Regularised nonlinear blind signal separation using sparsely connected network

A nonlinear approach based on the Tikhonov regularised cost function is presented for blind signal separation of nonlinear mixtures. The proposed approach uses a multilayer perceptron as the nonlinear demixer and combines both information theoretic learning and structural complexity learning into a single framework. It is shown that this approach can be jointly used to extract independent components while constraining the overall perceptron network to be as sparse as possible. The update algorithm for the nonlinear demixer is subsequently derived using the new cost function. Sparseness in the network connection is utilised to determine the total number of layers required in the multilayer perceptron and to prevent the nonlinear demixer from outputting arbitrary independent components. Experiments are meticulously conducted to study the performance of the new approach and the outcomes of these studies are critically assessed for performance comparison with existing methods.