Kernel Independent Component Analysis Research Articles

Process Monitoring of Multimode Processes Using Kernel Independent Component Analysis

In the paper, a new process monitoring approach is proposed for handling the multimode problem in the industrial processes. The original space can be separated into two different parts, which are the common part and the specific part. There are both similarity and dissimilarity in the underlying correlations of different modes, which play different roles in the industrial processes. Because the industrial processes have the non-Gaussian and nonlinear characteristics, modified kernel independent component analysis is used to monitor the multimode processes in this paper. The global multimode basis vector and the multimode sub-basis vector are obtained based on the modified KICA. Then, the common part and specific part in one mode are respectively analyzed. The proposed method is applied to monitor the continuous annealing process. The proposed approach effectively captures the non-Gaussian and nonlinear relationship in different modes in the industrial processes.

Hoist Machinery Fault Monitoring Based on HHT-DDKICA and Support Vector Data Description Method

The shortcomings of the existing feature extraction and machine fault detection methods are analyzed.Combining HHT-DDKICA with support vector data description(SVDD) method,a new fault monitoring algorithm for hoist machine is proposed.Vibration signals of hoist machine are filtered into multiple interesting frequency bands and intrinsic mode functions(IMFs) are obtained through empirical mode decomposition(EMD).Then HHT denoising method and a signal energy criterion are adopted to select effective IMFs.Since single IMF may consist of some nonlinear vibration sources,an alternative data dependent kernel independent component analysis(DDKICA) method is presented to separate source signals.The method can determine a proper kernel function according to training samples,and the optimal model parameter can be also achieved by solving a DDKICA model selection criterion in the empirical feature space.Considering distributions of features extracted by DDKICA,SVDD is adopted to extablish new statistics and confidence limits.Hoist machinery application shows the efficiency of the proposed method.

Decentralized fault diagnosis using multiblock kernel independent component analysis

In this paper, a multiblock kernel independent component analysis (MBKICA) algorithm is proposed. Then a new fault diagnosis approach based on MBKICA is proposed to monitor large-scale processes. MBKICA has superior fault diagnosis ability since variables are grouped and the non-Gaussianity is considered compared to standard kernel methods. The proposed method is applied to fault detection and diagnosis in the continuous annealing process. The proposed decentralized nonlinear approach effectively captures the nonlinear relationship and non-Gaussianity in the block process variables, and shows superior fault diagnosis ability compared to other methods.

A conditional Granger causality model approach for group analysis in functional magnetic resonance imaging

Granger causality model (GCM) derived from multivariate vector autoregressive models of data has been employed to identify effective connectivity in the human brain with functional magnetic resonance imaging (fMRI) and to reveal complex temporal and spatial dynamics underlying a variety of cognitive processes. In the most recent fMRI effective connectivity measures, pair-wise GCM has commonly been applied based on single-voxel values or average values from special brain areas at the group level. Although a few novel conditional GCM methods have been proposed to quantify the connections between brain areas, our study is the first to propose a viable standardized approach for group analysis of fMRI data with GCM. To compare the effectiveness of our approach with traditional pair-wise GCM models, we applied a well-established conditional GCM to preselected time series of brain regions resulting from general linear model (GLM) and group spatial kernel independent component analysis of an fMRI data set in the temporal domain. Data sets consisting of one task-related and one resting-state fMRI were used to investigate connections among brain areas with the conditional GCM method. With the GLM-detected brain activation regions in the emotion-related cortex during the block design paradigm, the conditional GCM method was proposed to study the causality of the habituation between the left amygdala and pregenual cingulate cortex during emotion processing. For the resting-state data set, it is possible to calculate not only the effective connectivity between networks but also the heterogeneity within a single network. Our results have further shown a particular interacting pattern of default mode network that can be characterized as both afferent and efferent influences on the medial prefrontal cortex and posterior cingulate cortex. These results suggest that the conditional GCM approach based on a linear multivariate vector autoregressive model can achieve greater accuracy in detecting network connectivity than the widely used pair-wise GCM, and this group analysis methodology can be quite useful to extend the information obtainable in fMRI.

Moving objects detection based on kernel independent component analysis

In video moving objects detection, the same illumination and perspective, will lead to that moving objects and background is nonlinearly mixed. In this paper, Kernel Independent Component Analysis (KICA) algorithm is proposed to detect the video moving objects. By the selection of kernel's parameters, the proposed algorithm avoids the unreasonable assumption which moving objects image and background image is completely independent in the video moving objects detection based on the Independent Component Analysis (ICA). And it achieves the separation of moving objects and background in the feature space by kernel function. Experimental results demonstrate that KICA is better than ICA in video motion detection.

Soft Sensor Modeling via Support Vector Regression with KICA-Based Feature Extraction

A novel two-level integrated soft sensor modeling method using kernel independent component analysis (KICA) and support vector regression (SVR) is proposed for chemical processes. In the first level, the KICA approach is adopted to extract information of input variables in the high dimensional feature space. Based on this strategy, the correlation of input variables can be eliminated and thus the complexity is reduced. Then, the model is established using SVR in the second level. The KICA-SVR soft sensor modeling method is applied to estimate product compositions in the Tennessee Eastman process. The obtained results show that it can exhibit better performance, compared to the traditional ICA, principal component analysis (PCA) and kernel PCA based information extraction methods, under different operating conditions.

Adaptive kernel independent component analysis and UV spectrometry applied to characterize the procedure for processing prepared rhubarb roots

By determination of the number of absorptive chemical components (ACCs) in mixtures using median absolute deviation (MAD) analysis and extraction of spectral profiles of ACCs using kernel independent component analysis (KICA), an adaptive KICA (AKICA) algorithm was proposed. The proposed AKICA algorithm was used to characterize the procedure for processing prepared rhubarb roots by resolution of the measured mixed raw UV spectra of the rhubarb samples that were collected at different steaming intervals. The results show that the spectral features of ACCs in the mixtures can be directly estimated without chemical and physical pre-separation and other prior information. The estimated three independent components (ICs) represent different chemical components in the mixtures, which are mainly polysaccharides (IC1), tannin (IC2), and anthraquinone glycosides (IC3). The variations of the relative concentrations of the ICs can account for the chemical and physical changes during the processing procedure: IC1 increases significantly before the first 5 h, and is nearly invariant after 6 h; IC2 has no significant changes or is slightly decreased during the processing procedure; IC3 decreases significantly before the first 5 h and decreases slightly after 6 h. The changes of IC1 can explain why the colour became black and darkened during the processing procedure, and the changes of IC3 can explain why the processing procedure can reduce the bitter and dry taste of the rhubarb roots. The endpoint of the processing procedure can be determined as 5-6 h, when the increasing or decreasing trends of the estimated ICs are insignificant. The AKICA-UV method provides an alternative approach for the characterization of the processing procedure of rhubarb roots preparation, and provides a novel way for determination of the endpoint of the traditional Chinese medicine (TCM) processing procedure by inspection of the change trends of the ICs.

3D reconstruction and face recognition using kernel-based ICA and neural networks

Kernel-based nonlinear characteristic extraction and classification algorithms are popular new research directions in machine learning. In this paper, we propose an improved photometric stereo scheme based on improved kernel-independent component analysis method to reconstruct 3D human faces. Next, we fetch the information of 3D faces for facial face recognition. For reconstruction, we obtain the correct normal vector’s sequence to form the surface, and use a method for enforcing integrability to reconstruct 3D objects. We test our algorithm on a number of real images captured from the Yale Face Database B, and use three kinds of methods to fetch characteristic values. Those methods are called contour-based, circle-based, and feature-based methods. Then, a three-layer, feed-forward neural network trained by a back-propagation algorithm is used to realize a classifier. All the experimental results were compared to those of the existing human face reconstruction and recognition approaches tested on the same images. The experimental results demonstrate that the proposed improved kernel independent component analysis (IKICA) method is efficient in reconstruction and face recognition applications.

Application of nonlinear feature extraction and least square support vector machines for fault diagnosis of chemical process

The nonlinear feature extraction(Kernel Principal Component Analysis(KPCA) and kernel Independent Component Analysis(ICA)) was used to eliminate the uncorrelated component from input data,and reduce dimension in the paper.Kernel principal component analysis adopted a kernel function to map input data into feature space and calculate linear PCA.Kernel independent component analysis extracted independent component by linear ICA transformation in the KPCA whitened space.The extracted features were taken as the input of Least Square Support Vector Machine(LSSVM) classifier.Incorporating nonlinear feature extraction into LSSVM served as a new method for intelligent fault diagnosis.The proposed method was applied to the fault diagnosis of lubricating oil process for a petro-plant.The effectiveness of the proposed method is verified.

Multivariate statistical process monitoring using an improved independent component analysis

An approach for multivariate statistical monitoring based on kernel independent component analysis (Kernel ICA) is presented. Different from the recently developed KICA which means kernel principal component analysis (KPCA) plus independent component analysis (ICA), Kernel ICA is an improvement of ICA and uses contrast functions based on canonical correlations in a reproducing kernel Hilbert space. The basic idea is to use Kernel ICA to extract independent components and later to provide enhanced monitoring of multivariate processes. I 2 (the sum of the squared independent scores) and squared prediction error (SPE) are adopted as statistical quantities. Besides, kernel density estimation (KDE) is described to calculate the confidence limits. The proposed monitoring method is applied to fault detection in the simulation benchmark of the wastewater treatment process and the Tennessee Eastman process, the simulation results clearly show the advantages of Kernel ICA monitoring in comparison to ICA and KICA monitoring.

Fast Kernel-Based Independent Component Analysis

Recent approaches to independent component analysis (ICA) have used kernel independence measures to obtain highly accurate solutions, particularly where classical methods experience difficulty (for instance, sources with near-zero kurtosis). FastKICA (fast HSIC-based kernel ICA) is a new optimization method for one such kernel independence measure, the Hilbert-Schmidt Independence Criterion (HSIC). The high computational efficiency of this approach is achieved by combining geometric optimization techniques, specifically an approximate Newton-like method on the orthogonal group, with accurate estimates of the gradient and Hessian based on an incomplete Cholesky decomposition. In contrast to other efficient kernel-based ICA algorithms, FastKICA is applicable to any twice differentiable kernel function. Experimental results for problems with large numbers of sources and observations indicate that FastKICA provides more accurate solutions at a given cost than gradient descent on HSIC. Comparing with other recently published ICA methods, FastKICA is competitive in terms of accuracy, relatively insensitive to local minima when initialized far from independence, and more robust towards outliers. An analysis of the local convergence properties of FastKICA is provided.

Local Image Descriptors Using Supervised Kernel ICA

PCA-SIFT is an extension to SIFT which aims to reduce SIFT's high dimensionality (128 dimensions) by applying PCA to the gradient image patches. However PCA is not a discriminative representation for recognition due to its global feature nature and unsupervised algorithm. In addition, linear methods such as PCA and ICA can fail in the case of non-linearity. In this paper, we propose a new discriminative method called Supervised Kernel ICA (SKICA) that uses a non-linear kernel approach combined with Supervised ICA-based local image descriptors. Our approach blends the advantages of supervised learning with nonlinear properties of kernels. Using five different test data sets we show that the SKICA descriptors produce better object recognition performance than other related approaches with the same dimensionality. The SKICA-based representation has local sensitivity, non-linear independence and high class separability providing an effective method for local image descriptors.

Damage localization in hydraulic turbine blades using kernel-independent component analysis and support vector machines

A hydraulic turbine runner has a complex structure, and traditional source location methods do not have the higher accuracy to meet engineering requirements. The source location of crack acoustic emission (AE) signals in hydraulic turbine blades has been researched by combining it with kernel-independent component analysis (KICA) as feature extraction, with support vector machines (SVMs) as position recognition. This method is compared with those applied SVMs with feature extraction using kernel principal components analysis without feature extraction. The results show that the recognition rate in the crack region is 100 per cent by using both original AE parameters and feature parameters. Support vector regression by feature extraction using KICA can perform better than the other methods. As a result, it is a better method for source location of complex big size structures to combine KICA with SVM. It decreases the dimensionality of input signals and also improves the accuracy of location.

Enhanced statistical analysis of nonlinear processes using KPCA, KICA and SVM

In this paper, some drawbacks of original kernel independent component analysis (KICA) and support vector machine (SVM) algorithms are analyzed for the purpose of multivariate statistical process monitoring (MSPM). When the measured variables follow non-Gaussian distribution, KICA provides more meaningful knowledge by extracting higher-order statistics compared with PCA and kernel principal component analysis (KPCA). However, in real industrial processes, process variables are complex and are not absolutely Gaussian or non-Gaussian distributed. Any single technique is not sufficient to extract the hidden information. Hence, both KICA (non-Gaussion part) and KPCA (Gaussion part) are used for fault detection in this paper, which combine the advantages of KPCA and KICA to develop a nonlinear dynamic approach to detect fault online compared to other nonlinear approaches. Because SVM is available for classifying faults, it is used to diagnose fault in this paper. For above mentioned kernel methods, the calculation of eigenvectors and support vectors will be time consuming when the sample number becomes large. Hence, some dissimilar data are analyzed in the input and feature space. The proposed approach is applied to the fault detection and diagnosis in the Tennessee Eastman process. Application of the proposed approach indicates that proposed method effectively captures the nonlinear dynamics in the process variables.

Improved nonlinear fault detection technique and statistical analysis

AbstractIn this article, first, some drawbacks of original Kernel Principal Component Analysis (KPCA) and Kernel Independent Component Analysis (KICA) are analyzed. Then the KPCA and KICA for multivariate statistical process monitoring (MSPM) are improved. The drawbacks of original KPCA and KICA are as follows: The data mapped into feature space become redundant; linear data introduce errors while the kernel trick is used; computation time increases with the number of samples. To solve the above problems, the original KPCA and KICA for MSPM are improved: similarity factors of the observed data in the input and feature space are defined; similar characteristics are measured; similar data are removed according to the similarity measurements; and k‐means clustering in feature space is used to isolate different classes. Specifically, the similarity concept of data in one group is first proposed. Applications of the proposed approach indicate that improved KPCA and KICA effectively capture the nonlinearities. © 2008 American Institute of Chemical Engineers AIChE J, 2008

Multiway kernel independent component analysis based on feature samples for batch process monitoring

Most batch processes generally exhibit the characteristics of nonlinear variation. In this paper, a nonlinear monitoring technique is proposed using a multiway kernel independent component analysis based on feature samples (FS-MKICA). This approach first unfolds the three-way dataset of a batch process into the two-way one and then chooses representative feature samples from the large two-way input training dataset. The nonlinear feature space abstracted from the unfolded two-way data space is then transformed into a high-dimensional linear space via kernel function and an independent component analysis (ICA) model is established in the mapped linear space. The proposed FS-MKICA method can significantly reduce the computational cost in extracting the kernel ICA model since it is based on the small subset of feature samples rather than on the entire input sample set. We supply two statistics, the I 2 statistic of process variation and the squared prediction error statistic of residual, for on-line monitoring of batch processes. The proposed method is applied to detecting faults in the fed-batch penicillin fermentation process. The standard linear ICA method for batch process monitoring, known as the multiway independent component analysis (MICA), is also applied to the same benchmark batch process. The simulation results obtained in this nonlinear batch process application clearly demonstrate the power and superiority of the new nonlinear monitoring method over the linear one. The FS-MKICA approach can extract the nonlinear features of the batch process while the MICA method cannot.

Sparse Kernel Independent Component Analysis for Blind Source Separation

We address the problem of Blind Source Separation (BSS) of superimposed signals in situations where one signal has constant or slowly varying intensities at some consecutive locations and at the corresponding locations the other signal has highly varying intensities. Independent Component Analysis (ICA) is a major technique for Blind Source Separation and the existing ICA algorithms fail to estimate the original intensities in the stated situation. We combine the advantages of existing sparse methods and Kernel ICA in our technique, by proposing wavelet packet based sparse decomposition of signals prior to the application of Kernel ICA. Simulations and experimental results illustrate the effectiveness and accuracy of the proposed approach. The approach is general in the way that it can be tailored and applied to a wide range of BSS problems concerning one-dimensional signals and images (two-dimensional signals).

Fault Detection and Diagnosis of Nonlinear Processes Using Improved Kernel Independent Component Analysis (KICA) and Support Vector Machine (SVM)

In this article, the nonlinear dynamic process monitoring method based on kernel independent component analysis (KICA) is developed. Compared to the Support Vector Machine (SVM) method, KICA is unsupervised and available for fault detection. Hence, in this article, KICA is used to detect faults. Because the dimension of the feature space is far less than the rank of kernel matrix, a basis in feature space is selected. Specifically, the basis in feature space is first constructed based on the similarity factor of data in one group in this article. A contribution plot is impossible, because the nonlinear mapping function from input space into feature space is unknown. Therefore, KICA is difficult for nonlinear fault diagnosis. In this article, once a fault is detected, the kernel-transformed scores from improved KICA will be directly introduced as the inputs of SVM to diagnose the fault. The classification rate of SVM plus improved KICA is higher than the classification rate of SVM plus KICA when the same n...

Fault Detection of Nonlinear Processes Using Multiway Kernel Independent Component Analysis

In this paper, a new nonlinear process monitoring method that is based on multiway kernel independent component analysis (MKICA) is developed. Its basic idea is to use MKICA to extract some dominant independent components that capture nonlinearity from normal operating process data and to combine them with statistical process monitoring techniques. The proposed method is applied to the fault detection in a fermentation process and is compared with modified independent component analysis (MICA). Applications of the proposed approach indicate that MKICA effectively captures the nonlinear relationship in the process variables and show superior fault detectability, compared to MICA.

Estimation of source infrared spectra profiles of acetylspiramycin active components from troches using kernel independent component analysis

Kernel independent component analysis (KICA), a kind of independent component analysis (ICA) algorithms based on kernel, was preliminarily investigated for blind source separation (BSS) of source spectra profiles from troches. The robustness of different ICA algorithms (KICA, FastICA and Infomax) was first checked by using them in the retrieval of source infrared (IR), ultraviolet (UV) and mass spectra (MS) from synthetic mixtures. It was found that KICA is the most robust method for retrieval of source spectra profiles. KICA algorithm is subsequently adopted in the analysis of diffuse reflection IR of acetylspiramycin (ASPM) troches. It is observed that KICA is able to isolate the theoretically predicted spectral features corresponding to the ASPM active components, excipients and other minor components as different independent (spectral) component. A troche can be authenticated and semi-quantified using the estimated ICs. KICA is an useful method for estimation of source spectral features of molecules with different geometry and stoichiometry, while features belonging to very similar molecules remain grouped.