Extract Independent Components Research Articles

Comparing feedforward neural networks using independent component analysis on hidden units.

Neural networks are widely used for classification and regression tasks, but they do not always perform well, nor explicitly inform us of the rationale for their predictions. In this study we propose a novel method of comparing a pair of different feedforward neural networks, which draws on independent components obtained by independent component analysis (ICA) on the hidden layers of these networks. It can compare different feedforward neural networks even when they have different structures, as well as feedforward neural networks that learned partially different datasets, yielding insights into their functionality or performance. We evaluate the proposed method by conducting three experiments with feedforward neural networks that have one hidden layer, and verify whether a pair of feedforward neural networks can be compared by the proposed method when the numbers of hidden units in the layer are different, when the datasets are partially different, and when activation functions are different. The results show that similar independent components are extracted from two feedforward neural networks, even when the three circumstances above are different. Our experiments also reveal that mere comparison of weights or activations does not lead to identifying similar relationships. Through the extraction of independent components, the proposed method can assess whether the internal processing of one neural network resembles that of another. This approach has the potential to help understand the performance of neural networks.

A Heuristic-Concatenated Feature Classification Algorithm (H-CFCA) for autism and epileptic seizure detection

ObjectiveThe objective is to implement a comprehensive Heuristic-CFCA (H-CFCA) a feature classification system using Machine Learning and Deep Learning principles for detecting Autism and Epileptic Seizures with EEG data. This involves a sequential integration of functional blocks that includes EEG Pre-Processing, Dimensionality Reduction, Feature Extraction, and a Concatenated Feature Classification Scheme (CFCA). MethodsThe methodology involves db4 Wavelet-based decomposition (WD) for pre-processing the EEG into five frequency bands, followed by Singular Value Decomposition (SVD) for dimensionality reduction. Subsequently, Heuristic Independent Component Analysis (H-ICA) is applied to extract Independent Components (ICs). These ICs are fed to H-CFCA comprises of Support Vector Machine (SVM) as first stage classifier, followed by a Bi-Stack Long Short-Term Memory (BiS-LSTM) as second stage classifier to detect the presence of autism and epileptic seizure. ResultsThe evaluation of the proposed algorithm achieves an accuracy of 99.51 % and a Sensitivity of 98.86 % and F1 score of 99.32 % ConclusionThe article realizes an effective and feasible H-CFCA, with an ability to handle large size EEG datasets and achieves reduced computational complexity, and improved feature classification. Comparative analysis shows that H-CFCA achieves the highest accuracy and sensitivity compared to existing ML/DL-based algorithms. SignificanceThe EEG feature classification system significantly advances diagnostic accuracy, improves social well-being by enabling personalized interventions, and drives technological innovation in healthcare by leveraging advanced classification methods and data analysis techniques.

Induction Motor Noise Source Separation and Identification Based on Adaptive Scale-Space Mode Extraction

Separating induction motor noise sources can provide an important reference basis for induction motor condition detection, noise reduction treatment, and fault diagnosis. Induction motors have different types of noise sources that partially overlap, and most radiate outward through the housing, so it is difficult to separate these noise sources. Therefore, a single-channel induction motor noise source separation and identification method, based on adaptive scale-space modal extraction (ASSME) is proposed. Firstly, the adaptive scale-space mode extraction method is proposed by constructing the electromagnetic feature scale space and the adaptive penalty factor. The simulation results show that this method solves over-decomposition problems in the classical scale-space variational mode decomposition and the difficulty in balancing the harmonic and shock modes. Secondly, motor noise experiments are conducted to construct blind source separation multi-channel inputs using the adaptive scale-space modal extraction method, judging the validity of the modal components using correlation and the variance contribution rate. Finally, robust independent component analysis (RobustICA) is used to extract independent noise components and identify these noise sources by power spectral density and envelope analysis. The results show that the multi-channel input signals obtained by the proposed method are more accurate and practical than those obtained by other methods. The independent components extracted through this noise source separation method are: electromagnetic noise of different orders, aerodynamic noise, and switching frequency noise.

Attention Deficit Hyperactivity Disorder Classification Based on Deep Learning.

Attention Deficit Hyperactivity Disorder (ADHD) is a type of mental health disorder that can be seen from children to adults and affects patients' normal life. Accurate diagnosis of ADHD as early as possible is very important for the treatment of patients in clinical applications. Some traditional classification methods, although having been shown powerful in many other classification tasks, are not as successful in the application of ADHD classification. In this paper, we propose two novel deep learning approaches for ADHD classification based on functional magnetic resonance imaging. The first method incorporates independent component analysis with convolutional neural network. It first extracts independent components from each subject. The independent components are then fed into a convolutional neural network as input features to classify the ADHD patient from typical controls. The second method, called the correlation autoencoder method, uses correlations between regions of interest of the brain as the input of an autoencoder to learn latent features, which are then used in the classification task by a new neural network. These two methods use different ways to extract the inter-voxel information from fMRI, but both use convolutional neural networks to further extract predictive features for the classification task. Empirical experiments show that both methods are able to outperform the classical methods such as logistic regression, support vector machines, and other methods used in previous studies.

Successive variational mode decomposition and blind source separation based on salp swarm optimization for bearing fault diagnosis

In this paper, we are interested in developing a new approach that combines successive variational mode decomposition and blind source separation based on salp swarm optimization for bearing fault diagnosis. Firstly, vibration signals are pre-processed using successive variational mode decomposition to increase the signal-to-noise ratio. Then, the dynamic time-warping algorithm is adopted to select the most effective modes which will be considered mixture signals. In the second step, we apply the salp swarm algorithm (SSA) for estimating the de-mixing matrix to extract independent components from mixture signals. However, SSA suffers from the problem of population diversity. Consequently, it offers somewhat different independent sources at every execution of the program. To overcome this shortcoming, the SSA-based source estimation will be executed several times with different ranges of initial positions. Then, a fuzzy C-mean algorithm is introduced to select the reliable independent components. The suggested method is tested based on two experiments and compared to state-of-the-art methods. The obtained results demonstrate the effectiveness of the suggested method in recovering reliable independent components and extracting the fault frequency of bearings.

Template Independent Component Analysis with Spatial Priors for Accurate Subject-Level Brain Network Estimation and Inference

Independent component analysis is commonly applied to functional magnetic resonance imaging (fMRI) data to extract independent components (ICs) representing functional brain networks. While ICA produces reliable group-level estimates, single-subject ICA often produces noisy results. Template ICA is a hierarchical ICA model using empirical population priors to produce more reliable subject-level estimates. However, this and other hierarchical ICA models assume unrealistically that subject effects are spatially independent. Here, we propose spatial template ICA (stICA), which incorporates spatial priors into the template ICA framework for greater estimation efficiency. Additionally, the joint posterior distribution can be used to identify brain regions engaged in each network using an excursions set approach. By leveraging spatial dependencies and avoiding massive multiple comparisons, stICA has high power to detect true effects. We derive an efficient expectation-maximization algorithm to obtain maximum likelihood estimates of the model parameters and posterior moments of the latent fields. Based on analysis of simulated data and fMRI data from the Human Connectome Project, we find that stICA produces estimates that are more accurate and reliable than benchmark approaches, and identifies larger and more reliable areas of engagement. The algorithm is computationally tractable, achieving convergence within 12 hr for whole-cortex fMRI analysis. Supplementary materials for this article are available online.

Brain state instability as a biomarker of Alzheimer’s disease progression: A dynamic functional network connectivity study

AbstractBackgroundFinding a biomarker predicting the Alzheimer’s disease (AD) progression from a healthy stage to mild cognitive impairment (MCI), mild dementia (MD) and further to AD is an essential step toward early medical intervention. In recent years, dynamic functional network connectivity (dFNC) from resting state‐fMRI, which estimates brain states during the scan, revealed excellent knowledge about AD progression's underlying mechanism. The current study explored whether the AD brain produces similar and stable dFNC states across different scanning sessions and introduced dFNC state (or brain) instability as a potential biomarker of AD progression.MethodThe data from the longitudinal Open Access Series of Imaging Studies (OASIS)‐3 cohort were used in this study. The dataset contains two rs‐fMRI sessions on the same day of 1048 healthy controls or HC (clinical dementia rating sum of boxes or CDR‐SOB=0, age= 70.80± 8.57), 193 MCI (4.5=<CDR‐SOB<=9, age= 71.00±9.48), and 75 MD patients (with clinical dementia rating 4.5=<CDR‐SOB<=9, age=71.90±9.65). We did not observe a significant age and gender difference across groups (ttest p>0.2). We used Group‐independent component analysis to extract independent components of the whole‐brain for each subject. Next, we applied the sliding window with Pearson correlation to estimate dFNC of each individual in each session. Subsequently, k‐means clustering (k=3, L1‐norm as a distance metric, and 1000 iterations) separately was applied to put the dFNCs of each group and session in three distinct states. Pearson correlation between the cluster (state) centroid was used to assess the similarity of state across two sessions.ResultWe found that HC and MCI groups generate similar dFNC states in both sessions in which The correlation between inter‐session “matched” states for HCs and MCIs was more than 0.99 (p<0.0001). But, MD patients did not generate similar brain states across two sessions. This points to a degradation in the stability of functional brain integration among MD patients.ConclusionOur findings highlight brain instability as a potential biomarker of Alzheimer’s disease progression. Also, the instability of dFNC observed in mild dementia patients suggests a need for multiple scanning sessions to analyze rs‐fMRI data in this group of patients.

Constrained independent vector extraction of quasi-periodic signals from multiple data sets

In this paper, we consider a problem to extract independent source component vector (SCV) formed by quasi-periodic signals from instantaneous mixtures in multiple data sets. We propose a method, termed as constrained independent vector extraction (CIVE), to uniquely determine the target quasi-periodic SCV. Specifically, the negentropy is taken to enforce the independence of the target SCV from the others, while the mutual information is used to determine the correlation of sources within the target SCV. A quasi-periodic constraint is further combined in the cost function to ensure the quasi-periodicity of the SCV. The demixing vectors of the target SCV are solved as a constrained optimization problem by the Lagrange multiplier method. The CIVE method is designed to work under diverse probability distributions for the mixed signals. In the experiments, the CIVE method is verified with both simulated and semi-simulated data. The comparison results with other methods indicate the effectiveness, applicability and stability of the proposed method for extracting quasi-periodic SCVs.

Identification of cardiovascular high-risk groups from dynamic retinal vessel signals using untargeted machine learning.

Dynamic retinal vessel analysis (DVA) provides a non-invasive way to assess microvascular function in patients and potentially to improve predictions of individual cardiovascular (CV) risk. The aim of our study was to use untargeted machine learning on DVA in order to improve CV mortality prediction and identify corresponding response alterations. We adopted a workflow consisting of noise reduction and extraction of independent components within DVA signals. Predictor performance was assessed in survival random forest models. Applying our technique to the prediction of all-cause mortality in a cohort of 214 haemodialysis patients resulted in the selection of a component which was highly correlated to maximal venous dilation following flicker stimulation (vMax), a previously identified predictor, confirming the validity of our approach. When fitting for CV mortality as the outcome of interest, a combination of three components derived from the arterial signal resulted in a marked improvement in predictive performance. Clustering analysis suggested that these independent components identified groups of patients with substantially higher CV mortality. Our results provide a machine learning workflow to improve the predictive performance of DVA and identify groups of haemodialysis patients at high risk of CV mortality. Our approach may also prove to be promising for DVA signal analysis in other CV disease states.

Statistical monitoring for non-Gaussian processes based on MICA-KDR method

The focus of the current work attempts to propose a purely data-based model for generating residuals for non-Gaussian process monitoring purposes, the idea of residual generation is borrowed from the field of model-based fault detection and applied in statistical monitoring, the generated residual instead of the measured variables is thus modeled and monitored. The proposed approach first employs the modified independent component analysis (MICA) algorithm to extract independent components (ICs) from a given dataset. Secondly, through assuming but only one variable is missing at one time, the known data regression (KDR) method dealing with missing data problem is then used for estimating the corresponding ICs. The inconsistency between the actual and estimated ICs is called residual and may present much lower level of non-Gaussianity, in contrast to the actual ICs. Thirdly, a principal component analysis based statistical monitoring model can be utilized for online fault detection based on the generated residual. Finally, the superiority and efficiency of the MICA-KDR approach over its counterparts are validated by implementing comparisons on two industrial processes, the proposed MICA-KDR method is demonstrated to be a comparative alternative in monitoring non-Gaussian processes

A Deep Learning-Based Compression Algorithm for 9-DOF Inertial Measurement Unit Signals Along With an Error Compensating Mechanism

Inertial sensors with microelectromechanical systems technology are an integral part of many modern electronic devices such as wearable medical products, which are inherently subject to memory, bandwidth, and energy constraints due to their size and purpose. One of the important challenges for the progress in this area is the storage, transmission, and processing of large quantities of inertial sensors signal. To address this issue, this paper presents a method for near-lossless compression of multi-axis inertial signals. To improve the inertial signal compression capability, the proposed compression method employs the independent component analysis method with a principal component analysis preprocessing step to extract independent components from the signals. A deep autoencoder is used to compress the independent components and later to estimate them in the reconstruction phase. The reconstruction error is also quantized and coded using arithmetic coding and transmitted alongside the compressed components. This paper also proposes a new approach for improving the quality of the reconstructed signals. In this approach, on the receiver side, the reconstruction error is fed to the Madgwick filter as an external noise and is compensated using this filter. The experimental results demonstrate the high compression rate and low reconstruction error of the proposed method compared to the state-of-the-art methods.

Classification of EEG signals for detection of epileptic seizure activities based on feature extraction from brain maps using image processing algorithms

This study presents a novel feature extraction approach based on image processing algorithms for the automated detection of epileptic seizure activities in brain map representation of electroencephalography (EEG) signal using an efficient classification technique. The proposed technique uses independent component analysis to extract independent components (ICs) from the EEG signal and each extracted IC is transformed into an image termed as brain maps. Two feature extraction techniques namely closed neighbourhood gradient pattern (CNGP) and combined texture pattern (CTP) are propounded for automatic elimination of artefact brain maps. The extracted features are fed into the least square support vector machine (LSSVM) for automatic detection of epileptic brain maps. Extensive experimental result over the existing image processing techniques in literature demonstrates that the texture pattern representations of CNGP and CTP are improved to obtain better features to enhance the performance of texture classification. The obtained result shows that the LSSVM classifier with Gaussian RBF kernel is able to detect the epileptic brain map with a high accuracy rate. The results are reliable and it assists the neurologist to diagnose epileptic signals effortlessly by visually locating the brain area being affected by seizure activities.

Non-linear ICA Analysis of Resting-State fMRI in Mild Cognitive Impairment

Compared to linear independent component analysis (ICA), non-linear ICA is more suitable for the decomposition of mixed components. Existing studies of functional magnetic resonance imaging (fMRI) data by using linear ICA assume that the brain's mixed signals, which are caused by the activity of brain, are formed through the linear combination of source signals. But the application of the non-linear combination of source signals is more suitable for the mixed signals of brain. For this reason, we investigated statistical differences in resting state networks (RSNs) on 32 healthy controls (HC) and 38 mild cognitive impairment (MCI) patients using post-nonlinear ICA. Post-nonlinear ICA is one of the non-linear ICA methods. Firstly, the fMRI data of all subjects was preprocessed. The second step was to extract independent components (ICs) of fMRI data of all subjects. In the third step, we calculated the correlation coefficient between ICs and RSN templates, and selected ICs of the largest spatial correlation coefficient. The ICs represent the corresponding RSNs. After finding out the eight RSNs of MCI group and HC group, one sample t-tests were performed. Finally, in order to compare the differences of RSNs between MCI and HC groups, the two-sample t-tests were carried out. We found that the functional connectivity (FC) of RSNs in MCI patients was abnormal. Compared with HC, MCI patients showed the increased and decreased FC in default mode network (DMN), central executive network (CEN), dorsal attention network (DAN), somato-motor network (SMN), visual network(VN), MCI patients displayed the specifically decreased FC in auditory network (AN), self-referential network (SRN). The FC of core network (CN) did not reveal significant group difference. The results indicate that the abnormal FC in RSNs is selective in MCI patients.

Quality-relevant independent component regression model for virtual sensing application

Independent component regression (ICR) is an efficient method for tackling non-Gaussian problems. In this work, the defects of the conventional ICR are analyzed, and a novel quality-relevant independent component regression (QR-ICR) method based on distance covariance and distance correlation is proposed. QR-ICR extracts independent components (ICs) using a quality-relevant independent component analysis (QR-ICA) algorithm, which simultaneously maximizes the non-Gaussianity of ICs and statistical dependency between ICs and quality variables. Meanwhile, two new types of statistical criteria, called cumulative percent relevance (CPR) and Max-Dependency (Max-Dep), are proposed to rank the order and determine the number of ICs according to their contributions to quality variables. The proposed QR-ICR(CPR) and QR-ICR(Max-Dep) methods were validated through a vinyl acetate monomer production process and a benchmark near-infrared spectral data. The results have demonstrated that the proposed QR-ICR(CPR) and QR-ICR(Max-Dep) provide simpler predictive models and give better prediction performances than PLS, ICR, ICR(CPR), and ICR(Max-Dep).

Statistical Process Monitoring with Biogeography-Based Optimization Independent Component Analysis

Independent Component Analysis (ICA), a type of typical data-driven fault detection techniques, has been widely applied for monitoring industrial processes. FastICA is a classical algorithm of ICA, which extracts independent components by using the Newton iteration method. However, the choice of the initial iterative point of Newton iteration method is difficult; sometimes, selection of different initial iterative points tends to show completely different effects for fault detection. So far, there is still no good strategy to get an ideal initial iterative point for ICA. To solve this problem, a modified ICA algorithm based on biogeography-based optimization (BBO) called BBO-ICA is proposed for the purpose of multivariate statistical process monitoring. The Newton iteration method is replaced with BBO here for extracting independent components. BBO is a novel and effective optimization method to search extremes or maximums. Comparing with the traditional intelligent optimization algorithm of particle swarm optimization (PSO) and so on, BBO behaves with stronger capability and accuracy of searching for solution space. Moreover, numerical simulations are finished with the platform of DAMADICS. Results demonstrate the practicability and effectiveness of BBO-ICA. The proposed BBO-ICA shows better performance of process monitoring than FastICA and PSO-ICA for DAMADICS.

Changing brain connectivity dynamics: From early childhood to adulthood.

Brain maturation through adolescence has been the topic of recent studies. Previous works have evaluated changes in morphometry and also changes in functional connectivity. However, most resting-state fMRI studies have focused on static connectivity. Here we examine the relationship between age/maturity and the dynamics of brain functional connectivity. Utilizing a resting fMRI dataset comprised 421 subjects ages 3-22 from the PING study, we first performed group ICA to extract independent components and their time courses. Next, dynamic functional network connectivity (dFNC) was calculated via a sliding window followed by clustering of connectivity patterns into 5 states. Finally, we evaluated the relationship between age and the amount of time each participant spent in each state as well as the transitions among different states. Results showed that older participants tend to spend more time in states which reflect overall stronger connectivity patterns throughout the brain. In addition, the relationship between age and state transition is symmetric. This can mean individuals change functional connectivity through time within a specific set of states. On the whole, results indicated that dynamic functional connectivity is an important factor to consider when examining brain development across childhood.

Identification of maturity of cuiguan pear based on RobustICA and PSO-ELM

In order to improve the stability of extreme learning machine (ELM), it was proposed to use the combination of robust independent component analysis (RobustICA) and particle swarm optimization ELM algorithm. It was applied to identify the maturity of cuiguan pear at different growing periods. First the two order derivative and the discrete wavelet transform were adopted to remove the background noise and compression spectra. Then RobustICA was used to decompose the spectra to extract independent components and mixed coefficient matrices. Finally, by using particle swarm optimization, the best input layer weight and hidden layer deviation were obtained and adjusted the number of hidden layer nodes to get the best ELM model to differentiate maturity. The method was used to identify the maturity of cuiguan pear at four different stages, and has strong robustness. The results showed that RobustICA-PSO-ELM had better discrimination ability compared with RobustICA - ELM and ELM, and the accuracy increases from 75% to 94.4%.

Application of Nonnegative Tensor Factorization for neutron-gamma discrimination of Monte Carlo simulated fission chamber’s output signals

For efficient exploitation of research reactors, it is important to discern neutron flux distribution inside the reactor with the best possible precision. For this reason, fission and ionization chambers are used to measure the neutron field. In these arrays, the sequences of the neutron interaction points in the fission chamber can correctly be identified in order to obtain true neutron energies emitted by nuclei of interest. However, together with the neutrons, gamma-rays are also emitted from nuclei and thereby affect neutron spectra. The originality of this study consists in the application of tensor based blind source separation methods to extract independent components from signals recorded at the fission chamber preamplifier’s output. The objective is to achieve software neutron-gamma discrimination using Nonnegative Tensor Factorization tools. For reasons of nuclear safety, we first simulate the neutron flux inside the TRIGA Mark II Reactor using Monte Carlo methods under Geant4 platform linked to Garfield++. Geant4 simulations allow the fission chamber construction whereas linking the model to Garfield++ permits to simulate drift parameters from the ionization of the filling gas, which is not possible otherwise.

An Anomaly Detection Framework Based on ICA and Bayesian Classification for IaaS Platforms

Infrastructure as a Service (IaaS) encapsulates computer hardware into a large amount of virtual and manageable instances mainly in the form of virtual machine (VM), and provides rental service for users. Currently, VM anomaly incidents occasionally occur, which leads to performance issues and even downtime. This paper aims at detecting anomalous VMs based on performance metrics data of VMs. Due to the dynamic nature and increasing scale of IaaS, detecting anomalous VMs from voluminous correlated and non-Gaussian monitored performance data is a challenging task. This paper designs an anomaly detection framework to solve this challenge. First, it collects 53 performance metrics to reflect the running state of each VM. The collected performance metrics are testified not to follow the Gaussian distribution. Then, it employs independent components analysis (ICA) instead of principal component analysis (PCA) to extract independent components from collected non-Gaussian performance metric data. For anomaly detection, it employs multi-class Bayesian classification to determine the current state of each VM. To evaluate the performance of the designed detection framework, four types of anomalies are separately or jointly injected into randomly selected VMs in a campus-wide testbed. The experimental results show that ICA-based detection mechanism outperforms PCA-based and LDA-based detection mechanisms in terms of sensitivity and specificity.

Dynamical analysis of contrastive divergence learning: Restricted Boltzmann machines with Gaussian visible units

The restricted Boltzmann machine (RBM) is an essential constituent of deep learning, but it is hard to train by using maximum likelihood (ML) learning, which minimizes the Kullback–Leibler (KL) divergence. Instead, contrastive divergence (CD) learning has been developed as an approximation of ML learning and widely used in practice. To clarify the performance of CD learning, in this paper, we analytically derive the fixed points where ML and CDn learning rules converge in two types of RBMs: one with Gaussian visible and Gaussian hidden units and the other with Gaussian visible and Bernoulli hidden units. In addition, we analyze the stability of the fixed points. As a result, we find that the stable points of CDn learning rule coincide with those of ML learning rule in a Gaussian–Gaussian RBM. We also reveal that larger principal components of the input data are extracted at the stable points. Moreover, in a Gaussian–Bernoulli RBM, we find that both ML and CDn learning can extract independent components at one of stable points. Our analysis demonstrates that the same feature components as those extracted by ML learning are extracted simply by performing CD1 learning. Expanding this study should elucidate the specific solutions obtained by CD learning in other types of RBMs or in deep networks.