Incremental Principal Component Analysis Research Articles

Self-Adaptive Incremental PCA-Based DBSCAN of Acoustic Features for Anomalous Sound Detection

In modern industry, maintaining continuous machine operations is important for improving production efficiency and reducing costs. Therefore, the smart technology of acoustic monitoring to detect anomalous machine conditions earlier before breakdowns works as part of predictive maintenance and is applied not only in industry fault detection but also in safety monitoring and surveillance systems. This paper proposes a self-adaptive unsupervised machine learning algorithm with dimension-reduction technology to detect anomalous sounds after extracting acoustic machine features. Technically, the automatic EPS calculation algorithm-based genetic algorithm optimizes the automatic clustering algorithm’s configuration for incremental principal component analysis and density-based spatial clustering algorithms with noise. IPCA is enhanced by the sequential Karhunen–Loeve (SKL) algorithm, and the condensation algorithm works as the second layer of the algorithm to reduce the number of effective components. This architecture could select an optimized set of parameters based on different test environments and keeps performance quality with fewer computational requirements. In the experiments, 228 sets of normal sounds and 100 sets of anomaly sounds are used. The sound files are collected from the same machine type (stepper motors) at a real plant site. We compare the proposed algorithm with K-means++, one-class SVM, agglomerative clustering, DCGAN and DCNN-Autoencoder, and this new algorithm performs best, with an AUC of 0.84 and the shortest execution time. The algorithm is generic and can be applied to detect anomalies in machines to provide early warning to people to avoid serious accidents or disasters.

Advanced wind turbine blade inspection with hyperspectral imaging and 3D convolutional neural networks for damage detection

In the context of global efforts to mitigate climate change by pursuing sustainable energy sources, wind energy has emerged as a critical contributor. However, the wind energy industry faces substantial challenges in maintaining and preserving the integrity of wind turbine blades. Timely and accurate detection and classification of blade faults, encompassing issues such as cracks, erosion, and ice buildup, are imperative to uphold wind turbines' ongoing efficiency and safety. This study introduces an inventive approach that amalgamates hyperspectral imaging and 3D Convolutional Neural Networks (CNNs) to augment the precision and efficiency of wind turbine blade fault detection and classification. Hyperspectral imaging is harnessed to capture comprehensive spectral information from blade surfaces, facilitating exact fault identification. The process is streamlined through Incremental Principal Component Analysis (IPCA), reducing data dimensions while maintaining integrity. The 3D CNN model demonstrates remarkable performance, achieving high accuracy in detecting all fault categories in full-band hyperspectral images. The model retains high accuracy even with dimensionality reduction to 20 spectral bands. The reduced processing time of the 20-band image enhances the practicality of real-world applications, thereby reducing downtime and maintenance expenditures. This research represents a significant advancement in wind turbine blade inspection, contributing to the sustainability and dependability of wind energy systems and furthering the cause of a cleaner and more sustainable energy future as part of the broader fight against climate change.

On the Search for Potentially Anomalous Traces of Cosmic Ray Particles in Images Acquired by Cmos Detectors for a Continuous Stream of Emerging Observational Data.

In this paper we propose the method for detecting potential anomalous cosmic ray particle tracks in big data image dataset acquired by Complementary Metal-Oxide-Semiconductors (CMOS). Those sensors are part of scientific infrastructure of Cosmic Ray Extremely Distributed Observatory (CREDO). The use of Incremental PCA (Principal Components Analysis) allowed approximation of loadings which might be updated at runtime. Incremental PCA with Sequential Karhunen-Loeve Transform results with almost identical embedding as basic PCA. Depending on image preprocessing method the weighted distance between coordinate frame and its approximation was at the level from 0.01 to 0.02 radian for batches with size of 10,000 images. This significantly reduces the necessary calculations in terms of memory complexity so that our method can be used for big data. The use of intuitive parameters of the potential anomalies detection algorithm based on object density in embedding space makes our method intuitive to use. The sets of anomalies returned by our proposed algorithm do not contain any typical morphologies of particle tracks shapes. Thus, one can conclude that our proposed method effectively filter-off typical (in terms of analysis of variance) shapes of particle tracks by searching for those that can be treated as significantly different from the others in the dataset. We also proposed method that can be used to find similar objects, which gives it the potential, for example, to be used in minimal distance-based classification and CREDO image database querying. The proposed algorithm was tested on more than half a million (570,000+) images that contains various morphologies of cosmic particle tracks. To our knowledge, this is the first study of this kind based on data collected using a distributed network of CMOS sensors embedded in the cell phones of participants collaborating within the citizen science paradigm.

A lightweight intrusion detection method for IoT based on deep learning and dynamic quantization.

Intrusion detection ensures that IoT can protect itself against malicious intrusions in extensive and intricate network traffic data. In recent years, deep learning has been extensively and effectively employed in IoT intrusion detection. However, the limited computing power and storage space of IoT devices restrict the feasibility of deploying resource-intensive intrusion detection systems on them. This article introduces the DL-BiLSTM lightweight IoT intrusion detection model. By combining deep neural networks (DNNs) and bidirectional long short-term memory networks (BiLSTMs), the model enables nonlinear and bidirectional long-distance feature extraction of complex network information. This capability allows the system to capture complex patterns and behaviors related to cyber-attacks, thus enhancing detection performance. To address the resource constraints of IoT devices, the model utilizes the incremental principal component analysis (IPCA) algorithm for feature dimensionality reduction. Additionally, dynamic quantization is employed to trim the specified cell structure of the model, thereby reducing the computational burden on IoT devices while preserving accurate detection capability. The experimental results on the benchmark datasets CIC IDS2017, N-BaIoT, and CICIoT2023 demonstrate that DL-BiLSTM surpasses traditional deep learning models and cutting-edge detection techniques in terms of detection performance, while maintaining a lower model complexity.

A novel hybrid feature extraction and ensemble C3D classification for anomaly detection in surveillance videos

Anomaly detection in several deep learning frameworks are recently presented on real-time video databases as a challenging task. However, these frameworks have high false positive rate (FPR) and error rate due to various backgrounds, motion appearance and semantic high-level and low-level features for anomaly detection through action classification. Also, extraction of features and classification are the major problems in traditional convolution neural network (CNN) on real-time video databases. The proposed work is a novel action classification framework which is designed and implemented on large video databases with high true positive rate (TPR) and error rate. In this framework, Kalman based incremental principal component analysis (IPCA) feature extraction method; C3D and non-linear support vector machine (SVM) classifier are used to improve the action prediction (anomaly detection) on the large real-time video databases. The proposed frame work shown new results of high computation performance than the traditional deep learning frameworks for action classification.

Hyperspectral Image Classification Using 3D Capsule-Net Based Architecture

Convolution neural networks have received much interest recently in the categorization of hyperspectral images (HSI). Deep learning requires a large number of labeled samples in order to optimize numerous parameters due to the expansion of architecture depth and feature aggregation. Unfortunately, only few examples with labels are accessible, and the majority of spectral images are not labeled. For HSI categorization, the difficulty is how to acquire richer features with constrained training data. In order to properly utilize HSI features at various scales, a 3D Capsule-Net based supervised architecture is presented in this paper for HSI classification. First, the input data undergo incremental principal component analysis (IPCA) for dimensionality reduction. The reduced data are then divided into windows and given to a 3D convolution layer to get the shallow features. These shallow features are then used by 3D Capsule-Net to compute high-level features for classification of HSI. Experimental investigation on three common datasets demonstrates that the categorization performance by Capsule-Net based architecture exceeds a number of other state-of-the-art approaches.

Introduction of a Novel Anomalous Sound Detection Methodology

This paper is to introduce a novel semi-supervised methodology, the enhanced incremental principal component analysis (“IPCA”) based deep convolutional neural network autoencoder (“DCNN-AE) for Anomalous Sound Detection (“ASD”) with high accuracy and computing efficiency. This hybrid methodology is to adopt Enhanced IPCA to reduce the dimensionality and then to use DCNN-AE to extract the features of the sample sound and detect the anomality. In this project, 228 sets of normal sounds and 100 sets of anomaly sounds of same machine are used for the experiments. And the sound files of machines (stepper motors) for the experiments are collected from a plant site. 50 random test cases are executed to evaluate the performance of the algorithm with AUC, PAUC, F measure and Accuracy Score. IPCA Based DCNN-AE shows high accuracy with the average AUC of 0.815793282, comparing with that of Kmeans++ of 0.499545351, of Incremental PCA based DBSCAN clustering of 0.636348073, of Incremental based PCA based One-class SVM of 0.506749433 and of DCGAN of 0.716528104. From the perspective of computing efficiency, because of the dimensions-reduction by the IPCA layer, the average execution time of the new methodology is 15 minutes in the CPU computing module of 2.3 GHz quad-core processors, comparing with that of DCGAN with 90 minutes in GPU computing module of 4 to 8 kernels.

Incremental learning of phase transition in Ising model: Preprocessing, finite-size scaling and critical exponents

We investigated an unsupervised machine learning to recognize the phase transition in Ising model by using principal component analysis (PCA) with a preprocessing of raw data, and we realized the large-scale calculations and finite-size scaling analysis based on the incremental PCA. It was shown that the two different phases are clearly and reasonably recognized by the k-means clustering of the first and second principal components. Taking the normalized first principal component as the order parameter, we calculated the critical properties near the phase transition point by using the finite-size scaling method. Our results of the critical temperature and critical exponents are consistent with the classical values, which indicates that the first principal component is capable of catching main features of the phase transition. The present scheme provides an alternative way with emphasis on data preprocessing and incremental PCA for the unsupervised machine learning of the phase transition in Ising model and its related spin models.

Incremental robust PCA for vessel segmentation in DSA sequences

In intervention surgery, DSA images provide a new way to observe the vessels and catheters inside the patient. Extracting coronary artery from the dynamic complex background fast improves the effectiveness directly in clinical interventional surgery. This article proposes an incremental robust principal component analysis (IRPCA) method to extract contrast-filled vessels from x-ray coronary angiograms. RPCA is a matrix decomposition method that decomposes a video matrix into foreground and background, commonly used to model complex backgrounds and extract target objects. IRPCA pre-optimizes an x-ray image sequence. When a new x-ray sequence is received, IRPCA optimizes it based on the pre-optimized matrix according to the strategy of minimizing the energy function to obtain the foreground matrix of the new sequence. Besides, based on the idea that the new x-ray sequence introduces new information to the pre-optimized matrix, we propose UIRPCA to improve the performence of IRPCA. Compared with the traditional RPCA method, IRPCA and UIRPCA save much time while ensuring that other indicators remain basically unchanged. The experiment results based on real data show the superiority of the proposed method over other RPCA algorithms.

Spatio-Temporal Coding-Based Helicopter Trajectory Planning for Pulsed Neural Membrane System.

For the trajectory planning problem under the nonlinear and strongly coupled characteristics of unmanned helicopters, membrane computing with distributed parallel processing capability is introduced for unmanned helicopter trajectory planning. The global and local spatial information is temporally characterized; the temporal characterization algorithm under mapping information is designed; the hierarchical discriminant regression algorithm is designed based on incremental principal component analysis to realize the process of building and identifying trees in trajectory planning; and the pulsed neural membrane system (PNMS) with spatio-temporal coding function under membrane computing is constructed. Compared with the RRT algorithm in two experimental environments, the original path length, the trimmed path length, the time used to plan the trajectory, and the number of search nodes have different levels of improvement; the feasibility and effectiveness of the PNMS in unmanned helicopter trajectory planning are verified. It expands the theoretical research of membrane computing in the field of optimal control and provides theoretical support for the subsequent application practice.

Efficient Driver Drunk Detection by Sensors: A Manifold Learning-Based Anomaly Detector

This study presents an effective data-driven anomaly detection scheme for drunk driving detection. Specifically, the proposed anomaly detection approach amalgamates the desirable features of the t-distributed stochastic neighbor embedding (t-SNE) as a feature extractor with the Isolation Forest (iF) scheme to detect drivers’ drunkenness status. We used the t-SNE model to exploit its capacity in reducing the dimensionality of nonlinear data by preserving the local and global structures of the input data in the feature space to obtain good detection. At the same time, the iF scheme is an effective and unsupervised tree-based approach to achieving good detection of anomalies in multivariate data. This approach only employs normal events data to train the detection model, making them more attractive for detecting drunk drivers in practice. To verify the detection capacity of the proposed t-SNE-iF approach in reliably detecting drivers with excess alcohol, we used publically available data collected using a gas sensor, temperature sensor, and a digital camera. The overall detection system proved a high detection performance with AUC around 95%, demonstrating the proposed approach’s robustness and reliability. Furthermore, compared to the Principal Component Analysis (PCA), Incremental PCA (IPCA), Independent component analysis (ICA), Kernel PCA (kPCA), and Multi-dimensional scaling (MDS)-based iForest, EE, and LOF detection schemes, the proposed t-SNE-based iF scheme offers superior detection performance of drunk driver status.

Lightweight Anomaly Detection Scheme Using Incremental Principal Component Analysis and Support Vector Machine.

Wireless Sensors Networks have been the focus of significant attention from research and development due to their applications of collecting data from various fields such as smart cities, power grids, transportation systems, medical sectors, military, and rural areas. Accurate and reliable measurements for insightful data analysis and decision-making are the ultimate goals of sensor networks for critical domains. However, the raw data collected by WSNs usually are not reliable and inaccurate due to the imperfect nature of WSNs. Identifying misbehaviours or anomalies in the network is important for providing reliable and secure functioning of the network. However, due to resource constraints, a lightweight detection scheme is a major design challenge in sensor networks. This paper aims at designing and developing a lightweight anomaly detection scheme to improve efficiency in terms of reducing the computational complexity and communication and improving memory utilization overhead while maintaining high accuracy. To achieve this aim, one-class learning and dimension reduction concepts were used in the design. The One-Class Support Vector Machine (OCSVM) with hyper-ellipsoid variance was used for anomaly detection due to its advantage in classifying unlabelled and multivariate data. Various One-Class Support Vector Machine formulations have been investigated and Centred-Ellipsoid has been adopted in this study due to its effectiveness. Centred-Ellipsoid is the most effective kernel among studies formulations. To decrease the computational complexity and improve memory utilization, the dimensions of the data were reduced using the Candid Covariance-Free Incremental Principal Component Analysis (CCIPCA) algorithm. Extensive experiments were conducted to evaluate the proposed lightweight anomaly detection scheme. Results in terms of detection accuracy, memory utilization, computational complexity, and communication overhead show that the proposed scheme is effective and efficient compared few existing schemes evaluated. The proposed anomaly detection scheme achieved the accuracy higher than 98%, with (nd) memory utilization and no communication overhead.

A Data-Driven Design for Gas Turbines Exit Temperature Spread Condition Monitoring System

One of the most complex and costly systems in the industry is the Gas turbine (GT). Because of the complexity of these assets, various indicators have been used to monitor the health condition of different parts of the gas turbine. Turbine exit temperature (TET) spread is one of the significant indicators that help monitor and detect faults such as overall engine deterioration and burner fault. The goal of this article is to use data-driven approaches to monitor TET data to detect faults early, as fault detection can have a significant impact on gas turbine reliability and availability. In this study, the TET data of v94.2 GT is measured by six temperature transmitters to show a detailed profile. According to the statistical tests, TET data are high dimensional and time-dependent in the real world industry. Hence, three distinctive methods in the field of the gas turbine are proposed in this study for early fault detection. Conventional principal component analysis (PCA), moving window PCA (MWPCA), and incremental PCA (IPCA) were implemented on TET data. According to the results, the conventional PCA model is a non-adaptive method, and the false alarm rate is high due to the incompatibility of this approach and the process. The MWPCA based on V-step-ahead and IPCA approaches overcame the non-stationary problem and reduced the false alarm rate. In fact, these approaches can distinguish between the normal time-varying and slow ramp fault processes. The results showed that IPCA could detect fault situations faster than MWPCA based on V-step-ahead in this study.

Fast and more accurate incremental-decremental principal component analysis algorithm for online learning of face recognition

Principal component analysis (PCA) has been successfully employed for face recognition. However, if the training process occurs frequently, owing to the update or downdate of the face images used for training, batch PCA becomes prohibitively expensive to recalculate. To overcome this limitation, incremental principal component analysis (IPCA) and decremental principal component analysis (DPCA) can be utilized as a good alternative to PCA because it reuses their previous results for its updates. Many IPCA or DPCA algorithms have been proposed; however, inaccurate tracking of the mean values of the face image data accumulates decomposition errors, which results in poor performance compared with batch PCA. We proposed faster and more accurate algorithms for IPCA and DPCA that maintain accurate decomposition results. The experimental results reveal that the proposed algorithms produce eigenvectors that are significantly close to the eigenvectors of batch PCA and exhibit faster execution speed for face recognition.

Learning to recognize while learning to speak: Self-supervision and developing a speaking motor

Traditionally, learning speech synthesis and speech recognition were investigated as two separate tasks. This separation hinders incremental development for concurrent synthesis and recognition, where partially-learned synthesis and partially-learned recognition must help each other throughout lifelong learning. This work is a paradigm shift—we treat synthesis and recognition as two intertwined aspects of a lifelong learning agent. Furthermore, in contrast to existing recognition or synthesis systems, babies do not need their mothers to directly supervise their vocal tracts at every moment during the learning. We argue that self-generated non-symbolic states/actions at fine-grained time level help such a learner as necessary temporal contexts. Here, we approach a new and challenging problem—how to enable an autonomous learning system to develop an artificial speaking motor for generating temporally-dense (e.g., frame-wise) actions on the fly without human handcrafting a set of symbolic states. The self-generated states/actions are Muscles-like, High-dimensional, Temporally-dense and Globally-smooth (MHTG), so that these states/actions are directly attended for concurrent synthesis and recognition for each time frame. Human teachers are relieved from supervising learner’s motor ends. The Candid Covariance-free Incremental (CCI) Principal Component Analysis (PCA) is applied to develop such an artificial speaking motor where PCA features drive the motor. Since each life must develop normally, each Developmental Network-2 (DN-2) reaches the same network (maximum likelihood, ML) regardless of randomly initialized weights, where ML is not just for a function approximator but rather an emergent Turing Machine. The machine-synthesized sounds are evaluated by both the neural network and humans with recognition experiments. Our experimental results showed learning-to-synthesize and learning-to-recognize-through-synthesis for phonemes. This work corresponds to a key step toward our goal to close a great gap toward fully autonomous machine learning directly from the physical world.

IPCARF: improving lncRNA-disease association prediction using incremental principal component analysis feature selection and a random forest classifier

BackgroundIdentifying lncRNA-disease associations not only helps to better comprehend the underlying mechanisms of various human diseases at the lncRNA level but also speeds up the identification of potential biomarkers for disease diagnoses, treatments, prognoses, and drug response predictions. However, as the amount of archived biological data continues to grow, it has become increasingly difficult to detect potential human lncRNA-disease associations from these enormous biological datasets using traditional biological experimental methods. Consequently, developing new and effective computational methods to predict potential human lncRNA diseases is essential.ResultsUsing a combination of incremental principal component analysis (IPCA) and random forest (RF) algorithms and by integrating multiple similarity matrices, we propose a new algorithm (IPCARF) based on integrated machine learning technology for predicting lncRNA-disease associations. First, we used two different models to compute a semantic similarity matrix of diseases from a directed acyclic graph of diseases. Second, a characteristic vector for each lncRNA-disease pair is obtained by integrating disease similarity, lncRNA similarity, and Gaussian nuclear similarity. Then, the best feature subspace is obtained by applying IPCA to decrease the dimension of the original feature set. Finally, we train an RF model to predict potential lncRNA-disease associations. The experimental results show that the IPCARF algorithm effectively improves the AUC metric when predicting potential lncRNA-disease associations. Before the parameter optimization procedure, the AUC value predicted by the IPCARF algorithm under 10-fold cross-validation reached 0.8529; after selecting the optimal parameters using the grid search algorithm, the predicted AUC of the IPCARF algorithm reached 0.8611.ConclusionsWe compared IPCARF with the existing LRLSLDA, LRLSLDA-LNCSIM, TPGLDA, NPCMF, and ncPred prediction methods, which have shown excellent performance in predicting lncRNA-disease associations. The compared results of 10-fold cross-validation procedures show that the predictions of the IPCARF method are better than those of the other compared methods.

Adaptive dimensionality reduction for neural network-based online principal component analysis.

“Principal Component Analysis” (PCA) is an established linear technique for dimensionality reduction. It performs an orthonormal transformation to replace possibly correlated variables with a smaller set of linearly independent variables, the so-called principal components, which capture a large portion of the data variance. The problem of finding the optimal number of principal components has been widely studied for offline PCA. However, when working with streaming data, the optimal number changes continuously. This requires to update both the principal components and the dimensionality in every timestep. While the continuous update of the principal components is widely studied, the available algorithms for dimensionality adjustment are limited to an increment of one in neural network-based and incremental PCA. Therefore, existing approaches cannot account for abrupt changes in the presented data. The contribution of this work is to enable in neural network-based PCA the continuous dimensionality adjustment by an arbitrary number without the necessity to learn all principal components. A novel algorithm is presented that utilizes several PCA characteristics to adaptivly update the optimal number of principal components for neural network-based PCA. A precise estimation of the required dimensionality reduces the computational effort while ensuring that the desired amount of variance is kept. The computational complexity of the proposed algorithm is investigated and it is benchmarked in an experimental study against other neural network-based and incremental PCA approaches where it produces highly competitive results.

Incremental Kernel Principal Components Subspace Inference With Nyström Approximation for Bayesian Deep Learning

As the state-of-the-art technology of Bayesian inference, based on low-dimensional principal components analysis (PCA) subspace inference methods can provide approximately accurate predictive distribution and well calibrated uncertainty. However, the main problem of PCA method is that it is a linear subspace feature extractor, and it cannot effectively represent the nonlinearly high-dimensional parameter space of deep neural networks (DNNs). Firstly, in this paper, in order to solve the main problem of the linear characteristics of PCA in high-dimensional space, we apply kernel PCA to extract higher-order statistical information in parameter space of DNNs. Secondly, to improve the efficiency of subsequent computation, we propose a strictly ordered incremental kernel PCA (InKPCA) subspace of parameter space within stochastic gradient descent (SGD) trajectories. In the proposed InKPCA subspace, we employ two approximation inference methods: elliptical slice sampling (ESS) and variational inference (VI). Finally, to further improve the memory efficiency of computing the kernel matrix, we apply Nyström approximation to determine the suitable size of subsets in the original datasets. The novelty of this paper is that it is the first time to apply the proposed InKPCA subspace with Nyström approximation for Bayesian inference in DNNs, and the results show that it can produce more accurate predictions and well-calibrated predictive uncertainty in regression and classification tasks of deep learning.

Data anomaly detection and Data fusion based on Incremental Principal Component Analysis in Fog Computing

Data anomaly detection and Data fusion based on Incremental Principal Component Analysis in Fog Computing

Photovoltaic Generation Prediction of CCIPCA Combined with LSTM

In order to remedy problems encompassing large-scale data being collected by photovoltaic (PV) stations, multiple dimensions of power prediction mode input, noise, slow model convergence speed, and poor precision, a power prediction model that combines the Candid Covariance-free Incremental Principal Component Analysis (CCIPCA) with Long Short-Term Memory (LSTM) network was proposed in this study. The corresponding model uses factor correlation coefficient to evaluate the factors that affect PV generation and obtains the most critical factor of PV generation. Then, it uses CCIPCA to reduce the dimension of PV super large-scale data to the factor dimension, avoiding the complex calculation of covariance matrix of algorithms such as Principal Component Analysis (PCA) and to some extent eliminating the influence of noise made by PV generation data acquisition equipment and transmission equipment such as sensors. The training speed and convergence speed of LSTM are improved by the dimension-reduced data. The PV generation data of a certain power station over a period is collected from SolarGIS as sample data. The model is compared with Markov chain power generation prediction model and GA-BP power generation prediction model. The experimental results indicate that the generation prediction error of the model is less than 3%.