Probabilistic Principal Component Analysis Model Research Articles

Publisher Correction: The Effectiveness of a Probabilistic Principal Component Analysis Model and Expectation Maximisation Algorithm in Treating Missing Daily Rainfall Data

Publisher Correction: The Effectiveness of a Probabilistic Principal Component Analysis Model and Expectation Maximisation Algorithm in Treating Missing Daily Rainfall Data

Missing Traffic Data Imputation with a Linear Generative Model Based on Probabilistic Principal Component Analysis

Even with the ubiquitous sensing data in intelligent transportation systems, such as the mobile sensing of vehicle trajectories, traffic estimation is still faced with the data missing problem due to the detector faults or limited number of probe vehicles as mobile sensors. Such data missing issue poses an obstacle for many further explorations, e.g., the link-based traffic status modeling. Although many studies have focused on tackling this kind of problem, existing studies mainly focus on the situation in which data are missing at random and ignore the distinction between links of missing data. In the practical scenario, traffic speed data are always missing not at random (MNAR). The distinction for recovering missing data on different links has not been studied yet. In this paper, we propose a general linear model based on probabilistic principal component analysis (PPCA) for solving MNAR traffic speed data imputation. Furthermore, we propose a metric, i.e., Pearson score (p-score), for distinguishing links and investigate how the model performs on links with different p-score values. Experimental results show that the new model outperforms the typically used PPCA model, and missing data on links with higher p-score values can be better recovered.

Fault detection of pressurized heavy water nuclear reactors with steady state and dynamic characteristics using data-driven techniques

Nuclear energy is a crucial source to bridge the deficit of energy demand as fossil fuel reserves are continuously depleting over time. However, nuclear reactors operation is highly safety-critical and any breach of safe operation would lead to catastrophic effects. Therefore, the operation of nuclear reactors have to monitored very carefully for any anomalous operational characteristics. However, a significant challenge in that front is the existence of various operational modes and dynamic transition in between them. Thus, a successful fault detection (FD) technique should clearly segregate the dynamic transitions and any anomalous faulty behaviour. This article presents a novel approach for data-driven FD of Pressurized Heavy Water Reactors. The proposed solution hinges on integrating Hidden Markov Models (HMM) with Probabilistic Principal Component Analysis (PPCA), and Dynamic Principal Component Analysis (DPCA). Initially an HMM is developed using a training data set which consist of multimode with transitions. Then, an HMM-based probability ratio strategy is employed for distinguishing transitional modes. After that, Viterbi algorithm is also used to separate different known modes, such as stable modes and transitional modes. Thus, HMM would help to identify the modal transitions and dynamic operation. Thereafter, PPCA models are used to deal with the steady-state operation while DPCA based models are used for detecting transitions and dynamic operation. Subsequently, statistical non-conformity to the developed models are used to flag faults. The superiority of the proposed framework is validated using a benchmark simulated data and industrial real time data representing the operation of PHWR.

An MPPCA-based approach for anomaly detection of structures under multiple operational conditions and missing data

Structural anomaly detection based on the structural health monitoring (SHM) data has attracted significant attention owing to its important role in the early warning of structural damage to existing civil structures. Data-driven approaches, where damage-sensitive features are extracted directly from the SHM data using statistical pattern recognition (SPR) techniques without physical models of structures, have been widely studied. Principal component analysis (PCA) and probabilistic PCA (PPCA) are powerful and efficient SPR methods for linear or weakly nonlinear cases. However, some special structures may be subjected to multiple operational conditions, wherein structural configurations such as geometry and mass distribution may change due to the movement of parts or the whole structure, as in retractable roof structures. These changes may give erroneous results in the SPR of the SHM data and eventually in the anomaly detection by a single PCA or PPCA model. This paper presents an improved approach using a mixture of probabilistic principal component analysis (MPPCA) for the anomaly detection of structures under multiple operational conditions with missing measurement data. First, the baseline MPPCA model was constructed for stress data collected under healthy conditions, where the estimation of the MPPCA parameters was reformulated for the missing data cases. Second, three anomaly statistics were presented for newly monitored incomplete data to detect and localize structural anomalies. The probability distributions of the anomaly statistics were estimated to obtain thresholds for outlier detection. Finally, the effectiveness of the MPPCA-based method was investigated by applying the method to the anomaly detection of a retractable roof structure with numerically simulated and real monitored stress data.

Nonlinear process monitoring using a mixture of probabilistic PCA with clusterings

Motivated by mixture of probabilistic principal component analysis (PCA), which is time-consuming due to expectation maximization, this paper investigates a novel mixture of probabilistic PCA with clusterings for process monitoring. The significant features are extracted by singular vector decomposition (SVD) or kernel PCA, and k-means is subsequently utilized as a clustering algorithm. Then, parameters of local PCA models are determined under each clustering model. Compared with PCA clustering, SVD based clustering only utilizes the nature basis for the components of the data instead of principal components of the data. Three clustering approaches are adopted and the effectiveness of the proposed approach is demonstrated by a practical coal pulverizing system.

Probabilistic principal component analysis‐based anomaly detection for structures with missing data

Structures are subjected to various kinds of structural deterioration and damage with use over long periods of service life. For the safety assurance of structures, it is very important to have a long-term monitoring system and continuous assessments of structural integrity using the measured data. The objective of this paper is to develop an anomaly detection algorithm for a long-term structural health monitoring (SHM) system based on probabilistic principal component analysis (PPCA). Static stress data were measured and used in this monitoring system. A baseline PPCA model is built under various environmental loading conditions. Then, newly monitored data are projected onto the principal vectors. Anomaly indices and their probability distributions are evaluated to determine the presence of structural damage indicated by outliers. This method is also capable of dealing with incomplete data and recovering the missing data. First, numerical simulation studies of a revolving auditorium are carried out to validate the proposed PPCA-based method. Then, real monitoring data collected from the SHM system are used to detect the presence and locations of anomalies in the revolving auditorium.

On multi-fault detection of rolling bearing through probabilistic principal component analysis denoising and Higuchi fractal dimension transformation

Bearing multi-fault detection from stochastic vibration signal is still a thorny task to dispose of because of the complex interplay between different fault components under severe noise interference. In such case, conventional techniques such as filter processing and envelope demodulation may cause undesired results. To overcome the limitation, this article explores a filtering-free technique combined probabilistic principal component analysis denoising with the Higuchi fractal dimension transformation to diagnose the bearing multi-faults. Fractal theory is used to optimize the model parameters and stabilize the random vibrational signal for fast Fourier transform spectrum analysis. Noise interference in the Higuchi transformation is capped using a probabilistic principal component analysis model whose parameters are optimized through embedding dimension Cao algorithm and correlation dimension Grassberger and Procaccia algorithm. The fault diagnostic scheme mainly falls into three steps. First, the original vibration signal is truncated into a series of sub-signal segments by moving window whose length is determined as twice the value of maximum time delay that is provided by examining the steady Higuchi fractal dimension value of a raw signal in a process of plotting the fractal dimension over a range of time delay. Then, the Higuchi approach is used to estimate the average fractal dimension for each segment to create a quasi-stationary Higuchi fractal dimension sequence on which, finally, the fault features are straightforwardly extracted by the fast Fourier transform algorithm. The effectiveness of the proposed method is validated using simulated and experimental compound bearing fault vibration signals. Some fault components may be clouded if applied Higuchi fractal dimension alone because of the noise interference, but using the probabilistic principal component analysis–Higuchi fractal dimension method leads to clear diagnostic results. It indicates that the proposed approach can be incorporated into bearing multi-fault extraction from raw vibration signals.

Fast, scalable and geo-distributed PCA for big data analytics

Principal Component Analysis (PCA) is a widely popular technique for reducing the dimensionality of a dataset. Interestingly, when dimensions of the dataset grow too large, existing state-of-the-art methods for PCA face scalability issue due to the explosion of intermediate data. Moreover, in a geographically distributed environment where most of today’s data are originally generated, these methods require unnecessary data transmissions as they apply centralized algorithms for PCA and thus are proven to be inefficient. To solve these problems, we take advantage of the zero-noise-limit Probabilistic PCA model, which provably outputs the correct principal components, and introduce a block-division method for it in order to suppress the explosion of intermediate data efficiently. We employ several optimization ideas such as mean propagation for preserving sparsity, dynamic tuning of the number of blocks to automatically adjust to large dimensions, etc. Additionally, in the geo-distributed environment, we propose a communication efficient solution by reducing idle time, passing only the required parameters, and choosing geographically ideal central datacenter for faster accumulation. We refer to our algorithm as TallnWide. Our empirical evaluation with real datasets shows that TallnWide can successfully handle significantly higher dimensional data (10×) than existing methods, and offer up to 2.9× improvement in running time in the geo-distributed environment compared to the conventional approaches. For reproducibility and extensibility of our work, we make the source code of TallnWide publicly available at https://github.com/tmadnan10/TallnWide.

The Effectiveness of a Probabilistic Principal Component Analysis Model and Expectation Maximisation Algorithm in Treating Missing Daily Rainfall Data

The reliability and accuracy of a risk assessment of extreme hydro-meteorological events are highly dependent on the quality of the historical rainfall time series data. However, missing data in a time series such as this could result in lower quality data. Therefore, this paper proposes a multiple-imputation algorithm for treating missing data without requiring information from adjoining monitoring stations. The proposed imputation algorithms are based on the M-component probabilistic principal component analysis model and an expectation maximisation algorithm (MPPCA-EM). In order to evaluate the effectiveness of the MPPCA-EM imputation algorithm, six distinct historical daily rainfall time series data were recorded from six monitoring stations. These stations were located at the coastal and inland regions of the East-Coast Economic Region (ECER) Malaysia. The results of analysis show that, when it comes to treating missing historical daily rainfall time series data recorded from coastal monitoring stations, the 2-component probabilistic principal component analysis model and expectation-maximisation algorithm (2PPCA-EM) were found to be superior to the single- and multiple-imputation algorithms proposed in previous studies. On the contrary, the single-imputation algorithms as proposed in previous studies were superior to the MPPCA-EM imputation algorithms when treating missing historical daily rainfall time series data recorded from inland monitoring stations.

An Improved Mixture of Probabilistic PCA for Nonlinear Data-Driven Process Monitoring.

An improved mixture of probabilistic principal component analysis (PPCA) has been introduced for nonlinear data-driven process monitoring in this paper. To realize this purpose, the technique of a mixture of probabilistic principal component analyzers is utilized to establish the model of the underlying nonlinear process with local PPCA models, where a novel composite monitoring statistic is proposed based on the integration of two monitoring statistics in modified PPCA-based fault detection approach. Besides, the weighted mean of the monitoring statistics aforementioned is utilized as a metrics to detect potential abnormalities. The virtues of the proposed algorithm are discussed in comparison with several unsupervised algorithms. Finally, Tennessee Eastman process and an autosuspension model are employed to demonstrate the effectiveness of the proposed scheme further.

Sparse probabilistic principal component analysis model for plant-wide process monitoring

In the industrial monitoring process, probabilistic principal component analysis (PPCA) is a popular algorithm for reducing the dimension. However, the principal components (PCs) are not easy to interpret and its preserved number cannot be determined automatically. In this paper, we propose a sparse PPCA (SPPCA) to improve the interpretability by adding a prior and introducing sparsification to the loading matrix of PPCA. An expectation-maximization (EM) algorithm is used to obtain the parameters of the probabilistic formulation, and the dimensionality of the latent variable space can be automatically determined during the iterative process. With the sparse representation, a process monitoring strategy is then developed with the construction of several partial PPCA models. Case studies of SPPCA to a numerical case and Tennessee Eastman (TE) benchmark process demonstrate its feasibility and efficiency.

Process monitoring based on entropy weight for a subspace containing probabilistic principal components and fault‐relevant noise factors

Probabilistic principal component analysis (PPCA) is widely used in process monitoring and has achieved effective fault detection and online fault identification. However, the relationship between fault information and certain probabilistic principal components has not been explicitly mapped. Therefore, fault information might be scattered across 2 different subspaces when probabilistic principal components are selected subjectively based on variance. Moreover, although the maximum useful information is collected into a new subspace by the selected elements, a large amount of useless information might cover up the fault‐relevant information. Considering these shortcomings, weighted PPCA based on entropy method and moving window (EMW‐PPCA) is applied in this study to improve the monitoring performance of traditional PPCA. After building a traditional PPCA model through normal observations, EMW‐PPCA proposes a novel strategy to identify and extract as much fault‐relevant information as possible from the residual subspace and then integrates the selected fault‐relevant noise factors into the dominant subspace to create a new specific subspace for process monitoring. The entropy method is applied to this new subspace to define the importance degree of each direction and evaluate the weighting values objectively. Then, the weighting values are assigned to each of the corresponding elements to highlight the fault‐relevant information for online monitoring within a moving window. Case studies on a simple numerical example are conducted, and the Tennessee Eastman process is applied to validate the effectiveness of the EMW‐PPCA scheme.

Urban Network Travel Time Prediction Based on a Probabilistic Principal Component Analysis Model of Probe Data

This paper proposes a network travel time prediction methodology based on probe data. The model is intended as a tool for traffic management, trip planning, and online vehicle routing, and is designed to be efficient and scalable in calibration and real-time prediction; flexible to changes in network, data, and model extensions; and robust against noisy and missing data. A multivariate probabilistic principal component analysis (PPCA) model is proposed. Spatio-temporal correlations are inferred from historical data based on MLE and an efficient EM algorithm for handling missing data. Prediction is performed in real time by computing the expected distribution of link travel times in future time intervals, conditional on recent current-day observations. A generalization of the methodology partitions the network and applies a distinct PPCA model to each subnetwork. The methodology is applied to the network of downtown Shenzhen, China, using taxi probe data. The model captures variability over months and weekdays as well as other factors. Prediction with PPCA outperforms $k$ -nearest neighbors prediction for horizons from 15 to 45 min, and a hybrid method of PPCA and local smoothing provides the highest accuracy.

A Corrected Criterion for Selecting the Optimum Number of Principal Components

Determining the optimum number of components to be retained is a key problem in principal component analysis (PCA). Besides the rule of thumb estimates there exist several sophisticated methods for automatically selecting the dimensionality of the data. Based on the probabilistic PCA model Minka (2001) proposed an approximate Bayesian model selection criterion. In this paper we correct this criterion and present a modified version. We compare the novel criterion with various other approaches in a simulationstudy. Furthermore, we use it for finding the optimum number of principal components in hyper-spectral skin cancer images.

On Estimation of the Noise Variance in High Dimensional Probabilistic Principal Component Analysis

Summary We develop new statistical theory for probabilistic principal component analysis models in high dimensions. The focus is the estimation of the noise variance, which is an important and unresolved issue when the number of variables is large in comparison with the sample size. We first unveil the reasons for an observed downward bias of the maximum likelihood estimator of the noise variance when the data dimension is high. We then propose a bias-corrected estimator by using random-matrix theory and establish its asymptotic normality. The superiority of the new and bias-corrected estimator over existing alternatives is checked by Monte Carlo experiments with various combinations of (p, n) (the dimension and sample size). Next, we construct a new criterion based on the bias-corrected estimator to determine the number of the principal components, and a consistent estimator is obtained. Its good performance is confirmed by a simulation study and real data analysis. The bias-corrected estimator is also used to derive new asymptotics for the related goodness-of-fit statistic under the high dimensional scheme.

Image Outlier Detection and Feature Extraction via L1-Norm-Based 2D Probabilistic PCA.

This paper introduces an L1-norm-based probabilistic principal component analysis model on 2D data (L1-2DPPCA) based on the assumption of the Laplacian noise model. The Laplacian or L1 density function can be expressed as a superposition of an infinite number of Gaussian distributions. Under this expression, a Bayesian inference can be established based on the variational expectation maximization approach. All the key parameters in the probabilistic model can be learned by the proposed variational algorithm. It has experimentally been demonstrated that the newly introduced hidden variables in the superposition can serve as an effective indicator for data outliers. Experiments on some publicly available databases show that the performance of L1-2DPPCA has largely been improved after identifying and removing sample outliers, resulting in more accurate image reconstruction than the existing PCA-based methods. The performance of feature extraction of the proposed method generally outperforms other existing algorithms in terms of reconstruction errors and classification accuracy.

Robust supervised probabilistic principal component analysis model for soft sensing of key process variables

In this paper, a robust and mixture form of supervised probabilistic principal component analysis model is proposed to deal with the soft sensing problem, particularly for those processes with multiple operating conditions and the collected datasets may contain outliers. Under the framework of latent variable models, the commonly adopted multivariate Gaussian distribution assumption is replaced by the multivariate student t-distribution so as to tolerate the notorious outliers by using the adjusted heavy tail. After the construction of robust probabilistic model, the iterative expectation–maximization algorithm is derived to perform the parameter estimation for both single and mixture models. For online soft sensing application, the Bayes rule is introduced for soft alignment of local prediction results. Two case studies are provided for performance evaluation of the proposed method, both in comparison with the conventional supervised model. Results indicate that the new model is much more reliable under outlier contaminated and multimode conditions.

Noise-Robust Classification of Ground Moving Targets Based on Time-Frequency Feature From Micro-Doppler Signature

A noise-robust classification method is proposed to discriminate the moving vehicle and walking human via the time-frequency feature extracted from the micro-Doppler signature of low resolution radar. Since the signal-to-noise ratio (SNR) directly relates to the distance between the target and radar for a given noise power and radar power, the denoising preprocessing is usually required to increase the classification distance between the target and radar in the real application. In this paper, we extend the real-valued probabilistic principal component analysis model to the complex-value domain, and develop the complex probabilistic principal component analysis (CPPCA) model for the complex-valued echoes from the ground moving targets. Then, the denoising preprocessing is accomplished based on signal reconstruction with CPPCA model, where we utilize the Bayesian inference criterion (BIC) to adaptively select the principal components. Compared with the existing CLEAN-based noise reduction method, the CPPCA-BIC-based method can work without SNR prior information. After denoising, a 3-D time-frequency feature vector is extracted from the denoised micro-Doppler signatures of the two kinds of ground targets, and the classification is performed via support vector machine classifier. In the experiments based on the measured data, the proposed classification scheme shows good classification and denoising performances under the relatively low SNR condition. The proposed method can also be applied to other classification problems based on micro-Doppler effect.

Robust modeling of mixture probabilistic principal component analysis and process monitoring application

In this article, a robust modeling strategy for mixture probabilistic principal component analysis (PPCA) is proposed. Different from the traditional Gaussian distribution driven model such as PPCA, the multivariate student t‐distribution is adopted for probabilistic modeling to reduce the negative effect of outliers, which is very common in the process industry. Furthermore, for handling the missing data problem, a partially updating algorithm is developed for parameter learning in the robust mixture PPCA model. Therefore, the new robust model can simultaneously deal with outliers and missing data. For process monitoring, a Bayesian soft decision fusion strategy is developed which is combined with the robust local monitoring models under different operating conditions. Two case studies demonstrate that the new robust model shows enhanced modeling and monitoring performance in both outlier and missing data cases, compared to the mixture probabilistic principal analysis model. © 2014 American Institute of Chemical Engineers AIChE J, 60: 2143–2157, 2014

A probabilistic model for image representation via multiple patterns

For image analysis, an important extension to principal component analysis (PCA) is to treat an image as multiple samples, which helps alleviate the small sample size problem. Various schemes of transforming an image to multiple samples have been proposed. Although having been shown effective in practice, the schemes are mainly based on heuristics and experience.In this paper, we propose a probabilistic PCA model, in which we explicitly represent the transformation scheme and incorporate the scheme as a stochastic component of the model. Therefore fitting the model automatically learns the transformation. Moreover, the learned model allows us to distinguish regions that can be well described by the PCA model from those that need further treatment. Experiments on synthetic images and face data sets demonstrate the properties and utility of the proposed model.