Mixture Of Probabilistic Principal Component Analyzers Research Articles

Unsupervised outlier detection using neural network-based mixtures of probabilistic principal component analyzers for building chiller fault diagnosis

The application of fault detection and diagnosis (FDD) for chillers is very crucial for ensuring the smooth operation, stability, and durability of HVAC systems. Recent FDD models with outstanding diagnostic performance have been proposed for chiller systems. However, the high performance of FDD models is only guaranteed in condition observed data for FDD model construction are not contaminated, i.e., do not contain outliers, which is inevitable during data recording process. To resolve this problem, this paper proposes a novel outlier detection (OD) method named NN-MPPCA. The proposed method uses a neural network (NN) to model a mixture of probabilistic principal component analyzers (MPPCA) framework which parameters are updated via back-propagation. The combining mechanism of MPPCA guarantees NN-MPPCA to fit well complex data distributions, and unseen instances with low reconstruction probability under MPPCA framework are detected as outliers. Comprehensive experimental results of exhaustive comparisons have demonstrated the superiority of NN-MPPCA over other state-of-the-art OD algorithms as well as its effectiveness for chiller FDD application. Under the scenario that datasets contain 20% outliers (i.e., both local outliers and global outliers), the outlier detection performance of NN-MPPCA is highest with F1-scores achieving from 95.4% to 99.7%. Also, with NN-MPPCA applied in the data purification step, FDD models can achieve a diagnostic accuracy up to 99.5%; while they can only reach a diagnostic accuracy of 82.6% at most if contaminated data are directly used.

An MPPCA-based approach for anomaly detection of structures under multiple operational conditions and missing data

Structural anomaly detection based on the structural health monitoring (SHM) data has attracted significant attention owing to its important role in the early warning of structural damage to existing civil structures. Data-driven approaches, where damage-sensitive features are extracted directly from the SHM data using statistical pattern recognition (SPR) techniques without physical models of structures, have been widely studied. Principal component analysis (PCA) and probabilistic PCA (PPCA) are powerful and efficient SPR methods for linear or weakly nonlinear cases. However, some special structures may be subjected to multiple operational conditions, wherein structural configurations such as geometry and mass distribution may change due to the movement of parts or the whole structure, as in retractable roof structures. These changes may give erroneous results in the SPR of the SHM data and eventually in the anomaly detection by a single PCA or PPCA model. This paper presents an improved approach using a mixture of probabilistic principal component analysis (MPPCA) for the anomaly detection of structures under multiple operational conditions with missing measurement data. First, the baseline MPPCA model was constructed for stress data collected under healthy conditions, where the estimation of the MPPCA parameters was reformulated for the missing data cases. Second, three anomaly statistics were presented for newly monitored incomplete data to detect and localize structural anomalies. The probability distributions of the anomaly statistics were estimated to obtain thresholds for outlier detection. Finally, the effectiveness of the MPPCA-based method was investigated by applying the method to the anomaly detection of a retractable roof structure with numerically simulated and real monitored stress data.

Nonlinear process monitoring using a mixture of probabilistic PCA with clusterings

Motivated by mixture of probabilistic principal component analysis (PCA), which is time-consuming due to expectation maximization, this paper investigates a novel mixture of probabilistic PCA with clusterings for process monitoring. The significant features are extracted by singular vector decomposition (SVD) or kernel PCA, and k-means is subsequently utilized as a clustering algorithm. Then, parameters of local PCA models are determined under each clustering model. Compared with PCA clustering, SVD based clustering only utilizes the nature basis for the components of the data instead of principal components of the data. Three clustering approaches are adopted and the effectiveness of the proposed approach is demonstrated by a practical coal pulverizing system.

MPPCANet: A feedforward learning strategy for few-shot image classification

The main learning strategy of the PCANet is using Principal Component Analysis (PCA) for learning the convolutional filters from the data. The assumption that all the image patches are sampled from a single Gaussian component is implicitly taken, which is too strong. In this paper, the image patches are modeled using mixtures of probabilistic principal component analysis (MPPCA) and the corresponding MPPCANet (PCANet constructed using mixtures of probabilistic principal component analysis) is proposed. The proposed model is applied to the few-shot learning scenario. In the proposed framework, the image patches are assumed to come from several suppositions of Gaussian components. In the process of estimating the parameters of the MPPCA model, the clustering of the training image patches and the principal components of each cluster are simultaneously obtained. The number of mixture components is automatically determined during the optimization procedure. The theoretical insights of the proposed MPPCANet is elaborated by comparing with our prior work, CPCANet (PCANet with clustering-based filters). The proposed MPPCANet is evaluated on several benchmarking visual data sets in the experiment. It is compared with the original PCANet, CPCANet and several state-of-the-art methods. The experimental results show that the proposed MPPCANet has improved significantly the recognition capability of the original PCANet under the few-shot learning scenario. The performance of the MPPCANet is also better than the CPCANet in most cases.

Region-based scalable smart system for anomaly detection in pedestrian walkways

Different-sized anomalies and its occurrence in a shorter period have always been an open research issue. To resolve the issue of detecting anomalies of different sizes, especially in pedestrian pathways, within a shorter time period, the current research article introduced a Region based Scalable Convolution Neural Network (RS-CNN). The proposed method used region based proposals for faster identification and performed well with the scalability issues. The RS-CNN model was validated using different video sequences from the UCSD anomaly detection dataset. When compared with state-of-the-art detection techniques such as Fast R-CNN, Minimization of Drive Testing (MDT), Mixtures of Probabilistic Principal Component Analyzers (MPPCA) and Social Force (SF), the RS-CNN model was found to be faster and efficient even in the presence of anomalies of various sizes.

Small-variance asymptotics for non-parametric online robot learning

Small-variance asymptotics is emerging as a useful technique for inference in large-scale Bayesian non-parametric mixture models. This paper analyzes the online learning of robot manipulation tasks with Bayesian non-parametric mixture models under small-variance asymptotics. The analysis yields a scalable online sequence clustering (SOSC) algorithm that is non-parametric in the number of clusters and the subspace dimension of each cluster. SOSC groups the new datapoint in low-dimensional subspaces by online inference in a non-parametric mixture of probabilistic principal component analyzers (MPPCA) based on a Dirichlet process, and captures the state transition and state duration information online in a hidden semi-Markov model (HSMM) based on a hierarchical Dirichlet process. A task-parameterized formulation of our approach autonomously adapts the model to changing environmental situations during manipulation. We apply the algorithm in a teleoperation setting to recognize the intention of the operator and remotely adjust the movement of the robot using the learned model. The generative model is used to synthesize both time-independent and time-dependent behaviors by relying on the principles of shared and autonomous control. Experiments with the Baxter robot yield parsimonious clusters that adapt online with new demonstrations and assist the operator in performing remote manipulation tasks.

Nonlinear sensor fault diagnosis using mixture of probabilistic PCA models

This paper presents a methodology for sensor fault diagnosis in nonlinear systems using a Mixture of Probabilistic Principal Component Analysis (MPPCA) models. This methodology separates the measurement space into several locally linear regions, each of which is associated with a Probabilistic PCA (PPCA) model. Using the transformation associated with each PPCA model, a parity relation scheme is used to construct a residual vector. Bayesian analysis of the residuals forms the basis for detection and isolation of sensor faults across the entire range of operation of the system. The resulting method is demonstrated in its application to sensor fault diagnosis of a fully instrumented HVAC system. The results show accurate detection of sensor faults under the assumption that a single sensor is faulty.

Mixture Probabilistic PCA for Process Monitoring - Collapsed Variational Bayesian Approach

Mixture of probabilistic principal component analyzers (MPPCA) has been used for modeling non-Gaussian process data and monitoring in the past. However, appropriate model structure selection in the case of MPPCA is a challenging task. Previously, variational Bayesian expectation maximization (VBEM) estimation has been used to handle this task. However, VBEM can be computationally expensive for practical purposes and also, may converge to spurious estimates. In this article, collapsed variational Bayesian technique with a new collapsing scheme as an alternative to VBEM is proposed. Advantages of the proposed scheme are demonstrated in simulated and industrial process data.

Efficient model selection for mixtures of probabilistic PCA via hierarchical BIC.

This paper concerns model selection for mixtures of probabilistic principal component analyzers (MPCA). The well known Bayesian information criterion (BIC) is frequently used for this purpose. However, it is found that BIC penalizes each analyzer implausibly using the whole sample size. In this paper, we present a new criterion for MPCA called hierarchical BIC in which each analyzer is penalized using its own effective sample size only. Theoretically, hierarchical BIC is a large sample approximation of variational Bayesian lower bound and BIC is a further approximation of hierarchical BIC. To learn hierarchical-BIC-based MPCA, we propose two efficient algorithms: two-stage and one-stage variants. The two-stage algorithm integrates model selection with respect to the subspace dimensions into parameter estimation, and the one-stage variant further integrates the selection of the number of mixture components into a single algorithm. Experiments on a number of synthetic and real-world data sets show that: 1) hierarchical BIC is more accurate than BIC and several related competitors and 2) the two proposed algorithms are not only effective but also much more efficient than the classical two-stage procedure commonly used for BIC.

Appearance Model Based Moving Object Matching across Disjoint Camera Views

A new object tracking scheme for multi-camera surveillance with non-overlapping views is proposed in this paper. Brightness transfer function (BTF) is used to establish relative appearance correspondence between different views. Mixtures of probabilistic principal component analysis (MPPCA) is incooperated to learn the subspace of brightness transfer function with the concern to deal with multiple different brightness areas in a scene. The incremental major color spectrum histogram (IMCSH) is used as similarity measure for reliable matching. Experimental results with real world videos show the effectiveness of the proposed algorithm.

A low-cost variational-Bayes technique for merging mixtures of probabilistic principal component analyzers

Mixtures of probabilistic principal component analyzers (MPPCA) have shown effective for modeling high-dimensional data sets living on non-linear manifolds. Briefly stated, they conduct mixture model estimation and dimensionality reduction through a single process. This paper makes two contributions: first, we disclose a Bayesian technique for estimating such mixture models. Then, assuming several MPPCA models are available, we address the problem of aggregating them into a single MPPCA model, which should be as parsimonious as possible. We disclose in detail how this can be achieved in a cost-effective way, without sampling nor access to data, but solely requiring mixture parameters. The proposed approach is based on a novel variational-Bayes scheme operating over model parameters. Numerous experimental results and discussion are provided.

Robust Face Tracking via Collaboration of Generic and Specific Models

Significant appearance changes of objects under different orientations could cause loss of tracking, "drifting." In this paper, we present a collaborative tracking framework to robustly track faces under large pose and expression changes and to learn their appearance models online. The collaborative tracking framework probabilistically combines measurements from an offline-trained generic face model with measurements from online-learned specific face appearance models in a dynamic Bayesian network. In this framework, generic face models provide the knowledge of the whole face class, while specific face models provide information on individual faces being tracked. Their combination, therefore, provides robust measurements for multiview face tracking. We introduce a mixture of probabilistic principal component analysis (MPPCA) model to represent the appearance of a specific face under multiple views, and we also present an online EM algorithm to incrementally update the MPPCA model using tracking results. Experimental results demonstrate that the collaborative tracking and online learning methods can handle large pose changes and are robust to distractions from the background.

Localization of Shapes Using Statistical Models and Stochastic Optimization

In this paper, we present a new model for deformations of shapes. A pseudo-likelihood is based on the statistical distribution of the gradient vector field of the gray level. The prior distribution is based on the Probabilistic Principal Component Analysis (PPCA). We also propose a new model based on mixtures of PPCA that is useful in the case of greater variability in the shape. A criterion of global or local object specificity based on a preliminary color segmentation of the image, is included into the model. The localization of a shape in an image is then viewed as minimizing the corresponding Gibbs field. We use the Exploration/Selection (E/S) stochastic algorithm in order to find the optimal deformation. This yields a new unsupervised statistical method for localization of shapes. In order to estimate the statistical parameters for the gradient vector field of the gray level, we use an Iterative Conditional Estimation (ICE) procedure. The color segmentation of the image can be computed with an Exploration/Selection/Estimation (ESE) procedure.

A new image representation algorithm inspired by image submodality models, redundancy reduction, and learning in biological vision

We develop a new biologically motivated algorithm for representing natural images using successive projections into complementary subspaces. An image is first projected into an edge subspace spanned using an ICA basis adapted to natural images which captures the sharp features of an image like edges and curves. The residual image obtained after extraction of the sharp image features is approximated using a mixture of probabilistic principal component analyzers (MPPCA) model. The model is consistent with cellular, functional, information theoretic, and learning paradigms in visual pathway modeling. We demonstrate the efficiency of our model for representing different attributes of natural images like color and luminance. We compare the performance of our model in terms of quality of representation against commonly used basis, like the discrete cosine transform (DCT), independent component analysis (ICA), and principal components analysis (PCA), based on their entropies. Chrominance and luminance components of images are represented using codes having lower entropy than DCT, ICA, or PCA for similar visual quality. The model attains considerable simplification for learning from images by using a sparse independent code for representing edges and explicitly evaluating probabilities in the residual subspace.