Probabilistic Principal Component Analysis Research Articles

Brain MR image classification using two-dimensional discrete wavelet transform and AdaBoost with random forests

This paper presents an automated and accurate computer-aided diagnosis (CAD) system for brain magnetic resonance (MR) image classification. The system first utilizes two-dimensional discrete wavelet transform (2D DWT) for extracting features from the images. After feature vector normalization, probabilistic principal component analysis (PPCA) is employed to reduce the dimensionality of the feature vector. The reduced features are applied to the classifier to categorize MR images into normal and abnormal. This scheme uses an AdaBoost algorithm with random forests as its base classifier. Three benchmark MR image datasets, Dataset-66, Dataset-160, and Dataset-255, have been used to validate the proposed system. A 5×5-fold stratified cross validation scheme is used to enhance the generalization capability of the proposed scheme. Simulation results are compared with the existing schemes and it is observed that the proposed scheme outperforms others in all the three datasets.

Analysis on Quranic Accents Automatic Identification with Acoustic Echo Cancellation using Affine Projection and Probabilistic Principal Component Analysis

Background/Objectives: Audio recordings or live recitation is easy for noise and reverberation prone and contributes to the classification results. Methods/Statistical Analysis: Therefore, this paper proposed suitable usage of Affine Projection (AP) as adaptive filtering can be used to remove the echo and improve the percentage of results after the classification method takes place. Findings: The room impulse response is added for the signals in the study and AP algorithm is applies to remove the echo existing on each signal. Later, 161 signals then are converted to feature vector by Mel Frequency Cepstral Coefficients (MFCC) and the classification method using Probabilistic Principal Component Analysis (PPCA). This finding is to find the accuracies results based on audio on Quranic accents recitations after acoustic echo cancellation is done on the echoed signal. Applications/Improvements: The percentage of accuracies increases when acoustic echo cancellation is in used from the overall classification processes done to the segmented verses of the Quranic verse for Surah Ad-Duhaa. The implementation of AP as adaptive algorithm able to reduce the echo of Quranic accents and consistent output on the pattern classification and with 20dB of room impulse reverberation is applied and finally it can improve the results of classification to 96%.

Nonlinear feature extraction for soft sensor modeling based on weighted probabilistic PCA

As industrial process plants are often instrumented with a large number of sensors, it is important to carry out feature extraction before soft sensor modeling. Probabilistic principal component analysis (PPCA) has been identified as an effective method for dimensional reduction. However, PPCA is a linear method, which cannot deal with nonlinear data distribution. To cope with this problem and enhance the performance of soft sensor model, a new nonlinear dimensional reduction method, weighted probabilistic principal component analysis (WPPCA), is proposed in this paper. By assigning different weights for training samples according to their similarities to the testing sample, nonlinear features can be extracted properly for regression modeling. For performance evaluation of the proposed method, detailed illustrations of a numerical example and an industrial process are provided.

Image Outlier Detection and Feature Extraction via L1-Norm-Based 2D Probabilistic PCA.

This paper introduces an L1-norm-based probabilistic principal component analysis model on 2D data (L1-2DPPCA) based on the assumption of the Laplacian noise model. The Laplacian or L1 density function can be expressed as a superposition of an infinite number of Gaussian distributions. Under this expression, a Bayesian inference can be established based on the variational expectation maximization approach. All the key parameters in the probabilistic model can be learned by the proposed variational algorithm. It has experimentally been demonstrated that the newly introduced hidden variables in the superposition can serve as an effective indicator for data outliers. Experiments on some publicly available databases show that the performance of L1-2DPPCA has largely been improved after identifying and removing sample outliers, resulting in more accurate image reconstruction than the existing PCA-based methods. The performance of feature extraction of the proposed method generally outperforms other existing algorithms in terms of reconstruction errors and classification accuracy.

Integration of Near Infrared Image and Probabilistic Classifier to Increase the Classification Accuracy of Point Clouds

Essential to the establishment of such 3D spatial information are the laser scanning technology to obtain high-precision 3D point clouds and the photography-metric camera to obtain high-resolution multispectral image information. TLS (Terrestrial laser scanner) is a high precision positioning technique to monitor the behavior and change of structures and natural topography. 3D point clouds of natural environments relevant to problems in geomorphology (rivers, coastal environments, cliffs, …) often require classification of the data into elementary relevant classes12. There are such old techniques to classify massive point clouds as NDVI and PCA (Principal Component Analysis). Although they have many different advantages, it is extremely difficult for them to maximize their advantages to function as stand-alone techniques and overcome their disadvantages. This study thus set out to investigate the integration of NDVI and a probabilistic classifier PCA, SVM (Support Vector Machine), LDA (Linear Discriminant Analysis))1 to obtain land cover information in a natural state through TLS and improve the classification accuracy of resulting massive point clouds.

Rolling element bearing fault detection using PPCA and spectral kurtosis

A hybrid approach using probabilistic principal component analysis (PPCA) and spectral kurtosis (SK) is proposed to detect rolling element bearing faults. The approach includes three main steps. In a first step, the signal-to-noise ratio (SNR) of PPCA denoising model is improved through the selection of two key parameters. In the model, the primary information and fault signals will be preserved by allotted in the principal component subspace, while noises and linear interrelated information will be discarded by projected to the residual subspace. In a second step a band-pass filter for the denoising signal is designed using rapid spectral kurtosis procedure to determine optimal center frequency and bandwidth. The third step is to perform a Hilbert envelope spectrum analysis of the filtered signal to extract the fault frequencies of the rolling element bearings. The effectiveness of the proposed approach is demonstrated by numerical simulation and experimental investigation of rolling element bearing with different kind of faults. It indicates that employing the proposed scheme with PPCA and SK results in the effective detection of faults in rolling element bearings.

Mixture of Subspaces Image Representation and Compact Coding for Large-Scale Image Retrieval.

There are two major approaches to content-based image retrieval using local image descriptors. One is descriptor-by-descriptor matching and the other is based on comparison of global image representation that describes the set of local descriptors of each image. In large-scale problems, the latter is preferred due to its smaller memory requirements; however, it tends to be inferior to the former in terms of retrieval accuracy. To achieve both low memory cost and high accuracy, we investigate an asymmetric approach in which the probability distribution of local descriptors is modeled for each individual database image while the local descriptors of a query are used as is. We adopt a mixture model of probabilistic principal component analysis. The model parameters constitute a global image representation to be stored in database. Then the likelihood function is employed to compute a matching score between each database image and a query. We also propose an algorithm to encode our image representation into more compact codes. Experimental results demonstrate that our method can represent each database image in less than several hundred bytes achieving higher retrieval accuracy than the state-of-the-art method using Fisher vectors.

Using latent features for short-term person re-identification with RGB-D cameras

This paper presents a system for people re-identification in uncontrolled scenarios using RGB-depth cameras. Compared to conventional RGB cameras, the use of depth information greatly simplifies the tasks of segmentation and tracking. In a previous work, we proposed a similar architecture where people were characterized using color-based descriptors that we named bodyprints. In this work, we propose the use of latent feature models to extract more relevant information from the bodyprint descriptors by reducing their dimensionality. Latent features can also cope with missing data in case of occlusions. Different probabilistic latent feature models, such as probabilistic principal component analysis and factor analysis, are compared in the paper. The main difference between the models is how the observation noise is handled in each case. Re-identification experiments have been conducted in a real store where people behaved naturally. The results show that the use of the latent features significantly improves the re-identification rates compared to state-of-the-art works.

Bayesian inference via projections

Bayesian inference often poses difficult computational problems. Even when off-the-shelf Markov chain Monte Carlo (MCMC) methods are available to the problem at hand, mixing issues might compromise the quality of the results. We introduce a framework for situations where the model space can be naturally divided into two components: (i) a baseline black-box probability distribution for the observed variables and (ii) constraints enforced on functionals of this probability distribution. Inference is performed by sampling from the posterior implied by the first component, and finding projections on the space defined by the second component. We discuss the implications of this separation in terms of priors, model selection, and MCMC mixing in latent variable models. Case studies include probabilistic principal component analysis, models of marginal independence, and a interpretable class of structured ordinal probit models.

MO‐FG‐303‐07: Quality Assurance of OAR Segmentation Using Machine Learning And Statistical Shape Models

Purpose:Automating the quality assurance of OAR segmentations is technically a challenging task due to the natural anatomical variations that cannot be easily quantified by computer algorithms. We report on QA method for OAR segmentations that uses recent advances in machine learning to automate the currently‐used manual interpretation of segmentation quality.Materials and Methods:The approach relies on building a statistical model of an organ's shapes and variation across patients from a set of high‐quality contours representative for the anatomy. When presented with a new OAR segmentation for evaluation, the QA procedure fits the shape to the statistical model to mark outlier contours and suggest a correction for their location based on the statistics inferred from the training datasets. Technically, the statistical model of OAR variation is build using a probabilistic PCA model where deformations between learning shapes are modeled as a Gaussian process to create a general non‐parametric model that is flexible in capturing inter‐patient variations.Results:The algorithm was tested in clinical practice by creating custom software that intercepts DICOM RT segmentation files send to the treatment planning system, verifies them against the statistical model, and presents to original and corrected segmentations side‐by‐side in the treatment planning for evaluating changes suggested by the QA procedure. Segmentation quality is quantified through the Dice and Hausdorff distance between the initial and corrected shape.Conclusion:Abnormalities in critical structures delineation can be quantified by a shape model derived from a database of verified segmentation to create a software tool that detects and quantifies point‐by point variations from a normal segmentation. The approach was used to verify OAR segmentation and streamline clinical practice.

Probabilistic Principal Component Analysis to Identify Profiles of Physical Activity Behaviours in the Presence of Non-Ignorable Missing Data

SummaryThe paper is motivated by an accelerometer-based study of physical activity (PA) behaviours in a large cohort of UK school-aged children. Advances in research on PA are accompanied by a growing number of results that are contributing to form a complex picture of PA behaviours in children. One source of such complexity is intimately related to the multiplicity of dimensions associated with PA. Currently a comprehensive individual accelerometer summary can include a large number of outcomes and this clearly poses challenges for the analysis. We explore the application of principal component analysis to accelerometer measurements that are aggregated daily over several days of the week and are affected by missingness. The probabilistic approach to principal component analysis with latent scores is extended to include non-ignorable missing data. The extended likelihood is maximized through a Monte Carlo EM algorithm via adaptive rejection Metropolis sampling. Our findings suggest that physical activity and inactivity are two dimensions over which children aggregate into distinct behavioural profiles, characterized by gender and season but not by anthropometric factors.

Dynamic mixture probabilistic PCA classifier modeling and application for fault classification

A dynamic classifier based on the mixture probabilistic principal component analyzer (MPPCA) is proposed for fault classification. Compared with traditional methods, both fault detection and diagnosis are combined into a single classification task. By introducing a state indicator, the conventional MPPCA model is first designed as a standard classifier. Then, the static MPPCA model based classifier is temporally extended to the dynamic form within the hidden Markov model framework. Both static and dynamic MPPCA classifiers are obtained by using the Expectation‐Maximization algorithm. For performance evaluation, case studies of the continuous stirred tank heater process and the Tennessee Eastman process are carried out. Results indicate that the dynamic MPPCA classifier performs better compared with the static MPPCA classifier and the hidden Markov model based classifier. Copyright © 2015 John Wiley & Sons, Ltd.

An aligned mixture probabilistic principal component analysis for fault detection of multimode chemical processes

A novel approach named aligned mixture probabilistic principal component analysis (AMPPCA) is proposed in this study for fault detection of multimode chemical processes. In order to exploit within-mode correlations, the AMPPCA algorithm first estimates a statistical description for each operating mode by applying mixture probabilistic principal component analysis (MPPCA). As a comparison, the combined MPPCA is employed where monitoring results are softly integrated according to posterior probabilities of the test sample in each local model. For exploiting the cross-mode correlations, which may be useful but are inadvertently neglected due to separately held monitoring approaches, a global monitoring model is constructed by aligning all local models together. In this way, both within-mode and cross-mode correlations are preserved in this integrated space. Finally, the utility and feasibility of AMPPCA are demonstrated through a non-isothermal continuous stirred tank reactor and the TE benchmark process.

Probabilistic principal component analysis for texture modelling of adaptive active appearance models and its application for head pose estimation

This study suggests an application of human–robot interaction based on three‐dimensional real‐time monocular head pose tracker in which active appearance models (AAMs) are utilised to extract facial features. In order to improve texture model, two probabilistic approaches are proposed for principal component analysis in the presence of missing values. It is observed that using the suggested Bayesian model not only increases the fitting accuracy of the model, but also reduces model parameters which may cause an increase in the speed of model fitting. Moreover, contrary to the common assumption in AAM, the gradient matrix must not be supposed to be constant. In this investigation, a method is suggested in which the gradient matrix is adapted with new images during model fitting of video sequences as much as possible. In the next step, by means of suggested methods, operator's head pose will be estimated by POSIT algorithm and by its implementation on PeopleBot robot, enhancement of the interaction between human and robot is presented in order to control the orientation of the robot camera.

Covariance-Guided Mixture Probabilistic Principal Component Analysis (C-MPPCA)

To extract information from high-dimensional data efficiently, visualization tools based on data projection methods have been developed and shown useful. However, a single two-dimensional visualization is often insufficient for capturing all or most interesting structures in complex high-dimensional datasets. For this reason, Tipping and Bishop developed mixture probabilistic principal component analysis (MPPCA) that separates data into multiple groups and enables a unique projection per group; that is, one probabilistic principal component analysis (PPCA) data visualization per group. Because the group labels are assigned to observations based on their high-dimensional coordinates, MPPCA works well to reveal homoscedastic structures in data that differ spatially. In the presence of heteroscedasticity, however, MPPCA may still mask noteworthy data structures. We propose a new method called covariance-guided MPPCA (C-MPPCA) that groups subsets of observations based on covariance, not locality, and, similar to MPPCA, displays them using PPCA. PPCA projects data in the dimensions with the highest variances, thus grouping by covariance makes sense and enables some data structures to be visible that were masked originally by MPPCA. We demonstrate the performance of C-MPPCA in an extensive simulation study. We also apply C-MPPCA to a real world dataset. Supplementary materials for this article are available online.

Process monitoring based on factor analysis: Probabilistic analysis of monitoring statistics in presence of both complete and incomplete measurements

In generic process monitoring approaches based on probabilistic latent variable models, such as probabilistic principal component analysis (PPCA) or factor analysis (FA) model, the online score and residual are characterized by probability distributions. However, only their expectations are involved in the calculations of monitoring statistics, square prediction error (SPE) and Hotelling T2, which ignore the information of their variances and may result in missed fault alarms. Based on the FA model, this paper investigates the probabilistic uncertainties of monitoring statistics arising from both inherent nature and missing measurements of the process data. The proposed method derives the distributions of both the online factor and residual at each sampling instant, and then transforms generic monitoring statistics into general quadratic forms. As a result, novel monitoring statistics are developed based on the probabilistic uncertainties of the generic statistics. In addition, the proposed monitoring statistics are extended to the case of incomplete measurements, in which the conditional distributions of the online measurement, factor and residual are computed and used to construct the statistics for process monitoring. Simulation examples illustrate the feasibility of the proposed method and demonstrate its effectiveness.

Process Monitoring Based on Recursive Probabilistic PCA for Multi-mode Process

A recursive probabilistic principal component analysis (PPCA) based data-driven fault identification method is proposed to handle the missing data samples and the mode transition in multi-mode process. This model is recursively obtained by using the increasing number of normal observations with partly missing data. First, based on the singular value of historic data matrix, the whole process is divided into different steady modes and mode transitions. For steady modes, the conventional PPCA is used to obtain the principal components, and to impute the missing data. When the mode is a mode transition, the proposed recursive PPCA is applied, which can actually reveal the between-mode dynamics for process monitoring and fault detection. After that, in order to identify the faults, a contribution analysis method is developed and used to identify the variables which make the major contributions to the occurrence of faults. The effectiveness of the proposed approach is demonstrated by the Tennessee Eastman chemical process. The results show that the presented approach can accurately detect abnormal events, identify the faults, and it is also robust to mode transitions.

A novel hybrid dimension reduction technique for undersized high dimensional gene expression data sets using information complexity criterion for cancer classification.

Gene expression data typically are large, complex, and highly noisy. Their dimension is high with several thousand genes (i.e., features) but with only a limited number of observations (i.e., samples). Although the classical principal component analysis (PCA) method is widely used as a first standard step in dimension reduction and in supervised and unsupervised classification, it suffers from several shortcomings in the case of data sets involving undersized samples, since the sample covariance matrix degenerates and becomes singular. In this paper we address these limitations within the context of probabilistic PCA (PPCA) by introducing and developing a new and novel approach using maximum entropy covariance matrix and its hybridized smoothed covariance estimators. To reduce the dimensionality of the data and to choose the number of probabilistic PCs (PPCs) to be retained, we further introduce and develop celebrated Akaike's information criterion (AIC), consistent Akaike's information criterion (CAIC), and the information theoretic measure of complexity (ICOMP) criterion of Bozdogan. Six publicly available undersized benchmark data sets were analyzed to show the utility, flexibility, and versatility of our approach with hybridized smoothed covariance matrix estimators, which do not degenerate to perform the PPCA to reduce the dimension and to carry out supervised classification of cancer groups in high dimensions.

Condition for perfect dimensionality recovery by variational Bayesian PCA

Having shown its good performance in many applications, variational Bayesian (VB) learning is known to be one of the best tractable approximations to Bayesian learning. However, its performance was not well understood theoretically. In this paper, we clarify the behavior of VB learning in probabilistic PCA (or fully-observed matrix factorization). More specifically, we establish a necessary and sufficient condition for perfect dimensionality (or rank) recovery in the large-scale limit when the matrix size goes to infinity. Our result theoretically guarantees the performance of VB-PCA. At the same time, it also reveals the conservative nature of VB learning--it offers a low false positive rate at the expense of low sensitivity. By contrasting with an alternative dimensionality selection method, we characterize VB learning in PCA. In our analysis, we obtain bounds of the noise variance estimator, and a new and simple analytic-form solution for the other parameters, which themselves are useful for implementation of VB-PCA.

Automated model selection in covariance estimation and spatial whitening of MEG and EEG signals

Magnetoencephalography and electroencephalography (M/EEG) measure non-invasively the weak electromagnetic fields induced by post-synaptic neural currents. The estimation of the spatial covariance of the signals recorded on M/EEG sensors is a building block of modern data analysis pipelines. Such covariance estimates are used in brain–computer interfaces (BCI) systems, in nearly all source localization methods for spatial whitening as well as for data covariance estimation in beamformers. The rationale for such models is that the signals can be modeled by a zero mean Gaussian distribution. While maximizing the Gaussian likelihood seems natural, it leads to a covariance estimate known as empirical covariance (EC). It turns out that the EC is a poor estimate of the true covariance when the number of samples is small. To address this issue the estimation needs to be regularized. The most common approach downweights off-diagonal coefficients, while more advanced regularization methods are based on shrinkage techniques or generative models with low rank assumptions: probabilistic PCA (PPCA) and factor analysis (FA). Using cross-validation all of these models can be tuned and compared based on Gaussian likelihood computed on unseen data.We investigated these models on simulations, one electroencephalography (EEG) dataset as well as magnetoencephalography (MEG) datasets from the most common MEG systems. First, our results demonstrate that different models can be the best, depending on the number of samples, heterogeneity of sensor types and noise properties. Second, we show that the models tuned by cross-validation are superior to models with hand-selected regularization. Hence, we propose an automated solution to the often overlooked problem of covariance estimation of M/EEG signals. The relevance of the procedure is demonstrated here for spatial whitening and source localization of MEG signals.