Gaussian Process Classification Research Articles

Variational Gaussian process for multisensor classification problems

This paper proposes a new model for multi-sensory data classification. To tackle this problem, probabilistic modeling and variational Bayesian inference are used. A Gaussian Process (GP) classifier is built upon the introduced modeling. Its posterior distribution is approximated using variational Bayesian inference. Finally, labels of test samples are predicted employing this classifier. Very importantly, and in contrast to alternative approaches, the proposed method does not discard samples with missing features and utilizes all available information for training. Furthermore, to take into account that the quality of the information provided by each sensor may differ (some modalities/sensors may provide more reliable/distinctive information than others), we introduce two versions of the algorithm. In the first one, the parameters modeling each sensor performance are shared while in the second one, each sensor parameters are estimated independently. Synthetic and real datasets are utilized to examine the validity of the proposed models. The results obtained for binary classification problems justify their use and confirm their superiority over existing fusion architectures.

Screening Fuels for Autoignition with Small-Volume Experiments and Gaussian Process Classification

Partially reacting candidate fuels under highly dilute conditions across a range of temperatures provides a means to classify the candidates based on traditional ignition characteristics using much lower quantities (sub-mL) than the full octane tests. Using a classifier based on a Gaussian Process model, synthetic species profiles obtained by plug flow reactor simulations at seven temperatures are used to demonstrate that the configuration can be used to classify 95% of the samples correctly for autoignition sensitivity exceeding a threshold (S ≥ 8) and 100%of the samples correctly for research octane number exceeding a threshold (RON ≥ 90). Molecular beam mass spectrometry (MBMS) experimental data at four temperatures is then used as the model input in a real-world test. Despite the nontrivial relationship between the MBMS measurements and speciation as well as experimental noise it is still possible to classify 95% of the samples correctly for RON and 85% of the samples correctly for S in a “leave-one-o...

A hybrid ensemble method for improved prediction of slope stability

SummaryAccurate prediction of slope stability is a significant issue in geomechanics with many artificial intelligence (AI) techniques being utilised. However, the application of AI has not reached its full potential because of the lack of more robust algorithms. In this paper, we proposed a hybrid ensemble method for the improved prediction of slope stability using classifier ensembles and genetic algorithm. Gaussian process classification, quadratic discriminant analysis, support vector machine, artificial neural networks, adaptive boosted decision trees, and k‐nearest neighbours were chosen to be individual AI techniques, and the weighted majority voting was used as the combination method. Validation method was chosen to be the 10‐fold cross‐validation, and performance measures were selected to be the accuracy, the receiver operating characteristic curve, and the area under the receiver operating characteristic curve (AUC). Grid search and genetic algorithm were used for the hyperparameter tuning and weight tuning respectively. The results show that the proposed hybrid ensemble method has great potential in improving the prediction of slope stability. Compared with individual classifiers, the optimum ensemble classifier achieved the highest AUC value (0.943) and the highest accuracy (0.902) on the testing set, denoting that the predictive performance has been improved. The optimum ensemble classifier with the Youden's cut‐off was recommended for slope stability prediction with respect to the AUC value, the accuracy, the true positive rate, and the true negative rate. This research indicates that the use of the classifier ensembles, rather than the search for the ideal individual classifiers, might help for the slope stability prediction.

Large-scale association analysis of climate drought and decline in groundwater quantity using Gaussian process classification (case study: 609 study area of Iran).

BackgroundThe level of groundwater resources is changing rapidly and this requires the discovery of newer groundwater resources. Drought is one of the most significant natural phenomena affecting different aspects of human life and environment. During the last decades, the application of artificial intelligent techniques has been recognized as effective approaches to forecast an annual precipitation rate.MethodIn this study, the association analysis of climate drought and a decline in groundwater level is addressed using Gaussian process classification (GPC) and backpropagation (BP) artificial neural network (ANN). This methodology is proposed to create a framework for decision making and reduce uncertainty in water resource management calculations, and in particular to optimize the management of groundwater drinking water sources.ResultsUnderground water levels in 609 study plains in Iran were used to predict drought over the test period, extending from 2017 to 2021. The artificial intelligence methods were implemented in the Python programming environment to achieve an annual precipitation rate. A statistical summary of the Rasterized Cells of the zoning maps was used to validate the prediction results. Considering the relationship between water quality reductions and drought in Iranian aquifers due to the occurrence of groundwater drought periods, the results were validated by analysis of the effect of climate drought using the Standardized Precipitation Index (SPI) on the occurrence of observed droughts with the Groundwater Resources Index (GRI). The results are well-illustrated by the observation of the predicted digits in the third dimension of the Gaussian distribution.ConclusionAccording to the SPI indicator, the southern regions of the country, and especially the central parts of the plain, can be considered the most affected areas by the most severe future droughts. The prediction results indicate a decrease in drought severity as part of a two-year sequence involving a recurrence of drought exacerbation and relative decline, as well as a failed state after the critical condition of aquifers.

Accurate Diabetes Risk Stratification Using Machine Learning: Role of Missing Value and Outliers

Diabetes mellitus is a group of metabolic diseases in which blood sugar levels are too high. About 8.8% of the world was diabetic in 2017. It is projected that this will reach nearly 10% by 2045. The major challenge is that when machine learning-based classifiers are applied to such data sets for risk stratification, leads to lower performance. Thus, our objective is to develop an optimized and robust machine learning (ML) system under the assumption that missing values or outliers if replaced by a median configuration will yield higher risk stratification accuracy. This ML-based risk stratification is designed, optimized and evaluated, where: (i) the features are extracted and optimized from the six feature selection techniques (random forest, logistic regression, mutual information, principal component analysis, analysis of variance, and Fisher discriminant ratio) and combined with ten different types of classifiers (linear discriminant analysis, quadratic discriminant analysis, naïve Bayes, Gaussian process classification, support vector machine, artificial neural network, Adaboost, logistic regression, decision tree, and random forest) under the hypothesis that both missing values and outliers when replaced by computed medians will improve the risk stratification accuracy. Pima Indian diabetic dataset (768 patients: 268 diabetic and 500 controls) was used. Our results demonstrate that on replacing the missing values and outliers by group median and median values, respectively and further using the combination of random forest feature selection and random forest classification technique yields an accuracy, sensitivity, specificity, positive predictive value, negative predictive value and area under the curve as: 92.26%, 95.96%, 79.72%, 91.14%, 91.20%, and 0.93, respectively. This is an improvement of 10% over previously developed techniques published in literature. The system was validated for its stability and reliability. RF-based model showed the best performance when outliers are replaced by median values.

Video Event Recognition and Anomaly Detection by Combining Gaussian Process and Hierarchical Dirichlet Process Models

In this paper, we present an unsupervised learning framework for analyzing activities and interactions in surveillance videos. In our framework, three levels of video events are connected by Hierarchical Dirichlet Process (<small>HDP</small>) model: low-level visual features, simple atomic activities, and multi-agent interactions. Atomic activities are represented as distribution of low-level features, while complicated interactions are represented as distribution of atomic activities. This learning process is unsupervised. Given a training video sequence, low-level visual features are extracted based on optic flow and then clustered into different atomic activities and video clips are clustered into different interactions. The <small>HDP</small> model automatically decides the number of clusters, i.e., the categories of atomic activities and interactions. Based on the learned atomic activities and interactions, a training dataset is generated to train the Gaussian Process (<small>GP</small>) classifier. Then, the trained <small>GP</small> models work in newly captured video to classify interactions and detect abnormal events in real time. Furthermore, the temporal dependencies between video events learned by HDP-Hidden Markov Models (<small>HMM</small>) are effectively integrated into <small>GP</small> classifier to enhance the accuracy of the classification in newly captured videos. Our framework couples the benefits of the generative model (<small>HDP</small>) with the discriminant model (<small>GP</small>). We provide detailed experiments showing that our framework enjoys favorable performance in video event classification in real-time in a crowded traffic scene.

Deep learning approach for classifying, detecting and predicting photometric redshifts of quasars in the Sloan Digital Sky Survey stripe 82

We have applied a convolutional neural network (CNN) to classify and detect quasars in the Sloan Digital Sky Survey Stripe 82 and also to predict the photometric redshifts of quasars. The network takes the variability of objects into account by converting light curves into images. The width of the images, noted w, corresponds to the five magnitudes ugriz and the height of the images, noted h, represents the date of the observation. The CNN provides good results since its precision is 0.988 for a recall of 0.90, compared to a precision of 0.985 for the same recall with a random forest classifier. Moreover 175 new quasar candidates are found with the CNN considering a fixed recall of 0.97. The combination of probabilities given by the CNN and the random forest makes good performance even better with a precision of 0.99 for a recall of 0.90. For the redshift predictions, the CNN presents excellent results which are higher than those obtained with a feature extraction step and different classifiers (a K-nearest-neighbors, a support vector machine, a random forest and a Gaussian process classifier). Indeed, the accuracy of the CNN within |Δz| < 0.1 can reach 78.09%, within |Δz| < 0.2 reaches 86.15%, within |Δz| < 0.3 reaches 91.2% and the value of root mean square (rms) is 0.359. The performance of the KNN decreases for the three |Δz| regions, since within the accuracy of |Δz| < 0.1, |Δz| < 0.2, and |Δz| < 0.3 is 73.72%, 82.46%, and 90.09% respectively, and the value of rms amounts to 0.395. So the CNN successfully reduces the dispersion and the catastrophic redshifts of quasars. This new method is very promising for the future of big databases such as the Large Synoptic Survey Telescope.

Gaussian Process Regression and Classification for Probabilistic Damage Assessment of Spatially Distributed Systems

We illustrate how methods for non-parametric regression and classification based on Gaussian Processes can be adapted for inferring the condition state of infrastructure components under spatially distributed stressors. When the stressor is modeled by a random field, observations collected in one location can reduce the uncertainty about the stressor intensity also in other locations. Exact inference is possible when the field is Gaussian and observations are perfect or affected by Gaussian noise. However, often the available observations are binary, as those related to the failure or survival of components, and indicate whether the local field is above or below a threshold whose value may also be uncertain. While no efficient scheme for exact inference is available in that setting, we can perform efficient approximate inference when the field is Gaussian and so is the uncertainty on the threshold value. The mathematical formulation for this problem is analogous to that of classification in machine learning, that can be based on latent Gaussian processes. We show how to formulate the problem and how to adapt deterministic methods, as Laplace’s method and Expectation Propagation, and methods based on random number generation, as Monte Carlo uniform sampling and importance sampling, to perform approximate inference. Our illustrative application is the condition assessment of assets exposed to a seismic event. Under specific assumptions, the seismic demand can be modeled as a Gaussian random field, and measures about the demand and about the survival and failure of assets can be processed globally, to update the risk assessment. Specifically, we evaluate methods for approximate inference and discuss their merits.

Gaussian process classification for prediction of in-hospital mortality among preterm infants

Abstract We present a method for predicting preterm infant in-hospital mortality using Bayesian Gaussian process classification. We combined features extracted from sensor measurements, made during the first 72 h of care for 598 Very Low Birth Weight infants of birth weight

Leveraging Statistical Machine Learning to Address Failure Localization in Optical Networks

In this work we consider the problem of fault localization in transparent optical networks.We attempt to localize single-link failures by utilizing statistical machine learning techniques trained on data that describe the network state upon current and past failure incidents. In particular, a Gaussian process classifier is trained on historical data extracted from the examined network, with the goal of modeling and predicting the failure probability of each link therein. To limit the set of suspect links for every failure incident, the proposed approach is complemented by the utilization of a graph-based correlation heuristic. The proposed approach is tested on a number of datasets generated for an orthogonal frequency-division multiplexing-based optical network, and demonstrates that the approach achieves a high localization accuracy (91%–99%) that is insignificantly affected as the size of the historical dataset is reduced. The approach is also compared to a conventional fault localization method that is based on the utilization of monitoring information. It is shown that the conventional method significantly increases the network cost, as measured by the number of monitoring nodes required to achieve the same accuracy as that achieved by the proposed approach. The proposed scheme can be used by service providers to reduce the network cost related to the fault localization procedure. As the approach is generic and does not depend on specific network technologies, it can be applied to different network types, e.g., fixed-grid or space-division multiplexing elastic optical networks.

Remote Sensing Image Classification With Large-Scale Gaussian Processes

Current remote sensing image classification problems have to deal with an unprecedented amount of heterogeneous and complex data sources. Upcoming missions will soon provide large data streams that will make land cover/use classification difficult. Machine learning classifiers can help at this, and many methods are currently available. A popular kernel classifier is the Gaussian process classifier (GPC), since it approaches the classification problem with a solid probabilistic treatment, thus yielding confidence intervals for the predictions as well as very competitive results to state-of-the-art neural networks and support vector machines. However, its computational cost is prohibitive for large scale applications, and constitutes the main obstacle precluding wide adoption. This paper tackles this problem by introducing two novel efficient methodologies for Gaussian Process (GP) classification. We first include the standard random Fourier features approximation into GPC, which largely decreases its computational cost and permits large scale remote sensing image classification. In addition, we propose a model which avoids randomly sampling a number of Fourier frequencies, and alternatively learns the optimal ones within a variational Bayes approach. The performance of the proposed methods is illustrated in complex problems of cloud detection from multispectral imagery and infrared sounding data. Excellent empirical results support the proposal in both computational cost and accuracy.

Applying Machine Learning Techniques for Classifying Cyclin-Dependent Kinase Inhibitors

The importance of protein kinases made them a target for many drug design studies. They play an essential role in cell cycle development and many other biological processes. Kinases are divided into different subfamilies according to the type and mode of their enzymatic activity. Computational studies targeting kinase inhibitors identification is widely considered for modelling kinase-inhibitor. This modelling is expected to help in solving the selectivity problem arising from the high similarity between kinases and their binding profiles. In this study, we explore the ability of two machine-learning techniques in classifying compounds as inhibitors or non-inhibitors for two members of the cyclin-dependent kinases as a subfamily of protein kinases. Random forest and genetic programming were used to classify CDK5 and CDK2 kinases inhibitors. This classification is based on calculated values of chemical descriptors. In addition, the response of the classifiers to adding prior information about compounds promiscuity was investigated. The results from each classifier for the datasets were analyzed by calculating different accuracy measures and metrics. Confusion matrices, accuracy, ROC curves, AUC values, F1 scores, and Matthews correlation, were obtained for the outputs. The analysis of these accuracy measures showed a better performance for the RF classifier in most of the cases. In addition, the results show that promiscuity information improves the classification accuracy, but its significant effect was notably clear with GP classifiers.

Bayesian active probabilistic classification for psychometric field estimation.

Psychometric functions are typically estimated by fitting a parametric model to categorical subject responses. Procedures to estimate unidimensional psychometric functions (i.e., psychometric curves) have been subjected to the most research, with modern adaptive methods capable of quickly obtaining accurate estimates. These capabilities have been extended to some multidimensional psychometric functions (i.e., psychometric fields) that are easily parameterizable, but flexible procedures for general psychometric field estimation are lacking. This study introduces a nonparametric Bayesian psychometric field estimator operating on subject queries sequentially selected to improve the estimate in some targeted way. This estimator implements probabilistic classification using Gaussian processes trained by active learning. The accuracy and efficiency of two different actively sampled estimators were compared to two non-actively sampled estimators for simulations of one of the simplest psychometric fields in common use: the pure-tone audiogram. The actively sampled methods achieved estimate accuracy equivalent to the non-actively sampled methods with fewer observations. This trend held for a variety of audiogram phenotypes representative of the range of human auditory perception. Gaussian process classification is a general estimation procedure capable of extending to multiple input variables and response classes. Its success with a two-dimensional psychometric field informed by binary subject responses holds great promise for extension to complex perceptual models currently inaccessible to practical estimation.

Classification of Solar Wind With Machine Learning

AbstractWe present a four‐category classification algorithm for the solar wind, based on Gaussian Process. The four categories are the ones previously adopted in Xu and Borovsky (2015): ejecta, coronal hole origin plasma, streamer belt origin plasma, and sector reversal origin plasma. The algorithm is trained and tested on a labeled portion of the OMNI data set. It uses seven inputs: the solar wind speed Vsw, the temperature standard deviation σT, the sunspot number R, the F10.7 index, the Alfven speed vA, the proton specific entropy Sp, and the proton temperature Tp compared to a velocity‐dependent expected temperature. The output of the Gaussian Process classifier is a four‐element vector containing the probabilities that an event (one reading from the hourly averaged OMNI database) belongs to each category. The probabilistic nature of the prediction allows for a more informative and flexible interpretation of the results, for instance, being able to classify events as “undecided.” The new method has a median accuracy larger than 90% for all categories, even using a small set of data for training. The Receiver Operating Characteristic curve and the reliability diagram also demonstrate the excellent quality of this new method. Finally, we use the algorithm to classify a large portion of the OMNI data set, and we present for the first time transition probabilities between different solar wind categories. Such probabilities represent the “climatological” statistics that determine the solar wind baseline.

Machine learning to design integral membrane channelrhodopsins for efficient eukaryotic expression and plasma membrane localization.

There is growing interest in studying and engineering integral membrane proteins (MPs) that play key roles in sensing and regulating cellular response to diverse external signals. A MP must be expressed, correctly inserted and folded in a lipid bilayer, and trafficked to the proper cellular location in order to function. The sequence and structural determinants of these processes are complex and highly constrained. Here we describe a predictive, machine-learning approach that captures this complexity to facilitate successful MP engineering and design. Machine learning on carefully-chosen training sequences made by structure-guided SCHEMA recombination has enabled us to accurately predict the rare sequences in a diverse library of channelrhodopsins (ChRs) that express and localize to the plasma membrane of mammalian cells. These light-gated channel proteins of microbial origin are of interest for neuroscience applications, where expression and localization to the plasma membrane is a prerequisite for function. We trained Gaussian process (GP) classification and regression models with expression and localization data from 218 ChR chimeras chosen from a 118,098-variant library designed by SCHEMA recombination of three parent ChRs. We use these GP models to identify ChRs that express and localize well and show that our models can elucidate sequence and structure elements important for these processes. We also used the predictive models to convert a naturally occurring ChR incapable of mammalian localization into one that localizes well.

Human Action Recognition Using Variational Bayesian Logistic Gaussian Process Classification Model

Human Action Recognition Using Variational Bayesian Logistic Gaussian Process Classification Model

Variational Bayesian multinomial logistic Gaussian process classification

The multinomial logistic Gaussian process is a flexible non-parametric model for multi-class classification tasks. These tasks are often involved in solving a pattern recognition problem in real life. In such contexts, the multinomial logistic function (or softmax function) is usually assumed to be the likelihood function. But, exact inferences for this model have proved challenging problem because it requires high-dimensional integration. In this paper, we propose approximate variational Bayesian inference for the multinomial logistic Gaussian process model. First, we compute the second-order approximation for the logarithm of the logistic likelihood function using Taylor series expansion, and derive the posterior distributions of all hidden variables and model parameters using the variational Bayesian inference method. Second, we derive the predictive distribution of the latent classification variable corresponding to the relevant test data point using the characteristics of the Cauchy product for a standard Gaussian process using a learning model parameter. We conducted experiments to verify the effectiveness of the proposed model using a number of synthetic and real datasets. The results show that the proposed model has superior classification capability to existing methods.

Comparative approaches for classification of diabetes mellitus data: Machine learning paradigm

Background and objectiveDiabetes is a silent killer. The main cause of this disease is the presence of excessive amounts of metabolites such as glucose. There were about 387 million diabetic people all over the world in 2014. The financial burden of this disease has been calculated to be about $13,700 per year. According to the World Health Organization (WHO), these figures will more than double by the year 2030. This cost will be reduced dramatically if someone can predict diabetes statistically on the basis of some covariates. Although several classification techniques are available, it is very difficult to classify diabetes. The main objectives of this paper are as follows: (i) Gaussian process classification (GPC), (ii) comparative classifier for diabetes data classification, (iii) data analysis using the cross-validation approach, (iv) interpretation of the data analysis and (v) benchmarking our method against others. MethodsTo classify diabetes, several classification techniques are used such as linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and Naive Bayes (NB). However, most of the medical data show non-normality, non-linearity and inherent correlation structure. So in this paper we adapted Gaussian process (GP)-based classification technique using three kernels namely: linear, polynomial and radial basis kernel. We also investigate the performance of a GP-based classification technique in comparison to existing techniques such as LDA, QDA and NB. Performances are evaluated by using the accuracy (ACC), sensitivity (SE), specificity (SP), positive predictive value (PPV), negative predictive value (NPV) and receiver-operating characteristic (ROC) curves. ResultsPima Indian diabetes dataset is taken as part of the study. This consists of 768 patients, of which 268 patients are diabetic and 500 patients are controls. Our machine learning system shows the performance of GP-based model as: ACC 81.97%, SE 91.79%, SP 63.33%, PPV 84.91% and NPV 62.50% which are larger compared to other methods.

Accelerating the computation of the volume of tissue activated during deep brain stimulation using Gaussian processes

The volume of tissue activated (VTA) is a well-established approach to model the direct effect of deep brain stimulation (DBS) on neural tissue. Previous studies have pointed to its potential clinical applications. However, the elevated computational runtime required to estimate the VTA with standard techniques used in biological neural modeling limits its suitability for practical use. The goal of this study was to develop a novel methodology to reduce the computation time of VTA estimation. To that end, we built a Gaussian process emulator. It combines multicompartment axon models coupled to the stimulating electric field with a Gaussian process classifier (GPC), following the premise that computing the VTA from a field of axons is in essence a binary classification problem. We achieved a considerable reduction in the average time required to estimate the VTA, under both ideal isotropic and realistic anisotropic brain tissue conductive conditions, limiting the loss of accuracy and overcoming other drawbacks entailed by alternative methods.

Parsing the Roles of the Frontal Lobes and Basal Ganglia in Task Control Using Multivoxel Pattern Analysis.

Cognitive control has traditionally been associated with pFC based on observations of deficits in patients with frontal lesions. However, evidence from patients with Parkinson disease indicates that subcortical regions also contribute to control under certain conditions. We scanned 17 healthy volunteers while they performed a task-switching paradigm that previously dissociated performance deficits arising from frontal lesions in comparison with Parkinson disease, as a function of the abstraction of the rules that are switched. From a multivoxel pattern analysis by Gaussian Process Classification, we then estimated the forward (generative) model to infer regional patterns of activity that predict Switch/Repeat behavior between rule conditions. At 1000 permutations, Switch/Repeat classification accuracy for concrete rules was significant in the BG, but at chance in the frontal lobe. The inverse pattern was obtained for abstract rules, whereby the conditions were successfully discriminated in the frontal lobe but not in the BG. This double dissociation highlights the difference between cortical and subcortical contributions to cognitive control and demonstrates the utility of multivariate approaches in investigations of functions that rely on distributed and overlapping neural substrates.