Gaussian Process Classification Research Articles

A Fault-Tolerant Early Classification Approach for Human Activities Using Multivariate Time Series

Activity classification has been an interesting area of research for many years, to better understand human behavior. Recent advancements in embedded computing systems allowed the emergence of several state-of-art solutions for human activity classification using sensors of a smartphone. The sensors generate temporal sequences of observations for human activity, which is called as Multivariate Time Series (MTS). Current state-of-art solutions for human activity classification suffer from two major limitations: first, the length of testing MTS should be equal to the training MTS and second, the MTS should not have any faulty time series. In real-time applications, it is desirable to classify a human activity using an incomplete MTS as early as possible. In this work, we propose a fault-tolerant early classification of MTS (FECM) approach to address these limitations. FECM builds a set of classification models using MTS training dataset. The approach employs Gaussian Process classifier to estimate minimum required length of time series, which is used to predict a class label of new MTS. Further, FECM uses an Auto Regressive Integrated Moving Average model to identify faulty time series in the new MTS. Finally, we conduct an experiment to evaluate the performance of FECM using accuracy and earliness metrics.

Fault diagnosis of reciprocating compressor using a novel ensemble empirical mode decomposition-convolutional deep belief network

In order to denoise the raw signal and fuse multiple sources of information for the fault diagnosis of reciprocating compressor, this paper proposes a novel convolutional deep belief network-based method and employs a novel framework fusing multi-source information to improve the performance of fault diagnosis. Firstly, signals from different sensors of the RC are input into an auto-denoising network, namely, ensemble empirical model decomposition-convolutional deep belief network, to denoise the signal and to extract more robust features by the unsupervised learning. Secondly, the extracted features of each source are input into multiple Gaussian process classifiers which are adopted as the members of probabilistic committee machine (PCM) to calculate the probabilities that each fault occurs. Finally, these probabilities are combined with an optimized weight to make a committee decision on fault type. The proposed method combines the information from multiple sources and enhances the robustness of fault diagnosis. Data from an industrial plant were collected to verify the proposed method. The obtained results demonstrate that the proposed method can effectively diagnose the RC faults with the accuracy rate of up to 91.89%. Furthermore, a comparison of the proposed method with the other methods illustrates the superiority of the proposed method for the diagnosis of RC faults.

Modeling Short Term Rainfall Forecast Using Neural Networks, and Gaussian Process Classification Based on the SPI Drought Index

Determining the quantitative occurrence of droughts, discovering the spatial correlation of droughts, and predicting the occurrence of undesirable classes of drinking water quality and aquifer farming is of high importance. In this research, the Standardized Precipitation Index (SPI) was calculated and analyzed with a monthly survey of 26,027 wells in 609 study areas over a period of 20 years. After analyzing the missing data, the annual rainfall was forecasted in 362 synoptic stations of the country based on an artificial intelligence model. In addition, statistical relationships were extracted in order to achieve a comprehensive and historical map of the state of shortages and surpluses of water resources, as well as verification of artificial intelligence relationships in predicting base data cultivars. The results indicated that the “mild drought” indicator was steeper than the “near-normal drought” indicator. Eventually, the southern and eastern regions and certain parts of the northeast of the country in the period from 2005 to 2015 were placed in the 7th and 8th classes, which indicates severe drought. The analysis of the period 1994–2014 showed that the plains of the Sistan and Baluchestan Province in the south-east region of the country have been significantly more affected by the droughts. With the exception of the central parts of Khorasan, the general eastern, southeast, and southern regions of the country can be considered as an absolute drought class for the long term.

Prediction of Black Sigatoka Disease in Banana Plants By Data Mining Classification Techniques using Scikit for Python

Agriculture has been evolving since humans started cultivating plants for food consumption. As the agriculture field evolves, the disease control measures too have evolved. Now in this modern era, disease in plants can be easily identified using computers. Data mining is the process of obtaining the useful information from the data. Before the electronic era, diseases in plants are identified just by seeing the symptoms of the plants. Similarly, we can identify the diseases in plants using data mining by supplying the disease symptoms data and classify them accordingly. The purpose of this paper is focusing on the prediction of the diseases from images of black sigatoka disease and uses the following methods: MultilayerPerceptrons, SVM,KNeighborsClassifier,K-NeighborsRegressor, Gaussian Process Regressor, Gaussian Process Classifier, GaussianNB, Decision Tree Classifier, Decision Tree Regressor, linear models such as Linear Regression, RidgeCV, Lasso, ElasticNet, Logistic RegressionCV, SGD Classifier, Perceptron and Passive Aggressive Classifier and ensemble models of the above classifiers. The results are compared, and multilayer perceptron model is seen to give better results for individual classifiers and ensemble of week classifiers gives better results when ensembled. In future, a new hybrid algorithm would be used from the above algorithms for attaining better accuracy. The scikit is a library used for classification, clustering, regression, dimensionality reduction,model selection and preprocessing. Our paper discusses various classifiers used in scikit-learn library for Python and their ensembling is done. This can be applied to all the classification tasks. Classification is done for classifying the black sigatoka disease in banana from healthy leaves.This disease is the most vulnerable one among banana plants.

Classifying Drugs by their Arrhythmogenic Risk Using Machine Learning

All medications have adverse effects. Among the most serious of these are cardiac arrhythmias. Current paradigms for drug safety evaluation are costly, lengthy, conservative, and impede efficient drug development. Here, we combine multiscale experiment and simulation, high-performance computing, and machine learning to create a risk estimator to stratify new and existing drugs according to their proarrhythmic potential. We capitalize on recent developments in machine learning and integrate information across 10 orders of magnitude in space and time to provide a holistic picture of the effects of drugs, either individually or in combination with other drugs. We show, both experimentally and computationally, that drug-induced arrhythmias are dominated by the interplay between two currents with opposing effects: the rapid delayed rectifier potassium current and the L-type calcium current. Using Gaussian process classification, we create a classifier that stratifies drugs into safe and arrhythmic domains for any combinations of these two currents. We demonstrate that our classifier correctly identifies the risk categories of 22 common drugs exclusively on the basis of their concentrations at 50% current block. Our new risk assessment tool explains under which conditions blocking the L-type calcium current can delay or even entirely suppress arrhythmogenic events. Using machine learning in drug safety evaluation can provide a more accurate and comprehensive mechanistic assessment of the proarrhythmic potential of new drugs. Our study paves the way toward establishing science-based criteria to accelerate drug development, design safer drugs, and reduce heart rhythm disorders.

On active learning methods for manifold data

Active learning is a major area of interest within the field of machine learning, especially when the labeled instances are very difficult, time-consuming or expensive to obtain. In this paper, we review various active learning methods for manifold data, where the intrinsic manifold structure of data is also incorporated into the active learning query strategies. In addition, we present a new manifold-based active learning algorithm for Gaussian process classification. This new method uses a data-dependent kernel derived from a semi-supervised model that considers both labeled and unlabeled data. The method performs a regularization on the smoothness of the fitted function with respect to both the ambient space and the manifold where the data lie. The regularization parameter is treated as an additional kernel (covariance) parameter and estimated from the data, permitting adaptation of the kernel to the given dataset manifold geometry. Comparisons with other AL methods for manifold data show faster learning performance in our empirical experiments. MATLAB code that reproduces all examples is provided as supplementary materials.

Classification of Non-Discriminant ERD/ERS Comprising Motor Imagery Electroencephalography Signals

Classification of Motor Imagery (MI) Electroencephalography (EEG) signals has always been an important aspect of Brain Computer Interface (BCI) systems. Event Related Desynchronization (ERD)/ Event Related Synchronization (ERS) plays a significant role in finding discriminant features of MI EEG signals. ERD/ERS is one type and Evoked Potential (EP) is another type of brain response. This study focuses upon the classification of MI EEG signals by Removing Evoked Potential (REP) from non-discriminant MI EEG data in filter band selection, called REP. This optimization is done to enhance the classification performance. A comprehensive comparison of several pipelines is presented by using famous feature extraction methods, namely Common Spatial Pattern (CSP), XDawn. The effectiveness of REP is demonstrated on the PhysioNet dataset which is an online data resource. Comparison is done between the performance of pipelines including proposed one (Common Spatial Pattern (CSP) and Gaussian Process Classifier (GPC)) as well as before and after applying REP. It is observed that the REP approach has improved the classification accuracy of all the subjects used as well as all the pipelines, including state of the art algorithms, up to 20%.

Convolutional Neural Network and Bayesian Gaussian Process in Driving Anger Recognition

With the development of motorization, road traffic crashes have become the leading cause of death in many countries. Among roadway traffic crashes, almost 90% of accidents are related to driver behaviors, wherein driving anger is one of the most leading causes to vehicle crash-related conditions. To some extent, angry driving is considered more dangerous than typical driving distraction due to emotion agitation. Aggressive driving behaviors create many kinds of roadway traffic safety hazards. Mitigating potential risk caused by road rage is essential to increase the overall level of traffic safety. This paper puts forward an integrated computer vision model composed of convolutional neural network in feature extraction and Bayesian Gaussian process in classification to recognize driver anger and distinguish angry driving from natural driving status. Histogram of gradients (HOG) was applied to extract facial features. Convolutional neural network extracted features on eye, eyebrow, and mouth, which are considered most related to anger emotion. Extracted features with its probability were sent to Bayesian Gaussian process classier as input. Integral analysis on three extracted features was conducted by Gaussian process classifier and output returned the likelihood of being anger from the overall study of all extracted features. An overall accuracy rate of 86.2% was achieved in this study. Tongji University 8-Degree-of-Freedom driving simulator was used to collect data from 30 recruited drivers and build test scenario.

A novel reliability analysis method based on Gaussian process classification for structures with discontinuous response

Reliability analysis techniques combining with various surrogate models have attracted increasing attention because of their accuracy and great efficiency. However, they primarily focus on the structures with continuous response, while very rare researches on the reliability analysis for structures with discontinuous response are carried out. Furthermore, existing adaptive reliability analysis methods based on importance sampling (IS) still have some intractable defects when dealing with small failure probability, and there is no related research on reliability analysis for structures involving discontinuous response and small failure probability. Therefore, this paper proposes a novel reliability analysis method called AGPC-IS for such structures, which combines adaptive Gaussian process classification (GPC) and adaptive-kernel-density-estimation-based IS. In AGPC-IS, an efficient adaptive strategy for design of experiments (DoE), taking into consideration the classification uncertainty, the sampling uniformity and the regional classification accuracy improvement, is developed with the purpose of improving the accuracy of Gaussian process classifier. The adaptive kernel density estimation is introduced for constructing the quasi-optimal density function of IS. In addition, a novel and more precise stopping criterion is also developed from the perspective of the stability of failure probability estimation.The efficiency, superiority and practicability of AGPC-IS are verified by three examples

Numerical simulation, clustering, and prediction of multicomponent polymer precipitation

AbstractMulticomponent polymer systems are of interest in organic photovoltaic and drug delivery applications, among others where diverse morphologies influence performance. An improved understanding of morphology classification, driven by composition-informed prediction tools, will aid polymer engineering practice. We use a modified Cahn–Hilliard model to simulate polymer precipitation. Such physics-based models require high-performance computations that prevent rapid prototyping and iteration in engineering settings. To reduce the required computational costs, we apply machine learning (ML) techniques for clustering and consequent prediction of the simulated polymer-blend images in conjunction with simulations. Integrating ML and simulations in such a manner reduces the number of simulations needed to map out the morphology of polymer blends as a function of input parameters and also generates a data set which can be used by others to this end. We explore dimensionality reduction, via principal component analysis and autoencoder techniques, and analyze the resulting morphology clusters. Supervised ML using Gaussian process classification was subsequently used to predict morphology clusters according to species molar fraction and interaction parameter inputs. Manual pattern clustering yielded the best results, but ML techniques were able to predict the morphology of polymer blends with ≥90% accuracy.

Classifying heterogeneous presentations of PTSD via the default mode, central executive, and salience networks with machine learning.

Intrinsic connectivity networks (ICNs), including the default mode network (DMN), the central executive network (CEN), and the salience network (SN) have been shown to be aberrant in patients with posttraumatic stress disorder (PTSD). The purpose of the current study was to a) compare ICN functional connectivity between PTSD, dissociative subtype PTSD (PTSD+DS) and healthy individuals; and b) to examine the use of multivariate machine learning algorithms in classifying PTSD, PTSD+DS, and healthy individuals based on ICN functional activation. Our neuroimaging dataset consisted of resting-state fMRI scans from 186 participants [PTSD (n=81); PTSD+DS (n=49); and healthy controls (n=56)]. We performed group-level independent component analyses to evaluate functional connectivity differences within each ICN. Multiclass Gaussian Process Classification algorithms within PRoNTo software were then used to predict the diagnosis of PTSD, PTSD+DS, and healthy individuals based on ICN functional activation. When comparing the functional connectivity of ICNs between PTSD, PTSD+DS and healthy controls, we found differential patterns of connectivity to brain regions involved in emotion regulation, in addition to limbic structures and areas involved in self-referential processing, interoception, bodily self-consciousness, and depersonalization/derealization. Machine learning algorithms were able to predict with high accuracy the classification of PTSD, PTSD+DS, and healthy individuals based on ICN functional activation. Our results suggest that alterations within intrinsic connectivity networks may underlie unique psychopathology and symptom presentation among PTSD subtypes. Furthermore, the current findings substantiate the use of machine learning algorithms for classifying subtypes of PTSD illness based on ICNs.

Individual prediction of psychotherapy outcome in posttraumatic stress disorder using neuroimaging data

Trauma-focused psychotherapy is the first-line treatment for posttraumatic stress disorder (PTSD) but 30–50% of patients do not benefit sufficiently. We investigated whether structural and resting-state functional magnetic resonance imaging (MRI/rs-fMRI) data could distinguish between treatment responders and non-responders on the group and individual level. Forty-four male veterans with PTSD underwent baseline scanning followed by trauma-focused psychotherapy. Voxel-wise gray matter volumes were extracted from the structural MRI data and resting-state networks (RSNs) were calculated from rs-fMRI data using independent component analysis. Data were used to detect differences between responders and non-responders on the group level using permutation testing, and the single-subject level using Gaussian process classification with cross-validation. A RSN centered on the bilateral superior frontal gyrus differed between responders and non-responder groups (PFWE < 0.05) while a RSN centered on the pre-supplementary motor area distinguished between responders and non-responders on an individual-level with 81.4% accuracy (P < 0.001, 84.8% sensitivity, 78% specificity and AUC of 0.93). No significant single-subject classification or group differences were observed for gray matter volume. This proof-of-concept study demonstrates the feasibility of using rs-fMRI to develop neuroimaging biomarkers for treatment response, which could enable personalized treatment of patients with PTSD.

Gaussian process classification for variable fidelity data

In this paper we address a classification problem where two sources of labels with different levels of fidelity are available. Our approach is to combine data from both sources by applying a co-kriging schema on latent functions, which allows the model to account item-dependent labeling discrepancy. We provide an extension of Laplace inference for Gaussian process classification, that takes into account multi-fidelity data. We evaluate the proposed method on real and synthetic datasets and show that it is more resistant to different levels of discrepancy between sources than other approaches for data fusion. Our method can provide accuracy/cost trade-off for a number of practical tasks such as crowd-sourced data annotation and feasibility regions construction in engineering design.

Analog/RF Post-silicon Tuning via Bayesian Optimization

Tunable analog/RF circuit has emerged as a promising technique to address the significant performance uncertainties caused by process variations. To optimize these tunable circuits after fabrication, most existing post-silicon programming methods are developed by using real-valued performance metrics. However, when measuring a performance of interest on silicon, it is often substantially more expensive to obtain a real-valued measurement than a binary testing outcome (i.e., pass or fail). In this article, we propose a Gaussian Process Classification model to capture the binary performance metrics of tunable analog/RF circuits. Based on these models, post-silicon programming is cast into an optimization problem that can be solved by a novel Bayesian optimization algorithm. Moreover, measurement noises are further incorporated into our proposed post-silicon programming to produce a robust circuit. Two circuit examples demonstrate that the proposed approach can efficiently program tunable circuits with binary performance metrics while other conventional methods are not applicable.

NIMG-30. REPRODUCIBLE RADIOMIC MAPPING OF TUMOR CELL DENSITY BY MACHINE LEARNING AND DOMAIN ADAPTATION

Abstract BACKGROUND An important challenge in radiomics research is reproducibility. Images are collected on different image scanners and protocols, which introduces significant variability even for the same type of image across institutions. In the present proof-of-concept study, we address the reproducibility issue by using domain adaptation – an algorithm that transforms the radiomic features of each new patient to align with the distribution of features formed by the patient samples in a training set. METHOD Our dataset included 18 patients in training with a total of 82 biopsy sample. The pathological tumor cell density was available for each sample. Radiomic (statistical + texture) features were extracted from the region of six image contrasts locally matched with each biopsy sample. A Gaussian Process (GP) classifier was built to predict tumor cell density using radiomic features. Another 6 patients were used to test the training model. These patients had a total of 31 biopsy samples. The images of each test patient were purposely normalized using a different approach, i.e., using the CSF instead of the whole brain as the reference. This was to mimic the practical scenario of image source discrepancy between different patients. Domain adaptation was applied to each test patient. RESULTS Among the 18 training patients, the leave-one-patient-out cross validation accuracy is 0.81 AUC, 0.78 sensitivity, and 0.83 specificity. When the trained model was applied to the 6 test patients (purposely normalized using a different approach than that of the training data), the accuracy dramatically reduced to 0.39 AUC, 0.08 sensitivity, and 0.61 specificity. After using domain adaption, the accuracy improved to 0.68 AUC, 0.62 sensitivity, and 0.72 specificity. CONCLUSION We provide candidate enabling tools to address reproducibility in radiomics models by using domain adaption algorithms to account for discrepancy of the images between different patients.

Alpha divergence minimization in multi-class Gaussian process classification

This paper analyzes the minimization of α-divergences in the context of multi-class Gaussian process classification. For this task, several methods are explored, including memory and computationally efficient variants of the Power Expectation Propagation algorithm, which allow for efficient training using stochastic gradients and mini-batches. When these methods are used for training, very large datasets (several millions of instances) can be considered. The proposed methods are also very general as they can interpolate between other popular approaches for approximate inference based on Expectation Propagation (EP) (α → 1) and Variational Bayes (VB) (α → 0) simply by varying the α parameter. An exhaustive empirical evaluation analyzes the generalization properties of each of the proposed methods for different values of the α parameter. The results obtained show that one can do better than EP and VB by considering intermediate values of α.

Online Identification of Interaction Behaviors From Haptic Data During Collaborative Object Transfer

Joint object transfer is a complex task, which is less structured and less specific than what is existing in several industrial settings. When two humans are involved in such a task, they cooperate through different modalities to understand the interaction states during operation and mutually adapt to one another's actions. Mutual adaptation implies that both partners can identify how well they collaborate (i.e. infer about the interaction state) and act accordingly. These interaction states can define whether the partners work in harmony, face conflicts, or remain passive during interaction. Understanding how two humans work together during physical interactions is important when exploring the ways a robotic assistant should operate under similar settings. This study acts as a first step to implement an automatic classification mechanism during ongoing collaboration to identify the interaction state during object co-manipulation. The classification is done on a dataset consisting of data from 40 subjects, who are partnered to form 20 dyads. The dyads experiment in a physical human-human interaction (pHHI) scenario to move an object in an haptics-enabled virtual environment to reach predefined goal configurations. In this study, we propose a sliding-window approach for feature extraction and demonstrate the online classification methodology to identify interaction patterns. We evaluate our approach using 1) a support vector machine classifier (SVMc) and 2) a Gaussian Process classifier (GPc) for multi-class classification, and achieve over ${\bf 80}\%$ accuracy with both classifiers when identifying general interaction types.

PTSD and its dissociative subtype through the lens of the insula: Anterior and posterior insula resting-state functional connectivity and its predictive validity using machine learning.

Individuals with post-traumatic stress disorder (PTSD) typically experience states of reliving and hypervigilance; however, the dissociative subtype of PTSD (PTSD+DS) presents with additional symptoms of depersonalization and derealization. Although the insula is critical to emotion processing, its association with these contrasting symptom profiles is yet to be fully delineated. Accordingly, we investigated insula subregion resting-state functional connectivity patterns among individuals with PTSD, PTSD+DS, and healthy controls. Using SPM12 and PRONTO software, we implemented a seed-based resting-state functional connectivity approach, along with multiclass Gaussian process classification machine learning, respectively, in order to evaluate unique patterns and the predictive validity of insula subregion connectivity among individuals with PTSD (n=84), PTSD+DS (n=49), and age-matched healthy controls (n=51). Ascompared to PTSD and PTSD+DS, healthy controls showed increased right anterior and posterior insula connectivity with frontal lobe structures. By contrast, PTSD showed increased bilateral posterior insula connectivity with subcortical structures, including the periaqueductal gray. Strikingly, ascompared to PTSD and controls, PTSD+DS showed increased bilateral anterior and posterior insula connectivity with posterior cortices, including the left lingual gyrus and the left precuneus. Moreover, machine learning analyses were able to classify PTSD, PTSD+DS, and controls using insula subregion connectivity patterns with 80.4% balanced accuracy (p < .01). These findings suggest a neurobiological distinction between PTSD and its dissociative subtype with regard to insula subregion functional connectivity patterns. Furthermore, machine learning algorithms were able to utilize insula resting-state connectivity patterns to discriminate between participant groups with high predictive accuracy.

RSSI-Power-Based Direction of Arrival Estimation of Partial Discharges in Substations

The localization of partial discharges in air-insulated substations using ultra-high frequency technology is widely studied for power equipment early warning purposes. Ultra-high frequency partial discharge localization systems are usually based on the time-difference of electromagnetic wave signals. However, the large size of test equipment and the need for a high sampling rate and time synchronization accuracy limit their practical application. To address this challenge, this paper proposes a power-based partial discharge direction of arrival method using a received signal strength indicator from an ultra-high frequency wireless sensor array. Furthermore, the Gaussian mixture model is used for noise suppression, and the Gaussian process classifier is used for line of sight received signal strength indicator data identification. Laboratory tests are performed and the results show the average error of direction of arrival is less than 5°. The results verify the effectiveness of the proposed partial discharge localization system.

White Matter Microstructure in Pediatric Bipolar Disorder and Disruptive Mood Dysregulation Disorder

Disruptive mood dysregulation disorder (DMDD) codifies severe, chronic irritability. Youths with bipolar disorder (BD) also present with irritability, but with an episodic course. To date, it is not clear whether aberrant white matter microstructure-a well-replicated finding in BD-can be observed in DMDD and relates to symptoms of irritability. We acquired diffusion tensor imaging data from 118 participants (BD= 36, DMDD= 44, healthy volunteers (HV= 38). Images of fractional anisotropy (FA), axial diffusivity (AD), and radial diffusivity (RD) were processed with tract-based spatial statistics controlling for age and sex. The data were also used to train Gaussian process classifiers to predict diagnostic group. In BD vs DMDD, FA in the corticospinal tract was reduced. In DMDD vs HV, reductions in FA and AD were confined to the anterior corpus callosum. In BD vs HV, widespread reductions in FA and increased RD were observed. FA in the anterior corpus callosum and corticospinal tract was negatively associated with irritability. The Gaussian process classifier could not discriminate between BD and DMDD, but achieved 68% accuracy in predicting DMDD vs HV and 75% accuracy in predicting BD vs HV. Aberrant white matter microstructure was associated with both categorical diagnosis and the dimension of irritability. Alterations in DMDD were regionally discrete and related to reduced AD. In BD, we observed widespread increases in RD, supporting the hypothesis of altered myelination in BD. These findings will contribute to the pathophysiological understanding of DMDD and its differentiation from BD. Studies of Brain Function and Course of Illness in Pediatric Bipolar Disorder; https://clinicaltrials.gov/; NCT00025935; Child & Adolescent Bipolar Disorder Brain Imaging and Treatment Study; https://clinicaltrials.gov/; NCT00006177.