Gaussian Process Classification Research Articles

Texture-Based Seafloor Characterization Using Gaussian Process Classification

Synthetic aperture sonar (SAS) allows for high-resolution imagery of the seafloor with a quality that approaches that of an optical camera. Identifying seabed types within SAS images can prove very useful in a setting where autonomous underwater vehicles (AUVs) are used, since information about the bottom type can be used to determine the behavior of the AUV. In this article, we present a new algorithm for the automatic detection of different seabed types based on SAS image texture. Using a Gaussian process classification (GPC) model, we were able to predict the seabed type, as well as an uncertainty estimate on this prediction. The GPC-based model is able to learn the most informative textures from a training data set, and by construction is robust against outliers or data points that are very different from those in the training set. An additional benefit of the model is that it can be easily configured to hit the desired tradeoff between computational complexity and accuracy.

Dense Incremental Metric-Semantic Mapping for Multiagent Systems via Sparse Gaussian Process Regression

In this article, we develop an online probabilistic metric-semantic mapping approach for mobile robot teams relying on streaming RGB-D observations. The generated maps contain full continuous distributional information about the geometric surfaces and semantic labels (e.g., chair, table, and wall). Our approach is based on online Gaussian process (GP) training and inference and avoids the complexity of GP classification by regressing a truncated signed distance function (TSDF) of the regions occupied by different semantic classes. Online regression is enabled through a sparse pseudo-point approximation of the GP posterior. To scale to large environments, we further consider spatial domain partitioning via a hierarchical tree structure with overlapping leaves. An extension to a multirobot setting is developed by having each robot execute its own online measurement update and then combine its posterior parameters via local weighted geometric averaging with those of its neighbors. This yields a distributed information processing architecture, in which the GP map estimates of all the robots converge to a common map of the environment while relying only on local one-hop communication. Our experiments demonstrate the effectiveness of the probabilistic metric-semantic mapping technique in 2-D and 3-D environments in both the single- and multirobot settings and in comparison to a deep TSDF neural network approach.

Design and Implementation of a Self-Learner Smart Home System Using Machine Learning Algorithms

Smart home systems are the integration of technology and services through the network for a better quality of life. Smart homes perform daily housework and activities more easily without user intervention or with remote control of the user. In this study, a machine learning-based smart home system has been developed. The aim of the study is to design a system that can continuously improve itself and learn instead of an ordinary smart home system that can be remotely controlled. The developed machine learning model predicts the routine activities of the users in the home and performs some operations for the user autonomously. The dataset used in the study consists of real data received from the sensors as a result of the daily use. Naive Bayes (NB) (i.e. Gaussian NB, Bernoulli NB, Multinomial NB, and Complement NB), ensemble (i.e. Random Forest, Gradient Tree Boosting and eXtreme Gradient Boosting), linear (i.e. Logistic Regression, Stochastic Gradient Descent, and Passive-Aggressive Classification), and other (i.e. Decision Tree, Support Vector Machine, K Nearest Neighbor, Gaussian Process Classifier (GPC), Multilayer Perceptron) machine learning-based algorithms were utilized. The performance of the proposed smart home system was evaluated using several performance metrics: The best results were obtained from the GPC algorithm (i.e. Precision: 0.97, Recall: 0.98, F1-score: 0.97, Accuracy: 0.97).

Bagged Ensemble of Gaussian Process Classifiers for Assessing Rockburst Damage Potential with an Imbalanced Dataset

The evaluation of rockburst damage potential plays a significant role in managing rockburst risk and guaranteeing the safety of personnel. However, it is still a challenging problem because of its complex mechanisms and numerous influencing factors. In this study, a bagged ensemble of Gaussian process classifiers (GPCs) is proposed to assess rockburst damage potential with an imbalanced dataset. First, a rockburst dataset including seven indicators and four levels is collected. To address classification problems with an imbalanced dataset, a novel model that integrates the under-sampling technique, Gaussian process classifier (GPC) and bagging method is constructed. Afterwards, the comprehensive performance of the proposed model is evaluated using the values of accuracy, precision, recall, and F1. Finally, the methodology is applied to assess rockburst damage potential in the Perseverance nickel mine. Results show that the performance of the proposed bagged ensemble of GPCs is acceptable, and the integration of data preprocessing, under-sampling technique, GPC, and bagging method can improve the model performance. The proposed methodology can provide an effective reference for the risk management of rockburst.

Adversarial vulnerability bounds for Gaussian process classification

Protecting ML classifiers from adversarial examples is crucial. We propose that the main threat is an attacker perturbing a confidently classified input to produce a confident misclassification. We consider in this paper the L_0 attack in which a small number of inputs can be perturbed by the attacker at test-time. To quantify the risk of this form of attack we have devised a formal guarantee in the form of an adversarial bound (AB) for a binary, Gaussian process classifier using the EQ kernel. This bound holds for the entire input domain, bounding the potential of any future adversarial attack to cause a confident misclassification. We explore how to extend to other kernels and investigate how to maximise the bound by altering the classifier (for example by using sparse approximations). We test the bound using a variety of datasets and show that it produces relevant and practical bounds for many of them.

Introducing ARTMO's Machine-Learning Classification Algorithms Toolbox: Application to Plant-Type Detection in a Semi-Steppe Iranian Landscape.

Accurate plant-type (PT) detection forms an important basis for sustainable land management maintaining biodiversity and ecosystem services. In this sense, Sentinel-2 satellite images of the Copernicus program offer spatial, spectral, temporal, and radiometric characteristics with great potential for mapping and monitoring PTs. In addition, the selection of a best-performing algorithm needs to be considered for obtaining PT classification as accurate as possible. To date, no freely downloadable toolbox exists that brings the diversity of the latest supervised machine-learning classification algorithms (MLCAs) together into a single intuitive user-friendly graphical user interface (GUI). To fill this gap and to facilitate and automate the usage of MLCAs, here we present a novel GUI software package that allows systematically training, validating, and applying pixel-based MLCA models to remote sensing imagery. The so-called MLCA toolbox has been integrated within ARTMO’s software framework developed in Matlab which implements most of the state-of-the-art methods in the machine learning community. To demonstrate its utility, we chose a heterogeneous case study scene, a landscape in Southwest Iran to map PTs. In this area, four main PTs were identified, consisting of shrub land, grass land, semi-shrub land, and shrub land–grass land vegetation. Having developed 21 MLCAs using the same training and validation, datasets led to varying accuracy results. Gaussian process classifier (GPC) was validated as the top-performing classifier, with an overall accuracy (OA) of 90%. GPC follows a Laplace approximation to the Gaussian likelihood under the supervised classification framework, emerging as a very competitive alternative to common MLCAs. Random forests resulted in the second-best performance with an OA of 86%. Two other types of ensemble-learning algorithms, i.e., tree-ensemble learning (bagging) and decision tree (with error-correcting output codes), yielded an OA of 83% and 82%, respectively. Following, thirteen classifiers reported OA between 70% and 80%, and the remaining four classifiers reported an OA below 70%. We conclude that GPC substantially outperformed all classifiers, and thus, provides enormous potential for the classification of a diversity of land-cover types. In addition, its probabilistic formulation provides valuable band ranking information, as well as associated predictive variance at a pixel level. Nevertheless, as these are supervised (data-driven) classifiers, performances depend on the entered training data, meaning that an assessment of all MLCAs is crucial for any application. Our analysis demonstrated the efficacy of ARTMO’s MLCA toolbox for an automated evaluation of the classifiers and subsequent thematic mapping.

Election Forecasting Using Macroeconomic and Social Indicators via Machine Learning

A comparative analysis of machine learning models is executed for the forecasting of incumbent party losses during federal elections in democratic countries. A proprietary dataset that encompasses a wide array of potential economic and social factors affecting election outcomes is compiled, and the most significant factors are identified and evaluated. A myriad of the most popular machine learning models for supervised learning is applied to the dataset, utilizing them as classifiers to predict whether the incumbent party stays in power during federal elections for eleven of the world’s most populous and democratic countries: the United States, Canada, the United Kingdom, the Netherlands, Austria, Norway, Sweden, Denmark, Australia, India, and New Zealand. The results show that the most significant factors for election outcomes are inflation growth rate, unemployment growth rate, and voter turnout growth rate. Multilayer perceptron produces the most accurate classifications. Additionally, Gaussian models such as Gaussian process classifier and Gaussian naive Bayes have the poorest classification accuracy.

Rail gage-based risk detection Using iPhone 12 pro

Federal Railroad Administration strictly regulates the inspection frequency of all track classes to ensure timely identification of rail defects including irregular gage which is a devastating rail geometry defect. Conventional rail inspection methods are both costly and labor-intensive, whereas existing novel technologies can be expensive and mostly focus on a specific inspection area, e.g. vertical alignment. iPhone 12 Pro was introduced to the public recently with a low-cost, low-resolution light detection and ranging (LiDAR) sensor that is purposed for better photography and virtual reality. Thanks to its portability and computational capacity, iPhone 12 Pro can potentially be used as a portable solution for irregular gage inspection, whose capacity and feasibility are unknown. This study first investigated the capability of the iPhone 12 Pro in calculating unloaded rail gages by its embedded LiDAR sensor. The results showed that uncalibrated raw gage values measured by the iPhone 12 Pro LiDAR sensor were systematically lower than the ground-truth values. The proposed method in this study then introduced logistic regression to calibrate the measured values through balancing the prediction performance and the efficiency, followed by validations using a Gaussian process classifier. The results show that the proposed method correctly identified all 39 high-risk locations with 227 false alarmed locations. The proposed method with the iPhone 12 Pro LiDAR sensor could potentially narrow down the possible “high-risk” gage sections and may result in a significant reduction in the field inspection workload by 48%.

A Machine Learning Framework for Detecting COVID-19 Infection Using Surface-Enhanced Raman Scattering.

In this study, we explored machine learning approaches for predictive diagnosis using surface-enhanced Raman scattering (SERS), applied to the detection of COVID-19 infection in biological samples. To do this, we utilized SERS data collected from 20 patients at the University of Maryland Baltimore School of Medicine. As a preprocessing step, the positive-negative labels are obtained using Polymerase Chain Reaction (PCR) testing. First, we compared the performance of linear and nonlinear dimensionality techniques for projecting the high-dimensional Raman spectra to a low-dimensional space where a smaller number of variables defines each sample. The appropriate number of reduced features used was obtained by comparing the mean accuracy from a 10-fold cross-validation. Finally, we employed Gaussian process (GP) classification, a probabilistic machine learning approach, to correctly predict the occurrence of a negative or positive sample as a function of the low-dimensional space variables. As opposed to providing rigid class labels, the GP classifier provides a probability (ranging from zero to one) that a given sample is positive or negative. In practice, the proposed framework can be used to provide high-throughput rapid testing, and a follow-up PCR can be used for confirmation in cases where the model’s uncertainty is unacceptably high.

Imitation learning for coordinated human–robot collaboration based on hidden state-space models

This paper proposes a novel coordinated human–robot collaboration framework based on the hidden state-space model, which probabilistically clones the human behaviour and presents dynamic features in a nonparametric form. Derived from the filter prediction techniques and the theory of exact moment matching, this framework could provide an analytical approximation of the posterior distribution, and hence infer the hidden state variables of the collaborative robot given the external observation and its uncertainties. Not akin to the other cutting-edge movement-primitive based algorithms or coordinated human–robot collaboration methods, our collaboration framework not only preserves the adaptation functionalities of imitation learning but also propagates state variables and their uncertainties during real-time coordinated implementation. By leveraging on the binary Gaussian process classification, additional functionality, such as multiple task recognition is proposed to enhance the generalisation capability of our framework. The application feasibility is verified from both theoretical comparison simulation and real-world experiments.

Improving the efficiency of machine learning models for predicting blood glucose levels and diabetes risk

Fasting <span>blood glucose is used as an indicator in the process of predicting diabetes risk. This research aims to, i) create a model for predicting blood glucose level using data mining algorithms, ii) a selection algorithm was used to select a feature from the correlation of the data, and iii) to compare the model's performance with the classical methods. All clinical data ware recorded and compiled in a database by hospital staff from 2014-2019. In our previous research, the blood glucose prediction model had an acceptable accuracy where 18 patient features were used as input data to the data mining process. In this research, we demonstrated that the random forest classifier and extra tree classifier algorithms have an outstanding in discarding non-critical attributes. And the process of reducing the number of those features has impacted the glycemic prediction model with higher efficiency. Seventeen machine learning algorithms are used to find the best performance models. Our results clearly show that the improved prediction model is more efficient. This experiment has shown that improvements to our proposed model were able to predict blood glucose levels with 99.69% and 99.63% accuracy for random forest classifier, extra tree classifier, and Gaussian process classifier, respectively.</span>

Comparison of breast cancer classification models on Wisconsin dataset

Breast cancer is the leading cause of death for women worldwide. Cancer can be discovered early, lowering the rate of death. Machine learning techniques are a hot field of research, and they have been shown to be helpful in cancer prediction and early detection. The primary purpose of this research is to identify which machine learning algorithms are the most successful in predicting and diagnosing breast cancer, according to five criteria: specificity, sensitivity, precision, accuracy, and F1 score. The project is finished in the Anaconda environment, which uses Python's NumPy and SciPy numerical and scientific libraries as well as matplotlib and Pandas. In this study, the Wisconsin diagnostic breast cancer dataset was used to evaluate eleven machine learning classifiers: decision tree, quadratic discriminant analysis, AdaBoost, Bagging meta estimator, Extra randomized trees, Gaussian process classifier, Ridge, Gaussian nave Bayes, k-Nearest neighbors, multilayer perceptron, and support vector classifier. During performance analysis, extremely randomized trees outperformed all other classifiers with an F1-score of 96.77% after data collection and data analysis.

Optimization strategy for coupled battery system design models using Gaussian Process Regression and Classification

In the development of battery systems for electric vehicles (EV), numerous components from different physical subareas must be harmonized with each other. Automotive engineers already use extensive simulation models to optimize individual components satisfying increasing demands of EV range and power. Yet, strategies for the combined optimization of battery systems have not been addressed in the literature. This work presents an optimization strategy for the holistic design of battery systems, which utilizes coupled simulations of technical submodels representing cellmodules, mechanics, cooling, and electronics. Given user-specified battery system requirements, methods of Gaussian Process Regression and Classification are combined to determine the optimal battery system design in terms of costs and feasibility. An inherited mixed-integer problem is addressed by using discretization of the solution space and refinement strategies in likely optimal regions. Moreover, the information gain per iteration is maximized by means of predictive calculations and parallelization methods. Testing the presented optimization strategy in different scenarios gives promising results, showcasing its robustness towards different technical requirements for battery systems. Also, exemplary analyses regarding the impact of the total installation space on costs and feasibility are conducted.

Combined Intrinsic Local Functional Connectivity With Multivariate Pattern Analysis to Identify Depressed Essential Tremor.

BackgroundAlthough depression is one of the most common neuropsychiatric symptoms in essential tremor (ET), the diagnosis biomarker and intrinsic brain activity remain unclear. We aimed to combine multivariate pattern analysis (MVPA) with local brain functional connectivity to identify depressed ET.MethodsBased on individual voxel-level local brain functional connectivity (regional homogeneity, ReHo) mapping from 41 depressed ET, 43 non-depressed ET, and 45 healthy controls (HCs), the binary support vector machine (BSVM) and multiclass Gaussian Process Classification (MGPC) algorithms were used to identify depressed ET patients from non-depressed ET and HCs, the accuracy and permutations test were used to assess the classification performance.ResultsThe MGPC algorithm was able to classify the three groups (depressed ET, non-depressed ET, and HCs) with a total accuracy of 84.5%. The BSVM algorithm achieved a better classification performance with total accuracy of 90.7, 88.64, and 90.48% for depressed ET vs. HCs, non-depressed ET vs. HCs, and depressed ET vs. non-depressed ET, and the sensitivity for them at 80.49, 76.64, and 80.49%, respectively. The significant discriminative features of depressed ET vs. HCs were primarily located in the cerebellar-motor-prefrontal gyrus-anterior cingulate cortex pathway, and for depressed ET vs. non-depressed ET located in the cerebellar-prefrontal gyrus-anterior cingulate cortex circuits. The partial correlation showed that the ReHo values in the bilateral middle prefrontal gyrus (positive) and the bilateral cerebellum XI (negative) were significantly correlated with clinical depression severity.ConclusionOur findings suggested that combined individual ReHo maps with MVPA not only could be used to identify depressed ET but also help to reveal the intrinsic brain activity changes and further act as the potential diagnosis biomarker in depressed ET patients.

A Holistic Strategy for Classification of Sleep Stages with EEG.

Manual sleep stage scoring is usually implemented with the help of sleep specialists by means of visual inspection of the neurophysiological signals of the patient. As it is a very hectic task to perform, automated sleep stage classification systems were developed in the past, and advancements are being made consistently by researchers. The various stages of sleep are identified by these automated sleep stage classification systems, and it is quite an important step to assist doctors for the diagnosis of sleep-related disorders. In this work, a holistic strategy named as clustering and dimensionality reduction with feature extraction cum selection for classification along with deep learning (CDFCD) is proposed for the classification of sleep stages with EEG signals. Though the methodology follows a similar structural flow as proposed in the past works, many advanced and novel techniques are proposed under each category in this work flow. Initially, clustering is applied with the help of hierarchical clustering, spectral clustering, and the proposed principal component analysis (PCA)-based subspace clustering. Then the dimensionality of it is reduced with the help of the proposed singular value decomposition (SVD)-based spectral algorithm and the standard variational Bayesian matrix factorization (VBMF) technique. Then the features are extracted and selected with the two novel proposed techniques, such as the sparse group lasso technique with dual-level implementation (SGL-DLI) and the ridge regression technique with limiting weight scheme (RR-LWS). Finally, the classification happens with the less explored multiclass Gaussian process classification (MGC), the proposed random arbitrary collective classification (RACC), and the deep learning technique using long short-term memory (LSTM) along with other conventional machine learning techniques. This methodology is validated on the sleep EDF database, and the results obtained with this methodology have surpassed the results of the previous studies in terms of the obtained classification accuracy reporting a high accuracy of 93.51% even for the six-classes classification problem.

Artificial intelligence model for prediction of cardiogenic shock in patients with acute coronary syndrome

Abstract Funding Acknowledgements Type of funding sources: Public grant(s) – National budget only. Main funding source(s): Ministry of Education, Science, Research and Sport of the Slovak Republic VEGA Background Cardiogenic shock (CS) is a serious life-threatening condition affecting almost 10% of patients suffering from acute coronary syndrome (ACS). Despite recent treatment improvements such as mechanical circulatory support devices (MCS), mortality remains 50%. It is hypothesized that early implantation of MCS before hemodynamic deterioration could prevent CS. Therefore, knowing which ACS patient will progress into CS would be of paramount importance. Purpose We aimed to develop a model based on machine learning algorithms for CS prediction in patients with ACS. Method Over 40 000 patients from critical care units of the Beth Israel Deaconess Medical Center database were extensively analyzed and sorted. Patients suffering from acute coronary syndrome undergoing cardiac catheterizations were selected and divided into two groups based on the development of CS. Patients in CS at the time of admission were excluded. The study population consisted of 3056 patients who didn’t develop CS and of 176 patients who did develop CS. Important information was also extracted manually from textual summaries of hospital stays. Potentially relevant clinical variables for shock prediction were selected using supervised feature selection, and missing values were supplemented using imputation methods. Seven well-known and established machine learning algorithms were used. Based on preliminary evaluation of classifier performance (AUC on random train-test split with 30% test data), we selected two best-performing algorithms: Logistic Regression and Gaussian Process classifier with Radial Basis Function (RBF) kernel. Both models were subsequently validated using Repeated Stratified K-Fold cross-validation with 5 folds and 20 repeats each. Results Age, heart rate, mean arterial pressure, respiratory rate, oxygen flow (liters per minute), peripheral oxygen saturation, blood glucose, pain type, heart rhythm and ectopy frequency were chosen as input variables. Both models showed good discrimination. The Logistic Regression model scored AUC of 0.76 ± 0.04. The Gaussian Process classifier with Radial Basis Function (RBF) kernel scored AUC of 0.77 ± 0.03. Conclusion According to our knowledge this is the first study that uses machine learning algorithms to predict CS in patients with ACS based on easily obtainable clinical variables. Further explorations and refinement of used imputation, feature selection, machine learning techniques and validation on an external cohort might result in even better performance of the proposed prediction models. Furthermore, these prediction models could be transformed into a simple predictive scoring system available in clinical practice.

Gaussian process regression model for the prediction of the compressive strength of polyurethane-based polymer concrete for runway repair: A comparative approach

Polyurethane (PU) composites have increasingly been used as construction materials to maintain civil engineering structures such as road pavement, runway, parking area, and floor systems in buildings. This study developed polyurethane polymer concrete (PC) mixtures by mixing aggregate-to-PU resin at 0.9: 0.1 and 0.85: 0.15 ratios by weight. The Machine Learning algorithms, including Gaussian Process Regression (GPR), Classification and Regression Tree (CART), and Support Vector Regression (SVR) model were employed to predict the compressive strength of PUPC mixtures as a repair material. The models were trained on the dataset of flexural strength (MPa), density (kg/m3), and PU composition (%), applied as input variables. The result revealed that the compressive stress-strain curves of PU-based polymer concrete exhibit linear elastic behavior under compression. The developed models demonstrate high prediction accuracy of PUPC’ strength. The Nash-Sutcliffe efficiency (NSE) was used to check the performance of each model, and the result obtained showed that the GPR model predicted the compressive strength with the highest accuracy with an NSE-values of 0.9619 and 0.9585 at the training and testing phase, respectively. The finding in this study could offer valuable insight into using these proposed models for compressive strength prediction of PU-based polymer concrete

Evaluation of Machine Learning Models for Breast Cancer Diagnosis Via Histogram of Oriented Gradients Method and Histopathology Images

Breast cancer is the main death rate from malignant growth worldwide and the most frequently diagnosed type of cancer in females. Machine learning systems have been developed to assist in the accurate detection of cancer. There are numerous methods for cancer detection. But histopathological images are thought to be more precise. In this study, we used the HOG features extractor to extract statistical features from histopathology images of invasive ductal carcinoma. We chose the following images at random from the histopathology images: 100, 200, 400, 1000, and 2000. These statistical features were then used to train several algorithms, including the decision tree, quadratic discriminant analysis, extra randomized trees, gradient boosting, gaussian process classifier, naive bayes, nearest centroid, multilayer perceptron, and support vector machine, to identify whether or not the images depict cancerous or noncancerous growth. The algorithms' performance was evaluated depending on the specificity, accuracy, sensitivity, precision, F1_score, and AUC. The algorithms used worked best when the number of images was set to 100. As the number of images went up, their effectiveness went down.

A chemiresistive sensor array based on polyaniline nanocomposites and machine learning classification.

The selective detection of ammonia (NH3), nitrogen dioxide (NO2), carbon oxides (CO2 and CO), acetone ((CH3)2CO), and toluene (C6H5CH3) is investigated by means of a gas sensor array based on polyaniline nanocomposites. The array composed by seven different conductive sensors with composite sensing layers are measured and analyzed using machine learning. Statistical tools, such as principal component analysis and linear discriminant analysis, are used as dimensionality reduction methods. Five different classification methods, namely k-nearest neighbors algorithm, support vector machine, random forest, decision tree classifier, and Gaussian process classification (GPC) are compared to evaluate the accuracy of target gas determination. We found the Gaussian process classification model trained on features extracted from the data by principal component analysis to be a highly accurate method reach to 99% of the classification of six different gases.

Predicting Durable Responses to Immune Checkpoint Inhibitors in Non-Small-Cell Lung Cancer Using a Multi-Feature Model.

Due to the complex mechanisms affecting anti-tumor immune response, a single biomarker is insufficient to identify patients who will benefit from immune checkpoint inhibitors (ICIs) treatment. Therefore, a comprehensive predictive model is urgently required to predict the response to ICIs. A total of 162 non-small-cell lung cancer (NSCLC) patients undergoing ICIs treatment from three independent cohorts were enrolled and used as training and test cohorts (training cohort = 69, test cohort1 = 72, test cohort2 = 21). Eight genomic markers were extracted or calculated for each patient. Ten machine learning classifiers, such as the gaussian process classifier, random forest, and support vector machine (SVM), were evaluated. Three genomic biomarkers, namely tumor mutation burden, intratumoral heterogeneity, and loss of heterozygosity in human leukocyte antigen were screened out, and the SVM_poly method was adopted to construct a durable clinical benefit (DCB) prediction model. Compared with a single biomarker, the DCB multi-feature model exhibits better predictive value with the area under the curve values equal to 0.77 and 0.78 for test cohort1 and cohort2, respectively. The patients predicted to have DCB showed improved median progression-free survival (mPFS) and median overall survival (mOS) than those predicted to have non-durable clinical benefit.