Gaussian Process Learning Research Articles

Resonant Frequency Modeling of Microwave Antennas Using Gaussian Process Based on Semisupervised Learning

For the optimal design of electromagnetic devices, it is the most time consuming to obtain the training samples from full wave electromagnetic simulation software, including HFSS, CST, and IE3D. Traditional machine learning methods usually use only labeled samples or unlabeled samples, but in practical problems, labeled samples and unlabeled samples coexist, and the acquisition cost of labeled samples is relatively high. This paper proposes a semisupervised learning Gaussian Process (GP), which combines unlabeled samples to improve the accuracy of the GP model and reduce the number of labeled training samples required. The proposed GP model consists two parts: initial training and self-training. In the process of initial training, a small number of labeled samples obtained by full wave electromagnetic simulation are used for training the initial GP model. Afterwards, the trained GP model is copied to another GP model in the process of self-training, and then the two GP models will update after crosstraining with different unlabeled samples. Using the same test samples for testing and updating, a model with a smaller error will replace another. Repeat the self-training process until a predefined stopping criterion is met. Four different benchmark functions and resonant frequency modeling problems of three different microstrip antennas are used to evaluate the effectiveness of the GP model. The results show that the proposed GP model has a good fitting effectiveness on benchmark functions. For microstrip antennas resonant frequency modeling problems, in the case of using the same labeled samples, its predictive ability is better than that of the traditional supervised GP model.

Ensemble Just-In-Time Learning-Based Soft Sensor for Mooney Viscosity Prediction in an Industrial Rubber Mixing Process

The lack of online sensors for Mooney viscosity measurement has posed significant challenges for enabling efficient monitoring, control, and optimization of industrial rubber mixing process. To obtain real-time and accurate estimations of Mooney viscosity, a novel soft sensor method, referred to as multimodal perturbation- (MP-) based ensemble just-in-time learning Gaussian process regression (MP-EJITGPR), is proposed by exploiting ensemble JIT learning. This method employs perturbations on similarity measure and input variables for generating the diversity of JIT learners. Furthermore, a set of accurate and diverse JIT learners are built through an evolutionary multiobjective optimization by balancing the accuracy and diversity objectives explicitly. Moreover, all base JIT learners are combined adaptively using a finite mixture mechanism. The proposed method is applied to an industrial rubber mixing process for Mooney viscosity prediction, and the experimental results demonstrate its effectiveness and superiority over traditional soft sensor methods.

An Early Classification Approach for Multivariate Time Series of On-Vehicle Sensors in Transportation

An important issue of research in the transportation system is timely classification of the inside-outside environment of the vehicle using sensors. The sensors generate the multivariate time series data, which requires a classification technique to classify it in real-time. Road surface classification is an example, where multivariate time series data can be used for early identification of the type of road surface. The challenge is to maintain the accuracy of the classification using a minimum number of data points of the multivariate time series. This work proposes an early classification approach for multivariate time series with a desired level of accuracy. It is assumed that the number of samples in the time series are not equal for a given period of time due to different type of sensors. Gaussian Process learning method is used to first estimate the minimum required length of the time series which helps to build an ensemble classifier with a desired level of accuracy. The ensemble classifier is used to predict the class label of an incoming multivariate time series. This work demonstrates a road surface classification system using the built ensemble classifier. Finally, the ensemble classifier is also evaluated on the various existing datasets from other domains. The results demonstrate the significance of early classification approach using accuracy, earliness, and confusion matrix, with the minimum required data points.

Constrained Gaussian Process Learning for Model Predictive Control

Many control tasks can be formulated as tracking problems of a known or unknown reference signal. examples are motion compensation in collaborative robotics, the synchronisation of oscillations for power systems or the reference tracking of recipes in chemical process operation. Both the tracking performance and the stability of the closed-loop system depend strongly on two factors: Firstly, they depend on whether the future reference signal required for tracking is known, and secondly, whether the system can track the reference at all. This paper shows how to use machine learning, i.e. Gaussian processes, to learn a reference from (noisy) data while guaranteeing trackability of the modified desired reference predictions within the framework of model predictive control. Guarantees are provided by adjusting the hyperparameters via a constrained optimisation. Two specific scenarios, i.e. asymptotically constant and periodic REFERENCES, are discussed.

Guest Editorial: Medpower 2018 Selected Papers

Guest Editorial: Medpower 2018 Selected Papers

Application of fuzzy multiplexing of learning Gaussian processes for the interval forecasting of wind speed

Robust forecasting of wind speed values is a key element to effectively accommodate renewable generation from wind in smart power systems. However, the stochastic nature of wind and the uncertainties associated with it impose high challenge in its forecasting. A new method for forecasting wind speed in renewable energy generation is introduced in this study. The goal of the method is to provide a forecast in the form of an interval, which is determined by a mean value and the variance around the mean. In particular, the forecasting interval is produced according to a two-step process: in the first step, a set of individual kernel modelled Gaussian processes (GP) are utilised to provide a respective set of interval forecasts, i.e. mean and variance values, over the future values of the wind. In the second step, the individual forecasts are evaluated using a fuzzy driven multiplexer, which selects one of them. The final output of the methodology is a single interval that has been identified as the best among the GP models. The presented methodology is tested on the set of real-world data and benchmarked against the individual GPs as well as the autoregressive moving average model.

A Gaussian process-based definition reveals new and bona fide genetic interactions compared to a multiplicative model in the Gram-negative Escherichia coli.

A digenic genetic interaction (GI) is observed when mutations in two genes within the same organism yield a phenotype that is different from the expected, given each mutation's individual effects. While multiplicative scoring is widely applied to define GIs, revealing underlying gene functions, it remains unclear if it is the most suitable choice for scoring GIs in Escherichia coli. Here, we assess many different definitions, including the multiplicative model, for mapping functional links between genes and pathways in E.coli. Using our published E.coli GI datasets, we show computationally that a machine learning Gaussian process (GP)-based definition better identifies functional associations among genes than a multiplicative model, which we have experimentally confirmed on a set of gene pairs. Overall, the GP definition improves the detection of GIs, biological reasoning of epistatic connectivity, as well as the quality of GI maps in E.coli, and, potentially, other microbes. The source code and parameters used to generate the machine learning models in WEKA software were provided in the Supplementary information. Supplementary data are available at Bioinformatics online.

Collective Online Learning of Gaussian Processes in Massive Multi-Agent Systems

This paper presents a novel Collective Online Learning of Gaussian Processes (COOL-GP) framework for enabling a massive number of GP inference agents to simultaneously perform (a) efficient online updates of their GP models using their local streaming data with varying correlation structures and (b) decentralized fusion of their resulting online GP models with different learned hyperparameter settings and inducing inputs. To realize this, we exploit the notion of a common encoding structure to encapsulate the local streaming data gathered by any GP inference agent into summary statistics based on our proposed representation, which is amenable to both an efficient online update via an importance sampling trick as well as multi-agent model fusion via decentralized message passing that can exploit sparse connectivity among agents for improving efficiency and enhance the robustness of our framework against transmission loss. We provide a rigorous theoretical analysis of the approximation loss arising from our proposed representation to achieve efficient online updates and model fusion. Empirical evaluations show that COOL-GP is highly effective in model fusion, resilient to information disparity between agents, robust to transmission loss, and can scale to thousands of agents.

Learning of Gaussian Processes in Distributed and Communication Limited Systems.

It is of fundamental importance to find algorithms obtaining optimal performance for learning of statistical models in distributed and communication limited systems. Aiming at characterizing the optimal strategies, we consider learning of Gaussian Processes (GP) in distributed systems as a pivotal example. We first address a very basic problem: how many bits are required to estimate the inner-products of some Gaussian vectors across distributed machines? Using information theoretic bounds, we obtain an optimal solution for the problem which is based on vector quantization. Two suboptimal and more practical schemes are also presented as substitutes for the vector quantization scheme. In particular, it is shown that the performance of one of the practical schemes which is called per-symbol quantization is very close to the optimal one. Schemes provided for the inner-product calculations are incorporated into our proposed distributed learning methods for GPs. Experimental results show that with spending few bits per symbol in our communication scheme, our proposed methods outperform previous zero rate distributed GP learning schemes such as Bayesian Committee Model (BCM) and Product of experts (PoE).

Robotic Active Information Gathering for Spatial Field Reconstruction with Rapidly-Exploring Random Trees and Online Learning of Gaussian Processes.

Information gathering (IG) algorithms aim to intelligently select a mobile sensor actions required to efficiently obtain an accurate reconstruction of a physical process, such as an occupancy map, or a magnetic field. Many recent works have proposed algorithms for IG that employ Gaussian processes (GPs) as underlying model of the process. However, most algorithms discretize the state space, which makes them computationally intractable for robotic systems with complex dynamics. Moreover, they are not suited for online information gathering tasks as they assume prior knowledge about GP parameters. This paper presents a novel approach that tackles the two aforementioned issues. Specifically, our approach includes two intertwined steps: (i) a Rapidly-Exploring Random Tree (RRT) search that allows a robot to identify unvisited locations, and to learn the GP parameters, and (ii) an RRT*-based informative path planning that guides the robot towards those locations by maximizing the information gathered while minimizing path cost. The combination of the two steps allows an online realization of the algorithm, while eliminating the need for discretization. We demonstrate that our proposed algorithm outperforms state-of-the-art both in simulations, and in a lab experiment in which a ground-based robot explores the magnetic field intensity within an indoor environment populated with obstacles.

Remote Sensing of Water Quality Parameters over Lake Balaton by Using Sentinel-3 OLCI

The Ocean and Land Color Instrument (OLCI) onboard Sentinel 3A satellite was launched in February 2016. Level 2 (L2) products have been available for the public since July 2017. OLCI provides the possibility to monitor aquatic environments on 300 m spatial resolution on 9 spectral bands, which allows to retrieve detailed information about the water quality of various type of waters. It has only been a short time since L2 data became accessible, therefore validation of these products from different aquatic environments are required. In this work we study the possibility to use S3 OLCI L2 products to monitor an optically highly complex shallow lake. We test S3 OLCI-derived Chlorophyll-a (Chl-a), Colored Dissolved Organic Matter (CDOM) and Total Suspended Matter (TSM) for complex waters against in situ measurements over Lake Balaton in 2017. In addition, we tested the machine learning Gaussian process regression model, trained locally as a potential candidate to retrieve water quality parameters. We applied the automatic model selection algorithm to select the combination and number of spectral bands for the given water quality parameter to train the Gaussian Process Regression model. Lake Balaton represents different types of aquatic environments (eutrophic, mesotrophic and oligotrophic), hence being able to establish a model to monitor water quality by using S3 OLCI products might allow the generalization of the methodology.

Gaussian Processes for Learning and Control: A Tutorial with Examples

Many challenging real-world control problems require adaptation and learning in the presence of uncertainty. Examples of these challenging domains include aircraft adaptive control under uncertain disturbances [1], [2], multiple-vehicle tracking with space-dependent uncertain dynamics [3], [4], robotic-arm control [5], blimp control [6], [7], mobile robot tracking and localization [8], [9], cart-pole systems and unicycle control [10], gait optimization in legged robots [11] and snake robots [12], and any other system whose dynamics are uncertain and for which limited data are available for model learning. Classical model reference adaptive control (MRAC) [13]-[15] and reinforcement learning (RL) methods [16]-[23] have been developed to address these challenges and rely on parametric adaptive elements or control policies whose number of parameters or features are fixed and determined a priori. One example of such an adaptive model are radial basis function networks (RBFNs), with RBF centers pre-allocated based on expected operating domains [24], [25].

Putting bandits into context: How function learning supports decision making.

The authors introduce the contextual multi-armed bandit task as a framework to investigate learning and decision making in uncertain environments. In this novel paradigm, participants repeatedly choose between multiple options in order to maximize their rewards. The options are described by a number of contextual features which are predictive of the rewards through initially unknown functions. From their experience with choosing options and observing the consequences of their decisions, participants can learn about the functional relation between contexts and rewards and improve their decision strategy over time. In three experiments, the authors explore participants' behavior in such learning environments. They predict participants' behavior by context-blind (mean-tracking, Kalman filter) and contextual (Gaussian process and linear regression) learning approaches combined with different choice strategies. Participants are mostly able to learn about the context-reward functions and their behavior is best described by a Gaussian process learning strategy which generalizes previous experience to similar instances. In a relatively simple task with binary features, they seem to combine this learning with a probability of improvement decision strategy which focuses on alternatives that are expected to lead to an improvement upon a current favorite option. In a task with continuous features that are linearly related to the rewards, participants seem to more explicitly balance exploration and exploitation. Finally, in a difficult learning environment where the relation between features and rewards is nonlinear, some participants are again well-described by a Gaussian process learning strategy, whereas others revert to context-blind strategies. (PsycINFO Database Record

Stability of Gaussian Process Learning Based Output Feedback Model Predictive Control

Abstract We present an output feedback nonlinear model predictive control approach that uses a Gaussian process model for prediction. We show nominal stability assuming that the Gaussian process model is able to represent the real process and establish input-to-state stability assuming a bounded error between the real process and the Gaussian model approximation. These results are achieved using a predictive control formulation without terminal region. The approach is illustrated using a continuous stirred-tank reactor benchmark problem.

Near-Optimal Active Learning of Multi-Output Gaussian Processes

This paper addresses the problem of active learning of a multi-output Gaussian process (MOGP) model representing multiple types of coexisting correlated environmental phenomena. In contrast to existing works, our active learning problem involves selecting not just the most informative sampling locations to be observed but also the types of measurements at each selected location for minimizing the predictive uncertainty (i.e., posterior joint entropy) of a target phenomenon of interest given a sampling budget. Unfortunately, such an entropy criterion scales poorly in the numbers of candidate sampling locations and selected observations when optimized. To resolve this issue, we first exploit a structure common to sparse MOGP models for deriving a novel active learning criterion. Then, we exploit a relaxed form of submodularity property of our new criterion for devising a polynomial-time approximation algorithm that guarantees a constant-factor approximation of that achieved by the optimal set of selected observations. Empirical evaluation on real-world datasets shows that our proposed approach outperforms existing algorithms for active learning of MOGP and single-output GP models.

Bayesian supervised learning and state estimation in a model of the cerebellum

The last decade has seen an explosion of research on the topic of Bayesian inference in neural systems. The cerebral cortex is now widely believed to form predictive internals models of the environment. But what does this mean for the cerebellum, which has historically been attributed just this function? And what might Bayesian inference mean in the cerebellar context? I present a cerebellar model unifying the Bayesian framework with classical cerebellar theories of supervised learning, state estimation, and control theory. This model builds on the dominant supervised learning theory of cerebellar function, which traces to the pioneering work of Marr. This model is extended to account for Bayesian optimal updating of the uncertainties inherent in probabilistic inference. Further, the model is capable of learning about states with little to no supervised input. It also provides theoretical explanation for a wide range of phenomenology observed of the circuit -- most importantly unsupervised synaptic plasticity upstream of cerebellar Purkinje cells. The connectionist architecture of this model is closely based on that of the cerebellum. In the input stage, unsupervised learning in the cerebellar granular layer is seen as essential for subsequent supervised learning in the Purkinje layer. Based on Golgi cell feedback, the granular layer learns to encode supervised learning covariances in a manner akin to the kernel learning of Gaussian processes. This circuit then predicts and filters mossy fiber input, conveying only select signals downstream for supervised learning. This granular representation is further influenced by a global serotonergic signal conveyed by Lugaro cells. In the cerebellar molecular layer, inhibitory interneurons then provide higher-order corrections to these estimates. Finally, learning in the cerebellar nuclei is seen as a parallel, simpler, regression model, allowing for faster, courser adjustments, corrected in time by the cerebellar cortical circuit.

Gaussian process for predicting CPU utilization and its application to energy efficiency

For the past ten years, Gaussian process has become increasingly popular for modeling numerous inferences and reasoning solutions due to the robustness and dynamic features. Particularly concerning regression and classification data, the combination of Gaussian process and Bayesian learning is considered to be one of the most appropriate supervised learning approaches in terms of accuracy and tractability. However, due to the high complexity in computation and data storage, Gaussian process performs poorly when processing large input dataset. Because of the limitation, this method is ill-equipped to deal with the large-scale system that requires reasonable precision and fast reaction rate. To improve the drawback, our research focuses on a comprehensive analysis of Gaussian process performance issues, highlighting ways to drastically reduce the complexity of hyper-parameter learning and training phases, which could be applicable in predicting the CPU utilization in the demonstrated application. In fact, the purpose of this application is to save the energy by distributively engaging the Gaussian process regression to monitor and predict the status of each computing node. Subsequently, a migration mechanism is applied to migrate the system-level processes between multi-core and turn off the idle one in order to reduce the power consumption while still maintaining the overall performance.

Incremental motion learning with locally modulated dynamical systems

Dynamical Systems (DS) for robot motion modeling are a promising approach for efficient robot learning and control. Our focus in this paper is on autonomous dynamical systems, which represent a motion plan without dependency on time. We develop a method that allows to locally reshape an existing, stable nonlinear autonomous DS while preserving important stability properties of the original system. Our system is based on local transformations of the dynamics. We propose an incremental learning algorithm based on Gaussian Processes for learning to reshape dynamical systems using this representation. The approach is validated in a 2d task of learning handwriting motions, a periodic polishing motion and in a manipulation task with the 7 degrees of freedom Barrett WAM manipulator.

Probabilistic electricity price forecasting with variational heteroscedastic Gaussian process and active learning

Electricity price forecasting is essential for the market participants in their decision making. Nevertheless, the accuracy of such forecasting cannot be guaranteed due to the high variability of the price data. For this reason, in many cases, rather than merely point forecasting results, market participants are more interested in the probabilistic price forecasting results, i.e., the prediction intervals of the electricity price. Focusing on this issue, this paper proposes a new model for the probabilistic electricity price forecasting. This model is based on the active learning technique and the variational heteroscedastic Gaussian process (VHGP). It provides the heteroscedastic Gaussian prediction intervals, which effectively quantify the heteroscedastic uncertainties associated with the price data. Because the high computational effort of VHGP hinders its application to the large-scale electricity price forecasting tasks, we design an active learning algorithm to select a most informative training subset from the whole available training set. By constructing the forecasting model on this smaller subset, the computational efforts can be significantly reduced. In this way, the practical applicability of the proposed model is enhanced. The forecasting performance and the computational time of the proposed model are evaluated using the real-world electricity price data, which is obtained from the ANEM, PJM, and New England ISO.

A New Method for Forecasting of Non-Point Source Pollutant Load

Gaussian process (GP) is a newly developed machine learning technology based on statistical theoretical fundamentals, which has successful application in the field of solving for highly nonlinear problems. Conventional methods for forecasting of non-point source pollutant load often meet great difficulty since relationship between pollutant load and its influencing factors is highly complicated nonlinear. A new method based on GP is proposed for forecasting of non-point source pollutant load. The monitoring data of a certain river since 1976 to 1990 are preformed to obtain the training samples and test samples. Nonlinear mapping relationship between non-point source pollutant load and its influencing factors can be constructed by GP learning with the training samples. The monitoring data of a certain river since 1991 to 1993 are preformed to testify the effects of the method above. The results of case studies show that the method is feasible, effective and simple to implement for forecasting of non-point source pollutant load. It has merits of self-adaptive parameters determination and better capacity for solving nonlinear small sample problems comparing with the artificial neural networks method and Support Vector Machine method. The good performance of GP model makes it very attractive for a wide range of application in environmental engineering.