Local Gaussian Process Regression Research Articles

A Local Gaussian Process Regression Approach to Mass Appraisal of Residential Properties

A Local Gaussian Process Regression Approach to Mass Appraisal of Residential Properties

Active sparse Bayesian committee machine potential for isothermal-isobaric molecular dynamics simulations.

Recent advancements in machine learning potentials (MLPs) have significantly impacted the fields of chemistry, physics, and biology by enabling large-scale first-principles simulations. Among different machine learning approaches, kernel-based MLPs distinguish themselves through their ability to handle small datasets, quantify uncertainties, and minimize over-fitting. Nevertheless, their extensive computational requirements present considerable challenges. To alleviate these, sparsification methods have been developed, aiming to reduce computational scaling without compromising accuracy. In the context of isothermal and isobaric ML molecular dynamics (MD) simulations, achieving precise pressure estimation is crucial for reproducing reliable system behavior under constant pressure. Despite progress, sparse kernel MLPs struggle with precise pressure prediction. Here, we introduce a virial kernel function that significantly enhances the pressure estimation accuracy of MLPs. Additionally, we propose the active sparse Bayesian committee machine (BCM) potential, an on-the-fly MLP architecture that aggregates local sparse Gaussian process regression (SGPR) MLPs. The sparse BCM potential overcomes the steep computational scaling with the kernel size, and a predefined restriction on the size of kernel allows for fast and efficient on-the-fly training. Our advancements facilitate accurate and computationally efficient machine learning-enhanced MD (MLMD) simulations across diverse systems, including ice-liquid coexisting phases, Li10Ge(PS6)2 lithium solid electrolyte, and high-pressure liquid boron nitride.

Nonparametric Modeling and Control of Ship Steering Motion Based on Local Gaussian Process Regression

This paper aims to study the nonparametric modeling and control of ship steering motion. Firstly, the black box response model is derived based on the Nomoto model. Then, the establishment of a nonparametric response model and prediction of ship steering motion are realized by applying the local Gaussian process regression (LGPR) algorithm. To assess the performance of LGPR, two cases are studied, including a Mariner class vessel by using simulation data and a KVLCC2 tanker model by using experimental data. The results reveal that the response model identified by LGPR presents good prediction accuracy and low computational burden. Finally, the identified response model is used as the basis for developing the ship heading controller, and the results demonstrate that the proposed controller is able to achieve good dynamic performance.

Latin hypercubes for constrained design of experiments for data-driven models

Abstract The quality of data used for data-driven modeling affects the model performance significantly. Thus, design of experiments (DoE) is an important part during model development. The design space is constrained in many applications. In this work, the constrained case is investigated. An Latin hypercube based approach is applied and analyzed for strongly constrained design spaces. Contrary to commonly used optimization techniques, an incremental procedure is proposed. In every step, new data are added to the design. Each new point is selected by a distance-based criterion. The performance of the created designs is evaluated by the quality of the trained models. For different constraints, artificial data sets are created with a function generator. The performance of local model networks and Gaussian process regression models trained with those designs is evaluated and compared to models trained on data sets based on Sobol’ sequences.

An efficient two‐stage algorithm for parameter identification of non‐linear state‐space models‐based on Gaussian process regression

AbstractThis paper aims to improve the efficiency of parameter identification of the nonlinear state‐space model (SSM). The commonly used particle Markov chain Monte Carlo (PMCMC) method is time‐consuming. The surrogate model is a useful acceleration strategy, but it is expensive to establish a global high‐precision surrogate model. This paper proposes an efficient algorithm that gradually estimates the unknown parameters in two stages. In the first stage, a reduced region is established based on the latest method. We train a local Gaussian Process regression (GPR) of the likelihood function in the reduced region based on the optimum Latin hypercube design (OLHD). In the second stage, we identify the unknown parameters more accurately based on the MCMC method. When the proposal sample is in the reduced region, we use GPR to estimate the likelihood; otherwise, BPF is used to estimate the likelihood. The reduced region is usually the high probability density region, which is why the algorithm is efficient. It is proved that the acceptance rate of any two samples based on the proposed algorithm is theoretically convergent to that of the PMCMC algorithm. Two examples demonstrate that the proposed method performs well in accuracy and efficiency.

Local Gaussian process regression with small sample data for temperature and humidity compensation of polyaniline-cerium dioxide NH3 sensor

Gas sensors have made great progress in gas sensing performances (such as response, sensitivity, detection limit and response speed), but they are generally affected by temperature and humidity. Here, we proposed a local Gaussian process regression with small samples for temperature and humidity compensation of gas sensors. Specifically, the above method is used to compensate the temperature and humidity influence of the polyaniline-cerium dioxide (PANI-CeO2) ammonia (NH3) sensor (10–50 °C, 20%−70% relative humidity (RH); It should be noted that the law of humidity influence is messy when the RH is greater than 70%.). The adaptive matching results show that the optimal number of K-neighbor points is 15, which greatly reduces the amount of computation. The temperature and humidity compensation results show that the predicted concentration achieved high accuracy (the mean absolute error is 0.19 ppm, and the mean relative error is 0.65%), and the absolute error showed a good normal distribution (N∼(0.00047, 0.3772)). This work provides an effective compensation strategy with small samples, high precision and low computational cost for the temperature and humidity influences of gas sensor.

Identification modeling of ship maneuvering motion based on local Gaussian process regression

A fast and accurate nonparametric modeling method based on local Gaussian process regression (LGPR) is proposed for the identification modeling and prediction of ship maneuvering motion. The training dataset collected from the free-running model tests of ship maneuvering is automatically divided into a number of clusters according to the similarity criterion by clustering analysis using k-means algorithm. Utilizing the data in each cluster, the corresponding local nonparametric model is identified. The computational cost of training and prediction based on LGPR is reduced compared to that based on the classic Gaussian process regression (CGPR) using the whole training dataset. Taking the KVLCC2 tanker and an unmanned surface vehicle (USV) as study objects, the nonparametric models are identified based on the experimental data of zigzag maneuvers of the KVLCC2 model and random maneuver of the USV. Using the identified models, the zigzag maneuvers of the KVLCC2 model and the random maneuver of the USV, which are not involved in the training data, are predicted. The results show that LGPR has higher computational efficiency than CGPR with acceptable prediction accuracy.

Single Image Super-Resolution Algorithm based on Fixed-Point Multi-Model Gaussian Process Regression

To solve the problem of large amount of computation in matrix inversion of Gaussian process regression model, a super-resolution algorithm based on local Gaussian process regression of fixed point multi-model is proposed. Firstly, the training samples are classified by Gaussian mixture model, and image patches are randomly selected as fixed points in each type of training samples, and its K nearest neighbor patch are searched. Secondly, local Gaussian process regression model by using its low-mid-frequency components and the corresponding high frequency components. Again, low resolution test image patch is classified, only the K nearest neighbor test image patch is searched during reconstruction. And then, find the nearest fixed point in each kind of image patch, and use the local Gaussian process regression model based on the fixed point to predict its corresponding high-frequency component. Finally, corresponding high frequency information is predicted by using this trained model, which improves the reconstruction efficiency. Experimental results have demonstrated that the proposed algorithm is superior in both quantitative and qualitative aspects against other algorithms.

Rapid accessibility evaluation for ballistic lunar capture via manifolds: A Gaussian process regression application

In this study, a supervised machine learning approach called Gaussian process regression (GPR) was applied to approximate optimal bi-impulse rendezvous maneuvers in the cis-lunar space. We demonstrate the use of the GPR approximation of the optimal bi-impulse transfer to patch points associated with various invariant manifolds in the cis-lunar space. The proposed method advances preliminary mission design operations by avoiding the computational costs associated with repeated solutions of the optimal bi-impulsive Lambert transfer because the learned map is computationally efficient. This approach promises to be useful for aiding in preliminary mission design. The use of invariant manifolds as part of the transfer trajectory design offers unique features for reducing propellant consumption while facilitating the solution of trajectory optimization problems. Long ballistic capture coasts are also very attractive for mission guidance, navigation, and control robustness. A multi-input single-output GPR model is presented to represent the fuel costs (in terms of the ΔV magnitude) associated with the class of orbital transfers of interest efficiently. The developed model is also proven to provide efficient approximations. The multi-resolution use of local GPRs over smaller sub-domains and their use for constructing a global GPR model are also demonstrated. One of the unique features of GPRs is that they provide an estimate of the quality of approximations in the form of covariance, which is proven to provide statistical consistency with the optimal trajectories generated through the approximation process. The numerical results demonstrate our basis for optimism for the utility of the proposed method.

Local approximate Gaussian process regression for data-driven constitutive models: development and comparison with neural networks

Hierarchical computational methods for multiscale mechanics such as the FE2 and FE-FFT methods are generally accompanied by high computational costs. Data-driven approaches are able to speed the process up significantly by enabling to incorporate the effective micromechanical response in macroscale simulations without the need of performing additional computations at each Gauss point explicitly. Traditionally artificial neural networks (ANNs) have been the surrogate modeling technique of choice in the solid mechanics community. However they suffer from severe drawbacks due to their parametric nature and suboptimal training and inference properties for the investigated datasets in a three dimensional setting. These problems can be avoided using local approximate Gaussian process regression (laGPR). This method can allow the prediction of stress outputs at particular strain space locations by training local regression models based on Gaussian processes, using only a subset of the data for each local model, offering better and more reliable accuracy than ANNs. A modified Newton–Raphson approach specific to laGPR is proposed to accommodate for the local nature of the laGPR approximation when solving the global structural problem in a FE setting. Hence, the presented work offers a complete and general framework enabling multiscale calculations combining a data-driven constitutive prediction using laGPR, and macroscopic calculations using an FE scheme that we test for finite-strain three-dimensional hyperelastic problems.

Splitting Gaussian processes for computationally-efficient regression.

Gaussian processes offer a flexible kernel method for regression. While Gaussian processes have many useful theoretical properties and have proven practically useful, they suffer from poor scaling in the number of observations. In particular, the cubic time complexity of updating standard Gaussian process models can be a limiting factor in applications. We propose an algorithm for sequentially partitioning the input space and fitting a localized Gaussian process to each disjoint region. The algorithm is shown to have superior time and space complexity to existing methods, and its sequential nature allows the model to be updated efficiently. The algorithm constructs a model for which the time complexity of updating is tightly bounded above by a pre-specified parameter. To the best of our knowledge, the model is the first local Gaussian process regression model to achieve linear memory complexity. Theoretical continuity properties of the model are proven. We demonstrate the efficacy of the resulting model on several multi-dimensional regression tasks.

Machine learning approaches for feature engineering of the crystal structure: Application to the prediction of the formation energy of cubic compounds

In this study, we present an approach (along with the needed computational strategies) for efficient and scalable feature engineering of the crystal structure in compounds of different chemical compositions. This approach utilizes a versatile and extensible framework for the quantification of a three-dimensional voxelized crystal structure in the form of 2-point spatial correlations of multiple atomic attributes and performs principal component analysis to extract the low-dimensional features that could be used to build surrogate models for material properties of interest. An application of the proposed feature engineering framework is demonstrated on a case study involving the prediction of the formation energies of crystalline compounds using two vastly different surrogate model building strategies; local Gaussian process regression and neural networks. Specifically, it is shown that the top 25 features (i.e., principal component scores) identified by the proposed framework serve as good regressors for the formation energy of the crystalline substance for both model building strategies.

Probabilistic wind power forecasting using selective ensemble of finite mixture Gaussian process regression models

Ensemble learning models have been widely used for wind power forecasting to facilitate efficient dispatching of power systems. However, traditional ensemble methods cannot always function well due to insufficient accuracy and diversity of base learners, ignorance of ensemble pruning, as well as the lack of adaptation capability. Therefore, a novel probabilistic wind power forecasting method is proposed based on selective ensemble of finite mixture Gaussian process regression models (SEFMGPR). First, a set of diverse local Gaussian process regression (GPR) models are constructed through multimodal perturbation mechanism, i.e., perturbing the training data and input attributes simultaneously. Then, a set of finite mixture GPR models (FMGPR) is built by integrating local GPR models through finite mixture mechanism (FMM). Next, the highly influential FMGPR models are selected using genetic algorithm (GA) based ensemble pruning. When a new test sample comes, the component predictions from the selected FMGPR models are adaptively combined by using FMM again and the probabilistic prediction results of the SEFMGPR model are obtained. Besides, an incremental adaptation mechanism is used to alleviate performance degradation of SEFMGPR. The application results from a real wind farm dataset show that SEFMGPR outperforms the traditional global and ensemble wind power prediction methods, and can maintain high prediction accuracy by effectively handling time-varying changes of wind power data.

Adaptive ranking based ensemble learning of Gaussian process regression models for quality-related variable prediction in process industries

The proper monitoring of quality-related but hard-to-measure variables is currently one of the bottlenecks limiting the safe and efficient operations of industrial processes. This paper proposes a novel ensemble learning algorithm by coordinating global and local Gaussian process regression (GPR) models, and this algorithm is able to capture global and local process behaviours for accurate prediction and timely process monitoring. To further address the deterioration in predictions when using the off-line training and online testing strategy, this paper proposes an adaptive ranking strategy to perform ensemble learning for the sub-GPR models. In this adaptive strategy, we use the moving-window technique to rank and select several of the best sub-model predictions and then average them together to make the final predictions. Last but not least, the least absolute shrinkage and selection operator (Lasso) works together with factor analysis (FA) in a two-step variable selection method to remove under-correlated model input variables in the first stage and to compress over-correlated model input variables in the second stage. The proposed prediction model is validated in two real wastewater treatment plants (WWTPs) with stationary and nonstationary behaviours. The results show that the proposed methodology achieves better performance than other standard methods in the context of their predictions of quality-related variables.

Smart Soft Sensor Design with Hierarchical Sampling Strategy of Ensemble Gaussian Process Regression for Fermentation Processes.

Accurate and real-time quality prediction to realize the optimal process control at a competitive price is an important issue in Industrial 4.0. This paper shows a successful engineering application of how smart soft sensors can be combined with machine learning technique to significantly save human resources and improve performance under complex industrial conditions. Ensemble learning based soft sensors succeed in capturing complex nonlinearities, frequent dynamic changes, as well as time-varying characteristics in industrial processes. However, local model regions under traditional ensemble modelling methods are highly dependent on labeled data samples and, hence, their prediction accuracy might get affected when labeled samples are limited. A novel active learning (AL) framework upon the ensemble Gaussian process regression (GPR) model is proposed for smart soft sensor design in order to overcome this drawback. Firstly, to iteratively select the most informative unlabeled samples for labeling with hierarchical sampling based AL strategy, to then apply Gaussian mixture model (GMM) technique to autonomously identify operation phases, to further construct local GPR models without human involvement, and finally to integrate the base predictors by applying the Bayesian fusion strategy. Comparative studies for the penicillin fermentation process demonstrate the reliability and superiority of the recommended smart soft sensing. The cost of human annotation can be dramatically reduced by at least half while the prediction performance simultaneously keeps high.

General Learning Modeling for AUV Position Tracking

In this article, we propose a nonlinear model based on hidden-layer neural networks and local Gaussian process regression. The hidden-layer neural networks have short training time, whereas the local Gaussian process regression is more suitable for nonlinearity. According to the abovementioned advantages of hidden-layer neural network and local Gaussian process regression, we get the measurement learning model and global process learning model offline and local process learning model online. Then, the proposed learning-based models have been applied to extended Kalman filter-simultaneous localization and mapping (EKF-SLAM), Rao-Blackwellised particle filter formulation of simultaneous localisation and mapping (FastSLAM), and unscented Kalman filter--simultaneous localization and mapping (UKF-SLAM) for autonomous underwater vehicles position tracking with field experiments. Compared with the conventional process and measurement models, improved position tracking performance can be achieved without cumbersome system analysis nor identification, benefiting from the proposed model. Experiments in more than 3883-m run show that the root-mean-square error has been improved by 36.95%, 37.14%, and 27.65% in EKF-SLAM, FastSLAM, and UKF-SLAM frameworks, respectively.

Vision-Based Online Learning Kinematic Control for Soft Robots Using Local Gaussian Process Regression

Soft robots, owing to their elastomeric material, ensure safe interaction with their surroundings. These robot compliance properties inevitably impose a tradeoff against precise motion control, as to which conventional model-based methods were proposed to approximate the robot kinematics. However, too many parameters, regarding robot deformation and external disturbance, are difficult to obtain, even if possible, which could be very nonlinear. Sensors self-contained in the robot are required to compensate modeling uncertainties and external disturbances. Camera (eye) integrated at the robot end-effector (hand) is a common setting. To this end, we propose an eye-in-hand visual servo that incorporates with learning-based controller to accomplish more precise robotic tasks. Local Gaussian process regression is used to initialize and refine the inverse mappings online, without prior knowledge of robot and camera parameters. Experimental validation is also conducted to demonstrate the hyperelastic robot can compensate an external variable loading during trajectory tracking.

Prognostics for structural health monitoring of historic masonry structures with a novel LL-GPR model

Structural health monitoring plays a significant role in civil engineering. To achieve more accurate prognostic evaluation of historic masonry structures, this paper presents a novel local linearization-based Gaussian process regression (LL-GPR) model. Local linearization is used to characterize the state of health (SOH) values of adjacent data points. Gaussian process regression is employed to predict the approximate linear relations. The proposed model is validated through a case study. The results demonstrate that the prediction results of LL-GPR can well reflect the degradation trend of SOH. Moreover, compared with the published methods, more accurate SOH prediction results can be obtained under this model

Active learning for modeling and prediction of dynamical fluid processes

Accurate prediction of the flow rate curve of a stroke for reciprocating multiphase pumps often encounters several challenges in practice, including process nonlinearity, dynamical characteristics, and changing multiphase transportation conditions. To enhance the prediction performance, an active learning method is proposed to efficiently design informative training data. Some initial training data are first collected from experiments to construct several local Gaussian process regression (GPR) models. Additionally, with the GPR-based probabilistic information, a relative variance-based criterion is proposed to explore which regions the new data should be introduced into the GPR prediction model. Moreover, an evaluation criterion is designed to implement the active learning procedure efficiently. Consequently, without time-consuming experiments, a set of new representative training data are sequentially introduced into the GPR prediction model. Experimental results and comparative studies for dynamical flow rate prediction of a stroke are carried out to demonstrate the effectiveness of the proposed method.

Auto-Switch Gaussian Process Regression-Based Probabilistic Soft Sensors for Industrial Multigrade Processes with Transitions

Prediction uncertainty has rarely been integrated into traditional soft sensors in industrial processes. In this work, a novel autoswitch probabilistic soft sensor modeling method is proposed for online quality prediction of a whole industrial multigrade process with several steady-state grades and transitional modes. It is different from traditional deterministic soft sensors. Several single Gaussian process regression (GPR) models are first constructed for each steady-state grade. A new index is proposed to evaluate each GPR-based steady-state grade model. For the online prediction of a new sample, a prediction variance-based Bayesian inference method is proposed to explore the reliability of existing GPR-based steady-state models. The prediction can be achieved using the related steady-state GPR model if its reliability using this model is large enough. Otherwise, the query sample can be treated as in transitional modes and a local GPR model in a just-in-time manner is online built. Moreover, to improve the efficiency, detailed implementation steps of the autoswitch GPR soft sensors for a whole multigrade process are developed. The superiority of the proposed method is demonstrated and compared with other soft sensors in an industrial process in Taiwan, in terms of online quality prediction.