Sparse Gaussian Process Regression Research Articles

The capacity degradation path prediction for the prismatic lithium-ion batteries based on the multi-features extraction with SGPR

Accurate capacity degradation path estimation of lithium-ion batteries plays a crucial role in ensuring the safety and reliability of electric vehicles. In recent years, the evolution of the status of health (SOH) prediction with machine learning techniques has become a research hotspot because of its powerful computing power and robustness. Thus, this study employes the sparse gaussian process regression (SGPR) data-driven approach with multi-features extracted from the battery cycling processes to project the potential degradation patterns of the lithium-ion batteries. Firstly, a battery life test platform was built. The accelerated life aging tests of batteries at different temperatures (25 °C, or 60 °C) and different discharge rates (1C, or 2C) were conducted to establish the dataset for the multi-features extraction and training of the SGPR algorithm. Secondly, different battery characteristic features were extracted based on the battery cycle charge-discharge curves. Various order-reduction treatments, i.e., the filter-based, embedding-based, and fusion-based selection algorithms, were performed to suppress the over-fitting and improve the estimate's accuracy. Finally, using the extracted features as inputs, SGPR models are constructed to estimate the degradation path of the battery. With the 50 % training data, the SGPR has a higher estimation accuracy than the regular GPR, and the average maximum absolute errors for the batteries are 2.80 %, 1.22 %, 3.57 %, and 1.83 %, respectively.

Modelling and estimating trajectory points from RTK-GNSS based on an integrated modelling approach

The sparse Gaussian process regression (GPR) has been used to model trajectory data from Real time kinematics-global navigation satellite system (RTK-GNSS). However, upon scrutinizing the model residuals; the sparse GPR model poorly fits the data and exhibits presence of correlated noise. This work attempts to address these issues by proposing an integrated modeling approach called GPR-LR-ARIMA where the sparse GPR was integrated with the linear regression with autoregressive integrated moving average errors (LR-ARIMA) to further enhance the description of the trajectory data. In this integrated approach, the predicted trajectory points from the GPR were further described by the LR-ARIMA. Simulation of the GPR-LR-ARIMA on three sets of trajectory data indicated better model fit, revealed in the normally distributed model residuals and symmetrically distributed scatter plots. Correlated noise was also successfully eliminated by the model. The GPR-LR-ARIMA outperformed both the GPR and LRARIMA by its ability to improve mean-absolute-error in 2-dimension positioning by up to 86%. The GPR-LR-ARIMA contributes to enhancement of positioning accuracy of dynamic GNSS measurements in localization and navigation system with good model fit.

Sample-Constrained Black Box Optimization for Audio Personalization

We consider the problem of personalizing audio to maximize user experience. Briefly, we aim to find a filter h*, which applied to any music or speech, will maximize the user’s satisfaction. This is a black-box optimization problem since the user’s satisfaction function is unknown. Substantive work has been done on this topic where the key idea is to play audio samples to the user, each shaped by a different filter hi, and query the user for their satisfaction scores f(hi). A family of “surrogate” functions is then designed to fit these scores and the optimization method gradually refines these functions to arrive at the filter ˆh* that maximizes satisfaction. In certain applications, we observe that a second type of querying is possible where users can tell us the individual elements h*[j] of the optimal filter h*. Consider an analogy from cooking where the goal is to cook a recipe that maximizes user satisfaction. A user can be asked to score various cooked recipes (e.g., tofu fried rice) or to score individual ingredients (say, salt, sugar, rice, chicken, etc.). Given a budget of B queries, where a query can be of either type, our goal is to find the recipe that will maximize this user’s satisfaction. Our proposal builds on Sparse Gaussian Process Regression (GPR) and shows how a hybrid approach can outperform any one type of querying. Our results are validated through simulations and real world experiments, where volunteers gave feedback on music/speech audio and were able to achieve high satisfaction levels. We believe this idea of hybrid querying opens new problems in black-box optimization and solutions can benefit other applications beyond audio personalization.

Treed Gaussian Process Regression for Solving Offline Data-Driven Continuous Multiobjective Optimization Problems.

For offline data-driven multiobjective optimization problems (MOPs), no new data is available during the optimization process. Approximation models (or surrogates) are first built using the provided offline data, and an optimizer, for example, a multiobjective evolutionary algorithm, can then be utilized to find Pareto optimal solutions to the problem with surrogates as objective functions. In contrast to online data-driven MOPs, these surrogates cannot be updated with new data and, hence, the approximation accuracy cannot be improved by considering new data during the optimization process. Gaussian process regression (GPR) models are widely used as surrogates because of their ability to provide uncertainty information. However, building GPRs becomes computationally expensive when the size of the dataset is large. Using sparse GPRs reduces the computational cost of building the surrogates. However, sparse GPRs are not tailored to solve offline data-driven MOPs, where good accuracy of the surrogates is needed near Pareto optimal solutions. Treed GPR (TGPR-MO) surrogates for offline data-driven MOPs with continuous decision variables are proposed in this paper. The proposed surrogates first split the decision space into subregions using regression trees and build GPRs sequentially in regions close to Pareto optimal solutions in the decision space to accurately approximate tradeoffs between the objective functions. TGPR-MO surrogates are computationally inexpensive because GPRs are built only in a smaller region of the decision space utilizing a subset of the data. The TGPR-MO surrogates were tested on distance-based visualizable problems with various data sizes, sampling strategies, numbers of objective functions, and decision variables. Experimental results showed that the TGPR-MO surrogates are computationally cheaper and can handle datasets of large size. Furthermore, TGPR-MO surrogates produced solutions closer to Pareto optimal solutions compared to full GPRs and sparse GPRs.

State-of-health estimation method for fast-charging lithium-ion batteries based on stacking ensemble sparse Gaussian process regression

Gaussian process regression (GPR) is extensively employed in lithium-ion battery state-of-health (SOH) estimation, which ensures the safe, reliable operation of electric vehicles (EVs). However, a single GPR can produce performance discrepancies across different fast-charge batteries, as well as high time consumption. Therefore, we propose an SOH estimation method for fast-charging batteries based on stacking ensemble sparse Gaussian process regression (SGPR). First, health factors are extracted in partial discharge fragments to reflect battery degradation. Then, SGPRs based on the fully independent training condition (FITC) are developed with different kernel functions as level-1 learners, and a genetic algorithm (GA) is used to optimize the parameters of the kernel function. Further, the level-2 learner integrates the features produced by the level-1 learner based on cross validation. Finally, the accuracy, robustness, and reliability of the proposed method were evaluated under various fast-charging experiments. The results show that the mean absolute error (MAE) and root mean square error (RMSE) of SOH estimation were within 1.0852% and 1.2123%, respectively, and that the average relative time consumption was reduced by 85.68% compared with stacking ensemble GPR. Thus, the proposed method has broad application prospects in processing vast datasets from numerous batteries in monitoring platforms or cloud data centers.

An effective surrogate model assisted algorithm for multi-objective optimization: application to wind farm layout design

Due to the intricate and diverse nature of industrial systems, traditional optimization algorithms require a significant amount of time to search for the optimal solution throughout the entire design space, making them unsuitable for meeting practical industrial demands. To address this issue, we propose a novel approach that combines surrogate models with optimization algorithms. Firstly, we introduce the Sparse Gaussian Process regression (SGP) into the surrogate model, proposing the SGP surrogate-assisted optimization method. This approach effectively overcomes the computational expense caused by the large amount of data required in Gaussian Process model. Secondly, we use grid partitioning to divide the optimization problem into multiple regions, and utilize the multi-objective particle swarm optimization algorithm to optimize particles in each region. By combining the advantages of grid partitioning and particle swarm optimization, which overcome the limitations of traditional optimization algorithms in handling multi-objective problems. Lastly, the effectiveness and robustness of the proposed method are verified through three types of 12 test functions and a wind farm layout optimization case study. The results show that the combination of meshing and SGP surrogate enables more accurate identification of optimal solutions, thereby improving the accuracy and speed of the optimization results. Additionally, the method demonstrates its applicability to a variety of complex multi-objective optimization problems.

Bridging semiempirical and abinitio QM/MM potentials by Gaussian process regression and its sparse variants for free energy simulation.

Free energy simulations that employ combined quantum mechanical and molecular mechanical (QM/MM) potentials at abinitio QM (AI) levels are computationally highly demanding. Here, we present a machine-learning-facilitated approach for obtaining AI/MM-quality free energy profiles at the cost of efficient semiempirical QM/MM (SE/MM) methods. Specifically, we use Gaussian process regression (GPR) to learn the potential energy corrections needed for an SE/MM level to match an AI/MM target along the minimum free energy path (MFEP). Force modification using gradients of the GPR potential allows us to improve configurational sampling and update the MFEP. To adaptively train our model, we further employ the sparse variational GP (SVGP) and streaming sparse GPR (SSGPR) methods, which efficiently incorporate previous sample information without significantly increasing the training data size. We applied the QM-(SS)GPR/MM method to the solution-phase SN2 Menshutkin reaction, NH3+CH3Cl→CH3NH3++Cl-, using AM1/MM and B3LYP/6-31+G(d,p)/MM as the base and target levels, respectively. For 4000 configurations sampled along the MFEP, the iteratively optimized AM1-SSGPR-4/MM model reduces the energy error in AM1/MM from 18.2 to 4.4 kcal/mol. Although not explicitly fitting forces, our method also reduces the key internal force errors from 25.5 to 11.1 kcal/mol/Å and from 30.2 to 10.3 kcal/mol/Å for the N-C and C-Cl bonds, respectively. Compared to the uncorrected simulations, the AM1-SSGPR-4/MM method lowers the predicted free energy barrier from 28.7 to 11.7 kcal/mol and decreases the reaction free energy from -12.4 to -41.9 kcal/mol, bringing these results into closer agreement with their AI/MM and experimental benchmarks.

Decentralized Sparse Gaussian Process Regression with Event-Triggered Adaptive Inducing Points

Decentralized Sparse Gaussian Process Regression with Event-Triggered Adaptive Inducing Points

Learning-Based Model Predictive Control for Autonomous Racing

In this paper, we present the adaptation of the terminal component learning-based model predictive control (TC-LMPC) architecture for autonomous racing to the Formula Student Driverless (FSD) context. We test the TC-LMPC architecture, a reference-free controller that is able to learn from previous iterations by building an appropriate terminal safe set and terminal cost from collected trajectories and input sequences, in a vehicle simulator dedicated to the FSD competition. One major problem in autonomous racing is the difficulty in obtaining accurate highly nonlinear vehicle models that cover the entire performance envelope. This is more severe as the controller pushes for incrementally more aggressive behavior. To address this problem, we use offline and online measurements and machine learning (ML) techniques for the online adaptation of the vehicle model. We test two sparse Gaussian process regression (GPR) approximations for model learning. The novelty in the model learning segment is the use of a selection method for the initial training dataset that maximizes the information gain criterion. The TC-LMPC with model learning achieves a 5.9 s reduction (3%) in the total 10-lap FSD race time.

Uncertainty-aware molecular dynamics from Bayesian active learning for phase transformations and thermal transport in SiC

Machine learning interatomic force fields are promising for combining high computational efficiency and accuracy in modeling quantum interactions and simulating atomistic dynamics. Active learning methods have been recently developed to train force fields efficiently and automatically. Among them, Bayesian active learning utilizes principled uncertainty quantification to make data acquisition decisions. In this work, we present a general Bayesian active learning workflow, where the force field is constructed from a sparse Gaussian process regression model based on atomic cluster expansion descriptors. To circumvent the high computational cost of the sparse Gaussian process uncertainty calculation, we formulate a high-performance approximate mapping of the uncertainty and demonstrate a speedup of several orders of magnitude. We demonstrate the autonomous active learning workflow by training a Bayesian force field model for silicon carbide (SiC) polymorphs in only a few days of computer time and show that pressure-induced phase transformations are accurately captured. The resulting model exhibits close agreement with both ab initio calculations and experimental measurements, and outperforms existing empirical models on vibrational and thermal properties. The active learning workflow readily generalizes to a wide range of material systems and accelerates their computational understanding.

Stochastic variational inference for scalable non-stationary Gaussian process regression

A natural extension to standard Gaussian process (GP) regression is the use of non-stationary Gaussian processes, an approach where the parameters of the covariance kernel are allowed to vary in time or space. The non-stationary GP is a flexible model that relaxes the strong prior assumption of standard GP regression, that the covariance properties of the inferred functions are constant across the input space. Non-stationary GPs typically model varying covariance kernel parameters as further lower-level GPs, thereby enabling sampling-based inference. However, due to the high computational costs and inherently sequential nature of MCMC sampling, these methods do not scale to large datasets. Here we develop a variational inference approach to fitting non-stationary GPs that combines sparse GP regression methods with a trajectory segmentation technique. Our method is scalable to large datasets containing potentially millions of data points. We demonstrate the effectiveness of our approach on both synthetic and real world datasets.

Correlated product of experts for sparse Gaussian process regression

Gaussian processes (GPs) are an important tool in machine learning and statistics. However, off-the-shelf GP inference procedures are limited to datasets with several thousand data points because of their cubic computational complexity. For this reason, many sparse GPs techniques have been developed over the past years. In this paper, we focus on GP regression tasks and propose a new approach based on aggregating predictions from several local and correlated experts. Thereby, the degree of correlation between the experts can vary between independent up to fully correlated experts. The individual predictions of the experts are aggregated taking into account their correlation resulting in consistent uncertainty estimates. Our method recovers independent Product of Experts, sparse GP and full GP in the limiting cases. The presented framework can deal with a general kernel function and multiple variables, and has a time and space complexity which is linear in the number of experts and data samples, which makes our approach highly scalable. We demonstrate superior performance, in a time vs. accuracy sense, of our proposed method against state-of-the-art GP approximations for synthetic as well as several real-world datasets with deterministic and stochastic optimization.

A hybrid proper orthogonal decomposition-heteroscedastic sparse Gaussian process regression model for evaluating structural reliability with correlated stochastic material properties

Data-driven material models have shown advantages in recent studies since they can directly use the existing data and improve the model performances further when additional data is available. However, few data-driven material models in the previous studies considered the uncertainty and stochastic correlations of the material properties. In this work, the hybrid Proper Orthogonal Decomposition-Heteroscedastic Sparse Gaussian Process Regression (POD-HSGPR) framework is proposed for predicting stochastic material behaviors and their correlation. The two material behaviors case studies on the metal strength and the rock joint behavior have demonstrated that the proposed POD-HSGPR-based stochastic material model can effectively capture the material properties, material uncertainty, and stochastic correlation between the material behaviors directly from the experimental dataset. The proposed POD-HSGPR model is then applied to a rock slope structure problem to investigate the influence of the correlation of material properties on structural behavior. The results indicate that the proposed POD-HSGPR model could effectively quantify the correlation effect in a non-parametric format and avoid the overestimation of structural reliability.

Sparse Gaussian Process Regression for Landslide Displacement Time-Series Forecasting

Landslide hazards are complex nonlinear systems with a highly dynamic nature. Accurate forecasting of landslide displacement and evolution is crucial for the prevention and mitigation of landslide hazards. In this study, a probabilistic landslide displacement forecasting model based on the quantification of epistemic uncertainty is proposed. In particular, the displacement forecasting problem is cast as a time-series regression problem with limited training samples and must be solved by statistical inference. The epistemic uncertainty of the landslide displacement series is depicted by the statistical properties of the function space constituted by the nonlinear mappings generated by the sparse Gaussian process regression. Data for our study was collected from the study area located in northwestern China. Other state-of-the-art probabilistic forecasting models have also been utilized for comparative analysis. The experimental results confirmed the superiority of the sparse Gaussian process in the modeling of landslide displacement series in terms of forecasting accuracy, uncertainty quantification, and robustness to overfitting.

Fast atomic structure optimization with on-the-fly sparse Gaussian process potentials *

We apply on-the-fly machine learning potentials (MLPs) using the sparse Gaussian process regression (SGPR) algorithm for fast optimization of atomic structures. Great acceleration is achieved even in the context of a single local optimization. Although for finding the exact local minimum, due to limited accuracy of MLPs, switching to another algorithm may be needed. For random gold clusters, the forces are reduced to ∼0.1 eV Å−1 within less than ten first-principles (FP) calculations. Because of highly transferable MLPs, this algorithm is specially suitable for global optimization methods such as random or evolutionary structure searching or basin hopping. This is demonstrated by sequential optimization of random gold clusters for which, after only a few optimizations, FP calculations were rarely needed.

On the impact of prior distributions on efficiency of sparse Gaussian process regression

On the impact of prior distributions on efficiency of sparse Gaussian process regression

Al‐Doping Driven Suppression of Capacity and Voltage Fadings in 4d‐Element Containing Li‐Ion‐Battery Cathode Materials: Machine Learning and Density Functional Theory

AbstractThe anion redox reaction in high‐energy‐density cathode materials such as Li‐excess layered oxides suffers from voltage/capacity fadings due to irreversible structural instability. Here, exploiting density functional theory (DFT) as well as fast simulations using the universal potential/forces generated from the newly developed sparse Gaussian process regression (SGPR) machine learning (ML) method, the very complicated/complex structures, X‐ray absorption near‐edge‐structure (XANES) spectra, redox phenomena, and Li diffusion of these battery materials depending on charging/discharging processes is investigated. It is found that voltage/capacity fadings are strongly suppressed in 4d‐element‐containing cathodes by Al‐doping. The suppressed fadings are discussed in view of the structural and electronic changes depending on charged/discharged states which are reflected in their extended X‐ray absorption fine structure and XANES spectra. According to crystal orbital Hamilton populations (COHP) and Bader charge analyses of Li1.22Ru0.61Ni0.11Al0.06O2 (Al‐LRNO), the Al‐doping helps in forming Ni–Al bonding and hence strengthens the bonding‐orbital characteristics in Al–O bonds. This strengthened Al–O bonding hinders oxygen oxidation and thus enhances structural stability, diminishing safety concerns. The Al‐doping driven suppression of capacity fading and voltage decay is expected to help in designing stable reversible layered cathode materials.

Data-Driven Battery State of Health Estimation Based on Random Partial Charging Data

The rapid development of battery technology has promoted the deployment of electric vehicles (EVs). To ensure the healthy and sustainable development of EVs, it is urgent to solve the problems of battery safety monitoring, residual value assessment, and predictive maintenance, which heavily depends on the accurate state-of-health (SOH) estimation of batteries. However, many published methods are unsuitable for actual vehicle conditions. To this end, a data-driven method based on the random partial charging process and sparse Gaussian process regression (GPR) is proposed in this article. First, the random capacity increment sequences (<i>&#x25B3;Q</i>) at different voltage segments are extracted from the partial charging process. The average value and standard deviation of <i>&#x25B3;Q</i> are used as features to indicate battery health. Second, correlation analysis is conducted for three types of batteries, and high correlations between the features and battery SOH are verified at different temperatures and discharging current rates. Third, by using the proposed features as inputs, sparse GPR models are constructed to estimate the SOH. Compared with other data-driven methods, the sparse GPR has the highest estimation accuracy, and its average maximum absolute errors are only 2.88&#x0025;, 2.52&#x0025;, and 1.51&#x0025; for three different types of batteries, respectively.

State-of-charge dependent equivalent circuit model identification for batteries using sparse Gaussian process regression

Due to their ease of implementation, equivalent circuit models (ECMs) of batteries are widely used in battery management systems. Generally, ECM parameters vary with operating conditions, thus how such parameter dependencies are addressed substantially influences the accuracy of an ECM over a wide operating range. In this paper, we identify an ECM whose parameters have nonlinear dependence on state-of-charge (SOC). By transforming the SOC-dependent ECM into a linear parameter varying (LPV) input–output model, we propose a non-parametric sparse Gaussian process regression (GPR) approach, which alleviates the difficulty of specifying parametric functional SOC-dependencies of model parameters. The proposed approach derives the posterior distributions of ECM parameters, thus is capable to provide both parameter estimates and their associated uncertainties. The computational cost over large datasets is significantly reduced by adopting the sparse GPR. The proposed approach is applied to the above LPV model with two noise model structures, i.e., white and colored noises. Identification results using experimental data illustrate the efficacy of the proposed approach. The use of colored noise enhances robustness under different noise levels, and achieves higher output prediction accuracy over experimental datasets.

Heteroscedastic sparse Gaussian process regression-based stochastic material model for plastic structural analysis

Describing the material flow stress and the associated uncertainty is essential for the plastic stochastic structural analysis. In this context, a data-driven approach-heteroscedastic sparse Gaussian process regression (HSGPR) with enhanced efficiency is introduced to model the material flow stress. Different from other machine learning approaches, e.g. artificial neural network (ANN), which only estimate the deterministic flow stress, the HSGPR model can capture the flow stress and its uncertainty simultaneously from the dataset. For validating the proposed model, the experimental data of the Al 6061 alloy is used here. Without setting a priori assumption on the mathematical expression, the proposed HSGPR-based flow stress model can produce a better prediction of the experimental stress data than the ANN model, the conventional GPR model, and Johnson Cook model at elevated temperatures. After the HSGPR-based flow stress model is implemented into finite element analysis, two numerical examples with synthetic material properties are performed to demonstrate the model’s capability in stochastic plastic structural analysis. The results have shown that with sufficient data, the distribution of the structural load carrying capacity at elevated temperatures and the variation of load–displacement curves during the loading and unloading processes can be accurately predicted by the HSGPR-based flow stress model.