Gaussian Process Research Articles

A Gaussian process model for stellar activity in 2-D line profile time-series

A Gaussian process model for stellar activity in 2-D line profile time-series

Internally-consistent and fully-unbiased multimodal MRI brain template construction from UK Biobank: Oxford-MM

Abstract Anatomical MRI templates of the brain are essential to group-level analyses and image processing pipelines, as they provide a reference space for spatial normalisation. While it has become common for studies to acquire multimodal MRI data, many templates are still limited to one type of modality, usually either scalar or tensor-based. Aligning each modality in isolation does not take full advantage of the available complementary information, such as strong contrast between tissue types in structural images, or axonal organisation in the white matter in diffusion tensor images. Most existing strategies for multimodal template construction either do not use all modalities of interest to inform the template construction process, or do not use them in a unified framework. Here, we present multimodal, cross-sectional templates constructed from UK Biobank data: the OMM-1 template, and age-dependent templates for each year of life between 45 to 81. All templates are fully unbiased to represent the average shape of the populations they were constructed from, and internally consistent through jointly informing the template construction process with T1, T2-FLAIR and DTI data. The OMM-1 template was constructed with a multi-resolution, iterative approach using 240 individuals in the 50-55 year age range. The age-dependent templates were estimated using a Gaussian Process, which describes the change in average brain shape with age in 37,330 individuals. All templates show excellent contrast and alignment within and between modalities. The global brain shape and size is not preconditioned on existing templates, although maximal possible compatibility with MNI-152 space was maintained through rigid alignment. We showed benefits in registration accuracy across two datasets (UK Biobank and HCP), when using the OMM-1 as the template compared with FSL’s MNI-152 template, and found that the use of age-dependent templates further improved accuracy to a small but detectable extent. All templates are publicly available and can be used as a new reference space for uni- or multimodal spatial alignment.

High precision accelerator for our hybrid model of the redshift space power spectrum

Abstract Upcoming Large Scale Structure surveys aim to achieve an unprecedented level of precision in measuring galaxy clustering. However, accurately modelling these statistics may require theoretical templates that go beyond two-loop order perturbation theory, especially for achieving precision at smaller scales. In our previous work, we introduced a hybrid model for the redshift space power spectrum of galaxies. This model combines two-loop order templates with N-body simulations to capture the influence of scale-independent parameters on the galaxy power spectrum. However, the impact of scale-dependent parameters was addressed by precomputing a set of input statistics derived from computationally expensive N-body simulations. As a result, exploring the scale-dependent parameter space was not feasible in this approach. To address this challenge, we present an accelerated methodology that utilizes Gaussian processes, a machine learning technique, to emulate these input statistics. Our emulators exhibit remarkable accuracy, achieving reliable results with just 13 N-body simulations for training. Our emulators can reproduce the set of statistics we are interested in with less than 0.1% error in the parameter space within 5σ of the Planck ΛCDM predictions, specifically for scales around k > 0.1 h Mpc−1. Following the training of our emulators, we can predict all inputs for our hybrid model in approximately 0.2 seconds at a specified redshift. Given that performing 13 N-body simulations is a manageable task, our present methodology enables us to construct efficient and highly accurate models of the galaxy power spectra within a manageable time frame.

Enhanced Crop Leaf Area Index Estimation via Random Forest Regression: Bayesian Optimization and Feature Selection Approach

The Leaf Area Index (LAI) is a crucial structural parameter linked to the photosynthetic capacity and biomass of crops. While integrating machine learning algorithms with spectral variables has improved LAI estimation over large areas, excessive input parameters can lead to data redundancy and reduced generalizability across different crop species. To address these challenges, we propose a novel framework based on Bayesian-Optimized Random Forest Regression (Bayes-RFR) for enhanced LAI estimation. This framework employs a tree model-based feature selection method to identify critical features, reducing redundancy and improving model interpretability. A Gaussian process serves as a prior model to optimize the hyperparameters of the Random Forest Regression. The field experiments conducted over two years on maize and wheat involved collecting LAI, hyperspectral, multispectral, and RGB data. The results indicate that the tree model-based feature selection outperformed the traditional correlation analysis and Recursive Feature Elimination (RFE). The Bayes-RFR model demonstrated a superior validation accuracy compared to the standard Random Forest Regression and Pso-optimized models, with the R2 values increasing by 27% for the maize hyperspectral data, 12% for the maize multispectral data, and 47% for the wheat hyperspectral data. These findings suggest that the proposed Bayes-RFR framework significantly enhances the stability and predictive capability of LAI estimation across various crop types, offering valuable insights for precision agriculture and crop monitoring.

Predicting shear strength of corroded RC columns: A probabilistic model with enhanced Gaussian Process Regression

This study proposes a new probabilistic model for predicting the shear strength of corroded reinforced concrete (RC) columns. The probabilistic model addresses limitations of traditional methods by combining mechanical understanding with enhanced Gaussian Process Regression (GPR). A novel mean function for enhanced GPR is developed first based on the shear resistance mechanism of corroded RC columns. Then the hyper-parameters of both the mean function and kernel function for the enhanced GPR are optimized using the maximum likelihood estimation method. This leads to the establishment of the probabilistic model that is based on the enhanced GPR. Finally, the accuracy and effectiveness of the enhanced GPR probabilistic model are validated by comparing it with both traditional mechanical models and machine learning models. The results indicate that the proposed probabilistic model can not only describes the probabilistic characteristics for shear strength of corroded RC columns based on probability density functions, but also provides an efficient calibration method for traditional prediction models based on confidence intervals.

Development of electromagnetic pollution maps utilizing Gaussian process spatial models

The rapid proliferation of wireless technologies in everyday environments demands the quick and precise estimation of electromagnetic field distribution. This distribution is commonly depicted through the electric field strength across various geographical areas. The objective of this research is to determine the most effective geospatial model for generating a national-level electric field strength map within the 30 MHz–6 GHz frequency range. To achieve this, we employed five different methodologies for constructing the electric field strength map. Four of these methodologies are based on Gaussian process regression, while the fifth utilizes the classical weighted-average method of the nearest neighbor. Our study focused on a country with a total area of 9251 km2, using a dataset comprising 3621 measurements. The findings reveal that Gaussian process spatial models, also known as Kriging models, generally outperform other methods when applied to spatial data. However, it was observed that, after excluding some outlier data points, the performance of the classical nearest neighbor models becomes comparable to that of the Gaussian process models. This indicates the potential for both approaches to be effective, depending on the data quality and the presence of outliers.

Resilient Coastal Protection Infrastructures: Probabilistic Sensitivity Analysis of Wave Overtopping Using Gaussian Process Surrogate Models

This paper presents a novel mathematical framework for assessing and predicting the resilience of critical coastal infrastructures against wave overtopping hazards and extreme climatic events. A probabilistic sensitivity analysis model is developed to evaluate the relative influence of hydrodynamic, geomorphological, and structural factors contributing to wave overtopping dynamics. Additionally, a stochastic Gaussian process (GP) model is introduced to predict the mean overtopping discharge from coastal defences. Both the sensitivity analysis and the predictive models are validated using a large homogeneous dataset comprising 163 laboratory and field-scale tests. Statistical evaluations demonstrate the superior performance of the GPs in identifying key parameters driving wave overtopping and predicting mean discharge rates, outperforming existing regression-based formulae. The proposed model offers a robust predictive tool for assessing the performance of critical coastal protection infrastructures under various climate scenarios.

Development and validation of a machine learning‐based model of ischemic stroke risk in the Chinese elderly hypertensive population

AbstractMachine learning (ML) has made some significant contributions to stroke prevention, but the stability and accuracy of existing models for clinical applications are uncertain. This study develops and validates an interpretable ML model using metabolic and coagulation biomarkers to predict ischemic stroke in elderly hypertensive patients in Northwest China. The prediction model used 453 electronic medical records for the model building (80% as a training set and 20% as a test set) and 132 for external validation. The final seven key features (D‐dimer, cystatin C, homocysteine, hemoglobin A1c, prothrombin time, low‐density lipoprotein C, and triglyceride glucose‐body mass index) were selected by the advanced approach, elastic net, and classical wrapping approaches. The final model, eXtreme gradient boosting, was identified as having superior performance than the other 9 classifers (random forest, Gaussian process, multilayer perceptron, logistic regression, support vector machine, K‐nearest neighbor, decision tree, Gaussian naive bayes, and ensemble model), with area under the receiver‐operating characteristic curves of 0.97 and 0.94 for the test and external validation sets, respectively. The final model demonstrates excellent stability, accuracy, and clinical usefulness through various metrics and decision curve analysis. Additionally, an online human–machine interface application has been developed for clinical practice to help early identification and intervention for ischemic stroke in elderly hypertensive patients.

Extreme-value modelling of migratory bird arrival dates: Insights from citizen science data

Abstract Citizen science mobilises many observers and gathers huge datasets but often without strict sampling protocols, resulting in observation biases due to heterogeneous sampling effort, which can lead to biased predictions. We develop a spatiotemporal Bayesian hierarchical model for bias-corrected estimation of arrival dates of the first migratory bird individuals at their breeding sites. Higher sampling effort could be correlated with earlier observed dates. We implement data fusion of two citizen-science datasets with fundamentally different protocols (BBS, eBird) and obtain posterior distributions of the latent process, which contains four spatial components endowed with Gaussian process priors: species niche; sampling effort; position and scale parameters of annual first arrival date. The data layer consists of four response variables: counts of observed eBird locations (Poisson); presence-absence at observed eBird locations (Binomial); BBS occurrence counts (Poisson); first arrival dates (Generalised Extreme-Value). We devise a Markov Chain Monte Carlo scheme and check by simulation that the latent process components are identifiable. We apply our model to several migratory bird species in the northeastern United States for 2001−2021 and find that the sampling effort significantly modulates the observed first arrival dates. We exploit this relationship to effectively bias-correct predictions of the true first arrivals.

A machine-learned kinetic energy model for light weight metals and compounds of group III-V elements

Abstract We present a machine-learned (ML) model of kinetic energy for orbital-free density functional theory (OF-DFT) suitable for bulk light weight metals and compounds made of group III–V elements. The functional is machine-learned with Gaussian process regression (GPR) from data computed with Kohn-Sham DFT with plane wave bases and local pseudopotentials. The dataset includes multiple phases of unary, binary, and ternary compounds containing Li, Al, Mg, Si, As, Ga, Sb, Na, Sn, P, and In. A total of 433 materials were used for training, and 18 strained structures were used for each material. Averaged (over the unit cell) kinetic energy density is fitted as a function of averaged terms of the 4th order gradient expansion and the product of the density and effective potential. The kinetic energy predicted by the model allows reproducing energy-volume curves around equilibrium geometry with good accuracy. We show that the GPR model beats linear and polynomial regressions. We also find that unary compounds sample a wider region of the descriptor space than binary and ternary compounds, and it is therefore important to include them in the training set; a GPR model trained on a small number of unary compounds is able to extrapolate relatively well to binary and ternary compounds but not vice versa.

Constitutive Modeling of Anisotropic Plasticity with ML Algorithms in Metals

Nuclear reactors hold significant promise for mitigating the environmental crisis and achieving a carbon-free world. The advancement of nuclear technology, however, is intricately tied to the materials used in reactors, with metals and alloys forming their core components. Understanding the plasticity and degradation of these materials is crucial to prevent critical reactor failures caused by factors such as radiation-induced embrittlement, creep, and fuel-cladding interactions. While traditional plasticity modeling has relied on constitutive laws and yield functions, the advent of machine learning (ML) and artificial intelligence has opened new avenues for this field. This paper explores the potential of data-driven approaches to enhance plasticity modeling for metals used in nuclear reactors. Recent studies have leveraged ML algorithms, including support vector machines and artificial neural networks, to model yield surfaces and correct theoretical yield functions with experimental data. Despite their accuracy, these models often lack interpretability and generality. To address these challenges, we investigate the applicability of various ML algorithms, i.e. neural networks, logistic regressions, decision trees, K-nearest neighbors, and Gaussian processes (GPs), in developing data-oriented flow rules for plasticity modeling. The paper demonstrates the superiority of GPs in terms of interpretability and generality. Using stress data generated from PyLabFEA, we compared the performance of these algorithms, highlighting the intrinsic uncertainties associated with GP predictions. Our findings indicate that Gaussian process classifiers (GPCs) offer a promising approach for modeling plasticity in metals, providing a balance between precision and physical insight. Additionally, this work presents a deep neural network (DNN) to model anisotropic Hill-type plasticity with perfect test data accuracy (accuracy score: 1.0), which is often hard to achieve with a classification problem. This shows the superiority of DNNs in terms of accuracy in modeling yield functions. The implications of our results for bridging the scale gaps in macrolevel simulations are discussed, and future directions for incorporating microstructural features into ML-based plasticity models are proposed. Overall, the ML framework presented in this paper can be employed for modeling constitutive relations that can be incorporated within traditional Finite Element Method (FEM)–based codes. Specifically, the GPC or DNN developed in this work can be readily swapped with the yield function within the so-called return mapping algorithm of nonlinear FEM-based plasticity subroutines to indicate yielding. Consequently, this merging of ML and FEM will allow us to leverage the geometric representational ability of FEM alongside the superior modeling capabilities of the ML algorithms.

Dynamic Programming-Based Approach to Model Antigen-Driven Immune Repertoire Synthesis

This paper presents a novel approach to modeling the repertoire of the immune system and its adaptation in response to the evolutionary dynamics of pathogens associated with their genetic variability. It is based on application of a dynamic programming-based framework to model the antigen-driven immune repertoire synthesis. The processes of formation of new receptor specificity of lymphocytes (the growth of their affinity during maturation) are described by an ordinary differential equation (ODE) with a piecewise-constant right-hand side. Optimal control synthesis is based on the solution of the Hamilton–Jacobi–Bellman equation implementing the dynamic programming approach for controlling Gaussian random processes generated by a stochastic differential equation (SDE) with the noise in the form of the Wiener process. The proposed description of the clonal repertoire of the immune system allows us to introduce an integral characteristic of the immune repertoire completeness or the integrative fitness of the whole immune system. The quantitative index for characterizing the immune system fitness is analytically derived using the Feynman–Kac–Kolmogorov equation.

Data-driven predictions of retained austenite content and yield strength and elucidation of the sliding wear performance of carbide-free bainitic steels

Carbide-free bainitic (CFB) steels have drawn much attention due to their high strength and toughness. The present work aims to build machine learning (ML) based surrogate models to predict retained austenite (RA) content and yield strength (YS) of isothermal transformed CFB steels and study the sliding wear performance. The input data for ML prediction were related to compositions, heat treatment process, and phases. The Random Forest regression and Gaussian process regression were respectively selected to build the predictive models for RA content and YS. Result shows that the predicted RA content agrees well with the experimentally determined ones in steel with high stability of untransformed austenite. For YS prediction, the present surrogate model can predict YS of CFB steels even without additional microstructural information such as dislocation and effective substructure size. Steels with different YS and RA content were designed to study the sliding wear performance. Result shows that the dominant wear mechanism of the tested steels is abrasive wear and the weight loss is inversely proportional to hardness when the load and wear distance are constant. Increasing YS and RA content can increase the initial hardness and work hardening ability at the worn surface, thereby enhancing wear resistance.

Smart Grid Forecasting with MIMO Models: A Comparative Study of Machine Learning Techniques for Day-Ahead Residual Load Prediction

With the fast expansion of intermittent renewable energy sources in the upcoming smart grids, simple and accurate day-ahead systems for residual load forecasts are urgently needed. Machine learning strategies can facilitate towards drastic cost minimizations in terms of operating-reserves avoidance to compensate the mismatches between the actual and forecasted values. In this study, a multi-input/multi-output model is developed based on artificial neural networks to map the relationship between different predictor inputs, including time indices, weather variables, human activity parameters, and energy price indicators, and target outputs such as wind and photovoltaic generation. While the information flows in only one direction (from the predictor nodes through the hidden layers to the target node), benchmark training algorithms are employed and assessed under different case studies. The model is evaluated under both parametric and non-parametric formulations, namely neural networks and Gaussian process regression. Essential improvements are achieved by enhancing the number of embedded predictors, while superior performance is observed by using Bayesian regularization mechanisms. In terms of mean-error indices and determination coefficient, this opens the pathway towards minimization via Bayesian inference-based approaches in the presence of increased and highly stochastic renewable inputs.

Computationally efficient and error aware surrogate construction for numerical solutions of subsurface flow through porous media

Limiting the injection rate to restrict the pressure below a threshold at a critical location can be an important goal of simulations that model the subsurface pressure between injection and extraction wells. The pressure is approximated by the solution of Darcy’s partial differential equation for a given permeability field. The subsurface permeability is modeled as a random field since it is known only up to statistical properties. This induces uncertainty in the computed pressure. Solving the partial differential equation for an ensemble of random permeability simulations enables estimating a probability distribution for the pressure at the critical location. These simulations are computationally expensive, and practitioners often need rapid online guidance for real-time pressure management. An ensemble of numerical partial differential equation solutions is used to construct a Gaussian process regression model that can quickly predict the pressure at the critical location as a function of the extraction rate and permeability realization. The Gaussian process surrogate analyzes the ensemble of numerical pressure solutions at the critical location as noisy observations of the true pressure solution, enabling robust inference using the conditional Gaussian process distribution.Our first novel contribution is to identify a sampling methodology for the random environment and matching kernel technology for which fitting the Gaussian process regression model scales as O(nlogn) instead of the typical O(n3) rate in the number of samples n used to fit the surrogate. The surrogate model allows almost instantaneous predictions for the pressure at the critical location as a function of the extraction rate and permeability realization. Our second contribution is a novel algorithm to calibrate the uncertainty in the surrogate model to the discrepancy between the true pressure solution of Darcy’s equation and the numerical solution. Although our method is derived for building a surrogate for the solution of Darcy’s equation with a random permeability field, the framework broadly applies to solutions of other partial differential equations with random coefficients.

Machine Learning-Aided Dual-Function Microfluidic SIW Sensor Antenna for Frost and Wildfire Detection Applications

In this paper, which represents a fundamental step in ongoing research, a new smart low-energy dual-function half-mode substrate integrated waveguide cavity-interdigital capacitor (HMSIWC-DIC) antenna-based sensor is developed and investigated for remote frost and wildfire detection applications at 5.7 GHz. The proposed methodology exploits the HMSIW antenna-based sensor, a microfluidic channel (microliter water channel (50 μL)), interdigital capacitor technologies, and the resonance frequency parameters combined with machine learning algorithms. This allows for superior interaction between the water channel and the TE101 mode, resulting in high sensitivity (∆f/∆ε = 5.5 MHz/ε (F/m) and ∆f/∆°C = 1.83 MHz/°C) within the sensing range. Additionally, it exhibits high decision-making ability and immunity to interference, demonstrating a best-in-class sensory response to weather temperature across two ranges: positive (≥0 °C, including frost and wildfire) and negative (<0 °C, including ice accumulation). To address the challenges posed by the non-linear, unpredictable behavior of resonance frequency results, even when dealing with weak sensor antenna responses, an innovative sensory intelligent system was proposed. This system utilizes resonance frequency results as features to classify and predict weather temperature ranges into three environmental states: Early Frost, Normal, and Early Wildfire, achieving an accuracy of 96.4%. Several machine learning techniques are employed, including artificial neural networks (ANNs), random forests (RF), decision trees (DT), support vector machines (SVMs), and Gaussian processes (GPs). This sensor serves as an ideal solution for energy management through its utilization in RF-based weather temperature sensing applications. It boasts stable performance, minimal energy consumption, and real-time sensitivity, eliminating the necessity for manual data recording.

A gaussian process framework for solving forward and inverse problems involving nonlinear partial differential equations

A gaussian process framework for solving forward and inverse problems involving nonlinear partial differential equations

Estimation of Urban Tree Chlorophyll Content and Leaf Area Index Using Sentinel-2 Images and 3D Radiative Transfer Model Inversion

Urban trees play an important role in mitigating effects of climate change and provide essential ecosystem services. However, the urban environment can stress trees, requiring the use of effective monitoring methods to assess their health and functionality. The objective of this study, which focused on four deciduous tree species in Rennes, France, was to evaluate the ability of hybrid inversion models to estimate leaf chlorophyll content (LCC), leaf area index (LAI), and canopy chlorophyll content (CCC) of urban trees using eight Sentinel-2 (S2) images acquired in 2021. Simulations were performed using the 3D radiative transfer model DART, and the hybrid inversion models were developed using machine-learning regression algorithms (random forest (RF) and gaussian process regression). Model performance was assessed using in situ measurements, and relations between satellite data and in situ measurements were investigated using spatial allocation (SA) methods at the pixel and tree scales. The influence of including environment features (EFs) as model inputs was also assessed. The results indicated that random forest models that included EFs and used the pixel-scale SA method were the most accurate with R2 values of 0.33, 0.29, and 0.46 for LCC, LAI, and CCC, respectively, with notable variability among species.

Bayesian protocols for high-throughput identification of kinematic hardening model forms

Constitutive models are essential for assessing the mechanical response of complex materials, yet uncertainties in model forms and parameters persist due to the influence of micromechanisms and microstructural features. We develop Bayesian protocols to iteratively refine both model forms and the associated material properties for complex constitutive models. Our aim is to provide rigorous, probabilistically informed evaluations of improvements achieved with increasing model complexity. Leveraging high-throughput experimental microindentation data, the protocols involve three steps: (i) emulating FE simulations using multi-output Gaussian process surrogate models, (ii) calibrating an initial simple constitutive model against experimental data, and (iii) progressively enhancing model complexity by iteratively improving agreement between simulations and experiments. The various model forms are compared using model form probabilities and aggregate discrepancies. Sobol indices are used to quantify the identifiability of material properties, aiming to prevent parameter proliferation. We apply this protocol to identify the optimal form of cyclic plasticity models for duplex Ti-6Al-4V. Although tailored for cyclic plasticity models, these protocols hold promise for calibrating and refining nonlinear constitutive models across diverse material classes.

A Cluster-Based Deep Learning Model Perceiving Series Correlation for Accurate Prediction of Phonon Spectrum.

The spectral properties are the most prevalent continuous representation for characterizing transport phenomena and excitation responses, yet their accurate predictions remain a challenge due to the inability to perceive series correlations by existing machine learning (ML) models. Herein, a ML model named cluster-based series graph networks (CSGN) is developed based on the dynamical theory of crystal lattices to predict phonon density of states (PDOS) spectrum for crystal materials. The multiple atomic cluster representation is constructed to capture the diverse vibration modes, while the mixture Gaussian process and dynamic time warping mechanism are compiled to project from clusters to PDOS spectrum. Accurate predictions of complicated spectra with multiple or overlapping peaks are achieved. The high performance of CSGN model can be attributed to the pertinent feature extraction and the appropriate similarity evaluation, which enable the natural perception of structure-property relation and intrinsic series correlations as confirmed in the predictive results. The transferable and interpretable CSGN model advances ML predictions of spectral properties and reveals the potential of designing ML methods based on physical mechanisms.