Gaussian Process Research Articles

Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling.

In this paper, we propose FGPR: a Federated Gaussian process ( GP) regression framework that uses an averaging strategy for model aggregation and stochastic gradient descent for local computations. Notably, the resulting global model excels in personalization as FGPR jointly learns a shared prior across all devices. The predictive posterior is then obtained by exploiting this shared prior and conditioning on local data, which encodes personalized features from a specific dataset. Theoretically, we show that FGPR converges to a critical point of the full log-marginal likelihood function, subject to statistical errors. This result offers standalone value as it brings federated learning theoretical results to correlated paradigms. Through extensive case studies on several regression tasks, we show that FGPR excels in a wide range of applications and is a promising approach for privacy-preserving multi-fidelity data modeling.

Multiobjective optimization of carbon fiber–reinforced epoxy composite properties using nonparametric modeling

Carbon fiber–reinforced epoxy composites are widely used in the marine and aviation sectors due to their superior mechanical properties. As research into these materials progresses, the demand for materials with even more complex and superior properties has increased. Traditional trial-and-error methods are limited to optimizing single factors and are plagued by heavy workloads and prohibitive costs when applied to multifactor composite designs. Addressing these problems, this study introduces a nonparametric multiobjective optimization process for composite materials. This approach employs a Gaussian process (GP) to establish a multiobjective regression model and uses nondominated sorting genetic algorithm II (NSGA II) to determine the Pareto front. Pareto solutions were ranked in accordance with their similarity to ideal solutions and on the basis of predetermined preference weights. The results demonstrate variations in optimal outcomes contingent on the weight values assigned. The differences between the optimized results and experimental validations reached 12.9%, with the minimum deviation being 0.6%. The effectiveness of the proposed method was demonstrated. The strategy combines GP and NSGA II to extend design methods for small samples and materials with multiple attributes. This method enables the simultaneous optimization of multiple conflicting objectives, offering greater efficiency than addressing them individually.

Machine learning assisted improved desalination pilot system design and experimentation for the circular economy

Desalination is among the most feasible solutions to supply sustainable and clean drinking water in water scarcity areas. In this regard, Multi-Effect Desalination (MED) systems are particularly preferred for harsh feeds (high temperature and salinity) because of their robust mode of operation for water production. However, maintaining the efficient operation of the MED systems is challenging because of the large system design and variables' interdependencies that are sensitive to the distillate production. Therefore, this research leverages the power of machine learning and optimization to estimate the optimal operating conditions for the maximum distillate production from the MED system. In the first step, detailed experimentation is conducted for distillate production against hot water temperature (HWT) varying from 38 to 70 °C, and feed water temperature (FWT) is changed from 34 to 42 °C. Whereas, the feed flow rate (FFR) is investigated to be varied nearly from 3.6 to 8.7 LPM in the three stages, i.e., FFR-S1, FFR-S2 and FFR-S3. The compiled dataset is used to make the process models of the MED system by three ML-based algorithms, i.e., Artificial Neural Network (ANN), Support Vector Machine (SVM), and Gaussian Process Regression (GPR) under rigorous hyperparameters optimization. GPR exhibited superior predictive performance than those of ANN and SVM on R2 value of 0.99 and RMSE of 0.026 LPM. Monte Carlo technique-based variable significance analysis revealed that the HWT has the highest effect on distillate production with a percentage significance of 95.6 %. Then Genetic Algorithm is used to maximize the distillate production with the GPR model embedded in the optimization problem. The GPR-GA driven maximum distillate production is estimated on HWT = 70 ± 0.5 °C, FWT = 40 ± 2.5 °C, FFR-S1 = 6 ± 2.6 LPM, FFR-S2 = 7 ± 1 LPM and FFR-S3 = 7 ± 1. The ML-GA-based system analysis and optimization of the MED system can boost the distillate production that promotes operation excellence and circular economy from the desalination sector.

A low-mass sub-Neptune planet transiting the bright active star HD 73344

Planets with radii of between $2$ and $4 R_ closely orbiting solar-type stars are of significant importance for studying the transition from rocky to giant planets, and are prime targets for atmospheric characterization by missions such as JWST and ARIEL. Unfortunately, only a handful of examples with precise mass measurements are known to orbit bright stars. Our goal is to determine the mass of a transiting planet around the very bright F6 star HD 73344 (Vmag=6.9). This star exhibits high activity and has a rotation period that is close to the orbital period of the planet ($P_ b =15.6$ days). The transiting planet, initially a K2 candidate, is confirmed through TESS observations (TOI 5140.01). We refined its parameters using TESS data and rule out a false positive with Spitzer observations. We analyzed high-precision radial velocity (RV) data from the SOPHIE and HIRES spectrographs. We conducted separate and joint analyses of K2, TESS, SOPHIE, and HIRES data using the PASTIS software. Given the star's early type and high activity, we used a novel observing strategy, targeting the star at high cadence for two consecutive nights with SOPHIE to understand the short-term stellar variability. We modeled stellar noise with two Gaussian processes: one for rotationally modulated stellar processes, and one for short-term stellar variability. High-cadence RV observations provide better constraints on stellar variability and precise orbital parameters for the transiting planet: a radius of $R_ b R_ and a mass of $M_ b M_ (upper-limit at $3 is $<10.48 M_ oplus$) . The derived mean density suggests a sub-Neptune-type composition, but uncertainties in the planet's mass prevent a detailed characterization. In addition, we find a periodic signal in the RV data that we attribute to the signature of a nontransiting exoplanet, without totally excluding the possibility of a nonplanetary origin. This planetary candidate would have a minimum mass of about $M_ c i_ c M_ and a period of $P_ c $ days. Dynamical analyses confirm the stability of the two-planet system and provide constraints on the inclination of the candidate planet; these findings favor a near-coplanar system. While the transiting planet orbits the bright star at a short period, stellar activity prevented us from precise mass measurements despite intensive RV follow-up. Long-term RV tracking of this planet could improve this measurement, as well as our understanding of the activity of the host star. The latter will be essential if we are to characterize the atmosphere of planets around F-type stars using transmission spectroscopy.

A Bayesian Framework to Quantify Uncertainty in Aerosol Optical Model Selection Applied to TROPOMI Measurements

This article presents a method within a Bayesian framework for quantifying uncertainty in satellite aerosol remote sensing when retrieving aerosol optical depth (AOD). By using a Bayesian model averaging technique, we take into account uncertainty in aerosol optical model selection and also obtain a shared inference about AOD based on the best-fitting optical models. In particular, uncertainty caused by forward-model approximations has been taken into account in the AOD retrieval process to obtain a more realistic uncertainty estimate. We evaluated a model discrepancy, i.e., forward-model uncertainty, empirically by exploiting the residuals of model fits and using a Gaussian process to characterise the discrepancy. We illustrate the method with examples using observations from the TROPOspheric Monitoring Instrument (TROPOMI) on the Sentinel-5 Precursor satellite. We evaluated the results against ground-based remote sensing aerosol data from the Aerosol Robotic Network (AERONET).

On the effect of stiffness discontinuities on the reeling of pipelines using FEM and machine learning strategy

ABSTRACT Under current DNV specifications, pipeline design for reeling installation involves extensive FE simulations considering wall thickness and yield stress mismatch. This process is laborious, requiring multiple setups and trial-and-error calculations, with each simulation taking several hours. This paper proposes a machine learning strategy to study the mismatch design during the reeling process. Four machine learning models—Gaussian Process regression, Multilayer Perceptron, Support Vector Machine, and Random Forest—are used. Parameters that may affect failure, including yield strength, wall thickness, hardening coefficient, back tension, reel diameter, and pipe diameter are considered as input features. FE analyses simulate the reeling process, extracting output parameters such as maximum ovality, peak strain, local curvature, and lift-off for training. Based on careful calibration, it is shown to form an accurate and efficient regression framework, which only takes less than one minute to finish, exhibiting significant advantages over the conventional process.

Bayesian dynamic modelling for probabilistic prediction of pavement condition

Significant funds have been allocated to maintain road networks each year in developed countries. Performance prediction is crucial for pavement management systems to adjust working plans and budget allocation. As a Bayesian nonparametric method, Gaussian process regression (GPR) is powerful in predicting nonlinear time series and quantifying uncertainty. However, it remains computationally intensive and fails to adapt to the time-varying characteristics. To address such issues, a dynamic GPR model is proposed for probabilistic prediction of the International Roughness Index (IRI) for flexible pavements. A moving window strategy is developed to substantially shrink the size of training data, which effectively alleviates computational cost and thus leads to a dynamic GPR. A genetic algorithm is then adopted to determine the optimal window size by considering the trade-off between computational efficiency and accuracy. A dataset acquired from Long-Term Pavement Performance (LTPP) is used to demonstrate the feasibility of the dynamic GPR. Its performance is compared to traditional GPR as well as dynamic and static Bayesian linear regression (BLR) models. The comparison results indicate that the proposed dynamic GPR can increase the accuracy by 0.86, 1.52, and 2.27 times for dynamic BLR, static GPR, and static BLR, respectively. It exhibits the best results in terms of accuracy and uncertainty metrics due to its nonlinear modelling and time-varying ability.

Utilizing undisturbed soil sampling approach to predict elastic modulus of cohesive soils: a Gaussian process regression model

Utilizing undisturbed soil sampling approach to predict elastic modulus of cohesive soils: a Gaussian process regression model

Sample size determination for prediction models via learning-type curves.

This article is concerned with sample size determination methodology for prediction models. We propose to combine the individual calculations via learning-type curves. We suggest two distinct ways of doing so, a deterministic skeleton of a learning curve and a Gaussian process centered upon its deterministic counterpart. We employ several learning algorithms for modeling the primary endpoint and distinct measures for trial efficacy. We find that the performance may vary with the sample size, but borrowing information across sample size universally improves the performance of such calculations. The Gaussian process-based learning curve appears more robust and statistically efficient, while computational efficiency is comparable. We suggest that anchoring against historical evidence when extrapolating sample sizes should be adopted when such data are available. The methods are illustrated on binary and survival endpoints.

A Comprehensive Survey of Predictive Maintenance Techniques for Aircraft Engines Utilizing the C-MAPSS Dataset

The application of deep learning and sophisticated machine learning techniques is driving the rapid advancement of aircraft engine prognostics and predictive maintenance. Remain ing Useful Life (RUL) of aviation engines has been the subject of numerous studies aimed at improving prediction accuracy and efficacy to improve aviation safety and maintenance plans. Innovative approaches and technologies are demonstrated by these projects, which use a variety of methodologies and datasets, including C-MAPSS and N-CMAPSS. Combining feature engi neering, ensemble learning, and deep learning models such as Restricted Boltzmann Machines (RBMs), Long Short-Term Mem ory (LSTM) networks, Convolutional Neural Networks (CNNs), and Deep Bidirectional Recurrent Neural Networks (DBRNNs) is one prominent method. Features are chosen and models are optimized using a variety of methods, including Genetic Algorithms, Recursive Feature Elimination, Lasso, and Feature Importances. In prognostic modeling, the research emphasize the need of interpretability, model adaptability, and measuring uncertainty. Additionally, in order to pinpoint important features and improve model transparency, the study investigates the use of explainable AI techniques such aggregated feature importances with cross-validation (AFICv) and Shapley additive explanation (SHAP). In order to capture prediction uncertainties, the inte gration of Gaussian Processes (GPs) and Bayesian Deep Neural Networks (DNNs) is also investigated. This provides insights into uncertainty-aware prognosis and predictive analytics for industrial assets. The development and publication of datasets such as the N-CMAPSS dataset also makes it possible to conduct more comprehensive and realistic assessments of prognostic models under real-world flight conditions, providing useful tools for benchmarking and improving machine learning algorithms in predictive maintenance. All things considered, these research projects highlight current developments and the possibility of combining cutting edge technology to improve system reliability, improve predictive maintenance techniques, and guarantee safer airline operations. Key Words: Aircraft engine, Prognostics, Predictive main tenance, Machine learning, Deep learning, Remaining Useful Life (RUL), Aviation safety, Feature engineering, Ensemble learning, Uncertainty quantification, Explainable AI, Bayesian Deep Neural Networks (DNNs), Gaussian Processes (GPs), N CMAPSS dataset, Real flight conditions, Industrial assets, System reliability, Benchmarking

Towards autonomous learning and optimisation in textile production: data-driven simulation approach for optimiser validation

AbstractThe textile industry is a traditional industry branch that remains highly relevant in Europe. The industry is under pressure to remain profitable in this high-wage region. As one promising approach, data-driven methods can be used for process optimisation in order to reduce waste, increase profitability and relieve mental burden on staff members. However, approaches from research rarely get adopted into practice. We identify the high dimensionality of textile production processes leading to high model uncertainty as well as an incomplete problem formulation as the two main problems. We argue that some form of an autonomous learning agent can address this challenge, when it safely explores advantageous, unknown new settings by interacting with the process. Our main goal is to facilitate the adoption of promising research into practical applications. The main contributions of this paper include the derivation and formulation of a probabilistic optimisation problem for high-dimensional, stationary production processes. We also create a highly adaptable simulation of the textile carded nonwovens production process in Python that implements the optimisation problem. Economic and technical behavior of the process is approximated using both Gaussian Process Regression (GPR) models trained with industrial data as well as physics-motivated explicit models. This ’simulation first’-approach makes the development of autonomous learning agents for practical applications feasible because it allows for cheap testing and validation before physical trials. Future work will include the comparison of the performance of different agent approaches.

Leveraging the concept of information entropy to improve a multi-fidelity design framework for early-stage design exploration of complex vessels

Early-stage design exploration is crucial since most of the major design decision are locked-in and only small design modifications are possible at later stages. To assess the performance of the various design candidates while performing design exploration, there are available methods and tools of various fidelities. These methods can be combined to form a multi-fidelity (MF) framework that guarantees accuracy through the high-fidelity model and achieves faster computational speeds through low-fidelity models. The present study proposes the adoption of information-theoretic entropy to improve a MF design framework based on Gaussian Processes (GPs). Entropy quantifies the uncertainty associated with the prediction of the design space. We propose using this uncertainty metric both as a criterion to determine whether further designs should be sampled to construct a reliable approximation of the design space and as a criterion to establish in which optimization step the optimization of the covariance matrix for the MF-GPs should be performed. The approach was tested to benchmark analytical functions and to a ship design problem of an AXEfrigate. The approach holds potential in practical applications, as it aids in the determination of whether additional resources should be allocated for high-fidelity analysis to support early-stage exploration.

Integrated form-position measurement of large-aperture transparent elements based on stereoscopic phase measuring deflectometry

Abstract In the field of ultra-precision manufacturing and measurement, sub-aperture stitching is a widely employed technique for the measurement of complex optical components. Those influencing factors like system errors and numerical bias introduce notable stitching errors in the overlapped areas between sub-apertures, consequently degrading measurement accuracy. In addition, the surfaces forms, refractive indices, and positions are difficult to be specified simultaneously. In this paper, an integrated measurement method based on the stereo deflectometry is proposed, which can measure the surface form, refractive index, and relative position of large-aperture transparent components together. This method utilizes a Gaussian process regression model to decouple and predict the refractive index and relative positions of the upper and lower surfaces. Then, a robust sub-aperture stitching technique with an adaptive distance function is employed to correct the positioning errors of sub-apertures. The feasibility and effectiveness of this method are demonstrated. The final stitched surface results exhibit a measurement error of 320 nm and a thickness deviation of 28 µm.

Dependency-aware deep generative models for multitasking analysis of spatial omics data.

Spatially resolved transcriptomics (SRT) technologies have significantly advanced biomedical research, but their data analysis remains challenging due to the discrete nature of the data and the high levels of noise, compounded by complex spatial dependencies. Here, we propose spaVAE, a dependency-aware, deep generative spatial variational autoencoder model that probabilistically characterizes count data while capturing spatial correlations. spaVAE introduces a hybrid embedding combining a Gaussian process prior with a Gaussian prior to explicitly capture spatial correlations among spots. It then optimizes the parameters of deep neural networks to approximate the distributions underlying the SRT data. With the approximated distributions, spaVAE can contribute to several analytical tasks that are essential for SRT data analysis, including dimensionality reduction, visualization, clustering, batch integration, denoising, differential expression, spatial interpolation, resolution enhancement and identification of spatially variable genes. Moreover, we have extended spaVAE to spaPeakVAE and spaMultiVAE to characterize spatial ATAC-seq (assay for transposase-accessible chromatin using sequencing) data and spatial multi-omics data, respectively.

Predicting weld pool metrics in laser welding of aluminum alloys using data-driven surrogate modeling: A FEA-DoE-GPRN hybrid approach

Multi-physics computational models based on finite element analysis, offer detailed insights into the dynamics and metrics in the weld pool formed by laser welding. Conversely, data-driven surrogate models provide a cost-effective means to predict desired responses. These models establish statistical or mathematical correlations with input–output data, eliminating the need for additional simulations during design optimization. This study proposes a data-driven surrogate model, employing the Gaussian process regression network (GPRN), to predict weld pool metrics, such as weld width and depth of penetration in laser welding of aluminum alloy. A 3D computational fluid dynamics-based numerical model is initially constructed and experimentally validated to predict weld pool metrics. Subsequent experimental runs, guided by the design of experiments, include various configurations of process parameter settings. The developed numerical model computes weld pool metrics for each experimental run, forming a dataset for training and testing the GPRN model. The GPRN model is evaluated against simulated data, showing adequacy with a mean square error of 1.7 µm and mean absolute percentage error of 10−7, with experimental validation further confirming its accuracy, revealing a minimum error of 1.7%, a maximum error of 8%, and an average error of 3%. The key contribution and novelty of this study lie in the development of the hybrid data-driven model, which accurately predicts weld pool metrics while minimizing experimental and computational efforts.

Mitigating Measurement Inaccuracies in Digital Twins of Construction Machinery through Multi-Objective Optimization

The advent of digital twins facilitates the generation of high-fidelity replicas of actual systems or assets, thereby enhancing the design’s performance and feasibility. When developing digital twins, precise measurement data is essential to ensure alignment between the actual and digital models. However, inherent uncertainties in sensors and models lead to disparities between observed and predicted (simulated) behaviors. To mitigate these uncertainties, this study originally proposes a multi-objective optimization strategy utilizing a Gaussian process regression surrogate model, which integrates various uncertain parameters, such as load angle, bucket cylinder stroke, arm cylinder stroke, and boom cylinder stroke. This optimization employs a genetic algorithm to indicate the Pareto frontiers regarding the pressure exerted on the boom, arm, and bucket cylinders. Subsequently, TOPSIS is applied to ascertain the optimal candidate among the identified Pareto optima. The findings reveal a substantial congruence between the experimental and numerical outcomes of the devised virtual model, in conjunction with the TOPSIS-derived optimal parameter configuration.

Multifidelity-based Gaussian process for quasi-site-specific probabilistic prediction of soil properties

Multifidelity-based Gaussian process for quasi-site-specific probabilistic prediction of soil properties

Augmenting a Simulation Campaign for Hybrid Computer Model and Field Data Experiments

The Kennedy and O’Hagan (KOH) calibration framework uses coupled Gaussian processes (GPs) to meta-model an expensive simulator (first GP), tune its “knobs” (calibration inputs) to best match observations from a real physical/field experiment and correct for any modeling bias (second GP) when predicting under new field conditions (design inputs). There are well-established methods for placement of design inputs for data-efficient planning of a simulation campaign in isolation, that is, without field data: space-filling, or via criterion like minimum integrated mean-squared prediction error (IMSPE). Analogues within the coupled GP KOH framework are mostly absent from the literature. Here we derive a closed form IMSPE criterion for sequentially acquiring new simulator data for KOH. We illustrate how acquisitions space-fill in design space, but concentrate in calibration space. Closed form IMSPE precipitates a closed-form gradient for efficient numerical optimization. We demonstrate that our KOH-IMSPE strategy leads to a more efficient simulation campaign on benchmark problems, and conclude with a showcase on an application to equilibrium concentrations of rare earth elements for a liquid–liquid extraction reaction.

Infinite-dimensional distances and divergences between positive definite operators, Gaussian measures, and Gaussian processes

Infinite-dimensional distances and divergences between positive definite operators, Gaussian measures, and Gaussian processes

Advancing Federated Learning by Addressing Data and System Heterogeneity

In the emerging field of federated learning (FL), the challenge of heterogeneity, both in data and systems, presents significant obstacles to efficient and effective model training. This talk focuses on the latest advancements and solutions addressing these challenges. The first part of the talk delves into data heterogeneity, a core issue in FL, where data distributions across different clients vary widely and affect FL convergence. We will introduce the FedCor framework addressing this by modeling loss correlations between clients using Gaussian Process and reducing expected global loss. External covariate shift in FL is uncovered, demonstrating that normalization layers are crucial, and layer normalization proves effective. Additionally, class imbalance in FL degrades performance, but our proposed Federated Class-balanced Sampling (Fed-CBS) mechanism reduces this imbalance by employing homomorphic encryption for privacy preservation. The second part of the talk shifts focus to system heterogeneity, an equally critical challenge in FL. System heterogeneity involves the varying computational capabilities, network speeds, and other resource-related constraints of participating devices in FL. To address this, we introduce FedSEA, which is a semi-asynchronous FL framework that addresses accuracy drops by balancing aggregation frequency and predicting local update arrival. Additionally, we discuss FedRepre, a framework specifically designed to enhance FL in real-world environments by addressing challenges including unbalanced local dataset distributions, uneven computational capabilities, and fluctuating network speeds. By introducing a client selection mechanism and a specialized server architecture, FedRepre notably improves the efficiency, scalability, and performance of FL systems. Our talk aims to provide a comprehensive overview of the current research and advancements in tackling both data and system heterogeneity in federated learning. We hope to highlight the path forward for FL, underlining its potential in diverse real-world applications while maintaining data privacy and optimizing resource usage.