Multivariate Gaussian Process Regression Research Articles

Unraveling the binary nature of HQ Tau

Context. Both the stellar activity and the accretion processes of young stellar objects can induce variations in their radial velocity (RV). This variation is often modulated on the stellar rotation period and may hide a RV signal from a planetary or even a stellar companion. Aims. The aim of this study is to detect the companion of HQ Tau, the existence of which is suspected based on our previous study of this object. We also aim to derive the orbital elements of the system. Methods. We used multi-variate Gaussian process regression on the RV and the bisector inverse slope of a six-month high-resolution spectroscopic follow-up observation of the system to model the stellar activity. This allowed us to extract the Keplerian RV modulation induced by the suspected companion. Results. Our analysis yields the detection of a ∼50 Mjup brown dwarf companion orbiting HQ Tau with a ∼126 day orbital period. Although this is consistent with the modulation seen on this dataset, it does not fit the measurements from our previous work three years earlier. In order to include these measurements in our analysis, we hypothesise the presence of a third component with orbital elements that are consistent with those of the secondary according to our previous analysis (MB ∼ 48 Mjup, Porb, B ∼ 126 days), and a ∼465 Mjup tertiary with a ∼767 day orbital period. However, the hypothesis of a single companion with MB ∼ 188 Mjup and Porb ∼ 247 days can fit both datasets and cannot be completely excluded at this stage of the analysis. Conclusions. At minima, HQ Tau is a single-lined spectroscopic binary, and several factors indicate that the companion is a brown dwarf and that a third component is responsible for larger RV variation on a longer timescale.

Multivariate Gaussian process surrogates for predicting basic structural parameters of refractory non-dilute random alloys

Refractory non-dilute random alloys consist of two or more principal refractory metals with complex interactions that modify their basic structural properties such as lattice parameters and elastic constants. Atomistic simulations (ASs) are an effective method to compute such basic structural parameters. However, accurate predictions from ASs are computationally expensive due to the size and number of atomistic structures required. To reduce the computational burden, multivariate Gaussian process regression (MVGPR) is proposed as a surrogate model that only requires computing a small number of configurations for training. The elemental atom percentage in the hyper-spherical coordinates is demonstrated to be an effective feature for surrogate modeling. An additive approximation of the full MVGPR model is also proposed to further reduce computations. To improve surrogate accuracy, active learning is used to select a small number of alloys to simulate. Numerical studies based on AS data show the accuracy of the surrogate methodology and the additive approximation, as well as the effectiveness and robustness of the active learning for selecting new alloy designs to simulate.

A multiscale computational framework using active learning to model complex suspension flows

We devise a novel computational framework to achieve efficient multiscale modeling for suspension flows. The modeling framework comprises a particle-resolving direct numerical simulation for microscopic computation, a single-fluid continuum model to capture the bulk flow behavior at the macroscale, and a Gaussian process regression that connects the two modeling components. The microscopic calculation simulates detailed flow fields around individual suspended particles. Along with a coarse-grained method, we are able to obtain the mean volume fractions and stresses for discrete particles, which are then further used to calculate the rheological properties of the bulk flow fields at the macroscale under specific forcing conditions. Using Gaussian process regression, a few data points from the microscopic calculations can be further interpolated/extrapolated to form complete constitutive relationships that cover the whole forcing range in macroscopic calculations (i.e., continuum modeling). Moreover, via the uncertainties returned from the Gaussian process regression, the resulting active learning strategy using the Gaussian process regression automatically decides on the required the microscopic calculations given by a specified tolerance. As a result, complex suspension problems can be efficiently solved by the macroscopic continuum model to the desired accuracy with the minimum computational effort (i.e., minimum number of microscopic runs). Four examples are demonstrated, among which two examples involve particle migration as a function of the particle stress, and the resulting nonuniform distribution of particles. This demands a multivariate Gaussian process regression, which is found to be particularly effective in achieving high computational efficiency compared with the off-line cases, i.e., a large number of off-line microscopic calculations to obtain the prescribed constitutive relationships.

Crowdsourced Indoor Positioning with Scalable WiFi Augmentation.

In recent years, crowdsourcing approaches have been proposed to record the WiFi signals annotated with the location of the reference points (RPs) extracted from the trajectories of common users to reduce the burden of constructing a fingerprint (FP) database for indoor positioning. However, crowdsourced data is usually sensitive to crowd density. The positioning accuracy degrades in some areas due to a lack of FPs or visitors. To improve the positioning performance, this paper proposes a scalable WiFi FP augmentation method with two major modules: virtual reference point generation (VRPG) and spatial WiFi signal modeling (SWSM). A globally self-adaptive (GS) and a locally self-adaptive (LS) approach are proposed in VRPG to determine the potential unsurveyed RPs. A multivariate Gaussian process regression (MGPR) model is designed to estimate the joint distribution of all WiFi signals and predicts the signals on unsurveyed RPs to generate more FPs. Evaluations are conducted on an open-source crowdsourced WiFi FP dataset based on a multi-floor building. The results show that combining GS and MGPR can improve the positioning accuracy by 5% to 20% from the benchmark, but with halved computation complexity compared to the conventional augmentation approach. Moreover, combining LS and MGPR can sharply reduce 90% of the computation complexity against the conventional approach while still providing moderate improvement in positioning accuracy from the benchmark.

An adaptive multivariate Student's t‐process recursive method for hypersonic glide vehicle trajectory prediction

AbstractThe trajectory prediction of the hypersonic glide vehicle (HGV) can provide hit point information for early warning systems, which is of great significance for near‐space defence operations. However, the heavy‐tailed noise caused by abnormal environmental disturbance and abrupt changes in vehicle trajectory seriously affects the accuracy of HGV trajectory prediction. To solve the problem of trajectory prediction for HGV under heavy‐tailed noise, we propose an adaptive multivariate Student's t‐process regression method called aEM‐MVTPR. Firstly, a heavy‐tailed noise model is developed using the Student's t‐distribution. Secondly, a multivariate Student's t‐process regression (MVTPR) for HGV trajectory prediction is derived, and the method can learn the HGV trajectory time series features and mine the correlations among the trajectory variables. Finally, to further improve the method's robustness, we use the accelerated expectation maximisation algorithm and Pearson correlation analysis to adaptively estimate and adjust the initial values of the MVTPR parameters. The simulation experimental results show that the proposed method has more accurate predicted values of trajectory variables than multivariate Gaussian process regression (MVGPR) under heavy‐tailed noise conditions. In addition, the ability of the aEM‐MVTPR to adaptively adjust the parameter makes it more robust than the MVTPR under different noise environments.

Multivariate Gaussian processes: definitions, examples and applications

Gaussian processes occupy one of the leading places in modern statistics and probability theory due to their importance and a wealth of strong results. The common use of Gaussian processes is in connection with problems related to estimation, detection, and many statistical or machine learning models. In this paper, we propose a precise definition of multivariate Gaussian processes based on Gaussian measures on vector-valued function spaces, and provide an existence proof. In addition, several fundamental properties of multivariate Gaussian processes, such as stationarity and independence, are introduced. We further derive two special cases of multivariate Gaussian processes, including multivariate Gaussian white noise and multivariate Brownian motion, and present a brief introduction to multivariate Gaussian process regression as a useful statistical learning method for multi-output prediction problems.

Research on surface roughness prediction in turning Inconel 718 based on Gaussian process regression

Nickel-based alloy Inconel 718 is widely used in aircraft engine industry because of its good mechanical properties. Inconel 718 is a typical difficult-to-machine material and its price is relatively expensive. Therefore, accurate prediction of Inconel 718 machined surface roughness with small sample space can improve machining efficiency, optimize process parameters and reduce machining cost. In this paper, a method is proposed to characterize the influence of cutting parameters on roughness by stablishing the corresponding relationship between the proportional hyperparameters in the multivariate kernel function and the cutting speed, cutting deep, feed rate and the rake angle of the tool. A multi input single output (MISO) multivariate Gaussian process regression (GPR) surface roughness prediction model with cutting speed, cutting depth, feed rate and tool rake angle as input variables and surface roughness as output variables is established. The model can not only output the predicted value of surface roughness, but also give the reliability of the predicted value. Experimental results show that the proportional hyperparameter has an independent adjustment function, and the influence of the process parameters characterized by the proportional hyperparameter on the surface roughness is consistent with the experimental results. The experimental results show that the average relative error of MISO multivariate GPR surface roughness prediction model proposed in this paper is 1.5%, which can accurately predict the surface roughness in small sample space.

Study of fluid–thermal–structural interaction in high-temperature high-speed flow using multi-fidelity multi-variate surrogates

This study investigates the impact of the high-temperature effect, especially the real gas effect and chemical reactions, on hypersonic aerothermodynamic solutions of double cone and double wedge configurations, as well as the fluid–thermal–structural interaction of a double wedge configuration in hypersonic flow. First, a high-temperature computational fluid dynamics (CFD) code was benchmarked and correlated with experimental results, emphasizing the impact of high-temperature effects as well as turbulence modeling on heat flux prediction. Subsequently, the multi-fidelity multi-variate Gaussian process regression (M2GPR ) method for problems with high-dimensional outputs was developed to create an aerothermal surrogate model. The model achieves a balance between model accuracy and computational cost of sample generation, using the combination of a few high-fidelity samples and many low-fidelity samples. The numerical examples show that, using the M2GPR formulation, the required number of high-fidelity samples may be reduced by over 80% while maintaining an accuracy comparable to the high-fidelity CFD solvers. In addition, a geodesic-distance-based metric is developed to inform the choice of high-dimensional datasets of different fidelities for the M2GPR surrogate with improved accuracy. Finally, the aerothermal surrogate was applied to study the impact of the high-temperature effect on the aerothermoelastic response of a hypersonic skin panel, emphasizing the necessity of the accurate characterization of the localized heat flux for reasonable assessment of the response of a compliant structure in high-speed high-temperature flowfield.

Diagnosis of obstructive sleep apnea with prediction of flow characteristics according to airway morphology automatically extracted from medical images: Computational fluid dynamics and artificial intelligence approach

BackgroundObstructive sleep apnea syndrome (OSAS) is being observed in an increasing number of cases. It can be diagnosed using several methods such as polysomnography. ObjectivesTo overcome the challenges of time and cost faced by conventional diagnostic methods, this paper proposes computational fluid dynamics (CFD) and machine-learning approaches that are derived from the upper-airway morphology with automatic segmentation using deep learning. MethodWe adopted a 3D UNet deep-learning model to perform medical image segmentation. 3D UNet prevents the feature-extraction loss that may occur by concatenating layers and extracts the anteroposterior coordination and width of the airway morphology. To create flow characteristics of the upper airway training data, we analyzed the changes in flow characteristics according to the upper-airway morphology using CFD. A multivariate Gaussian process regression (MVGPR) model was used to train the flow characteristic values. The trained MVGPR enables the prompt prediction of the aerodynamic features of the upper airway without simulation. Unlike conventional regression methods, MVGPR can be trained by considering the correlation between the flow characteristics. As a diagnostic step, a support vector machine (SVM) with predicted aerodynamic and biometric features was used in this study to classify patients as healthy or suffering from moderate OSAS. SVM is beneficial as it is easy to learn even with a small dataset, and it can diagnose various flow characteristics as factors while enhancing the feature via the kernel function. As the patient dataset is small, the Monte Carlo cross-validation was used to validate the trained model. Furthermore, to overcome the imbalanced data problem, the oversampling method was applied. ResultThe segmented upper-airway results of the high-resolution and low-resolution models present overall average dice coefficients of 0.76±0.041 and 0.74±0.052, respectively. Furthermore, the classification accuracy, sensitivity, specificity, and F1-score of the diagnosis algorithm were 81.5%, 89.3%, 86.2%, and 87.6%, respectively. ConclusionThe convenience and accuracy of sleep apnea diagnosis are improved using deep learning and machine learning. Further, the proposed method can aid clinicians in making appropriate decisions to evaluate the possible applications of OSAS.

Multivariate Gaussian and Student-t process regression for multi-output prediction

Gaussian process model for vector-valued function has been shown to be useful for multi-output prediction. The existing method for this model is to reformulate the matrix-variate Gaussian distribution as a multivariate normal distribution. Although it is effective in many cases, reformulation is not always workable and is difficult to apply to other distributions because not all matrix-variate distributions can be transformed to respective multivariate distributions, such as the case for matrix-variate Student-t distribution. In this paper, we propose a unified framework which is used not only to introduce a novel multivariate Student-t process regression model (MV-TPR) for multi-output prediction, but also to reformulate the multivariate Gaussian process regression (MV-GPR) that overcomes some limitations of the existing methods. Both MV-GPR and MV-TPR have closed-form expressions for the marginal likelihoods and predictive distributions under this unified framework and thus can adopt the same optimization approaches as used in the conventional GPR. The usefulness of the proposed methods is illustrated through several simulated and real-data examples. In particular, we verify empirically that MV-TPR has superiority for the datasets considered, including air quality prediction and bike rent prediction. At last, the proposed methods are shown to produce profitable investment strategies in the stock markets.

Robust Optimization of Industrial Process Operation Parameters Based on Data-Driven Model and Parameter Fluctuation Analysis

The fluctuation of industrial process operation parameters will severely influence the production process. How to find the robust optimal process operation parameters is an effective method to address this problem. In this paper, a scheme based on data-driven model and variable fluctuation analysis is proposed to obtain the robust optimal operation parameters of industrial process. The data-driven modelling method: multivariate Gaussian process regression (MGPR) based on Bayesian statistical learning theory can map the process operation parameters to objective performance with the flexibility in nonparameter inferring and the self-adaptiveness to determinate hyperparameters. According to the minimum variance criterion, the parameter fluctuation analysis can be performed through multiobjective evolutionary algorithm based on the MGPR model. To analyze the robustness influence of a single parameter, cross validation is applied to evaluate the model output with 2% fluctuation. After that, the robust optimal process operation parameters can be obtained and applied to guide the production. The effectiveness and reliability of the proposed method have been verified with the hydrogen cyanide production process and compared with other model methods and single objective optimization method.

Human Intention Understanding From Multiple Demonstrations and Behavior Generalization in Dynamic Movement Primitives Framework

Human’s interference in the process of skill learning can improve the performance of the robot greatly. However, learning from demonstration to generate a new action with human behavioral characteristics in the varying situation is challenging. Generally, dynamic movement primitives (DMPs) method can generalize the trajectory imitating the demonstration, but cannot integrate the feature of multiple trajectories of different targets. In this paper, the proposed method contains two aspects of learning and generating. The statistical method Gaussian mixture model and Gaussian mixture regression (GMM-GMR) is used to extract the common characteristic and eliminate the uncertainty of the multiple demonstrations. To exert the ability of DMPs to generate a human-like motion to a new goal, and we model the shape parameter with locally weighted regression (LWR) method. To enhance the ability of DMPs in multiple trajectories learning, we propose the multivariate Gaussian process regression (MV-GPR) method to construct the regression model of shape parameters to reflect the human intentions, with respect to the target position. To verify the feasibility of the proposed method, we design a peg-in-hole experiment with proving generalization and obstacle avoidance performance. The results have shown that the strategy integrated the generalization of DMPs and feature regeneration ability of MV-GPR method, and the generated valid trajectory could achieve the peg-in-hole task with 6-DOF whole-arm avoidance.

Multivariate Gaussian process regression for nonlinear modelling with colored noise

Nonlinearity of process systems along with colored noises is common in chemical processes. A multivariate (multiple inputs and multiple outputs) Gaussian process regression (MGPR) modelling approach, which can model multivariate nonlinear processes, is developed in this paper. The developed GPR model considers the Gaussian colored noise, rather than the traditional Gaussian white noise. The colored noise is described by the moving average (MA) model and the autoregressive (AR) model, respectively, with unknown parameters so that a MA-GPR model and an AR-GPR model are developed. These two colored noise based models are further extended to the MGPR model to generate the MA-MGPR model and the AR-MGPR model. The covariance functions of the MA-MGPR model or the AR-MGPR model are formulated with consideration of the autocorrelation of noises. Moreover, all parameters are estimated by using a unidimensional updated particle swarm optimization (PSO) algorithm, simultaneously. Numerical examples as well as a three-level drawing model of Carbon fiber production process are used to demonstrate the effectiveness of the proposed modelling approaches.

Multi-model multivariate Gaussian process modelling with correlated noises

A composite multiple-model approach based on multivariate Gaussian process regression (MGPR) with correlated noises is proposed in this paper. In complex industrial processes, observation noises of multiple response variables can be correlated with each other and process is nonlinear. In order to model the multivariate nonlinear processes with correlated noises, a dependent multivariate Gaussian process regression (DMGPR) model is developed in this paper. The covariance functions of this DMGPR model are formulated by considering the “between-data” correlation, the “between-output” correlation, and the correlation between noise variables. Further, owing to the complexity of nonlinear systems as well as possible multiple-mode operation of the industrial processes, to improve the performance of the proposed DMGPR model, this paper proposes a composite multiple-model DMGPR approach based on the Gaussian Mixture Model algorithm (GMM-DMGPR). The proposed modelling approach utilizes the weights of all the samples belonging to each sub-DMGPR model which are evaluated by utilizing the GMM algorithm when estimating model parameters through expectation and maximization (EM) algorithm. The effectiveness of the proposed GMM-DMGPR approach is demonstrated by two numerical examples and a three-level drawing process of Carbon fiber production.