Multivariate Gaussian Process Research Articles

Detection and prevention of man-in-the-middle attack in iot network using regression modeling

Security is the primary concern in any IoT application or network. Due to the rapid increase in the usage of IoT devices, data privacy becomes one of the most challenging issue to the researcher. In IoT applications, such as health care, smart homes or any wearables, transmission of human's personal data is more frequent. Man-in-the-Middle attack is one in which outsiders eavesdrops the communication between two trusted parties and steal the important information such as password, personal identification number, etc., and misuse it. So, this paper proposes a Regression Modelling technique to detect and mitigate the attack to provide attack-free path from source to destination in an IoT network. Three machine learning techniques Linear Regression (LR), Multi-variate Linear Regression (MLR) and Gaussian Process Regression (GPR) used and performance of these three algorithms analyzed on various metrics and shown Gaussian Process Regression provide higher rate for detecting the attacks and produces the lower rate for misclassification of attacks.

Instability of networks: effects of sampling frequency and extreme fluctuations in financial data

What determines the stability of networks inferred from dynamical behavior of a system? Internal and external shocks in a system can destabilize the topological properties of comovement networks. In real-world data, this creates a trade-off between identification of turbulent periods and the problem of high dimensionality. Longer time-series reduces the problem of high dimensionality, but suffers from mixing turbulent and non-turbulent periods. Shorter time-series can identify periods of turbulence more accurately, but introduces the problem of high dimensionality, so that the underlying linkages cannot be estimated precisely. In this paper, we exploit high-frequency multivariate financial data to analyze the origin of instability in the inferred networks during periods free from external disturbances. We show that the topological properties captured via centrality ordering is highly unstable even during such non-turbulent periods. Simulation results with multivariate Gaussian and fat-tailed stochastic process calibrated to financial data show that both sampling frequencies and the presence of outliers cause instability in the inferred network. We conclude that instability of network properties do not necessarily indicate systemic instability.Graphical abstract

COBALT: COnstrained Bayesian optimizAtion of computationaLly expensive grey-box models exploiting derivaTive information

Many engineering problems involve the optimization of computationally expensive models for which derivative information is not readily available. The Bayesian optimization (BO) framework is a particularly promising approach for solving these problems, which uses Gaussian process (GP) models and an expected utility function to systematically tradeoff between exploitation and exploration of the design space. BO, however, is fundamentally limited by the black-box model assumption that does not take into account any underlying problem structure. In this paper, we propose a new algorithm, COBALT, for constrained grey-box optimization problems that combines multivariate GP models with a novel constrained expected utility function whose structure can be exploited by state-of-the-art nonlinear programming solvers. COBALT is compared to traditional BO on seven test problems including the calibration of a genome-scale bioreactor model to experimental data. Overall, COBALT shows very promising performance on both unconstrained and constrained test problems.

Graphical Gaussian Process Models for Highly Multivariate Spatial Data.

For multivariate spatial Gaussian process (GP) models, customary specifications of cross-covariance functions do not exploit relational inter-variable graphs to ensure process-level conditional independence among the variables. This is undesirable, especially for highly multivariate settings, where popular cross-covariance functions such as the multivariate Matérn suffer from a "curse of dimensionality" as the number of parameters and floating point operations scale up in quadratic and cubic order, respectively, in the number of variables. We propose a class of multivariate "Graphical Gaussian Processes" using a general construction called "stitching" that crafts cross-covariance functions from graphs and ensures process-level conditional independence among variables. For the Matérn family of functions, stitching yields a multivariate GP whose univariate components are Matérn GPs, and conforms to process-level conditional independence as specified by the graphical model. For highly multivariate settings and decomposable graphical models, stitching offers massive computational gains and parameter dimension reduction. We demonstrate the utility of the graphical Matérn GP to jointly model highly multivariate spatial data using simulation examples and an application to air-pollution modelling.

A new Framework for the Spectral Information Decomposition of Multivariate Gaussian Processes.

Different information-theoretic measures are available in the literature for the study of pairwise and higher-order interactions in multivariate dynamical systems. While these measures operate in the time domain, several physiological and non-physiological systems exhibit a rich oscillatory content that is typically analyzed in the frequency domain through spectral and cross-spectral approaches. For Gaussian systems, the relation between information and spectral measures has been established considering coupling and causality measures, but not for higher-order interactions. To fill this gap, in this work we introduce an information-theoretic framework in the frequency domain to quantify the information shared between a target process and two sources, even multivariate, and to highlight the presence of redundancy and synergy in the analyzed dynamical system. Firstly, we simulate different linear interacting processes by showing the capability of the proposed framework to retrieve amounts of information shared by the processes in specific frequency bands which are not detectable by the related time-domain measures. Then, the framework is applied on EEG time series representative of the brain activity during a motor execution task in a group of healthy subjects.

Partial separability and functional graphical models for multivariate Gaussian processes

SummaryThe covariance structure of multivariate functional data can be highly complex, especially if the multivariate dimension is large, making extensions of statistical methods for standard multivariate data to the functional data setting challenging. For example, Gaussian graphical models have recently been extended to the setting of multivariate functional data by applying multivariate methods to the coefficients of truncated basis expansions. However, compared with multivariate data, a key difficulty is that the covariance operator is compact and thus not invertible. This paper addresses the general problem of covariance modelling for multivariate functional data, and functional Gaussian graphical models in particular. As a first step, a new notion of separability for the covariance operator of multivariate functional data is proposed, termed partial separability, leading to a novel Karhunen–Loève-type expansion for such data. Next, the partial separability structure is shown to be particularly useful in providing a well-defined functional Gaussian graphical model that can be identified with a sequence of finite-dimensional graphical models, each of identical fixed dimension. This motivates a simple and efficient estimation procedure through application of the joint graphical lasso. Empirical performance of the proposed method for graphical model estimation is assessed through simulation and analysis of functional brain connectivity during a motor task.

A high-efficiency simulation method of wind field and its application on transmission line

Generally, the fluctuating wind is simplified as several independent one-dimensional multivariate stationary Gaussian processes in simulating a natural wind field. The correlation in the lateral, longitudinal and vertical directions should all be considered in the simulation of longitudinal wind field for the large-span spatial structures. In fact, this type of structure has lots of simulation points. The calculation amount of wind field simulation by the harmonic superposition method depends on the scale of cross-spectral density matrix, which is directly related to the number of simulated points, leading to a low efficiency when generating the time-varying wind speed. This paper innovatively proposes a high-efficiency simulation method for the longitudinal wind field based on Taylor's hypothesis. Subsequently, the effectiveness of the proposed wind field method was verified by the numerical simulation. Finally, the dynamic responses of a transmission tower-line system under the wind loadings generated with the new method and traditional method are calculated and compared. The percentages difference of the mean and maximum axial force at the main tower members are less than 0.02% and 1%, respectively, indicating the effectiveness of the proposed time delay method. The results also show that the proposed simulation method of wind field can not only ensure the simulation accuracy, but also significantly improve the efficiency of wind speed generation, which is suitable for the wind load simulation of large-span spatial structures.

Gaussian process analysis of electron energy loss spectroscopy data: multivariate reconstruction and kernel control

Advances in hyperspectral imaging including electron energy loss spectroscopy bring forth the challenges of exploratory and physics-based analysis of multidimensional data sets. The multivariate linear unmixing methods generally explore similarities in the energy dimension, but ignore correlations in the spatial domain. At the same time, Gaussian process (GP) explicitly incorporate spatial correlations in the form of kernel functions but is computationally intensive. Here, we implement a GP method operating on the full spatial domain and reduced representations in the energy domain. In this multivariate GP, the information between the components is shared via a common spatial kernel structure, while allowing for variability in the relative noise magnitude or image morphology. We explore the role of kernel constraints on the quality of the reconstruction, and suggest an approach for estimating them from the experimental data. We further show that spatial information contained in higher-order components can be reconstructed and spatially localized.

Diagnosis of obstructive sleep apnea with prediction of flow characteristics according to airway morphology automatically extracted from medical images: Computational fluid dynamics and artificial intelligence approach

BackgroundObstructive sleep apnea syndrome (OSAS) is being observed in an increasing number of cases. It can be diagnosed using several methods such as polysomnography. ObjectivesTo overcome the challenges of time and cost faced by conventional diagnostic methods, this paper proposes computational fluid dynamics (CFD) and machine-learning approaches that are derived from the upper-airway morphology with automatic segmentation using deep learning. MethodWe adopted a 3D UNet deep-learning model to perform medical image segmentation. 3D UNet prevents the feature-extraction loss that may occur by concatenating layers and extracts the anteroposterior coordination and width of the airway morphology. To create flow characteristics of the upper airway training data, we analyzed the changes in flow characteristics according to the upper-airway morphology using CFD. A multivariate Gaussian process regression (MVGPR) model was used to train the flow characteristic values. The trained MVGPR enables the prompt prediction of the aerodynamic features of the upper airway without simulation. Unlike conventional regression methods, MVGPR can be trained by considering the correlation between the flow characteristics. As a diagnostic step, a support vector machine (SVM) with predicted aerodynamic and biometric features was used in this study to classify patients as healthy or suffering from moderate OSAS. SVM is beneficial as it is easy to learn even with a small dataset, and it can diagnose various flow characteristics as factors while enhancing the feature via the kernel function. As the patient dataset is small, the Monte Carlo cross-validation was used to validate the trained model. Furthermore, to overcome the imbalanced data problem, the oversampling method was applied. ResultThe segmented upper-airway results of the high-resolution and low-resolution models present overall average dice coefficients of 0.76±0.041 and 0.74±0.052, respectively. Furthermore, the classification accuracy, sensitivity, specificity, and F1-score of the diagnosis algorithm were 81.5%, 89.3%, 86.2%, and 87.6%, respectively. ConclusionThe convenience and accuracy of sleep apnea diagnosis are improved using deep learning and machine learning. Further, the proposed method can aid clinicians in making appropriate decisions to evaluate the possible applications of OSAS.

Emission-to-ash detoxification mechanisms of co-combustion of spent pot lining and pulverized coal

In response to the global initiative for greenhouse gas emission reduction, the co-combustion of coal and spent pot lining (SPL) may cost-effectively minimize waste streams and environmental risks. This study aimed to quantify the emission-to-ash detoxification mechanisms of the co-combustion of SPL and pulverized coal (PC) and their kinetics, gas emission, fluorine-leaching toxicity, mineral phases, and migrations. The main reaction covered the ranges of 335–540 °C and 540–870 °C while the interactions occurred at 360–780 °C. The apparent activation energy minimized (66.99 kJ/mol) with 90% PC addition. The rising PC fraction weakened the peak intensity of NaF and strengthened that of Ca2F, NaAlSiO4, and NaAlSi2O6. The addition of PC enhanced the combustion efficiency of SPL and raised the melting temperature by capturing Na. PC exhibited a positive effect on solidifying water-soluble fluorine and stabilizing alkali and alkaline earth metals. The leaching fluorine concentrations of the co-combustion ashes were lower than did SPL mono-combustion. The main gases emitted were HF, NH3, NOx, CO, and CO2. HF was largely released at above 800 °C. Multivariate Gaussian process model-based optimization of the operational conditions also verified the gas emissions results. Our study synchronizes the utilization and detoxification of SPL though co-combustion and provides insights into an eco-friendlier life-cycle control on the waste-to-energy conversion.

On multi-class automated vehicles: Car-following behavior and its implications for traffic dynamics

This paper develops a unifying framework to unveil the physical car-following (CF) behaviors of automated vehicles (AVs) under different control paradigms and parameter settings. The proposed framework adopts the flexible asymmetric behavior (AB) model to reveal the control mechanisms and their manifestation in the physical CF behavior, particularly their response to traffic disturbances. A mapping relationship between the AB model parameters and control parameters is then obtained to understand the range of CF behavior possible. Finally, a predictive modeling approach based on a logistic classifier coupled with a convoluted Multivariate Gaussian Process (MGP) is designed to predict the CF behavior of an AV. Analysis of two well-known controllers, linear state-feedback and Model Predictive Control (MPC), show how the proposed framework can uncover the CF mechanisms and provide insights into traffic-level disturbance evolution. The proposed analysis framework remains scalable and can be applied to a variety of controllers. Ultimately, it can guide AV control design that is not myopic, but considers traffic-level performance.

Simulation of stationary Gaussian/non-Gaussian stochastic processes based on stochastic harmonic functions

A new model is proposed to represent and simulate Gaussian/non-Gaussian stochastic processes. In the proposed model, stochastic harmonic function (SHF) is extended to represent multivariate Gaussian process firstly. Compared with the conventional spectral representation method (SRM), the SHF based model requires much fewer variables and Cholesky decompositions. Then, SHF based model is further extended to univariate/multivariate non-Gaussian stochastic process simulation. The target non-Gaussian process can be obtained from the corresponding underlying Gaussian processes by memoryless nonlinear transformation. For arbitrarily given marginal probability distribution function (PDF), the covariance function of the underlying multivariate Gaussian process can be determined easily by introducing the Mehler’s formula. And when the incompatibility between the target non-Gaussian power spectral density (PSD) or PSD matrix and marginal PDF exists, the calibration of the target non-Gaussian spectrum will be required. Hence, the proposed model can be regarded as SRM to efficiently generate Gaussian/non-Gaussian processes. Finally, several numerical examples are addressed to show the effectiveness of the proposed method.

New Formulation and Computation of NRDF for Time-Varying Multivariate Gaussian Processes With Correlated Noise

We derive a new formulation of nonanticipative rate distortion function (NRDF) for time-varying multivariate Gauss-Markov processes driven by correlated noise described by a first order autoregressive moving average (ARMA(1,1)) process with mean-squared error (MSE) distortion constraint. To arrive to this formulation, we first show that the Gauss-Markov process with correlated noise can be compactly written as a linear functional of the lagged by one sample of the sufficient statistic of the correlated noise and its orthogonal innovations process. Then, we use this structural result to a general low delay quantization problem where we choose to design the encoder and the decoder policies of a multi-input multi-output (MIMO) system with given the past sample of the sufficient statistic of the correlated noise process. For jointly Gaussian processes, we find the minimization problem that needs to be solved, obtain its optimal realization and solve it by showing that is semidefinite representable. Interestingly, the optimal realization of this problem reveals that it suffices only the decoder to have access to the given past sufficient statistic of the correlated noise process but not necessarily the encoder. For scalar-valued processes, we also derive a new analytical expression. The generality of our results (both for vector and scalar processes) is shown by recovering various special cases and known results obtained for independent noise processes.

Individualized Degradation Modeling and Prognostics in a Heterogeneous Group via Incorporating Intrinsic Covariate Information

This article focuses on individualized degradation modeling and prognostics for a heterogeneous group, where each individual unit shows a distinct degradation process. Existing degradation models usually treat each unit separately and do not fully utilize the distinct characteristics of each individual. In this study, we propose a generic framework to handle the heterogeneity across units by effectively leveraging the intrinsic covariate information, which is closely related to the unit’s degradation process. Specifically, we employ a multivariate Gaussian process (MGP) to nonparametrically establish the relation between the covariate information and degradation process. Through modeling the unit similarities based on the covariates, efficient information transfer among units is enabled for better degradation modeling and prognostics, as the collected degradation signals from one unit can be shared with the entire heterogeneous group. A theoretical justification for the proposed model is also investigated. Simulation studies are presented to evaluate the parameter estimation accuracy and the sensitivity of the proposed method. A case study on the Alzheimer’s disease (AD) neuroimaging initiative data set is further conducted, which demonstrates the advantage of the proposed method over existing benchmark approaches. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article is motivated by the practical issue of degradation modeling and prognostics for a heterogeneous group, where all units in a group share some similarities and each unit has its own distinct individual-level characteristics (covariates). The covariates in this study refer to static intrinsic characteristics instead of dynamic external environmental conditions. Several practical examples are explained in <xref ref-type="sec" rid="sec1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Section I</xref> with more details. There are two fundamental questions involved: 1) how to quantify the distinct individual characteristics of each unit while representing the group-level commonalities among all units and 2) how to perform degradation modeling and prognostics of a newly launched unit with few degradation signals available. The novelty of this article lies in encoding the available knowledge about individual covariates and group-level commonalities into the degradation modeling and prognostics. There are three main steps involved when implementing the proposed method: 1) collecting degradation signals, failure time, and covariate information of heterogeneous units; 2) constructing an MGP-based degradation model; and 3) predicting the degradation status and remaining useful life of the in-service units based on their covariates and signals. The proposed method is particularly useful when we collect various intrinsic covariates which can effectively represent the individual-level characteristics and when only sparse or no data are available for the units of interest.

Selection of surrogate modeling techniques for surface approximation and surrogate-based optimization

Surrogate models are used to map input data to output data when the actual relationship between the two is unknown or computationally expensive to evaluate for several applications, including surface approximation and surrogate-based optimization. This work evaluates the performance of eight surrogate modeling techniques for those two applications over a set of generated datasets with known characteristics. With this work, we aim to provide general rules for selecting an appropriate surrogate model form based solely on the characteristics of the data being modeled. The computational experiments revealed that there is a dependence of the surrogate modeling performance on the data characteristics. However, in general, multivariate adaptive regression spline models and Gaussian process regression yielded the most accurate predictions for approximating a surface. Random forests, support vector machine regression, and Gaussian process regression models most reliably identified the optimum locations and values when used for surrogate-based optimization.

Multivariate gaussian process incorporated predictive model for stream turbine power plant

Steam power turbine-based power plant approximately contributes 90% of the total electricity produced in the United States. Mainly steam turbine consists of multiple types of turbine, boiler, attemperator, reheater, etc. Power is produced through the steam with high pressure and temperature that is conducted by the turbines. The total power generation of the power plant is highly nonlinear considering all these elements in the model. We perform a predictive modeling approach to detect the power generation from these turbines by the Gaussian process (GP) model. As there are multiple interconnected turbines, we consider a multivariate Gaussian process (MGP) modeling to predict the power generation from these turbines which can capture the cross-correlations between the turbines. Also, the sensitivity analysis of the input parameters is constructed for each turbine to find out the most important parameters.

Combustion behaviors of complex incense stick residues: Multivariate Gaussian process-based optimization of thermal, kinetic, thermodynamic, emission, and ash responses

In order for second-generation feedstocks to be fed back to circular economies, both economic and environmental benefits of the thermochemical conversion need to be maximized in the context of the cradle-to-grave/cradle waste management. This study aimed to quantify and optimize the combustion performance and flue gas emissions of incense sticks (IS). The IS combustion consisted of three stages, with a main weight loss of 69.6% between 185 and 477 °C at 10 °C/min. Mean activation energy was estimated at 157.4 and 251.7 kJ/mol for the main second and third reaction stages. CO2 and H2O were the main evolved gases, while NH3 and SO2 exhibited a similar release pattern by peaking at 320 °C. Multivariate Gaussian process model-based optimizations pointed to 525 °C as the optimal setting for both three combustion responses and the emission reduction of nine flue gases. A 63% rise in the combustion temperature from 322 to 525 °C on average reduced the emissions by 79%.

Nonstationary multivariate Gaussian processes for electronic health records

Advances in the modeling and analysis of electronic health records (EHR) have the potential to improve patient risk stratification, leading to better patient outcomes. The modeling of complex temporal relations across the multiple clinical variables inherent in EHR data is largely unexplored. Existing approaches to modeling EHR data often lack the flexibility to handle time-varying correlations across multiple clinical variables, or they are too complex for clinical interpretation. Therefore, we propose a novel nonstationary multivariate Gaussian process model for EHR data to address the aforementioned drawbacks of existing methodologies. Our proposed model is able to capture time-varying scale, correlation and smoothness across multiple clinical variables. We also provide details on two inference approaches: Maximum a posteriori and Hamilton Monte Carlo. Our model is validated on synthetic data and then we demonstrate its effectiveness on EHR data from Kaiser Permanente Division of Research (KPDOR). Finally, we use the KPDOR EHR data to investigate the relationships between a clinical patient risk metric and the latent processes of our proposed model and demonstrate statistically significant correlations between these entities.

Learning a Generative Motion Model From Image Sequences Based on a Latent Motion Matrix.

We propose to learn a probabilistic motion model from a sequence of images for spatio-temporal registration. Our model encodes motion in a low-dimensional probabilistic space - the motion matrix - which enables various motion analysis tasks such as simulation and interpolation of realistic motion patterns allowing for faster data acquisition and data augmentation. More precisely, the motion matrix allows to transport the recovered motion from one subject to another simulating for example a pathological motion in a healthy subject without the need for inter-subject registration. The method is based on a conditional latent variable model that is trained using amortized variational inference. This unsupervised generative model follows a novel multivariate Gaussian process prior and is applied within a temporal convolutional network which leads to a diffeomorphic motion model. Temporal consistency and generalizability is further improved by applying a temporal dropout training scheme. Applied to cardiac cine-MRI sequences, we show improved registration accuracy and spatio-temporally smoother deformations compared to three state-of-the-art registration algorithms. Besides, we demonstrate the model's applicability for motion analysis, simulation and super-resolution by an improved motion reconstruction from sequences with missing frames compared to linear and cubic interpolation.

Compositional grading of a 316L-Cu multi-material part using machine learning for the determination of selective laser melting process parameters

The aim of this work is to propose a methodology for rapidly predicting suitable process parameters for additive manufacturing of a 316L-Cu multi-material part with a compositional gradient by using machine learning. Specifically, an algorithm based on a multivariate Gaussian process is developed to predict part density and surface roughness for a given set of laser power, velocity, and hatch spacing values. The training data for the algorithm is collected using a high-throughput experimentation method that allows for rapid measurement of part density, and surface roughness. After the model is validated using leave-one-out cross validation method, process parameter maps are generated for 316L-Cu parts manufactured using selective laser melting with premixed powder at mass fractions of 0.25, 0.50, and 0.75. A set of suitable process parameters are predicted using the process maps. It is shown that process parameters are a nonlinear function of gradient composition and neither process parameters of 316L or Cu are suitable for the graded region of a 316L-Cu multi-material part. Generated process maps provide a firsthand knowledge of process-property relationships for regions of compositional grading in 316L-Cu parts.