Gaussian Mixture Regression Model Research Articles

Gaussian Mixture Regression Model with Sparsity for Clustering of Territory Risk in Auto Insurance

Abstract Insurance rating territory design and accurate estimation of territory risk relativities are fundamental aspects of auto insurance rate regulation. It is crucial to develop methodologies that can facilitate the effective design of rating territories and their risk relativities estimate, as they directly impact the rate filing and the decision support of the rate change review process. This article proposes a Gaussian Mixture Regression model clustering approach for territory design. The proposed method incorporates a linear regression model, taking spatial location as model covariates, which helps estimate the cluster mean more accurately. Also, to further enhance the estimation of territory risk relativities, we impose sparsity through sparse matrix decomposition of the membership coefficient matrix obtained from the Gaussian Mixture Regression model. By transitioning from the current hard clustering method to a soft approach, our methodology could improve the evaluation of territory risk for rate-making purposes. Moreover, using non-negative sparse matrix approximation ensures that the estimation of risk relativities for basic rating units remains smooth, effectively eliminating data noise from the territory risk relativity estimate. Overall, our novel methodology aims to significantly enhance the accuracy and reliability of risk analysis in auto insurance. Furthermore, the proposed method exhibits potential for extension to various other domains that involve spatial clustering of data, thereby broadening its applicability and expanding its usefulness beyond auto insurance rate regulation.

Biosensor-Based Multimodal Deep Human Locomotion Decoding via Internet of Healthcare Things.

Multiple Internet of Healthcare Things (IoHT)-based devices have been utilized as sensing methodologies for human locomotion decoding to aid in applications related to e-healthcare. Different measurement conditions affect the daily routine monitoring, including the sensor type, wearing style, data retrieval method, and processing model. Currently, several models are present in this domain that include a variety of techniques for pre-processing, descriptor extraction, and reduction, along with the classification of data captured from multiple sensors. However, such models consisting of multiple subject-based data using different techniques may degrade the accuracy rate of locomotion decoding. Therefore, this study proposes a deep neural network model that not only applies the state-of-the-art Quaternion-based filtration technique for motion and ambient data along with background subtraction and skeleton modeling for video-based data, but also learns important descriptors from novel graph-based representations and Gaussian Markov random-field mechanisms. Due to the non-linear nature of data, these descriptors are further utilized to extract the codebook via the Gaussian mixture regression model. Furthermore, the codebook is provided to the recurrent neural network to classify the activities for the locomotion-decoding system. We show the validity of the proposed model across two publicly available data sampling strategies, namely, the HWU-USP and LARa datasets. The proposed model is significantly improved over previous systems, as it achieved 82.22% and 82.50% for the HWU-USP and LARa datasets, respectively. The proposed IoHT-based locomotion-decoding model is useful for unobtrusive human activity recognition over extended periods in e-healthcare facilities.

Morkaraman Kuzularının Sütten Kesim Öncesi Büyüme Özellikleri Üzerine Çevresel Faktörlerin Etkisi

The aim of this study was to analyze the environmental factors impacting preweaning growth traits in Morkaraman herds and to categorize weaning weight using the Gaussian mixture regression model. The mean live weights of lambs at birth, 60th, and 90th day (weaning) were determined to be 3.97 kg, 17.94 kg, and 24.92 kg, respectively. The effect of lambing years on live weights at all ages was found to be quite significant (p

Knowledge-informed Variational Bayesian Gaussian mixture regression model for predicting mixed oil length

Precise prediction of mixed oil length is a critical task in the multi-product pipeline. Current predictive models may not achieve desirable performance as they neglected the multimode characteristics of the data, which has been alleviated by using the finite mixture model (FMM) to identify the different modes and develop localized predictive models while it faces the challenge of assigning a suitable number of components. Variational Bayesian Gaussian mixture regression model (VBGMR) may remedy these deficiencies but the linear assumption between variables can result in unsatisfactory performance, and it may also not handle the model selection well since all input variables are forced to participate in the mode identification process, even if some are not important. To address such issues, this article proposes a new modeling approach named as the Knowledge-informed Variational Bayesian Gaussian mixture regression model (KIVBGMR) by incorporating the conventional VBGMR with the guidance of the domain knowledge. Subsequently, the learning procedure for the KIVBGMR based on the Variational Inference method is developed. The performance of the KIVBGMR modeling method is evaluated based on two case studies including a numerical example and a real-life industrial dataset, which demonstrates the performance superiority of the proposed method.

A Data-Driven Kernel Principal Component Analysis–Bagging–Gaussian Mixture Regression Framework for Pulverizer Soft Sensors Using Reduced Dimensions and Ensemble Learning

In light of the nonlinearity, high dimensionality, and time-varying nature of the operational conditions of the pulverizer in power plants, as well as the challenge of the real-time monitoring of quality variables in the process, a data-driven KPCA–Bagging–GMR framework for soft sensors using reduced dimensions and ensemble learning is proposed. Firstly, the methodology employs a Kernel Principal Component Analysis to effectively reduce the dimensionality of the collected process data in a nonlinear manner. Secondly, the reduced principal components are then utilized to reconstruct a refined set of input samples, followed by the application of the Bagging algorithm to obtain multiple subsets of the samples and develop corresponding Gaussian Mixture Regression models. Ultimately, the fusion output is achieved by calculating the weights of each local model based on Bayesian posterior probabilities. By conducting simulation experiments on the coal mill, the proposed approach has been validated as demonstrating superior predictive accuracy and excellent generalization capabilities.

De Novo Direct Inverse QSPR/QSAR: Chemical Variational Autoencoder and Gaussian Mixture Regression Models.

Herein, we propose a de novo direct inverse quantitative structure-property relationship/quantitative structure-activity relationship (QSPR/QSAR) analysis method, based on the chemical variational autoencoder (VAE) and Gaussian mixture regression (GMR) models, to generate molecules with the desired target variables of interest for properties and activities (y). A data set of molecules was analyzed, and an encoder was used to transform the simplified molecular input line entry system (SMILES) strings to latent variables (x), while a decoder was used to transform x to SMILES strings. A chemical VAE model was used for analysis and a GMR model (between x and y) was constructed for direct inverse analysis. The target y values were input into the GMR model to directly predict the x values. Following this, the predicted x values were input into the decoder associated with the chemical VAE model and the SMILES string representations (or chemical structures of molecules) were obtained as the output, indicating that the proposed method could be used to selectively obtain the molecules that were characterized by the target y values. We confirmed that the proposed method can be used to generate molecules within the target y ranges even when the conventional chemical VAE model failed to generate the target molecules.

Direct prediction of the batch time and process variable profiles using batch process data based on different batch times

This study aims to design both the batch time and the process variable (PV) profiles (x) to ensure that the endpoints (y), such as the product quality and the material properties, possess the target values following a batch process. To handle datasets based on different batch times, the time-series data of the PVs were either extrapolated for short batches or transformed by discrete Fourier transform. In addition, a Gaussian mixture regression model was constructed between x and y, allowing the direct prediction of x from y, enabling a direct inverse analysis. The effectiveness of the proposed method was verified via numerical simulations and a fed-batch bioreactor process. The results indicate that this method can accurately predict the end-point while appropriately designing the PV batch times and profiles. The proposed method successfully designed the batch profile by increasing the product concentration by 50% in the fed-batch bioreactor process.

Lifting the limitations of Gaussian mixture regression through coupling with principal component analysis and deep autoencoding

The mathematical modeling of correlations between target properties and their factors of influence, particularly that allowing inverse analysis, is an essential part of molecular, material, and process designs. In contrast to approaches employing pseudo-inverse analysis, Gaussian mixture regression (GMR), which assumes that the relationships between variables can be represented as a mixture of Gaussian distributions, allows for direct inverse analysis. However, as this model optimizes the means and variance–covariance matrices of all variables, parameter estimation becomes increasingly difficult with the increasing number of variables. Herein, this drawback is addressed by the transformation of explanatory variables X into latent variables Z before GMR modeling. As the inverse (Z to X) transformation is necessary for direct inverse analysis, principal component analysis (PCA) and deep autoencoding (DAE) are employed as dimensionality reduction methods. After X is transformed to Z with the help of PCA or DAE, a GMR model is constructed with Z and objective variables Y, and the proposed method is therefore denoted as PCA-GMR or DAE-GMR, respectively. As Z values can be predicted by inputting Y values into the GMR model and can be transformed to X values, direct inverse analysis is also possible. Given that unlabeled data can be employed to construct PCA and DAE models, the proposed methods can also be used as semi-supervised learning techniques. The predictive abilities of PCA-GMR and DAE-GMR are verified using molecular, material, and spectral datasets and surpass that of traditional GMR on all datasets, with the maximum reduction of prediction errors equaling 63%.

A Gaussian mixture regression model based adaptive filter for non-Gaussian noise without a priori statistic

In many engineering systems, the distribution of measurement noise is unknown and non-Gaussian, such as skewed, multimodal, time-varying distributions and we can not obtain these prior statistics in advance. How to achieve state estimation via this kind of non-Gaussian measurement noises without prior statistic information is a challenging problem. In this paper, a novel Gaussian mixture regression model (GMRM) is proposed to model the unknown non-Gaussian measurement likelihood for Bayesian update to achieve nonlinear state estimation. Without any prior assumption or limitation of measurement noises’ statistics and distributions, the GMRM can still describe the measurement likelihood accurately based on a group of parameters which are adjusted by maximizing the evidence lower bound. Based on the optimized GMRM, a new variational Bayesian Gaussian mixture filter is proposed by using the variational Bayesian approach. To eliminate the influence of the initialization of the introduced parameters, a learning scheme is proposed to adaptively optimize their hyper-parameters based on the historical measurements. Finally, simulation examples are employed to illustrate the effectiveness of the filter.

Analysis of Pulmonary Function Test Results By Using Gaussian Mixture Regression Model

Background: FEV1/FVC value is used in the diagnosis of obstructive and restrictive diseases of the lung. It is a parameter reported in the literature that it varies according to lung disease as well as weight, age and gender characteristics. The aim of this study is to investigate the relationship between age, weight, gender and height characteristics and FEV1/FVC value using a heterogeneous population using Gaussian mixture regression method. Material and Methods: GMR was used to separate the data into components and to make a parameter estimation for each component. The analysis performed on this model revealed that the patients were divided into 5 optimal groups and that these groups showed a regular transition from obstructive pattern to restrictive pattern. Results: The mean values of the components for FEV1/FVC were found as 50.071 (3.238), 67.034 (1.725), 82.156 (1.329), 93.592 (1.041), 98.466 (0.303), respectively. The effect of the weight on the components in terms of parameter estimation and standard errors of the components was determined as 0.445 (0.129) **, 0.226 (0.053) **, 0.173 (0.053) **, -0.036 (0.026), -0.040 (0.018) *, respectively. Conclusion: Direct proportional relationship between the patient's weight and the severity of the obstructive pattern, and between the severity of the disease and the age of the patient in both the obstructive and restrictive pattern are expilicitly proved. Furthermore, it has been revealed that data sets containing heterogeneity can be analyzed by dividing them into sub-components using the GMR model.

Direct inverse analysis based on Gaussian mixture regression for multiple objective variables in material design

In research and development of highly functional materials, new materials and compounds are required to achieve target values of multiple properties and activities Y. Because regression methods that construct a single model for each Y variable cannot consider the relationships between the Y variables, the search for candidates for materials that satisfy all of the target Y values cannot be efficiently performed. Furthermore, pseudo-inverse analysis of models, where promising candidates are selected based on the Y values predicted by substituting a large number of candidates for explanatory variables X, such as experimental conditions and molecular descriptors, into models, cannot search for candidates with desired Y values from X candidates. Therefore, in this study, we focused on Gaussian mixture regression (GMR), which can simultaneously handle multiple Y variables. GMR can simultaneously predict multiple Y variables while considering the relationships between the Y variables, and it can also directly predict X variables by substituting the values of multiple Y variables. We used numerical simulation data in which the Y variables were nonlinearly correlated and nonlinear relationships existed between X and Y, and verified the predictive ability of the GMR model and the advantages of direct inverse analysis of the GMR model. In addition, we analyzed a dataset of thermoelectric conversion materials and searched for new high-performance thermoelectric conversion materials.

Adaptive Segmentation and Sequence Learning of human activities from skeleton data

Discovering underlying patterns for predicting future actions from spatio-temporal human activity information is a fundamental component of research related to the development of expert systems in human activity recognition and assistive robotics. Current research focuses on classification or learning representations of activities for various applications. However, not much attention is given to the pattern discovery of activities which have a major role in the prediction of unseen actions. This paper proposes a novel Adaptive Segmentation and Sequence Learning (ASSL) framework which aims at segmenting unlabelled observations of human activities from extracted 3D joint information. Learning from these obtained segments provides information about the underlying patterns of activity sequences needed in predicting subsequent actions. In the proposed method, the temporal accumulated motion energy of body parts in an activity is utilised in the segmentation process to obtain key actions from unlabelled activity sequences since body parts show changes in acceleration and deceleration during an activity. Based on the segments obtained, the temporal sequence of transitions across activity segments are learned by employing a Long Short-Term Memory Recurrent Neural Network. This ASSL technique has been evaluated using both an experimental human activity dataset and a public activity dataset, and achieved a better performance when compared with other techniques including an Auto-regressive Integrated Moving Average, Support Vector Regression and Gaussian Mixture Regression Models in learning to predict patterns of activity sequences.

Nonlinear Gaussian Mixture Regression for Multimode Quality Prediction With Partially Labeled Data

An enhanced nonlinear Gaussian mixture regression (NLGMR) algorithm is proposed for quality prediction of a nonlinear multimode process. The traditional Gaussian mixture regression (GMR) model has been utilized for quality prediction with a linear model in each local mode, which will not suit for many cases that nonlinear relationships exist between input and output variables. Besides, large scales of process data that can be used for modeling are partially labeled on account of the low sampling rate of quality variables. Most of the unlabeled samples are discarded while building the GMR model, which leads to the loss of information and limits the improvement of prediction accuracy. To tackle these two problems, a locally weighted semisupervised factor analysis model is developed in each mode of GMR. The locally weighted model divides the nonlinear process into pieces of linear model and the semisupervised factor analysis model can effectively take advantage of the massive unlabeled data. Moreover, the variational inference (VI) algorithm is conducted on the GMR model to determine the amount of process modes automatically. The proposed method is first verified by a numerical example and then applied in a multimode primary reformer to predict the oxygen content, where prominent improvements are obtained, compared with traditional methods.

Comparing Coefficients Across Subpopulations in Gaussian Mixture Regression Models

When fitting a Gaussian mixture regression model to observed data, estimating a between-group contrast can be a practical issue. One can use the estimate to compare the effects of a particular covariate or a set of covariates across different subpopulations. By applying fiducial generalized pivotal quantities, a small-sample solution is proposed in this paper to obtain interval estimates of between-group contrasts. Specifically, a Markov chain Monte Carlo sampler, which takes the membership uncertainty of each individual into account, is designed to generate realizations from the target distributions for computing the required interval estimates. A plant virus transmission study is first introduced as a motivating example for the present study. Next, the observed data are analyzed to illustrate the proposed method. Based on the simulation results, it is further shown that the proposed method can maintain the empirical coverage rates sufficiently close to the nominal level. Supplementary materials accompanying this paper appear on-line.

Plane-extraction from depth-data using a Gaussian mixture regression model

We propose a novel algorithm for unsupervised extraction of piecewise planar models from depth-data. Among other applications, such models are a good way of enabling autonomous agents (robots, cars, drones, etc.) to effectively perceive their surroundings and to navigate in three dimensions. We propose to do this by fitting the data with a piecewise-linear Gaussian mixture regression model whose components are skewed over planes, making them flat in appearance rather than being ellipsoidal, by embedding an outlier-trimming process that is formally incorporated into the proposed expectation-maximization algorithm, and by selectively fusing contiguous, coplanar components. Part of our motivation is an attempt to estimate more accurate plane-extraction by allowing each model component to make use of all available data through probabilistic clustering. The algorithm is thoroughly evaluated against a standard benchmark and is shown to rank among the best of the existing state-of-the-art methods.

Modeling with Gaussian mixture regression for lactationmilk yield in Anatolian buffaloes

The purpose of this study was to classify Anatolian buffalo using Gaussian mixture regression model according to discrete and continuous environmental effects. Gaussian mixture model performs separately regression analysis both within and between groups. This is an important property of Gaussian mixture models which makes it different from other multivariate statistical methods. The data were obtained from 1455 Anatolian buffalo lactation milk yield records reared in seven different locations in Bitlis province, Turkey. Age of dam, lactation duration and locations were considered as environmental effects on lactation milk yield. Data set was divided into three homogenous subgroups with respect to AIC and BIC in the Gaussian mixture regression, based on environmental effects on lactation milk yield. Estimated mean for lactation milk yields and mixing probabilities for the first, second and third subgroups were determined as 1494.33 kg (16.9%), 540.33 kg (45.2%) and 847.61 (37.9%), respectively. The numbers of buffalo in each subgroup according to mixing probability were obtained as 159, 756, and 540 for the first, second, and third groups, respectively. The effects of lactation period, age of dam and villages were found statistically significant on lactation milk yield in subgroup 1 that was highest mean for lactation milk yield (p less than 0.01). In conclusion, results showed that Gaussian mixture regression was an important tool for classifying quantitative traits considering environmental effects in animal breeding.

A class of finite mixture of quantile regressions with its applications

Mixture of linear regression models provide a popular treatment for modeling nonlinear regression relationship. The traditional estimation of mixture of regression models is based on Gaussian error assumption. It is well known that such assumption is sensitive to outliers and extreme values. To overcome this issue, a new class of finite mixture of quantile regressions (FMQR) is proposed in this article. Compared with the existing Gaussian mixture regression models, the proposed FMQR model can provide a complete specification on the conditional distribution of response variable for each component. From the likelihood point of view, the FMQR model is equivalent to the finite mixture of regression models based on errors following asymmetric Laplace distribution (ALD), which can be regarded as an extension to the traditional mixture of regression models with normal error terms. An EM algorithm is proposed to obtain the parameter estimates of the FMQR model by combining a hierarchical representation of the ALD. Finally, the iterated weighted least square estimation for each mixture component of the FMQR model is derived. Simulation studies are conducted to illustrate the finite sample performance of the estimation procedure. Analysis of an aphid data set is used to illustrate our methodologies.

Comparisons of inverse modeling approaches for predicting building energy performance

In building retrofit projects, retrofit savings can be estimated by comparing building energy use before and after installing Energy Conservation Measures (ECMs). A complicating factor is that there is no direct measurement of the reduced energy use that is solely attributable to the retrofit. Indeed, simple comparisons by subtracting the post-retrofit energy use from the pre-retrofit would ignore the impact of other factors, such as weather and occupancy with constantly changing patterns, on the total building energy use. Data-driven models (i.e., derived by inverse modeling approaches) that are trained with monitored pre-retrofit building data can be used as the baseline models in a retrofit project. However, to be effective, the baseline energy models must be capable of singling out the impact of ECMs and ignoring the influence of other factors. A commonly used method to achieve this goal is to develop a statistical model that correlates energy use with weather and other independent variables.This paper first reviews four mainstream baseline data-driven energy models used to characterize building energy performance: change-point regression model, Gaussian process regression model, Gaussian Mixture Regression Model, and Artificial Neural Network model, These models are then applied to an office building to predict the Heating, Ventilation, and Air-Conditioning (HVAC) hot water energy consumption. Several model accuracy measures such as R2, RMSE, CV-RMSE, and sensitivity to sample frequency, and reliability, are evaluated and compared.

Accuracy of inverse treatment planning on substitute CT images derived from MR data for brain lesions.

BackgroundIn this pilot study we evaluated the performance of a substitute CT (s-CT) image derived from MR data of the brain, as a basis for optimization of intensity modulated rotational therapy, final dose calculation and derivation of reference images for patient positioning.MethodsS-CT images were created using a Gaussian mixture regression model on five patients previously treated with radiotherapy. Optimizations were compared using Dmax, Dmin, Dmedian and Dmean measures for the target volume and relevant risk structures. Final dose calculations were compared using gamma index with 1%/1 mm and 3%/3 mm acceptance criteria. 3D geometric evaluation was conducted using the DICE similarity coefficient for bony structures. 2D geometric comparison of digitally reconstructed radiographs (DRRs) was performed by manual delineation of relevant structures on the s-CT DRR that were transferred to the CT DRR and compared by visual inspection.ResultsDifferences for the target volumes in optimization comparisons were small in general, e.g. a mean difference in both Dmin and Dmax within ±0.3%. For the final dose calculation gamma evaluations, 100% of the voxels passed the 1%/1 mm criterion within the PTV. Within the entire external volume between 99.4% and 100% of the voxels passed the 3%/3 mm criterion. In the 3D geometric comparison, the DICE index varied between approximately 0.8-0.9, depending on the position in the skull. In the 2D DRR comparisons, no appreciable visual differences were found.ConclusionsEven though the present work involves a limited number of patients, the results provide a strong indication that optimization and dose calculation based on s-CT data is accurate regarding both geometry and dosimetry.

Soft sensor model development in multiphase/multimode processes based on Gaussian mixture regression

For complex industrial plants with multiphase/multimode data characteristic, Gaussian mixture model (GMM) has been used for soft sensor modeling. However, almost all GMM-based soft sensor modeling methods only employ GMM for identification of different operating modes, which means additional regression algorithms like PLS should be incorporated for quality prediction in different localized modes. In this paper, the Gaussian mixture regression (GMR) model is introduced for multiphase/multimode soft sensor modeling. In GMR, operating mode identification and variable regression are integrated into one model; thus, there is no need to switch prediction models when the operating mode changes from one to another. To improve the GMR model fitting performance, a heuristic algorithm is adopted for parameter initialization and component number optimization. Feasibility and efficiency of GMR based soft sensor are validated through a numerical example and two benchmark processes.