Mixture Regression Research Articles

Bayesian hierarchical finite mixture of regression for histopathological imaging-based cancer data analysis.

Cancer is heterogeneous, and for seemingly similar cancer patients, the associations between an outcome/phenotype and covariates can be different. To describe such differences, finite mixture of regression (FMR) and other modeling techniques have been developed. "Classic" FMR analysis has usually been based on clinical, demographic, and molecular variables. More recently, histopathological imaging data-which is a byproduct of biopsy and enjoys broader data availability and higher cost-effectiveness-has been increasingly used in cancer modeling, although it is noted that its application to cancer FMR analysis still remains limited. In this article, we further advance cancer FMR analysis based on histopathological imaging data. Significantly advancing from the existing analyses under heterogeneity and homogeneity, our goal is to simultaneously use two types of histopathological imaging features, which are extracted based on domain-specific biomedical knowledge and using automated signal processing software, respectively. A significant modeling/methodological advancement is that, to reflect the "increased resolution" of the second type of imaging features over the first type, we impose a hierarchy in the mixture structures. An effective and flexible Bayesian approach is proposed. Simulation shows its competitiveness over several alternatives. The TCGA lung cancer data is analyzed, and interesting heterogeneous structures different from using the alternatives are found. Overall, this study provides a new venue for FMR analysis for cancer and other complex diseases.

Jackknife for nonlinear estimating equations

In mixture with varying concentrations model (MVC) one deals with a nonhomogeneous sample which consists of subjects belonging to a fixed number of different populations (mixture components). The population which a subject belongs to is unknown, but the probabilities to belong to a given component are known and vary from observation to observation. The distribution of subjects’ observed features depends on the component which it belongs to. Generalized estimating equations (GEE) for Euclidean parameters in MVC models are considered. Under suitable assumptions the obtained estimators are asymptotically normal. A jackknife (JK) technique for the estimation of their asymptotic covariance matrices is described. Consistency of JK-estimators is demonstrated. An application to a model of mixture of nonlinear regressions and a real life example are presented.

Data-driven Steering Torque Behaviour Modelling with Hidden Markov Models

Modern Advanced Driver Assistance Systems (ADAS) are limited in their ability to consider the driver's intention, resulting in unnatural guidance and low customer acceptance. In this research, we focus on a novel data-driven approach to predict driver steering torque. In particular, driver behavior is modeled by learning the parameters of a Hidden Markov Model (HMM) and estimation is performed with Gaussian Mixture Regression (GMR). An extensive parameter selection framework enables us to objectively select the model hyper-parameters and prevents overfitting. The final model behavior is optimized with a cost function balancing between accuracy and smoothness. Naturalistic driving data covering seven participants is obtained using a static driving simulator at Toyota Motor Europe for the training, evaluation, and testing of the proposed model. The results demonstrate that our approach achieved a 92% steering torque accuracy with a 37% increase in signal smoothness and 90% fewer data compared to a baseline. In addition, our model captures the complex and nonlinear human behavior and inter-driver variability from novice to expert drivers, showing an interesting potential to become a steering performance predictor in future user-oriented ADAS.

State Prediction of Human-in-the-Loop Multi-rotor System with Stochastic Human Behavior Model

Reachability analysis is a widely used method to analyze the safety of a Human-in-the-Loop Cyber Physical System (HiLCPS). It allows the HiLCPS to respond against an imminent threat in advance by predicting reachable states of the system. However, it could lead to an unnecessarily conservative reachable set if the prediction only relies on the system dynamics without explicitly considering human behavior, and thus the risk might be overestimated. To avoid the conservativeness, we present a state probability distribution function (pdf) prediction method which takes into account a stochastic human behavior model represented as a Gaussian Mixture Model (GMM). In this paper, we focus on the multi-rotor controlled by a human operator in a near-collision situation. The stochastic human behavior model is trained using experimental data to represent the human operators’ evasive maneuver. Then, we can retrieve a human control input pdf from the trained stochastic human behavior model using the Gaussian Mixture Regression (GMR). The proposed algorithm predicts the multi-rotor's future state pdf by propagating the pdf of the retrieved human control input according to the given dynamics, which yields closed-loop analysis of the HiLCPS. Human subject experiment results are provided to demonstrate the effectiveness of the proposed algorithm.

Variable Selection for Robust Mixture Regression Model with Skew Scale Mixtures of Normal Distributions

In this paper, we propose a robust mixture regression model based on the skew scale mixtures of normal distributions (RMR-SSMN) which can accommodate asymmetric, heavy-tailed and contaminated data better. For the variable selection problem, the penalized likelihood approach with a new combined penalty function which balances the SCAD and l2 penalty is proposed. The adjusted EM algorithm is presented to get parameter estimates of RMR-SSMN models at a faster convergence rate. As simulations show, our mixture models are more robust than general FMR models and the new combined penalty function outperforms SCAD for variable selection. Finally, the proposed methodology and algorithm are applied to a real data set and achieve reasonable results.

Bayesian censored piecewise regression mixture models with skewness

ABSTRACT This paper presents censored mixture regression models with piecewise growth curves for assessing longitudinal data that exhibit multiphasic features. Such features may include censoring, skewness, measurement errors in covariates, and mixtures of unobserved subpopulations. In the process of describing those features, identification of differential effects of predictors on a response variable for a heterogeneous population (subpopulations) has recently been highly sought. Regression mixture models are key methods for assessing differential effects of predictors. In this article, we extend regression mixture models with normal distribution to incorporate (i) skew-normal distribution, (ii) left-censoring, (iii) measurement errors, and (iv) piecewise growth mixture modeling for describing multiphasic trajectories over time where the observed observations come from a mixture of unobserved subgroups. The proposed methods are illustrated using real data from an AIDS clinical study and a Bayesian approach.

Assessing the heterogeneity of social connectedness index via quantile regression mixture model

Assessing the heterogeneity of social connectedness index via quantile regression mixture model

Letter to the Editor: on the stability and internal consistency of component-wise sparse mixture regression-based clustering.

Understanding the relationship between molecular markers and a phenotype of interest is often obfuscated by patient-level heterogeneity. To address this challenge, Chang etal. recently published a novel method called Component-wise Sparse Mixture Regression (CSMR), a regression-based clustering method that promises to detect heterogeneous relationships between molecular markers and a phenotype of interest under high-dimensional settings. In this Letter to the Editor, we raise awareness to several issues concerning the assessment of CSMR in Chang etal., particularly its assessment in settings where the number of features, P, exceeds the study sample size, N, and advocate for additional metrics/approaches when assessing the performance of regression-based clustering methodologies.

Machine Learning Embedded Semiparametric Mixtures of Regressions with Covariate-Varying Mixing Proportions

A new class of semiparametric mixture regression models with covariate-varying mixing proportions is introduced by embedding machine learning methods into mixtures of regressions. The new method uses the neural network to estimate mixing proportions nonparametrically while using the maximum likelihood estimate to estimate all other component parameters. The new machine learning embedded semiparametric mixture regression models offer more flexible estimation compared to traditional parametric mixture regression models. More importantly, the new hybrid method could better estimate the effects of multivariate covariates nonparametrically than the traditional kernel regression methods, which suffer from the well known “curse of dimensionality”. The introduced hybrid idea can be easily extended to other semiparametric statistical models and other machine learning methods. Simulation studies and a real data application are used to demonstrate the effectiveness of the proposed new method and compare it with some other existing methods.

Assessing Differential Effects of Somatic Amplification to Positive Affect in Midlife and Late Adulthood-A Regression Mixture Approach.

This study aims to provide an empirical demonstration of a novel method, regression mixture model, by examining differential effects of somatic amplification to positive affect and identifying the predictors that contribute to the differential effects. Data derived from the second wave of Midlife in the United States. The analytic sample consisted of 1,766 adults aged from 33 to 84 years. Regression mixture models were fitted using Mplus 7.4, and a two-step model-building approach was adopted. Three latent groups were identified consisting of a maladaptive (32.1%), a vulnerable (62.5%), and a resilient (5.4%) group. Six covariates (i.e., age, education level, positive relations with others, purpose in life, depressive symptoms, and physical health) significantly predicted the latent class membership in the regression mixture model. The study demonstrated the regression mixture model to be a flexible and efficient statistical tool in assessing individual differences in response to adversity and identifying resilience factors, which contributes to aging research.

Online Bayesian learning for mixtures of spatial spline regressions with mixed effects

Classification and clustering methods based on univariate functions have been well developed. Recent work has extended the techniques to the domain of bivariate functions by incorporating the techniques based on mixtures of spatial spline regression with mixed-effects models. An Expectation Maximization (EM) algorithm is implemented to facilitate model inference. In this paper, we further extend the mixtures of spatial spline regression with mixed-effects model under the Bayesian framework to accommodate streaming image data. First, we derive a Markov chain Monte Carlo (MCMC) algorithm as an alternative approach to the EM algorithm to make inference on the model. However, MCMC is not scalable to streaming image data since it requires all observed information to update the posterior distribution of the parameters. To tackle this issue, we propose a sequential Monte Carlo (SMC) algorithm to analyse online fashion image data. The existence of model sufficient statistics improves the efficiency of the proposed online SMC algorithm. Instead of saving all batch data for inference, we only require storage of the model sufficient statistics and every data point is only used once, which is well suited for large-scale stream type data. In addition, the proposed algorithm provides an unbiased estimator of the marginal likelihood as a by-product of the approach, which can be used for model selection. Numerical experiments are used to demonstrate the effectiveness of our method. Our implementation is available at https://github.com/ShufeiGe/Online-Bayesian-learning-for-MMSRm.

Variety seeking behavior in the wine domain: A consumers segmentation using big data

This study investigates variety seeking behavior in the wine domain. Since variety seeking depends on brand strategies and consumers’ preferences for different types of vines and denominations, a bi-dimensional perspective is adopted. Two new variety seeking measures are defined, namely the Wine Index of Diversity and Brand Index of Diversity. A finite mixture regression model is implemented to identify and characterize groups of households sharing similar variety seeking behavior in a statistically representative sample of 8,313 Italian households. Four groups are identified based on consumer characteristics and their purchasing behavior. The largest group is “switchers,” which includes consumers showing a relatively higher wine diversity than brand diversity. Estimates reveal the “habitual” group, that lives in the southern Italy and consumes wine less frequently than all other groups. The “loyal” group includes the youngest consumers with an above average income, who reside in the northern regions. Finally, the “variety seekers” are older, have the highest incomes, and live in the central regions. This grouping provides insights into the effects of brand and wine typology on consumers’ choices.

Bayesian finite mixture of regression analysis for cancer based on histopathological imaging-environment interactions.

Cancer is a heterogeneous disease. Finite mixture of regression (FMR)-as an important heterogeneity analysis technique when an outcome variable is present-has been extensively employed in cancer research, revealing important differences in the associations between a cancer outcome/phenotype and covariates. Cancer FMR analysis has been based on clinical, demographic, and omics variables. A relatively recent and alternative source of data comes from histopathological images. Histopathological images have been long used for cancer diagnosis and staging. Recently, it has been shown that high-dimensional histopathological image features, which are extracted using automated digital image processing pipelines, are effective for modeling cancer outcomes/phenotypes. Histopathological imaging-environment interaction analysis has been further developed to expand the scope of cancer modeling and histopathological imaging-based analysis. Motivated by the significance of cancer FMR analysis and a still strong demand for more effective methods, in this article, we take the natural next step and conduct cancer FMR analysis based on models that incorporate low-dimensional clinical/demographic/environmental variables, high-dimensional imaging features, as well as their interactions. Complementary to many of the existing studies, we develop a Bayesian approach for accommodating high dimensionality, screening out noises, identifying signals, and respecting the "main effects, interactions" variable selection hierarchy. An effective computational algorithm is developed, and simulation shows advantageous performance of the proposed approach. The analysis of The Cancer Genome Atlas data on lung squamous cell cancer leads to interesting findings different from the alternative approaches.

Combining the strengths of inverse-variance weighting and Egger regression in Mendelian randomization using a mixture of regressions model.

With the increasing availability of large-scale GWAS summary data on various traits, Mendelian randomization (MR) has become commonly used to infer causality between a pair of traits, an exposure and an outcome. It depends on using genetic variants, typically SNPs, as instrumental variables (IVs). The inverse-variance weighted (IVW) method (with a fixed-effect meta-analysis model) is most powerful when all IVs are valid; however, when horizontal pleiotropy is present, it may lead to biased inference. On the other hand, Egger regression is one of the most widely used methods robust to (uncorrelated) pleiotropy, but it suffers from loss of power. We propose a two-component mixture of regressions to combine and thus take advantage of both IVW and Egger regression; it is often both more efficient (i.e. higher powered) and more robust to pleiotropy (i.e. controlling type I error) than either IVW or Egger regression alone by accounting for both valid and invalid IVs respectively. We propose a model averaging approach and a novel data perturbation scheme to account for uncertainties in model/IV selection, leading to more robust statistical inference for finite samples. Through extensive simulations and applications to the GWAS summary data of 48 risk factor-disease pairs and 63 genetically uncorrelated trait pairs, we showcase that our proposed methods could often control type I error better while achieving much higher power than IVW and Egger regression (and sometimes than several other new/popular MR methods). We expect that our proposed methods will be a useful addition to the toolbox of Mendelian randomization for causal inference.

Remaining useful life prediction of lithium-ion batteries based on stacked autoencoder and gaussian mixture regression

Lithium-ion batteries have been widely applied in energy storage systems and electric vehicles (EVs), the remaining useful life (RUL) prediction is one of the critical technologies for prognostics and health management. However, high accuracy RUL prediction with reliability is the biggest bottleneck. To improve RUL prediction and adaptively extract indirect health indicators (HIs), the RUL prediction framework based on the stacked autoencoder and Gaussian mixture regression (SAE-GMR) is proposed. Firstly, the indirect HIs are extracted from charging and discharging data, and the gray relation analysis (GRA) is adopted to analyze the relation with capacity. In this paper, the SAE neural network is proposed to reduce the dimensions and noise of battery and obtain a syncretic HI. Then, the GMR model is estiblished not only to improve accuracy of RUL prediction, but also describe the reliability. Finally, the proposed method is compared with esixting methods,which shows that the proposed model has superiority for other methods.

Variable selection in finite mixture of location and mean regression models using skew-normal distribution

In this article, a variable selection method for the finite mixture of location regression (FMLR) and the finite mixture of mean regression (FMMR) models with a skew-normal error are discussed. The univariate skew-normal distribution was introduced by Sahu et al. will be used in this work, which is attractive because estimation of the skewness parameter does not present the same degree of difficulty as in the case with Azzalini one and, moreover, it allows easy implementation of the EM algorithm. A penalized likelihood approach for variable selection in FMLR and FMMR models was introduced in this article. With a data-adaptive method for selecting tuning parameters, we establish the theoretical properties of our procedure, including consistency in variable selection, the oracle property in estimation. The EM algorithm facilitated by Gauss–Newton method for efficient numerical computations are developed. Simulation studies and a real data set are used to illustrate the proposed methodologies.

형성적 피드백이 고등학생의 수학성취에 미치는 차별적 효과 및 영향요인 검증: 회귀혼합모형의 적용

The main purpose of this study is to explore heterogeneous effects of formative feedback on math achievement and the factors affecting these patterns. To this end, a regression mixture analysis was conducted on 513 high school students in three regions nationwide. As a result of the analysis, two latent classes of feedback effects were derived. Specifically, the first pattern was labelled as ‘mixed effect group’ (35.87%), because monitoring had a negative effect while scaffolding had a positive effect on math achievement. The second group was labelled as ‘scaffolding effect group’ (64.13%), since only scaffolding had a positive effect on math achievement while monitoring did not. Next, a binomial logistic regression analysis was conducted to identify the factors predicting these differentiated feedback effects, and as a result, the effects of students’ feedback utilization ability and achievement goal orientation were significant. Lastly, the implications of this study through the verification of heterogeneous relationships of formative feedback with math achievement, and educational implications for improving the effectiveness of feedback were discussed.

Lifting the limitations of Gaussian mixture regression through coupling with principal component analysis and deep autoencoding

The mathematical modeling of correlations between target properties and their factors of influence, particularly that allowing inverse analysis, is an essential part of molecular, material, and process designs. In contrast to approaches employing pseudo-inverse analysis, Gaussian mixture regression (GMR), which assumes that the relationships between variables can be represented as a mixture of Gaussian distributions, allows for direct inverse analysis. However, as this model optimizes the means and variance–covariance matrices of all variables, parameter estimation becomes increasingly difficult with the increasing number of variables. Herein, this drawback is addressed by the transformation of explanatory variables X into latent variables Z before GMR modeling. As the inverse (Z to X) transformation is necessary for direct inverse analysis, principal component analysis (PCA) and deep autoencoding (DAE) are employed as dimensionality reduction methods. After X is transformed to Z with the help of PCA or DAE, a GMR model is constructed with Z and objective variables Y, and the proposed method is therefore denoted as PCA-GMR or DAE-GMR, respectively. As Z values can be predicted by inputting Y values into the GMR model and can be transformed to X values, direct inverse analysis is also possible. Given that unlabeled data can be employed to construct PCA and DAE models, the proposed methods can also be used as semi-supervised learning techniques. The predictive abilities of PCA-GMR and DAE-GMR are verified using molecular, material, and spectral datasets and surpass that of traditional GMR on all datasets, with the maximum reduction of prediction errors equaling 63%.

Measuring resilience to major life events

There is great interest in understanding who in the population is resilient in the face of major life events, and who is not. In this paper we construct a revealed measure of adulthood psychological resilience by modelling individuals’ responses to ten adverse life events using a dynamic finite mixture regression model applied to 17 years of panel data. Our methodology accounts for non-random selection into events, and differences between individuals in anticipation, immediate response, and speed of adaptation. We find considerable heterogeneity in individuals’ responses to events such as major financial shocks, redundancy and bereavement. We also find that our measure of resilience is correlated with clinical measures of mental health, and that it significantly predicts the psychological response to out-of-sample events. The strongest predictor of our measure of resilience is internal locus of control, which is an individual's belief that life outcomes are under their control.

3-D Epanechnikov Mixture Regression in integral imaging compression

Integral imaging is a kind of 3D display with no glasses, which represents the future developments. Elementary image array (EIA) is an essential component of integral imaging. Our coding framework includes pre-processing, modeling, and reconstruction. We acquire the sub-EIA from the original EIA and get the offsets between adjacent elementary images (EIs) through pre-processing. As for modeling, we get the optimal combination of 3-D Epanechnikov Mixture Regression (3-D EMR) or 3-D Gaussian Mixture Regression (3-D GMR) by Elementary Image Adaptive Model Selection (EI-AMLS) algorithm to achieve the best modeling of sub-EIA. Finally, the linear-based reconstruction is completed according to the correlation between adjacent EIs. Our decoded images realize a clearer outline reconstruction and more superior coding efficiency than HEVC and JPEG2000 below about 0.05bpp. Furthermore, the proposed method can achieve the same visual effect as HEVC with only 15% to 80% time consumed.