Finite Mixture Of Regression Models Research Articles

Sequential estimation for mixture of regression models for heterogeneous population

Heterogeneity among patients commonly exists in clinical studies and leads to challenges in medical research. It is widely accepted that there exist various sub-types in the population and they are distinct from each other. The approach of identifying the sub-types and thus tailoring disease prevention and treatment is known as precision medicine. The mixture model is a classical statistical model to cluster the heterogeneous population into homogeneous sub-populations. However, for the highly heterogeneous population with multiple components, its parameter estimation and clustering results may be ambiguous due to the dependence of the EM algorithm on the initial values. For sub-typing purposes, the finite mixture of regression models with concomitant variables is considered and a novel statistical method is proposed to identify the main components with large proportions in the mixture sequentially. Compared to existing typical statistical inferences, the new method not only requires no pre-specification on the number of components for model fitting, but also provides more reliable parameter estimation and clustering results. Simulation studies demonstrated the superiority of the proposed method. Real data analysis on the drug response prediction illustrated its reliability in the parameter estimation and capability to identify the important subgroup.

Finite mixture of regression models for censored data based on the skew-t distribution

Finite mixture of regression models for censored data based on the skew-t distribution

A finite mixture analysis of structural breaks in the G-7 gross domestic product series

In this paper we apply a clustering procedure to detect trend changes in macroeconomic data, focusing on the GDP time series for the G-7 countries. A finite mixture of regression models is considered to show different patterns and changes in GDP slopes over time in the long-trend component. Two popular trend-cycle decompositions (i.e., Beveridge and Nelson Decomposition and Hodrick and Prescott filter) are considered in a preliminary step of the analysis to stress the differences between the two methods in terms of the inferred clustering, if any. This approach can be used also to detect structural breaks or change points and it is an alternative to existing approaches in a probabilistic framework. We also discuss international changes in the GDP distribution for the G-7 countries, highlighting similarities, e.g., in break dates, aiming at adding more insights on the economic integration among countries. Our findings suggest that by looking at changes in slope over time a mixture of regression models is able to detect change points, also compared with alternative procedures.

A data-driven reversible jump for estimating a finite mixture of regression models

A data-driven reversible jump for estimating a finite mixture of regression models

Semiparametric finite mixture of regression models with Bayesian P-splines

AbstractMixture models provide a useful tool to account for unobserved heterogeneity and are at the basis of many model-based clustering methods. To gain additional flexibility, some model parameters can be expressed as functions of concomitant covariates. In this Paper, a semiparametric finite mixture of regression models is defined, with concomitant information assumed to influence both the component weights and the conditional means. In particular, linear predictors are replaced with smooth functions of the covariate considered by resorting to cubic splines. An estimation procedure within the Bayesian paradigm is suggested, where smoothness of the covariate effects is controlled by suitable choices for the prior distributions of the spline coefficients. A data augmentation scheme based on difference random utility models is exploited to describe the mixture weights as functions of the covariate. The performance of the proposed methodology is investigated via simulation experiments and two real-world datasets, one about baseball salaries and the other concerning nitrogen oxide in engine exhaust.

Sampling-based Gaussian Mixture Regression for Big Data

This paper proposes a nonuniform subsampling method for finite mixtures of regression models to reduce large data computational tasks. A general estimator based on a subsample is investigated, and its asymptotic normality is established. We assign optimal subsampling probabilities to data points that minimize the asymptotic mean squared errors of the general estimator and linearly transformed estimators. Since the proposed probabilities depend on unknown parameters, an implementable algorithm is developed. We first approximate the optimal subsampling probabilities using a pilot sample. After that, we select a subsample using the approximated subsampling probabilities and compute estimates using the subsample. We evaluate the proposed method in a simulation study and present a real data example using appliance energy data.

Variable selection in finite mixture of median regression models using skew-normal distribution

A regression model with skew-normal errors provides a useful extension for traditional normal regression models when the data involve asymmetric outcomes. Moreover, data that arise from a heterogeneous population can be efficiently analysed by a finite mixture of regression models. These observations motivate us to propose a novel finite mixture of median regression model based on a mixture of the skew-normal distributions to explore asymmetrical data from several subpopulations. With the appropriate choice of the tuning parameters, we establish the theoretical properties of the proposed procedure, including consistency for variable selection method and the oracle property in estimation. A productive nonparametric clustering method is applied to select the number of components, and an efficient EM algorithm for numerical computations is developed. Simulation studies and a real data set are used to illustrate the performance of the proposed methodologies.

Copula-based bivariate finite mixture regression models with an application for insurance claim count data

Modeling bivariate (or multivariate) count data has received increased interest in recent years. The aim is to model the number of different but correlated counts taking into account covariate information. Bivariate Poisson regression models based on the shock model approach are widely used because of their simple form and interpretation. However, these models do not allow for overdispersion or negative correlation, and thus, other models have been proposed in the literature to avoid these limitations. The present paper proposes copula-based bivariate finite mixture of regression models. These models offer some advantages since they have all the benefits of a finite mixture, allowing for unobserved heterogeneity and clustering effects, while the copula-based derivation can produce more flexible structures, including negative correlations and regressors. In this paper, the new approach is defined, estimation through an EM algorithm is presented, and then different models are applied to a Spanish insurance claim count database.

Finite Mixture Models for Clustering Auto-Correlated Sales Series Data Influenced by Promotions

The focus of the present paper is on clustering, namely the problem of finding distinct groups in a dataset so that each group consists of similar observations. We consider the finite mixtures of regression models, given their flexibility in modeling heterogeneous time series. Our study aims to implement a novel approach, which fits mixture models based on the spline and polynomial regression in the case of auto-correlated data, to cluster time series in an unsupervised machine learning framework. Given the assumption of auto-correlated data and the usage of exogenous variables in the mixture model, the usual approach of estimating the maximum likelihood parameters using the Expectation–Maximization (EM) algorithm is computationally prohibitive. Therefore, we provide a novel algorithm for model fitting combining auto-correlated observations with spline and polynomial regression. The case study of this paper consists of the task of clustering the time series of sales data influenced by promotional campaigns. We demonstrate the effectiveness of our method in a case study of 131 sales series data from a real-world company. Numerical outcomes demonstrate the efficacy of the proposed method for clustering auto-correlated time series. Despite the specific case study of this paper, the proposed method can be used in several real-world application fields.

The Effects of Postmaterialist Values on Average Life Satisfaction at the Country Level: What Generates Differences in Subjective Well-Being among European Countries?

The aim of this study is to clarify the social mechanisms that generate differences in the average life satisfaction among European countries by specifying the latent structures of life satisfaction. For this purpose, data from the European Values Study conducted from 2017 to 2020 (EVS 2017) were analyzed using finite mixtures of regression models. The results revealed that the respondents to the EVS 2017 could be categorized into two latent groups: a group dominated by materialist values and a group characterized by postmaterialist values. The results also imply that postmaterialist values increase average life satisfaction at the country level and are positively associated with economic prosperity and political stability at the country level. Consequently, the effect of postmaterialist values on life satisfaction is related to social inequalities among European countries. Finally, the sociological implications of these findings are discussed.

Robust order selection of mixtures of regression models with random effects

Finite mixtures of regression models with random effects are a very flexible statistical tool to model data, as these models allow to model the heterogeneity of the population and to account for multiple correlated observations from the same individual at the same time. The selection of the number of components for these models has been a long-standing challenging problem in statistics. However, the majority of the existent methods for the estimation of the number of components are not robust and, therefore, are quite sensitive to outliers. In this article we study a robust estimation of the number of components for mixtures of regression models with random effects, investigating the performance of trimmed information and classification criteria comparatively to the performance of the traditional information and classification criteria. The simulation study and a real-world application showcase the superiority of the trimmed information and classification criteria in the presence of contaminated data.

Variable selection for skew-normal mixture of joint location and scale models

Although there are many papers on variable selection methods based on mean model in the finite mixture of regression models, little work has been done on how to select significant explanatory variables in the modeling of the variance parameter. In this paper, we propose and study a novel class of models: a skew-normal mixture of joint location and scale models to analyze the heteroscedastic skew-normal data coming from a heterogeneous population. The problem of variable selection for the proposed models is considered. In particular, a modified Expectation-Maximization(EM) algorithm for estimating the model parameters is developed. The consistency and the oracle property of the penalized estimators is established. Simulation studies are conducted to investigate the finite sample performance of the proposed methodologies. An example is illustrated by the proposed methodologies.

Variable selection in finite mixture of regression models with an unknown number of components

A Bayesian framework for finite mixture models to deal with model selection and the selection of the number of mixture components simultaneously is presented. For that purpose, a feasible reversible jump Markov Chain Monte Carlo algorithm is proposed to model each component as a sparse regression model. This approach is made robust to outliers by using a prior that induces heavy tails and works well under multicollinearity and with high-dimensional data. Finally, the framework is applied to cross-sectional data investigating early warning indicators. The results reveal two distinct country groups for which estimated effects of vulnerability indicators vary considerably.

Cluster Analysis of Microbiome Data by Using Mixtures of Dirichlet–Multinomial Regression Models

SummaryThe human gut microbiome is one of the fundamental components of our physiology, and exploring the relationship between biological and environmental covariates and the resulting taxonomic composition of a given microbial community is an active area of research. Previously, a Dirichlet–multinomial regression framework has been suggested to model this relationship, but it did not account for any underlying latent group structure. An underlying group structure of guts (such as enterotypes) has been observed across gut microbiome samples in which guts in the same group share similar biota compositions. In the paper, a finite mixture of Dirichlet–multinomial regression models is proposed that accounts for this underlying group structure and to allow for a probabilistic investigation of the relationship between bacterial abundance and biological and/or environmental covariates within each inferred group. Furthermore, finite mixtures of regression models which incorporate the concomitant effect of the covariates on the resulting mixing proportions are also proposed and examined within the Dirichlet–multinomial framework. We utilize the proposed mixture model to gain insight on underlying subgroups in a microbiome data set comprising tumour and healthy samples and the relationships between covariates and microbial abundance in those subgroups.

Self-assessed life expectancy among older adults in C\xf4te d\u2019Ivoire

BackgroundThe purpose of this study was to estimate individuals’ expected longevity based on self-assessed survival probabilities and determine the predictors of such subjective life expectancy in a sample of elderly people (50 years and older) in Côte d’Ivoire.MethodsPaper-based questionnaires were administered to a sample (n = 267) of older adults residing in the city of Dabou, Côte d’Ivoire in May 2017. Information on subjective expectations regarding health, comorbidities, and self-assessed survival probabilities was collected. We estimated self-assessed life expectancy and its determinants using a two-pronged approach by: (i) estimating individuals’ life expectancy using the self-assessed survival probabilities (SSPs), and (ii) applying a finite mixture of regression models to form homogenous groups of individuals (clusters/components) and investigate the determinants.A spline-based approach was used to estimate the overall distribution of life expectancy for each individual using two to four points of self-assessed survival probabilities. A finite mixture of regression models was used to identify homogeneous groups of individuals (i.e. clusters/components) of the overall subjective life expectancy distribution of the study participants.ResultsThe mean subjective life expectancy in older people varied according to four components/clusters. The average subjective life expectancy among the elderly was 79.51, 78.89, 80.02, and 77.79 years in the first, second, third, and fourth component of the subjects’ overall subjective life expectancy, respectively. The effect of sociodemographic characteristics, comorbidities, and lifestyle on subjective life expectancy varied across components. For instance, a U-shape relationship between household per capita income and subjective life expectancy was found for individuals classified into the third component, and an inverse U-shape relationship was found for individuals classified into the fourth component.ConclusionsWe extended the estimation of subjective life expectancy by accounting for heterogeneity in the distribution of the estimated subjective life expectancy. This approach improved the usual methods for estimating individual subjective life expectancies and may provide insight into the elderly’s perception of aging, which could be used to forecast the demand for health services and long-term care needs.

Two Types of Support for Redistribution of Wealth: Consistent and Inconsistent Policy Preferences

This article aims to clarify the latent structure of support for redistribution. To this end, the author analyzed data from the National Survey of Social Stratification and Social Mobility in 2015 (SSM 2015), which was conducted in Japan, using finite mixtures of regression models. The results revealed that the population could be categorized into two latent groups: one that determines preferences for social policies based on self-interest and another that does so based on ideology. Surprisingly, the results also showed that, compared to those who supported redistribution of wealth based on ideology, those who supported them based on self-interest were more likely to hold inconsistent preferences (e.g., simultaneous support for redistribution of wealth and free-market competition). This observation implies that, even when individuals want to determine their policy preferences rationally, they often do not have enough information to correctly assess the influence of each social policy on their self-interest.

Penalized proportion estimation for non parametric mixture of regressions

In this article, we propose a penalized local log-likelihood method to locally select the number of components in non parametric finite mixture of regression models via proportion shrinkage method. Mean functions and variance functions are estimated simultaneously. We show that the number of components can be estimated consistently, and further establish asymptotic normality of functional estimates. We use a modified EM algorithm to estimate the unknown functions. Simulations are conducted to demonstrate the performance of the proposed method. We illustrate our method via an empirical analysis of the housing price index data of United States.

Variable selection in finite mixture of regression models using the skew-normal distribution

Variable selection in finite mixture of regression (FMR) models is frequently used in statistical modeling. The majority of applications of variable selection in FMR models use a normal distribution for regression error. Such assumptions are unsuitable for a set of data containing a group or groups of observations with asymmetric behavior. In this paper, we introduce a variable selection procedure for FMR models using the skew-normal distribution. With appropriate choice of the tuning parameters, we establish the theoretical properties of our procedure, including consistency in variable selection and the oracle property in estimation. To estimate the parameters of the model, a modified EM algorithm for numerical computations is developed. The methodology is illustrated through numerical experiments and a real data example.

Structured analysis of the high-dimensional FMR model

The finite mixture of regression (FMR) model is a popular tool for accommodating data heterogeneity. In the analysis of FMR models with high-dimensional covariates, it is necessary to conduct regularized estimation and identify important covariates rather than noises. In the literature, there has been a lack of attention paid to the differences among important covariates, which can lead to the underlying structure of covariate effects. Specifically, important covariates can be classified into two types: those that behave the same in different subpopulations and those that behave differently. It is of interest to conduct structured analysis to identify such structures, which will enable researchers to better understand covariates and their associations with outcomes. Specifically, the FMR model with high-dimensional covariates is considered. A structured penalization approach is developed for regularized estimation, selection of important variables, and, equally importantly, identification of the underlying covariate effect structure. The proposed approach can be effectively realized, and its statistical properties are rigorously established. Simulation demonstrates its superiority over alternatives. In the analysis of cancer gene expression data, interesting models/structures missed by the existing analysis are identified.

Finite mixture of regression models for a stratified sample

ABSTRACTDespite the popularity and importance, there is limited work on modelling data which come from complex survey design using finite mixture models. In this work, we explored the use of finite mixture regression models when the samples were drawn using a complex survey design. In particular, we considered modelling data collected based on stratified sampling design. We developed a new design-based inference where we integrated sampling weights in the complete-data log-likelihood function. The expectation–maximisation algorithm was developed accordingly. A simulation study was conducted to compare the new methodology with the usual finite mixture of a regression model. The comparison was done using bias-variance components of mean square error. Additionally, a simulation study was conducted to assess the ability of the Bayesian information criterion to select the optimal number of components under the proposed modelling approach. The methodology was implemented on real data with good results.