Finite Mixture Regression Research Articles

Change point detection via Gaussian mixture model

Change point detection aims to find abrupt changes in time series data. These changes denote substantial modifications to the process; these can be modeled as a change in the distribution (in location, scale, or trend). Traditional changepoint detection methods often rely on a cost function to assess if a change occurred in a series. Here, change point detection is investigated in a mixture-model-based clustering framework and a novel change point detection algorithm is developed using a finite mixture of regressions with concomitant variables. Through the introduction of a label correction mechanism, the unstructured clustering-based labels are treated as ordered and distinct segment labels. This approach can detect change points in both univariate and multivariate time series, and different kinds of change can be captured using a parsimonious family of models. Performance is illustrated on both simulated and real data. Journal of Statistical Research 2024, Vol. 58, No. 1, pp. 197-219.

Evaluating regression techniques for service advisor performance analysis in automotive dealerships

Service advisors are a crucial contributor to the profit and loss of an automotive dealership. It is thus crucial that their performances are continuously evaluated to make decisions that ensure continued profitability. Artificial intelligence (AI) algorithms like the finite mixture of regression (FMR) have been previously explored as a tool for benchmarking the performance of dealerships. However, it is essential that the AI algorithm is trustworthy and that its predictions can be explained. This study aims to develop a performance analytics platform that is reliable and explainable. It develops and evaluates variants of FMR based on different regression techniques, focusing on both reliability and explainability. The results show that the implemented FMR variants not only provide accurate performance assessments but also offer transparent insights into the factors influencing dealership performance.

Financial Inclusion, Globalization and Structural Transformation in Developing Countries: A Finite Mixture-of-Regressions Approach

Industrialization plays a key role in job creation and economic development. We extend the abundant literature on the determinants of industrialization by exploring the potential effects of financial inclusion in 64 developing countries over the period 2000-2020. Using a finite mixture regression model, we find that our sample is best described by a model with three classes of countries. We then incorporate the potential presence of hidden heterogeneity and, we explain the class membership of countries based on globalization. We find that the impact of financial inclusion on industrialization differs across these three classes of countries. Specifically, financial inclusion is negatively associated with industrialization in the first class of countries where the level of industrialization is low, while it positively affects industrialization in relatively more industrialized countries (class 2 and class 3). We also show that better-integrated countries into the global economy are more likely to be in the class where financial inclusion promotes industrialization. Our results are robust to several robustness checks. Our findings suggest that developing countries could reap more benefits from financial inclusion by strengthening their integration into the global economy.

Financial inclusion, household consumption instability and democracy in developing countries: A finite mixture of regressions approach

The literature on the impact of financial inclusion on the instability of household consumption remains controversial. In this article, we considered the possibility that countries follow different regimes of household consumption instability and tested the hypothesis of whether financial inclusion is countercyclical or procyclical to a country's regime of consumption instability. Thus, relying on recent works that used the finite mixture regression method, we identify the different regimes of household consumption instability. Drawing on a sample of 28 developing countries over the period 2011–2020, the paper highlights that the impact of financial inclusion on consumption instability differs across classes and a three-class model best describes the sample. Specifically, we show that financial inclusion is positively associated with consumption instability in the first class, while financial inclusion significantly reduces consumption instability in the second class. Financial inclusion does not have a significant impact on the third class. Furthermore, exploring the potential role of democracy in determining class affiliation, the results emphasize that developing countries would fully benefit from financial inclusion by undertaking strong reforms to improve democracy.

Modeling agricultural and methane emission data: a finite mixture regression approach

In this paper, the method for unsupervised learning of finite mixture regression (FMR) models is presented for evaluation using agricultural and emissions data sets. The FMR models can be written as problems with incomplete data, and the expectation–maximization (EM) algorithm can be used to estimate unknown variables. The goals of this research are to find the best clustering model with different sets of training and test data and examine the relationship between crop production index and methane emissions in 22 countries from 1990 to 2019 using FMR. In this study also use machine learning process for a FMR model from real world data. According to the findings, the performance of the random training data (RDM) in time series is preferable to that of the fixed training data (FXM). In addition, both RDM and FXM are capable of classifying the 22 countries into two distinct groups and constructing the parameters for the regression model. However, selecting training and test data will result in a good prediction; it is dependent on the data collected. Picking the right training and test data is crucial for accurate predictions-it all comes down to having good data in the first place.

Do natural resources rents reduce income inequality? A finite mixture of regressions approach

The World Bank report (2023) highlights that almost all types of natural capital, the global stock of resources and services provided by nature, are in decline while inequalities persist. Based on this context, the objective of this article is to analyze the effect of natural resources on income inequality in 73 developing countries over the period 2005–2020. Unlike previous works, this article examines the possibility that countries follow different inequality regimes and test the hypothesis that whether natural resources, reduce or increase income inequality depending on the inequality regime to which countries belong. We used a finite mixture regression model, and we found that our sample is best described by a model with five regimes of countries. The results show that the effect of natural resource rents on income inequality varies across five distinct regimes of countries. In regimes 1 and 2, natural resources have a positive effect on inequality, while in regimes 3 and 4, they have a negative effect on inequality. In regime 5, the effect is not significant. Furthermore, our analysis shows that countries that have low levels of political risk are likely to be in the regime where the dependence of natural resources decreases income inequality. These results clearly emphasize that for countries with abundant natural resources to fully benefit from reduced inequality, they must undertake strong reforms to reduce political risk. These reforms include improving the quality of the judicial system, intensifying the fight against corruption, and reducing investment risks.

Family-oriented versus career seekers: mixture regression separation

Using a finite mixture regression approach, we identify two groups of individuals, family- and career-oriented. Depending on whether the employment status is treated as a binomial or trinomial (with part-time work) variable, 55–70% of women and 4–10% of men belong to the family-oriented group. Children variables are the main source of differences between the two identified groups and between the family-oriented group and the average woman. Treating all women as one group leads to averaged-out estimates of children effects and underestimates the importance of part-time work for family-oriented women. Our results are important for studying women’s labour supply and for designing policies aimed at encouraging women’s labour market involvement.

HYFIS vs FMR, LWR and Least squares regression methods in estimating uniaxial compressive strength of evaporitic rocks

The uniaxial compressive strength (UCS) of the rock is one of the most important design parameters in various engineering applications. Therefore, the UCS requires to be either preciously measured through extensive field and laboratory studies or could be estimated by employing machine learning techniques and several other measured physical and mechanical explanatory rock parameters. This study is proposed to estimate the UCS of the evaporitic rocks by using a simple, measured point load index (PLI) and Schmidt Hammer (SHVRB) test rock blocks of evaporitic rocks. Finite mixture regression model (FMR), hybrid fuzzy inference systems model (HYFIS), multiple regression model (MLR), and locally weighted regression (LWR) are employed to predict the UCS. Different algorithms are implemented, including expectation–maximization (EM) algorithm, Mamdani fuzzy rule structures, Gradient descent-based learning algorithm with multilayer perceptron (MLP), and the least squares. Coefficient of Determination (R2), Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and A20-index accuracy measures are used to compare the performances of the competing models. Based on all the above measures, LWR outperformed with the other models whereas the HYFIS model has a slight advantage over the other two models.

Sparse estimation in semiparametric finite mixture of varying coefficient regression models.

Finite mixture of regressions (FMR) are commonly used to model heterogeneous effects of covariates on a response variable in settings where there are unknown underlying subpopulations. FMRs, however, cannot accommodate situations where covariates' effects also vary according to an "index" variable-known as finite mixture of varying coefficient regression (FM-VCR). Although complex, this situation occurs in real data applications: the osteocalcin (OCN) data analyzed in this manuscript presents a heterogeneous relationship where the effect of a genetic variant on OCN in each hidden subpopulation varies over time. Oftentimes, the number of covariates with varying coefficients also presents a challenge: in the OCN study, genetic variants on the same chromosome are considered jointly. The relative proportions of hidden subpopulations may also change over time. Nevertheless, existing methods cannot provide suitable solutions for accommodating all these features in real data applications. To fill this gap, we develop statistical methodologies based on regularized local-kernel likelihood for simultaneous parameter estimation and variable selection in sparse FM-VCR models. We study large-sample properties of the proposed methods. We then carry out a simulation study to evaluate the performance of various penalties adopted for our regularized approach and ascertain the ability of a BIC-type criterion for estimating the number of subpopulations. Finally, we applied the FM-VCR model to analyze the OCN data and identified several covariates, including genetic variants, that have age-dependent effects onOCN.

HETEROGENEITY ANALYSIS VIA INTEGRATING MULTI-SOURCES HIGH-DIMENSIONAL DATA WITH APPLICATIONS TO CANCER STUDIES.

This study has been motivated by cancer research, in which heterogeneity analysis plays an important role and can be roughly classified as unsupervised or supervised. In supervised heterogeneity analysis, the finite mixture of regression (FMR) technique is used extensively, under which the covariates affect the response differently in subgroups. High-dimensional molecular and, very recently, histopathological imaging features have been analyzed separately and shown to be effective for heterogeneity analysis. For simpler analysis, they have been shown to contain overlapping, but also independent information. In this article, our goal is to conduct the first and more effective FMR-based cancer heterogeneity analysis by integrating high-dimensional molecular and histopathological imaging features. A penalization approach is developed to regularize estimation, select relevant variables, and, equally importantly, promote the identification of independent information. Consistency properties are rigorously established. An effective computational algorithm is developed. A simulation and an analysis of The Cancer Genome Atlas (TCGA) lung cancer data demonstrate the practical effectiveness of the proposed approach. Overall, this study provides a practical and useful new way of conducting supervised cancer heterogeneity analysis.

Sparse Estimation in Finite Mixture of Accelerated Failure Time and Mixture of Regression Models with R Package fmrs

Variable selection in large-dimensional data has been extensively studied in different settings over the past decades. In a recent article, Shokoohi et. al. [29, DOI:10.1214/18-AOAS1198] proposed a method for variable selection in finite mixture of accelerated failure time regression models for studies on time-to-event data to capture heterogeneity within the population and account for censoring. In this paper, we introduce the fmrs package, which implements the variable selection methodology for such models. Furthermore, as a byproduct, the fmrs package facilitates variable selection in finite mixture regression models. The package also incorporates a tuning parameter selection mechanism based on component-wise bic. Commonly used penalties, such as Least Absolute Shrinkage and Selection Operator, and Smoothly Clipped Absolute Deviation, are integrated into fmrs. Additionally, the package offers an option for non-mixture regression models. The C language is chosen to boost the optimization speed. We provide an overview of the fmrs principles and the strategies employed for optimization. Hands-on illustrations are presented to help users get acquainted with fmrs. Finally, we apply fmrs to a lung cancer dataset and observe that a two-component mixture model reveals a subgroup with a more aggressive form of the disease, displaying a lower survival time.

Performance evaluation of automotive dealerships using grouped mixture of regressions

Finite Mixture of Regressions (FMR) are among the most widely used models for dealing with heterogeneity in regression problems. FMR is a model-based clustering approach that models the data by assigning individual observations to one of the K latent regression clusters. In some applications, it is desired to cluster groups of observations together rather than individually. We present an extension to the regular FMR, that we call the Grouped Mixture of Regressions (GMR), which allows for a known group structure among observations, in addition to their possibly unknown heterogeneity. The research is motivated by a large financial longitudinal dataset from a brand of automotive dealerships across the United States. The task was to cluster the dealers based on their financial performance and also improve performance prediction for individual dealers. We derive a fast and salable algorithm for estimating the model parameters using Expectation–Maximization (EM). We also show how the group structure can improve prediction by sharing information among members of each group, as reflected in the posterior predictive density under GMR. The performance of the approach is assessed using both synthetic data as well as the dealership data. Among our findings are the superior predictive performance of GMR and the ability to correctly detect the number of clusters using simple cross-validation.

Copula-based bivariate finite mixture regression models with an application for insurance claim count data

Modeling bivariate (or multivariate) count data has received increased interest in recent years. The aim is to model the number of different but correlated counts taking into account covariate information. Bivariate Poisson regression models based on the shock model approach are widely used because of their simple form and interpretation. However, these models do not allow for overdispersion or negative correlation, and thus, other models have been proposed in the literature to avoid these limitations. The present paper proposes copula-based bivariate finite mixture of regression models. These models offer some advantages since they have all the benefits of a finite mixture, allowing for unobserved heterogeneity and clustering effects, while the copula-based derivation can produce more flexible structures, including negative correlations and regressors. In this paper, the new approach is defined, estimation through an EM algorithm is presented, and then different models are applied to a Spanish insurance claim count database.

Variable Selection Algorithm for a Mixture of Poisson Regression for Handling Overdispersion in Claims Frequency Modeling Using Telematics Car Driving Data

In automobile insurance, it is common to adopt a Poisson regression model to predict the number of claims as part of the actuarial pricing process. The Poisson assumption can rarely be justified, often due to overdispersion, and alternative modeling is often considered, typically zero-inflated models, which are special cases of finite mixture distributions. Finite mixture regression modeling of telematics data is challenging to implement since the huge number of covariates computationally prohibits the essential variable selection needed to attain a model with desirable predictive power devoid of overfitting. This paper aims at devising an algorithm that can carry the task of variable selection in the presence of a large number of covariates. This is achieved by generating sub-samples of the data corresponding to each component of the Poisson mixture, and wherein variable selection is applied following the enhancement of the Poisson assumption by means of controlling the number of zero claims. The resulting algorithm is assessed by measuring the out-of-sample AUC (Area Under the Curve), a Machine Learning tool for quantifying predictive power. Finally, the application of the algorithm is demonstrated by using data of claim history and telematics data describing driving behavior. It transpires that unlike alternative algorithms related to Poisson regression, the proposed algorithm is both implementable and enjoys an improved AUC (0.71). The proposed algorithm allows more accurate pricing in an era where telematics data is used for automobile insurance.

Estimation for finite mixture of mode regression models using skew-normal distribution

The mode of a distribution provides a significant summary. A regression model with skew-normal errors provides a useful extension for traditional normal regression models when the data involve asymmetric outcomes. Finite mixture of regression (FMR) models are frequently used to analyze data that arise from a heterogeneous population. This work develops a new data analysis tool called finite mixture of mode regression (FMMoR) in order to explore skewed data from several subpopulations. The main virtue of considering the FMMoR models for skewed data is that this class of models has a nice hierarchical representation which allows easy implementation of inferences. A productive clustering method via mode identification is applied to select the number of components. A modified expectation-maximization algorithm facilitated by the two-point step size gradient descent method (GDEM), simultaneously, is developed for the inference. The proposed methods are evaluated through some simulation studies and illustrated by a real dataset.

Mixture regression for longitudinal data based on joint mean–covariance model

In the process of modeling longitudinal data, we focus on the case that the studied population is comprised of different groups of individuals and individuals within the same group share the similar kind of mean progression trajectories, where finite mixture models (FMM) are often used to address this kind of unobserved heterogeneity in terms of mean. Existing methods, such as parametric and semiparametric mixture regression, usually model the mean in each subpopulation with assumption that observations sharing a common trajectory are independent or their covariance structure is pre-specified, but less research considers modeling of covariance structures while accounting for heterogeneity. In this paper, we introduce a joint model which models the mean and covariance structures simultaneously in a finite normal mixture regression, demonstrating how important the within-subject correlation is in clustering longitudinal data. Model parameters are estimated with an iteratively re-weighted least squares EM (IRLS-EM) algorithm. Our estimators are shown to be consistent and asymptotically normal. We can identify different mean trajectories and covariance structures in all clusters. Simulations show that the proposed method performs well and gives more accurate clustering results by introducing covariance modeling. Real data analysis is also used to illustrate the usefulness of the proposed method, and it presents good performance in clustering COVID-19 deaths for European countries in terms of progression trajectory.

Bayesian hierarchical finite mixture of regression for histopathological imaging-based cancer data analysis.

Cancer is heterogeneous, and for seemingly similar cancer patients, the associations between an outcome/phenotype and covariates can be different. To describe such differences, finite mixture of regression (FMR) and other modeling techniques have been developed. "Classic" FMR analysis has usually been based on clinical, demographic, and molecular variables. More recently, histopathological imaging data-which is a byproduct of biopsy and enjoys broader data availability and higher cost-effectiveness-has been increasingly used in cancer modeling, although it is noted that its application to cancer FMR analysis still remains limited. In this article, we further advance cancer FMR analysis based on histopathological imaging data. Significantly advancing from the existing analyses under heterogeneity and homogeneity, our goal is to simultaneously use two types of histopathological imaging features, which are extracted based on domain-specific biomedical knowledge and using automated signal processing software, respectively. A significant modeling/methodological advancement is that, to reflect the "increased resolution" of the second type of imaging features over the first type, we impose a hierarchy in the mixture structures. An effective and flexible Bayesian approach is proposed. Simulation shows its competitiveness over several alternatives. The TCGA lung cancer data is analyzed, and interesting heterogeneous structures different from using the alternatives are found. Overall, this study provides a new venue for FMR analysis for cancer and other complex diseases.

Variety seeking behavior in the wine domain: A consumers segmentation using big data

This study investigates variety seeking behavior in the wine domain. Since variety seeking depends on brand strategies and consumers’ preferences for different types of vines and denominations, a bi-dimensional perspective is adopted. Two new variety seeking measures are defined, namely the Wine Index of Diversity and Brand Index of Diversity. A finite mixture regression model is implemented to identify and characterize groups of households sharing similar variety seeking behavior in a statistically representative sample of 8,313 Italian households. Four groups are identified based on consumer characteristics and their purchasing behavior. The largest group is “switchers,” which includes consumers showing a relatively higher wine diversity than brand diversity. Estimates reveal the “habitual” group, that lives in the southern Italy and consumes wine less frequently than all other groups. The “loyal” group includes the youngest consumers with an above average income, who reside in the northern regions. Finally, the “variety seekers” are older, have the highest incomes, and live in the central regions. This grouping provides insights into the effects of brand and wine typology on consumers’ choices.

Bayesian finite mixture of regression analysis for cancer based on histopathological imaging-environment interactions.

Cancer is a heterogeneous disease. Finite mixture of regression (FMR)-as an important heterogeneity analysis technique when an outcome variable is present-has been extensively employed in cancer research, revealing important differences in the associations between a cancer outcome/phenotype and covariates. Cancer FMR analysis has been based on clinical, demographic, and omics variables. A relatively recent and alternative source of data comes from histopathological images. Histopathological images have been long used for cancer diagnosis and staging. Recently, it has been shown that high-dimensional histopathological image features, which are extracted using automated digital image processing pipelines, are effective for modeling cancer outcomes/phenotypes. Histopathological imaging-environment interaction analysis has been further developed to expand the scope of cancer modeling and histopathological imaging-based analysis. Motivated by the significance of cancer FMR analysis and a still strong demand for more effective methods, in this article, we take the natural next step and conduct cancer FMR analysis based on models that incorporate low-dimensional clinical/demographic/environmental variables, high-dimensional imaging features, as well as their interactions. Complementary to many of the existing studies, we develop a Bayesian approach for accommodating high dimensionality, screening out noises, identifying signals, and respecting the "main effects, interactions" variable selection hierarchy. An effective computational algorithm is developed, and simulation shows advantageous performance of the proposed approach. The analysis of The Cancer Genome Atlas data on lung squamous cell cancer leads to interesting findings different from the alternative approaches.

Measuring resilience to major life events

There is great interest in understanding who in the population is resilient in the face of major life events, and who is not. In this paper we construct a revealed measure of adulthood psychological resilience by modelling individuals’ responses to ten adverse life events using a dynamic finite mixture regression model applied to 17 years of panel data. Our methodology accounts for non-random selection into events, and differences between individuals in anticipation, immediate response, and speed of adaptation. We find considerable heterogeneity in individuals’ responses to events such as major financial shocks, redundancy and bereavement. We also find that our measure of resilience is correlated with clinical measures of mental health, and that it significantly predicts the psychological response to out-of-sample events. The strongest predictor of our measure of resilience is internal locus of control, which is an individual's belief that life outcomes are under their control.