Finite Mixture Regression Models Research Articles

Modeling agricultural and methane emission data: a finite mixture regression approach

In this paper, the method for unsupervised learning of finite mixture regression (FMR) models is presented for evaluation using agricultural and emissions data sets. The FMR models can be written as problems with incomplete data, and the expectation–maximization (EM) algorithm can be used to estimate unknown variables. The goals of this research are to find the best clustering model with different sets of training and test data and examine the relationship between crop production index and methane emissions in 22 countries from 1990 to 2019 using FMR. In this study also use machine learning process for a FMR model from real world data. According to the findings, the performance of the random training data (RDM) in time series is preferable to that of the fixed training data (FXM). In addition, both RDM and FXM are capable of classifying the 22 countries into two distinct groups and constructing the parameters for the regression model. However, selecting training and test data will result in a good prediction; it is dependent on the data collected. Picking the right training and test data is crucial for accurate predictions-it all comes down to having good data in the first place.

Do natural resources rents reduce income inequality? A finite mixture of regressions approach

The World Bank report (2023) highlights that almost all types of natural capital, the global stock of resources and services provided by nature, are in decline while inequalities persist. Based on this context, the objective of this article is to analyze the effect of natural resources on income inequality in 73 developing countries over the period 2005–2020. Unlike previous works, this article examines the possibility that countries follow different inequality regimes and test the hypothesis that whether natural resources, reduce or increase income inequality depending on the inequality regime to which countries belong. We used a finite mixture regression model, and we found that our sample is best described by a model with five regimes of countries. The results show that the effect of natural resource rents on income inequality varies across five distinct regimes of countries. In regimes 1 and 2, natural resources have a positive effect on inequality, while in regimes 3 and 4, they have a negative effect on inequality. In regime 5, the effect is not significant. Furthermore, our analysis shows that countries that have low levels of political risk are likely to be in the regime where the dependence of natural resources decreases income inequality. These results clearly emphasize that for countries with abundant natural resources to fully benefit from reduced inequality, they must undertake strong reforms to reduce political risk. These reforms include improving the quality of the judicial system, intensifying the fight against corruption, and reducing investment risks.

HYFIS vs FMR, LWR and Least squares regression methods in estimating uniaxial compressive strength of evaporitic rocks

The uniaxial compressive strength (UCS) of the rock is one of the most important design parameters in various engineering applications. Therefore, the UCS requires to be either preciously measured through extensive field and laboratory studies or could be estimated by employing machine learning techniques and several other measured physical and mechanical explanatory rock parameters. This study is proposed to estimate the UCS of the evaporitic rocks by using a simple, measured point load index (PLI) and Schmidt Hammer (SHVRB) test rock blocks of evaporitic rocks. Finite mixture regression model (FMR), hybrid fuzzy inference systems model (HYFIS), multiple regression model (MLR), and locally weighted regression (LWR) are employed to predict the UCS. Different algorithms are implemented, including expectation–maximization (EM) algorithm, Mamdani fuzzy rule structures, Gradient descent-based learning algorithm with multilayer perceptron (MLP), and the least squares. Coefficient of Determination (R2), Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and A20-index accuracy measures are used to compare the performances of the competing models. Based on all the above measures, LWR outperformed with the other models whereas the HYFIS model has a slight advantage over the other two models.

Sparse Estimation in Finite Mixture of Accelerated Failure Time and Mixture of Regression Models with R Package fmrs

Variable selection in large-dimensional data has been extensively studied in different settings over the past decades. In a recent article, Shokoohi et. al. [29, DOI:10.1214/18-AOAS1198] proposed a method for variable selection in finite mixture of accelerated failure time regression models for studies on time-to-event data to capture heterogeneity within the population and account for censoring. In this paper, we introduce the fmrs package, which implements the variable selection methodology for such models. Furthermore, as a byproduct, the fmrs package facilitates variable selection in finite mixture regression models. The package also incorporates a tuning parameter selection mechanism based on component-wise bic. Commonly used penalties, such as Least Absolute Shrinkage and Selection Operator, and Smoothly Clipped Absolute Deviation, are integrated into fmrs. Additionally, the package offers an option for non-mixture regression models. The C language is chosen to boost the optimization speed. We provide an overview of the fmrs principles and the strategies employed for optimization. Hands-on illustrations are presented to help users get acquainted with fmrs. Finally, we apply fmrs to a lung cancer dataset and observe that a two-component mixture model reveals a subgroup with a more aggressive form of the disease, displaying a lower survival time.

Copula-based bivariate finite mixture regression models with an application for insurance claim count data

Modeling bivariate (or multivariate) count data has received increased interest in recent years. The aim is to model the number of different but correlated counts taking into account covariate information. Bivariate Poisson regression models based on the shock model approach are widely used because of their simple form and interpretation. However, these models do not allow for overdispersion or negative correlation, and thus, other models have been proposed in the literature to avoid these limitations. The present paper proposes copula-based bivariate finite mixture of regression models. These models offer some advantages since they have all the benefits of a finite mixture, allowing for unobserved heterogeneity and clustering effects, while the copula-based derivation can produce more flexible structures, including negative correlations and regressors. In this paper, the new approach is defined, estimation through an EM algorithm is presented, and then different models are applied to a Spanish insurance claim count database.

Variety seeking behavior in the wine domain: A consumers segmentation using big data

This study investigates variety seeking behavior in the wine domain. Since variety seeking depends on brand strategies and consumers’ preferences for different types of vines and denominations, a bi-dimensional perspective is adopted. Two new variety seeking measures are defined, namely the Wine Index of Diversity and Brand Index of Diversity. A finite mixture regression model is implemented to identify and characterize groups of households sharing similar variety seeking behavior in a statistically representative sample of 8,313 Italian households. Four groups are identified based on consumer characteristics and their purchasing behavior. The largest group is “switchers,” which includes consumers showing a relatively higher wine diversity than brand diversity. Estimates reveal the “habitual” group, that lives in the southern Italy and consumes wine less frequently than all other groups. The “loyal” group includes the youngest consumers with an above average income, who reside in the northern regions. Finally, the “variety seekers” are older, have the highest incomes, and live in the central regions. This grouping provides insights into the effects of brand and wine typology on consumers’ choices.

Measuring resilience to major life events

There is great interest in understanding who in the population is resilient in the face of major life events, and who is not. In this paper we construct a revealed measure of adulthood psychological resilience by modelling individuals’ responses to ten adverse life events using a dynamic finite mixture regression model applied to 17 years of panel data. Our methodology accounts for non-random selection into events, and differences between individuals in anticipation, immediate response, and speed of adaptation. We find considerable heterogeneity in individuals’ responses to events such as major financial shocks, redundancy and bereavement. We also find that our measure of resilience is correlated with clinical measures of mental health, and that it significantly predicts the psychological response to out-of-sample events. The strongest predictor of our measure of resilience is internal locus of control, which is an individual's belief that life outcomes are under their control.

Modelling Healthcare Demand Count Data with Excessive Zeros and Overdispersion

ABSTRACT In healthcare economics count datasets often exhibit excessive zeros or right-skewed tails. When covariates are available, such datasets are typically modelled using the zero-inflated (ZI) or finite mixture (FM) regression models. However, neither model performs adequately when the dataset has both excessive zeros and a long tail, which is often the case in practice. In this paper we combine these two models to create a more flexible, versatile class of ZIFM models. With this model we perform a comprehensive analysis on the number of visits to a physician’s office using the US healthcare demand dataset that has been used in numerous healthcare studies in the literature. After comparing to other existing models which have been reported to perform well on this dataset, we find that the ZIFM model substantially outperforms alternative models. In addition, the model offers a new interpretation that is in contrast to previous empirical findings regarding the factors associated with the demand for the physicians, which can shed a fresh light on the healthcare utilisation policies.

Bayesian Inference for Finite Mixture Regression Model Based on Non-Iterative Algorithm

Finite mixtures normal regression (FMNR) models are widely used to investigate the relationship between a response variable and a set of explanatory variables from several unknown latent homogeneous groups. However, the classical EM algorithm and Gibbs sampling to deal with this model have several weak points. In this paper, a non-iterative sampling algorithm for fitting FMNR model is proposed from a Bayesian perspective. The procedure can generate independently and identically distributed samples from the posterior distributions of the parameters and produce more reliable estimations than the EM algorithm and Gibbs sampling. Simulation studies are conducted to illustrate the performance of the algorithm with supporting results. Finally, a real data is analyzed to show the usefulness of the methodology.

A generalized ordinal finite mixture regression model for market segmentation

Model-based market segmentation analyses often involve an ordinal dependent variable as ordinal responses are frequently collected in marketing research. In the Bayesian segmentation literature, there are models for an interval- or ratio-scaled dependent variable but there is not any general model for an ordinal dependent variable. In this manuscript, the authors propose a new Bayesian procedure to simultaneously perform segmentation and ordinal regression with variable selection within each derived segment. The procedure is robust to outliers and it also provides an option to include concomitant variables that allows the simultaneous profiling of the derived segments. The authors demonstrate that the practice of treating ordinal responses as interval- or ratio-scales to apply existing Bayesian segmentation procedures can lead to very misleading results and conclusions. Through simulation studies, the authors show that the proposed procedure outperforms several benchmark Bayesian segmentation models in parameter recovery, segment retention, and segment membership prediction for such data. Finally, they provide a commercial business customer satisfaction empirical application to illustrate the usefulness of the proposed model.

Hierarchical mixture-of-experts models for count variables with excessive zeros

We propose a tree-structured hierarchical architecture for estimating count variables that exhibit both excessive zeros and over-dispersion, which is commonly observed in practice. The underlying statistical model is based on the Mixture-of-experts (MoE) model, a generalization of finite mixture regression models. With two levels of experts, the proposed model can efficiently model both excessive zeros and the long right-tail, controlled by two gating networks at different levels in the hierarchy. We develop the general form of the maximum likelihood estimator for the proposed model based on the EM algorithm with two latent variables, which allows a natural interpretation and convenient optimization for the likelihood function. As a numerical application, we apply the proposed model to a well-known real dataset and found that it shows superior performance and allows better interpretations compared to other existing alternatives.

Variable Selection in Finite Mixture of Time-Varying Regression Models

In this paper, we research the regression problem of time series data from heterogeneous populations on the basis of the finite mixture regression model. We propose two finite mixed time-varying regression models to solve this. A regularization method for variable selection of the models is proposed, which is a mixture of the appropriate penalty functions and l2 penalty. A Block-wise minimization maximization (MM) algorithm is used for maximum penalized log quasi-likelihood estimation of these models. The procedure is illustrated by analyzing simulations and with an application to analyze the behavior of urban vehicular traffic of the city of São Paulo in the period from 14 to 18 December 2009, which shows that the proposed models outperform the FMR models.

Finite mixture of regression models for a stratified sample

ABSTRACTDespite the popularity and importance, there is limited work on modelling data which come from complex survey design using finite mixture models. In this work, we explored the use of finite mixture regression models when the samples were drawn using a complex survey design. In particular, we considered modelling data collected based on stratified sampling design. We developed a new design-based inference where we integrated sampling weights in the complete-data log-likelihood function. The expectation–maximisation algorithm was developed accordingly. A simulation study was conducted to compare the new methodology with the usual finite mixture of a regression model. The comparison was done using bias-variance components of mean square error. Additionally, a simulation study was conducted to assess the ability of the Bayesian information criterion to select the optimal number of components under the proposed modelling approach. The methodology was implemented on real data with good results.

Robust finite mixture regression for heterogeneous targets

Finite Mixture Regression (FMR) refers to the mixture modeling scheme which learns multiple regression models from the training data set. Each of them is in charge of a subset. FMR is an effective scheme for handling sample heterogeneity, where a single regression model is not enough for capturing the complexities of the conditional distribution of the observed samples given the features. In this paper, we propose an FMR model that 1) finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously, 2) achieves shared feature selection among tasks and cluster components, and 3) detects anomaly tasks or clustered structure among tasks, and accommodates outlier samples. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The proposed model is evaluated on both synthetic and real-world data sets. The results show that our model can achieve state-of-the-art performance.

Prediction with a flexible finite mixture-of-regressions

Finite mixture regression (FMR) is widely used for modeling data that originate from heterogeneous populations. In these settings, FMR can offer increased predictive power compared to more traditional one-class models. However, existing FMR methods rely heavily on mixtures of linear models, where the linear predictor must be given as an input. A flexible FMR model is presented using a combination of the random forest learner and a penalized linear FMR. The performance of the new method is assessed by predictive log-likelihood in extensive simulation studies. The method is shown to achieve equal performance with the existing FMR methods when the true regression functions are in fact linear and superior performance in cases where at least one of the regression functions is nonlinear. The method can handle a large number of covariates, and its predictive ability is not greatly affected by surplus variables.

Revealed individual attendance at Italian theatre: a microeconomic investigation

This paper investigates theatre consumption in Italy in 3 years, 1995, 2000 and 2006 using data provided by the Italian Institute of Statistics (2006 the latest year for such data) and the Italian Society of Authors and Publishers. These data sets are very large and rarely analysed in the current economic literature. Explanatory variables are determined by identifying their contributions to both the probability to attend, and the probability to attend more, using the finite mixture regression model. Socio-demographic and socio-economic characteristics, participation in other cultural activities, ticket price and theatre supply are taken into account to varying extents. Three main findings emerge based on a more generous sample than past analyses. The strong confirmation of a priori expectations for education and occupation variables. The different impact of explanatory variables on the decision to attend and the decision to attend more. Finally, a remarkable consistency of results across the 3 year surveys and the different model specifications.

Robust variable selection for finite mixture regression models

Finite mixture regression (FMR) models are frequently used in statistical modeling, often with many covariates with low significance. Variable selection techniques can be employed to identify the covariates with little influence on the response. The problem of variable selection in FMR models is studied here. Penalized likelihood-based approaches are sensitive to data contamination, and their efficiency may be significantly reduced when the model is slightly misspecified. We propose a new robust variable selection procedure for FMR models. The proposed method is based on minimum-distance techniques, which seem to have some automatic robustness to model misspecification. We show that the proposed estimator has the variable selection consistency and oracle property. The finite-sample breakdown point of the estimator is established to demonstrate its robustness. We examine small-sample and robustness properties of the estimator using a Monte Carlo study. We also analyze a real data set.

Joint rank and variable selection for parsimonious estimation in a high-dimensional finite mixture regression model

We study a dimensionality reduction technique for finite mixtures of high-dimensional multivariate response regression models. Both the dimension of the response and the number of predictors are allowed to exceed the sample size. We consider predictor selection and rank reduction to obtain lower-dimensional approximations. A class of estimators with a fast rate of convergence is introduced. We apply this result to a specific procedure, introduced in Devijver (in press), where the relevant predictors are selected by the Group-Lasso.

A new method for robust mixture regression.

Finite mixture regression models have been widely used for modelling mixed regression relationships arising from a clustered and thus heterogenous population. The classical normal mixture model, despite its simplicity and wide applicability, may fail in the presence of severe outliers. Using a sparse, case-specific, and scale-dependent mean-shift mixture model parameterization, we propose a robust mixture regression approach for simultaneously conducting outlier detection and robust parameter estimation. A penalized likelihood approach is adopted to induce sparsity among the mean-shift parameters so that the outliers are distinguished from the remainder of the data, and a generalized Expectation-Maximization (EM) algorithm is developed to perform stable and efficient computation. The proposed approach is shown to have strong connections with other robust methods including the trimmed likelihood method and M-estimation approaches. In contrast to several existing methods, the proposed methods show outstanding performance in our simulation studies.

Modèles de mélange fini (FMM) appliqués à la segmentation du marché du travail à Bogota

Finite Mixture Models (FMM) Applied to the Segmentation of the Bogota Labor Market. The purpose of this article is to demonstrate that the semi-parametric approach with Finite Mixture Models (FMM) is a real alternative to clustering procedures to explain social phenomena. We present the general framework of FMM before detailing the three stages of Finite Mixture Regression Models (FMRM). An empirical application of this new method is used to analyze the segmentation between formal and informal economy on the labor market in Bogota.