Mixture Regression Model Research Articles

Unsupervised learning of mixture regression models for longitudinal data

This paper is concerned with learning of mixture regression models for individuals that are measured repeatedly. The adjective “unsupervised” implies that the number of mixing components is unknown and has to be determined, ideally by data driven tools. For this purpose, a novel penalized method is proposed to simultaneously select the number of mixing components and to estimate the mixture proportions and unknown parameters in the models. The proposed method is capable of handling both continuous and discrete responses by only requiring the first two moment conditions of the model distribution. It is shown to be consistent in both selecting the number of components and estimating the mixture proportions and unknown regression parameters. Further, a modified EM algorithm is developed to seamlessly integrate model selection and estimation. Simulation studies are conducted to evaluate the finite sample performance of the proposed procedure. And it is further illustrated via an analysis of a primary biliary cirrhosis data set.

Plane-extraction from depth-data using a Gaussian mixture regression model

We propose a novel algorithm for unsupervised extraction of piecewise planar models from depth-data. Among other applications, such models are a good way of enabling autonomous agents (robots, cars, drones, etc.) to effectively perceive their surroundings and to navigate in three dimensions. We propose to do this by fitting the data with a piecewise-linear Gaussian mixture regression model whose components are skewed over planes, making them flat in appearance rather than being ellipsoidal, by embedding an outlier-trimming process that is formally incorporated into the proposed expectation-maximization algorithm, and by selectively fusing contiguous, coplanar components. Part of our motivation is an attempt to estimate more accurate plane-extraction by allowing each model component to make use of all available data through probabilistic clustering. The algorithm is thoroughly evaluated against a standard benchmark and is shown to rank among the best of the existing state-of-the-art methods.

A Command for Fitting Mixture Regression Models for Bounded Dependent Variables Using the Beta Distribution

In this article, we describe the betamix command, which fits mixture regression models for dependent variables bounded in an interval. The model is a generalization of the truncated inflated beta regression model introduced in Pereira, Botter, and Sandoval (2012, Communications in Statistics—Theory and Methods 41: 907–919) and the mixture beta regression model in Verkuilen and Smithson (2012, Journal of Educational and Behavioral Statistics 37: 82–113) for variables with truncated supports at either the top or the bottom of the distribution. betamix accepts dependent variables defined in any range that are then transformed to the interval (0, 1) before estimation.

Target-based catch-per-unit-effort standardization in multispecies fisheries

Standardized catch per unit effort (CPUE) is a fundamental component of fishery stock assessment. In multispecies fisheries, catchability can differ depending on which species is being targeted, and so the yearly trend extracted from the standardized CPUE is likely to be biased. We have, therefore, developed a method for predicting the unobserved variable related to targeted species from among multispecies composition data using a mixture regression model for the transformed residuals. In contrast with traditional methods, the proposed method predicts the target variable in CPUE standardization without removing a subset of the data. Keeping the entire data set avoids information loss, and so CPUE standardization with the predicted target variable should yield an unbiased estimate of the yearly trend. Simple simulation tests demonstrate that our method outperforms traditional methods. We illustrate the use of our method by applying it to CPUE data on arabesque greenling (Pleurogrammus azonus) caught in multispecies trawl fisheries in Hokkaido, Japan.

On the Use of the Sub-Gaussian $$\alpha $$ α -Stable Distribution in the Cluster-Weighted Model

The Gaussian cluster-weighted model (CWM) is a mixture of regression models with random covariates that allows for flexible clustering of a random vector composed of a response variable and some covariates. In each mixture component, a Gaussian distribution is adopted for both the covariates and the response given the covariates. To make the approach robust with respect to the presence of atypical observations, we propose to replace the Gaussian distribution with the sub-Gaussian $$\alpha $$ -stable (SG $$\alpha $$ S) distribution, an elliptical generalization of the Gaussian distribution having one additional parameter, $$\alpha $$ , governing the tails’ weight. The resulting SG $$\alpha $$ S CWM is able to accommodate outliers and leverage points, concepts of primary importance in the robust regression analysis. Advantageously with respect to the t-distribution, the tails of the SG $$\alpha $$ S distribution can be heavier, thus allowing robustness also with respect to gross atypical observations. A new algorithm, based on a combination of stochastic and conditional expectation maximizations, is used to obtain maximum likelihood estimates of the model parameters. Simulated and real data are used to illustrate and compare the proposal with CWMs based on Gaussian and t distributions.

Relationship between sleep and exercise as colorectal cancer survivors transition off treatment.

The primary objective of this study was to evaluate the relationship between exercise and sleep disturbance in a sample of individuals diagnosed with stage I, II, and III colorectal cancer (CRC) as patients transitioned off first-line treatment. We also sought to identify heterogeneity in the relationship between sleep disturbance and exercise. Data were obtained from the MY-Health study, a community-based observational study of adults diagnosed with cancer. Patient-Reported Outcomes Measurement Information System® (PROMIS) measures (e.g., PROMIS Sleep) were administered, and participants self-reported demographics, comorbidities, cancer treatment, and exercise. Regression mixture and multiple regression models were used to evaluate the relationship between sleep disturbance and exercise cross-sectionally at an average of 10months after diagnosis, and the change in sleep disturbance over a 7-month period, from approximately 10 to 17months post-diagnosis. Patients whose exercise was categorized as likely at or above American College of Sports Medicine's guidelines did not report statistically better sleep quality compared to patients who were classified as not active. However, retirement (B = - 2.4), anxiety (B = 0.21), and fatigue (B = 0.24) had statistically significant relationships with sleep disturbance (p < 0.05). Increase in exercise was not significantly associated with a decrease in sleep disturbance. No statistical heterogeneity was revealed in the relationship between sleep and exercise. Further prospective research using an objective measure of exercise is warranted to confirm or refute the nature of the relationship between exercise and sleep disturbance in individuals diagnosed with CRC transitioning off first-line treatment.

Fault detection and diagnosis for nonlinear systems: A new adaptive Gaussian mixture modeling approach

In heating, ventilation and air-conditioning (HVAC) systems, early detection and diagnosis of faults are of critical importance to save energy and improve the performance of system components. The challenge is developing an automatic fault detection and diagnosis algorithm for monitoring of multi-mode nonlinear systems. This paper proposes a novel adaptive Gaussian mixture model (AGMM) approach for automatic fault detection and diagnosis in nonlinear systems. The concept of this method relies on developing a time-varying probabilistic machine-learning model for non-linear systems. In this study, Gaussian mixture model regression (GMMR) technique is used to model a nonlinear system based on measurement data. Unscented Kalman filter (UKF) is then integrated with GMMR for adjusting the model parameters based on the feedback of residuals between observation and model prediction. The proposed algorithm is able to detect and diagnose simultaneous faults in systems from monitoring variations of key GMMR parameters. The application of AGMM method is demonstrated for detection and diagnosis of common faults in a water-cooled multi-chiller plant system. Faults including energy efficiency degradation in chillers were tested with the proposed method. Results indicate that the AGMM approach is successful in detection and diagnosis of simultaneous faults.

Modelling of road traffic fatalities in India

Passenger modes in India include walking, cycling, buses, trains, intermediate public transport modes (IPT) such as three-wheeled auto rickshaws or tuk-tuks, motorised two-wheelers (2W) as well as cars. However, epidemiological studies of traffic crashes in India have been limited in their approach to account for the exposure of these road users. In 2011, for the first time, census in India reported travel distance and mode of travel for workers. A Poisson-lognormal mixture regression model is developed at the state level to explore the relationship of road deaths of all the road users with commute travel distance by different on-road modes. The model controlled for diesel consumption (proxy for freight traffic), length of national highways, proportion of population in urban areas, and built-up population density. The results show that walking, cycling and, interestingly, IPT are associated with lower risk of road deaths, while 2W, car and bus are associated with higher risk. Promotion of IPT has twofold benefits of increasing safety as well as providing a sustainable mode of transport. The mode shift scenarios show that, for similar mode shift across the states, the resulting trends in road deaths are highly dependent on the baseline mode shares. The most worrying trend is the steep growth of death burden resulting from mode shift of walking and cycling to 2W. While the paper illustrates a limited set of mode shift scenarios involving two modes at a time, the model can be applied to assess safety impacts resulting from a more complex set of scenarios.

Mixture regression models for the gap time distributions and illness-death processes.

The aim of this study is to provide an analysis of gap event times under the illness-death model, where some subjects experience "illness" before "death" and others experience only "death." Which event is more likely to occur first and how the duration of the "illness" influences the "death" event are of interest. Because the occurrence of the second event is subject to dependent censoring, it can lead to bias in the estimation of model parameters. In this work, we generalize the semiparametric mixture models for competing risks data to accommodate the subsequent event and use a copula function to model the dependent structure between the successive events. Under the proposed method, the survival function of the censoring time does not need to be estimated when developing the inference procedure. We incorporate the cause-specific hazard functions with the counting process approach and derive a consistent estimation using the nonparametric maximum likelihood method. Simulations are conducted to demonstrate the performance of the proposed analysis, and its application in a clinical study on chronic myeloid leukemia is reported to illustrate its utility.

A new analysis of VIX using mixture of regressions: Examination and short‐term forecasting for the S &amp; P 500 market

AbstractA novel approach to the analysis of S &amp; P 500 market fluctuations is proposed using a K‐component mixture of regressions model. The Barndorff‐Nielsen and Shephard stochastic model is employed where the estimates of jumps of log‐returns are governed by Lévy subordinators. Daily VIX and VIX2 close prices are analyzed as the indicators of log‐return volatility and the corresponding variance of the S &amp; P 500 index using the mixture model. The behavior of the S &amp; P 500 market from 1 August 2005 to 31 December 2009 is analyzed and forecasted. A set of rules are provided to predict monthly fluctuation in the S &amp; P 500 market. The procedure used in this paper gives a novel approach for constructing an “indicator”of non‐Gaussian jump of an empirical data set in finance using mixture of regression (Gaussian) analysis.

FlexCWM: A Flexible Framework for Cluster-Weighted Models

Cluster-weighted models (CWMs) are mixtures of regression models with random covariates. However, besides having recently become rather popular in statistics and data mining, there is still a lack of support for CWMs within the most popular statistical suites. In this paper, we introduce flexCWM, an R package specifically conceived for fitting CWMs. The package supports modeling the conditioned response variable by means of the most common distributions of the exponential family and by the t distribution. Covariates are allowed to be of mixed-type and parsimonious modeling of multivariate normal covariates, based on the eigenvalue decomposition of the component covariance matrices, is supported. Furthermore, either the response or the covariates distributions can be omitted, yielding to mixtures of distributions and mixtures of regression models with fixed covariates, respectively. The expectation-maximization (EM) algorithm is used to obtain maximum-likelihood estimates of the parameters and likelihood-based information criteria are adopted to select the number of groups and/or a parsimonious model. For the component regression coefficients, standard errors and significance tests are also provided. Parallel computation can be used on multicore PCs and computer clusters, when several models have to be fitted. To exemplify the use of the package, applications to artificial and real datasets, included in the package, are presented.

Finite mixture of compositional regression with gaussian errors

In this paper, we consider to evaluate the efficiency of volleyball players according to the performance of attack, block and serve, but considering the compositional structure of the data related to the fundaments. The finite mixture of regression models better fitted the data in comparison with the usual regression model. The maximum likelihood estimates are obtained via an EM algorithm. A simulation study revels that the estimates are closer to the real values, the estimators are asymptotically unbiased for the parameters. A real Brazilian volleyball dataset related to the efficiency of the players is considered for the analysis.

Which Firm Characteristics Affect Foreign Analyst Coverage? Evidence from the Taiwan Stock Market

This study investigates the association between firm characteristics and foreign analyst coverage. Using a sample of 366 firms listed on the Taiwan Stock Exchange that were followed by foreign analysts from January 2006 to June 2012, I employ the finite Tobit regression mixture model to analyze which firm characteristics influence foreign analyst coverage. The results find that mixture model may express differential influences on analyst coverage based on the heterogeneity of firm characteristics in different subgroups. Price-to-book ratio, firm size, and institutional ownership are the important characteristic variables that influence foreign analyst coverage. Among them, price-to-book ratio and foreign analyst coverage may present an inverted U-shaped or a U-shaped correlation in different subgroups. Also, there is an endogenous relationship between foreign analyst coverage and foreign institutional ownership, and thus, the characteristics of target firms followed by foreign analysts resulting from changes in foreign institutional ownership cannot be ignored. Finally, I also find that large-sized firms with high institutional ownership and medium stock turnover rate have the highest probability of being followed by foreign analysts.

Revealed individual attendance at Italian theatre: a microeconomic investigation

This paper investigates theatre consumption in Italy in 3 years, 1995, 2000 and 2006 using data provided by the Italian Institute of Statistics (2006 the latest year for such data) and the Italian Society of Authors and Publishers. These data sets are very large and rarely analysed in the current economic literature. Explanatory variables are determined by identifying their contributions to both the probability to attend, and the probability to attend more, using the finite mixture regression model. Socio-demographic and socio-economic characteristics, participation in other cultural activities, ticket price and theatre supply are taken into account to varying extents. Three main findings emerge based on a more generous sample than past analyses. The strong confirmation of a priori expectations for education and occupation variables. The different impact of explanatory variables on the decision to attend and the decision to attend more. Finally, a remarkable consistency of results across the 3 year surveys and the different model specifications.

Statistical Assessment of the OWZ Tropical Cyclone Tracking Scheme in ERA-Interim

AbstractThe Okubo–Weiss–Zeta (OWZ) tropical cyclone (TC) detection scheme, which has been used to detect TCs in climate, seasonal prediction, and weather forecast models, is assessed on its ability to produce a realistic TC track climatology in the ERA-Interim product over the 25-yr period 1989 to 2013. The analysis focuses on TCs that achieve gale-force (17 m s−1) sustained winds. Objective criteria were established to define TC tracks once they reach gale force for both observed and detected TCs. A lack of consistency between storm tracks preceding this level of intensity led these track segments to be removed from the analysis. A subtropical jet (STJ) diagnostic is used to terminate transitioning TCs and is found to be preferable to a fixed latitude cutoff point. TC tracks were analyzed across seven TC basins, using a probabilistic clustering technique that is based on regression mixture models. The technique grouped TC tracks together based on their geographical location and shape of trajectory in five separate “cluster regions” around the globe. A mean trajectory was then regressed for each cluster that showed good agreement between the detected and observed tracks. Other track measures such as interannual TC days and translational speeds were also replicated to a satisfactory level, with TC days showing limited sensitivity to different latitude cutoff points. Successful validation in reanalysis data allows this model- and grid-resolution-independent TC tracking scheme to be applied to climate models with confidence in its ability to identify TC tracks in coarse-resolution climate models.

Sleep quality in individuals diagnosed with colorectal cancer: Factors associated with sleep disturbance as patients transition off treatment.

To identify patient characteristics associated with sleep disturbance and worsening of sleep in individuals diagnosed with localized colorectal cancer and assess heterogeneity in these relationships. Data were from the MY-Health study, a community-based observational study of adults diagnosed with cancer. Patient-Reported Outcomes Measurement Information System® Sleep Disturbance, Anxiety, Depression, Fatigue, and Pain Interference measures were administered. Participants self-reported demographics, comorbidities, and treatment information. Regression mixture and multiple regression models were used to evaluate the relationship between sleep disturbance and patient characteristics cross-sectionally at an average of 10months after diagnosis (n=613) as well as change in sleep disturbance over a 6-month period (n=361). Pain, anxiety, fatigue, and the existence of multiple comorbid conditions had statistically significant relationships with sleep disturbance (B=0.09, 0.22, 0.29, and 1.53, respectively; P<0.05). Retirement (B=-2.49) was associated with sleep quality in the cross-sectional model. Worsening anxiety (B=0.14) and fatigue (B=0.20) were associated with worsening sleep disturbance, and more severe sleep disturbance 10months after diagnosis (B=-0.21) was associated with improvement in sleep quality after diagnosis (P<0.05). No evidence of latent subgroups of patients (heterogeneity) was present. Pain, anxiety, fatigue, employment, and comorbid conditions were associated with sleep disturbance, but regression coefficients were small (< |2.5|). Results suggest that screening for anxiety, depression, fatigue, or pain is not sufficient for identifying sleep disturbance. Given the negative consequences of sleep disturbance, sleep disturbance screening may be warranted.

Mixtures of multivariate contaminated normal regression models

Mixtures of regression models (MRMs) are widely used to investigate the relationship between variables coming from several unknown latent homogeneous groups. Usually, the conditional distribution of the response in each mixture component is assumed to be (multivariate) normal (MN-MRM). To robustify the approach with respect to possible elliptical heavy-tailed departures from normality, due to the presence of mild outliers, the multivariate contaminated normal MRM is here introduced. In addition to the parameters of the MN-MRM, each mixture component has a parameter controlling the proportion of outliers and one specifying the degree of contamination with respect to the response variable(s). Crucially, these parameters do not have to be specified a priori, adding flexibility to our approach. Furthermore, once the model is estimated and the observations are assigned to the groups, a finer intra-group classification in typical points and (mild) outliers, can be directly obtained. Identifiability conditions are provided, an expectation-conditional maximization algorithm is outlined for parameter estimation, and various implementation and operational issues are discussed. Properties of the estimators of the regression coefficients are evaluated through Monte Carlo experiments and compared with other procedures. The performance of this novel family of models is also illustrated on artificial and real data, with particular emphasis to the application in allometric studies.

이상점이 존재하는 이질적 자료에 대한 로버스트 다변량 혼합회귀모형

다변량 회귀모형(multivariate regression models)은 두 개 이상의 반응변수와 여러 개의 설명변수들 간의 관계를 조사하기 위해 사용된다. 자료가 이질적 모집단(heterogeneous population)으로부터 추출된 표본자료인 경우에는 부모집단(subpopulation)마다 회귀계수들이 달라지는 것을 허용하는 다변량 혼합회귀모형(multivariate mixture regression models)으로 분석할 수 있다. 다변량 혼합회귀모형은 일반적으로 정규분포 가정 하에서 모형화 되는데, 자료에 이상점(outliers) 또는 노이즈(noise)가 포함되어 있는 경우에는 로버스트(robust) 추정을 위해 t-분포를 사용하여 모형화 될 수 있다. 본 연구에서는 이질적 모집단으로부터 추출된 표본 자료에 이상점 또는 노이즈가 존재하는 경우, 표본 내에 잠재되어 있는 서브그룹(latent subgroup)들을 식별하고 각 서브그룹의 회귀계수들을 로버스트하게 추정하기 위해 다변량 t-분포를 기반으로 한 혼합회귀모형을 제안한다. 본 연구에서 제안하는 모형은 반응변수들 간의 편상관관계(partial correlation)를 파악하는데도 유용하며, 이러한 편상관관계는 그림으로 시각화(visualization)가 가능하여 서브그룹 내 반응변수 간의 고유 연관성을 쉽게 파악할 수 있다. 모의실험 결과 제안된 모형이 다른 비교 모형들에 비해 우수함을 확인하였고, 고등학생들의 시험 성적 자료에 제안된 모형을 적용하여 분석하였다.Multivariate regression models are used to examine the relationship between two or more response variables and multiple explanatory variables. Data from a heterogeneous population can be analyzed using the multivariate mixture regression models that allow regression coefficients to vary across subgroups. Multivariate mixture regression models are typically modeled using normal distributions, but can be modeled using t-distributions for robust estimation if data contain outliers or noise. In this paper, a robust multivariate mixture regression model based on the multivariate t-distributions (MVTmixreg) is proposed to identify latent subgroups and robustly estimate the regression coefficients of each subgroup when there are outliers or noise in the sample from a heterogeneous population. The proposed model can also be used to find the partial correlation between the response variables in each latent subgroup. This partial correlation can be visualized by a plot. Simulation results show that the proposed model is superior to the other comparative models, and the proposed model is applied to the data of high school students’ test scores.

Adaptive learning based data-driven models for predicting hourly building energy use

Accurately predicting energy usage in buildings is of great importance in various efforts on improving building energy efficiencies such as fault detection and diagnostics, building-grid interactions, and building commissioning. Data-driven approach and first-principle approach are two commonly used methods in developing models for predicting building energy use. In this paper, several data-driven methods including multiple linear regression, adaptive linear filter algorithms (least mean square (LMS), normalized least mean square (nLMS), and recursive least square (RLS)), and Gaussian mixture model regression (GMMR) are employed to predict hourly energy usages in two buildings. One building is a synthetic large-size office building from DOE reference building models. The hourly building energy consumption was predicted using the energy simulation model for one year under Chicago climate. The other building is an existing office building located in Des Moines, Iowa. The actual hourly building energy consumption of the existing building was obtained through building submeters. The accuracies of these data-driven models for predicting energy usages of the two buildings are compared. The GMMR models outperform the adaptive filter methods in this study. Both the GMMR and adaptive filter methods meet the model calibration criteria defined by the ASHRAE Guideline 14.

Modeling and clustering water demand patterns from real-world smart meter data

Abstract. Nowadays, drinking water utilities need an acute comprehension of the water demand on their distribution network, in order to efficiently operate the optimization of resources, manage billing and propose new customer services. With the emergence of smart grids, based on automated meter reading (AMR), a better understanding of the consumption modes is now accessible for smart cities with more granularities. In this context, this paper evaluates a novel methodology for identifying relevant usage profiles from the water consumption data produced by smart meters. The methodology is fully data-driven using the consumption time series which are seen as functions or curves observed with an hourly time step. First, a Fourier-based additive time series decomposition model is introduced to extract seasonal patterns from time series. These patterns are intended to represent the customer habits in terms of water consumption. Two functional clustering approaches are then used to classify the extracted seasonal patterns: the functional version of K-means, and the Fourier REgression Mixture (FReMix) model. The K-means approach produces a hard segmentation and K representative prototypes. On the other hand, the FReMix is a generative model and also produces K profiles as well as a soft segmentation based on the posterior probabilities. The proposed approach is applied to a smart grid deployed on the largest water distribution network (WDN) in France. The two clustering strategies are evaluated and compared. Finally, a realistic interpretation of the consumption habits is given for each cluster. The extensive experiments and the qualitative interpretation of the resulting clusters allow one to highlight the effectiveness of the proposed methodology.