Mixture Regression Research Articles

Generalized predictive recursion maximum likelihood for robust mixture regression

In the application of econometric model, the error distribution is unknown and is not easily to specify in the likelihood function. In some situations, there might exist a mixture distribution in the errors and thus the traditional estimation method would probably yield a biased result. In this study, this mixture distribution of the error term is taken into account and the generalized semiparametric estimation is presented and applied in regression model. We also use an experiment study and the real application analysis to check the performance of this estimator in regression model. The performance of this estimation is then compared with that of conventional Least Squares method in the real data analysis.

Bayesian Bernoulli Mixture Regression Model for Bidikmisi Scholarship Classification

Bidikmisi scholarship grantees are determined based on criteria related to the socioeconomic conditions of the parent of the scholarship grantee. Decision process of Bidikmisi acceptance is not easy to do, since there are sufficient big data of prospective applicants and variables of varied criteria. Based on these problems, a new approach is proposed to determine Bidikmisi grantees by using the Bayesian Bernoulli mixture regression model. The modeling procedure is performed by compiling the accepted and unaccepted cluster of applicants which are estimated for each cluster by the Bernoulli mixture regression model. The model parameter estimation process is done by building an algorithm based on Bayesian Markov Chain Monte Carlo (MCMC) method. The accuracy of acceptance process through Bayesian Bernoulli mixture regression model is measured by determining acceptance classification percentage of model which is compared with acceptance classification percentage of the dummy regression model and the polytomous regression model. The comparative results show that Bayesian Bernoulli mixture regression model approach gives higher percentage of acceptance classification accuracy than dummy regression model and polytomous regression model

Exploration of Muscle Fatigue Effects in Bioinspired Robot Learning from sEMG Signals

To investigate the effects of muscle fatigue on bioinspired robot learning quality in teaching by demonstration (TbD) tasks, in this work, we propose to first identify the emerging muscle fatigue phenomenon of the human demonstrator by analyzing his/her surface Electromyography (sEMG) recordings and then guide the robot learning curve with this knowledge in mind. The time-varying amplitude and frequency sequences determining the subband sEMG signals have been estimated and their dominant values over short time intervals have been explored as fatigue-indicating features. These features are found carrying muscle fatigue cues of the human demonstrator in the course of robot manipulation. In robot learning tasks requiring multiple demonstrations, the fatiguing status of human demonstrator can be acquired by tracking the changes of the proposed features over time. In order to model data from multiple demonstrations, Gaussian mixture models (GMMs) have been employed. According to the identified muscle fatigue factor, a weight has been assigned to each of the demonstration trials in training stage, which is therefore termed as weighted GMMs (W-GMMs) algorithm. Six groups of data with various fatiguing status, as well as their corresponding weights, are taken as input data to get the adapted W-GMMs parameters. After that, Gaussian mixture regression (GMR) algorithm has been applied to regenerate the movement trajectory for the robot. TbD experiments on Baxter robot with 30 human demonstration trials show that the robot can successfully accomplish the taught task with a generated trajectory much closer to that of the desirable condition where little fatigue exists.

Motion-Sound Mapping through Interaction

Technologies for sensing movement are expanding toward everyday use in virtual reality, gaming, and artistic practices. In this context, there is a need for methodologies to help designers and users create meaningful movement experiences. This article discusses a user-centered approach for the design of interactive auditory feedback using interactive machine learning. We discuss Mapping through Interaction, a method for crafting sonic interactions from corporeal demonstrations of embodied associations between motion and sound. It uses an interactive machine learning approach to build the mapping from user demonstrations, emphasizing an iterative design process that integrates acted and interactive experiences of the relationships between movement and sound. We examine Gaussian Mixture Regression and Hidden Markov Regression for continuous movement recognition and real-time sound parameter generation. We illustrate and evaluate this approach through an application in which novice users can create interactive sound feedback based on coproduced gestures and vocalizations. Results indicate that Gaussian Mixture Regression and Hidden Markov Regression can efficiently learn complex motion-sound mappings from few examples.

Convex mixture regression for quantitative risk assessment.

There is wide interest in studying how the distribution of a continuous response changes with a predictor. We are motivated by environmental applications in which the predictor is the dose of an exposure and the response is a health outcome. A main focus in these studies is inference on dose levels associated with a given increase in risk relative to a baseline. In addressing this goal, popular methods either dichotomize the continuous response or focus on modeling changes with the dose in the expectation of the outcome. Such choices may lead to information loss and provide inaccurate inference on dose-response relationships. We instead propose a Bayesian convex mixture regression model that allows the entire distribution of the health outcome to be unknown and changing with the dose. To balance flexibility and parsimony, we rely on a mixture model for the density at the extreme doses, and express the conditional density at each intermediate dose via a convex combination of these extremal densities. This representation generalizes classical dose-response models for quantitative outcomes, and provides a more parsimonious, but still powerful, formulation compared to nonparametric methods, thereby improving interpretability and efficiency in inference on risk functions. A Markov chain Monte Carlo algorithm for posterior inference is developed, and the benefits of our methods are outlined in simulations, along with a study on the impact of dde exposure on gestational age.

An Extension of Generalized Linear Models to Finite Mixture Outcome Distributions

ABSTRACTFinite mixture distributions arise in sampling a heterogeneous population. Data drawn from such a population will exhibit extra variability relative to any single subpopulation. Statistical models based on finite mixtures can assist in the analysis of categorical and count outcomes when standard generalized linear models (GLMs) cannot adequately express variability observed in the data. We propose an extension of GLMs where the response follows a finite mixture distribution and the regression of interest is linked to the mixture’s mean. This approach may be preferred over a finite mixture of regressions when the population mean is of interest; here, only one regression must be specified and interpreted in the analysis. A technical challenge is that the mixture’s mean is a composite parameter that does not appear explicitly in the density. The proposed model maintains its link to the regression through a certain random effects structure and is completely likelihood-based. We consider typical GLM cases where means are either real-valued, constrained to be positive, or constrained to be on the unit interval. The resulting model is applied to two example datasets through Bayesian analysis. Supporting the extra variation is seen to improve residual plots and produce widened prediction intervals reflecting the uncertainty. Supplementary materials for this article are available online.

A learning from demonstration framework for implementation of a feeding task

In this paper, a learning from demonstration (LFD) approach is used to design an autonomous meal-assistance agent. The feeding task is modeled as a mixture of Gaussian distributions. Using the data collected via kinesthetic teaching, the parameters of the Gaussian mixture model (GMM) are learnt using Gaussian mixture regression (GMR) and expectation maximization (EM) algorithm. Reproduction of feeding trajectories for different environments is obtained by solving a constrained optimization problem. In this method we show that obstacles can be avoided by robot’s end-effector by adding a set of extra constraints to the optimization problem. Finally, the performance of the designed meal assistant is evaluated in two feeding scenario experiments: one considering obstacles in the path between the bowl and the mouth and the other without.

A DIAGNOSTIC TOOL FOR CHECKING ASSUMPTIONS OF REGRESSION MIXTURE MODELS.

Regression mixture models are becoming more widely used in applied research. It has been recognized that these models are quite sensitive to underlying assumptions, yet many of these assumptions are not directly testable. We discuss a diagnostic tool based on reconstructed residuals that can help uncover violations of model assumptions. These residuals are found by using the posterior probability of class membership to assign, based on a multinomial distribution, a class to each observation. Standard residual checks can be applied to these posterior draw residuals to explore violations of the model assumptions. We present several illustrations of the diagnostic tool.

Robust finite mixture regression for heterogeneous targets

Finite Mixture Regression (FMR) refers to the mixture modeling scheme which learns multiple regression models from the training data set. Each of them is in charge of a subset. FMR is an effective scheme for handling sample heterogeneity, where a single regression model is not enough for capturing the complexities of the conditional distribution of the observed samples given the features. In this paper, we propose an FMR model that 1) finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously, 2) achieves shared feature selection among tasks and cluster components, and 3) detects anomaly tasks or clustered structure among tasks, and accommodates outlier samples. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The proposed model is evaluated on both synthetic and real-world data sets. The results show that our model can achieve state-of-the-art performance.

A k-Inflated Negative Binomial Mixture Regression Model: Application to Rate–Making Systems

Abstract This article introduces a k-Inflated Negative Binomial mixture distribution/regression model as a more flexible alternative to zero-inflated Poisson distribution/regression model. An EM algorithm has been employed to estimate the model’s parameters. Then, such new model along with a Pareto mixture model have employed to design an optimal rate–making system. Namely, this article employs number/size of reported claims of Iranian third party insurance dataset. Then, it employs the k-Inflated Negative Binomial mixture distribution/regression model as well as other well developed counting models along with a Pareto mixture model to model frequency/severity of reported claims in Iranian third party insurance dataset. Such numerical illustration shows that: (1) the k-Inflated Negative Binomial mixture models provide more fair rate/pure premiums for policyholders under a rate–making system; and (2) in the situation that number of reported claims uniformly distributed in past experience of a policyholder (for instance k 1 = 1 $k_1=1$ and k 2 = 1 $k_2=1$ instead of k 1 = 0 $k_1=0$ and k 2 = 2 $k_2=2$ ). The rate/pure premium under the k-Inflated Negative Binomial mixture models are more appealing and acceptable.

Characterizing sex ratios of sea turtle populations: A Bayesian mixture modeling approach applied to juvenile loggerheads (Caretta caretta)

Sex ratio is fundamental to the demography of animal populations. For sea turtles, the operational sex ratio (i.e., that of breeding adults) can be inferred from juvenile sex ratio. In this study, we characterize sex ratio of n = 1401 juvenile loggerhead sea turtles (Caretta caretta) sampled from foraging grounds in North Carolina, USA. For a subset (n = 170), sex was confirmed by laparoscopy, and for the remaining individuals, sex was classified probabilistically using a mixture regression model. The model predicted plasma testosterone concentration as a function of sex, which was treated as a latent variable, and of other potential covariates, namely water temperature and carapace length. Furthermore, it quantified uncertainty in the sex classification of each individual. For the full data set, the predicted sex ratio was 81% female. Using the subset of turtles with known sex and a leave-one-out validation approach, we found the model to have classification accuracy of 94%. We further used that subset to examine how many laparoscopies are sufficient for accurate classification of turtles with unknown sex, in an attempt to provide guidance for other studies. We found diminishing returns as sample size increased, and recommend 100–140 turtles as a sufficient range.

Unsupervised learning of mixture regression models for longitudinal data

This paper is concerned with learning of mixture regression models for individuals that are measured repeatedly. The adjective “unsupervised” implies that the number of mixing components is unknown and has to be determined, ideally by data driven tools. For this purpose, a novel penalized method is proposed to simultaneously select the number of mixing components and to estimate the mixture proportions and unknown parameters in the models. The proposed method is capable of handling both continuous and discrete responses by only requiring the first two moment conditions of the model distribution. It is shown to be consistent in both selecting the number of components and estimating the mixture proportions and unknown regression parameters. Further, a modified EM algorithm is developed to seamlessly integrate model selection and estimation. Simulation studies are conducted to evaluate the finite sample performance of the proposed procedure. And it is further illustrated via an analysis of a primary biliary cirrhosis data set.

Plane-extraction from depth-data using a Gaussian mixture regression model

We propose a novel algorithm for unsupervised extraction of piecewise planar models from depth-data. Among other applications, such models are a good way of enabling autonomous agents (robots, cars, drones, etc.) to effectively perceive their surroundings and to navigate in three dimensions. We propose to do this by fitting the data with a piecewise-linear Gaussian mixture regression model whose components are skewed over planes, making them flat in appearance rather than being ellipsoidal, by embedding an outlier-trimming process that is formally incorporated into the proposed expectation-maximization algorithm, and by selectively fusing contiguous, coplanar components. Part of our motivation is an attempt to estimate more accurate plane-extraction by allowing each model component to make use of all available data through probabilistic clustering. The algorithm is thoroughly evaluated against a standard benchmark and is shown to rank among the best of the existing state-of-the-art methods.

Monthly streamflow forecasting based on hidden Markov model and Gaussian Mixture Regression

Reliable streamflow forecasts can be highly valuable for water resources planning and management. In this study, we combined a hidden Markov model (HMM) and Gaussian Mixture Regression (GMR) for probabilistic monthly streamflow forecasting. The HMM is initialized using a kernelized K-medoids clustering method, and the Baum–Welch algorithm is then executed to learn the model parameters. GMR derives a conditional probability distribution for the predictand given covariate information, including the antecedent flow at a local station and two surrounding stations. The performance of HMM–GMR was verified based on the mean square error and continuous ranked probability score skill scores. The reliability of the forecasts was assessed by examining the uniformity of the probability integral transform values. The results show that HMM–GMR obtained reasonably high skill scores and the uncertainty spread was appropriate. Different HMM states were assumed to be different climate conditions, which would lead to different types of observed values. We demonstrated that the HMM–GMR approach can handle multimodal and heteroscedastic data.

Recovering task fMRI signals from highly under-sampled data with low-rank and temporal subspace constraints

Recent developments in highly accelerated fMRI data acquisition have employed low-rank and/or sparsity constraints for image reconstruction, as an alternative to conventional, time-independent parallel imaging. When under-sampling factors are high or the signals of interest are low-variance, however, functional data recovery can be poor or incomplete. We introduce a method for improving reconstruction fidelity using external constraints, like an experimental design matrix, to partially orient the estimated fMRI temporal subspace. Combining these external constraints with low-rank constraints introduces a new image reconstruction model that is analogous to using a mixture of subspace-decomposition (PCA/ICA) and regression (GLM) models in fMRI analysis.We show that this approach improves fMRI reconstruction quality in simulations and experimental data, focusing on the model problem of detecting subtle 1-s latency shifts between brain regions in a block-design task-fMRI experiment. Successful latency discrimination is shown at acceleration factors up to R = 16 in a radial-Cartesian acquisition. We show that this approach works with approximate, or not perfectly informative constraints, where the derived benefit is commensurate with the information content contained in the constraints. The proposed method extends low-rank approximation methods for under-sampled fMRI data acquisition by leveraging knowledge of expected task-based variance in the data, enabling improvements in the speed and efficiency of fMRI data acquisition without the loss of subtle features.

A Command for Fitting Mixture Regression Models for Bounded Dependent Variables Using the Beta Distribution

In this article, we describe the betamix command, which fits mixture regression models for dependent variables bounded in an interval. The model is a generalization of the truncated inflated beta regression model introduced in Pereira, Botter, and Sandoval (2012, Communications in Statistics—Theory and Methods 41: 907–919) and the mixture beta regression model in Verkuilen and Smithson (2012, Journal of Educational and Behavioral Statistics 37: 82–113) for variables with truncated supports at either the top or the bottom of the distribution. betamix accepts dependent variables defined in any range that are then transformed to the interval (0, 1) before estimation.

Target-based catch-per-unit-effort standardization in multispecies fisheries

Standardized catch per unit effort (CPUE) is a fundamental component of fishery stock assessment. In multispecies fisheries, catchability can differ depending on which species is being targeted, and so the yearly trend extracted from the standardized CPUE is likely to be biased. We have, therefore, developed a method for predicting the unobserved variable related to targeted species from among multispecies composition data using a mixture regression model for the transformed residuals. In contrast with traditional methods, the proposed method predicts the target variable in CPUE standardization without removing a subset of the data. Keeping the entire data set avoids information loss, and so CPUE standardization with the predicted target variable should yield an unbiased estimate of the yearly trend. Simple simulation tests demonstrate that our method outperforms traditional methods. We illustrate the use of our method by applying it to CPUE data on arabesque greenling (Pleurogrammus azonus) caught in multispecies trawl fisheries in Hokkaido, Japan.

Bayesian Analysis of Skew Normal Mixture Regression

Bayesian Analysis of Skew Normal Mixture Regression

An enhanced teaching interface for a robot using DMP and GMR

This paper develops an enhanced teaching interface tested on both a Baxter robot and a KUKA iiwa robot. Movements are collected from a human demonstrator by using a Kinect v2 sensor, and then the data is sent to a remote PC for the teleoperation with Baxter. Meanwhile, data is saved locally for the playback process of the Baxter. The dynamic movement primitive (DMP) is used to model and generalize the movements. In order to learn from multiple demonstrations accurately, dynamic time warping (DTW), is used to pretreat the data recorded by the robot platform and Gaussian mixture model (GMM), aiming to generate multiple patterns after the teaching process, are employed for the calculation of the DMP. Then the Gaussian mixture regression (GMR) algorithm is applied to generate a synthesized trajectory with smaller position errors in 3D space. This proposed approach is tested by performing two tasks on a KUKA iiwa and a Baxter robot.

Robust feature selection via l2,1-norm in finite mixture of regression

Finite mixture of Gaussian regression (FMR) is a widely-used modeling technique in supervised learning problems. In cases where the number of features is large, feature selection is desirable to enhance model interpretability and to avoid overfitting. In this paper, we propose a robust feature selection method via l2,1-norm penalized maximum likelihood estimation (MLE) in FMR, with extension to sparse l2,1 penalty by combining l1-norm with l2,1-norm for increasing flexibility. To solve the non-convex and non-smooth problem of (sparse) penalized MLE in FMR, we develop an new EM-based algorithm for numerical optimization, with combination of block coordinate descent and majorizing-minimization scheme in M-step. We finally apply our method in six simulations and one real dataset to demonstrate its superior performance.