Mixture Regression Research Articles

Asymptotic normality of modified LS estimator for mixture of nonlinear regressions

Asymptotic normality of modified LS estimator for mixture of nonlinear regressions

Just-in-Time Learning-Based Soft Sensor for Mechanical Properties of Strip Steel via Multi-Block Weighted Semisupervised Models

Mechanical properties are important indexes to evaluate the quality of hot rolling strips. It is a research hotspot in the field of hot rolling that realizing timely and accurate soft sensing of mechanical properties. Traditional soft sensing methods have poor performance in the application of strong nonlinearity and multiple working conditions. Moreover, the utilization rate of data is relatively low, which limit the improvement of prediction accuracy. To solve the problems above, a just-in-time learning (JITL) based multi-block weighted semisupervised Gaussian mixture regression (JMWSSGMR) soft sensor is proposed in the paper. There are two stages in the soft sensor: off-line variable blocking and on-line local modeling. In the off-line phase, process variables are divided into different sub-blocks by partial least square (PLS) according to distinct principal component directions. In each sub-block, original variables with high contribution rate are retained. In the on-line phase, optimized Mahalanobis distance is constructed to select the most similar historical samples to the query sample. Next, various real-time semisupervised sub-models are built to estimate the output of the query sample. Finally, predicted values of sub-models are fused and ultimate prediction of mechanical properties is obtained. Case studies are carried out on a numerical example and a hot rolling process. The feasibility and effectiveness of proposed soft sensor are verified by the predicted results.

Variable Selection in Finite Mixture of Time-Varying Regression Models

In this paper, we research the regression problem of time series data from heterogeneous populations on the basis of the finite mixture regression model. We propose two finite mixed time-varying regression models to solve this. A regularization method for variable selection of the models is proposed, which is a mixture of the appropriate penalty functions and l2 penalty. A Block-wise minimization maximization (MM) algorithm is used for maximum penalized log quasi-likelihood estimation of these models. The procedure is illustrated by analyzing simulations and with an application to analyze the behavior of urban vehicular traffic of the city of São Paulo in the period from 14 to 18 December 2009, which shows that the proposed models outperform the FMR models.

Adaptive Sparse Group Variable Selection for a Robust Mixture Regression Model Based on Laplace Distribution

The traditional estimation of Gaussian mixture model is sensitive to heavy-tailed errors; thus we propose a robust mixture regression model by assuming that the error terms follow a Laplace distribution in this article. And for the variable selection problem in our new robust mixture regression model, we introduce the adaptive sparse group Lasso penalty to achieve sparsity at both the group-level and within-group-level. As numerical experiments show, compared with other alternative methods, our method has better performances in variable selection and parameter estimation. Finally, we apply our proposed method to analyze NBA salary data during the period from 2018 to 2019.

The influence of flood exposure and subsequent stressors on youth social-emotional health.

Most disaster mental health research focuses on the relationship between disaster exposure and distress, often neglecting its influence on social-emotional health, despite implications for resilience and well-being after the disaster. Following multiple floods in Texas, a sample of 486 youth aged 10-19 years old (M = 13.74 years, SD = 2.57; 52.9% male) completed measures of disaster exposure, life stressors since the disaster, and social-emotional health. Using mixture regression modeling, we examined differences in the relationship between life stressors and social-emotional health across latent classes of disaster exposure (high, moderate, community, and low exposure). After accounting for mean levels of life stressors, the mean levels of social-emotional health did not differ across exposure classes; however, the strength of the relationship between life stressors and social-emotional health did. Youth in the high exposure group had the highest mean level of life stressors since the disaster. Thus, each additional life stressor did not result in changes in social-emotional health, suggesting saturated stress levels. For youth in the moderate and community exposure classes, increases in life stressors did lower social-emotional health, perhaps pushing them into stress overload. For the low exposure group, life stressors did not have an influence. This has implications for postdisaster mental health screening and support, tailored by levels of exposure and attuned to ongoing stressors that may impact long-term social-emotional health. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Cluster validity indices for mixture hazards regression models.

In the analysis of survival data, the problems of competing risks arise frequently in medical applications where individuals fail from multiple causes. Semiparametric mixture regression models have become a prominent approach in competing risks analysis due to their flexibility and easy interpretation of resultant estimates. The literature presents several semiparametric methods on the estimations for mixture Cox proportional hazards models, but fewer works appear on the determination of the number of model components and the estimation of baseline hazard functions using kernel approaches. These two issues are important because both incorrect number of components and inappropriate baseline functions can lead to insufficient estimates of mixture Cox hazard models. This research thus proposes four validity indices to select the optimal number of model components based on the posterior probabilities and residuals resulting from the application of an EM-based algorithm on a mixture Cox regression model. We also introduce a kernel approach to produce a smooth estimate of the baseline hazard function in a mixture model. The effectiveness and the preference of the proposed cluster indices are demonstrated through a simulation study. An analysis on a prostate cancer dataset illustrates the practical use of the proposed method.

Variable selection in finite mixture of regression models using the skew-normal distribution

Variable selection in finite mixture of regression (FMR) models is frequently used in statistical modeling. The majority of applications of variable selection in FMR models use a normal distribution for regression error. Such assumptions are unsuitable for a set of data containing a group or groups of observations with asymmetric behavior. In this paper, we introduce a variable selection procedure for FMR models using the skew-normal distribution. With appropriate choice of the tuning parameters, we establish the theoretical properties of our procedure, including consistency in variable selection and the oracle property in estimation. To estimate the parameters of the model, a modified EM algorithm for numerical computations is developed. The methodology is illustrated through numerical experiments and a real data example.

A regularized spatial market segmentation method with Dirichlet process—Gaussian mixture prior

Spatially referenced data are increasingly available thanks to the development of modern GPS technology. They also provide rich opportunities for spatial analytics in the field of marketing science. Our main interest is to propose a new efficient statistical framework to conduct spatial segmentation analysis for restaurants located in a metropolitan area in the U.S. The spatial segmentation problem poses important statistical challenges: selecting the optimal number of underlying structures of market segments, capturing complex and flexible spatial structures, and resolving any possible small sample size issue which can be typical in latent class analysis. Existing approaches try to tackle these issues in heuristic ways or seem silent on them. To overcome these challenges, we propose a new statistical framework based on regularized spatial mixture regressions with Dirichlet process integrating ridge or lasso regularization. Our simulation study demonstrates that the proposed models successfully recover the underlying spatial clustering structures and outperforms two existing benchmark models. In the empirical analysis using online customer satisfaction data from the Yelp, our model provides interesting insights on segment-level key drivers of customer satisfaction and interpretable relationships between regional demographics and restaurants’ characteristics.

SOC estimation of Li-ion battery based on gaussian mixture regression

SOC estimation of Li-ion battery based on gaussian mixture regression

Bayesian Nonlinear Gaussian Mixture Regression and its Application to Virtual Sensing for Multimode Industrial Processes

Virtual sensors have established themselves as effective tools in process industries for online estimating variables that are crucial but difficult to measure. However, multimode industrial processes render developing high-accuracy virtual sensors quite challenging. The main difficulties lie in that, in multimode processes, the distributions of process data are strongly non-Gaussian and the mathematical relationships between the explanatory and primary variables are highly nonlinear. Even within one operating mode, the primary variables could depend on the explanatory variables in nonlinear ways. In order to address these issues, this article proposes a virtual sensing approach named Bayesian nonlinear Gaussian mixture regression (BNGMR) with the aid of single-hidden layer feedforward neural networks (SLFNs). In the BNGMR, a fully Bayesian model structure that absorbs the merits of SLFNs and the mixture models is designed. In addition, we develop a training algorithm for the BNGMR to realize predictive virtual sensor development based on variational inference. Extensive assessments of the performance of the BNGMR are carried out using both artificial example and real-world industrial processes. The experiments have demonstrated the predictive advantage of the BNGMR over several benchmark methods and also have provided practitioners with good illustrations. Note to Practitioners —Although the case studies on two real-world industrial processes have verified the effectiveness and feasibility of the Bayesian nonlinear Gaussian mixture regression (BNGMR) for industrial applications, it is still worth emphasizing that before training the BNGMR, some important works on data preprocessing should be done, which include: 1) careful selection of explanatory variables that are closely related to primary variables; 2) time alignment matching between primary and explanatory variables; and 3) dealing with samples with missing values (for example, removing them) and elimination of outliers. We suggest to complete 1) and 2) based on process mechanisms and to pay particular attention to removing the outlier, as the presence of outliers could skew the estimations of Gaussian components and result in significant performance deterioration.

A Selective Overview and Comparison of Robust Mixture Regression Estimators

SummaryMixture regression models have been widely used in business, marketing and social sciences to model mixed regression relationships arising from a clustered and thus heterogeneous population. The unknown mixture regression parameters are usually estimated by maximum likelihood estimators using the expectation–maximisation algorithm based on the normality assumption of component error density. However, it is well known that the normality‐based maximum likelihood estimation is very sensitive to outliers or heavy‐tailed error distributions. This paper aims to give a selective overview of the recently proposed robust mixture regression methods and compare their performance using simulation studies.

Using Regression Mixture Analysis in Educational Research

Using Regression Mixture Analysis in Educational Research

A Generalized Model of Complex Allometry I: Formal Setup, Identification Procedures and Applications to Non-Destructive Estimation of Plant Biomass Units

(1) Background: We previously demonstrated that customary regression protocols for curvature in geometrical space all derive from a generalized model of complex allometry combining scaling parameters expressing as continuous functions of covariate. Results highlighted the relevance of addressing suitable complexity in enhancing the accuracy of allometric surrogates of plant biomass units. Nevertheless, examination was circumscribed to particular characterizations of the generalized model. Here we address the general identification problem. (2) Methods: We first suggest a log-scales protocol composing a mixture of linear models weighted by exponential powers. Alternatively, adopting an operating regime-based modeling slant we offer mixture regression or Takagi–Sugeno–Kang arrangements. This last approach allows polyphasic identification in direct scales. A derived index measures the extent on what complexity in arithmetic space drives curvature in arithmetical space. (3) Results: Fits on real and simulated data produced proxies of outstanding reproducibility strength indistinctly of data scales. (4) Conclusions: Presented analytical constructs are expected to grant efficient allometric projection of plant biomass units and also for the general settings of allometric examination. A traditional perspective deems log-transformation and allometry inseparable. Recent views assert that this leads to biased results. The present examination suggests this controversy can be resolved by addressing adequately the complexity of geometrical space protocols

Machine learning models for predicting the use of different animal breeding services in smallholder dairy farms in Sub-Saharan Africa

This study is concerned with developing predictive models using machine learning techniques to be used in identifying factors that influence farmers' decisions, predict farmers' decisions, and forecast farmers' demands relating to breeding service. The data used to develop the models comes from a survey of small-scale dairy farmers from Tanzania (n = 3500 farmers), Kenya (n = 6190 farmers), Ethiopia (n = 4920 farmers), and Uganda (n = 5390 farmers) and more than 120 variables were identified to influence breeding decisions. Feature engineering process was used to reduce the number of variables to a practical level and to identify the most influential ones. Three algorithms were used for feature selection, namely: logistic regression, random forest, and Boruta. Subsequently, six predictive models, using features selected by feature selection method, were tested for each country-neural network, logistic regression, K-nearest neighbor, decision tree, random forest, and Gaussian mixture model. A combination of decision tree and random forest algorithms was used to develop the final models. Each country model showed high predictive power (up to 93%) and are ready for practical use. The use of ML techniques assisted in identifying the key factors that influence the adoption of breeding method that can be taken and prioritized to improve the dairy sector among countries. Moreover, it provided various alternatives for policymakers to compare the consequences of different courses of action which can assist in determining which alternative at any particular choice point had a high probability to succeed, given the information and alternatives pertinent to the breeding decision. Also, through the use of ML, results to the identification of different clusters of farmers, who were classified based on their farm, and farmers' characteristics, i.e., farm location, feeding system, animal husbandry practices, etc. This information had significant value to decision-makers in finding the appropriate intervention for a particular cluster of farmers. In the future, such predictive models will assist decision-makers in planning and managing resources by allocating breeding services and capabilities where they would be most in demand.

Structured analysis of the high-dimensional FMR model

The finite mixture of regression (FMR) model is a popular tool for accommodating data heterogeneity. In the analysis of FMR models with high-dimensional covariates, it is necessary to conduct regularized estimation and identify important covariates rather than noises. In the literature, there has been a lack of attention paid to the differences among important covariates, which can lead to the underlying structure of covariate effects. Specifically, important covariates can be classified into two types: those that behave the same in different subpopulations and those that behave differently. It is of interest to conduct structured analysis to identify such structures, which will enable researchers to better understand covariates and their associations with outcomes. Specifically, the FMR model with high-dimensional covariates is considered. A structured penalization approach is developed for regularized estimation, selection of important variables, and, equally importantly, identification of the underlying covariate effect structure. The proposed approach can be effectively realized, and its statistical properties are rigorously established. Simulation demonstrates its superiority over alternatives. In the analysis of cancer gene expression data, interesting models/structures missed by the existing analysis are identified.

Nonlinear logistic regression mixture experiment modeling for binary data using dimensionally reduced components

AbstractThis article motivates, presents, and illustrates an approach using nonlinear logistic regression (NLR) for modeling binary response data from a mixture experiment when the components can be partitioned into groups used to form dimensionally reduced components (DRCs). A DRC is formed from a linear combination of the components in a group having similar roles and/or effects of the same sign, where the linear combinations over all groups are normalized so that the DRC proportions sum to one. Linear combinations of a particular form provide for quantifying the effects of the remaining components in a group relative to a chosen component. This reason, plus dimensional reduction, are the primary motivations for the proposed DRC mixture experiment modeling approach. NLR is required because models expressed in terms of the DRCs are nonlinear in the parameters that specify the linear combinations. A method for obtaining nonparametric tolerance limits on the probability of a “success” for the binary response variable using a bootstrap approach is also presented. Finally, the article shows how DRCs provide for visualizing data and modeling results that otherwise would be impossible. A real database on the presence or absence of nepheline crystals in simulated nuclear waste glass is used to illustrate the NLR modeling and nonparametric tolerance limit approaches. The methodology is general and can be applied to other applications.

Learning quasi-periodic robot motions from demonstration

The goal of Learning from Demonstration is to automatically transfer the skill knowledge from human to robot. Current researches focus on the problem of modeling aperiodic/periodic robot motions and extracting dynamic task parameters from the recorded sensory information. However, it is still not adequate for describing complex behaviors in an unstructured environment, such as searching for an unknown fitting position or painting/polishing an irregular surface. The quasi-periodic and stochastic properties cause a high demand for generalization ability of the modeling techniques. This paper proposes a systematic framework for learning quasi-periodic robot motions, which contains three steps: decomposition, modeling, and synthesization. Firstly FFT transform is performed to identify all the frequencies in the quasi-periodic motion. Then the motion is decomposed into an offset component, a series of harmonic and corresponding envelop components based on the concept of equivalent transformation. The offset component is extracted by Empirical Mode Decomposition, harmonic is separated by notch filter, and envelope component is extracted by Hilbert Transform. These components are either periodic or aperiodic. The aperiodic motions can be modeled by conventional techniques such as Gaussian Mixture Model and recovered by Gaussian Mixture Regression. The periodic motions are modeled in closed-form expressions. Finally, they are synthesized together to regenerate the robot motion. This modeling process captures both the aperiodicity and periodicity of a quasi-periodic motion. Simulation and experiment show that the proposed methods are feasible, effective and can predict robot motions beyond demonstrations. With this generalization ability, it is able to reduce the programming difficulty and demonstration complexity.

A robot learning framework based on adaptive admittance control and generalizable motion modeling with neural network controller

Robot learning from demonstration (LfD) enables robots to be fast programmed. This paper presents a novel LfD framework involving a teaching phase, a learning phase and a reproduction phase, and proposes methods in each of these phases to guarantee the overall system performance. An adaptive admittance controller is developed to take into account the unknown human dynamics so that the human tutor can smoothly move the robot around in the teaching phase. The task model in this controller is formulated by the Gaussian mixture regression to extract the human-related motion characteristics. In the learning and reproduction phases, the dynamic movement primitive is employed to model a robotic motion that is generalizable. A neural network-based controller is designed for the robot to track the trajectories generated from the motion model, and a radial basis function neural network is used to compensate for the effect caused by the dynamic environments. Experiments have been performed using a Baxter robot and the results have confirmed the validity of the proposed robot learning framework.

Generating realistic Arabic handwriting dataset

During the previous year's holistic approach showing satisfactory results to solve ‎the ‎problem of Arabic handwriting word recognition instead of word letters ‎‎segmentation.‎ ‎In this paper, we present an efficient system for ‎ generation realistic Arabic handwriting dataset from ASCII input ‎text. We carefully selected simple word list that contains most Arabic ‎letters normal and ligature connection cases. To improve the ‎performance of new letters reproduction we developed our ‎normalization method that adapt its clustering action according to ‎created Arabic letters families. We enhanced Gaussian Mixture ‎Model process to learn letters template by detecting the ‎number and position of Gaussian component by implementing ‎Ramer-Douglas-Peucker‎ algorithm which improve the new letters ‎shapes reproduced by using and Gaussian Mixture Regression. ‎‎We learn the translation distance between word-part to achieve ‎real handwriting word generation shape.‎ Using combination of LSTM and CTC layer as a recognizer to validate the ‎efficiency of our approach in generating new realistic Arabic handwriting words inherit user handwriting style as shown by the experimental results.‎

Trajectory analysis for postoperative pain using electronic health records: A nonparametric method with robust linear regression and K-medians cluster analysis.

Postoperative pain scores are widely monitored and collected in the electronic health record, yet current methods fail to fully leverage the data with fast implementation. A robust linear regression was fitted to describe the association between the log-scaled pain score and time from discharge after total knee replacement. The estimated trajectories were used for a subsequent K-medians cluster analysis to categorize the longitudinal pain score patterns into distinct clusters. For each cluster, a mixture regression model estimated the association between pain score and time to discharge adjusting for confounding. The fitted regression model generated the pain trajectory pattern for given cluster. Finally, regression analyses examined the association between pain trajectories and patient outcomes. A total of 3442 surgeries were identified with a median of 22 pain scores at an academic hospital during 2009-2016. Four pain trajectory patterns were identified and one was associated with higher rates of outcomes. In conclusion, we described a novel approach with fast implementation to model patients' pain experience using electronic health records. In the era of big data science, clinical research should be learning from all available data regarding a patient's episode of care instead of focusing on the "average" patient outcomes.