Mixture Of Student Research Articles

Tail Conditional Moments for Location-Scale Mixture of Elliptical Distributions

We present the general results on the univariate tail conditional moments for a location-scale mixture of elliptical distributions. Examples include the location-scale mixture of normal, location-scale mixture of Student’s t, location-scale mixture of logistic, and location-scale mixture of Laplace distributions. More specifically, we give the tail variance, the tail conditional skewness, and the tail conditional kurtosis of generalised hyperbolic distribution and Student–GIG mixture distribution. We give an illustrative example, which discusses the TCE, TV, TCS and TCK of three stocks, including Amazon, Google and Apple.

Tail conditional risk measures for location-scale mixture of elliptical distributions

We present general results on the univariate tail conditional expectation (TCE) and multivariate tail conditional expectation (MTCE) for location-scale mixture of elliptical distributions. Examples include the location-scale mixture of normal distributions, location-scale mixture of Student- distributions, location-scale mixture of logistic distributions and location-scale mixture of Laplace distributions. We also consider portfolio risk decomposition with TCE for location-scale mixture of elliptical distributions. More specifically, we give MTCEs of generalized hyperbolic and slash distributions, and discuss the difference of MTCEs for generalized hyperbolic and slash distributions. As an illustrative example, we discuss the MTCE of five stocks including Amazon, Goldman Sachs, IBM, Google and Apple.

Bayesian robust multi-extreme learning machine

Outliers usually exist in the collected data due to human or instrumentation errors. Their existence makes the regression error obey a non-Gaussian distribution. So, people assume that the regression error obeys finite mixture distributions, such as mixture of Gaussians and mixture of Student’s t-distributions. This paper proposed two Bayesian robust regression models based on multiple extreme learning machines. First, the finite mixture error distributions are extended to their infinite counterparts according to the theory of stick-breaking construction, which can avoid selecting the optimal number of components in mixture models. Second, a multi-extreme learning machine, which combines many single extreme learning machines with different numbers of hidden nodes, is constructed, which can avoid the effects of different numbers of hidden nodes on the final regression performance. Besides, sparse Bayesian learning has also been derived to perform the sparse model weight and output weight inference automatically. The experimental results on artificial datasets and eight benchmark datasets show that the proposed two models outperform the other compared models under different rates of outliers. Also, they generate better multi-step ahead wind speed and traffic speed forecasts in real applications.

Robust Task Learning Based on Nonlinear Regression With Mixtures of Student-t Distributions

We propose a robust task learning method based on nonlinear regression model with mixtures of t-distributions. The model can adaptively reduce the effects of complex noises and accurately learn the nonlinear structure of targets. By introducing latent variables, the model is expressed into a hierarchical structure, which helps explain the advantage of flexibility compared to the traditional Gaussian based learning model. We develop a two-stage efficient estimation procedure to obtain penalized likelihood estimator of the parameters combined an expectation-maximization algorithm with Lagrange multiplier method. The learning performances of the model are investigated through experiments on both synthetic and real data sets.

Flexible Models for Stock Returns Based on Student'sTDistribution

Models based on the Student'stdistribution are proposed with its scale parameter randomized. Mathematical properties of the models such as their probability density functions, cumulative distribution functions, moments and characteristic functions are derived. Three of the models are fitted to daily log returns of six financial indices. They were shown to provide better fits than mixtures of Student'stdistributions and the popular generalized hyperbolic distribution.

A Student's t Mixture Probability Hypothesis Density Filter for Multi-Target Tracking with Outliers.

In multi-target tracking, the outliers-corrupted process and measurement noises can reduce the performance of the probability hypothesis density (PHD) filter severely. To solve the problem, this paper proposed a novel PHD filter, called Student’s t mixture PHD (STM-PHD) filter. The proposed filter models the heavy-tailed process noise and measurement noise as a Student’s t distribution as well as approximates the multi-target intensity as a mixture of Student’s t components to be propagated in time. Then, a closed PHD recursion is obtained based on Student’s t approximation. Our approach can make full use of the heavy-tailed characteristic of a Student’s t distribution to handle the situations with heavy-tailed process and the measurement noises. The simulation results verify that the proposed filter can overcome the negative effect generated by outliers and maintain a good tracking accuracy in the simultaneous presence of process and measurement outliers.

Robust Student’s t Mixture Probability Hypothesis Density Filter for Multi-Target Tracking With Heavy-Tailed Noises

In order to improve filtering accuracy and restrain the degradation of filtering performance caused by the heavy-tailed process and measurement noises in multi-target tracking, this paper proposes a robust Student’s t mixture probability hypothesis density (PHD) filter. In the proposed method, a Student’s t mixture is implemented to the PHD filter, which recursively propagates the intensity as a mixture of Student’s t components in PHD filtering framework. Furthermore, with the advantage of a designed judging and re-weighting mechanism, an M-estimation-based dual-gating strategy is designed for the Student’s t mixture implementation to suppress the negative effect of the heavy-tailed noises. Our proposed approach not only utilizes the Student’s t distribution to match the real heavy-tailed non-Gaussian noise well but also enhances the robustness of the Student’s t mixture-based approach via the designed dual-gating strategy. The simulation results verify that the proposed algorithm can keep good filtering accuracy in the presence of the process and measurement outliers simultaneously.

Merging Student’s-t and Rayleigh distributions regression mixture model for clustering time-series

This paper addresses an efficient scheme for clustering time-series through a novel regression mixture strategy (RMM) that simultaneously utilizes the benefits of the Markov random field (MRF) and mean template. Each component of the proposed RMM is a mixture of Student’s-t and non-symmetric Rayleigh distributions. The Student’s-t distribution has been validated to be effective in reducing the influence of outliers. Another advantage is it considers the property of time-series in which spatially adjacent voxels have higher probabilities when belonging to the same cluster; thus, this study seeks to impose the spatially interdependent constraints in the form of a prior through a Bayesian theorem. By introducing the non-symmetric Rayleigh distribution, the proposed method has flexibility to fit various types of observed time-series. To optimize the model parameters, the proposed algorithm is formulated as a maximum a posteriori (MAP) estimation problem. An expectation maximization (EM)-type approach is eventually executed to obtain the update equations. Additionally, an efficient energy function is applied in the proposed approach such that the EM-MAP algorithm can be directly applied to calculate the objective function, which makes the proposed approach easier to implement. To illustrate the suitability of the proposed approach, we have extensively compared clustering accuracy, curve square error, and intraclass index with those of existing similarity methods on synthetic spatiotemporal datasets, synthetic fMRI time-series, and real life fMRI time-series.

Distributed Density Estimation Based on a Mixture of Factor Analyzers in a Sensor Network.

Distributed density estimation in sensor networks has received much attention due to its broad applicability. When encountering high-dimensional observations, a mixture of factor analyzers (MFA) is taken to replace mixture of Gaussians for describing the distributions of observations. In this paper, we study distributed density estimation based on a mixture of factor analyzers. Existing estimation algorithms of the MFA are for the centralized case, which are not suitable for distributed processing in sensor networks. We present distributed density estimation algorithms for the MFA and its extension, the mixture of Student’s t-factor analyzers (MtFA). We first define an objective function as the linear combination of local log-likelihoods. Then, we give the derivation process of the distributed estimation algorithms for the MFA and MtFA in details, respectively. In these algorithms, the local sufficient statistics (LSS) are calculated at first and diffused. Then, each node performs a linear combination of the received LSS from nodes in its neighborhood to obtain the combined sufficient statistics (CSS). Parameters of the MFA and the MtFA can be obtained by using the CSS. Finally, we evaluate the performance of these algorithms by numerical simulations and application example. Experimental results validate the promising performance of the proposed algorithms.

MODEL-BASED CLUSTERING WITH GENE RANKING USING PENALIZED MIXTURES OF HEAVY-TAILED DISTRIBUTIONS

Cluster analysis of biological samples using gene expression measurements is a common task which aids the discovery of heterogeneous biological sub-populations having distinct mRNA profiles. Several model-based clustering algorithms have been proposed in which the distribution of gene expression values within each sub-group is assumed to be Gaussian. In the presence of noise and extreme observations, a mixture of Gaussian densities may over-fit and overestimate the true number of clusters. Moreover, commonly used model-based clustering algorithms do not generally provide a mechanism to quantify the relative contribution of each gene to the final partitioning of the data. We propose a penalized mixture of Student's t distributions for model-based clustering and gene ranking. Together with a resampling procedure, the proposed approach provides a means for ranking genes according to their contributions to the clustering process. Experimental results show that the algorithm performs well comparably to traditional Gaussian mixtures in the presence of outliers and longer tailed distributions. The algorithm also identifies the true informative genes with high sensitivity, and achieves improved model selection. An illustrative application to breast cancer data is also presented which confirms established tumor sub-classes.

Joint Independent Metropolis-Hastings Methods for Nonlinear Non-Gaussian State Space Models

We propose a new methodology for the Bayesian analysis of nonlinear nonGaussian state space models with a Gaussian time-varying signal, where the signal is a function of a possibly high-dimensional state vector. The novelty of our approach is the development of proposal densities for the joint posterior density of parameter and state vectors: a mixture of Student’s t-densities as the marginal proposal density for the parameter vector, and a Gaussian density as the conditional proposal density for the signal given the parameter vector. We argue that a highly efficient procedure emerges when these proposal densities are used in an independent Metropolis-Hastings algorithm. A particular feature of our approach is that smoothed estimates of the states and an estimate of the marginal likelihood are obtained directly as an output of the algorithm. Our methods are computationally efficient and produce more accurate estimates when compared to recently proposed alternatives. We present extensive simulation evidence for stochastic volatility and stochastic intensity models. For our empirical study, we analyse the performance of our method for stock return data and corporate default panel data. JEL classification: C11, C15, C22, C32, C58.

The infinite Student's t-factor mixture analyzer for robust clustering and classification

Recently, the Student's t-factor mixture analyzer (tFMA) has been proposed. Compared with the mixture of Student's t-factor analyzers (MtFA), the tFMA has better performance when processing high-dimensional data. Moreover, the factors estimated by the tFMA can be visualized in a low-dimensional latent space, which is not shared by the MtFA. However, as the tFMA belongs to finite mixtures and the related parameter estimation method is based on the maximum likelihood criterion, it could not automatically determine the appropriate model complexity according to the observed data, leading to overfitting. In this paper, we propose an infinite Student's t-factor mixture analyzer (itFMA) to handle this issue. The itFMA is based on the nonparametric Bayesian statistics which assumes infinite number of mixing components in advance, and automatically determines the proper number of components after observing the high-dimensional data. Moreover, we derive an efficient variational inference algorithm for the itFMA. The proposed itFMA and the related variational inference algorithm are used to cluster and classify high-dimensional data. Experimental results of some applications show that the itFMA has good generalization capacity, offering a more robust and powerful performance than other competing approaches.

A Bayesian Analysis of Body Mass Index Data From Small Domains Under Nonignorable Nonresponse and Selection

Here we analyze body mass index (BMI) data for children and adolescents from the Third National Health and Nutrition Examination Survey (NHANES III). Because of the lack of BMI values for a considerable number of the children and adolescents, and the differential probabilities of selection of these individuals, serious nonresponse and selection bias in inference can be present. To analyze the NHANES III BMI data, a nonignorable nonresponse model has been proposed to estimate the finite population means of small domains formed by crossing age, race and sex within counties. In this approach, the log-BMI values are used to obtain more normally distributed data, and the model includes a spline regression of log-BMI on age, adjusted for race, sex, and the interaction of race and sex. In this work, to assess the status of overweight and obesity in children and adolescents, our new model predicts the more informative finite population percentiles of BMI for these small domains, incorporating additional measures to minimize possible biases. These measures are the most appropriate transformation for the skewed BMI data, incorporation of an intraclass correlation within the households, and inclusion of selection probabilities into the nonignorable nonresponse model to reflect the higher probabilities of selection among black non-Hispanics and Hispanic-Americans. We also consider robustness and sensitivity to the assumptions of the nonignorable nonresponse model by fitting several versions of our proposed model, as well as a very different ignorable nonresponse model that uses a mixture of Student t densities, selection probabilities, and BMI values. It is noteworthy that in the likelihood-based inference literature, we have seen no work by others that includes both nonignorable nonresponse and nonignorable selection. Based on NHANES III data, we show that there are differences in the 85th or 95th percentile for overweight by county, race, and particularly age, and a small difference in sex.

The mixtures of Student’s t-distributions as a robust framework for rigid registration

The problem of registering images or point sets is addressed. At first, a pixel similarity-based algorithm for the rigid registration between single and multimodal images is presented. The images may present dissimilarities due to noise, missing data or outlying measures. The method relies on the partitioning of a reference image by a Student’s t-mixture model (SMM). This partition is then projected onto the image to be registered. The main idea is that a t-component in the reference image corresponds to a t-component in the image to be registered. If the images are correctly registered the distances between the corresponding components is minimized. Moreover, the extension of the method to the registration of point clouds is also proposed. The use of SMM components is justified by the property that they have heavier tails than standard Gaussians, thus providing robustness to outliers. Experimental results indicate that, even in the case of low SNR or important amount of dissimilarities due to temporal changes, the proposed algorithm compares favorably to the mutual information method for image registration and to the Iterative Closest Points (ICP) algorithm for the alignment of point sets.

Possibly Ill-Behaved Posteriors in Econometric Models

textabstractHighly non-elliptical posterior distributions may occur in several econometric models, in particular, when the likelihood information is allowed to dominate and data information is weak. We explain the issue of highly non-elliptical posteriors in a model for the effect of education on income using data from the well-known Angrist and Krueger (1991) study and discuss how a so-called Information Matrix or Jeffreys' prior may be used as a `regularization prior' that in combination with the likelihood yields posteriors with desirable properties. We further consider an 8-dimensional bimodal posterior distribution in a 2-regime mixture model for the real US GNP growth. In order to perform a Bayesian posterior analysis using indirect sampling methods in these models, one has to find a good candidate density. In a recent paper - Hoogerheide, Kaashoek and Van Dijk (2007) - a class of neural network functions was introduced as candidate densities in case of non-elliptical posteriors. In the present paper, the connection between canonical model structures, non-elliptical credible sets, and more sophisticated neural network simulation techniques is explored. In all examples considered in this paper – a bimodal distribution of Gelman and Meng (1991) and posteriors in IV and mixture models - the mixture of Student's t distributions is clearly a much better candidate than a Student's t candidate, yielding far more precise estimates of posterior means after the same amount of computing time, whereas the Student's t candidate almost completely misses substantial parts of the parameter space.

Robust analysis of MRS brain tumour data using [formula omitted]-GTM

This paper proposes a principled, self-organized, framework to manage two sources of uncertainty that are inherent in intelligent systems for medical decision support, namely outliers and missing data. The framework is applied to magnetic resonance spectra (MRS), which are indicators of the grade of malignancy in brain tumours. A model for multivariate data clustering and visualization, the generative topographic mapping (GTM), is re-formulated as a mixture of Student's t-distributions making it more robust to outliers while supporting the imputation of missing values. An important new development is the extension of the model to provide automatic feature relevance determination. Its effectiveness on the MRS data is demonstrated empirically.