Heteroscedastic Linear Model Research Articles

A bootstrap method for estimation in linear mixed models with heteroscedasticity

Bootstrap is a widely applicable computational statistical method. The focus of this paper lies in developing a bootstrap method for linear mixed models under homoscedasticity violation (heteroscedasticity) in variance of errors. We assume that the form of heteroscedasticity is unknown. To generate bootstrap response data as close as possible to the actual response data, we transform the marginal residuals to ensure that the variance of the modified marginal residuals in bootstrap samples is an unbiased estimator for the variance of the response variable in the linear mixed models. The consistency is proved, implying that the parameter estimators by the proposed method are asymptotically unbiased. The simulations are conducted with varying error terms and sample sizes to show improvement of the proposed method in parameter estimation. The simulation results are compared to demonstrate that the proposed method outperforms the other considered approaches including the wild method in the heteroscedastic linear mixed models under small samples and performs competitively with the parametric method and both methods outperform the other considered methods in homoscedastic linear mixed models. The proposed bootstrap method is applied to squid data for illustration of its effectiveness.

Order types and natural price change: Model and empirical study of the Chinese market

Order type plays an important role in algorithmic trading and is a key factor of price impact. In this paper, we propose a new framework for studying the discrete price change process, which focuses on the impacts of aggressive orders (market orders and aggressive limit orders) and cancelations. The price change process is driven by states and events of best quotes, and we define the event-based price change as the “natural price change” (NPC). Under the framework, we propose a heteroscedastic linear econometric model for the NPC to explore the impact of different types of orders on the price dynamics. To verify the usability of our model and explore the driving factors of price dynamics, we conduct a thorough empirical analysis for 786 large-tick stocks traded on the Shenzhen Stock Exchange. Empirical results statistically demonstrate that aggressive orders can introduce stronger impact on the NPC than cancelations. Meanwhile, splitting a big order into several small orders can lead to a larger NPC. Our framework can also be applied for the prediction of price change.

Robust estimation and variable selection in heteroscedastic linear regression

ABSTRACTThe paper concerns robust estimation and variable selection in heteroscedastic linear regression models. After a brief review of existing methods for estimation in such models, a robust S-estimation approach is discussed. For all methods concise descriptions of algorithms are provided. Little is available upon robust variable selection methods for heteroscedastic linear models. The paper gives essential contributions in the area of simultaneous robust estimation and variable selection, relying on basics of the nonnegative garrote method which has been proven to have very good practical as well as theoretical properties in the homoscedastic linear model context. Several numerical examples, simulations and analysis of real data, demonstrate the performances and practical use of the discussed methods. Moreover, we provide expressions for the influence functions of the estimators of the mean and the error variance parameters. Influence functions are plotted in a simple setting providing insights in the sensitivity of the estimators for a single outlying observation.

Application and relative importance of spatial analysis for replicated multi-environment yield trials in sugarcane (Saccharum Spp.)

Data obtained from multi-environment yield trials of sugarcane are usually analyzed using traditional models that detect only those variations existed between plots or test genotypes. However, blocking does not control spatial heterogeneity existed within and between trials. This investigation was conducted to evaluate the application and relative importance of spatial analysis for sugarcane multi-environment trials conducted under different designs. Yield data obtained from 20 yield trials were analyzed using classical linear and autoregressive models on the basis of mixed model. Results of the within trial analysis indicated both Akaike information criterion (AIC) and relative efficiency selected the classical linear model as best model in most of the cases. The anisotropic autoregressive model was the best model in modeling cane and sugar yields data of plant cane crop trials which received poor management practices and should be applied under these circumstances. When we analyze the combined multi-environment trials data, the heteroscedastic anisotropic autoregressive model showed better adjustments for cane and sugar yield data modeled across locations, while heteroscedastic linear model was the best model for cane and sugar yields data modeled over crop years. Thus, the heteroscedastic isotropic autoregressive model is recommended to be applied in sugarcane METs data analysis. Moreover, our investigation insures the applicability of the spatial analysis for data obtained from yield trials conducted under simple lattice design.

Empirical Likelihood Ratio Tests for Coefficients in High Dimensional Heteroscedastic Linear Models

Empirical Likelihood Ratio Tests for Coefficients in High Dimensional Heteroscedastic Linear Models

On the finite-sample properties of conditional empirical likelihood estimators

ABSTRACTWe provide Monte Carlo evidence on the finite-sample behavior of the conditional empirical likelihood (CEL) estimator of Kitamura, Tripathi, and Ahn and the conditional Euclidean empirical likelihood (CEEL) estimator of Antoine, Bonnal, and Renault in the context of a heteroscedastic linear model with an endogenous regressor. We compare these estimators with three heteroscedasticity-consistent instrument-based estimators and the Donald, Imbens, and Newey estimator in terms of various performance measures. Our results suggest that the CEL and CEEL with fixed bandwidths may suffer from the no-moment problem, similarly to the unconditional generalized empirical likelihood estimators studied by Guggenberger. We also study the CEL and CEEL estimators with automatic bandwidths selected through cross-validation. We do not find evidence that these suffer from the no-moment problem. When the instruments are weak, we find CEL and CEEL to have finite-sample properties—in terms of mean squared error and coverage probability of confidence intervals—poorer than the heteroscedasticity-consistent Fuller (HFUL) estimator. In the strong instruments case, the CEL and CEEL estimators with automatic bandwidths tend to outperform HFUL in terms of mean squared error, while the reverse holds in terms of the coverage probability, although the differences in numerical performance are rather small.

Optimal designs for multiple treatments with unequal variances

The response of a patient in a clinical trial usually depends on both the selected treatment and some latent covariates, while its variance varies across the treatment groups. A general heteroscedastic linear additive model incorporating the treatment effect and the covariate effects is often used in such studies. In this paper, under D- and DA-optimality criteria, it is shown that the product of an optimal treatment allocation and an optimal design for covariates is also optimal among all possible designs for this linear additive model. Moreover, the optimal treatment allocation is characterized by a unique set of solutions to a system of equations. The connection between D- and DA-optimal designs is also revealed. Several examples are presented to illustrate the applications of the above results to some selected models.

FVGWAS: Fast voxelwise genome wide association analysis of large-scale imaging genetic data

More and more large-scale imaging genetic studies are being widely conducted to collect a rich set of imaging, genetic, and clinical data to detect putative genes for complexly inherited neuropsychiatric and neurodegenerative disorders. Several major big-data challenges arise from testing genome-wide (NC>12 million known variants) associations with signals at millions of locations (NV~106) in the brain from thousands of subjects (n~103). The aim of this paper is to develop a Fast Voxelwise Genome Wide Association analysiS (FVGWAS) framework to efficiently carry out whole-genome analyses of whole-brain data. FVGWAS consists of three components including a heteroscedastic linear model, a global sure independence screening (GSIS) procedure, and a detection procedure based on wild bootstrap methods. Specifically, for standard linear association, the computational complexity is O (nNVNC) for voxelwise genome wide association analysis (VGWAS) method compared with O ((NC+NV)n2) for FVGWAS. Simulation studies show that FVGWAS is an efficient method of searching sparse signals in an extremely large search space, while controlling for the family-wise error rate. Finally, we have successfully applied FVGWAS to a large-scale imaging genetic data analysis of ADNI data with 708 subjects, 193,275voxels in RAVENS maps, and 501,584 SNPs, and the total processing time was 203,645s for a single CPU. Our FVGWAS may be a valuable statistical toolbox for large-scale imaging genetic analysis as the field is rapidly advancing with ultra-high-resolution imaging and whole-genome sequencing.

Efficiency of heterosceastic linear model

In order to investigate the asymptotic efficiency of estimators under two different simulation techniques, normal-normal double sided Heteroscedastic error structure was adopted. We explored Direct Monte Carlo method of A. Zellner (2010) and Metropolis Hasting Algorithm experiments, an approach of Markov Chain Monte Carlo. We truncated the model with one error component of two sided error structure. A Metropolis-Hasting Algorithm and Direct Monte Carlo adopted to perform simulation on joint posterior distribution of heteroscedastic linear econometric model. Since Ordinary Least squares is invalid and inefficient in the presence of heteroscedastic, heteroscedastic linear model was conjugated with informative priors to form posterior distribution. Maximum Likelihood Estimation was compared with Bayesian Maximum Likelihood Estimation, Mean Squares Error criterion was use to identify which estimator and/or simulation method outperform other. We chose the following sample sizes: 25; 50; 100; and 200. Thus 10,000 simulations with varying degree of heteroscedastic error structures were adopted. This is subjected to the level of convergence. The overall using minimum mean squares error criterion revealed improving performance asymptotically regardless of the degree of heteroscedasticity. The results showed that Direct Monte Carlo Method outperformed Markov Chain Monte Carlo Method and Maximum Likelihood Estimator with minimum mean square error at any degree of heteroscedasticity.

Spatial Shrinkage Estimation of Diffusion Tensors on Diffusion-Weighted Imaging Data

Diffusion tensor imaging (DTI), based on the diffusion-weighted imaging (DWI) data acquired from magnetic resonance experiments, has been widely used to analyze the physical structure of white-matter fibers in the human brain in vivo. The raw DWI data, however, carry noise; this contaminates the diffusion tensor (DT) estimates and introduces systematic bias into the induced eigenvalues. These bias components affect the effectiveness of fiber-tracking algorithms. In this article, we propose a two-stage spatial shrinkage estimation (SpSkE) procedure to accommodate the spatial information carried in DWI data in DT estimation and to reduce the bias components in the corresponding derived eigenvalues. To this end, in the framework of the heteroscedastic linear model, SpSkE incorporates L 1-type penalization and the locally weighted least-square function. The theoretical properties of SpSkE are explored. The effectiveness of SpSkE is further illustrated by simulation and real-data examples. Supplementary materials for this article are available online.

Bayesian forecasting of demand time-series data with zero values

This paper describes the development of a Bayesian procedure to analyse and forecast positive demand time-series data with a proportion of zero values and a high level of variability for the non-zero data. The resulting forecasts play decisive roles in organisational planning, budgeting, and performance monitoring. Exponential smoothing methods are widely used as forecasting techniques in industry and business. However, they can be unsuitable for the analysis of non-negative demand time-series data with the aforementioned features. In this paper, an unconstrained latent demand underlying the observed demand is introduced into the linear heteroscedastic model associated with the Holt-Winters model. Accurate forecasts for the observed demand can readily be derived from those obtained with exponential smoothing for the latent demand. The performance of the proposed procedure is illustrated using a simulation study and two real time-series datasets which correspond to tourism demand and book sales.

Variational approximation for heteroscedastic linear models and matching pursuit algorithms

Modern statistical applications involving large data sets have focused attention on statistical methodologies which are both efficient computationally and able to deal with the screening of large numbers of different candidate models. Here we consider computationally efficient variational Bayes approaches to inference in high-dimensional heteroscedastic linear regression, where both the mean and variance are described in terms of linear functions of the predictors and where the number of predictors can be larger than the sample size. We derive a closed form variational lower bound on the log marginal likelihood useful for model selection, and propose a novel fast greedy search algorithm on the model space which makes use of one step optimization updates to the variational lower bound in the current model for screening large numbers of candidate predictor variables for inclusion/exclusion in a computationally thrifty way. We show that the model search strategy we suggest is related to widely used orthogonal matching pursuit algorithms for model search but yields a framework for potentially extending these algorithms to more complex models. The methodology is applied in simulations and in two real examples involving prediction for food constituents using NIR technology and prediction of disease progression in diabetes.

Simultaneous variable selection for heteroscedastic regression models

In this paper, we propose a new criterion, named PICa, to simultaneously select explanatory variables in the mean model and variance model in heteroscedastic linear models based on the model structure. We show that the new criterion can select the true mean model and a correct variance model with probability tending to 1 under mild conditions. Simulation studies and a real example are presented to evaluate the new criterion, and it turns out that the proposed approach performs well.

Bayesian forecasting with the Holt–Winters model

Exponential smoothing methods are widely used as forecasting techniques in inventory systems and business planning, where reliable prediction intervals are also required for a large number of series. This paper describes a Bayesian forecasting approach based on the Holt–Winters model, which allows obtaining accurate prediction intervals. We show how to build them incorporating the uncertainty due to the smoothing unknowns using a linear heteroscedastic model. That linear formulation simplifies obtaining the posterior distribution on the unknowns; a random sample from such posterior, which is not analytical, is provided using an acceptance sampling procedure and a Monte Carlo approach gives the predictive distributions. On the basis of this scheme, point-wise forecasts and prediction intervals are obtained. The accuracy of the proposed Bayesian forecasting approach for building prediction intervals is tested using the 3003 time series from the M3-competition.

Weighted least squares method for censored linear models

For estimation of linear models with randomly censored data, a class of data transformations is used to construct synthetic data. It is shown that the conditional variance of the synthetic data depends on the covariates in the model regardless of the homoscedasticity of the error. Therefore, linear models based on the synthetic data are always heteroscedastic models. To improve efficiency, we propose a weighted least squares (WLS) method, where the conditional variance of the synthetic data is estimated nonparametrically, then the standard WLS principle is applied to the synthetic data in the estimation procedure. The resultant estimator is asymptotically normal and the limiting variance is estimated using the plug-in method. In general, the proposed method improves the existing synthetic data methods for censored linear models, and gains more efficiency. For the censored heteroscedastic linear models, where the Buckley–James (BJ) and rank-based methods cannot be used since the condition of homoscedastic errors is violated, the new method provides a solution for better estimation. Monte Carlo simulations are conducted to compare the proposed method with the unweighted least squares method and the BJ method under different error conditions.

A theorem on the covariance matrix of a generalized least squares estimator under an elliptically symmetric error

In a general linear regression model, a generalized least squares estimator (GLSE) is widely employed as an estimator of regression coefficient. The efficiency of the GLSE is usually measured by its covariance (or risk) matrix. In this paper, it is shown that the covariance matrix remains the same as long as the distribution of the error term is elliptically symmetric. This implies that every efficiency result obtained under normal distribution assumption is still valid under elliptical symmetry. An application to a heteroscedastic linear model is also illustrated.

Robust Testing Procedures in Heteroscedastic Linear Models

There exist many studies which treat the robust tests in homoscedastic linear models. However, the robust testing procedure in heteroscedastic linear models has not been examined. In this article, three classes of testing procedures for testing subhypothesis in heteroscedastic linear models are developed. These are Wald-type, score-type, and drop-in dispersion tests. The asymptotic distributions of these tests are obtained under the null hypothesis and contiguous alternatives. For a robustness criterion, the maximum asymptotic bias of the level of the test for distributions in a shrinking contamination neighborhood is used and the most-efficient robust test is derived. Finally, the performance of these tests in small sample is studied by Monte Carlo simulation.

A RESTRICTED MAXIMUM LIKELIHOOD PROCEDURE FOR ESTIMATING THE VARIANCE FUNCTION OF AN IMMUNOASSAY

SummaryRestricted maximum likelihood (REML) is a procedure for estimating a variance function in a heteroscedastic linear model. Although REML has been extended to non‐linear models, the case in which the data are dominated by replicated observations with unknown values of the independent variable of interest, such as the concentration of a substance in a blood sample, has not been considered. We derive a REML procedure for an immunoassay and show that the resulting estimator is superior to those currently being used. Some interesting properties of the REML estimator are derived, and its relationship to other estimators is discussed.

A Statistical Analysis of Brain Morphology Using Wild Bootstrapping

Methods for the analysis of brain morphology, including voxel-based morphology and surface-based morphometries, have been used to detect associations between brain structure and covariates of interest, such as diagnosis, severity of disease, age, IQ, and genotype. The statistical analysis of morphometric measures usually involves two statistical procedures: 1) invoking a statistical model at each voxel (or point) on the surface of the brain or brain subregion, followed by mapping test statistics (e.g., t test) or their associated p values at each of those voxels; 2) correction for the multiple statistical tests conducted across all voxels on the surface of the brain region under investigation. We propose the use of new statistical methods for each of these procedures. We first use a heteroscedastic linear model to test the associations between the morphological measures at each voxel on the surface of the specified subregion (e.g., cortical or subcortical surfaces) and the covariates of interest. Moreover, we develop a robust test procedure that is based on a resampling method, called wild bootstrapping. This procedure assesses the statistical significance of the associations between a measure of given brain structure and the covariates of interest. The value of this robust test procedure lies in its computationally simplicity and in its applicability to a wide range of imaging data, including data from both anatomical and functional magnetic resonance imaging (fMRI). Simulation studies demonstrate that this robust test procedure can accurately control the family-wise error rate. We demonstrate the application of this robust test procedure to the detection of statistically significant differences in the morphology of the hippocampus over time across gender groups in a large sample of healthy subjects.

Heteroscedastic symmetrical linear models

We discuss in this paper heteroscedastic linear models with symmetrical errors. The symmetrical class includes all symmetrical continuous distributions such as normal, Student- t, power exponential, logistics I and II, contaminated normal, so on. The variety of error distributions with different kurtosis coefficients than the normal one may give more flexibility in the choice of an appropriate error distribution, particularly to accommodate outlying and influential observations. We derive a joint iterative process for estimating the location and dispersion coefficients and we discuss some robustness aspects of the maximum likelihood estimates against outlying and large variance observations. The score test proposed by Cook and Weisberg [1983. Diagnostics for heteroscedasticity in regression. Biometrika 70, 1–10] is generalized and some diagnostic procedures such as leverage, local influence and residual analysis are derived. Finally, a data set is analyzed under heteroscedastic linear models with normal and heavy-tailed error distributions.