Heteroscedastic Structure Research Articles

Shrinkage and LASSO strategies in high-dimensional heteroscedastic models

ABSTRACTIn this paper, we consider the estimation problem of the parameter vector in the linear regression model with heteroscedastic errors. First, under heteroscedastic errors, we study the performance of shrinkage-type estimators and their performance as compared to theunrestricted and restricted least squares estimators. In order to accommodate the heteroscedastic structure, we generalize an identity which is useful in deriving the risk function. Thanks to the established identity, we prove that shrinkage estimators dominate the unrestricted estimator. Finally, we explore the performance of high-dimensional heteroscedastic regression estimator as compared to classical LASSO and shrinkage estimators.

Total least‐squares formulation of Wahba's problem

The Wahba's problem is widely recognised as a least-square problem in three-axis attitude estimation from vector measurements. However, both the observation and the reference vectors can be corrupted by errors, so a total least-square problem considering both errors should be formulated to be rigorous. It is found that for errors with heteroscedastic and isotropic covariance structures, the total least-square problem degenerates to be the Wahba's problem. A method for solving this problem is presented. Errors corrupting both observation and reference vectors can be equivalent to pseudo errors corrupting only observation vectors in the case of isotropic covariance structures, so the Wahba's problem can also be formulated as a least-square problem adjusting these pseudo errors in a somewhat simpler manner.

A Heterogeneous Growth Curve Model for Nonnormal Data

The heterogeneous growth curve model (HGM; Klein & Muthén, 2006) is a method for modeling heterogeneity of growth rates with a heteroscedastic residual structure for the slope factor. It has been developed as an extension of a conventional growth curve model and a complementary tool to growth curve mixture models. In this article, a robust version of the heterogeneous growth curve model (HGM-R) is presented that extends the original HGM with a mixture model to allow for an unbiased parameter estimation under the condition of nonnormal data. In two simulation studies, the performance of the method is examined under the condition of nonnormality and a misspecified heteroscedastic residual structure. The results of the simulation studies suggest an unbiased estimation of the heterogeneity by the HGM-R when sample size was large enough and a good approximation of the heteroscedastic residual structure even when the functional form of the heteroscedasticity was misspecified. The practical application of the approach is demonstrated for a data set from HIV-infected patients.

Time-varying coefficient models with ARMA–GARCH structures for longitudinal data analysis

Time-varying coefficient models with autoregressive and moving-average–generalized autoregressive conditional heteroscedasticity structure are proposed for examining the time-varying effects of risk factors in longitudinal studies. Compared with existing models in the literature, the proposed models give explicit patterns for the time-varying coefficients. Maximum likelihood and marginal likelihood (based on a Laplace approximation) are used to estimate the parameters in the proposed models. Simulation studies are conducted to evaluate the performance of these two estimation methods, which is measured in terms of the Kullback–Leibler divergence and the root mean square error. The marginal likelihood approach leads to the more accurate parameter estimates, although it is more computationally intensive. The proposed models are applied to the Framingham Heart Study to investigate the time-varying effects of covariates on coronary heart disease incidence. The Bayesian information criterion is used for specifying the time series structures of the coefficients of the risk factors.

A new stochastic model for simulating daily solar radiation from sunshine hours

ABSTRACTMany empirical models have been developed that use sunshine hours (SSH) data to estimate global solar radiation. Most of these models use the Angstrom‐Prescott equation to produce deterministic estimates of monthly radiation and do not provide uncertainty estimates in their predictions. This study develops five stochastic models that use daily SSH data to produce probabilistic simulations of global radiation, including associated uncertainties. These models can be used to estimate historical daily radiation or to estimate radiation without the use of satellite data. Two sources of predictive uncertainty are considered: (1) the timing of the SSH during the day (estimated using Monte Carlo simulation) and (2) external errors such as variability in cloud type and amount (estimated using residual error modelling). The models differ in the parameterization of the diffuse and direct radiation, using either no scaling, linear or quadratic scaling of the radiation by the daily SSH fraction to account for cloud attenuation. The models are calibrated under several residual error assumptions, including constant, linear and quadratic variances dependent on the SSH fraction and simulated radiation. The five models perform equally well in simulating global radiation, with an average error of approximately 9% for all locations studied. The results suggest that SSH uncertainty does not dominate predictive errors in global radiation. The residual errors appear to be best described by a linear heteroscedastic structure with larger variance during cloudy days and smaller variance during sunny days. The developed methodology provides a novel approach for estimating the uncertainty in radiation estimates based on SSH data.

Height-diameter Models of Chinese Fir (<em>Cunninghamia lanceolata</em>) Based on Nonlinear Mixed Effects Models in Southeast China

Tree height and diameter at breast height are two important forest factors. The best model from 23 height-diameter equations was selected as the basic model to fit the height-diameter relationships of Chinese fir with one level (sites or plots effects) and nested two levels (nested effects of sites and plots) Nonlinear Mixed Effects (NLME) models. The best model was chosen by smaller Bias, RMSE and larger R²_adj. Then the best random-effects combinations for the NLME models were determined by AIC, BIC and -2LL. The results showed that the basic model with three random effects parameters φ₁, φ₂ and φ₃ was considered the best mixed model. The nested two levels NLME model considering heteroscedasticity structure (power function) possessed with higher predictable accuracy and significantly improved model performance (LRT = 469.43, p<0.0001). The NLME model would be allowed for estimating accuracy the height-diameter relationships of Chinese fir and provided better height predictions than the models using only fixed-effects parameters.

Skew-normal distribution for growth curve models in presence of a heteroscedasticity structure

In general, growth models are adjusted under the assumptions that the error terms are homoscedastic and normally distributed. However, these assumptions are often not verified in practice. In this work we propose four growth models (Morgan–Mercer–Flodin, von Bertalanffy, Gompertz, and Richards) considering different distributions (normal, skew-normal) for the error terms and three different covariance structures. Maximum likelihood estimation procedure is addressed. A simulation study is performed in order to verify the appropriateness of the proposed growth curve models. The methodology is also illustrated on a real dataset.

Optimal Designs for Random Coefficient Regression Models with Heteroscedastic Errors

The purpose of this article is to present the optimal designs based on D-, G-, A-, I-, and D β-optimality criteria for random coefficient regression (RCR) models with heteroscedastic errors. A sufficient condition for the heteroscedastic structure is given to make sure that the search of optimal designs can be confined at extreme settings of the design region when the criteria satisfy the assumption of the real valued monotone design criteria. Analytical solutions of D-, G-, A-, I-, and D β-optimal designs for the RCR models are derived. Two examples are presented for random slope models with specific heteroscedastic errors.

Sample size determinations for Welch's test in one‐way heteroscedastic ANOVA

For one-way fixed effects ANOVA, it is well known that the conventional F test of the equality of means is not robust to unequal variances, and numerous methods have been proposed for dealing with heteroscedasticity. On the basis of extensive empirical evidence of Type I error control and power performance, Welch's procedure is frequently recommended as the major alternative to the ANOVA F test under variance heterogeneity. To enhance its practical usefulness, this paper considers an important aspect of Welch's method in determining the sample size necessary to achieve a given power. Simulation studies are conducted to compare two approximate power functions of Welch's test for their accuracy in sample size calculations over a wide variety of model configurations with heteroscedastic structures. The numerical investigations show that Levy's (1978a) approach is clearly more accurate than the formula of Luh and Guo (2011) for the range of model specifications considered here. Accordingly, computer programs are provided to implement the technique recommended by Levy for power calculation and sample size determination within the context of the one-way heteroscedastic ANOVA model.

Influence diagnostics in heteroscedastic and/or autoregressive nonlinear elliptical models for correlated data

In this paper, we propose nonlinear elliptical models for correlated data with heteroscedastic and/or autoregressive structures. Our aim is to extend the models proposed by Russo et al. 22 by considering a more sophisticated scale structure to deal with variations in data dispersion and/or a possible autocorrelation among measurements taken throughout the same experimental unit. Moreover, to avoid the possible influence of outlying observations or to take into account the non-normal symmetric tails of the data, we assume elliptical contours for the joint distribution of random effects and errors, which allows us to attribute different weights to the observations. We propose an iterative algorithm to obtain the maximum-likelihood estimates for the parameters and derive the local influence curvatures for some specific perturbation schemes. The motivation for this work comes from a pharmacokinetic indomethacin data set, which was analysed previously by Bocheng and Xuping 1 under normality.

Discussion: The forward search: Theory and data analysis

In the last decade the forward search has emerged not only as an algorithm for finding robust estimates, but also as a method for providing a broad range of exploratory (and confirmatory) procedures for analyzing data, such as data transformation and variable selection. For its original purpose of being a “fast very robust” procedure (Atkinson, 1994), the forward search cannot compete with the “fast algorithm” of Rousseeuw and van Driessen (1999, 2006) in terms of velocity. The basic idea behind the fast algorithm consists of carrying out many two-step procedures: a trial step followed by a refinement step (the so-called Concentration step). The latter step especially shows a “fast” (convergence) property. The authors are to be congratulated on the achievement of filling in the theoretical part for the forward search. The forward search also has several advantages for diagnostic plotting, such as the stalactite plot (Atkinson & Mulira, 1993), fan plot (Atkinson & Riani, 2000), and candlestick plot (Atkinson, Riani, & Cerioli, 2010). These contribute insights into the data analysis that no other available algorithms for robust estimators can. We present herein another advantage of the forward search this time for the heteroscedastic regression model (Cheng, submitted for publication). The benefit is from the nature of the forward search algorithm — the number of observations is augmented in a way that not only excludes the potentially outlying cases, but also takes into account the heteroscedastic structure of the data.

Real-Time Short-Term Traffic Speed Level Forecasting and Uncertainty Quantification Using Layered Kalman Filters

Short-term traffic condition forecasting has long been argued as essential for developing proactive traffic control systems that could alleviate the growing congestion in the United States. In this field, short-term traffic condition level forecasting and short-term traffic condition uncertainty forecasting play an equally important role. Past literature showed that linear stochastic time series models are promising in modeling and hence forecasting traffic condition levels and traffic conditional variance with workable performance. On the basis of this finding, an autoregressive moving average plus generalized autoregressive conditional heteroscedasticity structure was proposed for modeling the station-by-station traffic speed series. An online algorithm based on layered Kalman filter was developed for processing this structure in real time. Empirical results based on real-world station-by-station traffic speed data showed that the proposed online algorithm can generate workable short-term traffic speed level forecasts and associated prediction confidence intervals. Future work is recommended to develop and test a proactive traffic control system in a simulated environment, to refine the uncertainty modeling through a stochastic volatility model, and to extend uncertainty modeling and forecasting to link level and network level.

Microdeterminants of Poverty among the Farming Population in Bangladesh

This paper examines the determinants of poverty among the farming population of Bangladesh by applying a probit model with heteroscedastic structure (hetprobit). The model diagnostic reveals that the choice of hetprobit instead of a standard probit is more appropriate in this case. Among the socioeconomic factors, land ownership, farm resource endowments and non-agricultural income significantly reduce the probability of becoming poor. On the other hand, the number of dependants and education of female members significantly increase the likelihood of becoming poor. Adoption of green revolution technology does not seem to have any significant influence on the likelihood of being poor. However, regional and village level factors have significant influence on poverty. The likelihood of poverty is significantly lower in regions with developed infrastructure and high soil fertility. Poverty is also significantly lower in Comilla and Jamalpur, implying that geographical location matters. Policies to promote land ownership and farm resource endowments, investment in rural infrastructure development and soil fertility improvement will significantly reduce poverty among the farming population of Bangladesh.

The compass rose pattern in electricity prices

The "compass rose pattern" is known to appear in the phase portraits, or scatter diagrams, of the high-frequency returns of financial series. We first show that this pattern is also present in the returns of spot electricity prices. Early researchers investigating these phenomena hoped that these patterns signaled the presence of rich dynamics, possibly chaotic or fractal in nature. Although there is a definite autoregressive and conditional heteroscedasticity structure in electricity returns, we find that after simple filtering no pattern remains. While the series is non-normal in terms of their distribution and statistical tests fail to identify significant chaos, there is evidence of fractal structures in periodic price returns when measured over the trading day. The phase diagram of the filtered returns provides a useful visual check on independence, a property necessary for pricing and trading derivatives and portfolio construction, as well as providing useful insights into the market dynamics.

Data Collection Time Intervals for Stochastic Short-Term Traffic Flow Forecasting

The specification of time intervals for data collection is a fundamental determinant of the nature and utility of the resulting traffic condition data streams. In the context of short-term traffic flow forecasting, the establishment of the data collection time interval should play a key role in determining the corresponding appropriate forecasting approach. The data collection time interval provides the forecasting horizon for one-step-ahead forecasting. Nevertheless, the need for more rigorous understanding of the effects of data collection time interval specification within the context of short-term traffic flow forecasting is not well recognized. By contrast, it has been common practice in previous research to select the data collection time interval and forecasting approach without explicit consideration of time interval effects or systematic evaluation of available forecasting methods. A stochastic seasonal autoregressive integrated moving average plus generalized autoregressive conditional heteroscedasticity (SARIMA+GARCH) structure proposed in previous work holds promise in providing accurate point forecasts and reasonable forecasting confidence intervals. In this paper, a spectrum of data collection time intervals is tested with an online forecasting algorithm developed based on the SARIMA+GARCH structure to determine the applicable data collection time intervals for this structure. In this test, both the forecast accuracy and the validity of the forecasting confidence intervals are investigated. This work serves as an important step toward establishing a short-term traffic condition forecasting framework that identifies appropriate forecasting approaches for candidate data collection time intervals based on the functional requirements of specific applications.

Probit with heteroscedasticity: an application to Indian poverty analysis

This study argues that in limited dependent variable models, when there is heteroscedasticity, a probit model with a heteroscedastic structure should be estimated. The problem is illustrated using unit record data from the Indian National Sample Survey to analyse the determinants of poverty at household level. It is found that these biases are large even with large number of observations because in the limited dependent variable case, the bias does not vanish asymptotically when the assumption of homoscedasticity breaks down. Both regression coefficients and marginal effects differ widely between probit and hetprobit models in this study.

Constructing Bayesian formulations of sparse kernel learning methods

We present here a simple technique that simplifies the construction of Bayesian treatments of a variety of sparse kernel learning algorithms. An incomplete Cholesky factorisation is employed to modify the dual parameter space, such that the Gaussian prior over the dual model parameters is whitened. The regularisation term then corresponds to the usual weight-decay regulariser, allowing the Bayesian analysis to proceed via the evidence framework of MacKay. There is in addition a useful by-product associated with the incomplete Cholesky factorisation algorithm, it also identifies a subset of the training data forming an approximate basis for the entire dataset in the kernel-induced feature space, resulting in a sparse model. Bayesian treatments of the kernel ridge regression (KRR) algorithm, with both constant and heteroscedastic (input dependent) variance structures, and kernel logistic regression (KLR) are provided as illustrative examples of the proposed method, which we hope will be more widely applicable.

Weighted empirical likelihood inference

A weighted empirical likelihood approach is proposed to take account of the heteroscedastic structure of the data. The resulting weighted empirical likelihood ratio statistic is shown to have a limiting chisquare distribution. A limited simulation study shows that the associated confidence intervals for a population mean or a regression coefficient have more accurate coverage probabilities and more balanced two-sided tail errors when the sample size is small or moderate. The proposed weighted empirical likelihood method also provides more efficient point estimators for a population mean in the presence of side information. Large sample resemblances between the weighted and the unweighted empirical likelihood estimators are characterized through high-order asymptotics and small sample discrepancies of these estimators are investigated through simulation. The proposed weighted approach reduces to the usual unweighted empirical likelihood method under a homogeneous variance structure.

A note on optimal designs in weighted polynomial regression for the classical efficiency functions

In this note we consider the D-optimal design problem for the heteroscedastic polynomial regression model. Karlin and Studden (Ann. Math. Statist. 37 (1966a) 783) found explicit solutions for three types of efficiency functions. We introduce two “new” functions to model the heteroscedastic structure, for which the D-optimal designs can also be found explicitly. The optimal designs have equal masses at the roots of generalized Bessel- and Jacobi-polynomials with complex parameters. It is also demonstrated that there exist no other efficiency functions such that the supporting polynomial of the D-optimal design satisfies a generalized Rodrigues’ formula.

Maximin designs for exponential growth models and heteroscedastic polynomial models

This paper is concerned with nonsequential optimal designs for a class of nonlinear growth models, which includes the asymptotic regression model. This design problem is intimately related to the problem of finding optimal designs for polynomial regression models with only partially known heteroscedastic structure. In each case, a straightforward application of the usual D­optimality criterion would lead to designs which depend on the unknown underlying parameters. To overcome this undesirable dependence a maximin approach is adopted. The theorem of Perron and Frobenius on primitive matrices plays a crucial role in the analysis.