Underestimation Of Standard Errors Research Articles

Interviewer Effects in the European Social Survey

In this paper, we focus on interviewer effects in the European Social Survey, and seek to demonstrate that academic publications seldom take these effects into consideration. An analysis is provided of interviewer effects for 48 continuous items, covering 36 countries in six rounds. The analysis does not only deal with the means of variables. Using multilevel covariance structure analysis, interviewer effects on the relationships between these variables is also assessed. Results indicate that first, countries showing considerable interviewer effects regarding means are also more at risk regarding regression coefficients and second, ignoring interviewer effects leads to an overestimation of the effect size of the relationships between variables and an underestimation of standard errors.

Analysis of occupant injury severity in winter weather crashes: A fully Bayesian multivariate approach

Multivariate injury severity models that consider the cross-group heterogeneity in the crash data where individuals or occupants are nested within vehicles and vehicles are nested within crashes are limited in the literature. Most previous studies on crash injury severity were conducted at the crash level ignoring the potential correlation in severity for the vehicles involved in the same crashes or individuals involved in the same vehicles. Ignoring these correlation and dependence effects might result in underestimation of standard errors and erroneous inferences. The objective of this paper is to correctly determine the factors affecting occupant injury severity in winter seasons by addressing the within-crash and between-crash correlation of injury severity. To achieve this, fully Bayesian hierarchical multinomial logit models were developed for estimating occupant injury severity in weather-related crashes, non weather-related crashes, and all crashes. These models were developed using disaggregate crash data with occupants nested within crashes for four winter seasons in Iowa. Significant factors affecting occupant injury severity included factors related to occupants (gender, seating position, occupant trapped status, ejection status, and occupant protection used), as well as crash-level factors (road junction type, first harmful event and major cause of crash). Weather-related variables, such as visibility, pavement and air temperature, were also significant factors in winter weather crashes. Interaction effects involving crash-level variables and occupant-level variables were also found significant. Overall, the model diagnostics suggested significant within-crash correlation in the study dataset justifying the use of a multivariate model specification that addresses multivariate error term correlation issues.

Comparing interval estimates for small sample ordinal CFA models.

Robust maximum likelihood (RML) and asymptotically generalized least squares (AGLS) methods have been recommended for fitting ordinal structural equation models. Studies show that some of these methods underestimate standard errors. However, these studies have not investigated the coverage and bias of interval estimates. An estimate with a reasonable standard error could still be severely biased. This can only be known by systematically investigating the interval estimates. The present study compares Bayesian, RML, and AGLS interval estimates of factor correlations in ordinal confirmatory factor analysis models (CFA) for small sample data. Six sample sizes, 3 factor correlations, and 2 factor score distributions (multivariate normal and multivariate mildly skewed) were studied. Two Bayesian prior specifications, informative and relatively less informative were studied. Undercoverage of confidence intervals and underestimation of standard errors was common in non-Bayesian methods. Underestimated standard errors may lead to inflated Type-I error rates. Non-Bayesian intervals were more positive biased than negatively biased, that is, most intervals that did not contain the true value were greater than the true value. Some non-Bayesian methods had non-converging and inadmissible solutions for small samples and non-normal data. Bayesian empirical standard error estimates for informative and relatively less informative priors were closer to the average standard errors of the estimates. The coverage of Bayesian credibility intervals was closer to what was expected with overcoverage in a few cases. Although some Bayesian credibility intervals were wider, they reflected the nature of statistical uncertainty that comes with the data (e.g., small sample). Bayesian point estimates were also more accurate than non-Bayesian estimates. The results illustrate the importance of analyzing coverage and bias of interval estimates, and how ignoring interval estimates can be misleading. Therefore, editors and policymakers should continue to emphasize the inclusion of interval estimates in research.

A Simulation Study on the Performances of Classical Var and Sims-Zha Bayesian Var Models in the Presence of Autocorrelated Errors

It is well known that a high degree of positive dependency among the errors generally leads to 1) serious underestimation of standard errors for regression coefficients; 2) prediction intervals that are excessively wide. This paper set out to study the performances of classical VAR and Sims-Zha Bayesian VAR models in the presence of autocorrelated errors. Autocorrelation levels of (-0.99, -0.95, -0.9, -0.85, -0.8, 0.8, 0.85, 0.9, 0.95, 0.99) were considered for short term (T = 8, 16); medium term (T = 32, 64) and long term (T = 128, 256). The results from 10,000 simulation revealed that BVAR model with loose prior is suitable for negative autocorrelations and BVAR model with tight prior is suitable for positive autocorrelations in the short term. While for medium term, the BVAR model with loose prior is suitable for the autocorrelation levels considered except in few cases. Lastly, for long term, the classical VAR is suitable for all the autocorrelation levels considered except in some cases where the BVAR models are preferred. This work therefore concludes that the performance of the classical VAR and Sims-Zha Bayesian VAR varies in terms of the autocorrelation levels and the time series lengths.

A reply

We are grateful to Dr Butwick for spotting the error in our article 1, for which we apologise; we have corrected the bias and graphs accordingly in an Erratum 2. We also thank Dr Butwick for raising an important point about treating separate measurements on the same subjects as independent, in our Bland-Altman plots. There is no simple answer. Normally, not treating paired observations as dependent would lead to serious under-estimation of standard errors and misleading significance tests. However, significance testing is not used in the Bland-Altman method. Bland and Altman themselves point out that their approach ‘is based on the philosophy that the key to method comparison studies is to quantify disagreements between individual measurements. It follows that we do not see a place for methods of analysis based on hypothesis testing’ 3. Myles and Cui 4 suggest various methods of analysis for multiple measurements producing varying results, presumably with the purpose of avoiding estimated limits of agreement that appear more accurate than is actually the case. We feel that in this context, not accounting for dependency can be defended. We are not seeking to estimate the average difference or trend as accurately as possible while minimising the standard error. Instead, we are interested in describing the differences between the observations as accurately as possible. As such we are assured by the fact that Butwick and colleagues' findings are in accord with ours: there is a clinically unacceptable variation of the values obtained by the Masimo device, which renders it less useful in clinical practice.

Power of Tests for Overdispersion Parameter in Negative Binomial Regression Mode

In this paper we focus on a negative binomial (NB) regression model to take account of overdispersion in Poisson counts. Moreover, we present the power of score test for testing the overdispersion parameter in the negative binomial regression model. The power of the proposed score test was compared with the LRT and Wald test via Monte Carlo simulation technique using SAS 9.2 software. The application of the test was shown using two real datasets such as using numerical illustration and real datasets. Keywords- Count data, Negative binomial regression, Overdispersion, Score test I. INTRODUCTION Poisson regression is one of the most popular techniques for the analysis of count data. Whereas the Poisson regression model may be the foremost candidate, it rarely explains the data due to several important constraints. One important constraint is the mean of the distribution must be equal to the variance. In this case the standard errors, usually estimated by the Maximum Likelihood method, will be biased and the test statistics derived from the models will be incorrect. Therefore, this problem leads an overdispersion. Failure to take overdispersion into account leads to serious underestimation of standard errors and misleading inference for the regression parameters. Consequently, a number of models and associated estimation methods have been proposed for handling overdispersed data. Such models include those based on the negative binomial distributions as well as regression models based on mixtures of Poisson (Lawless

The Impact of Territorially Concentrated FDI on Local Labor Markets: Evidence from the Czech Republic

This paper investigates the impact of a large territorially concentrated FDI inflow on local labor market outcomes using district panel data from the Czech Republic. Toyota-Peugeot's joint investment in Kolin is used to quantify the effect of FDI on the district unemployment outflow and inflow rates, the aggregate unemployment exit hazard rates, and subsequently both the unemployment rate and the employment rate. Using difference-in-differences analysis, labor market performance of 'treatment' and 'control' districts for two periods (before and after the investment) are compared. Placebo simulations reveal that conventional least squares estimates lead to serious underestimation of standard errors. Therefore, in order to account for serial correlation, the block bootstrapping technique is used to compute consistent standard errors. The results indicate a positive significant impact of the investment on the local unemployment outflow rate driven mainly by increases in the aggregate unemployment hazard rates for durations less than nine months. However, the impact on longer unemployment durations remained negligible. Consequently, the local unemployment rate decreased and the employment rate increased in the 'treated' district.

Multilevel Modeling in Traffic Safety Research: Two Empirical Examples Illustrating the Consequences of Ignoring Hierarchies

Objectives: This commentary illustrates the advantages of multilevel modeling compared to statistical techniques that ignore hierarchies, based on two empirical traffic safety examples. Methods: The common concept shared by different definitions of multilevel modeling is identified and illustrated. Each definition defines multilevel modeling in its own way but they all refer to hierarchies. Conceptual issues inherently related to hierarchies are then pointed out and illustrated. Results: Broadly speaking there are two important consequences of ignoring a hierarchical structure in the data. The first consequence, underestimation of standard errors, is illustrated with data from an observational study on seatbelt behavior. Two effects that were significant at the 5% level in a single-level model were no longer found to be significant in a two-level model. The single-level model is therefore bound to lead to erroneous conclusions regarding variables that could have an impact on seatbelt use and, ultimately, on increasing the level of traffic safety. The second consequence, related to contextual information, is illustrated with data from a roadside survey on drink driving. Of particular interest is the relationship between Traffic Count, an aggregated level 2 explanatory variable and Odds of drink driving, an individual level 1 dependent variable. Conclusions: Like every statistical technique, multilevel models should be used with caution and reservation. However, given certain limitations, multilevel modeling is very useful and valuable to traffic safety research.

Underestimation of standard errors in multi-site time series studies.

Multi-site time series studies of the association of air pollution with mortality and morbidity have figured prominently in the literature as comprehensive approaches for estimating short-term effects of air pollution on health. Hierarchical models are generally used to combine site-specific information and to estimate pooled air pollution effects while taking into account both within-site statistical uncertainty and across-site heterogeneity. Within a site, characteristics of time series data of air pollution and health (small pollution effects, missing data, and highly correlated predictors) make the modeling of all sources of uncertainty challenging. One potential consequence is underestimation of the statistical variance of the site-specific effects to be combined.In this paper, we investigate the impact of variance underestimation on the pooled relative rate estimate. We focused on two-stage normal-normal hierarchical models and on underestimation of the statistical variance at the first stage. By mathematical considerations and simulation studies, we found that variance underestimation did not affect the pooled estimate substantially. However, the pooled estimate was somewhat sensitive to variance underestimation when the number of sites was small and underestimation was severe. These simulation results are applicable to any two-stage normal-normal hierarchical model for combining information of site-specific results (including meta-analyses), and they can easily be extended to more general hierarchical formulations. We also examined the impact of variance underestimation on the national average relative rate estimate from the National Morbidity, Mortality and Air Pollution Study. We found that variance underestimation as large as 40% had little effect on the national average.

Empirical Bayes and Item-Clustering Effects in a Latent Variable Hierarchical Model

Empirical Bayes regression procedures are often used in educational and psychological testing as extensions to latent variables models. The National Assessment of Educational Progress (NAEP) is an important national survey using such procedures. The NAEP applies empirical Bayes methods to models from item response theory to calibrate student responses to questions of varying difficulty. Due partially to the limited computing technology that existed when NAEP was first conceived, NAEP analyses are carried out using a two-stage estimation procedure that ignores uncertainty about some model parameters. Furthermore, the item response theory model that NAEP uses ignores the effect of item clustering created by the design of a test form. Using Markov chain Monte Carlo, we simultaneously estimate all parameters of an expanded model that considers item clustering to investigate the impact of item clustering and ignoring uncertainty about model parameters on an important outcome measure that NAEP report. Ignoring these two effects causes substantial underestimation of standard errors and induces a modest bias in location estimates.