Biased Standard Errors Research Articles

Collinear Latent Variables in Multilevel Confirmatory Factor Analysis: A Comparison of Maximum Likelihood and Bayesian Estimations.

Because variables may be correlated in the social and behavioral sciences, multicollinearity might be problematic. This study investigates the effect of collinearity manipulated in within and between levels of a two-level confirmatory factor analysis by Monte Carlo simulation. Furthermore, the influence of the size of the intraclass correlation coefficient (ICC) and estimation method; maximum likelihood estimation with robust chi-squares and standard errors and Bayesian estimation, on the convergence rate are investigated. The other variables of interest were rate of inadmissible solutions and the relative parameter and standard error bias on the between level. The results showed that inadmissible solutions were obtained when there was between level collinearity and the estimation method was maximum likelihood. In the within level multicollinearity condition, all of the solutions were admissible but the bias values were higher compared with the between level collinearity condition. Bayesian estimation appeared to be robust in obtaining admissible parameters but the relative bias was higher than for maximum likelihood estimation. Finally, as expected, high ICC produced less biased results compared to medium ICC conditions.

Estimating cross-sectional regressions in event studies with conditional heteroskedasticity and regression designs that have leverage

Purpose– In the finance literature, fitting a cross-sectional regression with (estimated) abnormal returns as the dependent variable and firm-specific variables (e.g. financial ratios) as independent variables has becomede rigueurfor a publishable event study. In the absence of skewness and/or kurtosis the explanatory variable, the regression design does not exhibit leverage – an issue that has been addressed in the econometrics literature on the finite sample properties of heteroskedastic-consistent (HC) standard errors, but not in the finance literature on event studies. The paper aims to discuss this issue.Design/methodology/approach– In this paper, simulations are designed to evaluate the potential bias in the standard error of the regression coefficient when the regression design includes “points of high leverage” (Chesher and Jewitt, 1987) and heteroskedasticity. The empirical distributions of test statistics are tabulated from ordinary least squares, weighted least squares, and HC standard errors.Findings– None of the test statistics examined in these simulations are uniformly robust with regard to conditional heteroskedasticity when the regression includes “points of high leverage.” In some cases the bias can be quite large: an empirical rejection rate as high as 25 percent for a 5 percent nominal significance level. Further, the bias in OLS HC standard errors may be attenuated but not fully corrected with a “wild bootstrap.”Research limitations/implications– If the researcher suspects an event-induced increase in return variances, tests for conditional heteroskedasticity should be conducted and the regressor matrix should be evaluated for observations that exhibit a high degree of leverage.Originality/value– This paper is a modest step toward filling a gap on the finite sample properties of HC standard errors in the event methodology literature.

A Comparison of Diagonal Weighted Least Squares Robust Estimation Techniques for Ordinal Data

This study compared diagonal weighted least squares robust estimation techniques available in 2 popular statistical programs: diagonal weighted least squares (DWLS; LISREL version 8.80) and weighted least squares–mean (WLSM) and weighted least squares—mean and variance adjusted (WLSMV; Mplus version 6.11). A 20-item confirmatory factor analysis was estimated using item-level ordered categorical data. Three different nonnormality conditions were applied to 2- to 7-category data with sample sizes of 200, 400, and 800. Convergence problems were seen with nonnormal data when DWLS was used with few categories. Both DWLS and WLSMV produced accurate parameter estimates; however, bias in standard errors of parameter estimates was extreme for select conditions when nonnormal data were present. The robust estimators generally reported acceptable model–data fit, unless few categories were used with nonnormal data at smaller sample sizes; WLSMV yielded better fit than WLSM for most indices.

Planned missing data designs with small sample sizes: How small is too small?

Utilizing planned missing data (PMD) designs (ex. 3-form surveys) enables researchers to ask participants fewer questions during the data collection process. An important question, however, is just how few participants are needed to effectively employ planned missing data designs in research studies. This article explores this question by using simulated three-form planned missing data to assess analytic model convergence, parameter estimate bias, standard error bias, mean squared error (MSE), and relative efficiency (RE).Three models were examined: a one-time-point, cross-sectional model with 3 constructs; a two-time-point model with 3 constructs at each time point; and a three-time-point, mediation model with 3 constructs over three time points. Both full-information maximum likelihood (FIML) and multiple imputation (MI) were used to handle the missing data. Models were found to meet convergence rate and acceptable bias criteria with FIML at smaller sample sizes than with MI.

Statistische Methoden der kleinräumigen Versorgungsforschung

Accurate modeling of spatial dependencies between observations is a significant challenge in research on regional health-care services. This article provides insight into current methods of modeling relationships in regional health-care service research, with consideration of spatial dependencies. Spatial dependencies may be triggered by spillover effects between neighboring regions and spatially distributed differences in - e.g., morbidity - which are not observable. If not considered in the model, the results of the analyses may be biased. Spatial dependencies can be added to the regression model as a spatial lag or a spatial error term. Using an example study, we illustrate that failing to consider spatial autocorrelation may lead to biased coefficients and/or standard errors. Research on regional health-care services should, therefore, if possible, test for spatial autocorrelation in the data and adjust the model accordingly.

Planned missing designs to optimize the efficiency of latent growth parameter estimates

We examine the performance of planned missing (PM) designs for correlated latent growth curve models. Using simulated data from a model where latent growth curves are fitted to two constructs over five time points, we apply three kinds of planned missingness. The first is item-level planned missingness using a three-form design at each wave such that 25% of data are missing. The second is wave-level planned missingness such that each participant is missing up to two waves of data. The third combines both forms of missingness. We find that three-form missingness results in high convergence rates, little parameter estimate or standard error bias, and high efficiency relative to the complete data design for almost all parameter types. In contrast, wave missingness and the combined design result in dramatically lowered efficiency for parameters measuring individual variability in rates of change (e.g., latent slope variances and covariances), and bias in both estimates and standard errors for these same parameters. We conclude that wave missingness should not be used except with large effect sizes and very large samples.

Relative Performance of Categorical Diagonally Weighted Least Squares and Robust Maximum Likelihood Estimation

Robust maximum likelihood (ML) and categorical diagonally weighted least squares (cat-DWLS) estimation have both been proposed for use with categorized and nonnormally distributed data. This study compares results from the 2 methods in terms of parameter estimate and standard error bias, power, and Type I error control, with unadjusted ML and WLS estimation methods included for purposes of comparison. Conditions manipulated include model misspecification, level of asymmetry, level and categorization, sample size, and type and size of the model. Results indicate that cat-DWLS estimation method results in the least parameter estimate and standard error bias under the majority of conditions studied. Cat-DWLS parameter estimates and standard errors were generally the least affected by model misspecification of the estimation methods studied. Robust ML also performed well, yielding relatively unbiased parameter estimates and standard errors. However, both cat-DWLS and robust ML resulted in low power under conditions of high data asymmetry, small sample sizes, and mild model misspecification. For more optimal conditions, power for these estimators was adequate.

Review of Sample Size for Structural Equation Models in Second Language Testing and Learning Research: A Monte Carlo Approach

The importance of sample size, although widely discussed in the literature on structural equation modeling (SEM), has not been widely recognized among applied SEM researchers. To narrow this gap, we focus on second language testing and learning studies and examine the following: (a) Is the sample size sufficient in terms of precision and power of parameters in a model using Monte Carlo analysis? (b) How are the results from Monte Carlo sample size analysis comparable with those from the N ≥ 100 rule and from the N: q ≥ 10 (sample size–free parameter ratio) rule? Regarding (a), parameter bias, standard error bias, coverage, and power were overall satisfactory, suggesting that sample size for SEM models in second language testing and learning studies is generally appropriate. Regarding (b), both rules were often inconsistent with the Monte Carlo analysis, suggesting that they do not serve as guidelines for sample size. We encourage applied SEM researchers to perform Monte Carlo analyses to estimate the requisite sample size of a model.

Sample Size Limits for Estimating Upper Level Mediation Models Using Multilevel SEM

This simulation study investigated use of the multilevel structural equation model (MLSEM) for handling measurement error in both mediator and outcome variables (M and Y) in an upper level multilevel mediation model. Mediation and outcome variable indicators were generated with measurement error. Parameter and standard error bias, confidence interval coverage, and power to detect the ab mediated effect using Empirical-M confidence interval estimates were assessed for the correct MLSEM versus a conventional multilevel model (MM) that used composite scores for M and Y. The following conditions were manipulated: level 1 and 2 sample sizes, intraclass correlation, degree of measurement error in M, and the true value of ab. The MLSEM more accurately recovered the ab effect's value, but serious convergence issues were encountered with MLSEM estimates based on fewer than 80 clusters. More power for detecting a nonzero ab was found for MM than for MLSEM estimates.

A Logistic -Moment-Based Analog for the Tukey -, , , and - System of Distributions

This paper introduces a standard logisticL-moment-based system of distributions. The proposed system is an analog to the standard normal conventional moment-based Tukeyg-h,g,h, andh-hsystem of distributions. The system also consists of four classes of distributions and is referred to as (i) asymmetric -, (ii) log-logistic , (iii) symmetric , and (iv) asymmetric -. The system can be used in a variety of settings such as simulation or modeling events—most notably when heavy-tailed distributions are of interest. A procedure is also described for simulating -, , , and - distributions with specifiedL-moments andL-correlations. The Monte Carlo results presented in this study indicate that estimates ofL-skew,L-kurtosis, andL-correlation associated with the -, , , and - distributions are substantially superior to their corresponding conventional product-moment estimators in terms of relative bias and relative standard error.

Applying Directed Acyclic Graphs to Assist Specification of a Hedonic Model

This research empirically tests the hypothesis that utilizing directed acyclic graphs (DAGs) as an ex-ante process to select variables for a hedonic model improves the model's performance. The results for both new and existing house submarkets indicated that DAG analysis mitigated the multicollinearity issue commonly observed in hedonic models. Using DAG analysis also improved the goodness-of-fit of the hedonic model for the new submarket. However, model specification through DAG analysis does not offer clear implications for improving forecasting accuracy, efficiency, and spatial error autocorrelation. The findings imply that DAG analysis for model specification can be a complementary step in the process of estimating hedonic models, especially when reducing standard error bias by alleviating potential multicollinearity is important in determining the attributes that affect housing prices.

Properties of Estimators of the Process Capability Index

The capability index is designed for an in-control process with asymmetric tolerances. In this article, the density function of the natural estimator is derived. The expression is in terms of finite sum and only involves standard normal distribution. Four ensemble estimators for are proposed. A simulation study is conducted due to the complexity of the mean-squared error of the estimators. The comparisons among estimators are based on the mean-squared error, standard error and absolute bias of estimators. Furthermore, the ability for an estimator to reach a given capability level is also considered.

Strategies for handling missing data in randomised trials

Missing outcome data in randomised trials are a major potential source of bias in trial results, and their correct handling can be a major source of difficulty for investigators [1]. Missing outcome data matter for three main reasons: 1. They lead to a loss of power. This cannot be reversed, and so all efforts should be made to maximise completeness of follow-up. 2. Any analysis of incomplete data makes untestable assumptions and is likely to be biased if those assumptions do not hold. Because the correct assumption is usually unknown, it is important to justify the assumption used from subject-matter knowledge, and to perform sensitivity analyses. 3. Some popular analysis methods give inefficient estimates and/or biased standard errors, even if their assumptions are correct. This talk will outline some of the statistical methods available for handling missing data in randomised trials, and what their underlying assumptions are. Some methods are simple to implement but harder to support: for example, Last Observation Carried Forward is based on an assumption that is rarely justified, while analysis of complete cases can be statistically inefficient. More complex methods such as mixed models and multiple imputation are usually based around a missing at random assumption, which is popular because it is perceived to become more plausible as models become more complex. The intention-to-treat principle is sometimes seen as requiring missing values to be imputed. By focussing instead on assumptions, I will show that it is sufficient for all observed data to be included in the main analysis, but that sensitivity analyses must take account of all randomised individuals. This approach will be summarised in an intention-to-treat analysis strategy [2]. I will also briefly describe how to perform a sensitivity analysis and how to handle missing baseline variables.

Weighting Large Datasets with Complex Sampling Designs: Choosing the Appropriate Variance Estimation Method

Using the Canadian Workplace and Employee Survey (WES), three variance estimation methods for weighting large datasets with complex sampling designs are compared: simple final weighting, standard bootstrapping and mean bootstrapping. Using a logit analysis, it is shown - depending on which weighting method is used - different predictor variables are significant. The potential lack of independence inherent in a multi-stage cluster sample design, as in the WES, results in a downward bias in the variance when conducting statistical inference (using the simple final weight), which in turn results in increased Type I errors. Bootstrap methods can account for the survey’s design and adjust the variance so that it is inference appropriate and corrected for downward bias. The WES provides mean, as opposed to standard, bootstrap weights with the data; thus, a further adjustment to account for the reduced variation inherent when information is grouped is required. Failure to use mean bootstrap weights appropriately leads to biased standard errors and inappropriate inference.

The Impact of Ignoring a Level of Nesting Structure in Multilevel Growth Mixture Models: A Monte Carlo Study

Growth mixture modeling (GMM) is a relatively new technique for analyzing longitudinal data. However, when applying GMM, researchers might assume that the higher level (nonrepeated measure) units (e.g., students) are independent from each other even though it might not always be true. This article reports the results of a simulation study examining the impact of ignoring a higher level nesting structure in multilevel GMM. Three-level longitudinal clustered data were generated and then analyzed with the correct 3-level model and the incorrect model that ignored the highest level of nesting structure separately. The simulation results showed that although over 90% of the replications resulted in the correct class solution under both true and misspecified models, ignoring a higher level nesting structure could still result in lower classification accuracy, overestimated lower level variance components, and biased standard errors. The biased standard errors in turn affect the tests of significance for the fixed effects.

Partial misspecification of survey design features sufficed to severely bias estimates of health-related outcomes

Objective Surveys frequently deviate from simple random sampling through the use of unequal probability sampling, stratified sampling, and multistage sampling. This work uses a survey of public health to systematically illustrate the effects of incompletely accounting for strata, clustering, and weights. Study Design and Setting Data analysis was based on the Study of Health in Pomerania ( n = 4,308, 20–79 years), a two-stage regional survey with high sampling fractions at the first stage. Effects of survey design features comprising weights, stratification, clustering, and finite population correction on point and variance estimates of lifestyle indicators and clinical parameters were assessed. Results Misspecifications of the survey design substantially affected both the point estimates of health characteristics and their standard errors (SEs). The strongest bias in SEs concerned the omission of the second sampling stage. Ignoring the sampling design led to minor differences in variance estimates from the complete setup. Weighting predominantly affected point estimates of lifestyle factors. Conclusion A partial misspecification of survey design elements may bias variance estimates severely and is sometimes even more harmful compared with completely neglecting design elements. If subgroups are sampled at different rates, weighting is of particular relevance with regard to prevalence estimates of lifestyle indicators.

Maximum likelihood estimation of regression parameters with spatially dependent discrete data

Generalized estimating equations (GEEs) have been successfully used to estimate regression parameters from discrete longitudinal data. GEEs have been adapted for spatially correlated count data with less success. It is convenient to model correlated counts as lognormal-Poisson, where a latent lognormal random process carries all correlation. This model limits correlation and can lead to negative bias of standard errors. Moreover, correlation is not the best dependence measure for highly nonnormal data. This article proposes a model which yields maximum likelihood (ML) estimates of regression parameters when the response is discrete and spatially dependent. This model employs a spatial Gaussian copula, bringing the discrete distribution into the Gaussian geostatistical framework, where correlation completely describes dependence. The model yields a log-likelihood for regression parameters that can be maximized using established numerical methods. The proposed procedure is used to estimate the relationship between Japanese beetle grub counts and soil organic matter. These data exhibit residual correlation well above the lognormal-Poisson correlation limit, so that model is not appropriate. The data and MATLAB code are available online. Simulations demonstrate that negative bias in GEE standard errors leads to nominal 95% confidence coverage less than 62% for moderate or strong spatial dependence, whereas ML coverage remains above 82%.

Bias of Standard Errors in Latent Class Model Applications Using Newton-Raphson and EM Algorithms

The EM algorithm is a popular method for computing maximum likelihood estimates. It tends to be numerically stable, reduces execution time compared to other estimation procedures and is easy to implement in latent class models. However, the EM algorithm fails to provide a consistent estimator of the standard errors of maximum likelihood estimates in incomplete data applications. Correct standard errors can be obtained by numerical differentiation. The technique requires computation of a complete-data gradient vector and Hessian matrix, but not those associated with the incomplete data likelihood. Obtaining first and second derivatives numerically is computationally very intensive and execution time may become very expensive when fitting Latent class models using a Newton-type algorithm. When the execution time is too high one is motivated to use the EM algorithm solution to initialize the Newton Raphson algorithm. We also investigate the effect on the execution time when a final Newton-Raphson step follows the EM algorithm after convergence. In this paper we compare the standard errors provided by the EM and Newton-Raphson algorithms for two models and analyze how this bias is affected by the number of parameters in the model fit.

Estimation of seal pup production from aerial surveys using generalized additive models

In this paper, we estimate the pup production of harp seals ( Pagophilus groenlandicus ) using generalized additive models (GAMs) based on thin-plate regression splines. The spatial distribution of seal pups in a patch is modelled using GAMs, and the pup production is estimated by numerically integrating the model over a fine grid area of the patch. Closed form expression for estimation of the the standard error of the pup production estimate is derived. The estimators are applied to simulated seal populations to investigate their properties. The results show that the proposed pup production estimator is comparable with the conventional pup production estimator. However, the bias of the standard error estimator of the proposed method is much lower than the bias of the conventional standard error estimator. The decrease of standard error bias results in a considerable reduction of the coefficient of variation estimate using the proposed GAM-based method. The proposed method is also applied to real survey data of harp seals obtained from aerial surveys in the Greenland Sea pack ice in 2002. We show that the number of pups counted from aerial photographs possess a good fit to the negative binomial distribution when a logarithmic link function is applied. The approach described here is applicable to many situations where georeferenced counts or measurements are available.

A Monte Carlo sample size study: How many countries are needed for accurate multilevel SEM?

Recently, there has been growing scientific interest for cross-national survey research. Various scholars have used multilevel techniques to link individual characteristics to aspects of the national context. At first sight, multilevel SEM seems to be a promising tool for this purpose, as it integrates multilevel modeling within a latent variable framework. However, due to the fact that the number of countries in most international surveys does not exceed 30, the application of multilevel SEM in cross-national research is problematic. Taking European Social Survey (ESS) data as a point of departure, this paper uses Monte Carlo studies to assess the estimation accuracy of multilevel SEM with small group sample sizes. The results indicate that a group sample size of 20 ‐ a situation common in cross-national research ‐ does not guarantee accurate estimation at all. Unacceptable amounts of parameter and standard error bias are present for the between-level estimates. Unless the standardized e ect is very large (0.75), statistical power for detecting a significant between-level structural e ect is seriously lacking. Required group sample sizes depend strongly on the specific interests of the researcher, the expected e ect sizes and the complexity of the model. If the between-level model is relatively simple and one is merely interested in the between-level factor structure, a group sample size of 40 could be su cient. To detect large (>0.50) structural e ects at the between level, at least 60 groups are required. To have an acceptable probability of detecting smaller e ects, more than 100 groups are needed. These guidelines are shown to be quite robust for varying cluster sizes and intra-class correlations (ICCs).