Biased Standard Errors Research Articles

Hierarchical Nearest Neighbor Gaussian Process models for discrete choice: Mode choice in New York City

Standard Discrete Choice Models (DCMs) assume that unobserved effects that influence decision-making are independently and identically distributed among individuals. When unobserved effects are spatially correlated, the independence assumption does not hold, leading to biased standard errors and potentially biased parameter estimates. This paper proposes an interpretable Hierarchical Nearest Neighbor Gaussian Process (HNNGP) model to account for spatially correlated unobservables in discrete choice analysis. Gaussian Processes (GPs) are often regarded as lacking interpretability due to their non-parametric nature. However, we demonstrate how to incorporate GPs directly into the latent utility specification to flexibly model spatially correlated unobserved effects without sacrificing structural economic interpretation. To empirically test our proposed HNNGP models, we analyze binary and multinomial mode choices for commuting to work in New York City. For the multinomial case, we formulate and estimate HNNGPs with and without independence from irrelevant alternatives (IIA). Building on the interpretability of our modeling strategy, we provide both point estimates and credible intervals for the value of travel time savings in NYC. Finally, we compare the results from all proposed specifications with those derived from a standard logit model and a probit model with spatially autocorrelated errors (SAE) to showcase how accounting for different sources of spatial correlation in discrete choice can significantly impact inference. We also show that the HNNGP models attain better out-of-sample prediction performance when compared to the logit and probit SAE models, especially in the multinomial case.

Small sample adjustment for inference without assuming orthogonality in a mixed model for repeated measures analysis

ABSTRACT The mixed model for repeated measures (MMRM) analysis is sometimes used as a primary statistical analysis for a longitudinal randomized clinical trial. When the MMRM analysis is implemented in ordinary statistical software, the standard error of the treatment effect is estimated by assuming orthogonality between the fixed effects and covariance parameters, based on the characteristics of the normal distribution. However, orthogonality does not hold unless the normality assumption of the error distribution holds, and/or the missing data are derived from the missing completely at random structure. Therefore, assuming orthogonality in the MMRM analysis is not preferable. However, without the assumption of orthogonality, the small-sample bias in the standard error of the treatment effect is significant. Nonetheless, there is no method to improve small-sample performance. Furthermore, there is no software that can easily implement inferences on treatment effects without assuming orthogonality. Hence, we propose two small-sample adjustment methods inflating standard errors that are reasonable in ideal situations and achieve empirical conservatism even in general situations. We also provide an R package to implement these inference processes. The simulation results show that one of the proposed small-sample adjustment methods performs particularly well in terms of underestimation bias of standard errors; consequently, the proposed method is recommended. When using the MMRM analysis, our proposed method is recommended if the sample size is not large and between-group heteroscedasticity is expected.

Large-ensemble Monte Carlo: a researcher’s guide to better climate trend uncertainties

Abstract Internal climate variability (ICV) often violates the assumptions of statistical methods, and the climate research community does not have an established approach for addressing resulting biases. Here we argue for a technique we call climate model Large-Ensemble Monte-Carlo (LENS-MC) to inform the selection of statistical methods for real-world application. Until now, scientists have often made best efforts to select methods based on assumptions about the mathematical properties of ICV. LENS-MC relaxes these assumptions and justifies method selection, potentially for a wide range of statistical analyses. We demonstrate LENS-MC using a case study of statistical errors in 20 year trends in global temperature and top-of-atmosphere flux series, comparing results with standard ordinary least squares (OLS). OLS commonly underestimates trend uncertainties, resulting in a higher likelihood of falsely reporting statistically significant trends or changes in trends, for example reporting p < 0.05 in 20 year temperature trends when the statistics are actually equivalent to p < 0.56. LENS-MC tests result in the selection of methods that almost eliminate the low bias in OLS trend standard errors. Using the suggested methods, researchers are less likely to mistakenly report significant trends, and LENS-MC could be widely applied to statistical climate analysis for which model output is available, provided that model ICV displays similar statistical structure, such as in autocorrelation, to observed ICV.

Impact of missing data and ICC on full information maximum-likelihood estimation in multilevel SEMs

A Monte Carlo simulation study was conducted to investigate the performance of full information maximum-likelihood (FIML) estimator in multilevel structural equation modeling (SEM) with missing data and different intra-class correlations (ICCs) coefficients. The study simulated the influence of two independent variables (missing data patterns, and ICC coefficients) in multilevel SEM on five outcome measures (model rejection rates, parameter estimate bias, standard error bias, coverage, and power). Results indicated that FIML parameter estimates were generally robust for data missing on outcomes and/or higher-level predictor variables under the data completely at random (MCAR) and for data missing at random (MAR). However, FIML estimation yielded substantially lower parameter and standard error bias when data was not missing on higher-level variables, and in high rather than in low ICC conditions (0.50 vs 0.20). Future research should extend to further examination of the impacts of data distribution, complexity of the between-level model, and missingness on the between-level variables on FIML estimation performance.

Understanding the Consequences of Collinearity for Multilevel Models: The Importance of Disaggregation Across Levels

In multilevel models, disaggregating predictors into level-specific parts (typically accomplished via centering) benefits parameter estimates and their interpretations. However, the importance of level-specificity has been sparsely addressed in multilevel literature concerning collinearity. In this study, we develop novel insights into the interactivity of centering and collinearity in multilevel models. After integrating the broad literatures on centering and collinearity, we review level-specific and conflated correlations in multilevel data. Next, by deriving formal relationships between predictor collinearity and multilevel model estimates, we demonstrate how the consequences of collinearity change across different centering specifications and identify data characteristics that may exacerbate or mitigate those consequences. We show that when all or some level-1 predictors are uncentered, slope estimates can be greatly biased by collinearity. Disaggregation of all predictors eliminates the possibility that fixed effect estimates will be biased due to collinearity alone; however, under some data conditions, collinearity is associated with biased standard errors and random effect (co)variance estimates. Finally, we illustrate the importance of disaggregation for diagnosing collinearity in multilevel data and provide recommendations for the use of level-specific collinearity diagnostics. Overall, the necessity of disaggregation for identifying and managing collinearity’s consequences in multilevel models is clarified in novel ways.

Application of the professional maturity scale as a computerized adaptive testing

This study has two main purposes. Firstly, to compare the different item selection methods and stopping rules used in Computerized Adaptive Testing (CAT) applications with simulative data generated based on the item parameters of the Vocational Maturity Scale. Secondly, to test the validity of CAT application scores. For the first purpose, simulative data produced based on Vocational Maturity Scale item parameters were analyzed under different item selection methods (Maximum Fisher Information [MFI],Maximum Likelihood Weighted Information [MLWI] Maximum Posterior Weighted Information [MPWI] Maximum Expected Information [MEI] Minimum Expected Posterior Variance [MEPV] Maximum Expected Posterior Weighted Information [MEPWI]) and stopping rules (Standard Error [SE]<0.30, SE<0.50, SE <0.70, Number of Item [NI]=10, NI=20) by calculating the average number of items, standard error averages, correlation coefficients, bias, and RMSE statistics. For all the conditions of the item selection methods, standard error averages, correlation coefficients, bias, and RMSE statistics showed similar results. When the average number of items is considered, MFI and SE<0.30 were found as most appropriate methods to be used in CAT application. For the second purpose of the study, the paper-pencil form of the Vocational Maturity scale and CAT version were administered to 33 students. A moderate, positive, and statistically significant relationship was found between the CAT application scores and the paper-pencil form scores on the vocational maturity scale. As a result, it can be said that the vocational maturity scale can be applied as a computerized adaptive test and can be used in career guidance processes.

Accurate Standard Errors in Multilevel Modeling with Heteroscedasticity: A Computationally More Efficient Jackknife Technique

In random-effects models, hierarchical linear models, or multilevel models, it is typically assumed that the variances within higher-level units are homoscedastic, meaning that they are equal across these units. However, this assumption is often violated in research. Depending on the degree of violation, this can lead to biased standard errors of higher-level parameters and thus to incorrect inferences. In this article, we describe a resampling technique for obtaining standard errors—Zitzmann’s jackknife. We conducted a Monte Carlo simulation study to compare the technique with the commonly used delete-1 jackknife, the robust standard error in Mplus, and a modified version of the commonly used delete-1 jackknife. Findings revealed that the resampling techniques clearly outperformed the robust standard error in rather small samples with high levels of heteroscedasticity. Moreover, Zitzmann’s jackknife tended to perform somewhat better than the two versions of the delete-1 jackknife and was much faster.

Individual Mobility across Clusters: The Impact of Ignoring Cross-Classified Data Structures in Discrete-Time Survival Analysis

A multilevel-discrete time survival model may be appropriate for purely hierarchical data, but when data are non-purely hierarchical due to individual mobility across clusters, a cross-classified discrete time survival model may be necessary. The purpose of this research was to investigate the performance of a cross-classified discrete-time survival model and assess the impact of ignoring a cross-classified data structure on the model parameters of a conventional discrete-time survival model and a multilevel discrete-time survival model. A Monte Carlo simulation was used to examine the performance of three discrete-time survival models when individuals are mobile across clusters. Simulation factors included the value of the between-clusters variance, number of clusters, within-cluster sample size, Weibull scale parameter, and mobility rate. The results suggest that substantial relative parameter bias, unacceptable coverage of the 95% confidence intervals, and severely biased standard errors are possible for all model parameters when a discrete-time survival model is used that ignores the cross-classified data structure. The findings presented in this study are useful for methodologists and practitioners in educational research, public health, and other social sciences where discrete-time survival analysis is a common methodological technique for analyzing event-history data.

Standard Error Biases When Using Generated Regressors in Accounting Research

ABSTRACTWe analyze the standard error bias associated with the use of generated regressors—independent variables generated from first‐step regressions—in accounting research settings. Under general conditions, generated regressors do not affect the consistency of coefficient estimates. However, commonly used generated regressors can cause standard errors to be understated. Problematic generated regressors include predicted values, coefficient estimates, and measures derived from these estimates. Widely used generated regressors in accounting include measures of earnings persistence, normal accruals, litigation risk, and conditional conservatism. Using simple regression models and simulation, we demonstrate how generated regressors can produce understated standard errors in accounting research settings. We also demonstrate how the magnitude of the standard error bias is inversely related to the precision of the generated regressor. Finally, we discuss bootstrapping as a correction for the bias and demonstrate the pairs cluster bootstrap as a tool to improve inferences in common accounting settings involving generated regressors.

Two-stage path analysis with definition variables: An alternative framework to account for measurement error.

When estimating path coefficients among psychological constructs measured with error, structural equation modeling (SEM), which simultaneously estimates the measurement and structural parameters, is generally regarded as the gold standard. In practice, however, researchers usually first compute composite scores or factor scores, and use those as observed variables in a path analysis, for purposes of simplifying the model or avoiding model convergence issues. Whereas recent approaches, such as reliability adjustment methods and factor score regression, has been proposed to mitigate the bias induced by ignoring measurement error in composite/factor scores with continuous indicators, those approaches are not yet applicable to models with categorical indicators. In this article, we introduce the two-stage path analysis (2S-PA) with definition variables as a general framework for path modeling to handle categorical indicators, in which estimation of factor scores and path coefficients are separated. It thus allows for different estimation methods in the measurement and the structural path models and easier diagnoses of violations of model assumptions. We conducted three simulation studies, ranging from latent regression to mediation analysis with categorical indicators, and showed that 2S-PA generally produced similar estimates to those using SEM in large samples, but gave better convergence rates, less standard error bias, and better control of Type I error rates in small samples. We illustrate 2S-PA using data from a national data set, and show how researchers can implement it in Mplus and OpenMx. Possible extensions and future directions of 2S-PA are discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A new approach fits multivariate genomic prediction models efficiently

BackgroundFast, memory-efficient, and reliable algorithms for estimating genomic estimated breeding values (GEBV) for multiple traits and environments are needed to make timely decisions in breeding. Multivariate genomic prediction exploits genetic correlations between traits and environments to increase accuracy of GEBV compared to univariate methods. These genetic correlations are estimated simultaneously with GEBV, because they are specific to year, environment, and management. However, estimating genetic parameters is computationally demanding with restricted maximum likelihood (REML) and Bayesian samplers, and canonical transformations or orthogonalizations cannot be used for unbalanced experimental designs.MethodsWe propose a multivariate randomized Gauss–Seidel algorithm for simultaneous estimation of model effects and genetic parameters. Two previously proposed methods for estimating genetic parameters were combined with a Gauss–Seidel (GS) solver, and were called Tilde-Hat-GS (THGS) and Pseudo-Expectation-GS (PEGS). Balanced and unbalanced experimental designs were simulated to compare runtime, bias and accuracy of GEBV, and bias and standard errors of estimates of heritabilities and genetic correlations of THGS, PEGS, and REML. Models with 10 to 400 response variables, 1279 to 42,034 genetic markers, and 5990 to 1.85 million observations were fitted.ResultsRuntime of PEGS and THGS was a fraction of REML. Accuracies of GEBV were slightly lower than those from REML, but higher than those from the univariate approach, hence THGS and PEGS exploited genetic correlations. For 500 to 600 observations per response variable, biases of estimates of genetic parameters of THGS and PEGS were small, but standard errors of estimates of genetic correlations were higher than for REML. Bias and standard errors decreased as sample size increased. For balanced designs, GEBV and estimates of genetic correlations from THGS were unbiased when only an intercept and eigenvectors of genotype scores were fitted.ConclusionsTHGS and PEGS are fast and memory-efficient algorithms for multivariate genomic prediction for balanced and unbalanced experimental designs. They are scalable for increasing numbers of environments and genetic markers. Accuracy of GEBV was comparable to REML. Estimates of genetic parameters had little bias, but their standard errors were larger than for REML. More studies are needed to evaluate the proposed methods for datasets that contain selection.

Six Methods for Latent Moderation Analysis in Marketing Research: A Comparison and Guidelines

It is common in moderation analysis that at least one of the target moderation variables is latent and measured with measurement error. This article compares six methods for latent moderation analysis: multigroup, means, corrected means, factor scores, product indicators, and latent product. It reviews their use in marketing research, describes their assumptions, and compares their performance with Monte Carlo simulations. Several recommendations follow from the results. First, although the means method is the most frequently used method in the review (95% of articles), it should only be used when reliabilities of the moderation variables are close to 1, which is rare. Then, all methods except the multigroup method perform similarly well. Second, the results support using the factor scores method and latent product method when reliabilities are smaller than 1. These methods perform best with parameter and standard error bias less than or equal to 5% under most investigated conditions. Third, specific settings can warrant using the multigroup method (if the moderator is discrete), the corrected means method (if moderation variables are single indicators), and the product indicators method (if indicators are nonnormally distributed). Practical guidelines and sample code for four statistical platforms (SPSS, Stata, R, and Mplus) are provided.

Evaluating tests for diagnosing COVID-19 in the absence of a reliable reference standard: pitfalls and potential solutions

Evaluating tests for diagnosing COVID-19 in the absence of a reliable reference standard: pitfalls and potential solutions

A unified model-implied instrumental variable approach for structural equation modeling with mixed variables

The model-implied instrumental variable (MIIV) estimator is an equation-by-equation estimator of structural equation models that is more robust to structural misspecifications than full information estimators. Previous studies have concentrated on endogenous variables that are all continuous (MIIV-2SLS) or all ordinal . We develop a unified MIIV approach that applies to a mixture of binary, ordinal, censored, or continuous endogenous observed variables. We include estimates of factor loadings, regression coefficients, variances, and covariances along with their asymptotic standard errors. In addition, we create new goodness of fit tests of the model and overidentification tests of single equations. Our simulation study shows that the proposed MIIV approach is more robust to structural misspecifications than diagonally weighted least squares (DWLS) and that both the goodness of fit model tests and the overidentification equations tests can detect structural misspecifications. We also find that the bias in asymptotic standard errors for the MIIV estimators of factor loadings and regression coefficients are often lower than the DWLS ones, though the differences are small in large samples. Our analysis shows that scaling indicators with low reliability can adversely affect the MIIV estimators. Also, using a small subset of MIIVs reduces small sample bias of coefficient estimates, but can lower the power of overidentification tests of equations.

Comparison of confirmatory factor analysis estimation methods on mixed-format data

Weighted least squares (WLS), weighted least squares mean-and-variance-adjusted (WLSMV), unweighted least squares mean-and-variance-adjusted (ULSMV), maximum likelihood (ML), robust maximum likelihood (MLR) and Bayesian estimation methods were compared in mixed item response type data via Monte Carlo simulation. The percentage of polytomous items, distribution of polytomous items, categories of polytomous items, average factor loading, sample size and test length conditions were manipulated. ULSMV and WLSMV were found to be the more accurate methods under all simulation conditions. All methods except WLS had acceptable relative bias and relative standard error bias. No method gives accurate results with small sample sizes and low factor loading, however, the ULSMV method can be recommended to researchers because it gives more appropriate results in all conditions.

The Impact of Ignoring a Crossed Factor in Cross-Classified Multilevel Modeling.

The present study investigated estimate biases in cross-classified random effect modeling (CCREM) and hierarchical linear modeling (HLM) when ignoring a crossed factor in CCREM considering the impact of the feeder and the magnitude of coefficients. There were six simulation factors: the magnitude of coefficient, the correlation between the level 2 residuals, the number of groups, the average number of individuals sampled from each group, the intra-unit correlation coefficient, and the number of feeders. The targeted interests of the coefficients were four fixed effects and two random effects. The results showed that ignoring a crossed factor in cross-classified data causes a parameter bias for the random effects of level 2 predictors and a standard error bias for the fixed effects of intercepts, level 1 predictors, and level 2 predictors. Bayesian information criteria generally outperformed Akaike information criteria in detecting the correct model.

Bias in Local Projections

<ns3:p>Local projections (LPs) are a popular tool in macroeconomic research. We show that LPs are often used with very small samples in the time dimension. Consequently, LP point estimates can be severely biased. We derive simple expressions for this bias and propose a way to bias-correct LPs. Small sample bias can also lead autocorrelation-robust standard errors to dramatically understate sampling uncertainty. We argue they should be avoided in LPs like the ones we study. Using identified monetary policy shocks, we demonstrate that the bias in point estimates can be economically meaningful and the bias in standard errors can affect inference.</ns3:p>

Evaluating Methods for Dealing with Missing Outcomes in Discrete-Time Event History Analysis: A Simulation Study

Background: In discrete-time event history analysis, subjects are measured once each time period until they experience the event, prematurely drop out, or when the study concludes. This implies measuring event status of a subject in each time period determines whether (s)he should be measured in subsequent time periods. For that reason, intermittent missing event status causes a problem because, unlike other repeated measurement designs, it does not make sense to simply ignore the corresponding missing event status from the analysis (as long as the dropout is ignorable). Method: We used Monte Carlo simulation to evaluate and compare various alternatives, including event occurrence recall, event (non-)occurrence, case deletion, period deletion, and single and multiple imputation methods, to deal with missing event status. Moreover, we showed the methods’ performance in the analysis of an empirical example on relapse to drug use. Result: The strategies assuming event (non-)occurrence and the recall strategy had the worst performance because of a substantial parameter bias and a sharp decrease in coverage rate. Deletion methods suffered from either loss of power or undercoverage issues resulting from a biased standard error. Single imputation recovered the bias issue but showed an undercoverage estimate. Multiple imputations performed reasonably with a negligible standard error bias leading to a gradual decrease in power. Conclusion: On the basis of the simulation results and real example, we provide practical guidance to researches in terms of the best ways to deal with missing event history data.

Reflection on modern methods: demystifying robust standard errors for epidemiologists.

All statistical estimates from data have uncertainty due to sampling variability. A standard error is one measure of uncertainty of a sample estimate (such as the mean of a set of observations or a regression coefficient). Standard errors are usually calculated based on assumptions underpinning the statistical model used in the estimation. However, there are situations in which some assumptions of the statistical model including the variance or covariance of the outcome across observations are violated, which leads to biased standard errors. One simple remedy is to use robust standard errors, which are robust to violations of certain assumptions of the statistical model. Robust standard errors are frequently used in clinical papers (e.g. to account for clustering of observations), although the underlying concepts behind robust standard errors and when to use them are often not well understood. In this paper, we demystify robust standard errors using several worked examples in simple situations in which model assumptions involving the variance or covariance of the outcome are misspecified. These are: (i) when the observed variances are different, (ii) when the variance specified in the model is wrong and (iii) when the assumption of independence is wrong.

Generalized Linear Factor Score Regression: A Comparison of Four Methods.

Factor score regression has recently received growing interest as an alternative for structural equation modeling. However, many applications are left without guidance because of the focus on normally distributed outcomes in the literature. We perform a simulation study to examine how a selection of factor scoring methods compare when estimating regression coefficients in generalized linear factor score regression. The current study evaluates the regression method and the correlation-preserving method as well as two sum score methods in ordinary, logistic, and Poisson factor score regression. Our results show that scoring method performance can differ notably across the considered regression models. In addition, the results indicate that the choice of scoring method can substantially influence research conclusions. The regression method generally performs the best in terms of coefficient and standard error bias, accuracy, and empirical Type I error rates. Moreover, the regression method and the correlation-preserving method mostly outperform the sum score methods.