Extent Of Measurement Error Research Articles

Measurement error in longitudinal earnings data: evidence from Germany

We present evidence on the extent of measurement error in German longitudinal earnings data. Qualitatively, we confirm the main result of the international literature: longitudinal earnings data are relatively reliable in a cross section but much less so in first differences. Quantitatively, in the cross section our findings are very similar to those of Bound and Krueger (J Labor Econ 9:1–24, 1991) and Pischke (J Bus Econ Stat 13:305–314, 1995) for the United States while we find even stronger evidence that first-differencing exacerbates measurement error problems. We also show that measurement error in our survey data is not “classical” as it is negatively correlated with administrative earnings and positively autocorrelated over an extended period of time. Additionally, we estimate a model of measurement error stemming from underreporting of transitory earnings shocks in combination with a white-noise component and make a number of methodological contributions. Our results are robust to the use of two different linked survey-administrative data sets and various other sensitivity checks.

Misreporting of program take-up in survey data and its consequences for measuring non-take-up: new evidence from linked administrative and survey data

The international literature studies non-take-up behavior of eligible populations to evaluate the effectiveness of government programs. A major challenge in this literature is the measurement error regarding benefit take-up. In our data, we observe both actual welfare receipt and respondents’ survey information on their program take-up. This allows us to observe the measurement errors that other researchers must estimate. We describe survey misreporting and investigate how it biases the estimates of the magnitude and patterns of benefit take-up among eligible households. Our findings suggest that the extent of measurement error can be substantial. It varies with the characteristics of the misreporting population and is associated with the drivers of misreporting. This indicates that survey-based analyses of take-up behavior are likely subject to severe biases.

Evaluating the Quality of Survey and Administrative Data with Generalized Multitrait-Multimethod Models

ABSTRACTAdministrative data are increasingly important in statistics, but, like other types of data, may contain measurement errors. To prevent such errors from invalidating analyses of scientific interest, it is therefore essential to estimate the extent of measurement errors in administrative data. Currently, however, most approaches to evaluate such errors involve either prohibitively expensive audits or comparison with a survey that is assumed perfect. We introduce the “generalized multitrait-multimethod” (GMTMM) model, which can be seen as a general framework for evaluating the quality of administrative and survey data simultaneously. This framework allows both survey and administrative data to contain random and systematic measurement errors. Moreover, it accommodates common features of administrative data such as discreteness, nonlinearity, and nonnormality, improving similar existing models. The use of the GMTMM model is demonstrated by application to linked survey-administrative data from the German Federal Employment Agency on income from of employment, and a simulation study evaluates the estimates obtained and their robustness to model misspecification. Supplementary materials for this article are available online.

Measurement error in geometric morphometrics.

Geometric morphometrics-a set of methods for the statistical analysis of shape once saluted as a revolutionary advancement in the analysis of morphology -is now mature and routinely used in ecology and evolution. However, a factor often disregarded in empirical studies is the presence and the extent of measurement error. This is potentially a very serious issue because random measurement error can inflate the amount of variance and, since many statistical analyses are based on the amount of "explained" relative to "residual" variance, can result in loss of statistical power. On the other hand, systematic bias can affect statistical analyses by biasing the results (i.e. variation due to bias is incorporated in the analysis and treated as biologically-meaningful variation). Here, I briefly review common sources of error in geometric morphometrics. I then review the most commonly used methods to measure and account for both random and non-random measurement error, providing a worked example using a real dataset.

Measuring Test Measurement Error

Test-based accountability as well as value-added asessments and much experimental and quasi-experimental research in education rely on achievement tests to measure student skills and knowledge. Yet, we know little regarding fundamental properties of these tests, an important example being the extent of measurement error and its implications for educational policy and practice. While test vendors provide estimates of split-test reliability, these measures do not account for potentially important day-to-day differences in student performance. In this article, we demonstrate a credible, low-cost approach for estimating the overall extent of measurement error that can be applied when students take three or more tests in the subject of interest (e.g., state assessments in consecutive grades). Our method generalizes the test–retest framework by allowing for (a) growth or decay in knowledge and skills between tests, (b) tests being neither parallel nor vertically scaled, and (c) the degree of measurement error varying across tests. The approach maintains relatively unrestrictive, testable assumptions regarding the structure of student achievement growth. Estimation only requires descriptive statistics (e.g., test-score correlations). With student-level data, the extent and pattern of measurement-error heteroscedasticity also can be estimated. In turn, one can compute Bayesian posterior means of achievement and achievement gains given observed scores—estimators having statistical properties superior to those for the observed score (score gain). We employ math and English language arts test-score data from New York City to demonstrate these methods and estimate the overall extent of test measurement error is at least twice as large as that reported by the test vendor.

Measurement error in earnings data in the health and retirement study

We provide new evidence on the extent of measurement error in respondentreported earnings data by exploiting detailed W-2 records matched to older workers in the Health and Retirement Study (HRS). Our empirical findings are qualitatively consistent with the findings of previous studies. Mean measurement error in the 1991 HRS earnings data for men is somewhat larger than what has been found in other validation studies, but is still modest, averaging about 0.059 log points, approximately 5.9 percent, or $1,500. For women in 1991, it is 0.067 log points, approximately 6.7 percent, or $916. We find a negative correlation between the measurement error and the true value of earnings as measured by the W-2 records, which indicates the presence of non-classical measurement error. For men and women, this error shows little correlation with a standard set of cross-sectional earnings determinants. The one exception is that the measurement error rises with reported education. The bias on the OLS parameter estimate of the impact of having a college degree or higher (relative to a high school drop-out) from using the respondent-reported rather than the W-2 earnings is positive and estimated to be 0.071 log points, or roughly a bias of 7 percent.

Measurement Error in Earnings Data in the Health and Retirement Study

We provide new evidence on the extent of measurement error in respondent-reported earnings data by exploiting detailed W-2 records matched to older workers in the Health and Retirement Study (HRS). Our empirical findings are qualitatively consistent with the findings of previous studies. Mean measurement error in the 1991 HRS earnings data for men is somewhat larger than what has been found in other validation studies, but is still modest, averaging about 0.059 log points, approximately 5.9 percent, or $1,500. For women in 1991, it is 0.067 log points, approximately 6.7 percent, or $916. We find a negative correlation between the measurement error and the true value of earnings as measured by the W-2 records, which indicates the presence of non-classical measurement error. For men and women, this error shows little correlation with a standard set of cross-sectional earnings determinants. The one exception is that the measurement error rises with reported education. The bias on the OLS parameter estimate of the impact of having a college degree or higher (relative to a high school drop-out) from using the respondent-reported rather than the W-2 earnings is positive and estimated to be 0.071 log points, or roughly a bias of 7 percent.

A Monte Carlo Study of Growth Regressions

Using Monte Carlo simulations, this paper evaluates the bias properties of common estimators used in growth regressions derived from the Solow model. We explicitly allow for measurement error in the right-hand side variables, as well as country-specific effects that are correlated with the regressors. Our results suggest that using an OLS estimator applied to a single cross-section of variables averaged over time (the between estimator) performs best in terms of the extent of bias on each of the estimated coefficients. The fixed-effects estimator and the Arellano-Bond estimator greatly overstate the speed of convergence under a wide variety of assumptions concerning the type and extent of measurement error, while between understates it somewhat. Finally, fixed effects and Arellano-Bond bias towards zero the slope estimates on the human and physical capital accumulation variables.

A Monte Carlo Study of Regressions

Using Monte Carlo simulations, this paper evaluates the bias properties of common estimators used in growth regressions derived from the Solow model. We explicitly allow for measurement error in the right-hand side variables, as well as country-specific effects that are correlated with the regressors. Our results suggest that using an OLS estimator applied to a single cross-section of variables averaged over time (the between estimator) performs best in terms of the extent of bias on each of the estimated coeffcients. The fixed-effects estimator and the Arellano-Bond estimator greatly overstate the speed of convergence under a wide variety of assumptions concerning the type and extent of measurement error, while between understates it somewhat. Finally, fixed effects and Arellano-Bond bias towards zero the slope estimates on the human and physical capital accumulation variables.

Biomarkers in nutritional epidemiology.

To illustrate biomarkers of diet that can be used to validate estimates of dietary intake in the study of gene-environment interactions in complex diseases. Prospective cohort studies, studies of biomarkers where diet is carefully controlled. Free-living individuals, volunteers in metabolic suites. Male and female human volunteers. Recent studies using biomarkers have demonstrated substantial differences in the extent of measurement error from those derived by comparison with other methods of dietary assessment. The interaction between nutritional and genetic factors has so far largely gone uninvestigated, but can be studied in epidemiological trials that include collections of biological material. Large sample sizes are required to study interactions, and these are made larger in the presence of measurement errors. Diet is of key importance in affecting the risk of most chronic diseases in man. Nutritional epidemiology provides the only direct approach to the quantification of risks. The introduction of biomarkers to calibrate the measurement error in dietary reports, and as additional measures of exposure, is a significant development in the effort to improve estimates of the magnitude of the contribution of diet in affecting individual disease risk within populations. The extent of measurement error has important implications for correction for regression dilution and for sample size. The collection of biological samples to improve and validate estimates of exposure, enhance the pursuit of scientific hypotheses, and enable gene-nutrient interactions to be studied, should become the routine in nutritional epidemiology.

The role of expectations in arms acquisition processes: The U.S.‐U.S.S.R. case

Recent empirical studies of arms races have led to the disturbing conclusion that for several important arms races, particularly the U.S.‐U.S.S.R. case, there appears to be no interaction. Instead arms acquisition appears to be driven only by internal factors. A number of interpretations for these results have been provided. They include, inappropriate model specification, inappropriate operationalization of military threat, and error ridden military expenditure data. Recent results, coupled with our more descriptive understanding of arms processes, suggest that expectations of current and future arms expenditures by the opponent are relevant to an understanding of current arms expenditures. This implies that such expectations must be taken into account in the arms modeling process. In this paper several alternative expectations mechanisms are formulated and tested for the U.S.‐U.S.S.R. case. The extent of measurement error in Soviet military expenditures and its effect on the empirical tests of the expec...