Measurement Errors In Survey Data Research Articles

Dependent interviewing: a remedy or a curse for measurement error in surveys?

Longitudinal surveys often rely on dependent interviewing (DI) to lower the levels of random measurement error in survey data and reduce the incidence of spurious change. DI refers to a data collection technique that incorporates information from prior interview rounds into subsequent waves. While this method is considered an effective remedy for random measurement error, it can also introduce more systematic errors, in particular when respondents are first reminded of their previously provided answer and then asked about their current status. The aim of this paper is to assess the impact of DI on measurement error in employment mobility. We take advantage of a unique experimental situation that was created by the roll-out of dependent interviewing in the Dutch Labour Force Survey (LFS). We apply Hidden Markov Modeling (HMM) to linked LFS and Employment Register (ER) data that cover a period before and after dependent interviewing was abolished, which in turn enables the modeling of systematic errors in the LFS data. Our results indicate that DI lowered the probability of obtaining random measurement error but had no significant effect on the systematic component of the error. The lack of a significant effect might be partially due to the fact that the probability of repeating the same error was extremely high at baseline (i.e when using standard, independent interviewing); therefore the use of DI could not increase this probability any further.

Age is measured with systematic measurement error in developing country surveys: A diagnosis and analysis of consequences

Research in political science and other social sciences often relies on survey data to study a range of questions about politics in the developing world. This study identifies systematic measurement error in some of the most frequently used datasets with respect to one commonly employed variable: respondent’s age. It shows evidence of substantial measurement error that is correlated with observable characteristics, and discusses and illustrates the implications for empirical analysis with an example from a recently published study. In doing so, it demonstrates tools for identifying and diagnosing systematic measurement error in survey data, as well as for investigating the robustness of one’s findings when the problem arises.

Using sensors to measure technology adoption in the social sciences

Empirical social sciences rely heavily on surveys to measure human behavior. Previous studies show that such data are prone to random errors and systematic biases caused by social desirability, recall challenges, and the Hawthorne effect. Moreover, collecting high frequency survey data is often impossible, which is important for outcomes that fluctuate. Innovation in sensor technology might address these challenges. In this study, we use sensors to describe solar light adoption in Kenya and analyze the extent to which survey data are limited by systematic and random error. Sensor data reveal that households used lights for about 4 h per day. Frequent surveyor visits for a random sub-sample increased light use in the short term, but had no long-term effects. Despite large measurement errors in survey data, self-reported use does not differ from sensor measurements on average and differences are not correlated with household characteristics. However, mean-reverting measurement error stands out: households that used the light a lot tend to underreport, while households that used it little tend to overreport use. Last, general usage questions provide more accurate information than asking about each hour of the day. Sensor data can serve as a benchmark to test survey questions and seem especially useful for small-sample analyses.

Impact of Look-Back Periods on Misclassification

s / Annals of Epidemiology 22 (2012) 661e680 673 P55. Impact of Look-Back Periods on Misclassification C.D. O'Malley, R.I. Griffiths, R.J. Herbert, M.D. Danese. CfOR, Amgen, South San Francisco, CA; Outcomes Insights, Inc. Westlake Village, CA; Johns Hopkins Bloomberg School of Public Health, Baltimore, MD Purpose: Estimating disease incidence using claims data often requires constructing a look-back period to exclude those with prevalent disease. Those missed during the look-back period may be misclassified as incident cases (false positives). Using Medicare claims, we examined the impact of varying the look-back period lengths on the incidence of 12 conditions. Methods: Two cohorts of women were included: 33,731 diagnosed with breast cancer and 101,649 without cancer, all with 39 months of Medicare eligibility. Cancer patients were followed from 36 months before diagnosis (prevalence period) up to 3 months after diagnosis (incidence period). Noncancer patients were followed for up to 39 months after the beginning of Medicare eligibility with a sham date inserted after 36 months to separate the prevalence and incidence periods. Using 36 months as the gold standard, the look-back period was shortened in 6-month increments to examine the impact on false positives during the incidence period. Results: In the cancer cohort, > 20% of the total incident cases for 11/12 conditions were false positives using a 6 month period. Lengthening the look-back period from 6 to 12 months resulted in the greatest decline in false positives. The impact varied by condition; false positive rates for diabetes and liver disease were 57% and 22% using 6months, 27%, and 15%with 12months compared with 0% as the 36 month gold standard. Misclassification patterns were similar but lower in non-cancer patients. Conclusions: Shortening the look-back period to rule out pre-existing disease can substantially reduce the misclassification of incident disease. P56-S. Measurement Error in Survey Data on Income From a Publicly Funded Financial Credit F. Pega, K. Carter, T. Blakely. Department of Public Health, University of Otago, Wellington, New Zealand Purpose: Measurement errors in survey data reporting of income from publicly funded financial credits are a central methodological concern in research on the health impact of such credits. This study sought to quantify the measurement error in reported income from the Family Tax Credit (FTC), New Zealand's equivalent of the U.S. Earned Income Tax Credit. Methods: Seven waves of data (2002-2009) from Statistics New Zealand's Survey of Family, Income and Employment were extracted (N1⁄427,795). These data were restricted to a balanced panel of working-age adults in families with children aged under 13 years (N1⁄45,710). FTC receipt (any vs. no) and amount received were derived from survey data and, in a second step, estimated by applying eligibility and entitlement criteria. The reported FTC receipt and amount were compared with the estimated FTC receipt and amount, using cross-sectional tabular and correlation analyses. Results: The reported FTC receipt did not correspond with the estimated receipt for 12.1-12.4% of participants. Underreporting of FTC receipt was 10.410.8%, and overreporting was 34.5%. The reported and estimated FTC amounts correlated weakly (Pearson's r 1⁄4 0.17233). Conclusion: This study identified large measurement errors in survey data reporting of income from FTC, raising concerns for the validity of these data. Survey data on income from publicly funded financial credits should be scrutinized for measurement errors, and may not be fit for use in epidemiological studies.

Measurement Error in Survey Data on Income From a Publicly Funded Financial Credit

Measurement Error in Survey Data on Income From a Publicly Funded Financial Credit

Some Consequences of Measurement Error in Survey Data

Using the 1956-1958-1960 SRC American panel data and the 1965 Jennings-Niemi socialization data, this paper first presents some estimates of the extent of measurement error in several standard face sheet items. After the presence of measurement error is demonstrated, two techniques involving multiple indicators and observations over time are employed to estimate the effects of measurement error on bivariate correlation coefficients with party identification providing the substantive vehicle of the analysis. In general, the analysis suggests that random measurement error may have a major impact on our coefficients and thereby result in misleading inferences.