The Validity of Survey Responses: Insights from Interviews of Married Couples in a Survey of Soviet Emigrants

Abstract

Survey researchers have given little attention to the reliability of respondents as informants about others in the household. Data from the Soviet Interview Project General Survey a study based on recent emigrants from the Soviet Union to the US provide a special opportunity to test for the accuracy of recall and of reporting about others by measuring the level of agreement between the answers by 163 married couples who lived in the same household in the USSR. It also tests for the response effects of 2 sources of potential contamination in the interview setting: having multiple respondents in a family who are interviewed at different times and who thus could discuss the survey; and having 3rd parties present during the interview. The analysis shows a very high level of agreement between spouses on measures of household material status in the USSR (income wealth housing space) probably stemming from the high saliency of these measures as status indicators. Contrary to expectations neither the agreement on objective questions nor on subjective questions is affected by possible contamination of the interviews. A low level of agreement on answers to some ostensibly objective questions occurs however because the behavior was variant over time definitions were ambiguous or the question was not properly administered. The analysis also shows that studying divergent answers by associated pairs of respondents to questions to which there should be substantial agreement can reveal unanticipated ambiguities in question wording and thus can be an aid in debugging and interpreting survey data. (authors)