Monitoring Social Change Via Survey Replication: Prospects and Pitfalls from a Replication Survey of Social Roles and Mental Health

Abstract

Recent methodological criticisms of longitudinal studies employing a panel design suggest that greater care is needed in determining what can and cannot be studied best using that approach, and where other diachronic approaches may be as appropriate or more appropriate. In this paper a few of the key strengths and weaknesses of panel studies relative to one alternative approach—a sequence of cross‐sectional surveys on samples from the same population—are reviewed. That review suggests that the latter design holds considerable promise as a method for studying women and social change. In spite of this great potential, a number of possible pitfalls associated with the analysis of successive cross‐sectional surveys for that purpose are described and illustrated using examples from a recent twenty‐year replication study. Such potential problems include: (1) differences in sample design; (2) differences in response rates; (3) demographic shifts in the population; (4) problems in repeating flawed questions; (5) changes in question meanings and context; (6) problems of maintaining comparable codes; and (7) potential changes in response bias. In spite of these inherent difficulties, however, additional examples derived from the same study are used to demonstrate the great potential value of using survey replication for the examination of important research questions in the study of social change.