The Problem of Response Stability: Some Correlates and Consequences

Abstract

The study of political attitudes and behavior has long been one of the major areas of research in political science. Through survey techniques including interviews and written questionnaires, numerous measures of political attitudes have been obtained and thoroughly analyzed. Considerable scholarly effort has been devoted to developing rigorous research designs and innovating research instruments. Similarly, the use of complex and sophisticated statistical techniques to describe and analyze data has increased steadily over the last few years. However, despite this emphasis on rigor and precision, the most fundamental problems of reliability and validity are often ignored and obscured. The analysis undertaken here will focus on the reliability of survey data, specifically, data generated by written questionnaires. Following the conceptualization of reliability, an attempt will be made to explain differential levels of reliability. Finally, the effects of identifying and removing unreliable data will be investigated. Political scientists are most familiar with discussions of reliability couched in terms of accuracy and internal consistency. Accuracy, of course, entails the absence of any systematic errors such as coding irregularities from the measurement process while internal consistency usually refers to the similarity and homogeneity of data patterns generated by different indicators of the same construct. Both these facets of reliability are obviously important and have been the subject of considerable research.1 On the other hand, the issue of response stability, while of equal imhportance, has generally been ignored by political scientists. This is indeed unfortunate since the issue raises impor-