The CANIS Method of Reducing Bias in Survey Research

Abstract

Although survey research is one of the best tools yet found for use by the social sciences, it is far from perfect. Error creeps in at almost every stage of the procedure. There is sampling error, error in questionnaire construction, and error in recording responses just to name a few. However, it has been noted that the most vulnerable stage in survey research is the interview situation itself, and the most uncontrollable aspect of this stage involves the dependence on the respondent to answer truthfully the questions he is asked. In fact, one of the main assumptions of survey research is that respondents in fact do not conceal their real feelings and do respond to questions truthfully. Were the researcher not allowed to accept this assumption, there would be little use in venturing into the field. As important as this assumption is, it must be admitted that there are certain respondents who will not cooperate for one reason or another and the more controversial the content of the questions, the more one must suspect that the questions are not answered truthfully. This article describes a research strategy designed to eliminate the inherent bias in such an interview situation. Still more important, the experiment pointed the way to a superior method which eliminated even the possibility of the respondent falsifying his responses. This project started with the search for a standard by which to evaluate responses during the interview so they could be validated as truthful. The usual technique of putting a number of different questions in the interview schedule designed to tap the same attitude and then checking to see if, in fact, they were answered in the same way was found to be unsatisfactory. Most respondents were found to be too sophisticated to be duped by such a naive method. We also examined another possibility, including in the schedule questions that could be checked by an outside source. For example, we included questions about the respondent's place of work, his social security number, and whether he had contributed to the Community Chest. We then checked these answers to see if he was telling the truth. It was finally concluded that this method was unsatisfactory in that it was not possible to generalize from this information to responses to attitude questions. Other methods proved equally subject to attack, but it was during an exploration of a suggestion to use a lie-detector during the interview that we came upon a way to check on the truth of the respondent's answers. A lie-detector would have given us the check on the validity of the answers, but by requesting a respondent to use such an apparatus, we would have been including in the interview an even more biasing factor. The respondent would have been put in an uneasy situation by being asked to use such a controversial device. The question arose whether there might be something already in the house, which the respondent would not be aware of, that would belie his true emotional response to the questions he was answering. If the immense literature on animal psychology has proved one thing, it is that a dog comes to reflect the personality of its owner. Furthermore, dogs sense emotional distress in their owners and, in effect, are excellent indicators of whether their owners are un-